2024-07-03T15:23:09.020 INFO:root:teuthology version: 0.0.1.dev347+ga4a89a3 2024-07-03T15:23:09.020 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_gabrioux@teuthology --name gabrioux-2024-07-03_14:50:01-orch:cephadm-squid-release-distro-default-smithi --verbose --description orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} --archive /home/teuthworker/archive/gabrioux-2024-07-03_14:50:01-orch:cephadm-squid-release-distro-default-smithi/7785842 -- /home/teuthworker/archive/gabrioux-2024-07-03_14:50:01-orch:cephadm-squid-release-distro-default-smithi/7785842/orig.config.yaml 2024-07-03T15:23:09.068 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-07-03T15:23:09.154 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/gabrioux-2024-07-03_14:50:01-orch:cephadm-squid-release-distro-default-smithi/7785842 branch: squid-release description: orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: gabrioux@ibm.com first_in_suite: false job_id: '7785842' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/quincy ' name: gabrioux-2024-07-03_14:50:01-orch:cephadm-squid-release-distro-default-smithi no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid-release ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN sha1: c00506dc10003928f8cb40769168571f611cbd67 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: c00506dc10003928f8cb40769168571f611cbd67 kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid-release sha1: 47d23ed6cd9bd552d37bff09d8392220d7ba7b11 owner: scheduled_gabrioux@teuthology priority: 30 repo: https://git.ceph.com/ceph-ci.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 3319 sha1: c00506dc10003928f8cb40769168571f611cbd67 sleep_before_teardown: 0 subset: 1/8 suite: orch:cephadm suite_branch: squid-release suite_path: /home/teuthworker/src/github.com_guits_ceph_47d23ed6cd9bd552d37bff09d8392220d7ba7b11/qa suite_relpath: qa suite_repo: https://github.com/guits/ceph suite_sha1: 47d23ed6cd9bd552d37bff09d8392220d7ba7b11 targets: smithi028.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEB7erQ+D8UdluL0AWh0KjkRVU/9qGgXiguyh8Q3vFxd79cWWRkFyTU/pf478/VhBLhBYZ6+8KVGcaBakv6PlK4= smithi148.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLlGOPCHFJg7h0HHV/FOhXJbVOY8yafBBw7Nc+Ha7rjzKOTTUeFBIfk/7IKwFymORjGXq2S3+ZlE6U3wSDJ+ZzY= tasks: - install: branch: quincy exclude_packages: - ceph-volume - print: '**** done install task...' - cephadm: cephadm_branch: quincy cephadm_git_url: https://github.com/ceph/ceph conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.ceph.io/ceph-ci/ceph:quincy roleless: true - print: '**** done end installing quincy cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 1 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay true - cephadm.shell: host.a: - ceph fs set cephfs inline_data false - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - kclient: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: - /cephfs/home/gabrioux/src/github.com_guits_ceph_squid-release/qa/suites/orch/cephadm/mds_upgrade_sequence/tasks/3-upgrade-mgr-staggered.yaml meta: {} postmerge: [] variables: fail_fs: false teuthology_branch: main teuthology_sha1: a4a89a3d72be587ba6bd8509090508e8a19e74c5 timestamp: 2024-07-03_14:50:01 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs false || true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: gabrioux verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698093 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-07-03T15:23:09.155 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_guits_ceph_47d23ed6cd9bd552d37bff09d8392220d7ba7b11/qa; will attempt to use it 2024-07-03T15:23:09.155 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_guits_ceph_47d23ed6cd9bd552d37bff09d8392220d7ba7b11/qa/tasks 2024-07-03T15:23:09.155 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-07-03T15:23:09.157 INFO:teuthology.task.internal:Checking packages... 2024-07-03T15:23:09.176 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'c00506dc10003928f8cb40769168571f611cbd67' 2024-07-03T15:23:09.176 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-07-03T15:23:09.177 INFO:teuthology.packaging:ref: None 2024-07-03T15:23:09.177 INFO:teuthology.packaging:tag: None 2024-07-03T15:23:09.177 INFO:teuthology.packaging:branch: squid-release 2024-07-03T15:23:09.177 INFO:teuthology.packaging:sha1: c00506dc10003928f8cb40769168571f611cbd67 2024-07-03T15:23:09.177 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid-release 2024-07-03T15:23:09.323 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2550.gc00506dc 2024-07-03T15:23:09.325 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-07-03T15:23:09.336 INFO:teuthology.task.internal:no buildpackages task found 2024-07-03T15:23:09.336 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-07-03T15:23:09.347 INFO:teuthology.task.internal:Saving configuration 2024-07-03T15:23:09.364 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-07-03T15:23:09.373 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-07-03T15:23:09.400 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi028.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-07-03_14:50:01-orch:cephadm-squid-release-distro-default-smithi/7785842', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-07-03 15:17:56.352879', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEB7erQ+D8UdluL0AWh0KjkRVU/9qGgXiguyh8Q3vFxd79cWWRkFyTU/pf478/VhBLhBYZ6+8KVGcaBakv6PlK4='} 2024-07-03T15:23:09.421 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi148.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-07-03_14:50:01-orch:cephadm-squid-release-distro-default-smithi/7785842', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-07-03 15:17:56.354619', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLlGOPCHFJg7h0HHV/FOhXJbVOY8yafBBw7Nc+Ha7rjzKOTTUeFBIfk/7IKwFymORjGXq2S3+ZlE6U3wSDJ+ZzY='} 2024-07-03T15:23:09.421 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-07-03T15:23:09.428 INFO:teuthology.task.internal:roles: ubuntu@smithi028.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-07-03T15:23:09.428 INFO:teuthology.task.internal:roles: ubuntu@smithi148.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-07-03T15:23:09.428 INFO:teuthology.run_tasks:Running task console_log... 2024-07-03T15:23:09.489 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f843de83ac0>, signals=[15]) 2024-07-03T15:23:09.489 INFO:teuthology.run_tasks:Running task internal.connect... 2024-07-03T15:23:09.497 INFO:teuthology.task.internal:Opening connections... 2024-07-03T15:23:09.497 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi028.front.sepia.ceph.com 2024-07-03T15:23:09.498 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-03T15:23:09.573 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi148.front.sepia.ceph.com 2024-07-03T15:23:09.574 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-03T15:23:09.645 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-07-03T15:23:09.654 DEBUG:teuthology.orchestra.run.smithi028:> uname -m 2024-07-03T15:23:09.673 INFO:teuthology.orchestra.run.smithi028.stdout:x86_64 2024-07-03T15:23:09.673 DEBUG:teuthology.orchestra.run.smithi028:> cat /etc/os-release 2024-07-03T15:23:09.732 INFO:teuthology.orchestra.run.smithi028.stdout:NAME="CentOS Stream" 2024-07-03T15:23:09.732 INFO:teuthology.orchestra.run.smithi028.stdout:VERSION="9" 2024-07-03T15:23:09.732 INFO:teuthology.orchestra.run.smithi028.stdout:ID="centos" 2024-07-03T15:23:09.732 INFO:teuthology.orchestra.run.smithi028.stdout:ID_LIKE="rhel fedora" 2024-07-03T15:23:09.732 INFO:teuthology.orchestra.run.smithi028.stdout:VERSION_ID="9" 2024-07-03T15:23:09.733 INFO:teuthology.orchestra.run.smithi028.stdout:PLATFORM_ID="platform:el9" 2024-07-03T15:23:09.733 INFO:teuthology.orchestra.run.smithi028.stdout:PRETTY_NAME="CentOS Stream 9" 2024-07-03T15:23:09.733 INFO:teuthology.orchestra.run.smithi028.stdout:ANSI_COLOR="0;31" 2024-07-03T15:23:09.733 INFO:teuthology.orchestra.run.smithi028.stdout:LOGO="fedora-logo-icon" 2024-07-03T15:23:09.733 INFO:teuthology.orchestra.run.smithi028.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-07-03T15:23:09.733 INFO:teuthology.orchestra.run.smithi028.stdout:HOME_URL="https://centos.org/" 2024-07-03T15:23:09.733 INFO:teuthology.orchestra.run.smithi028.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-07-03T15:23:09.733 INFO:teuthology.orchestra.run.smithi028.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-07-03T15:23:09.733 INFO:teuthology.orchestra.run.smithi028.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-07-03T15:23:09.734 INFO:teuthology.lock.ops:Updating smithi028.front.sepia.ceph.com on lock server 2024-07-03T15:23:09.759 DEBUG:teuthology.orchestra.run.smithi148:> uname -m 2024-07-03T15:23:09.777 INFO:teuthology.orchestra.run.smithi148.stdout:x86_64 2024-07-03T15:23:09.777 DEBUG:teuthology.orchestra.run.smithi148:> cat /etc/os-release 2024-07-03T15:23:09.834 INFO:teuthology.orchestra.run.smithi148.stdout:NAME="CentOS Stream" 2024-07-03T15:23:09.835 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION="9" 2024-07-03T15:23:09.835 INFO:teuthology.orchestra.run.smithi148.stdout:ID="centos" 2024-07-03T15:23:09.835 INFO:teuthology.orchestra.run.smithi148.stdout:ID_LIKE="rhel fedora" 2024-07-03T15:23:09.835 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION_ID="9" 2024-07-03T15:23:09.835 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM_ID="platform:el9" 2024-07-03T15:23:09.835 INFO:teuthology.orchestra.run.smithi148.stdout:PRETTY_NAME="CentOS Stream 9" 2024-07-03T15:23:09.835 INFO:teuthology.orchestra.run.smithi148.stdout:ANSI_COLOR="0;31" 2024-07-03T15:23:09.835 INFO:teuthology.orchestra.run.smithi148.stdout:LOGO="fedora-logo-icon" 2024-07-03T15:23:09.835 INFO:teuthology.orchestra.run.smithi148.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-07-03T15:23:09.835 INFO:teuthology.orchestra.run.smithi148.stdout:HOME_URL="https://centos.org/" 2024-07-03T15:23:09.835 INFO:teuthology.orchestra.run.smithi148.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-07-03T15:23:09.835 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-07-03T15:23:09.836 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-07-03T15:23:09.836 INFO:teuthology.lock.ops:Updating smithi148.front.sepia.ceph.com on lock server 2024-07-03T15:23:09.859 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-07-03T15:23:09.869 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-07-03T15:23:09.877 INFO:teuthology.task.internal:Checking for old test directory... 2024-07-03T15:23:09.877 DEBUG:teuthology.orchestra.run.smithi028:> test '!' -e /home/ubuntu/cephtest 2024-07-03T15:23:09.879 DEBUG:teuthology.orchestra.run.smithi148:> test '!' -e /home/ubuntu/cephtest 2024-07-03T15:23:09.895 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-07-03T15:23:09.903 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-07-03T15:23:09.903 DEBUG:teuthology.orchestra.run.smithi028:> test -z $(ls -A /var/lib/ceph) 2024-07-03T15:23:09.937 DEBUG:teuthology.orchestra.run.smithi148:> test -z $(ls -A /var/lib/ceph) 2024-07-03T15:23:09.972 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-07-03T15:23:10.024 INFO:teuthology.run_tasks:Running task kernel... 2024-07-03T15:23:10.040 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-07-03T15:23:10.040 INFO:teuthology.task.kernel:config {'host.a': {'kdb': 1, 'sha1': 'distro'}, 'host.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-07-03T15:23:10.040 DEBUG:teuthology.orchestra.run.smithi028:> test -f /run/.containerenv -o -f /.dockerenv 2024-07-03T15:23:10.041 DEBUG:teuthology.orchestra.run.smithi148:> test -f /run/.containerenv -o -f /.dockerenv 2024-07-03T15:23:10.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-03T15:23:10.058 DEBUG:teuthology.orchestra.run.smithi148:> uname -r 2024-07-03T15:23:10.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-03T15:23:10.059 DEBUG:teuthology.orchestra.run.smithi028:> uname -r 2024-07-03T15:23:10.115 INFO:teuthology.orchestra.run.smithi148.stdout:5.14.0-437.el9.x86_64 2024-07-03T15:23:10.115 INFO:teuthology.task.kernel:Running kernel on smithi148: 5.14.0-437.el9.x86_64 2024-07-03T15:23:10.116 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum install -y kernel 2024-07-03T15:23:10.117 INFO:teuthology.orchestra.run.smithi028.stdout:5.14.0-437.el9.x86_64 2024-07-03T15:23:10.117 INFO:teuthology.task.kernel:Running kernel on smithi028: 5.14.0-437.el9.x86_64 2024-07-03T15:23:10.118 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum install -y kernel 2024-07-03T15:23:14.270 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - BaseOS 3.2 MB/s | 8.1 MB 00:02 2024-07-03T15:23:15.619 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - BaseOS 2.1 MB/s | 8.1 MB 00:03 2024-07-03T15:23:17.790 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - AppStream 8.3 MB/s | 20 MB 00:02 2024-07-03T15:23:20.695 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - AppStream 5.1 MB/s | 20 MB 00:03 2024-07-03T15:23:22.329 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - CRB 7.7 MB/s | 6.4 MB 00:00 2024-07-03T15:23:25.289 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - Extras packages 10 kB/s | 17 kB 00:01 2024-07-03T15:23:26.159 INFO:teuthology.orchestra.run.smithi148.stdout:Extra Packages for Enterprise Linux 31 MB/s | 22 MB 00:00 2024-07-03T15:23:27.129 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - CRB 2.5 MB/s | 6.4 MB 00:02 2024-07-03T15:23:28.590 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - Extras packages 98 kB/s | 17 kB 00:00 2024-07-03T15:23:29.451 INFO:teuthology.orchestra.run.smithi028.stdout:Extra Packages for Enterprise Linux 31 MB/s | 22 MB 00:00 2024-07-03T15:23:31.440 INFO:teuthology.orchestra.run.smithi148.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-07-03T15:23:33.554 INFO:teuthology.orchestra.run.smithi148.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-07-03T15:23:33.554 INFO:teuthology.orchestra.run.smithi148.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-07-03T15:23:33.653 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-07-03T15:23:33.667 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-07-03T15:23:33.667 INFO:teuthology.orchestra.run.smithi148.stdout: Package Architecture Version Repository Size 2024-07-03T15:23:33.667 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-07-03T15:23:33.667 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-07-03T15:23:33.667 INFO:teuthology.orchestra.run.smithi148.stdout: kernel x86_64 5.14.0-467.el9 baseos 1.2 M 2024-07-03T15:23:33.667 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-07-03T15:23:33.667 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-core x86_64 5.14.0-467.el9 baseos 16 M 2024-07-03T15:23:33.667 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-modules x86_64 5.14.0-467.el9 baseos 35 M 2024-07-03T15:23:33.668 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-modules-core x86_64 5.14.0-467.el9 baseos 29 M 2024-07-03T15:23:33.668 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:23:33.668 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-07-03T15:23:33.668 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-07-03T15:23:33.668 INFO:teuthology.orchestra.run.smithi148.stdout:Install 4 Packages 2024-07-03T15:23:33.668 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:23:33.668 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 82 M 2024-07-03T15:23:33.669 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 125 M 2024-07-03T15:23:33.669 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-07-03T15:23:34.200 INFO:teuthology.orchestra.run.smithi148.stdout:(1/4): kernel-5.14.0-467.el9.x86_64.rpm 2.8 MB/s | 1.2 MB 00:00 2024-07-03T15:23:34.948 INFO:teuthology.orchestra.run.smithi028.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-07-03T15:23:35.675 INFO:teuthology.orchestra.run.smithi148.stdout:(2/4): kernel-core-5.14.0-467.el9.x86_64.rpm 8.6 MB/s | 16 MB 00:01 2024-07-03T15:23:37.155 INFO:teuthology.orchestra.run.smithi028.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-07-03T15:23:37.155 INFO:teuthology.orchestra.run.smithi028.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-07-03T15:23:37.246 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-07-03T15:23:37.252 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-07-03T15:23:37.252 INFO:teuthology.orchestra.run.smithi028.stdout: Package Architecture Version Repository Size 2024-07-03T15:23:37.252 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-07-03T15:23:37.252 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-07-03T15:23:37.252 INFO:teuthology.orchestra.run.smithi028.stdout: kernel x86_64 5.14.0-467.el9 baseos 1.2 M 2024-07-03T15:23:37.252 INFO:teuthology.orchestra.run.smithi028.stdout:Installing dependencies: 2024-07-03T15:23:37.252 INFO:teuthology.orchestra.run.smithi028.stdout: kernel-core x86_64 5.14.0-467.el9 baseos 16 M 2024-07-03T15:23:37.252 INFO:teuthology.orchestra.run.smithi028.stdout: kernel-modules x86_64 5.14.0-467.el9 baseos 35 M 2024-07-03T15:23:37.252 INFO:teuthology.orchestra.run.smithi028.stdout: kernel-modules-core x86_64 5.14.0-467.el9 baseos 29 M 2024-07-03T15:23:37.252 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:23:37.253 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-07-03T15:23:37.253 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-07-03T15:23:37.253 INFO:teuthology.orchestra.run.smithi028.stdout:Install 4 Packages 2024-07-03T15:23:37.253 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:23:37.253 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 82 M 2024-07-03T15:23:37.253 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 125 M 2024-07-03T15:23:37.253 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-07-03T15:23:37.441 INFO:teuthology.orchestra.run.smithi148.stdout:(3/4): kernel-modules-5.14.0-467.el9.x86_64.rpm 9.7 MB/s | 35 MB 00:03 2024-07-03T15:23:37.717 INFO:teuthology.orchestra.run.smithi148.stdout:(4/4): kernel-modules-core-5.14.0-467.el9.x86_6 8.3 MB/s | 29 MB 00:03 2024-07-03T15:23:37.717 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-07-03T15:23:37.717 INFO:teuthology.orchestra.run.smithi148.stdout:Total 20 MB/s | 82 MB 00:04 2024-07-03T15:23:37.908 INFO:teuthology.orchestra.run.smithi028.stdout:(1/4): kernel-5.14.0-467.el9.x86_64.rpm 2.2 MB/s | 1.2 MB 00:00 2024-07-03T15:23:38.434 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-07-03T15:23:38.773 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-07-03T15:23:38.773 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-07-03T15:23:39.774 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-07-03T15:23:39.774 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-07-03T15:23:39.950 INFO:teuthology.orchestra.run.smithi028.stdout:(2/4): kernel-core-5.14.0-467.el9.x86_64.rpm 6.3 MB/s | 16 MB 00:02 2024-07-03T15:23:41.900 INFO:teuthology.orchestra.run.smithi028.stdout:(3/4): kernel-modules-core-5.14.0-467.el9.x86_6 7.3 MB/s | 29 MB 00:03 2024-07-03T15:23:42.221 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-07-03T15:23:42.833 INFO:teuthology.orchestra.run.smithi028.stdout:(4/4): kernel-modules-5.14.0-467.el9.x86_64.rpm 6.5 MB/s | 35 MB 00:05 2024-07-03T15:23:42.834 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-07-03T15:23:42.834 INFO:teuthology.orchestra.run.smithi028.stdout:Total 15 MB/s | 82 MB 00:05 2024-07-03T15:23:43.100 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : kernel-modules-core-5.14.0-467.el9.x86_64 1/4 2024-07-03T15:23:43.200 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : kernel-core-5.14.0-467.el9.x86_64 2/4 2024-07-03T15:23:43.582 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-07-03T15:23:43.988 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-07-03T15:23:43.988 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-07-03T15:23:44.993 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-07-03T15:23:44.993 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-07-03T15:23:45.899 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-core-5.14.0-467.el9.x86_64 2/4 2024-07-03T15:23:46.150 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : kernel-modules-5.14.0-467.el9.x86_64 3/4 2024-07-03T15:23:47.563 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-07-03T15:23:48.224 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : kernel-modules-core-5.14.0-467.el9.x86_64 1/4 2024-07-03T15:23:48.317 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : kernel-core-5.14.0-467.el9.x86_64 2/4 2024-07-03T15:23:50.928 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-modules-5.14.0-467.el9.x86_64 3/4 2024-07-03T15:23:51.059 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : kernel-5.14.0-467.el9.x86_64 4/4 2024-07-03T15:23:51.116 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: kernel-core-5.14.0-467.el9.x86_64 2/4 2024-07-03T15:23:51.254 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : kernel-modules-5.14.0-467.el9.x86_64 3/4 2024-07-03T15:23:55.877 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-modules-core-5.14.0-467.el9.x86_64 4/4 2024-07-03T15:23:56.414 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: kernel-modules-5.14.0-467.el9.x86_64 3/4 2024-07-03T15:23:56.579 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : kernel-5.14.0-467.el9.x86_64 4/4 2024-07-03T15:24:01.761 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: kernel-modules-core-5.14.0-467.el9.x86_64 4/4 2024-07-03T15:24:25.363 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-core-5.14.0-467.el9.x86_64 4/4 2024-07-03T15:24:25.403 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-modules-5.14.0-467.el9.x86_64 4/4 2024-07-03T15:24:25.748 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-5.14.0-467.el9.x86_64 4/4 2024-07-03T15:24:25.749 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : kernel-5.14.0-467.el9.x86_64 1/4 2024-07-03T15:24:25.749 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : kernel-core-5.14.0-467.el9.x86_64 2/4 2024-07-03T15:24:25.749 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : kernel-modules-5.14.0-467.el9.x86_64 3/4 2024-07-03T15:24:26.556 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : kernel-modules-core-5.14.0-467.el9.x86_64 4/4 2024-07-03T15:24:26.556 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:24:26.556 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-07-03T15:24:26.556 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-5.14.0-467.el9.x86_64 2024-07-03T15:24:26.556 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-core-5.14.0-467.el9.x86_64 2024-07-03T15:24:26.556 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-modules-5.14.0-467.el9.x86_64 2024-07-03T15:24:26.556 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-modules-core-5.14.0-467.el9.x86_64 2024-07-03T15:24:26.556 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:24:26.556 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-07-03T15:24:26.946 DEBUG:teuthology.orchestra.run.smithi148:> rpm -q kernel | sort -rV | head -n 1 2024-07-03T15:24:27.033 INFO:teuthology.orchestra.run.smithi148.stdout:kernel-5.14.0-467.el9.x86_64 2024-07-03T15:24:27.033 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-467.el9.x86_64 2024-07-03T15:24:27.033 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-467.el9.x86_64 2024-07-03T15:24:27.034 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-07-03T15:24:27.034 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-07-03T15:24:27.034 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2024-07-03T15:24:27.034 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi148.front.sepia.ceph.com, path=None, version=distro) 2024-07-03T15:24:27.034 DEBUG:teuthology.orchestra.run.smithi148:> rpm -q kernel | sort -rV | head -n 1 2024-07-03T15:24:27.087 INFO:teuthology.orchestra.run.smithi148.stdout:kernel-5.14.0-467.el9.x86_64 2024-07-03T15:24:27.088 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-467.el9.x86_64 2024-07-03T15:24:27.088 DEBUG:teuthology.orchestra.run.smithi148:> sudo rpm -qi grub2-tools 2024-07-03T15:24:27.118 INFO:teuthology.orchestra.run.smithi148.stdout:Name : grub2-tools 2024-07-03T15:24:27.118 INFO:teuthology.orchestra.run.smithi148.stdout:Epoch : 1 2024-07-03T15:24:27.118 INFO:teuthology.orchestra.run.smithi148.stdout:Version : 2.06 2024-07-03T15:24:27.118 INFO:teuthology.orchestra.run.smithi148.stdout:Release : 68.el9 2024-07-03T15:24:27.118 INFO:teuthology.orchestra.run.smithi148.stdout:Architecture: x86_64 2024-07-03T15:24:27.118 INFO:teuthology.orchestra.run.smithi148.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-07-03T15:24:27.118 INFO:teuthology.orchestra.run.smithi148.stdout:Group : Unspecified 2024-07-03T15:24:27.118 INFO:teuthology.orchestra.run.smithi148.stdout:Size : 8264254 2024-07-03T15:24:27.118 INFO:teuthology.orchestra.run.smithi148.stdout:License : GPLv3+ 2024-07-03T15:24:27.119 INFO:teuthology.orchestra.run.smithi148.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-07-03T15:24:27.119 INFO:teuthology.orchestra.run.smithi148.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-07-03T15:24:27.119 INFO:teuthology.orchestra.run.smithi148.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-07-03T15:24:27.119 INFO:teuthology.orchestra.run.smithi148.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-07-03T15:24:27.119 INFO:teuthology.orchestra.run.smithi148.stdout:Packager : builder@centos.org 2024-07-03T15:24:27.119 INFO:teuthology.orchestra.run.smithi148.stdout:Vendor : CentOS 2024-07-03T15:24:27.119 INFO:teuthology.orchestra.run.smithi148.stdout:URL : http://www.gnu.org/software/grub/ 2024-07-03T15:24:27.119 INFO:teuthology.orchestra.run.smithi148.stdout:Summary : Support tools for GRUB. 2024-07-03T15:24:27.119 INFO:teuthology.orchestra.run.smithi148.stdout:Description : 2024-07-03T15:24:27.119 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:24:27.119 INFO:teuthology.orchestra.run.smithi148.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-07-03T15:24:27.119 INFO:teuthology.orchestra.run.smithi148.stdout:customizable bootloader with modular architecture. It supports a rich 2024-07-03T15:24:27.119 INFO:teuthology.orchestra.run.smithi148.stdout:variety of kernel formats, file systems, computer architectures and 2024-07-03T15:24:27.119 INFO:teuthology.orchestra.run.smithi148.stdout:hardware devices. 2024-07-03T15:24:27.120 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:24:27.120 INFO:teuthology.orchestra.run.smithi148.stdout:This subpackage provides tools for support of all platforms. 2024-07-03T15:24:27.120 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-07-03T15:24:27.120 INFO:teuthology.task.kernel:Updating grub on smithi148 to boot 5.14.0-467.el9.x86_64 2024-07-03T15:24:27.121 DEBUG:teuthology.orchestra.run.smithi148:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-07-03T15:24:27.721 INFO:teuthology.orchestra.run.smithi148.stderr:Generating grub configuration file ... 2024-07-03T15:24:29.118 INFO:teuthology.orchestra.run.smithi148.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-07-03T15:24:29.146 INFO:teuthology.orchestra.run.smithi148.stderr:done 2024-07-03T15:24:29.148 DEBUG:teuthology.orchestra.run.smithi148:> mktemp 2024-07-03T15:24:29.162 INFO:teuthology.orchestra.run.smithi148.stdout:/tmp/tmp.hGpB3sKANf 2024-07-03T15:24:29.163 DEBUG:teuthology.orchestra.run.smithi148:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.hGpB3sKANf 2024-07-03T15:24:29.229 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 0666 /tmp/tmp.hGpB3sKANf 2024-07-03T15:24:29.365 DEBUG:teuthology.orchestra.remote:smithi148:/tmp/tmp.hGpB3sKANf is 6KB 2024-07-03T15:24:29.385 DEBUG:teuthology.orchestra.run.smithi148:> rm -fr /tmp/tmp.hGpB3sKANf 2024-07-03T15:24:29.400 DEBUG:teuthology.orchestra.run.smithi148:> sudo /bin/ls /boot/loader/entries || true 2024-07-03T15:24:29.468 INFO:teuthology.orchestra.run.smithi148.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-07-03T15:24:29.468 INFO:teuthology.orchestra.run.smithi148.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-07-03T15:24:29.468 INFO:teuthology.orchestra.run.smithi148.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-07-03T15:24:29.468 INFO:teuthology.orchestra.run.smithi148.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-467.el9.x86_64.conf 2024-07-03T15:24:29.470 DEBUG:teuthology.orchestra.run.smithi148:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-467.el9.x86_64 2024-07-03T15:24:29.615 DEBUG:teuthology.orchestra.run.smithi148:> sudo shutdown -r now 2024-07-03T15:24:32.921 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: kernel-core-5.14.0-467.el9.x86_64 4/4 2024-07-03T15:24:32.965 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: kernel-modules-5.14.0-467.el9.x86_64 4/4 2024-07-03T15:24:33.347 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: kernel-5.14.0-467.el9.x86_64 4/4 2024-07-03T15:24:33.348 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : kernel-5.14.0-467.el9.x86_64 1/4 2024-07-03T15:24:33.348 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : kernel-core-5.14.0-467.el9.x86_64 2/4 2024-07-03T15:24:33.348 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : kernel-modules-5.14.0-467.el9.x86_64 3/4 2024-07-03T15:24:34.163 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : kernel-modules-core-5.14.0-467.el9.x86_64 4/4 2024-07-03T15:24:34.163 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:24:34.163 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-07-03T15:24:34.163 INFO:teuthology.orchestra.run.smithi028.stdout: kernel-5.14.0-467.el9.x86_64 2024-07-03T15:24:34.163 INFO:teuthology.orchestra.run.smithi028.stdout: kernel-core-5.14.0-467.el9.x86_64 2024-07-03T15:24:34.163 INFO:teuthology.orchestra.run.smithi028.stdout: kernel-modules-5.14.0-467.el9.x86_64 2024-07-03T15:24:34.164 INFO:teuthology.orchestra.run.smithi028.stdout: kernel-modules-core-5.14.0-467.el9.x86_64 2024-07-03T15:24:34.164 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:24:34.164 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-07-03T15:24:34.535 DEBUG:teuthology.orchestra.run.smithi028:> rpm -q kernel | sort -rV | head -n 1 2024-07-03T15:24:34.612 INFO:teuthology.orchestra.run.smithi028.stdout:kernel-5.14.0-467.el9.x86_64 2024-07-03T15:24:34.613 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-467.el9.x86_64 2024-07-03T15:24:34.613 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-467.el9.x86_64 2024-07-03T15:24:34.613 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-07-03T15:24:34.613 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-07-03T15:24:34.613 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2024-07-03T15:24:34.613 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi028.front.sepia.ceph.com, path=None, version=distro) 2024-07-03T15:24:34.613 DEBUG:teuthology.orchestra.run.smithi028:> rpm -q kernel | sort -rV | head -n 1 2024-07-03T15:24:34.670 INFO:teuthology.orchestra.run.smithi028.stdout:kernel-5.14.0-467.el9.x86_64 2024-07-03T15:24:34.671 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-467.el9.x86_64 2024-07-03T15:24:34.671 DEBUG:teuthology.orchestra.run.smithi028:> sudo rpm -qi grub2-tools 2024-07-03T15:24:34.701 INFO:teuthology.orchestra.run.smithi028.stdout:Name : grub2-tools 2024-07-03T15:24:34.701 INFO:teuthology.orchestra.run.smithi028.stdout:Epoch : 1 2024-07-03T15:24:34.701 INFO:teuthology.orchestra.run.smithi028.stdout:Version : 2.06 2024-07-03T15:24:34.701 INFO:teuthology.orchestra.run.smithi028.stdout:Release : 68.el9 2024-07-03T15:24:34.701 INFO:teuthology.orchestra.run.smithi028.stdout:Architecture: x86_64 2024-07-03T15:24:34.701 INFO:teuthology.orchestra.run.smithi028.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-07-03T15:24:34.702 INFO:teuthology.orchestra.run.smithi028.stdout:Group : Unspecified 2024-07-03T15:24:34.702 INFO:teuthology.orchestra.run.smithi028.stdout:Size : 8264254 2024-07-03T15:24:34.702 INFO:teuthology.orchestra.run.smithi028.stdout:License : GPLv3+ 2024-07-03T15:24:34.702 INFO:teuthology.orchestra.run.smithi028.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-07-03T15:24:34.702 INFO:teuthology.orchestra.run.smithi028.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-07-03T15:24:34.702 INFO:teuthology.orchestra.run.smithi028.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-07-03T15:24:34.702 INFO:teuthology.orchestra.run.smithi028.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-07-03T15:24:34.702 INFO:teuthology.orchestra.run.smithi028.stdout:Packager : builder@centos.org 2024-07-03T15:24:34.702 INFO:teuthology.orchestra.run.smithi028.stdout:Vendor : CentOS 2024-07-03T15:24:34.702 INFO:teuthology.orchestra.run.smithi028.stdout:URL : http://www.gnu.org/software/grub/ 2024-07-03T15:24:34.702 INFO:teuthology.orchestra.run.smithi028.stdout:Summary : Support tools for GRUB. 2024-07-03T15:24:34.702 INFO:teuthology.orchestra.run.smithi028.stdout:Description : 2024-07-03T15:24:34.703 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:24:34.703 INFO:teuthology.orchestra.run.smithi028.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-07-03T15:24:34.703 INFO:teuthology.orchestra.run.smithi028.stdout:customizable bootloader with modular architecture. It supports a rich 2024-07-03T15:24:34.703 INFO:teuthology.orchestra.run.smithi028.stdout:variety of kernel formats, file systems, computer architectures and 2024-07-03T15:24:34.703 INFO:teuthology.orchestra.run.smithi028.stdout:hardware devices. 2024-07-03T15:24:34.703 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:24:34.703 INFO:teuthology.orchestra.run.smithi028.stdout:This subpackage provides tools for support of all platforms. 2024-07-03T15:24:34.704 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-07-03T15:24:34.704 INFO:teuthology.task.kernel:Updating grub on smithi028 to boot 5.14.0-467.el9.x86_64 2024-07-03T15:24:34.704 DEBUG:teuthology.orchestra.run.smithi028:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-07-03T15:24:35.312 INFO:teuthology.orchestra.run.smithi028.stderr:Generating grub configuration file ... 2024-07-03T15:24:36.684 INFO:teuthology.orchestra.run.smithi028.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-07-03T15:24:36.713 INFO:teuthology.orchestra.run.smithi028.stderr:done 2024-07-03T15:24:36.715 DEBUG:teuthology.orchestra.run.smithi028:> mktemp 2024-07-03T15:24:36.738 INFO:teuthology.orchestra.run.smithi028.stdout:/tmp/tmp.EmOF4aHtzy 2024-07-03T15:24:36.738 DEBUG:teuthology.orchestra.run.smithi028:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.EmOF4aHtzy 2024-07-03T15:24:36.804 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 0666 /tmp/tmp.EmOF4aHtzy 2024-07-03T15:24:36.936 DEBUG:teuthology.orchestra.remote:smithi028:/tmp/tmp.EmOF4aHtzy is 6KB 2024-07-03T15:24:36.958 DEBUG:teuthology.orchestra.run.smithi028:> rm -fr /tmp/tmp.EmOF4aHtzy 2024-07-03T15:24:36.973 DEBUG:teuthology.orchestra.run.smithi028:> sudo /bin/ls /boot/loader/entries || true 2024-07-03T15:24:37.036 INFO:teuthology.orchestra.run.smithi028.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-07-03T15:24:37.036 INFO:teuthology.orchestra.run.smithi028.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-07-03T15:24:37.036 INFO:teuthology.orchestra.run.smithi028.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-07-03T15:24:37.036 INFO:teuthology.orchestra.run.smithi028.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-467.el9.x86_64.conf 2024-07-03T15:24:37.038 DEBUG:teuthology.orchestra.run.smithi028:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-467.el9.x86_64 2024-07-03T15:24:37.179 DEBUG:teuthology.orchestra.run.smithi028:> sudo shutdown -r now 2024-07-03T15:24:59.638 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-07-03T15:24:59.639 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-07-03T15:24:59.640 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-03T15:25:07.190 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-07-03T15:25:07.190 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi028.front.sepia.ceph.com' 2024-07-03T15:25:07.191 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-03T15:25:17.982 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.148 2024-07-03T15:25:25.663 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.28 2024-07-03T15:25:26.984 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-07-03T15:25:26.985 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-03T15:25:30.046 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.148 2024-07-03T15:25:34.668 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi028.front.sepia.ceph.com' 2024-07-03T15:25:34.669 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-03T15:25:42.054 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-07-03T15:25:42.055 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-03T15:25:53.054 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.28 2024-07-03T15:26:05.066 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi028.front.sepia.ceph.com' 2024-07-03T15:26:05.067 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-03T15:26:05.446 DEBUG:teuthology.orchestra.run.smithi028:> true 2024-07-03T15:26:05.891 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi028.front.sepia.ceph.com' 2024-07-03T15:26:05.891 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "5.14.0-467.el9.x86_64"... 2024-07-03T15:26:05.892 DEBUG:teuthology.orchestra.run.smithi028:> uname -r 2024-07-03T15:26:05.907 INFO:teuthology.orchestra.run.smithi028.stdout:5.14.0-467.el9.x86_64 2024-07-03T15:26:05.907 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-467.el9.x86_64 vs 5.14.0-467.el9.x86_64 2024-07-03T15:26:05.907 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-07-03T15:26:05.907 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-07-03T15:26:06.908 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-07-03T15:26:06.908 DEBUG:teuthology.orchestra.run.smithi028:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-07-03T15:26:06.988 INFO:teuthology.orchestra.run.smithi028.stdout:ttyS1 2024-07-03T15:26:07.026 DEBUG:teuthology.parallel:result is None 2024-07-03T15:26:42.057 DEBUG:teuthology.orchestra.remote:timed out 2024-07-03T15:26:57.058 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-07-03T15:26:57.060 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-03T15:26:57.480 DEBUG:teuthology.orchestra.run.smithi148:> true 2024-07-03T15:26:57.812 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-07-03T15:26:57.812 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "5.14.0-467.el9.x86_64"... 2024-07-03T15:26:57.812 DEBUG:teuthology.orchestra.run.smithi148:> uname -r 2024-07-03T15:26:57.828 INFO:teuthology.orchestra.run.smithi148.stdout:5.14.0-467.el9.x86_64 2024-07-03T15:26:57.828 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-467.el9.x86_64 vs 5.14.0-467.el9.x86_64 2024-07-03T15:26:57.828 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-07-03T15:26:57.828 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-07-03T15:26:58.830 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-07-03T15:26:58.830 DEBUG:teuthology.orchestra.run.smithi148:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-07-03T15:26:58.912 INFO:teuthology.orchestra.run.smithi148.stdout:ttyS1 2024-07-03T15:26:58.948 DEBUG:teuthology.parallel:result is None 2024-07-03T15:26:58.948 INFO:teuthology.run_tasks:Running task internal.base... 2024-07-03T15:26:58.956 INFO:teuthology.task.internal:Creating test directory... 2024-07-03T15:26:58.956 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-07-03T15:26:58.960 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-07-03T15:26:58.977 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-07-03T15:26:58.987 INFO:teuthology.run_tasks:Running task internal.archive... 2024-07-03T15:26:59.004 INFO:teuthology.task.internal:Creating archive directory... 2024-07-03T15:26:59.004 DEBUG:teuthology.orchestra.run.smithi028:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-07-03T15:26:59.019 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-07-03T15:26:59.061 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-07-03T15:26:59.068 INFO:teuthology.task.internal:Enabling coredump saving... 2024-07-03T15:26:59.069 DEBUG:teuthology.orchestra.run.smithi028:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-07-03T15:26:59.095 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-07-03T15:26:59.131 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-03T15:26:59.132 INFO:teuthology.orchestra.run.smithi028.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-03T15:26:59.142 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-03T15:26:59.144 INFO:teuthology.orchestra.run.smithi028.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-03T15:26:59.146 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-07-03T15:26:59.153 INFO:teuthology.task.internal:Configuring sudo... 2024-07-03T15:26:59.153 DEBUG:teuthology.orchestra.run.smithi028:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-07-03T15:26:59.188 DEBUG:teuthology.orchestra.run.smithi148:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-07-03T15:26:59.216 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-07-03T15:26:59.225 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-07-03T15:26:59.225 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-07-03T15:26:59.258 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-07-03T15:26:59.275 DEBUG:teuthology.orchestra.run.smithi028:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-03T15:26:59.359 DEBUG:teuthology.orchestra.run.smithi028:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-03T15:26:59.406 DEBUG:teuthology.orchestra.run.smithi028:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-03T15:26:59.493 DEBUG:teuthology.orchestra.run.smithi028:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-03T15:26:59.536 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-07-03T15:26:59.536 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-07-03T15:26:59.602 DEBUG:teuthology.orchestra.run.smithi148:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-03T15:26:59.643 DEBUG:teuthology.orchestra.run.smithi148:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-03T15:26:59.705 DEBUG:teuthology.orchestra.run.smithi148:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-03T15:26:59.785 DEBUG:teuthology.orchestra.run.smithi148:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-03T15:26:59.833 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-07-03T15:26:59.833 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-07-03T15:26:59.901 DEBUG:teuthology.orchestra.run.smithi028:> sudo service rsyslog restart 2024-07-03T15:26:59.904 DEBUG:teuthology.orchestra.run.smithi148:> sudo service rsyslog restart 2024-07-03T15:26:59.950 INFO:teuthology.orchestra.run.smithi028.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-03T15:26:59.980 INFO:teuthology.orchestra.run.smithi148.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-03T15:27:00.405 INFO:teuthology.run_tasks:Running task internal.timer... 2024-07-03T15:27:00.415 INFO:teuthology.task.internal:Starting timer... 2024-07-03T15:27:00.415 INFO:teuthology.run_tasks:Running task pcp... 2024-07-03T15:27:00.440 INFO:teuthology.run_tasks:Running task selinux... 2024-07-03T15:27:00.449 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-07-03T15:27:00.450 DEBUG:teuthology.orchestra.run.smithi028:> sudo service auditd rotate 2024-07-03T15:27:00.533 INFO:teuthology.orchestra.run.smithi028.stdout:Rotating logs: 2024-07-03T15:27:00.535 DEBUG:teuthology.orchestra.run.smithi148:> sudo service auditd rotate 2024-07-03T15:27:00.582 INFO:teuthology.orchestra.run.smithi148.stdout:Rotating logs: 2024-07-03T15:27:00.583 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-07-03T15:27:00.584 DEBUG:teuthology.orchestra.run.smithi028:> /usr/sbin/getenforce 2024-07-03T15:27:00.626 INFO:teuthology.orchestra.run.smithi028.stdout:Permissive 2024-07-03T15:27:00.626 DEBUG:teuthology.orchestra.run.smithi148:> /usr/sbin/getenforce 2024-07-03T15:27:00.661 INFO:teuthology.orchestra.run.smithi148.stdout:Permissive 2024-07-03T15:27:00.661 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi028.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi148.front.sepia.ceph.com': 'permissive'} 2024-07-03T15:27:00.661 DEBUG:teuthology.orchestra.run.smithi028:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-03T15:27:00.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-03T15:27:00.695 DEBUG:teuthology.orchestra.run.smithi148:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-03T15:27:00.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-03T15:27:00.729 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-07-03T15:27:00.729 DEBUG:teuthology.orchestra.run.smithi028:> sudo /usr/sbin/setenforce permissive 2024-07-03T15:27:00.764 DEBUG:teuthology.orchestra.run.smithi148:> sudo /usr/sbin/setenforce permissive 2024-07-03T15:27:00.805 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-07-03T15:27:00.814 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-07-03T15:27:00.825 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-07-03T15:27:00.826 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi028.front.sepia.ceph.com,smithi148.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-07-03T15:31:30.434 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi028.front.sepia.ceph.com'), Remote(name='ubuntu@smithi148.front.sepia.ceph.com')] 2024-07-03T15:31:30.435 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi028.front.sepia.ceph.com' 2024-07-03T15:31:30.437 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-03T15:31:30.515 DEBUG:teuthology.orchestra.run.smithi028:> true 2024-07-03T15:31:30.590 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi028.front.sepia.ceph.com' 2024-07-03T15:31:30.590 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-07-03T15:31:30.591 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-03T15:31:30.667 DEBUG:teuthology.orchestra.run.smithi148:> true 2024-07-03T15:31:30.745 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-07-03T15:31:30.745 INFO:teuthology.run_tasks:Running task clock... 2024-07-03T15:31:30.756 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-07-03T15:31:30.756 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-07-03T15:31:30.756 DEBUG:teuthology.orchestra.run.smithi028:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-03T15:31:30.759 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-07-03T15:31:30.759 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-03T15:31:30.789 INFO:teuthology.orchestra.run.smithi028.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-07-03T15:31:30.804 INFO:teuthology.orchestra.run.smithi028.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-07-03T15:31:30.814 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-07-03T15:31:30.828 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-07-03T15:31:30.830 INFO:teuthology.orchestra.run.smithi028.stderr:sudo: ntpd: command not found 2024-07-03T15:31:30.842 INFO:teuthology.orchestra.run.smithi028.stdout:506 Cannot talk to daemon 2024-07-03T15:31:30.852 INFO:teuthology.orchestra.run.smithi148.stderr:sudo: ntpd: command not found 2024-07-03T15:31:30.857 INFO:teuthology.orchestra.run.smithi028.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-07-03T15:31:30.863 INFO:teuthology.orchestra.run.smithi148.stdout:506 Cannot talk to daemon 2024-07-03T15:31:30.871 INFO:teuthology.orchestra.run.smithi028.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-07-03T15:31:30.877 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-07-03T15:31:30.891 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-07-03T15:31:30.930 INFO:teuthology.orchestra.run.smithi028.stderr:bash: line 1: ntpq: command not found 2024-07-03T15:31:30.933 INFO:teuthology.orchestra.run.smithi028.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-03T15:31:30.933 INFO:teuthology.orchestra.run.smithi028.stdout:=============================================================================== 2024-07-03T15:31:30.933 INFO:teuthology.orchestra.run.smithi028.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-03T15:31:30.933 INFO:teuthology.orchestra.run.smithi028.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-03T15:31:30.933 INFO:teuthology.orchestra.run.smithi028.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-03T15:31:30.933 INFO:teuthology.orchestra.run.smithi028.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-03T15:31:30.949 INFO:teuthology.orchestra.run.smithi148.stderr:bash: line 1: ntpq: command not found 2024-07-03T15:31:30.952 INFO:teuthology.orchestra.run.smithi148.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-03T15:31:30.952 INFO:teuthology.orchestra.run.smithi148.stdout:=============================================================================== 2024-07-03T15:31:30.952 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-03T15:31:30.952 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-03T15:31:30.953 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-03T15:31:30.953 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-03T15:31:30.953 INFO:teuthology.run_tasks:Running task install... 2024-07-03T15:31:30.962 DEBUG:teuthology.task.install:project ceph 2024-07-03T15:31:30.962 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'c00506dc10003928f8cb40769168571f611cbd67'}} 2024-07-03T15:31:30.962 DEBUG:teuthology.task.install:config {'branch': 'quincy', 'exclude_packages': ['ceph-volume'], 'flavor': 'default', 'sha1': 'c00506dc10003928f8cb40769168571f611cbd67'} 2024-07-03T15:31:30.962 INFO:teuthology.task.install:Using flavor: default 2024-07-03T15:31:30.967 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-07-03T15:31:30.967 INFO:teuthology.task.install:extra packages: [] 2024-07-03T15:31:30.968 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c00506dc10003928f8cb40769168571f611cbd67', 'tag': None, 'wait_for_package': False} 2024-07-03T15:31:30.968 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-07-03T15:31:30.969 INFO:teuthology.packaging:ref: None 2024-07-03T15:31:30.969 INFO:teuthology.packaging:tag: None 2024-07-03T15:31:30.969 INFO:teuthology.packaging:branch: quincy 2024-07-03T15:31:30.970 INFO:teuthology.packaging:sha1: c00506dc10003928f8cb40769168571f611cbd67 2024-07-03T15:31:30.970 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-07-03T15:31:30.972 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c00506dc10003928f8cb40769168571f611cbd67', 'tag': None, 'wait_for_package': False} 2024-07-03T15:31:30.972 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-07-03T15:31:30.972 INFO:teuthology.packaging:ref: None 2024-07-03T15:31:30.972 INFO:teuthology.packaging:tag: None 2024-07-03T15:31:30.972 INFO:teuthology.packaging:branch: quincy 2024-07-03T15:31:30.972 INFO:teuthology.packaging:sha1: c00506dc10003928f8cb40769168571f611cbd67 2024-07-03T15:31:30.972 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-07-03T15:31:31.173 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/quincy/c602137526bb0ca8affc8939bb917d8d9bdb3a36/centos/9/flavors/default/ 2024-07-03T15:31:31.174 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1045.gc6021375 2024-07-03T15:31:31.282 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/quincy/c602137526bb0ca8affc8939bb917d8d9bdb3a36/centos/9/flavors/default/ 2024-07-03T15:31:31.282 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1045.gc6021375 2024-07-03T15:31:31.345 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/quincy/c602137526bb0ca8affc8939bb917d8d9bdb3a36/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/c602137526bb0ca8affc8939bb917d8d9bdb3a36/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/c602137526bb0ca8affc8939bb917d8d9bdb3a36/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-07-03T15:31:31.345 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-07-03T15:31:31.345 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-07-03T15:31:31.372 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-07-03T15:31:31.372 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-07-03T15:31:31.372 INFO:teuthology.packaging:ref: None 2024-07-03T15:31:31.372 INFO:teuthology.packaging:tag: None 2024-07-03T15:31:31.372 INFO:teuthology.packaging:branch: quincy 2024-07-03T15:31:31.372 INFO:teuthology.packaging:sha1: c00506dc10003928f8cb40769168571f611cbd67 2024-07-03T15:31:31.372 DEBUG:teuthology.orchestra.run.smithi148:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-07-03T15:31:31.438 DEBUG:teuthology.orchestra.run.smithi148:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-07-03T15:31:31.451 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/quincy/c602137526bb0ca8affc8939bb917d8d9bdb3a36/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/c602137526bb0ca8affc8939bb917d8d9bdb3a36/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/c602137526bb0ca8affc8939bb917d8d9bdb3a36/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-07-03T15:31:31.452 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-07-03T15:31:31.452 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-07-03T15:31:31.481 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-07-03T15:31:31.481 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-07-03T15:31:31.481 INFO:teuthology.packaging:ref: None 2024-07-03T15:31:31.481 INFO:teuthology.packaging:tag: None 2024-07-03T15:31:31.481 INFO:teuthology.packaging:branch: quincy 2024-07-03T15:31:31.481 INFO:teuthology.packaging:sha1: c00506dc10003928f8cb40769168571f611cbd67 2024-07-03T15:31:31.481 DEBUG:teuthology.orchestra.run.smithi028:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-07-03T15:31:31.513 DEBUG:teuthology.orchestra.run.smithi148:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-07-03T15:31:31.551 DEBUG:teuthology.orchestra.run.smithi028:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-07-03T15:31:31.577 INFO:teuthology.orchestra.run.smithi148.stdout:check_obsoletes = 1 2024-07-03T15:31:31.578 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum clean all 2024-07-03T15:31:31.632 DEBUG:teuthology.orchestra.run.smithi028:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-07-03T15:31:31.698 INFO:teuthology.orchestra.run.smithi028.stdout:check_obsoletes = 1 2024-07-03T15:31:31.700 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum clean all 2024-07-03T15:31:31.807 INFO:teuthology.orchestra.run.smithi148.stdout:45 files removed 2024-07-03T15:31:31.834 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-radosgw 2024-07-03T15:31:31.951 INFO:teuthology.orchestra.run.smithi028.stdout:45 files removed 2024-07-03T15:31:31.978 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install ceph-radosgw 2024-07-03T15:31:32.458 INFO:teuthology.orchestra.run.smithi148.stdout:ceph packages for x86_64 223 kB/s | 77 kB 00:00 2024-07-03T15:31:32.644 INFO:teuthology.orchestra.run.smithi028.stdout:ceph packages for x86_64 207 kB/s | 77 kB 00:00 2024-07-03T15:31:32.942 INFO:teuthology.orchestra.run.smithi148.stdout:ceph noarch packages 25 kB/s | 11 kB 00:00 2024-07-03T15:31:33.020 INFO:teuthology.orchestra.run.smithi028.stdout:ceph noarch packages 33 kB/s | 11 kB 00:00 2024-07-03T15:31:33.334 INFO:teuthology.orchestra.run.smithi148.stdout:ceph source packages 5.1 kB/s | 1.8 kB 00:00 2024-07-03T15:31:33.429 INFO:teuthology.orchestra.run.smithi028.stdout:ceph source packages 4.9 kB/s | 1.8 kB 00:00 2024-07-03T15:31:34.034 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.1 MB 00:00 2024-07-03T15:31:34.929 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - BaseOS 5.6 MB/s | 8.1 MB 00:01 2024-07-03T15:31:37.947 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - AppStream 11 MB/s | 20 MB 00:01 2024-07-03T15:31:40.110 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - AppStream 4.0 MB/s | 20 MB 00:04 2024-07-03T15:31:43.622 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - CRB 3.5 MB/s | 6.4 MB 00:01 2024-07-03T15:31:45.224 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - Extras packages 56 kB/s | 17 kB 00:00 2024-07-03T15:31:45.919 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - CRB 3.0 MB/s | 6.4 MB 00:02 2024-07-03T15:31:46.086 INFO:teuthology.orchestra.run.smithi028.stdout:Extra Packages for Enterprise Linux 31 MB/s | 22 MB 00:00 2024-07-03T15:31:47.486 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - Extras packages 48 kB/s | 17 kB 00:00 2024-07-03T15:31:48.465 INFO:teuthology.orchestra.run.smithi148.stdout:Extra Packages for Enterprise Linux 26 MB/s | 22 MB 00:00 2024-07-03T15:31:51.692 INFO:teuthology.orchestra.run.smithi028.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-07-03T15:31:53.630 INFO:teuthology.orchestra.run.smithi148.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-07-03T15:31:54.303 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-07-03T15:31:54.317 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-07-03T15:31:54.317 INFO:teuthology.orchestra.run.smithi028.stdout: Package Arch Version Repository Size 2024-07-03T15:31:54.317 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-07-03T15:31:54.317 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-07-03T15:31:54.317 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-radosgw x86_64 2:17.2.7-1045.gc6021375.el9 ceph 14 M 2024-07-03T15:31:54.317 INFO:teuthology.orchestra.run.smithi028.stdout:Upgrading: 2024-07-03T15:31:54.317 INFO:teuthology.orchestra.run.smithi028.stdout: librados2 x86_64 2:17.2.7-1045.gc6021375.el9 ceph 3.1 M 2024-07-03T15:31:54.317 INFO:teuthology.orchestra.run.smithi028.stdout: librbd1 x86_64 2:17.2.7-1045.gc6021375.el9 ceph 2.9 M 2024-07-03T15:31:54.318 INFO:teuthology.orchestra.run.smithi028.stdout:Installing dependencies: 2024-07-03T15:31:54.318 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-base x86_64 2:17.2.7-1045.gc6021375.el9 ceph 5.0 M 2024-07-03T15:31:54.318 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-common x86_64 2:17.2.7-1045.gc6021375.el9 ceph 22 M 2024-07-03T15:31:54.318 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-selinux x86_64 2:17.2.7-1045.gc6021375.el9 ceph 25 k 2024-07-03T15:31:54.318 INFO:teuthology.orchestra.run.smithi028.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-07-03T15:31:54.318 INFO:teuthology.orchestra.run.smithi028.stdout: libcephfs2 x86_64 2:17.2.7-1045.gc6021375.el9 ceph 636 k 2024-07-03T15:31:54.318 INFO:teuthology.orchestra.run.smithi028.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-07-03T15:31:54.318 INFO:teuthology.orchestra.run.smithi028.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-07-03T15:31:54.318 INFO:teuthology.orchestra.run.smithi028.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-07-03T15:31:54.318 INFO:teuthology.orchestra.run.smithi028.stdout: libradosstriper1 x86_64 2:17.2.7-1045.gc6021375.el9 ceph 457 k 2024-07-03T15:31:54.319 INFO:teuthology.orchestra.run.smithi028.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-07-03T15:31:54.319 INFO:teuthology.orchestra.run.smithi028.stdout: librgw2 x86_64 2:17.2.7-1045.gc6021375.el9 ceph 4.9 M 2024-07-03T15:31:54.319 INFO:teuthology.orchestra.run.smithi028.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-07-03T15:31:54.319 INFO:teuthology.orchestra.run.smithi028.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-07-03T15:31:54.319 INFO:teuthology.orchestra.run.smithi028.stdout: python3-ceph-argparse x86_64 2:17.2.7-1045.gc6021375.el9 ceph 45 k 2024-07-03T15:31:54.319 INFO:teuthology.orchestra.run.smithi028.stdout: python3-ceph-common x86_64 2:17.2.7-1045.gc6021375.el9 ceph 118 k 2024-07-03T15:31:54.319 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cephfs x86_64 2:17.2.7-1045.gc6021375.el9 ceph 156 k 2024-07-03T15:31:54.319 INFO:teuthology.orchestra.run.smithi028.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-07-03T15:31:54.319 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rados x86_64 2:17.2.7-1045.gc6021375.el9 ceph 320 k 2024-07-03T15:31:54.319 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rbd x86_64 2:17.2.7-1045.gc6021375.el9 ceph 293 k 2024-07-03T15:31:54.320 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rgw x86_64 2:17.2.7-1045.gc6021375.el9 ceph 99 k 2024-07-03T15:31:54.320 INFO:teuthology.orchestra.run.smithi028.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-07-03T15:31:54.320 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:31:54.320 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-07-03T15:31:54.320 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-07-03T15:31:54.320 INFO:teuthology.orchestra.run.smithi028.stdout:Install 22 Packages 2024-07-03T15:31:54.320 INFO:teuthology.orchestra.run.smithi028.stdout:Upgrade 2 Packages 2024-07-03T15:31:54.320 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:31:54.321 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 58 M 2024-07-03T15:31:54.321 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-07-03T15:31:55.310 INFO:teuthology.orchestra.run.smithi028.stdout:(1/24): ceph-base-17.2.7-1045.gc6021375.el9.x86 10 MB/s | 5.0 MB 00:00 2024-07-03T15:31:55.377 INFO:teuthology.orchestra.run.smithi028.stdout:(2/24): ceph-selinux-17.2.7-1045.gc6021375.el9. 378 kB/s | 25 kB 00:00 2024-07-03T15:31:55.469 INFO:teuthology.orchestra.run.smithi028.stdout:(3/24): libcephfs2-17.2.7-1045.gc6021375.el9.x8 6.8 MB/s | 636 kB 00:00 2024-07-03T15:31:55.552 INFO:teuthology.orchestra.run.smithi028.stdout:(4/24): libradosstriper1-17.2.7-1045.gc6021375. 5.4 MB/s | 457 kB 00:00 2024-07-03T15:31:55.728 INFO:teuthology.orchestra.run.smithi028.stdout:(5/24): ceph-radosgw-17.2.7-1045.gc6021375.el9. 16 MB/s | 14 MB 00:00 2024-07-03T15:31:55.770 INFO:teuthology.orchestra.run.smithi028.stdout:(6/24): python3-ceph-argparse-17.2.7-1045.gc602 1.1 MB/s | 45 kB 00:00 2024-07-03T15:31:55.845 INFO:teuthology.orchestra.run.smithi028.stdout:(7/24): librgw2-17.2.7-1045.gc6021375.el9.x86_6 17 MB/s | 4.9 MB 00:00 2024-07-03T15:31:55.870 INFO:teuthology.orchestra.run.smithi028.stdout:(8/24): python3-ceph-common-17.2.7-1045.gc60213 1.1 MB/s | 118 kB 00:00 2024-07-03T15:31:55.895 INFO:teuthology.orchestra.run.smithi028.stdout:(9/24): python3-cephfs-17.2.7-1045.gc6021375.el 3.1 MB/s | 156 kB 00:00 2024-07-03T15:31:55.921 INFO:teuthology.orchestra.run.smithi028.stdout:(10/24): python3-rados-17.2.7-1045.gc6021375.el 6.2 MB/s | 320 kB 00:00 2024-07-03T15:31:55.946 INFO:teuthology.orchestra.run.smithi028.stdout:(11/24): python3-rbd-17.2.7-1045.gc6021375.el9. 5.7 MB/s | 293 kB 00:00 2024-07-03T15:31:55.971 INFO:teuthology.orchestra.run.smithi028.stdout:(12/24): python3-rgw-17.2.7-1045.gc6021375.el9. 1.9 MB/s | 99 kB 00:00 2024-07-03T15:31:56.112 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-07-03T15:31:56.127 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-07-03T15:31:56.128 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-07-03T15:31:56.128 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-07-03T15:31:56.128 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-07-03T15:31:56.128 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-radosgw x86_64 2:17.2.7-1045.gc6021375.el9 ceph 14 M 2024-07-03T15:31:56.128 INFO:teuthology.orchestra.run.smithi148.stdout:Upgrading: 2024-07-03T15:31:56.128 INFO:teuthology.orchestra.run.smithi148.stdout: librados2 x86_64 2:17.2.7-1045.gc6021375.el9 ceph 3.1 M 2024-07-03T15:31:56.128 INFO:teuthology.orchestra.run.smithi148.stdout: librbd1 x86_64 2:17.2.7-1045.gc6021375.el9 ceph 2.9 M 2024-07-03T15:31:56.128 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-07-03T15:31:56.128 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-base x86_64 2:17.2.7-1045.gc6021375.el9 ceph 5.0 M 2024-07-03T15:31:56.128 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-common x86_64 2:17.2.7-1045.gc6021375.el9 ceph 22 M 2024-07-03T15:31:56.128 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-selinux x86_64 2:17.2.7-1045.gc6021375.el9 ceph 25 k 2024-07-03T15:31:56.128 INFO:teuthology.orchestra.run.smithi148.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-07-03T15:31:56.128 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs2 x86_64 2:17.2.7-1045.gc6021375.el9 ceph 636 k 2024-07-03T15:31:56.129 INFO:teuthology.orchestra.run.smithi148.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-07-03T15:31:56.129 INFO:teuthology.orchestra.run.smithi148.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-07-03T15:31:56.129 INFO:teuthology.orchestra.run.smithi148.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-07-03T15:31:56.129 INFO:teuthology.orchestra.run.smithi148.stdout: libradosstriper1 x86_64 2:17.2.7-1045.gc6021375.el9 ceph 457 k 2024-07-03T15:31:56.129 INFO:teuthology.orchestra.run.smithi148.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-07-03T15:31:56.129 INFO:teuthology.orchestra.run.smithi148.stdout: librgw2 x86_64 2:17.2.7-1045.gc6021375.el9 ceph 4.9 M 2024-07-03T15:31:56.129 INFO:teuthology.orchestra.run.smithi148.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-07-03T15:31:56.129 INFO:teuthology.orchestra.run.smithi148.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-07-03T15:31:56.129 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-argparse x86_64 2:17.2.7-1045.gc6021375.el9 ceph 45 k 2024-07-03T15:31:56.129 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-common x86_64 2:17.2.7-1045.gc6021375.el9 ceph 118 k 2024-07-03T15:31:56.129 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cephfs x86_64 2:17.2.7-1045.gc6021375.el9 ceph 156 k 2024-07-03T15:31:56.129 INFO:teuthology.orchestra.run.smithi148.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-07-03T15:31:56.130 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rados x86_64 2:17.2.7-1045.gc6021375.el9 ceph 320 k 2024-07-03T15:31:56.130 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rbd x86_64 2:17.2.7-1045.gc6021375.el9 ceph 293 k 2024-07-03T15:31:56.130 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rgw x86_64 2:17.2.7-1045.gc6021375.el9 ceph 99 k 2024-07-03T15:31:56.130 INFO:teuthology.orchestra.run.smithi148.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-07-03T15:31:56.130 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:31:56.130 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-07-03T15:31:56.130 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-07-03T15:31:56.130 INFO:teuthology.orchestra.run.smithi148.stdout:Install 22 Packages 2024-07-03T15:31:56.130 INFO:teuthology.orchestra.run.smithi148.stdout:Upgrade 2 Packages 2024-07-03T15:31:56.130 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:31:56.131 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 58 M 2024-07-03T15:31:56.131 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-07-03T15:31:56.255 INFO:teuthology.orchestra.run.smithi028.stdout:(13/24): ceph-common-17.2.7-1045.gc6021375.el9. 16 MB/s | 22 MB 00:01 2024-07-03T15:31:56.797 INFO:teuthology.orchestra.run.smithi028.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 55 kB/s | 45 kB 00:00 2024-07-03T15:31:56.864 INFO:teuthology.orchestra.run.smithi028.stdout:(15/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 175 kB/s | 160 kB 00:00 2024-07-03T15:31:57.189 INFO:teuthology.orchestra.run.smithi028.stdout:(16/24): librdkafka-1.6.1-102.el9.x86_64.rpm 710 kB/s | 662 kB 00:00 2024-07-03T15:31:57.264 INFO:teuthology.orchestra.run.smithi028.stdout:(17/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 4.0 MB/s | 309 kB 00:00 2024-07-03T15:31:57.289 INFO:teuthology.orchestra.run.smithi028.stdout:(18/24): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-07-03T15:31:57.315 INFO:teuthology.orchestra.run.smithi028.stdout:(19/24): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-07-03T15:31:57.373 INFO:teuthology.orchestra.run.smithi028.stdout:(20/24): thrift-0.15.0-2.el9.x86_64.rpm 27 MB/s | 1.6 MB 00:00 2024-07-03T15:31:57.439 INFO:teuthology.orchestra.run.smithi148.stdout:(1/24): ceph-common-17.2.7-1045.gc6021375.el9.x 24 MB/s | 22 MB 00:00 2024-07-03T15:31:57.489 INFO:teuthology.orchestra.run.smithi148.stdout:(2/24): ceph-selinux-17.2.7-1045.gc6021375.el9. 508 kB/s | 25 kB 00:00 2024-07-03T15:31:57.498 INFO:teuthology.orchestra.run.smithi028.stdout:(21/24): python3-prettytable-0.7.2-27.el9.noarc 67 kB/s | 42 kB 00:00 2024-07-03T15:31:57.548 INFO:teuthology.orchestra.run.smithi148.stdout:(3/24): libcephfs2-17.2.7-1045.gc6021375.el9.x8 11 MB/s | 636 kB 00:00 2024-07-03T15:31:57.624 INFO:teuthology.orchestra.run.smithi028.stdout:(22/24): librados2-17.2.7-1045.gc6021375.el9.x8 12 MB/s | 3.1 MB 00:00 2024-07-03T15:31:57.640 INFO:teuthology.orchestra.run.smithi148.stdout:(4/24): ceph-base-17.2.7-1045.gc6021375.el9.x86 4.3 MB/s | 5.0 MB 00:01 2024-07-03T15:31:57.658 INFO:teuthology.orchestra.run.smithi028.stdout:(23/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 340 kB/s | 292 kB 00:00 2024-07-03T15:31:57.665 INFO:teuthology.orchestra.run.smithi148.stdout:(5/24): libradosstriper1-17.2.7-1045.gc6021375. 3.8 MB/s | 457 kB 00:00 2024-07-03T15:31:57.707 INFO:teuthology.orchestra.run.smithi148.stdout:(6/24): python3-ceph-argparse-17.2.7-1045.gc602 1.1 MB/s | 45 kB 00:00 2024-07-03T15:31:57.757 INFO:teuthology.orchestra.run.smithi148.stdout:(7/24): python3-ceph-common-17.2.7-1045.gc60213 2.3 MB/s | 118 kB 00:00 2024-07-03T15:31:57.800 INFO:teuthology.orchestra.run.smithi028.stdout:(24/24): librbd1-17.2.7-1045.gc6021375.el9.x86_ 9.8 MB/s | 2.9 MB 00:00 2024-07-03T15:31:57.804 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-07-03T15:31:57.804 INFO:teuthology.orchestra.run.smithi028.stdout:Total 17 MB/s | 58 MB 00:03 2024-07-03T15:31:57.807 INFO:teuthology.orchestra.run.smithi148.stdout:(8/24): python3-cephfs-17.2.7-1045.gc6021375.el 3.1 MB/s | 156 kB 00:00 2024-07-03T15:31:57.866 INFO:teuthology.orchestra.run.smithi148.stdout:(9/24): python3-rados-17.2.7-1045.gc6021375.el9 5.4 MB/s | 320 kB 00:00 2024-07-03T15:31:57.872 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-07-03T15:31:57.911 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-07-03T15:31:57.911 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-07-03T15:31:57.925 INFO:teuthology.orchestra.run.smithi148.stdout:(10/24): python3-rbd-17.2.7-1045.gc6021375.el9. 4.9 MB/s | 293 kB 00:00 2024-07-03T15:31:57.984 INFO:teuthology.orchestra.run.smithi148.stdout:(11/24): python3-rgw-17.2.7-1045.gc6021375.el9. 1.7 MB/s | 99 kB 00:00 2024-07-03T15:31:58.125 INFO:teuthology.orchestra.run.smithi148.stdout:(12/24): librgw2-17.2.7-1045.gc6021375.el9.x86_ 10 MB/s | 4.9 MB 00:00 2024-07-03T15:31:58.192 INFO:teuthology.orchestra.run.smithi148.stdout:(13/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 768 kB/s | 160 kB 00:00 2024-07-03T15:31:58.268 INFO:teuthology.orchestra.run.smithi148.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 319 kB/s | 45 kB 00:00 2024-07-03T15:31:58.343 INFO:teuthology.orchestra.run.smithi148.stdout:(15/24): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-07-03T15:31:58.402 INFO:teuthology.orchestra.run.smithi148.stdout:(16/24): python3-prettytable-0.7.2-27.el9.noarc 725 kB/s | 42 kB 00:00 2024-07-03T15:31:58.442 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-07-03T15:31:58.442 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-07-03T15:31:58.452 INFO:teuthology.orchestra.run.smithi148.stdout:(17/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.6 MB/s | 292 kB 00:00 2024-07-03T15:31:58.510 INFO:teuthology.orchestra.run.smithi148.stdout:(18/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 2.8 MB/s | 309 kB 00:00 2024-07-03T15:31:58.536 INFO:teuthology.orchestra.run.smithi148.stdout:(19/24): libunwind-1.6.2-1.el9.x86_64.rpm 2.9 MB/s | 67 kB 00:00 2024-07-03T15:31:58.561 INFO:teuthology.orchestra.run.smithi148.stdout:(20/24): liboath-2.6.7-2.el9.x86_64.rpm 456 kB/s | 49 kB 00:00 2024-07-03T15:31:58.711 INFO:teuthology.orchestra.run.smithi148.stdout:(21/24): ceph-radosgw-17.2.7-1045.gc6021375.el9 6.3 MB/s | 14 MB 00:02 2024-07-03T15:31:58.753 INFO:teuthology.orchestra.run.smithi148.stdout:(22/24): thrift-0.15.0-2.el9.x86_64.rpm 7.3 MB/s | 1.6 MB 00:00 2024-07-03T15:31:58.887 INFO:teuthology.orchestra.run.smithi148.stdout:(23/24): librbd1-17.2.7-1045.gc6021375.el9.x86_ 17 MB/s | 2.9 MB 00:00 2024-07-03T15:31:58.937 INFO:teuthology.orchestra.run.smithi148.stdout:(24/24): librados2-17.2.7-1045.gc6021375.el9.x8 8.2 MB/s | 3.1 MB 00:00 2024-07-03T15:31:58.942 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-07-03T15:31:58.942 INFO:teuthology.orchestra.run.smithi148.stdout:Total 21 MB/s | 58 MB 00:02 2024-07-03T15:31:59.004 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-07-03T15:31:59.037 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-07-03T15:31:59.037 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-07-03T15:31:59.583 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-07-03T15:31:59.584 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-07-03T15:31:59.647 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-07-03T15:32:00.049 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-07-03T15:32:00.119 INFO:teuthology.orchestra.run.smithi028.stdout: Upgrading : librados2-2:17.2.7-1045.gc6021375.el9.x86_64 2/26 2024-07-03T15:32:00.212 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: librados2-2:17.2.7-1045.gc6021375.el9.x86_64 2/26 2024-07-03T15:32:00.293 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-rados-2:17.2.7-1045.gc6021375.el9.x86_64 3/26 2024-07-03T15:32:00.328 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libcephfs2-2:17.2.7-1045.gc6021375.el9.x86_64 4/26 2024-07-03T15:32:00.429 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: libcephfs2-2:17.2.7-1045.gc6021375.el9.x86_64 4/26 2024-07-03T15:32:00.480 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-07-03T15:32:00.517 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-07-03T15:32:00.779 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-07-03T15:32:00.975 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-07-03T15:32:00.998 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : librgw2-2:17.2.7-1045.gc6021375.el9.x86_64 8/26 2024-07-03T15:32:01.044 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: librgw2-2:17.2.7-1045.gc6021375.el9.x86_64 8/26 2024-07-03T15:32:01.081 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-07-03T15:32:01.121 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-07-03T15:32:01.125 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-ceph-argparse-2:17.2.7-1045.gc6021375.el9. 10/26 2024-07-03T15:32:01.161 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-cephfs-2:17.2.7-1045.gc6021375.el9.x86_64 11/26 2024-07-03T15:32:01.217 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-rgw-2:17.2.7-1045.gc6021375.el9.x86_64 12/26 2024-07-03T15:32:01.225 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : librados2-2:17.2.7-1045.gc6021375.el9.x86_64 2/26 2024-07-03T15:32:01.242 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libradosstriper1-2:17.2.7-1045.gc6021375.el9.x86_6 13/26 2024-07-03T15:32:01.292 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1045.gc6021375.el9.x86_6 13/26 2024-07-03T15:32:01.318 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librados2-2:17.2.7-1045.gc6021375.el9.x86_64 2/26 2024-07-03T15:32:01.333 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-07-03T15:32:01.368 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-07-03T15:32:01.400 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-07-03T15:32:01.401 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rados-2:17.2.7-1045.gc6021375.el9.x86_64 3/26 2024-07-03T15:32:01.427 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephfs2-2:17.2.7-1045.gc6021375.el9.x86_64 4/26 2024-07-03T15:32:01.501 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libcephfs2-2:17.2.7-1045.gc6021375.el9.x86_64 4/26 2024-07-03T15:32:01.545 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-07-03T15:32:01.582 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-07-03T15:32:01.722 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-07-03T15:32:01.771 INFO:teuthology.orchestra.run.smithi028.stdout: Upgrading : librbd1-2:17.2.7-1045.gc6021375.el9.x86_64 18/26 2024-07-03T15:32:01.854 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: librbd1-2:17.2.7-1045.gc6021375.el9.x86_64 18/26 2024-07-03T15:32:01.930 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-rbd-2:17.2.7-1045.gc6021375.el9.x86_64 19/26 2024-07-03T15:32:01.966 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-ceph-common-2:17.2.7-1045.gc6021375.el9.x8 20/26 2024-07-03T15:32:02.013 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-07-03T15:32:02.038 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librgw2-2:17.2.7-1045.gc6021375.el9.x86_64 8/26 2024-07-03T15:32:02.085 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librgw2-2:17.2.7-1045.gc6021375.el9.x86_64 8/26 2024-07-03T15:32:02.113 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-07-03T15:32:02.156 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ceph-argparse-2:17.2.7-1045.gc6021375.el9. 10/26 2024-07-03T15:32:02.193 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cephfs-2:17.2.7-1045.gc6021375.el9.x86_64 11/26 2024-07-03T15:32:02.255 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rgw-2:17.2.7-1045.gc6021375.el9.x86_64 12/26 2024-07-03T15:32:02.283 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libradosstriper1-2:17.2.7-1045.gc6021375.el9.x86_6 13/26 2024-07-03T15:32:02.349 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1045.gc6021375.el9.x86_6 13/26 2024-07-03T15:32:02.389 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-07-03T15:32:02.433 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-07-03T15:32:02.457 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-07-03T15:32:02.766 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-07-03T15:32:02.804 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : librbd1-2:17.2.7-1045.gc6021375.el9.x86_64 18/26 2024-07-03T15:32:02.892 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librbd1-2:17.2.7-1045.gc6021375.el9.x86_64 18/26 2024-07-03T15:32:02.966 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rbd-2:17.2.7-1045.gc6021375.el9.x86_64 19/26 2024-07-03T15:32:02.998 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ceph-common-2:17.2.7-1045.gc6021375.el9.x8 20/26 2024-07-03T15:32:04.514 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-common-2:17.2.7-1045.gc6021375.el9.x86_64 21/26 2024-07-03T15:32:04.629 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-common-2:17.2.7-1045.gc6021375.el9.x86_64 21/26 2024-07-03T15:32:05.194 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-common-2:17.2.7-1045.gc6021375.el9.x86_64 21/26 2024-07-03T15:32:05.233 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-base-2:17.2.7-1045.gc6021375.el9.x86_64 22/26 2024-07-03T15:32:05.331 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-base-2:17.2.7-1045.gc6021375.el9.x86_64 22/26 2024-07-03T15:32:05.331 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-07-03T15:32:05.332 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-07-03T15:32:05.332 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:32:05.359 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-selinux-2:17.2.7-1045.gc6021375.el9.x86_64 23/26 2024-07-03T15:32:05.402 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-common-2:17.2.7-1045.gc6021375.el9.x86_64 21/26 2024-07-03T15:32:05.436 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-common-2:17.2.7-1045.gc6021375.el9.x86_64 21/26 2024-07-03T15:32:05.964 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-common-2:17.2.7-1045.gc6021375.el9.x86_64 21/26 2024-07-03T15:32:06.007 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-base-2:17.2.7-1045.gc6021375.el9.x86_64 22/26 2024-07-03T15:32:06.089 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-base-2:17.2.7-1045.gc6021375.el9.x86_64 22/26 2024-07-03T15:32:06.089 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-07-03T15:32:06.089 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-07-03T15:32:06.089 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:32:06.124 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-selinux-2:17.2.7-1045.gc6021375.el9.x86_64 23/26 2024-07-03T15:32:15.338 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1045.gc6021375.el9.x86_64 23/26 2024-07-03T15:32:15.338 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /sys 2024-07-03T15:32:15.338 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /proc 2024-07-03T15:32:15.338 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /mnt 2024-07-03T15:32:15.338 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /var/tmp 2024-07-03T15:32:15.338 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /home 2024-07-03T15:32:15.338 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /root 2024-07-03T15:32:15.338 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /tmp 2024-07-03T15:32:15.339 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:32:15.506 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1045.gc6021375.el9.x86_64 23/26 2024-07-03T15:32:15.506 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /sys 2024-07-03T15:32:15.506 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /proc 2024-07-03T15:32:15.506 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /mnt 2024-07-03T15:32:15.506 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /var/tmp 2024-07-03T15:32:15.506 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /home 2024-07-03T15:32:15.506 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /root 2024-07-03T15:32:15.506 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /tmp 2024-07-03T15:32:15.506 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:32:16.663 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-radosgw-2:17.2.7-1045.gc6021375.el9.x86_64 24/26 2024-07-03T15:32:16.742 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1045.gc6021375.el9.x86_64 24/26 2024-07-03T15:32:16.742 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-03T15:32:16.742 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-07-03T15:32:16.742 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-03T15:32:16.742 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-03T15:32:16.743 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:32:16.744 INFO:teuthology.orchestra.run.smithi028.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-07-03T15:32:16.766 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-radosgw-2:17.2.7-1045.gc6021375.el9.x86_64 24/26 2024-07-03T15:32:16.841 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-07-03T15:32:16.841 INFO:teuthology.orchestra.run.smithi028.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-07-03T15:32:16.846 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1045.gc6021375.el9.x86_64 24/26 2024-07-03T15:32:16.846 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-03T15:32:16.846 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-07-03T15:32:16.846 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-03T15:32:16.846 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-03T15:32:16.846 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:32:16.848 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-07-03T15:32:16.935 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-07-03T15:32:16.936 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-07-03T15:32:18.097 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-07-03T15:32:18.097 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-base-2:17.2.7-1045.gc6021375.el9.x86_64 1/26 2024-07-03T15:32:18.097 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-common-2:17.2.7-1045.gc6021375.el9.x86_64 2/26 2024-07-03T15:32:18.097 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-radosgw-2:17.2.7-1045.gc6021375.el9.x86_64 3/26 2024-07-03T15:32:18.098 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-selinux-2:17.2.7-1045.gc6021375.el9.x86_64 4/26 2024-07-03T15:32:18.098 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libcephfs2-2:17.2.7-1045.gc6021375.el9.x86_64 5/26 2024-07-03T15:32:18.098 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libradosstriper1-2:17.2.7-1045.gc6021375.el9.x86_6 6/26 2024-07-03T15:32:18.098 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librgw2-2:17.2.7-1045.gc6021375.el9.x86_64 7/26 2024-07-03T15:32:18.098 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1045.gc6021375.el9. 8/26 2024-07-03T15:32:18.098 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-ceph-common-2:17.2.7-1045.gc6021375.el9.x8 9/26 2024-07-03T15:32:18.098 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-cephfs-2:17.2.7-1045.gc6021375.el9.x86_64 10/26 2024-07-03T15:32:18.098 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-rados-2:17.2.7-1045.gc6021375.el9.x86_64 11/26 2024-07-03T15:32:18.098 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-rbd-2:17.2.7-1045.gc6021375.el9.x86_64 12/26 2024-07-03T15:32:18.098 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-rgw-2:17.2.7-1045.gc6021375.el9.x86_64 13/26 2024-07-03T15:32:18.098 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-07-03T15:32:18.099 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-07-03T15:32:18.099 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-07-03T15:32:18.099 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-07-03T15:32:18.099 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-07-03T15:32:18.099 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-07-03T15:32:18.099 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-07-03T15:32:18.099 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-07-03T15:32:18.099 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-07-03T15:32:18.099 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librados2-2:17.2.7-1045.gc6021375.el9.x86_64 23/26 2024-07-03T15:32:18.099 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-07-03T15:32:18.099 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librbd1-2:17.2.7-1045.gc6021375.el9.x86_64 25/26 2024-07-03T15:32:18.217 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-07-03T15:32:18.217 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-base-2:17.2.7-1045.gc6021375.el9.x86_64 1/26 2024-07-03T15:32:18.218 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-common-2:17.2.7-1045.gc6021375.el9.x86_64 2/26 2024-07-03T15:32:18.218 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-radosgw-2:17.2.7-1045.gc6021375.el9.x86_64 3/26 2024-07-03T15:32:18.218 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-selinux-2:17.2.7-1045.gc6021375.el9.x86_64 4/26 2024-07-03T15:32:18.218 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephfs2-2:17.2.7-1045.gc6021375.el9.x86_64 5/26 2024-07-03T15:32:18.218 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libradosstriper1-2:17.2.7-1045.gc6021375.el9.x86_6 6/26 2024-07-03T15:32:18.218 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librgw2-2:17.2.7-1045.gc6021375.el9.x86_64 7/26 2024-07-03T15:32:18.218 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1045.gc6021375.el9. 8/26 2024-07-03T15:32:18.218 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ceph-common-2:17.2.7-1045.gc6021375.el9.x8 9/26 2024-07-03T15:32:18.218 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cephfs-2:17.2.7-1045.gc6021375.el9.x86_64 10/26 2024-07-03T15:32:18.218 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rados-2:17.2.7-1045.gc6021375.el9.x86_64 11/26 2024-07-03T15:32:18.218 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rbd-2:17.2.7-1045.gc6021375.el9.x86_64 12/26 2024-07-03T15:32:18.218 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rgw-2:17.2.7-1045.gc6021375.el9.x86_64 13/26 2024-07-03T15:32:18.218 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-07-03T15:32:18.219 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-07-03T15:32:18.219 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-07-03T15:32:18.219 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-07-03T15:32:18.219 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-07-03T15:32:18.219 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-07-03T15:32:18.219 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-07-03T15:32:18.219 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-07-03T15:32:18.219 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-07-03T15:32:18.219 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados2-2:17.2.7-1045.gc6021375.el9.x86_64 23/26 2024-07-03T15:32:18.219 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-07-03T15:32:18.219 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librbd1-2:17.2.7-1045.gc6021375.el9.x86_64 25/26 2024-07-03T15:32:18.950 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-07-03T15:32:18.950 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:32:18.951 INFO:teuthology.orchestra.run.smithi028.stdout:Upgraded: 2024-07-03T15:32:18.951 INFO:teuthology.orchestra.run.smithi028.stdout: librados2-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:32:18.951 INFO:teuthology.orchestra.run.smithi028.stdout: librbd1-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:32:18.951 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-07-03T15:32:18.951 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-base-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:32:18.951 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-common-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:32:18.951 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-radosgw-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:32:18.951 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-selinux-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:32:18.951 INFO:teuthology.orchestra.run.smithi028.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-07-03T15:32:18.951 INFO:teuthology.orchestra.run.smithi028.stdout: libcephfs2-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:32:18.951 INFO:teuthology.orchestra.run.smithi028.stdout: liboath-2.6.7-2.el9.x86_64 2024-07-03T15:32:18.951 INFO:teuthology.orchestra.run.smithi028.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-07-03T15:32:18.952 INFO:teuthology.orchestra.run.smithi028.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-07-03T15:32:18.952 INFO:teuthology.orchestra.run.smithi028.stdout: libradosstriper1-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:32:18.952 INFO:teuthology.orchestra.run.smithi028.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-07-03T15:32:18.952 INFO:teuthology.orchestra.run.smithi028.stdout: librgw2-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:32:18.952 INFO:teuthology.orchestra.run.smithi028.stdout: libunwind-1.6.2-1.el9.x86_64 2024-07-03T15:32:18.952 INFO:teuthology.orchestra.run.smithi028.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-07-03T15:32:18.952 INFO:teuthology.orchestra.run.smithi028.stdout: python3-ceph-argparse-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:32:18.952 INFO:teuthology.orchestra.run.smithi028.stdout: python3-ceph-common-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:32:18.952 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cephfs-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:32:18.952 INFO:teuthology.orchestra.run.smithi028.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-07-03T15:32:18.952 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rados-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:32:18.952 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rbd-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:32:18.953 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rgw-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:32:18.953 INFO:teuthology.orchestra.run.smithi028.stdout: thrift-0.15.0-2.el9.x86_64 2024-07-03T15:32:18.953 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:32:18.953 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-07-03T15:32:19.030 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-07-03T15:32:19.030 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:32:19.030 INFO:teuthology.orchestra.run.smithi148.stdout:Upgraded: 2024-07-03T15:32:19.030 INFO:teuthology.orchestra.run.smithi148.stdout: librados2-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:32:19.030 INFO:teuthology.orchestra.run.smithi148.stdout: librbd1-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:32:19.030 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-07-03T15:32:19.030 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-base-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:32:19.031 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-common-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:32:19.031 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-radosgw-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:32:19.031 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-selinux-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:32:19.031 INFO:teuthology.orchestra.run.smithi148.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-07-03T15:32:19.031 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs2-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:32:19.031 INFO:teuthology.orchestra.run.smithi148.stdout: liboath-2.6.7-2.el9.x86_64 2024-07-03T15:32:19.031 INFO:teuthology.orchestra.run.smithi148.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-07-03T15:32:19.031 INFO:teuthology.orchestra.run.smithi148.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-07-03T15:32:19.031 INFO:teuthology.orchestra.run.smithi148.stdout: libradosstriper1-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:32:19.031 INFO:teuthology.orchestra.run.smithi148.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-07-03T15:32:19.031 INFO:teuthology.orchestra.run.smithi148.stdout: librgw2-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:32:19.031 INFO:teuthology.orchestra.run.smithi148.stdout: libunwind-1.6.2-1.el9.x86_64 2024-07-03T15:32:19.031 INFO:teuthology.orchestra.run.smithi148.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-07-03T15:32:19.032 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-argparse-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:32:19.032 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-common-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:32:19.032 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cephfs-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:32:19.032 INFO:teuthology.orchestra.run.smithi148.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-07-03T15:32:19.032 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rados-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:32:19.032 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rbd-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:32:19.032 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rgw-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:32:19.032 INFO:teuthology.orchestra.run.smithi148.stdout: thrift-0.15.0-2.el9.x86_64 2024-07-03T15:32:19.032 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:32:19.032 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-07-03T15:32:19.295 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install ceph-test 2024-07-03T15:32:19.420 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-test 2024-07-03T15:32:19.875 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:00:28 ago on Wed 03 Jul 2024 03:31:51 PM UTC. 2024-07-03T15:32:19.989 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:00:26 ago on Wed 03 Jul 2024 03:31:53 PM UTC. 2024-07-03T15:32:20.006 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-07-03T15:32:20.007 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-07-03T15:32:20.007 INFO:teuthology.orchestra.run.smithi028.stdout: Package Arch Version Repository Size 2024-07-03T15:32:20.007 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-07-03T15:32:20.007 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-07-03T15:32:20.008 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-test x86_64 2:17.2.7-1045.gc6021375.el9 ceph 48 M 2024-07-03T15:32:20.008 INFO:teuthology.orchestra.run.smithi028.stdout:Installing dependencies: 2024-07-03T15:32:20.008 INFO:teuthology.orchestra.run.smithi028.stdout: libcephsqlite x86_64 2:17.2.7-1045.gc6021375.el9 ceph 163 k 2024-07-03T15:32:20.008 INFO:teuthology.orchestra.run.smithi028.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-07-03T15:32:20.008 INFO:teuthology.orchestra.run.smithi028.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-07-03T15:32:20.008 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:32:20.008 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-07-03T15:32:20.008 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-07-03T15:32:20.008 INFO:teuthology.orchestra.run.smithi028.stdout:Install 4 Packages 2024-07-03T15:32:20.008 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:32:20.009 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 48 M 2024-07-03T15:32:20.009 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 192 M 2024-07-03T15:32:20.009 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-07-03T15:32:20.116 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-07-03T15:32:20.116 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-07-03T15:32:20.116 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-07-03T15:32:20.116 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-07-03T15:32:20.116 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-07-03T15:32:20.117 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-test x86_64 2:17.2.7-1045.gc6021375.el9 ceph 48 M 2024-07-03T15:32:20.117 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-07-03T15:32:20.117 INFO:teuthology.orchestra.run.smithi148.stdout: libcephsqlite x86_64 2:17.2.7-1045.gc6021375.el9 ceph 163 k 2024-07-03T15:32:20.117 INFO:teuthology.orchestra.run.smithi148.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-07-03T15:32:20.117 INFO:teuthology.orchestra.run.smithi148.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-07-03T15:32:20.117 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:32:20.117 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-07-03T15:32:20.117 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-07-03T15:32:20.117 INFO:teuthology.orchestra.run.smithi148.stdout:Install 4 Packages 2024-07-03T15:32:20.117 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:32:20.118 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 48 M 2024-07-03T15:32:20.118 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 192 M 2024-07-03T15:32:20.118 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-07-03T15:32:20.417 INFO:teuthology.orchestra.run.smithi148.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.9 MB/s | 304 kB 00:00 2024-07-03T15:32:20.419 INFO:teuthology.orchestra.run.smithi028.stdout:(1/4): libcephsqlite-17.2.7-1045.gc6021375.el9. 443 kB/s | 163 kB 00:00 2024-07-03T15:32:20.459 INFO:teuthology.orchestra.run.smithi148.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-07-03T15:32:20.500 INFO:teuthology.orchestra.run.smithi148.stdout:(3/4): libcephsqlite-17.2.7-1045.gc6021375.el9. 686 kB/s | 163 kB 00:00 2024-07-03T15:32:20.904 INFO:teuthology.orchestra.run.smithi028.stdout:(2/4): socat-1.7.4.1-6.el9.x86_64.rpm 357 kB/s | 304 kB 00:00 2024-07-03T15:32:21.095 INFO:teuthology.orchestra.run.smithi028.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 94 kB/s | 64 kB 00:00 2024-07-03T15:32:22.267 INFO:teuthology.orchestra.run.smithi148.stdout:(4/4): ceph-test-17.2.7-1045.gc6021375.el9.x86_ 24 MB/s | 48 MB 00:02 2024-07-03T15:32:22.270 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-07-03T15:32:22.270 INFO:teuthology.orchestra.run.smithi148.stdout:Total 22 MB/s | 48 MB 00:02 2024-07-03T15:32:22.297 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-07-03T15:32:22.317 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-07-03T15:32:22.317 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-07-03T15:32:22.620 INFO:teuthology.orchestra.run.smithi028.stdout:(4/4): ceph-test-17.2.7-1045.gc6021375.el9.x86_ 19 MB/s | 48 MB 00:02 2024-07-03T15:32:22.622 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-07-03T15:32:22.623 INFO:teuthology.orchestra.run.smithi028.stdout:Total 19 MB/s | 48 MB 00:02 2024-07-03T15:32:22.651 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-07-03T15:32:22.673 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-07-03T15:32:22.673 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-07-03T15:32:22.681 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-07-03T15:32:22.681 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-07-03T15:32:23.056 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-07-03T15:32:23.056 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-07-03T15:32:23.210 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-07-03T15:32:23.291 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-07-03T15:32:23.334 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-07-03T15:32:23.361 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephsqlite-2:17.2.7-1045.gc6021375.el9.x86_64 3/4 2024-07-03T15:32:23.611 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-07-03T15:32:23.682 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-07-03T15:32:23.737 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-07-03T15:32:23.764 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libcephsqlite-2:17.2.7-1045.gc6021375.el9.x86_64 3/4 2024-07-03T15:32:27.730 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1045.gc6021375.el9.x86_64 3/4 2024-07-03T15:32:27.819 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-test-2:17.2.7-1045.gc6021375.el9.x86_64 4/4 2024-07-03T15:32:28.408 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1045.gc6021375.el9.x86_64 3/4 2024-07-03T15:32:28.490 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-test-2:17.2.7-1045.gc6021375.el9.x86_64 4/4 2024-07-03T15:32:30.093 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-test-2:17.2.7-1045.gc6021375.el9.x86_64 4/4 2024-07-03T15:32:30.093 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-test-2:17.2.7-1045.gc6021375.el9.x86_64 1/4 2024-07-03T15:32:30.093 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephsqlite-2:17.2.7-1045.gc6021375.el9.x86_64 2/4 2024-07-03T15:32:30.093 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-07-03T15:32:30.398 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-07-03T15:32:30.398 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:32:30.398 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-07-03T15:32:30.398 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-test-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:32:30.398 INFO:teuthology.orchestra.run.smithi148.stdout: libcephsqlite-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:32:30.399 INFO:teuthology.orchestra.run.smithi148.stdout: socat-1.7.4.1-6.el9.x86_64 2024-07-03T15:32:30.399 INFO:teuthology.orchestra.run.smithi148.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-07-03T15:32:30.399 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:32:30.399 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-07-03T15:32:30.566 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-test-2:17.2.7-1045.gc6021375.el9.x86_64 4/4 2024-07-03T15:32:30.567 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-test-2:17.2.7-1045.gc6021375.el9.x86_64 1/4 2024-07-03T15:32:30.567 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libcephsqlite-2:17.2.7-1045.gc6021375.el9.x86_64 2/4 2024-07-03T15:32:30.567 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-07-03T15:32:30.689 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph 2024-07-03T15:32:30.828 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-07-03T15:32:30.828 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:32:30.828 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-07-03T15:32:30.828 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-test-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:32:30.829 INFO:teuthology.orchestra.run.smithi028.stdout: libcephsqlite-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:32:30.829 INFO:teuthology.orchestra.run.smithi028.stdout: socat-1.7.4.1-6.el9.x86_64 2024-07-03T15:32:30.829 INFO:teuthology.orchestra.run.smithi028.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-07-03T15:32:30.829 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:32:30.829 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-07-03T15:32:31.103 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install ceph 2024-07-03T15:32:31.249 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:00:38 ago on Wed 03 Jul 2024 03:31:53 PM UTC. 2024-07-03T15:32:31.377 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-07-03T15:32:31.379 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-07-03T15:32:31.379 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-07-03T15:32:31.379 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-07-03T15:32:31.379 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-07-03T15:32:31.379 INFO:teuthology.orchestra.run.smithi148.stdout: ceph x86_64 2:17.2.7-1045.gc6021375.el9 ceph 6.5 k 2024-07-03T15:32:31.379 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-07-03T15:32:31.379 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mds x86_64 2:17.2.7-1045.gc6021375.el9 ceph 2.1 M 2024-07-03T15:32:31.379 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr x86_64 2:17.2.7-1045.gc6021375.el9 ceph 1.4 M 2024-07-03T15:32:31.380 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1045.gc6021375.el9 ceph-noarch 240 k 2024-07-03T15:32:31.380 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mon x86_64 2:17.2.7-1045.gc6021375.el9 ceph 3.9 M 2024-07-03T15:32:31.380 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-osd x86_64 2:17.2.7-1045.gc6021375.el9 ceph 15 M 2024-07-03T15:32:31.380 INFO:teuthology.orchestra.run.smithi148.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-07-03T15:32:31.380 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-07-03T15:32:31.380 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-07-03T15:32:31.380 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-07-03T15:32:31.380 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-07-03T15:32:31.380 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-07-03T15:32:31.380 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-07-03T15:32:31.380 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-collections 2024-07-03T15:32:31.380 INFO:teuthology.orchestra.run.smithi148.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-07-03T15:32:31.381 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-07-03T15:32:31.381 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-07-03T15:32:31.381 INFO:teuthology.orchestra.run.smithi148.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-07-03T15:32:31.381 INFO:teuthology.orchestra.run.smithi148.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-07-03T15:32:31.381 INFO:teuthology.orchestra.run.smithi148.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-07-03T15:32:31.381 INFO:teuthology.orchestra.run.smithi148.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-07-03T15:32:31.381 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-07-03T15:32:31.381 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-07-03T15:32:31.381 INFO:teuthology.orchestra.run.smithi148.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-07-03T15:32:31.381 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-07-03T15:32:31.381 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-07-03T15:32:31.381 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-07-03T15:32:31.382 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-07-03T15:32:31.382 INFO:teuthology.orchestra.run.smithi148.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-07-03T15:32:31.382 INFO:teuthology.orchestra.run.smithi148.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-07-03T15:32:31.382 INFO:teuthology.orchestra.run.smithi148.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-07-03T15:32:31.382 INFO:teuthology.orchestra.run.smithi148.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-07-03T15:32:31.382 INFO:teuthology.orchestra.run.smithi148.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-07-03T15:32:31.382 INFO:teuthology.orchestra.run.smithi148.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-07-03T15:32:31.382 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:32:31.382 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-07-03T15:32:31.382 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-07-03T15:32:31.382 INFO:teuthology.orchestra.run.smithi148.stdout:Install 33 Packages 2024-07-03T15:32:31.382 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:32:31.383 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 27 M 2024-07-03T15:32:31.383 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 96 M 2024-07-03T15:32:31.383 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-07-03T15:32:31.686 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:00:40 ago on Wed 03 Jul 2024 03:31:51 PM UTC. 2024-07-03T15:32:31.821 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-07-03T15:32:31.823 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-07-03T15:32:31.823 INFO:teuthology.orchestra.run.smithi028.stdout: Package Arch Version Repository Size 2024-07-03T15:32:31.823 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-07-03T15:32:31.823 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-07-03T15:32:31.823 INFO:teuthology.orchestra.run.smithi028.stdout: ceph x86_64 2:17.2.7-1045.gc6021375.el9 ceph 6.5 k 2024-07-03T15:32:31.823 INFO:teuthology.orchestra.run.smithi028.stdout:Installing dependencies: 2024-07-03T15:32:31.824 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mds x86_64 2:17.2.7-1045.gc6021375.el9 ceph 2.1 M 2024-07-03T15:32:31.824 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr x86_64 2:17.2.7-1045.gc6021375.el9 ceph 1.4 M 2024-07-03T15:32:31.824 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1045.gc6021375.el9 ceph-noarch 240 k 2024-07-03T15:32:31.824 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mon x86_64 2:17.2.7-1045.gc6021375.el9 ceph 3.9 M 2024-07-03T15:32:31.824 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-osd x86_64 2:17.2.7-1045.gc6021375.el9 ceph 15 M 2024-07-03T15:32:31.824 INFO:teuthology.orchestra.run.smithi028.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-07-03T15:32:31.824 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-07-03T15:32:31.824 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-07-03T15:32:31.824 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-07-03T15:32:31.824 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-07-03T15:32:31.824 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-07-03T15:32:31.824 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-07-03T15:32:31.824 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-collections 2024-07-03T15:32:31.825 INFO:teuthology.orchestra.run.smithi028.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-07-03T15:32:31.825 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-07-03T15:32:31.825 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-07-03T15:32:31.825 INFO:teuthology.orchestra.run.smithi028.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-07-03T15:32:31.825 INFO:teuthology.orchestra.run.smithi028.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-07-03T15:32:31.825 INFO:teuthology.orchestra.run.smithi028.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-07-03T15:32:31.825 INFO:teuthology.orchestra.run.smithi028.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-07-03T15:32:31.825 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-07-03T15:32:31.825 INFO:teuthology.orchestra.run.smithi028.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-07-03T15:32:31.825 INFO:teuthology.orchestra.run.smithi028.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-07-03T15:32:31.825 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-07-03T15:32:31.825 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-07-03T15:32:31.825 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-07-03T15:32:31.826 INFO:teuthology.orchestra.run.smithi028.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-07-03T15:32:31.826 INFO:teuthology.orchestra.run.smithi028.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-07-03T15:32:31.826 INFO:teuthology.orchestra.run.smithi028.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-07-03T15:32:31.826 INFO:teuthology.orchestra.run.smithi028.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-07-03T15:32:31.826 INFO:teuthology.orchestra.run.smithi028.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-07-03T15:32:31.826 INFO:teuthology.orchestra.run.smithi028.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-07-03T15:32:31.826 INFO:teuthology.orchestra.run.smithi028.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-07-03T15:32:31.826 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:32:31.826 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-07-03T15:32:31.826 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-07-03T15:32:31.826 INFO:teuthology.orchestra.run.smithi028.stdout:Install 33 Packages 2024-07-03T15:32:31.826 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:32:31.827 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 27 M 2024-07-03T15:32:31.827 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 96 M 2024-07-03T15:32:31.827 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-07-03T15:32:32.027 INFO:teuthology.orchestra.run.smithi148.stdout:(1/33): ceph-17.2.7-1045.gc6021375.el9.x86_64.r 38 kB/s | 6.5 kB 00:00 2024-07-03T15:32:32.202 INFO:teuthology.orchestra.run.smithi148.stdout:(2/33): ceph-mgr-17.2.7-1045.gc6021375.el9.x86_ 4.0 MB/s | 1.4 MB 00:00 2024-07-03T15:32:32.253 INFO:teuthology.orchestra.run.smithi148.stdout:(3/33): ceph-mds-17.2.7-1045.gc6021375.el9.x86_ 5.2 MB/s | 2.1 MB 00:00 2024-07-03T15:32:32.345 INFO:teuthology.orchestra.run.smithi148.stdout:(4/33): ceph-mgr-modules-core-17.2.7-1045.gc602 2.6 MB/s | 240 kB 00:00 2024-07-03T15:32:32.414 INFO:teuthology.orchestra.run.smithi148.stdout:(5/33): ceph-mon-17.2.7-1045.gc6021375.el9.x86_ 10 MB/s | 3.9 MB 00:00 2024-07-03T15:32:32.446 INFO:teuthology.orchestra.run.smithi028.stdout:(1/33): ceph-17.2.7-1045.gc6021375.el9.x86_64.r 26 kB/s | 6.5 kB 00:00 2024-07-03T15:32:32.545 INFO:teuthology.orchestra.run.smithi148.stdout:(6/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-07-03T15:32:32.629 INFO:teuthology.orchestra.run.smithi148.stdout:(7/33): python3-cryptography-36.0.1-4.el9.x86_6 6.4 MB/s | 1.2 MB 00:00 2024-07-03T15:32:32.779 INFO:teuthology.orchestra.run.smithi148.stdout:(8/33): ceph-osd-17.2.7-1045.gc6021375.el9.x86_ 27 MB/s | 15 MB 00:00 2024-07-03T15:32:32.863 INFO:teuthology.orchestra.run.smithi148.stdout:(9/33): python3-ply-3.11-14.el9.noarch.rpm 336 kB/s | 106 kB 00:00 2024-07-03T15:32:32.921 INFO:teuthology.orchestra.run.smithi148.stdout:(10/33): python3-pycparser-2.20-6.el9.noarch.rp 465 kB/s | 135 kB 00:00 2024-07-03T15:32:32.947 INFO:teuthology.orchestra.run.smithi148.stdout:(11/33): python3-requests-2.25.1-8.el9.noarch.r 882 kB/s | 125 kB 00:00 2024-07-03T15:32:33.005 INFO:teuthology.orchestra.run.smithi148.stdout:(12/33): python3-urllib3-1.26.5-5.el9.noarch.rp 1.5 MB/s | 215 kB 00:00 2024-07-03T15:32:33.080 INFO:teuthology.orchestra.run.smithi028.stdout:(2/33): ceph-mds-17.2.7-1045.gc6021375.el9.x86_ 2.3 MB/s | 2.1 MB 00:00 2024-07-03T15:32:33.114 INFO:teuthology.orchestra.run.smithi028.stdout:(3/33): ceph-mgr-17.2.7-1045.gc6021375.el9.x86_ 1.5 MB/s | 1.4 MB 00:00 2024-07-03T15:32:33.214 INFO:teuthology.orchestra.run.smithi028.stdout:(4/33): ceph-mgr-modules-core-17.2.7-1045.gc602 2.3 MB/s | 240 kB 00:00 2024-07-03T15:32:33.456 INFO:teuthology.orchestra.run.smithi028.stdout:(5/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.0 MB/s | 253 kB 00:00 2024-07-03T15:32:33.572 INFO:teuthology.orchestra.run.smithi148.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 56 kB/s | 35 kB 00:00 2024-07-03T15:32:33.648 INFO:teuthology.orchestra.run.smithi028.stdout:(6/33): python3-cryptography-36.0.1-4.el9.x86_6 6.4 MB/s | 1.2 MB 00:00 2024-07-03T15:32:33.689 INFO:teuthology.orchestra.run.smithi148.stdout:(14/33): python3-mako-1.1.4-6.el9.noarch.rpm 225 kB/s | 172 kB 00:00 2024-07-03T15:32:33.714 INFO:teuthology.orchestra.run.smithi148.stdout:(15/33): python3-pytz-2021.1-5.el9.noarch.rpm 72 kB/s | 51 kB 00:00 2024-07-03T15:32:33.740 INFO:teuthology.orchestra.run.smithi028.stdout:(7/33): ceph-mon-17.2.7-1045.gc6021375.el9.x86_ 3.0 MB/s | 3.9 MB 00:01 2024-07-03T15:32:33.773 INFO:teuthology.orchestra.run.smithi028.stdout:(8/33): python3-ply-3.11-14.el9.noarch.rpm 851 kB/s | 106 kB 00:00 2024-07-03T15:32:33.789 INFO:teuthology.orchestra.run.smithi148.stdout:(16/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 437 kB/s | 43 kB 00:00 2024-07-03T15:32:33.823 INFO:teuthology.orchestra.run.smithi148.stdout:(17/33): python3-cherrypy-18.6.1-2.el9.noarch.r 11 MB/s | 358 kB 00:00 2024-07-03T15:32:33.849 INFO:teuthology.orchestra.run.smithi148.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 425 kB/s | 11 kB 00:00 2024-07-03T15:32:33.857 INFO:teuthology.orchestra.run.smithi028.stdout:(9/33): python3-requests-2.25.1-8.el9.noarch.rp 1.5 MB/s | 125 kB 00:00 2024-07-03T15:32:33.898 INFO:teuthology.orchestra.run.smithi148.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 933 kB/s | 172 kB 00:00 2024-07-03T15:32:33.924 INFO:teuthology.orchestra.run.smithi148.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 343 kB/s | 18 kB 00:00 2024-07-03T15:32:33.924 INFO:teuthology.orchestra.run.smithi028.stdout:(10/33): python3-pycparser-2.20-6.el9.noarch.rp 737 kB/s | 135 kB 00:00 2024-07-03T15:32:33.941 INFO:teuthology.orchestra.run.smithi148.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 552 kB/s | 23 kB 00:00 2024-07-03T15:32:33.957 INFO:teuthology.orchestra.run.smithi148.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 579 kB/s | 19 kB 00:00 2024-07-03T15:32:33.958 INFO:teuthology.orchestra.run.smithi028.stdout:(11/33): python3-urllib3-1.26.5-5.el9.noarch.rp 2.1 MB/s | 215 kB 00:00 2024-07-03T15:32:33.983 INFO:teuthology.orchestra.run.smithi148.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 469 kB/s | 20 kB 00:00 2024-07-03T15:32:34.000 INFO:teuthology.orchestra.run.smithi148.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 1.1 MB/s | 46 kB 00:00 2024-07-03T15:32:34.016 INFO:teuthology.orchestra.run.smithi148.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 2.3 MB/s | 79 kB 00:00 2024-07-03T15:32:34.033 INFO:teuthology.orchestra.run.smithi148.stdout:(26/33): python3-portend-3.1.0-2.el9.noarch.rpm 983 kB/s | 16 kB 00:00 2024-07-03T15:32:34.059 INFO:teuthology.orchestra.run.smithi148.stdout:(27/33): python3-pecan-1.4.2-3.el9.noarch.rpm 4.5 MB/s | 272 kB 00:00 2024-07-03T15:32:34.066 INFO:teuthology.orchestra.run.smithi028.stdout:(12/33): python3-markupsafe-1.1.1-12.el9.x86_64 321 kB/s | 35 kB 00:00 2024-07-03T15:32:34.084 INFO:teuthology.orchestra.run.smithi148.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-07-03T15:32:34.091 INFO:teuthology.orchestra.run.smithi028.stdout:(13/33): python3-mako-1.1.4-6.el9.noarch.rpm 1.0 MB/s | 172 kB 00:00 2024-07-03T15:32:34.101 INFO:teuthology.orchestra.run.smithi148.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 852 kB/s | 36 kB 00:00 2024-07-03T15:32:34.135 INFO:teuthology.orchestra.run.smithi148.stdout:(30/33): python3-webob-1.8.7-6.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-07-03T15:32:34.142 INFO:teuthology.orchestra.run.smithi028.stdout:(14/33): python3-toml-0.10.2-6.el9.noarch.rpm 837 kB/s | 42 kB 00:00 2024-07-03T15:32:34.167 INFO:teuthology.orchestra.run.smithi028.stdout:(15/33): python3-pytz-2021.1-5.el9.noarch.rpm 508 kB/s | 51 kB 00:00 2024-07-03T15:32:34.168 INFO:teuthology.orchestra.run.smithi148.stdout:(31/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 6.2 MB/s | 427 kB 00:00 2024-07-03T15:32:34.185 INFO:teuthology.orchestra.run.smithi148.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 397 kB/s | 20 kB 00:00 2024-07-03T15:32:34.244 INFO:teuthology.orchestra.run.smithi148.stdout:(33/33): python3-toml-0.10.2-6.el9.noarch.rpm 62 kB/s | 42 kB 00:00 2024-07-03T15:32:34.248 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-07-03T15:32:34.248 INFO:teuthology.orchestra.run.smithi148.stdout:Total 9.5 MB/s | 27 MB 00:02 2024-07-03T15:32:34.251 INFO:teuthology.orchestra.run.smithi028.stdout:(16/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 400 kB/s | 43 kB 00:00 2024-07-03T15:32:34.276 INFO:teuthology.orchestra.run.smithi028.stdout:(17/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.5 MB/s | 172 kB 00:00 2024-07-03T15:32:34.293 INFO:teuthology.orchestra.run.smithi028.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 653 kB/s | 11 kB 00:00 2024-07-03T15:32:34.318 INFO:teuthology.orchestra.run.smithi028.stdout:(19/33): python3-cherrypy-18.6.1-2.el9.noarch.r 5.4 MB/s | 358 kB 00:00 2024-07-03T15:32:34.335 INFO:teuthology.orchestra.run.smithi028.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 421 kB/s | 18 kB 00:00 2024-07-03T15:32:34.352 INFO:teuthology.orchestra.run.smithi028.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 692 kB/s | 23 kB 00:00 2024-07-03T15:32:34.362 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-07-03T15:32:34.369 INFO:teuthology.orchestra.run.smithi028.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 581 kB/s | 19 kB 00:00 2024-07-03T15:32:34.386 INFO:teuthology.orchestra.run.smithi028.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 587 kB/s | 20 kB 00:00 2024-07-03T15:32:34.397 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-07-03T15:32:34.398 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-07-03T15:32:34.403 INFO:teuthology.orchestra.run.smithi028.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 1.3 MB/s | 46 kB 00:00 2024-07-03T15:32:34.428 INFO:teuthology.orchestra.run.smithi028.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 1.8 MB/s | 79 kB 00:00 2024-07-03T15:32:34.454 INFO:teuthology.orchestra.run.smithi028.stdout:(26/33): python3-portend-3.1.0-2.el9.noarch.rpm 656 kB/s | 16 kB 00:00 2024-07-03T15:32:34.470 INFO:teuthology.orchestra.run.smithi028.stdout:(27/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 5.3 MB/s | 90 kB 00:00 2024-07-03T15:32:34.496 INFO:teuthology.orchestra.run.smithi028.stdout:(28/33): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-07-03T15:32:34.521 INFO:teuthology.orchestra.run.smithi028.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 713 kB/s | 36 kB 00:00 2024-07-03T15:32:34.555 INFO:teuthology.orchestra.run.smithi028.stdout:(30/33): python3-webob-1.8.7-6.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-07-03T15:32:34.580 INFO:teuthology.orchestra.run.smithi028.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 801 kB/s | 20 kB 00:00 2024-07-03T15:32:34.605 INFO:teuthology.orchestra.run.smithi028.stdout:(32/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 5.0 MB/s | 427 kB 00:00 2024-07-03T15:32:34.757 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-07-03T15:32:34.757 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-07-03T15:32:34.764 INFO:teuthology.orchestra.run.smithi028.stdout:(33/33): ceph-osd-17.2.7-1045.gc6021375.el9.x86 9.1 MB/s | 15 MB 00:01 2024-07-03T15:32:34.768 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-07-03T15:32:34.768 INFO:teuthology.orchestra.run.smithi028.stdout:Total 9.2 MB/s | 27 MB 00:02 2024-07-03T15:32:34.892 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-07-03T15:32:34.932 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-07-03T15:32:34.932 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-07-03T15:32:35.300 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-07-03T15:32:35.300 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-07-03T15:32:35.896 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-07-03T15:32:35.947 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-07-03T15:32:35.987 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-07-03T15:32:36.020 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-07-03T15:32:36.112 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-07-03T15:32:36.182 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-07-03T15:32:36.222 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-07-03T15:32:36.258 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-07-03T15:32:36.288 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-07-03T15:32:36.322 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-07-03T15:32:36.355 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-07-03T15:32:36.395 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-07-03T15:32:36.430 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-07-03T15:32:36.450 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-07-03T15:32:36.465 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-07-03T15:32:36.525 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-07-03T15:32:36.531 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-07-03T15:32:36.558 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-07-03T15:32:36.591 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-07-03T15:32:36.610 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-07-03T15:32:36.687 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-07-03T15:32:36.757 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-07-03T15:32:36.772 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-07-03T15:32:36.827 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-07-03T15:32:36.845 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-07-03T15:32:36.870 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-07-03T15:32:36.897 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-07-03T15:32:36.899 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-07-03T15:32:36.934 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-07-03T15:32:36.938 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-07-03T15:32:36.967 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-07-03T15:32:36.985 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-07-03T15:32:37.008 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-07-03T15:32:37.040 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-07-03T15:32:37.052 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-07-03T15:32:37.086 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-07-03T15:32:37.124 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-07-03T15:32:37.201 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-07-03T15:32:37.234 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-07-03T15:32:37.290 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-07-03T15:32:37.354 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-07-03T15:32:37.358 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-07-03T15:32:37.435 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-07-03T15:32:37.448 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-07-03T15:32:37.491 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-07-03T15:32:37.495 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-07-03T15:32:37.533 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-07-03T15:32:37.592 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-07-03T15:32:37.612 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-07-03T15:32:37.655 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-07-03T15:32:37.760 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1045.gc6021375.el9. 28/33 2024-07-03T15:32:37.783 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-2:17.2.7-1045.gc6021375.el9.x86_64 29/33 2024-07-03T15:32:37.809 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1045.gc6021375.el9.x86_64 29/33 2024-07-03T15:32:37.809 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-03T15:32:37.809 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-07-03T15:32:37.809 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-03T15:32:37.809 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-03T15:32:37.810 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:32:37.858 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-07-03T15:32:37.912 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-07-03T15:32:37.992 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-07-03T15:32:38.082 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-07-03T15:32:38.127 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-07-03T15:32:38.248 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-07-03T15:32:38.405 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1045.gc6021375.el9. 28/33 2024-07-03T15:32:38.429 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mgr-2:17.2.7-1045.gc6021375.el9.x86_64 29/33 2024-07-03T15:32:38.456 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1045.gc6021375.el9.x86_64 29/33 2024-07-03T15:32:38.456 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-03T15:32:38.456 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-07-03T15:32:38.457 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-03T15:32:38.457 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-03T15:32:38.457 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:32:39.126 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-osd-2:17.2.7-1045.gc6021375.el9.x86_64 30/33 2024-07-03T15:32:39.153 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-osd-2:17.2.7-1045.gc6021375.el9.x86_64 30/33 2024-07-03T15:32:39.153 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-03T15:32:39.153 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-07-03T15:32:39.153 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-03T15:32:39.153 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-03T15:32:39.153 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:32:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mon-2:17.2.7-1045.gc6021375.el9.x86_64 31/33 2024-07-03T15:32:39.567 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mon-2:17.2.7-1045.gc6021375.el9.x86_64 31/33 2024-07-03T15:32:39.567 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-03T15:32:39.567 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-07-03T15:32:39.567 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-03T15:32:39.567 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-03T15:32:39.567 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:32:39.779 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mds-2:17.2.7-1045.gc6021375.el9.x86_64 32/33 2024-07-03T15:32:39.803 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mds-2:17.2.7-1045.gc6021375.el9.x86_64 32/33 2024-07-03T15:32:39.803 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-03T15:32:39.803 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-07-03T15:32:39.803 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-03T15:32:39.803 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-03T15:32:39.803 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:32:39.859 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-2:17.2.7-1045.gc6021375.el9.x86_64 33/33 2024-07-03T15:32:39.863 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-osd-2:17.2.7-1045.gc6021375.el9.x86_64 30/33 2024-07-03T15:32:39.893 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-osd-2:17.2.7-1045.gc6021375.el9.x86_64 30/33 2024-07-03T15:32:39.894 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-03T15:32:39.894 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-07-03T15:32:39.894 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-03T15:32:39.894 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-03T15:32:39.894 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:32:40.289 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mon-2:17.2.7-1045.gc6021375.el9.x86_64 31/33 2024-07-03T15:32:40.315 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mon-2:17.2.7-1045.gc6021375.el9.x86_64 31/33 2024-07-03T15:32:40.315 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-03T15:32:40.315 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-07-03T15:32:40.315 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-03T15:32:40.315 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-03T15:32:40.315 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:32:40.532 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mds-2:17.2.7-1045.gc6021375.el9.x86_64 32/33 2024-07-03T15:32:40.557 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mds-2:17.2.7-1045.gc6021375.el9.x86_64 32/33 2024-07-03T15:32:40.557 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-03T15:32:40.557 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-07-03T15:32:40.557 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-03T15:32:40.557 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-03T15:32:40.558 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:32:40.623 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-2:17.2.7-1045.gc6021375.el9.x86_64 33/33 2024-07-03T15:32:41.582 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-2:17.2.7-1045.gc6021375.el9.x86_64 33/33 2024-07-03T15:32:41.582 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-2:17.2.7-1045.gc6021375.el9.x86_64 1/33 2024-07-03T15:32:41.582 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mds-2:17.2.7-1045.gc6021375.el9.x86_64 2/33 2024-07-03T15:32:41.582 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-2:17.2.7-1045.gc6021375.el9.x86_64 3/33 2024-07-03T15:32:41.582 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mon-2:17.2.7-1045.gc6021375.el9.x86_64 4/33 2024-07-03T15:32:41.582 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-osd-2:17.2.7-1045.gc6021375.el9.x86_64 5/33 2024-07-03T15:32:41.582 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1045.gc6021375.el9. 6/33 2024-07-03T15:32:41.583 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-07-03T15:32:41.583 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-07-03T15:32:41.583 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-07-03T15:32:41.583 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-07-03T15:32:41.583 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-07-03T15:32:41.583 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-07-03T15:32:41.583 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-07-03T15:32:41.583 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-07-03T15:32:41.583 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-07-03T15:32:41.583 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-07-03T15:32:41.583 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-07-03T15:32:41.583 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-07-03T15:32:41.584 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-07-03T15:32:41.584 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-07-03T15:32:41.584 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-07-03T15:32:41.584 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-07-03T15:32:41.584 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-07-03T15:32:41.584 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-07-03T15:32:41.584 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-07-03T15:32:41.584 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-07-03T15:32:41.584 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-07-03T15:32:41.584 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-07-03T15:32:41.584 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-07-03T15:32:41.584 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-07-03T15:32:41.584 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-07-03T15:32:41.584 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-07-03T15:32:42.269 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-2:17.2.7-1045.gc6021375.el9.x86_64 33/33 2024-07-03T15:32:42.270 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-2:17.2.7-1045.gc6021375.el9.x86_64 1/33 2024-07-03T15:32:42.270 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mds-2:17.2.7-1045.gc6021375.el9.x86_64 2/33 2024-07-03T15:32:42.270 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mgr-2:17.2.7-1045.gc6021375.el9.x86_64 3/33 2024-07-03T15:32:42.270 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mon-2:17.2.7-1045.gc6021375.el9.x86_64 4/33 2024-07-03T15:32:42.270 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-osd-2:17.2.7-1045.gc6021375.el9.x86_64 5/33 2024-07-03T15:32:42.270 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1045.gc6021375.el9. 6/33 2024-07-03T15:32:42.270 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-07-03T15:32:42.270 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-07-03T15:32:42.270 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-07-03T15:32:42.270 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-07-03T15:32:42.270 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-07-03T15:32:42.270 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-07-03T15:32:42.271 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-07-03T15:32:42.271 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-07-03T15:32:42.271 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-07-03T15:32:42.271 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-07-03T15:32:42.271 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-07-03T15:32:42.271 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-07-03T15:32:42.271 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-07-03T15:32:42.271 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-07-03T15:32:42.271 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-07-03T15:32:42.271 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-07-03T15:32:42.271 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-07-03T15:32:42.271 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-07-03T15:32:42.272 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-07-03T15:32:42.272 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-07-03T15:32:42.273 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-07-03T15:32:42.273 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-07-03T15:32:42.273 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-07-03T15:32:42.273 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-07-03T15:32:42.273 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-07-03T15:32:42.273 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-07-03T15:32:42.579 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-07-03T15:32:42.579 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:32:42.580 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-07-03T15:32:42.580 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:32:42.580 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mds-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:32:42.580 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:32:42.580 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-modules-core-2:17.2.7-1045.gc6021375.el9.noarch 2024-07-03T15:32:42.580 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mon-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:32:42.580 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-osd-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:32:42.580 INFO:teuthology.orchestra.run.smithi148.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-07-03T15:32:42.580 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-07-03T15:32:42.580 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-07-03T15:32:42.580 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-07-03T15:32:42.580 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-07-03T15:32:42.581 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-07-03T15:32:42.581 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-07-03T15:32:42.581 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-07-03T15:32:42.581 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-07-03T15:32:42.581 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-07-03T15:32:42.581 INFO:teuthology.orchestra.run.smithi148.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-07-03T15:32:42.581 INFO:teuthology.orchestra.run.smithi148.stdout: python3-mako-1.1.4-6.el9.noarch 2024-07-03T15:32:42.581 INFO:teuthology.orchestra.run.smithi148.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-07-03T15:32:42.581 INFO:teuthology.orchestra.run.smithi148.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-07-03T15:32:42.581 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-07-03T15:32:42.581 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ply-3.11-14.el9.noarch 2024-07-03T15:32:42.581 INFO:teuthology.orchestra.run.smithi148.stdout: python3-portend-3.1.0-2.el9.noarch 2024-07-03T15:32:42.581 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-07-03T15:32:42.582 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pycparser-2.20-6.el9.noarch 2024-07-03T15:32:42.582 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pytz-2021.1-5.el9.noarch 2024-07-03T15:32:42.582 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-2.25.1-8.el9.noarch 2024-07-03T15:32:42.582 INFO:teuthology.orchestra.run.smithi148.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-07-03T15:32:42.582 INFO:teuthology.orchestra.run.smithi148.stdout: python3-toml-0.10.2-6.el9.noarch 2024-07-03T15:32:42.582 INFO:teuthology.orchestra.run.smithi148.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-07-03T15:32:42.582 INFO:teuthology.orchestra.run.smithi148.stdout: python3-webob-1.8.7-6.el9.noarch 2024-07-03T15:32:42.582 INFO:teuthology.orchestra.run.smithi148.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-07-03T15:32:42.582 INFO:teuthology.orchestra.run.smithi148.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-07-03T15:32:42.582 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:32:42.582 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-07-03T15:32:42.941 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-base 2024-07-03T15:32:43.101 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-07-03T15:32:43.101 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:32:43.101 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-07-03T15:32:43.101 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:32:43.101 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mds-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:32:43.101 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:32:43.101 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-modules-core-2:17.2.7-1045.gc6021375.el9.noarch 2024-07-03T15:32:43.101 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mon-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:32:43.101 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-osd-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:32:43.102 INFO:teuthology.orchestra.run.smithi028.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-07-03T15:32:43.102 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-07-03T15:32:43.102 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-07-03T15:32:43.102 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-07-03T15:32:43.102 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-07-03T15:32:43.102 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-07-03T15:32:43.102 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-07-03T15:32:43.102 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-07-03T15:32:43.102 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-07-03T15:32:43.102 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-07-03T15:32:43.102 INFO:teuthology.orchestra.run.smithi028.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-07-03T15:32:43.102 INFO:teuthology.orchestra.run.smithi028.stdout: python3-mako-1.1.4-6.el9.noarch 2024-07-03T15:32:43.103 INFO:teuthology.orchestra.run.smithi028.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-07-03T15:32:43.103 INFO:teuthology.orchestra.run.smithi028.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-07-03T15:32:43.103 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-07-03T15:32:43.103 INFO:teuthology.orchestra.run.smithi028.stdout: python3-ply-3.11-14.el9.noarch 2024-07-03T15:32:43.103 INFO:teuthology.orchestra.run.smithi028.stdout: python3-portend-3.1.0-2.el9.noarch 2024-07-03T15:32:43.103 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-07-03T15:32:43.103 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pycparser-2.20-6.el9.noarch 2024-07-03T15:32:43.103 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pytz-2021.1-5.el9.noarch 2024-07-03T15:32:43.103 INFO:teuthology.orchestra.run.smithi028.stdout: python3-requests-2.25.1-8.el9.noarch 2024-07-03T15:32:43.103 INFO:teuthology.orchestra.run.smithi028.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-07-03T15:32:43.103 INFO:teuthology.orchestra.run.smithi028.stdout: python3-toml-0.10.2-6.el9.noarch 2024-07-03T15:32:43.103 INFO:teuthology.orchestra.run.smithi028.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-07-03T15:32:43.103 INFO:teuthology.orchestra.run.smithi028.stdout: python3-webob-1.8.7-6.el9.noarch 2024-07-03T15:32:43.104 INFO:teuthology.orchestra.run.smithi028.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-07-03T15:32:43.104 INFO:teuthology.orchestra.run.smithi028.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-07-03T15:32:43.104 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:32:43.104 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-07-03T15:32:43.444 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install ceph-base 2024-07-03T15:32:43.504 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:00:50 ago on Wed 03 Jul 2024 03:31:53 PM UTC. 2024-07-03T15:32:43.590 INFO:teuthology.orchestra.run.smithi148.stdout:Package ceph-base-2:17.2.7-1045.gc6021375.el9.x86_64 is already installed. 2024-07-03T15:32:43.632 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-07-03T15:32:43.633 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-07-03T15:32:43.633 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-07-03T15:32:43.702 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install cephadm 2024-07-03T15:32:44.030 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:00:53 ago on Wed 03 Jul 2024 03:31:51 PM UTC. 2024-07-03T15:32:44.119 INFO:teuthology.orchestra.run.smithi028.stdout:Package ceph-base-2:17.2.7-1045.gc6021375.el9.x86_64 is already installed. 2024-07-03T15:32:44.164 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-07-03T15:32:44.165 INFO:teuthology.orchestra.run.smithi028.stdout:Nothing to do. 2024-07-03T15:32:44.165 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-07-03T15:32:44.237 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install cephadm 2024-07-03T15:32:44.258 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:00:51 ago on Wed 03 Jul 2024 03:31:53 PM UTC. 2024-07-03T15:32:44.386 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-07-03T15:32:44.387 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-07-03T15:32:44.387 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-07-03T15:32:44.387 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-07-03T15:32:44.387 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-07-03T15:32:44.387 INFO:teuthology.orchestra.run.smithi148.stdout: cephadm noarch 2:17.2.7-1045.gc6021375.el9 ceph-noarch 82 k 2024-07-03T15:32:44.387 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:32:44.387 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-07-03T15:32:44.387 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-07-03T15:32:44.388 INFO:teuthology.orchestra.run.smithi148.stdout:Install 1 Package 2024-07-03T15:32:44.388 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:32:44.388 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 82 k 2024-07-03T15:32:44.388 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 372 k 2024-07-03T15:32:44.388 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-07-03T15:32:44.704 INFO:teuthology.orchestra.run.smithi148.stdout:cephadm-17.2.7-1045.gc6021375.el9.noarch.rpm 260 kB/s | 82 kB 00:00 2024-07-03T15:32:44.704 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-07-03T15:32:44.704 INFO:teuthology.orchestra.run.smithi148.stdout:Total 258 kB/s | 82 kB 00:00 2024-07-03T15:32:44.705 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-07-03T15:32:44.710 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-07-03T15:32:44.710 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-07-03T15:32:44.712 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-07-03T15:32:44.712 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-07-03T15:32:44.783 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-07-03T15:32:44.818 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:00:53 ago on Wed 03 Jul 2024 03:31:51 PM UTC. 2024-07-03T15:32:44.951 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-07-03T15:32:44.952 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-07-03T15:32:44.952 INFO:teuthology.orchestra.run.smithi028.stdout: Package Arch Version Repository Size 2024-07-03T15:32:44.952 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-07-03T15:32:44.952 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-07-03T15:32:44.952 INFO:teuthology.orchestra.run.smithi028.stdout: cephadm noarch 2:17.2.7-1045.gc6021375.el9 ceph-noarch 82 k 2024-07-03T15:32:44.953 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:32:44.953 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-07-03T15:32:44.953 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-07-03T15:32:44.953 INFO:teuthology.orchestra.run.smithi028.stdout:Install 1 Package 2024-07-03T15:32:44.953 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:32:44.953 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 82 k 2024-07-03T15:32:44.953 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 372 k 2024-07-03T15:32:44.953 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-07-03T15:32:45.227 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: cephadm-2:17.2.7-1045.gc6021375.el9.noarch 1/1 2024-07-03T15:32:45.263 INFO:teuthology.orchestra.run.smithi028.stdout:cephadm-17.2.7-1045.gc6021375.el9.noarch.rpm 264 kB/s | 82 kB 00:00 2024-07-03T15:32:45.264 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-07-03T15:32:45.264 INFO:teuthology.orchestra.run.smithi028.stdout:Total 263 kB/s | 82 kB 00:00 2024-07-03T15:32:45.265 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-07-03T15:32:45.270 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-07-03T15:32:45.270 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-07-03T15:32:45.272 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-07-03T15:32:45.272 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-07-03T15:32:45.324 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephadm-2:17.2.7-1045.gc6021375.el9.noarch 1/1 2024-07-03T15:32:45.344 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-07-03T15:32:45.692 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: cephadm-2:17.2.7-1045.gc6021375.el9.noarch 1/1 2024-07-03T15:32:45.808 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: cephadm-2:17.2.7-1045.gc6021375.el9.noarch 1/1 2024-07-03T15:32:45.922 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : cephadm-2:17.2.7-1045.gc6021375.el9.noarch 1/1 2024-07-03T15:32:45.955 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephadm-2:17.2.7-1045.gc6021375.el9.noarch 1/1 2024-07-03T15:32:45.955 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:32:45.955 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-07-03T15:32:45.956 INFO:teuthology.orchestra.run.smithi148.stdout: cephadm-2:17.2.7-1045.gc6021375.el9.noarch 2024-07-03T15:32:45.956 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:32:45.956 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-07-03T15:32:46.178 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-immutable-object-cache 2024-07-03T15:32:46.271 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: cephadm-2:17.2.7-1045.gc6021375.el9.noarch 1/1 2024-07-03T15:32:46.539 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : cephadm-2:17.2.7-1045.gc6021375.el9.noarch 1/1 2024-07-03T15:32:46.539 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:32:46.539 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-07-03T15:32:46.539 INFO:teuthology.orchestra.run.smithi028.stdout: cephadm-2:17.2.7-1045.gc6021375.el9.noarch 2024-07-03T15:32:46.539 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:32:46.540 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-07-03T15:32:46.737 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:00:53 ago on Wed 03 Jul 2024 03:31:53 PM UTC. 2024-07-03T15:32:46.741 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install ceph-immutable-object-cache 2024-07-03T15:32:46.867 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-07-03T15:32:46.868 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-07-03T15:32:46.868 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repo Size 2024-07-03T15:32:46.868 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-07-03T15:32:46.868 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-07-03T15:32:46.868 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1045.gc6021375.el9 ceph 137 k 2024-07-03T15:32:46.868 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:32:46.868 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-07-03T15:32:46.868 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-07-03T15:32:46.868 INFO:teuthology.orchestra.run.smithi148.stdout:Install 1 Package 2024-07-03T15:32:46.868 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:32:46.869 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 137 k 2024-07-03T15:32:46.869 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 379 k 2024-07-03T15:32:46.869 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-07-03T15:32:47.176 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-immutable-object-cache-17.2.7-1045.gc60213 447 kB/s | 137 kB 00:00 2024-07-03T15:32:47.176 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-07-03T15:32:47.177 INFO:teuthology.orchestra.run.smithi148.stdout:Total 445 kB/s | 137 kB 00:00 2024-07-03T15:32:47.177 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-07-03T15:32:47.185 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-07-03T15:32:47.185 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-07-03T15:32:47.241 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-07-03T15:32:47.241 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-07-03T15:32:47.325 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:00:56 ago on Wed 03 Jul 2024 03:31:51 PM UTC. 2024-07-03T15:32:47.378 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-07-03T15:32:47.461 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-07-03T15:32:47.462 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-07-03T15:32:47.462 INFO:teuthology.orchestra.run.smithi028.stdout: Package Arch Version Repo Size 2024-07-03T15:32:47.462 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-07-03T15:32:47.462 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-07-03T15:32:47.462 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1045.gc6021375.el9 ceph 137 k 2024-07-03T15:32:47.462 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:32:47.462 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-07-03T15:32:47.463 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-07-03T15:32:47.463 INFO:teuthology.orchestra.run.smithi028.stdout:Install 1 Package 2024-07-03T15:32:47.463 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:32:47.463 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 137 k 2024-07-03T15:32:47.463 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 379 k 2024-07-03T15:32:47.463 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-07-03T15:32:47.469 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1045.gc6021375. 1/1 2024-07-03T15:32:47.494 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1045.gc6021375. 1/1 2024-07-03T15:32:47.494 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-03T15:32:47.494 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-07-03T15:32:47.494 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:32:47.688 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-immutable-object-cache-17.2.7-1045.gc60213 612 kB/s | 137 kB 00:00 2024-07-03T15:32:47.688 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-07-03T15:32:47.688 INFO:teuthology.orchestra.run.smithi028.stdout:Total 608 kB/s | 137 kB 00:00 2024-07-03T15:32:47.689 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-07-03T15:32:47.697 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-07-03T15:32:47.697 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-07-03T15:32:47.756 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-07-03T15:32:47.757 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-07-03T15:32:47.905 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-07-03T15:32:47.972 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1045.gc6021375. 1/1 2024-07-03T15:32:47.998 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1045.gc6021375. 1/1 2024-07-03T15:32:47.999 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-03T15:32:47.999 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-07-03T15:32:47.999 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:32:48.470 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1045.gc6021375. 1/1 2024-07-03T15:32:48.470 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:32:48.470 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-07-03T15:32:48.470 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-immutable-object-cache-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:32:48.470 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:32:48.470 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-07-03T15:32:48.670 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-mgr 2024-07-03T15:32:48.972 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1045.gc6021375. 1/1 2024-07-03T15:32:48.972 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:32:48.973 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-07-03T15:32:48.973 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-immutable-object-cache-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:32:48.973 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:32:48.973 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-07-03T15:32:49.184 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install ceph-mgr 2024-07-03T15:32:49.236 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:00:56 ago on Wed 03 Jul 2024 03:31:53 PM UTC. 2024-07-03T15:32:49.327 INFO:teuthology.orchestra.run.smithi148.stdout:Package ceph-mgr-2:17.2.7-1045.gc6021375.el9.x86_64 is already installed. 2024-07-03T15:32:49.369 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-07-03T15:32:49.370 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-07-03T15:32:49.370 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-07-03T15:32:49.447 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-mgr-dashboard 2024-07-03T15:32:49.767 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:00:58 ago on Wed 03 Jul 2024 03:31:51 PM UTC. 2024-07-03T15:32:49.857 INFO:teuthology.orchestra.run.smithi028.stdout:Package ceph-mgr-2:17.2.7-1045.gc6021375.el9.x86_64 is already installed. 2024-07-03T15:32:49.901 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-07-03T15:32:49.902 INFO:teuthology.orchestra.run.smithi028.stdout:Nothing to do. 2024-07-03T15:32:49.902 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-07-03T15:32:49.969 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install ceph-mgr-dashboard 2024-07-03T15:32:50.000 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:00:56 ago on Wed 03 Jul 2024 03:31:53 PM UTC. 2024-07-03T15:32:50.127 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-07-03T15:32:50.128 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-07-03T15:32:50.128 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-07-03T15:32:50.128 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-07-03T15:32:50.128 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-07-03T15:32:50.128 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1045.gc6021375.el9 ceph-noarch 3.4 M 2024-07-03T15:32:50.128 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-07-03T15:32:50.128 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1045.gc6021375.el9 ceph-noarch 24 k 2024-07-03T15:32:50.128 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1045.gc6021375.el9 ceph-noarch 15 k 2024-07-03T15:32:50.128 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-07-03T15:32:50.128 INFO:teuthology.orchestra.run.smithi148.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-07-03T15:32:50.128 INFO:teuthology.orchestra.run.smithi148.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-07-03T15:32:50.129 INFO:teuthology.orchestra.run.smithi148.stdout:Installing weak dependencies: 2024-07-03T15:32:50.129 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-07-03T15:32:50.129 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:32:50.129 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-07-03T15:32:50.129 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-07-03T15:32:50.129 INFO:teuthology.orchestra.run.smithi148.stdout:Install 7 Packages 2024-07-03T15:32:50.129 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:32:50.130 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 3.7 M 2024-07-03T15:32:50.130 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 80 M 2024-07-03T15:32:50.130 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-07-03T15:32:50.548 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:00:59 ago on Wed 03 Jul 2024 03:31:51 PM UTC. 2024-07-03T15:32:50.682 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-07-03T15:32:50.683 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-07-03T15:32:50.683 INFO:teuthology.orchestra.run.smithi028.stdout: Package Arch Version Repository Size 2024-07-03T15:32:50.683 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-07-03T15:32:50.683 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-07-03T15:32:50.683 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1045.gc6021375.el9 ceph-noarch 3.4 M 2024-07-03T15:32:50.683 INFO:teuthology.orchestra.run.smithi028.stdout:Installing dependencies: 2024-07-03T15:32:50.683 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1045.gc6021375.el9 ceph-noarch 24 k 2024-07-03T15:32:50.683 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1045.gc6021375.el9 ceph-noarch 15 k 2024-07-03T15:32:50.683 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-07-03T15:32:50.684 INFO:teuthology.orchestra.run.smithi028.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-07-03T15:32:50.684 INFO:teuthology.orchestra.run.smithi028.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-07-03T15:32:50.684 INFO:teuthology.orchestra.run.smithi028.stdout:Installing weak dependencies: 2024-07-03T15:32:50.684 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-07-03T15:32:50.684 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:32:50.684 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-07-03T15:32:50.684 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-07-03T15:32:50.684 INFO:teuthology.orchestra.run.smithi028.stdout:Install 7 Packages 2024-07-03T15:32:50.684 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:32:50.685 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 3.7 M 2024-07-03T15:32:50.685 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 80 M 2024-07-03T15:32:50.685 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-07-03T15:32:50.813 INFO:teuthology.orchestra.run.smithi148.stdout:(1/7): ceph-prometheus-alerts-17.2.7-1045.gc602 56 kB/s | 15 kB 00:00 2024-07-03T15:32:50.839 INFO:teuthology.orchestra.run.smithi148.stdout:(2/7): ceph-grafana-dashboards-17.2.7-1045.gc60 86 kB/s | 24 kB 00:00 2024-07-03T15:32:50.922 INFO:teuthology.orchestra.run.smithi148.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 83 kB/s | 9.0 kB 00:00 2024-07-03T15:32:50.941 INFO:teuthology.orchestra.run.smithi028.stdout:(1/7): ceph-grafana-dashboards-17.2.7-1045.gc60 127 kB/s | 24 kB 00:00 2024-07-03T15:32:50.947 INFO:teuthology.orchestra.run.smithi148.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 1.3 MB/s | 31 kB 00:00 2024-07-03T15:32:50.992 INFO:teuthology.orchestra.run.smithi028.stdout:(2/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 181 kB/s | 9.0 kB 00:00 2024-07-03T15:32:50.997 INFO:teuthology.orchestra.run.smithi148.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 257 kB/s | 41 kB 00:00 2024-07-03T15:32:51.008 INFO:teuthology.orchestra.run.smithi028.stdout:(3/7): python3-jwt-2.4.0-1.el9.noarch.rpm 2.4 MB/s | 41 kB 00:00 2024-07-03T15:32:51.023 INFO:teuthology.orchestra.run.smithi148.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 2.4 MB/s | 188 kB 00:00 2024-07-03T15:32:51.025 INFO:teuthology.orchestra.run.smithi028.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 1.8 MB/s | 31 kB 00:00 2024-07-03T15:32:51.042 INFO:teuthology.orchestra.run.smithi028.stdout:(5/7): ceph-prometheus-alerts-17.2.7-1045.gc602 50 kB/s | 15 kB 00:00 2024-07-03T15:32:51.067 INFO:teuthology.orchestra.run.smithi028.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 4.4 MB/s | 188 kB 00:00 2024-07-03T15:32:51.081 INFO:teuthology.orchestra.run.smithi148.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1045.gc6021375 6.4 MB/s | 3.4 MB 00:00 2024-07-03T15:32:51.085 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-07-03T15:32:51.085 INFO:teuthology.orchestra.run.smithi148.stdout:Total 3.8 MB/s | 3.7 MB 00:00 2024-07-03T15:32:51.085 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-07-03T15:32:51.093 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-07-03T15:32:51.093 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-07-03T15:32:51.155 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-07-03T15:32:51.156 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-07-03T15:32:51.359 INFO:teuthology.orchestra.run.smithi028.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1045.gc6021375 5.5 MB/s | 3.4 MB 00:00 2024-07-03T15:32:51.363 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-07-03T15:32:51.363 INFO:teuthology.orchestra.run.smithi028.stdout:Total 5.4 MB/s | 3.7 MB 00:00 2024-07-03T15:32:51.363 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-07-03T15:32:51.372 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-07-03T15:32:51.372 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-07-03T15:32:51.439 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-07-03T15:32:51.441 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-07-03T15:32:51.475 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-07-03T15:32:51.533 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-07-03T15:32:51.566 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-07-03T15:32:51.612 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-07-03T15:32:51.645 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-07-03T15:32:51.674 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1045.gc6021375.el9.n 5/7 2024-07-03T15:32:51.742 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-07-03T15:32:51.794 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-07-03T15:32:51.829 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-07-03T15:32:51.876 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-07-03T15:32:51.906 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-07-03T15:32:51.936 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1045.gc6021375.el9.n 5/7 2024-07-03T15:32:52.669 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1045.gc6021375.el9. 6/7 2024-07-03T15:32:52.711 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1045.gc6021375.el9.noarc 7/7 2024-07-03T15:32:52.990 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1045.gc6021375.el9. 6/7 2024-07-03T15:32:53.030 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1045.gc6021375.el9.noarc 7/7 2024-07-03T15:32:53.760 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1045.gc6021375.el9.noarc 7/7 2024-07-03T15:32:53.760 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1045.gc6021375.el9. 1/7 2024-07-03T15:32:53.760 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1045.gc6021375.el9.noarc 2/7 2024-07-03T15:32:53.760 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1045.gc6021375.el9.n 3/7 2024-07-03T15:32:53.760 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-07-03T15:32:53.760 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-07-03T15:32:53.761 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-07-03T15:32:54.056 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1045.gc6021375.el9.noarc 7/7 2024-07-03T15:32:54.056 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1045.gc6021375.el9. 1/7 2024-07-03T15:32:54.056 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1045.gc6021375.el9.noarc 2/7 2024-07-03T15:32:54.056 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1045.gc6021375.el9.n 3/7 2024-07-03T15:32:54.056 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-07-03T15:32:54.056 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-07-03T15:32:54.057 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-07-03T15:32:54.077 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-07-03T15:32:54.078 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:32:54.078 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-07-03T15:32:54.078 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-grafana-dashboards-2:17.2.7-1045.gc6021375.el9.noarch 2024-07-03T15:32:54.078 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-dashboard-2:17.2.7-1045.gc6021375.el9.noarch 2024-07-03T15:32:54.078 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-prometheus-alerts-2:17.2.7-1045.gc6021375.el9.noarch 2024-07-03T15:32:54.078 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-07-03T15:32:54.078 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-07-03T15:32:54.078 INFO:teuthology.orchestra.run.smithi148.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-07-03T15:32:54.078 INFO:teuthology.orchestra.run.smithi148.stdout: python3-routes-2.5.1-5.el9.noarch 2024-07-03T15:32:54.078 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:32:54.078 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-07-03T15:32:54.286 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-mgr-diskprediction-local 2024-07-03T15:32:54.389 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-07-03T15:32:54.389 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:32:54.390 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-07-03T15:32:54.390 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-grafana-dashboards-2:17.2.7-1045.gc6021375.el9.noarch 2024-07-03T15:32:54.390 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-dashboard-2:17.2.7-1045.gc6021375.el9.noarch 2024-07-03T15:32:54.390 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-prometheus-alerts-2:17.2.7-1045.gc6021375.el9.noarch 2024-07-03T15:32:54.390 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-07-03T15:32:54.390 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-07-03T15:32:54.390 INFO:teuthology.orchestra.run.smithi028.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-07-03T15:32:54.390 INFO:teuthology.orchestra.run.smithi028.stdout: python3-routes-2.5.1-5.el9.noarch 2024-07-03T15:32:54.390 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:32:54.390 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-07-03T15:32:54.563 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install ceph-mgr-diskprediction-local 2024-07-03T15:32:54.844 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:01 ago on Wed 03 Jul 2024 03:31:53 PM UTC. 2024-07-03T15:32:54.970 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-07-03T15:32:54.971 INFO:teuthology.orchestra.run.smithi148.stdout:======================================================================================= 2024-07-03T15:32:54.972 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-07-03T15:32:54.972 INFO:teuthology.orchestra.run.smithi148.stdout:======================================================================================= 2024-07-03T15:32:54.972 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-07-03T15:32:54.972 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1045.gc6021375.el9 ceph-noarch 7.4 M 2024-07-03T15:32:54.972 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-07-03T15:32:54.972 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-07-03T15:32:54.972 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-07-03T15:32:54.972 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-07-03T15:32:54.972 INFO:teuthology.orchestra.run.smithi148.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-07-03T15:32:54.972 INFO:teuthology.orchestra.run.smithi148.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-07-03T15:32:54.972 INFO:teuthology.orchestra.run.smithi148.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-07-03T15:32:54.972 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-07-03T15:32:54.973 INFO:teuthology.orchestra.run.smithi148.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-07-03T15:32:54.973 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-07-03T15:32:54.973 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-07-03T15:32:54.973 INFO:teuthology.orchestra.run.smithi148.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-07-03T15:32:54.973 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:32:54.973 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-07-03T15:32:54.973 INFO:teuthology.orchestra.run.smithi148.stdout:======================================================================================= 2024-07-03T15:32:54.973 INFO:teuthology.orchestra.run.smithi148.stdout:Install 12 Packages 2024-07-03T15:32:54.973 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:32:54.974 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 38 M 2024-07-03T15:32:54.974 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 209 M 2024-07-03T15:32:54.974 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-07-03T15:32:55.141 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:04 ago on Wed 03 Jul 2024 03:31:51 PM UTC. 2024-07-03T15:32:55.239 INFO:teuthology.orchestra.run.smithi148.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-07-03T15:32:55.274 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-07-03T15:32:55.275 INFO:teuthology.orchestra.run.smithi028.stdout:======================================================================================= 2024-07-03T15:32:55.275 INFO:teuthology.orchestra.run.smithi028.stdout: Package Arch Version Repository Size 2024-07-03T15:32:55.275 INFO:teuthology.orchestra.run.smithi028.stdout:======================================================================================= 2024-07-03T15:32:55.275 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-07-03T15:32:55.275 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1045.gc6021375.el9 ceph-noarch 7.4 M 2024-07-03T15:32:55.275 INFO:teuthology.orchestra.run.smithi028.stdout:Installing dependencies: 2024-07-03T15:32:55.275 INFO:teuthology.orchestra.run.smithi028.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-07-03T15:32:55.275 INFO:teuthology.orchestra.run.smithi028.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-07-03T15:32:55.276 INFO:teuthology.orchestra.run.smithi028.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-07-03T15:32:55.276 INFO:teuthology.orchestra.run.smithi028.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-07-03T15:32:55.276 INFO:teuthology.orchestra.run.smithi028.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-07-03T15:32:55.276 INFO:teuthology.orchestra.run.smithi028.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-07-03T15:32:55.276 INFO:teuthology.orchestra.run.smithi028.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-07-03T15:32:55.276 INFO:teuthology.orchestra.run.smithi028.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-07-03T15:32:55.276 INFO:teuthology.orchestra.run.smithi028.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-07-03T15:32:55.276 INFO:teuthology.orchestra.run.smithi028.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-07-03T15:32:55.276 INFO:teuthology.orchestra.run.smithi028.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-07-03T15:32:55.276 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:32:55.276 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-07-03T15:32:55.276 INFO:teuthology.orchestra.run.smithi028.stdout:======================================================================================= 2024-07-03T15:32:55.277 INFO:teuthology.orchestra.run.smithi028.stdout:Install 12 Packages 2024-07-03T15:32:55.277 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:32:55.277 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 38 M 2024-07-03T15:32:55.277 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 209 M 2024-07-03T15:32:55.277 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-07-03T15:32:55.339 INFO:teuthology.orchestra.run.smithi148.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 2.7 MB/s | 803 kB 00:00 2024-07-03T15:32:55.581 INFO:teuthology.orchestra.run.smithi148.stdout:(3/12): ceph-mgr-diskprediction-local-17.2.7-10 14 MB/s | 7.4 MB 00:00 2024-07-03T15:32:55.606 INFO:teuthology.orchestra.run.smithi148.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 89 kB/s | 33 kB 00:00 2024-07-03T15:32:55.759 INFO:teuthology.orchestra.run.smithi028.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 1.4 MB/s | 195 kB 00:00 2024-07-03T15:32:55.790 INFO:teuthology.orchestra.run.smithi148.stdout:(5/12): openblas-0.3.21-2.el9.x86_64.rpm 192 kB/s | 35 kB 00:00 2024-07-03T15:32:55.809 INFO:teuthology.orchestra.run.smithi028.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 4.2 MB/s | 803 kB 00:00 2024-07-03T15:32:55.849 INFO:teuthology.orchestra.run.smithi148.stdout:(6/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 68 kB/s | 18 kB 00:00 2024-07-03T15:32:55.893 INFO:teuthology.orchestra.run.smithi028.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 245 kB/s | 33 kB 00:00 2024-07-03T15:32:55.951 INFO:teuthology.orchestra.run.smithi028.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 312 kB/s | 18 kB 00:00 2024-07-03T15:32:56.010 INFO:teuthology.orchestra.run.smithi028.stdout:(5/12): openblas-0.3.21-2.el9.x86_64.rpm 603 kB/s | 35 kB 00:00 2024-07-03T15:32:56.235 INFO:teuthology.orchestra.run.smithi028.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 7.0 MB/s | 3.0 MB 00:00 2024-07-03T15:32:56.302 INFO:teuthology.orchestra.run.smithi028.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 3.6 MB/s | 244 kB 00:00 2024-07-03T15:32:56.307 INFO:teuthology.orchestra.run.smithi148.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 533 kB/s | 244 kB 00:00 2024-07-03T15:32:56.491 INFO:teuthology.orchestra.run.smithi148.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.6 MB/s | 3.0 MB 00:01 2024-07-03T15:32:56.510 INFO:teuthology.orchestra.run.smithi028.stdout:(8/12): ceph-mgr-diskprediction-local-17.2.7-10 8.3 MB/s | 7.4 MB 00:00 2024-07-03T15:32:56.599 INFO:teuthology.orchestra.run.smithi148.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.3 MB/s | 250 kB 00:00 2024-07-03T15:32:56.661 INFO:teuthology.orchestra.run.smithi028.stdout:(9/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 14 MB/s | 5.1 MB 00:00 2024-07-03T15:32:56.775 INFO:teuthology.orchestra.run.smithi148.stdout:(10/12): openblas-openmp-0.3.21-2.el9.x86_64.rp 4.8 MB/s | 4.7 MB 00:00 2024-07-03T15:32:56.819 INFO:teuthology.orchestra.run.smithi028.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 813 kB/s | 250 kB 00:00 2024-07-03T15:32:56.919 INFO:teuthology.orchestra.run.smithi028.stdout:(11/12): openblas-openmp-0.3.21-2.el9.x86_64.rp 5.2 MB/s | 4.7 MB 00:00 2024-07-03T15:32:57.117 INFO:teuthology.orchestra.run.smithi148.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 6.3 MB/s | 5.1 MB 00:00 2024-07-03T15:32:57.336 INFO:teuthology.orchestra.run.smithi028.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-07-03T15:32:57.337 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-07-03T15:32:57.337 INFO:teuthology.orchestra.run.smithi028.stdout:Total 18 MB/s | 38 MB 00:02 2024-07-03T15:32:57.700 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-07-03T15:32:57.728 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-07-03T15:32:57.728 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-07-03T15:32:58.259 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-07-03T15:32:58.259 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-07-03T15:32:59.180 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-07-03T15:32:59.276 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-07-03T15:32:59.329 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-07-03T15:32:59.432 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-07-03T15:32:59.478 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-07-03T15:32:59.661 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-07-03T15:32:59.693 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-07-03T15:32:59.787 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-07-03T15:33:00.480 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-07-03T15:33:00.602 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-07-03T15:33:01.924 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-07-03T15:33:03.003 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-07-03T15:33:03.032 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1045.gc6021 12/12 2024-07-03T15:33:05.551 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1045.gc6021 12/12 2024-07-03T15:33:05.551 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1045.gc6021 1/12 2024-07-03T15:33:05.551 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-07-03T15:33:05.551 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-07-03T15:33:05.551 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-07-03T15:33:05.551 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-07-03T15:33:05.551 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-07-03T15:33:05.551 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-07-03T15:33:05.551 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-07-03T15:33:05.551 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-07-03T15:33:05.552 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-07-03T15:33:05.552 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-07-03T15:33:05.958 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-07-03T15:33:05.958 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:33:05.958 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-07-03T15:33:05.958 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1045.gc6021375.el9.noarch 2024-07-03T15:33:05.958 INFO:teuthology.orchestra.run.smithi028.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-07-03T15:33:05.959 INFO:teuthology.orchestra.run.smithi028.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-07-03T15:33:05.959 INFO:teuthology.orchestra.run.smithi028.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-07-03T15:33:05.959 INFO:teuthology.orchestra.run.smithi028.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-07-03T15:33:05.959 INFO:teuthology.orchestra.run.smithi028.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-07-03T15:33:05.960 INFO:teuthology.orchestra.run.smithi028.stdout: openblas-0.3.21-2.el9.x86_64 2024-07-03T15:33:05.960 INFO:teuthology.orchestra.run.smithi028.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-07-03T15:33:05.960 INFO:teuthology.orchestra.run.smithi028.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-07-03T15:33:05.960 INFO:teuthology.orchestra.run.smithi028.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-07-03T15:33:05.960 INFO:teuthology.orchestra.run.smithi028.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-07-03T15:33:05.960 INFO:teuthology.orchestra.run.smithi028.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-07-03T15:33:05.960 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:33:05.960 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-07-03T15:33:06.304 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install ceph-mgr-rook 2024-07-03T15:33:06.884 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:15 ago on Wed 03 Jul 2024 03:31:51 PM UTC. 2024-07-03T15:33:07.017 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-07-03T15:33:07.018 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-07-03T15:33:07.019 INFO:teuthology.orchestra.run.smithi028.stdout: Package Arch Version Repository Size 2024-07-03T15:33:07.019 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-07-03T15:33:07.019 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-07-03T15:33:07.019 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-rook noarch 2:17.2.7-1045.gc6021375.el9 ceph-noarch 47 k 2024-07-03T15:33:07.019 INFO:teuthology.orchestra.run.smithi028.stdout:Installing dependencies: 2024-07-03T15:33:07.019 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-07-03T15:33:07.019 INFO:teuthology.orchestra.run.smithi028.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-07-03T15:33:07.019 INFO:teuthology.orchestra.run.smithi028.stdout: python3-google-auth noarch 1:2.30.0-1.el9 epel 213 k 2024-07-03T15:33:07.019 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-07-03T15:33:07.019 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-07-03T15:33:07.019 INFO:teuthology.orchestra.run.smithi028.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-07-03T15:33:07.019 INFO:teuthology.orchestra.run.smithi028.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-07-03T15:33:07.019 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-07-03T15:33:07.020 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-07-03T15:33:07.020 INFO:teuthology.orchestra.run.smithi028.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-07-03T15:33:07.020 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-07-03T15:33:07.020 INFO:teuthology.orchestra.run.smithi028.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-07-03T15:33:07.020 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:33:07.020 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-07-03T15:33:07.020 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-07-03T15:33:07.020 INFO:teuthology.orchestra.run.smithi028.stdout:Install 13 Packages 2024-07-03T15:33:07.020 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:33:07.021 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 2.2 M 2024-07-03T15:33:07.021 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 26 M 2024-07-03T15:33:07.021 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-07-03T15:33:07.831 INFO:teuthology.orchestra.run.smithi028.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 190 kB/s | 19 kB 00:00 2024-07-03T15:33:07.857 INFO:teuthology.orchestra.run.smithi028.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 202 kB/s | 26 kB 00:00 2024-07-03T15:33:07.982 INFO:teuthology.orchestra.run.smithi028.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.5 MB/s | 222 kB 00:00 2024-07-03T15:33:08.008 INFO:teuthology.orchestra.run.smithi028.stdout:(4/13): ceph-mgr-rook-17.2.7-1045.gc6021375.el9 171 kB/s | 47 kB 00:00 2024-07-03T15:33:08.033 INFO:teuthology.orchestra.run.smithi028.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 904 kB/s | 159 kB 00:00 2024-07-03T15:33:08.058 INFO:teuthology.orchestra.run.smithi028.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 3.6 MB/s | 279 kB 00:00 2024-07-03T15:33:08.142 INFO:teuthology.orchestra.run.smithi028.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 297 kB/s | 32 kB 00:00 2024-07-03T15:33:08.167 INFO:teuthology.orchestra.run.smithi028.stdout:(8/13): python3-requests-oauthlib-1.3.0-12.el9. 337 kB/s | 54 kB 00:00 2024-07-03T15:33:08.192 INFO:teuthology.orchestra.run.smithi028.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 141 kB/s | 15 kB 00:00 2024-07-03T15:33:08.243 INFO:teuthology.orchestra.run.smithi028.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.4 MB/s | 59 kB 00:00 2024-07-03T15:33:08.268 INFO:teuthology.orchestra.run.smithi028.stdout:(11/13): python3-google-auth-2.30.0-1.el9.noarc 1.7 MB/s | 213 kB 00:00 2024-07-03T15:33:08.293 INFO:teuthology.orchestra.run.smithi028.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.8 MB/s | 90 kB 00:00 2024-07-03T15:33:08.335 INFO:teuthology.orchestra.run.smithi028.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.1 MB/s | 1.0 MB 00:00 2024-07-03T15:33:08.337 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-07-03T15:33:08.337 INFO:teuthology.orchestra.run.smithi028.stdout:Total 1.7 MB/s | 2.2 MB 00:01 2024-07-03T15:33:08.404 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-07-03T15:33:08.420 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-07-03T15:33:08.420 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-07-03T15:33:08.513 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-07-03T15:33:08.514 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-07-03T15:33:09.149 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-07-03T15:33:09.315 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-07-03T15:33:09.371 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-07-03T15:33:09.426 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-07-03T15:33:09.479 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-07-03T15:33:09.521 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-07-03T15:33:09.631 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-07-03T15:33:09.742 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-google-auth-1:2.30.0-1.el9.noarch 7/13 2024-07-03T15:33:09.803 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-07-03T15:33:10.559 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-07-03T15:33:10.616 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-07-03T15:33:10.649 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-07-03T15:33:10.695 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-07-03T15:33:10.743 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mgr-rook-2:17.2.7-1045.gc6021375.el9.noarch 13/13 2024-07-03T15:33:11.542 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1045.gc6021375.el9.noarch 13/13 2024-07-03T15:33:11.542 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1045.gc6021375.el9.noarch 1/13 2024-07-03T15:33:11.542 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-07-03T15:33:11.542 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-07-03T15:33:11.542 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-07-03T15:33:11.542 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-07-03T15:33:11.542 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-07-03T15:33:11.542 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-07-03T15:33:11.542 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-07-03T15:33:11.542 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-07-03T15:33:11.543 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-google-auth-1:2.30.0-1.el9.noarch 10/13 2024-07-03T15:33:11.543 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-07-03T15:33:11.543 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-07-03T15:33:11.863 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-07-03T15:33:11.863 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:33:11.863 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-07-03T15:33:11.864 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-rook-2:17.2.7-1045.gc6021375.el9.noarch 2024-07-03T15:33:11.864 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-07-03T15:33:11.864 INFO:teuthology.orchestra.run.smithi028.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-07-03T15:33:11.864 INFO:teuthology.orchestra.run.smithi028.stdout: python3-google-auth-1:2.30.0-1.el9.noarch 2024-07-03T15:33:11.864 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-07-03T15:33:11.864 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-07-03T15:33:11.864 INFO:teuthology.orchestra.run.smithi028.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-07-03T15:33:11.864 INFO:teuthology.orchestra.run.smithi028.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-07-03T15:33:11.864 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-07-03T15:33:11.864 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-07-03T15:33:11.864 INFO:teuthology.orchestra.run.smithi028.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-07-03T15:33:11.864 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rsa-4.9-2.el9.noarch 2024-07-03T15:33:11.865 INFO:teuthology.orchestra.run.smithi028.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-07-03T15:33:11.865 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:33:11.865 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-07-03T15:33:12.087 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install ceph-mgr-cephadm 2024-07-03T15:33:12.671 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:21 ago on Wed 03 Jul 2024 03:31:51 PM UTC. 2024-07-03T15:33:12.806 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-07-03T15:33:12.807 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-07-03T15:33:12.807 INFO:teuthology.orchestra.run.smithi028.stdout: Package Arch Version Repository Size 2024-07-03T15:33:12.808 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-07-03T15:33:12.808 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-07-03T15:33:12.808 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1045.gc6021375.el9 ceph-noarch 123 k 2024-07-03T15:33:12.808 INFO:teuthology.orchestra.run.smithi028.stdout:Installing dependencies: 2024-07-03T15:33:12.808 INFO:teuthology.orchestra.run.smithi028.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-07-03T15:33:12.808 INFO:teuthology.orchestra.run.smithi028.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-07-03T15:33:12.808 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-07-03T15:33:12.808 INFO:teuthology.orchestra.run.smithi028.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-07-03T15:33:12.808 INFO:teuthology.orchestra.run.smithi028.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-07-03T15:33:12.808 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:33:12.809 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-07-03T15:33:12.809 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-07-03T15:33:12.809 INFO:teuthology.orchestra.run.smithi028.stdout:Install 6 Packages 2024-07-03T15:33:12.809 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:33:12.809 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 7.0 M 2024-07-03T15:33:12.810 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 33 M 2024-07-03T15:33:12.810 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-07-03T15:33:13.249 INFO:teuthology.orchestra.run.smithi028.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.7 MB/s | 248 kB 00:00 2024-07-03T15:33:13.316 INFO:teuthology.orchestra.run.smithi028.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 8.0 MB/s | 548 kB 00:00 2024-07-03T15:33:13.341 INFO:teuthology.orchestra.run.smithi028.stdout:(3/6): ceph-mgr-cephadm-17.2.7-1045.gc6021375.e 531 kB/s | 123 kB 00:00 2024-07-03T15:33:13.367 INFO:teuthology.orchestra.run.smithi028.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.1 MB/s | 58 kB 00:00 2024-07-03T15:33:13.425 INFO:teuthology.orchestra.run.smithi028.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 610 kB/s | 51 kB 00:00 2024-07-03T15:33:13.501 INFO:teuthology.orchestra.run.smithi028.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-07-03T15:33:13.503 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-07-03T15:33:13.503 INFO:teuthology.orchestra.run.smithi028.stdout:Total 10 MB/s | 7.0 MB 00:00 2024-07-03T15:33:13.576 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-07-03T15:33:13.588 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-07-03T15:33:13.588 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-07-03T15:33:13.692 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-07-03T15:33:13.693 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-07-03T15:33:14.024 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-07-03T15:33:14.210 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-07-03T15:33:14.262 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-07-03T15:33:14.813 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-07-03T15:33:14.909 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-07-03T15:33:14.979 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-07-03T15:33:15.005 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1045.gc6021375.el9.noarch 6/6 2024-07-03T15:33:15.771 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1045.gc6021375.el9.noarch 6/6 2024-07-03T15:33:15.772 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1045.gc6021375.el9.noarch 1/6 2024-07-03T15:33:15.772 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-07-03T15:33:15.772 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-07-03T15:33:15.772 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-07-03T15:33:15.772 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-07-03T15:33:15.898 INFO:teuthology.orchestra.run.smithi148.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 862 kB/s | 16 MB 00:19 2024-07-03T15:33:15.899 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-07-03T15:33:15.899 INFO:teuthology.orchestra.run.smithi148.stdout:Total 1.8 MB/s | 38 MB 00:20 2024-07-03T15:33:16.037 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-07-03T15:33:16.038 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:33:16.038 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-07-03T15:33:16.038 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-cephadm-2:17.2.7-1045.gc6021375.el9.noarch 2024-07-03T15:33:16.038 INFO:teuthology.orchestra.run.smithi028.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-07-03T15:33:16.038 INFO:teuthology.orchestra.run.smithi028.stdout: python3-babel-2.9.1-2.el9.noarch 2024-07-03T15:33:16.038 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-07-03T15:33:16.038 INFO:teuthology.orchestra.run.smithi028.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-07-03T15:33:16.038 INFO:teuthology.orchestra.run.smithi028.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-07-03T15:33:16.038 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:33:16.038 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-07-03T15:33:16.254 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-07-03T15:33:16.261 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install ceph-fuse 2024-07-03T15:33:16.281 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-07-03T15:33:16.281 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-07-03T15:33:16.790 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-07-03T15:33:16.791 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-07-03T15:33:16.843 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:25 ago on Wed 03 Jul 2024 03:31:51 PM UTC. 2024-07-03T15:33:16.976 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-07-03T15:33:16.976 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-07-03T15:33:16.976 INFO:teuthology.orchestra.run.smithi028.stdout: Package Arch Version Repository Size 2024-07-03T15:33:16.976 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-07-03T15:33:16.977 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-07-03T15:33:16.977 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-fuse x86_64 2:17.2.7-1045.gc6021375.el9 ceph 813 k 2024-07-03T15:33:16.977 INFO:teuthology.orchestra.run.smithi028.stdout:Installing dependencies: 2024-07-03T15:33:16.977 INFO:teuthology.orchestra.run.smithi028.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-07-03T15:33:16.977 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:33:16.977 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-07-03T15:33:16.977 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-07-03T15:33:16.977 INFO:teuthology.orchestra.run.smithi028.stdout:Install 2 Packages 2024-07-03T15:33:16.977 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:33:16.978 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 893 k 2024-07-03T15:33:16.978 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 2.6 M 2024-07-03T15:33:16.978 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-07-03T15:33:17.193 INFO:teuthology.orchestra.run.smithi028.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 464 kB/s | 80 kB 00:00 2024-07-03T15:33:17.310 INFO:teuthology.orchestra.run.smithi028.stdout:(2/2): ceph-fuse-17.2.7-1045.gc6021375.el9.x86_ 2.7 MB/s | 813 kB 00:00 2024-07-03T15:33:17.310 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-07-03T15:33:17.311 INFO:teuthology.orchestra.run.smithi028.stdout:Total 2.6 MB/s | 893 kB 00:00 2024-07-03T15:33:17.323 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-07-03T15:33:17.347 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-07-03T15:33:17.348 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-07-03T15:33:17.430 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-07-03T15:33:17.430 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-07-03T15:33:17.572 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-07-03T15:33:17.628 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-07-03T15:33:17.649 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-07-03T15:33:17.705 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-07-03T15:33:17.760 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-07-03T15:33:17.802 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-07-03T15:33:17.829 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-fuse-2:17.2.7-1045.gc6021375.el9.x86_64 2/2 2024-07-03T15:33:17.839 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-07-03T15:33:18.015 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-07-03T15:33:18.054 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-07-03T15:33:18.139 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-07-03T15:33:18.519 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1045.gc6021375.el9.x86_64 2/2 2024-07-03T15:33:18.519 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-fuse-2:17.2.7-1045.gc6021375.el9.x86_64 1/2 2024-07-03T15:33:18.776 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-07-03T15:33:18.794 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-07-03T15:33:18.794 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:33:18.794 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-07-03T15:33:18.795 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-fuse-2:17.2.7-1045.gc6021375.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-07-03T15:33:18.795 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:33:18.795 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-07-03T15:33:18.909 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-07-03T15:33:19.094 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install librados-devel 2024-07-03T15:33:19.683 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:28 ago on Wed 03 Jul 2024 03:31:51 PM UTC. 2024-07-03T15:33:19.815 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-07-03T15:33:19.816 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-07-03T15:33:19.816 INFO:teuthology.orchestra.run.smithi028.stdout: Package Arch Version Repo Size 2024-07-03T15:33:19.816 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-07-03T15:33:19.816 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-07-03T15:33:19.816 INFO:teuthology.orchestra.run.smithi028.stdout: librados-devel x86_64 2:17.2.7-1045.gc6021375.el9 ceph 126 k 2024-07-03T15:33:19.816 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:33:19.816 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-07-03T15:33:19.816 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-07-03T15:33:19.816 INFO:teuthology.orchestra.run.smithi028.stdout:Install 1 Package 2024-07-03T15:33:19.817 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:33:19.817 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 126 k 2024-07-03T15:33:19.817 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 448 k 2024-07-03T15:33:19.817 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-07-03T15:33:20.039 INFO:teuthology.orchestra.run.smithi028.stdout:librados-devel-17.2.7-1045.gc6021375.el9.x86_64 567 kB/s | 126 kB 00:00 2024-07-03T15:33:20.040 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-07-03T15:33:20.040 INFO:teuthology.orchestra.run.smithi028.stdout:Total 564 kB/s | 126 kB 00:00 2024-07-03T15:33:20.040 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-07-03T15:33:20.046 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-07-03T15:33:20.046 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-07-03T15:33:20.111 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-07-03T15:33:20.112 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-07-03T15:33:20.150 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-07-03T15:33:20.264 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-07-03T15:33:20.372 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : librados-devel-2:17.2.7-1045.gc6021375.el9.x86_64 1/1 2024-07-03T15:33:20.815 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: librados-devel-2:17.2.7-1045.gc6021375.el9.x86_64 1/1 2024-07-03T15:33:21.077 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librados-devel-2:17.2.7-1045.gc6021375.el9.x86_64 1/1 2024-07-03T15:33:21.078 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:33:21.078 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-07-03T15:33:21.078 INFO:teuthology.orchestra.run.smithi028.stdout: librados-devel-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:33:21.078 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:33:21.078 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-07-03T15:33:21.194 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-07-03T15:33:21.229 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1045.gc6021 12/12 2024-07-03T15:33:21.316 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install libcephfs2 2024-07-03T15:33:21.905 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:30 ago on Wed 03 Jul 2024 03:31:51 PM UTC. 2024-07-03T15:33:21.993 INFO:teuthology.orchestra.run.smithi028.stdout:Package libcephfs2-2:17.2.7-1045.gc6021375.el9.x86_64 is already installed. 2024-07-03T15:33:22.038 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-07-03T15:33:22.039 INFO:teuthology.orchestra.run.smithi028.stdout:Nothing to do. 2024-07-03T15:33:22.039 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-07-03T15:33:22.102 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install libcephfs-devel 2024-07-03T15:33:22.700 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:31 ago on Wed 03 Jul 2024 03:31:51 PM UTC. 2024-07-03T15:33:22.833 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-07-03T15:33:22.833 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-07-03T15:33:22.833 INFO:teuthology.orchestra.run.smithi028.stdout: Package Arch Version Repo Size 2024-07-03T15:33:22.834 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-07-03T15:33:22.834 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-07-03T15:33:22.834 INFO:teuthology.orchestra.run.smithi028.stdout: libcephfs-devel x86_64 2:17.2.7-1045.gc6021375.el9 ceph 25 k 2024-07-03T15:33:22.834 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:33:22.834 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-07-03T15:33:22.834 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-07-03T15:33:22.834 INFO:teuthology.orchestra.run.smithi028.stdout:Install 1 Package 2024-07-03T15:33:22.834 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:33:22.834 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 25 k 2024-07-03T15:33:22.834 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 110 k 2024-07-03T15:33:22.834 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-07-03T15:33:23.084 INFO:teuthology.orchestra.run.smithi028.stdout:libcephfs-devel-17.2.7-1045.gc6021375.el9.x86_6 100 kB/s | 25 kB 00:00 2024-07-03T15:33:23.085 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-07-03T15:33:23.085 INFO:teuthology.orchestra.run.smithi028.stdout:Total 99 kB/s | 25 kB 00:00 2024-07-03T15:33:23.085 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-07-03T15:33:23.088 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-07-03T15:33:23.089 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-07-03T15:33:23.090 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-07-03T15:33:23.090 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-07-03T15:33:23.172 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-07-03T15:33:23.283 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libcephfs-devel-2:17.2.7-1045.gc6021375.el9.x86_64 1/1 2024-07-03T15:33:23.641 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1045.gc6021375.el9.x86_64 1/1 2024-07-03T15:33:23.678 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1045.gc6021 12/12 2024-07-03T15:33:23.678 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1045.gc6021 1/12 2024-07-03T15:33:23.679 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-07-03T15:33:23.679 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-07-03T15:33:23.679 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-07-03T15:33:23.679 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-07-03T15:33:23.679 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-07-03T15:33:23.679 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-07-03T15:33:23.679 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-07-03T15:33:23.679 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-07-03T15:33:23.679 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-07-03T15:33:23.680 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-07-03T15:33:23.851 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libcephfs-devel-2:17.2.7-1045.gc6021375.el9.x86_64 1/1 2024-07-03T15:33:23.851 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:33:23.851 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-07-03T15:33:23.851 INFO:teuthology.orchestra.run.smithi028.stdout: libcephfs-devel-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:33:23.851 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:33:23.852 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-07-03T15:33:24.016 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install librados2 2024-07-03T15:33:24.069 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-07-03T15:33:24.069 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:33:24.069 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-07-03T15:33:24.069 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1045.gc6021375.el9.noarch 2024-07-03T15:33:24.069 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-07-03T15:33:24.069 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-07-03T15:33:24.069 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-07-03T15:33:24.070 INFO:teuthology.orchestra.run.smithi148.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-07-03T15:33:24.070 INFO:teuthology.orchestra.run.smithi148.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-07-03T15:33:24.070 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-0.3.21-2.el9.x86_64 2024-07-03T15:33:24.070 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-07-03T15:33:24.070 INFO:teuthology.orchestra.run.smithi148.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-07-03T15:33:24.070 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-07-03T15:33:24.070 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-07-03T15:33:24.070 INFO:teuthology.orchestra.run.smithi148.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-07-03T15:33:24.070 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:33:24.070 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-07-03T15:33:24.425 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-mgr-rook 2024-07-03T15:33:24.602 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:33 ago on Wed 03 Jul 2024 03:31:51 PM UTC. 2024-07-03T15:33:24.690 INFO:teuthology.orchestra.run.smithi028.stdout:Package librados2-2:17.2.7-1045.gc6021375.el9.x86_64 is already installed. 2024-07-03T15:33:24.734 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-07-03T15:33:24.735 INFO:teuthology.orchestra.run.smithi028.stdout:Nothing to do. 2024-07-03T15:33:24.736 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-07-03T15:33:24.807 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install librbd1 2024-07-03T15:33:24.997 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:31 ago on Wed 03 Jul 2024 03:31:53 PM UTC. 2024-07-03T15:33:25.125 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-07-03T15:33:25.126 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-07-03T15:33:25.126 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-07-03T15:33:25.126 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-07-03T15:33:25.126 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-07-03T15:33:25.126 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-rook noarch 2:17.2.7-1045.gc6021375.el9 ceph-noarch 47 k 2024-07-03T15:33:25.126 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-07-03T15:33:25.126 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-07-03T15:33:25.127 INFO:teuthology.orchestra.run.smithi148.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-07-03T15:33:25.127 INFO:teuthology.orchestra.run.smithi148.stdout: python3-google-auth noarch 1:2.30.0-1.el9 epel 213 k 2024-07-03T15:33:25.127 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-07-03T15:33:25.127 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-07-03T15:33:25.127 INFO:teuthology.orchestra.run.smithi148.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-07-03T15:33:25.127 INFO:teuthology.orchestra.run.smithi148.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-07-03T15:33:25.127 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-07-03T15:33:25.127 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-07-03T15:33:25.127 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-07-03T15:33:25.127 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-07-03T15:33:25.127 INFO:teuthology.orchestra.run.smithi148.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-07-03T15:33:25.127 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:33:25.127 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-07-03T15:33:25.128 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-07-03T15:33:25.128 INFO:teuthology.orchestra.run.smithi148.stdout:Install 13 Packages 2024-07-03T15:33:25.128 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:33:25.128 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 2.2 M 2024-07-03T15:33:25.128 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 26 M 2024-07-03T15:33:25.128 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-07-03T15:33:25.378 INFO:teuthology.orchestra.run.smithi148.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 188 kB/s | 26 kB 00:00 2024-07-03T15:33:25.394 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:34 ago on Wed 03 Jul 2024 03:31:51 PM UTC. 2024-07-03T15:33:25.395 INFO:teuthology.orchestra.run.smithi148.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 125 kB/s | 19 kB 00:00 2024-07-03T15:33:25.419 INFO:teuthology.orchestra.run.smithi148.stdout:(3/13): ceph-mgr-rook-17.2.7-1045.gc6021375.el9 269 kB/s | 47 kB 00:00 2024-07-03T15:33:25.482 INFO:teuthology.orchestra.run.smithi028.stdout:Package librbd1-2:17.2.7-1045.gc6021375.el9.x86_64 is already installed. 2024-07-03T15:33:25.519 INFO:teuthology.orchestra.run.smithi148.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-07-03T15:33:25.526 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-07-03T15:33:25.528 INFO:teuthology.orchestra.run.smithi028.stdout:Nothing to do. 2024-07-03T15:33:25.528 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-07-03T15:33:25.544 INFO:teuthology.orchestra.run.smithi148.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.5 MB/s | 222 kB 00:00 2024-07-03T15:33:25.578 INFO:teuthology.orchestra.run.smithi148.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 916 kB/s | 54 kB 00:00 2024-07-03T15:33:25.596 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install python3-rados 2024-07-03T15:33:25.661 INFO:teuthology.orchestra.run.smithi148.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 276 kB/s | 32 kB 00:00 2024-07-03T15:33:25.687 INFO:teuthology.orchestra.run.smithi148.stdout:(8/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.0 MB/s | 279 kB 00:00 2024-07-03T15:33:25.712 INFO:teuthology.orchestra.run.smithi148.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 112 kB/s | 15 kB 00:00 2024-07-03T15:33:25.762 INFO:teuthology.orchestra.run.smithi148.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.2 MB/s | 59 kB 00:00 2024-07-03T15:33:25.787 INFO:teuthology.orchestra.run.smithi148.stdout:(11/13): python3-google-auth-2.30.0-1.el9.noarc 1.7 MB/s | 213 kB 00:00 2024-07-03T15:33:25.812 INFO:teuthology.orchestra.run.smithi148.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.8 MB/s | 90 kB 00:00 2024-07-03T15:33:25.854 INFO:teuthology.orchestra.run.smithi148.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.1 MB/s | 1.0 MB 00:00 2024-07-03T15:33:25.857 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-07-03T15:33:25.857 INFO:teuthology.orchestra.run.smithi148.stdout:Total 3.0 MB/s | 2.2 MB 00:00 2024-07-03T15:33:25.921 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-07-03T15:33:25.936 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-07-03T15:33:25.936 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-07-03T15:33:26.020 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-07-03T15:33:26.021 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-07-03T15:33:26.176 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:35 ago on Wed 03 Jul 2024 03:31:51 PM UTC. 2024-07-03T15:33:26.264 INFO:teuthology.orchestra.run.smithi028.stdout:Package python3-rados-2:17.2.7-1045.gc6021375.el9.x86_64 is already installed. 2024-07-03T15:33:26.308 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-07-03T15:33:26.310 INFO:teuthology.orchestra.run.smithi028.stdout:Nothing to do. 2024-07-03T15:33:26.310 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-07-03T15:33:26.375 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install python3-rgw 2024-07-03T15:33:26.526 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-07-03T15:33:26.693 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-07-03T15:33:26.755 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-07-03T15:33:26.820 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-07-03T15:33:26.857 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-07-03T15:33:26.898 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-07-03T15:33:26.958 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:35 ago on Wed 03 Jul 2024 03:31:51 PM UTC. 2024-07-03T15:33:26.995 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-07-03T15:33:27.046 INFO:teuthology.orchestra.run.smithi028.stdout:Package python3-rgw-2:17.2.7-1045.gc6021375.el9.x86_64 is already installed. 2024-07-03T15:33:27.091 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-07-03T15:33:27.092 INFO:teuthology.orchestra.run.smithi028.stdout:Nothing to do. 2024-07-03T15:33:27.092 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-07-03T15:33:27.110 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-google-auth-1:2.30.0-1.el9.noarch 7/13 2024-07-03T15:33:27.155 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install python3-cephfs 2024-07-03T15:33:27.158 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-07-03T15:33:27.739 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:36 ago on Wed 03 Jul 2024 03:31:51 PM UTC. 2024-07-03T15:33:27.826 INFO:teuthology.orchestra.run.smithi028.stdout:Package python3-cephfs-2:17.2.7-1045.gc6021375.el9.x86_64 is already installed. 2024-07-03T15:33:27.870 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-07-03T15:33:27.871 INFO:teuthology.orchestra.run.smithi028.stdout:Nothing to do. 2024-07-03T15:33:27.871 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-07-03T15:33:27.872 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-07-03T15:33:27.922 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-07-03T15:33:27.937 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install python3-rbd 2024-07-03T15:33:27.956 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-07-03T15:33:28.002 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-07-03T15:33:28.025 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-rook-2:17.2.7-1045.gc6021375.el9.noarch 13/13 2024-07-03T15:33:28.522 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:37 ago on Wed 03 Jul 2024 03:31:51 PM UTC. 2024-07-03T15:33:28.611 INFO:teuthology.orchestra.run.smithi028.stdout:Package python3-rbd-2:17.2.7-1045.gc6021375.el9.x86_64 is already installed. 2024-07-03T15:33:28.655 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-07-03T15:33:28.656 INFO:teuthology.orchestra.run.smithi028.stdout:Nothing to do. 2024-07-03T15:33:28.656 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-07-03T15:33:28.727 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install rbd-fuse 2024-07-03T15:33:28.774 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1045.gc6021375.el9.noarch 13/13 2024-07-03T15:33:28.774 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1045.gc6021375.el9.noarch 1/13 2024-07-03T15:33:28.774 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-07-03T15:33:28.774 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-07-03T15:33:28.774 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-07-03T15:33:28.774 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-07-03T15:33:28.775 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-07-03T15:33:28.775 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-07-03T15:33:28.775 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-07-03T15:33:28.775 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-07-03T15:33:28.775 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-google-auth-1:2.30.0-1.el9.noarch 10/13 2024-07-03T15:33:28.775 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-07-03T15:33:28.775 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-07-03T15:33:29.090 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-07-03T15:33:29.090 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:33:29.090 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-07-03T15:33:29.090 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-rook-2:17.2.7-1045.gc6021375.el9.noarch 2024-07-03T15:33:29.090 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-07-03T15:33:29.090 INFO:teuthology.orchestra.run.smithi148.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-07-03T15:33:29.091 INFO:teuthology.orchestra.run.smithi148.stdout: python3-google-auth-1:2.30.0-1.el9.noarch 2024-07-03T15:33:29.091 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-07-03T15:33:29.091 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-07-03T15:33:29.091 INFO:teuthology.orchestra.run.smithi148.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-07-03T15:33:29.091 INFO:teuthology.orchestra.run.smithi148.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-07-03T15:33:29.091 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-07-03T15:33:29.091 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-07-03T15:33:29.091 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-07-03T15:33:29.091 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rsa-4.9-2.el9.noarch 2024-07-03T15:33:29.091 INFO:teuthology.orchestra.run.smithi148.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-07-03T15:33:29.091 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:33:29.091 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-07-03T15:33:29.307 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:38 ago on Wed 03 Jul 2024 03:31:51 PM UTC. 2024-07-03T15:33:29.344 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-mgr-cephadm 2024-07-03T15:33:29.439 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-07-03T15:33:29.440 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-07-03T15:33:29.440 INFO:teuthology.orchestra.run.smithi028.stdout: Package Architecture Version Repository Size 2024-07-03T15:33:29.440 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-07-03T15:33:29.440 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-07-03T15:33:29.440 INFO:teuthology.orchestra.run.smithi028.stdout: rbd-fuse x86_64 2:17.2.7-1045.gc6021375.el9 ceph 82 k 2024-07-03T15:33:29.440 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:33:29.440 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-07-03T15:33:29.440 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-07-03T15:33:29.440 INFO:teuthology.orchestra.run.smithi028.stdout:Install 1 Package 2024-07-03T15:33:29.440 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:33:29.441 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 82 k 2024-07-03T15:33:29.441 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 222 k 2024-07-03T15:33:29.441 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-07-03T15:33:29.754 INFO:teuthology.orchestra.run.smithi028.stdout:rbd-fuse-17.2.7-1045.gc6021375.el9.x86_64.rpm 262 kB/s | 82 kB 00:00 2024-07-03T15:33:29.754 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-07-03T15:33:29.754 INFO:teuthology.orchestra.run.smithi028.stdout:Total 261 kB/s | 82 kB 00:00 2024-07-03T15:33:29.755 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-07-03T15:33:29.762 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-07-03T15:33:29.762 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-07-03T15:33:29.828 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-07-03T15:33:29.828 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-07-03T15:33:29.902 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:36 ago on Wed 03 Jul 2024 03:31:53 PM UTC. 2024-07-03T15:33:29.982 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-07-03T15:33:30.034 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-07-03T15:33:30.034 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-07-03T15:33:30.034 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-07-03T15:33:30.035 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-07-03T15:33:30.035 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-07-03T15:33:30.035 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1045.gc6021375.el9 ceph-noarch 123 k 2024-07-03T15:33:30.035 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-07-03T15:33:30.035 INFO:teuthology.orchestra.run.smithi148.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-07-03T15:33:30.035 INFO:teuthology.orchestra.run.smithi148.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-07-03T15:33:30.035 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-07-03T15:33:30.035 INFO:teuthology.orchestra.run.smithi148.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-07-03T15:33:30.035 INFO:teuthology.orchestra.run.smithi148.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-07-03T15:33:30.035 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:33:30.035 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-07-03T15:33:30.036 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-07-03T15:33:30.036 INFO:teuthology.orchestra.run.smithi148.stdout:Install 6 Packages 2024-07-03T15:33:30.036 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:33:30.036 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 7.0 M 2024-07-03T15:33:30.036 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 33 M 2024-07-03T15:33:30.036 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-07-03T15:33:30.094 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : rbd-fuse-2:17.2.7-1045.gc6021375.el9.x86_64 1/1 2024-07-03T15:33:30.470 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1045.gc6021375.el9.x86_64 1/1 2024-07-03T15:33:30.727 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : rbd-fuse-2:17.2.7-1045.gc6021375.el9.x86_64 1/1 2024-07-03T15:33:30.728 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:33:30.728 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-07-03T15:33:30.728 INFO:teuthology.orchestra.run.smithi028.stdout: rbd-fuse-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:33:30.728 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:33:30.728 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-07-03T15:33:30.873 INFO:teuthology.orchestra.run.smithi148.stdout:(1/6): ceph-mgr-cephadm-17.2.7-1045.gc6021375.e 466 kB/s | 123 kB 00:00 2024-07-03T15:33:30.940 INFO:teuthology.orchestra.run.smithi148.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 8.0 MB/s | 548 kB 00:00 2024-07-03T15:33:30.953 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install rbd-mirror 2024-07-03T15:33:30.965 INFO:teuthology.orchestra.run.smithi148.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.3 MB/s | 58 kB 00:00 2024-07-03T15:33:30.991 INFO:teuthology.orchestra.run.smithi148.stdout:(4/6): python3-typing-extensions-4.4.0-2.el9.no 2.0 MB/s | 51 kB 00:00 2024-07-03T15:33:31.258 INFO:teuthology.orchestra.run.smithi148.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 382 kB/s | 248 kB 00:00 2024-07-03T15:33:31.533 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:40 ago on Wed 03 Jul 2024 03:31:51 PM UTC. 2024-07-03T15:33:31.633 INFO:teuthology.orchestra.run.smithi148.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 5.8 MB/s | 6.0 MB 00:01 2024-07-03T15:33:31.637 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-07-03T15:33:31.637 INFO:teuthology.orchestra.run.smithi148.stdout:Total 4.4 MB/s | 7.0 MB 00:01 2024-07-03T15:33:31.665 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-07-03T15:33:31.666 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-07-03T15:33:31.666 INFO:teuthology.orchestra.run.smithi028.stdout: Package Arch Version Repo Size 2024-07-03T15:33:31.666 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-07-03T15:33:31.666 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-07-03T15:33:31.666 INFO:teuthology.orchestra.run.smithi028.stdout: rbd-mirror x86_64 2:17.2.7-1045.gc6021375.el9 ceph 2.9 M 2024-07-03T15:33:31.666 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:33:31.666 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-07-03T15:33:31.666 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-07-03T15:33:31.667 INFO:teuthology.orchestra.run.smithi028.stdout:Install 1 Package 2024-07-03T15:33:31.667 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:33:31.667 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 2.9 M 2024-07-03T15:33:31.667 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 12 M 2024-07-03T15:33:31.667 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-07-03T15:33:31.707 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-07-03T15:33:31.718 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-07-03T15:33:31.718 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-07-03T15:33:31.815 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-07-03T15:33:31.815 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-07-03T15:33:32.031 INFO:teuthology.orchestra.run.smithi028.stdout:rbd-mirror-17.2.7-1045.gc6021375.el9.x86_64.rpm 8.1 MB/s | 2.9 MB 00:00 2024-07-03T15:33:32.031 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-07-03T15:33:32.032 INFO:teuthology.orchestra.run.smithi028.stdout:Total 8.0 MB/s | 2.9 MB 00:00 2024-07-03T15:33:32.032 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-07-03T15:33:32.042 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-07-03T15:33:32.042 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-07-03T15:33:32.116 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-07-03T15:33:32.126 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-07-03T15:33:32.126 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-07-03T15:33:32.281 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-07-03T15:33:32.342 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-07-03T15:33:32.564 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-07-03T15:33:32.615 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : rbd-mirror-2:17.2.7-1045.gc6021375.el9.x86_64 1/1 2024-07-03T15:33:32.642 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1045.gc6021375.el9.x86_64 1/1 2024-07-03T15:33:32.642 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-03T15:33:32.642 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-07-03T15:33:32.642 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-03T15:33:32.642 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-03T15:33:32.642 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:33:32.867 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-07-03T15:33:32.953 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-07-03T15:33:33.018 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-07-03T15:33:33.047 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1045.gc6021375.el9.noarch 6/6 2024-07-03T15:33:33.627 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : rbd-mirror-2:17.2.7-1045.gc6021375.el9.x86_64 1/1 2024-07-03T15:33:33.627 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:33:33.627 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-07-03T15:33:33.627 INFO:teuthology.orchestra.run.smithi028.stdout: rbd-mirror-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:33:33.627 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:33:33.628 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-07-03T15:33:33.827 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install rbd-nbd 2024-07-03T15:33:33.841 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1045.gc6021375.el9.noarch 6/6 2024-07-03T15:33:33.842 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1045.gc6021375.el9.noarch 1/6 2024-07-03T15:33:33.842 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-07-03T15:33:33.842 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-07-03T15:33:33.842 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-07-03T15:33:33.842 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-07-03T15:33:34.113 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-07-03T15:33:34.113 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:33:34.113 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-07-03T15:33:34.113 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-cephadm-2:17.2.7-1045.gc6021375.el9.noarch 2024-07-03T15:33:34.113 INFO:teuthology.orchestra.run.smithi148.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-07-03T15:33:34.113 INFO:teuthology.orchestra.run.smithi148.stdout: python3-babel-2.9.1-2.el9.noarch 2024-07-03T15:33:34.113 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-07-03T15:33:34.113 INFO:teuthology.orchestra.run.smithi148.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-07-03T15:33:34.113 INFO:teuthology.orchestra.run.smithi148.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-07-03T15:33:34.114 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:33:34.114 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-07-03T15:33:34.346 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-fuse 2024-07-03T15:33:34.421 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:43 ago on Wed 03 Jul 2024 03:31:51 PM UTC. 2024-07-03T15:33:34.553 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-07-03T15:33:34.554 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-07-03T15:33:34.554 INFO:teuthology.orchestra.run.smithi028.stdout: Package Architecture Version Repository Size 2024-07-03T15:33:34.554 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-07-03T15:33:34.555 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-07-03T15:33:34.555 INFO:teuthology.orchestra.run.smithi028.stdout: rbd-nbd x86_64 2:17.2.7-1045.gc6021375.el9 ceph 165 k 2024-07-03T15:33:34.555 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:33:34.555 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-07-03T15:33:34.555 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-07-03T15:33:34.555 INFO:teuthology.orchestra.run.smithi028.stdout:Install 1 Package 2024-07-03T15:33:34.555 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:33:34.555 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 165 k 2024-07-03T15:33:34.555 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 477 k 2024-07-03T15:33:34.555 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-07-03T15:33:34.906 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:41 ago on Wed 03 Jul 2024 03:31:53 PM UTC. 2024-07-03T15:33:34.917 INFO:teuthology.orchestra.run.smithi028.stdout:rbd-nbd-17.2.7-1045.gc6021375.el9.x86_64.rpm 457 kB/s | 165 kB 00:00 2024-07-03T15:33:34.918 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-07-03T15:33:34.918 INFO:teuthology.orchestra.run.smithi028.stdout:Total 454 kB/s | 165 kB 00:00 2024-07-03T15:33:34.918 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-07-03T15:33:34.927 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-07-03T15:33:34.927 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-07-03T15:33:34.995 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-07-03T15:33:34.996 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-07-03T15:33:35.032 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-07-03T15:33:35.032 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-07-03T15:33:35.033 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-07-03T15:33:35.033 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-07-03T15:33:35.033 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-07-03T15:33:35.033 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-fuse x86_64 2:17.2.7-1045.gc6021375.el9 ceph 813 k 2024-07-03T15:33:35.033 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-07-03T15:33:35.033 INFO:teuthology.orchestra.run.smithi148.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-07-03T15:33:35.033 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:33:35.033 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-07-03T15:33:35.033 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-07-03T15:33:35.033 INFO:teuthology.orchestra.run.smithi148.stdout:Install 2 Packages 2024-07-03T15:33:35.033 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:33:35.034 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 893 k 2024-07-03T15:33:35.034 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 2.6 M 2024-07-03T15:33:35.034 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-07-03T15:33:35.176 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-07-03T15:33:35.266 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : rbd-nbd-2:17.2.7-1045.gc6021375.el9.x86_64 1/1 2024-07-03T15:33:35.343 INFO:teuthology.orchestra.run.smithi148.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 422 kB/s | 80 kB 00:00 2024-07-03T15:33:35.418 INFO:teuthology.orchestra.run.smithi148.stdout:(2/2): ceph-fuse-17.2.7-1045.gc6021375.el9.x86_ 3.0 MB/s | 813 kB 00:00 2024-07-03T15:33:35.419 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-07-03T15:33:35.419 INFO:teuthology.orchestra.run.smithi148.stdout:Total 2.3 MB/s | 893 kB 00:00 2024-07-03T15:33:35.431 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-07-03T15:33:35.454 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-07-03T15:33:35.454 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-07-03T15:33:35.531 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-07-03T15:33:35.532 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-07-03T15:33:35.668 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1045.gc6021375.el9.x86_64 1/1 2024-07-03T15:33:35.732 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-07-03T15:33:35.863 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-07-03T15:33:35.918 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : rbd-nbd-2:17.2.7-1045.gc6021375.el9.x86_64 1/1 2024-07-03T15:33:35.918 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:33:35.918 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-07-03T15:33:35.918 INFO:teuthology.orchestra.run.smithi028.stdout: rbd-nbd-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:33:35.919 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:33:35.919 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-07-03T15:33:35.934 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-fuse-2:17.2.7-1045.gc6021375.el9.x86_64 2/2 2024-07-03T15:33:36.163 DEBUG:teuthology.parallel:result is None 2024-07-03T15:33:36.597 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1045.gc6021375.el9.x86_64 2/2 2024-07-03T15:33:36.597 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-fuse-2:17.2.7-1045.gc6021375.el9.x86_64 1/2 2024-07-03T15:33:36.888 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-07-03T15:33:36.888 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:33:36.888 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-07-03T15:33:36.888 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-fuse-2:17.2.7-1045.gc6021375.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-07-03T15:33:36.889 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:33:36.889 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-07-03T15:33:37.199 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install librados-devel 2024-07-03T15:33:37.774 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:44 ago on Wed 03 Jul 2024 03:31:53 PM UTC. 2024-07-03T15:33:37.904 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-07-03T15:33:37.904 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-07-03T15:33:37.904 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repo Size 2024-07-03T15:33:37.904 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-07-03T15:33:37.904 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-07-03T15:33:37.904 INFO:teuthology.orchestra.run.smithi148.stdout: librados-devel x86_64 2:17.2.7-1045.gc6021375.el9 ceph 126 k 2024-07-03T15:33:37.904 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:33:37.904 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-07-03T15:33:37.905 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-07-03T15:33:37.905 INFO:teuthology.orchestra.run.smithi148.stdout:Install 1 Package 2024-07-03T15:33:37.905 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:33:37.905 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 126 k 2024-07-03T15:33:37.905 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 448 k 2024-07-03T15:33:37.905 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-07-03T15:33:38.222 INFO:teuthology.orchestra.run.smithi148.stdout:librados-devel-17.2.7-1045.gc6021375.el9.x86_64 398 kB/s | 126 kB 00:00 2024-07-03T15:33:38.223 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-07-03T15:33:38.223 INFO:teuthology.orchestra.run.smithi148.stdout:Total 396 kB/s | 126 kB 00:00 2024-07-03T15:33:38.223 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-07-03T15:33:38.228 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-07-03T15:33:38.229 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-07-03T15:33:38.291 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-07-03T15:33:38.291 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-07-03T15:33:38.442 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-07-03T15:33:38.536 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librados-devel-2:17.2.7-1045.gc6021375.el9.x86_64 1/1 2024-07-03T15:33:38.949 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librados-devel-2:17.2.7-1045.gc6021375.el9.x86_64 1/1 2024-07-03T15:33:39.195 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados-devel-2:17.2.7-1045.gc6021375.el9.x86_64 1/1 2024-07-03T15:33:39.195 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:33:39.195 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-07-03T15:33:39.196 INFO:teuthology.orchestra.run.smithi148.stdout: librados-devel-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:33:39.196 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:33:39.196 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-07-03T15:33:39.424 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install libcephfs2 2024-07-03T15:33:39.997 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:46 ago on Wed 03 Jul 2024 03:31:53 PM UTC. 2024-07-03T15:33:40.085 INFO:teuthology.orchestra.run.smithi148.stdout:Package libcephfs2-2:17.2.7-1045.gc6021375.el9.x86_64 is already installed. 2024-07-03T15:33:40.130 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-07-03T15:33:40.130 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-07-03T15:33:40.130 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-07-03T15:33:40.194 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install libcephfs-devel 2024-07-03T15:33:40.751 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:47 ago on Wed 03 Jul 2024 03:31:53 PM UTC. 2024-07-03T15:33:40.879 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-07-03T15:33:40.879 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-07-03T15:33:40.879 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repo Size 2024-07-03T15:33:40.879 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-07-03T15:33:40.879 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-07-03T15:33:40.880 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs-devel x86_64 2:17.2.7-1045.gc6021375.el9 ceph 25 k 2024-07-03T15:33:40.880 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:33:40.880 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-07-03T15:33:40.880 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-07-03T15:33:40.880 INFO:teuthology.orchestra.run.smithi148.stdout:Install 1 Package 2024-07-03T15:33:40.880 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:33:40.880 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 25 k 2024-07-03T15:33:40.880 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 110 k 2024-07-03T15:33:40.880 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-07-03T15:33:41.126 INFO:teuthology.orchestra.run.smithi148.stdout:libcephfs-devel-17.2.7-1045.gc6021375.el9.x86_6 101 kB/s | 25 kB 00:00 2024-07-03T15:33:41.127 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-07-03T15:33:41.127 INFO:teuthology.orchestra.run.smithi148.stdout:Total 101 kB/s | 25 kB 00:00 2024-07-03T15:33:41.127 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-07-03T15:33:41.130 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-07-03T15:33:41.130 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-07-03T15:33:41.132 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-07-03T15:33:41.132 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-07-03T15:33:41.206 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-07-03T15:33:41.314 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephfs-devel-2:17.2.7-1045.gc6021375.el9.x86_64 1/1 2024-07-03T15:33:41.699 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1045.gc6021375.el9.x86_64 1/1 2024-07-03T15:33:41.921 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephfs-devel-2:17.2.7-1045.gc6021375.el9.x86_64 1/1 2024-07-03T15:33:41.921 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:33:41.921 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-07-03T15:33:41.922 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs-devel-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:33:41.922 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:33:41.922 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-07-03T15:33:42.101 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install librados2 2024-07-03T15:33:42.662 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:49 ago on Wed 03 Jul 2024 03:31:53 PM UTC. 2024-07-03T15:33:42.746 INFO:teuthology.orchestra.run.smithi148.stdout:Package librados2-2:17.2.7-1045.gc6021375.el9.x86_64 is already installed. 2024-07-03T15:33:42.789 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-07-03T15:33:42.790 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-07-03T15:33:42.790 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-07-03T15:33:42.862 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install librbd1 2024-07-03T15:33:43.423 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:50 ago on Wed 03 Jul 2024 03:31:53 PM UTC. 2024-07-03T15:33:43.509 INFO:teuthology.orchestra.run.smithi148.stdout:Package librbd1-2:17.2.7-1045.gc6021375.el9.x86_64 is already installed. 2024-07-03T15:33:43.550 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-07-03T15:33:43.551 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-07-03T15:33:43.551 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-07-03T15:33:43.619 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install python3-rados 2024-07-03T15:33:44.180 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:51 ago on Wed 03 Jul 2024 03:31:53 PM UTC. 2024-07-03T15:33:44.266 INFO:teuthology.orchestra.run.smithi148.stdout:Package python3-rados-2:17.2.7-1045.gc6021375.el9.x86_64 is already installed. 2024-07-03T15:33:44.308 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-07-03T15:33:44.308 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-07-03T15:33:44.309 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-07-03T15:33:44.369 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install python3-rgw 2024-07-03T15:33:44.929 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:51 ago on Wed 03 Jul 2024 03:31:53 PM UTC. 2024-07-03T15:33:45.015 INFO:teuthology.orchestra.run.smithi148.stdout:Package python3-rgw-2:17.2.7-1045.gc6021375.el9.x86_64 is already installed. 2024-07-03T15:33:45.057 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-07-03T15:33:45.058 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-07-03T15:33:45.058 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-07-03T15:33:45.119 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install python3-cephfs 2024-07-03T15:33:45.671 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:52 ago on Wed 03 Jul 2024 03:31:53 PM UTC. 2024-07-03T15:33:45.755 INFO:teuthology.orchestra.run.smithi148.stdout:Package python3-cephfs-2:17.2.7-1045.gc6021375.el9.x86_64 is already installed. 2024-07-03T15:33:45.797 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-07-03T15:33:45.798 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-07-03T15:33:45.798 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-07-03T15:33:45.861 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install python3-rbd 2024-07-03T15:33:46.423 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:53 ago on Wed 03 Jul 2024 03:31:53 PM UTC. 2024-07-03T15:33:46.508 INFO:teuthology.orchestra.run.smithi148.stdout:Package python3-rbd-2:17.2.7-1045.gc6021375.el9.x86_64 is already installed. 2024-07-03T15:33:46.549 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-07-03T15:33:46.551 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-07-03T15:33:46.551 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-07-03T15:33:46.617 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install rbd-fuse 2024-07-03T15:33:47.176 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:54 ago on Wed 03 Jul 2024 03:31:53 PM UTC. 2024-07-03T15:33:47.307 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-07-03T15:33:47.307 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-07-03T15:33:47.307 INFO:teuthology.orchestra.run.smithi148.stdout: Package Architecture Version Repository Size 2024-07-03T15:33:47.308 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-07-03T15:33:47.308 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-07-03T15:33:47.308 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-fuse x86_64 2:17.2.7-1045.gc6021375.el9 ceph 82 k 2024-07-03T15:33:47.308 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:33:47.308 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-07-03T15:33:47.308 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-07-03T15:33:47.308 INFO:teuthology.orchestra.run.smithi148.stdout:Install 1 Package 2024-07-03T15:33:47.308 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:33:47.308 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 82 k 2024-07-03T15:33:47.308 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 222 k 2024-07-03T15:33:47.308 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-07-03T15:33:47.520 INFO:teuthology.orchestra.run.smithi148.stdout:rbd-fuse-17.2.7-1045.gc6021375.el9.x86_64.rpm 386 kB/s | 82 kB 00:00 2024-07-03T15:33:47.521 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-07-03T15:33:47.521 INFO:teuthology.orchestra.run.smithi148.stdout:Total 384 kB/s | 82 kB 00:00 2024-07-03T15:33:47.521 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-07-03T15:33:47.528 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-07-03T15:33:47.528 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-07-03T15:33:47.590 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-07-03T15:33:47.591 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-07-03T15:33:47.735 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-07-03T15:33:47.834 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-fuse-2:17.2.7-1045.gc6021375.el9.x86_64 1/1 2024-07-03T15:33:48.184 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1045.gc6021375.el9.x86_64 1/1 2024-07-03T15:33:48.428 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-fuse-2:17.2.7-1045.gc6021375.el9.x86_64 1/1 2024-07-03T15:33:48.428 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:33:48.428 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-07-03T15:33:48.428 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-fuse-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:33:48.428 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:33:48.429 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-07-03T15:33:48.649 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install rbd-mirror 2024-07-03T15:33:49.222 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:56 ago on Wed 03 Jul 2024 03:31:53 PM UTC. 2024-07-03T15:33:49.350 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-07-03T15:33:49.351 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-07-03T15:33:49.351 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repo Size 2024-07-03T15:33:49.351 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-07-03T15:33:49.351 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-07-03T15:33:49.352 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-mirror x86_64 2:17.2.7-1045.gc6021375.el9 ceph 2.9 M 2024-07-03T15:33:49.352 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:33:49.352 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-07-03T15:33:49.352 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-07-03T15:33:49.352 INFO:teuthology.orchestra.run.smithi148.stdout:Install 1 Package 2024-07-03T15:33:49.352 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:33:49.352 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 2.9 M 2024-07-03T15:33:49.352 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 12 M 2024-07-03T15:33:49.352 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-07-03T15:33:49.732 INFO:teuthology.orchestra.run.smithi148.stdout:rbd-mirror-17.2.7-1045.gc6021375.el9.x86_64.rpm 7.7 MB/s | 2.9 MB 00:00 2024-07-03T15:33:49.733 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-07-03T15:33:49.733 INFO:teuthology.orchestra.run.smithi148.stdout:Total 7.7 MB/s | 2.9 MB 00:00 2024-07-03T15:33:49.733 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-07-03T15:33:49.743 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-07-03T15:33:49.743 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-07-03T15:33:49.823 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-07-03T15:33:49.823 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-07-03T15:33:50.242 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-07-03T15:33:50.300 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-mirror-2:17.2.7-1045.gc6021375.el9.x86_64 1/1 2024-07-03T15:33:50.325 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1045.gc6021375.el9.x86_64 1/1 2024-07-03T15:33:50.325 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-03T15:33:50.325 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-07-03T15:33:50.325 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-03T15:33:50.326 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-03T15:33:50.326 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:33:51.411 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-mirror-2:17.2.7-1045.gc6021375.el9.x86_64 1/1 2024-07-03T15:33:51.412 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:33:51.412 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-07-03T15:33:51.412 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-mirror-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:33:51.412 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:33:51.412 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-07-03T15:33:51.640 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install rbd-nbd 2024-07-03T15:33:52.217 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:59 ago on Wed 03 Jul 2024 03:31:53 PM UTC. 2024-07-03T15:33:52.352 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-07-03T15:33:52.353 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-07-03T15:33:52.353 INFO:teuthology.orchestra.run.smithi148.stdout: Package Architecture Version Repository Size 2024-07-03T15:33:52.353 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-07-03T15:33:52.353 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-07-03T15:33:52.353 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-nbd x86_64 2:17.2.7-1045.gc6021375.el9 ceph 165 k 2024-07-03T15:33:52.353 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:33:52.353 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-07-03T15:33:52.353 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-07-03T15:33:52.353 INFO:teuthology.orchestra.run.smithi148.stdout:Install 1 Package 2024-07-03T15:33:52.353 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:33:52.353 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 165 k 2024-07-03T15:33:52.353 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 477 k 2024-07-03T15:33:52.354 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-07-03T15:33:52.586 INFO:teuthology.orchestra.run.smithi148.stdout:rbd-nbd-17.2.7-1045.gc6021375.el9.x86_64.rpm 711 kB/s | 165 kB 00:00 2024-07-03T15:33:52.587 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-07-03T15:33:52.587 INFO:teuthology.orchestra.run.smithi148.stdout:Total 705 kB/s | 165 kB 00:00 2024-07-03T15:33:52.587 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-07-03T15:33:52.595 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-07-03T15:33:52.595 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-07-03T15:33:52.663 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-07-03T15:33:52.663 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-07-03T15:33:52.816 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-07-03T15:33:52.901 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-nbd-2:17.2.7-1045.gc6021375.el9.x86_64 1/1 2024-07-03T15:33:53.361 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1045.gc6021375.el9.x86_64 1/1 2024-07-03T15:33:53.603 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-nbd-2:17.2.7-1045.gc6021375.el9.x86_64 1/1 2024-07-03T15:33:53.603 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:33:53.603 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-07-03T15:33:53.603 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-nbd-2:17.2.7-1045.gc6021375.el9.x86_64 2024-07-03T15:33:53.603 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:33:53.604 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-07-03T15:33:53.911 DEBUG:teuthology.parallel:result is None 2024-07-03T15:33:53.911 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-07-03T15:33:53.911 INFO:teuthology.packaging:ref: None 2024-07-03T15:33:53.911 INFO:teuthology.packaging:tag: None 2024-07-03T15:33:53.911 INFO:teuthology.packaging:branch: quincy 2024-07-03T15:33:53.911 INFO:teuthology.packaging:sha1: c00506dc10003928f8cb40769168571f611cbd67 2024-07-03T15:33:53.912 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-07-03T15:33:54.134 DEBUG:teuthology.orchestra.run.smithi028:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-07-03T15:33:54.160 INFO:teuthology.orchestra.run.smithi028.stdout:17.2.7-1045.gc6021375.el9 2024-07-03T15:33:54.161 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1045.gc6021375.el9 2024-07-03T15:33:54.161 INFO:teuthology.task.install:The correct ceph version 17.2.7-1045.gc6021375 is installed. 2024-07-03T15:33:54.163 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-07-03T15:33:54.163 INFO:teuthology.packaging:ref: None 2024-07-03T15:33:54.163 INFO:teuthology.packaging:tag: None 2024-07-03T15:33:54.163 INFO:teuthology.packaging:branch: quincy 2024-07-03T15:33:54.164 INFO:teuthology.packaging:sha1: c00506dc10003928f8cb40769168571f611cbd67 2024-07-03T15:33:54.164 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-07-03T15:33:54.384 DEBUG:teuthology.orchestra.run.smithi148:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-07-03T15:33:54.407 INFO:teuthology.orchestra.run.smithi148.stdout:17.2.7-1045.gc6021375.el9 2024-07-03T15:33:54.407 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1045.gc6021375.el9 2024-07-03T15:33:54.408 INFO:teuthology.task.install:The correct ceph version 17.2.7-1045.gc6021375 is installed. 2024-07-03T15:33:54.409 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-07-03T15:33:54.410 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-07-03T15:33:54.410 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-07-03T15:33:54.442 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-07-03T15:33:54.442 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-07-03T15:33:54.475 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-07-03T15:33:54.475 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-07-03T15:33:54.475 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/usr/bin/daemon-helper 2024-07-03T15:33:54.511 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-07-03T15:33:54.580 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-07-03T15:33:54.580 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/daemon-helper 2024-07-03T15:33:54.617 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-07-03T15:33:54.682 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-07-03T15:33:54.682 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-07-03T15:33:54.682 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/usr/bin/adjust-ulimits 2024-07-03T15:33:54.711 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-07-03T15:33:54.779 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-07-03T15:33:54.779 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/adjust-ulimits 2024-07-03T15:33:54.804 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-07-03T15:33:54.868 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-07-03T15:33:54.868 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-07-03T15:33:54.868 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/usr/bin/stdin-killer 2024-07-03T15:33:54.895 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-07-03T15:33:54.961 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-07-03T15:33:54.961 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/stdin-killer 2024-07-03T15:33:54.985 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-07-03T15:33:55.050 INFO:teuthology.run_tasks:Running task print... 2024-07-03T15:33:55.059 INFO:teuthology.task.print:**** done install task... 2024-07-03T15:33:55.059 INFO:teuthology.run_tasks:Running task cephadm... 2024-07-03T15:33:55.165 INFO:tasks.cephadm:Config: {'cephadm_branch': 'quincy', 'cephadm_git_url': 'https://github.com/ceph/ceph', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}}, 'image': 'quay.ceph.io/ceph-ci/ceph:quincy', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'sha1': 'c00506dc10003928f8cb40769168571f611cbd67'} 2024-07-03T15:33:55.165 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:quincy 2024-07-03T15:33:55.166 INFO:tasks.cephadm:Cluster fsid is a85e4be6-3951-11ef-bcab-c7b262605968 2024-07-03T15:33:55.166 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-07-03T15:33:55.166 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-07-03T15:33:55.166 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi028': '172.21.15.28', 'mon.smithi148': '172.21.15.148'} 2024-07-03T15:33:55.166 INFO:tasks.cephadm:Normalizing hostnames... 2024-07-03T15:33:55.166 DEBUG:teuthology.orchestra.run.smithi028:> sudo hostname $(hostname -s) 2024-07-03T15:33:55.193 DEBUG:teuthology.orchestra.run.smithi148:> sudo hostname $(hostname -s) 2024-07-03T15:33:55.217 INFO:tasks.cephadm:Downloading cephadm (repo https://github.com/ceph/ceph ref quincy)... 2024-07-03T15:33:55.217 DEBUG:teuthology.orchestra.run.smithi028:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-07-03T15:33:55.495 INFO:teuthology.orchestra.run.smithi028.stdout:-rw-r--r--. 1 ubuntu ubuntu 375153 Jul 3 15:33 /home/ubuntu/cephtest/cephadm 2024-07-03T15:33:55.496 DEBUG:teuthology.orchestra.run.smithi148:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-07-03T15:33:55.688 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r--. 1 ubuntu ubuntu 375153 Jul 3 15:33 /home/ubuntu/cephtest/cephadm 2024-07-03T15:33:55.688 DEBUG:teuthology.orchestra.run.smithi028:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-07-03T15:33:55.705 DEBUG:teuthology.orchestra.run.smithi148:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-07-03T15:33:55.729 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:quincy on all hosts... 2024-07-03T15:33:55.729 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-07-03T15:33:55.747 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-07-03T15:33:56.604 INFO:teuthology.orchestra.run.smithi028.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-07-03T15:33:56.686 INFO:teuthology.orchestra.run.smithi148.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-07-03T15:34:44.447 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-07-03T15:34:44.448 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_version": "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)", 2024-07-03T15:34:44.448 INFO:teuthology.orchestra.run.smithi148.stdout: "image_id": "4ca399d9e65db9fa94fcda22a8b06b0fd12b8ab90081173322ce426b15957323", 2024-07-03T15:34:44.448 INFO:teuthology.orchestra.run.smithi148.stdout: "repo_digests": [ 2024-07-03T15:34:44.448 INFO:teuthology.orchestra.run.smithi148.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:0b7651540cdb24c1b97466c79c298d54c7794c5f53d5d6b6c5d76a2d3b130d85" 2024-07-03T15:34:44.448 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-07-03T15:34:44.448 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-07-03T15:34:47.289 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-07-03T15:34:47.289 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph_version": "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)", 2024-07-03T15:34:47.289 INFO:teuthology.orchestra.run.smithi028.stdout: "image_id": "4ca399d9e65db9fa94fcda22a8b06b0fd12b8ab90081173322ce426b15957323", 2024-07-03T15:34:47.290 INFO:teuthology.orchestra.run.smithi028.stdout: "repo_digests": [ 2024-07-03T15:34:47.290 INFO:teuthology.orchestra.run.smithi028.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:0b7651540cdb24c1b97466c79c298d54c7794c5f53d5d6b6c5d76a2d3b130d85" 2024-07-03T15:34:47.290 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-07-03T15:34:47.290 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-07-03T15:34:47.312 DEBUG:teuthology.orchestra.run.smithi028:> sudo mkdir -p /etc/ceph 2024-07-03T15:34:47.352 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /etc/ceph 2024-07-03T15:34:47.384 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 777 /etc/ceph 2024-07-03T15:34:47.419 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 777 /etc/ceph 2024-07-03T15:34:47.452 INFO:tasks.cephadm:Writing seed config... 2024-07-03T15:34:47.453 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-07-03T15:34:47.454 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-07-03T15:34:47.454 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-07-03T15:34:47.454 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-07-03T15:34:47.454 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-07-03T15:34:47.454 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-07-03T15:34:47.454 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-07-03T15:34:47.454 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-07-03T15:34:47.454 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-07-03T15:34:47.454 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-07-03T15:34:47.454 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-07-03T15:34:47.454 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-07-03T15:34:47.455 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-07-03T15:34:47.455 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-07-03T15:34:47.455 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-07-03T15:34:47.455 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-07-03T15:34:47.455 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-07-03T15:34:47.455 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-07-03T15:34:47.455 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-07-03T15:34:47.455 INFO:tasks.cephadm: override: [client] debug client = 20 2024-07-03T15:34:47.455 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-07-03T15:34:47.455 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-07-03T15:34:47.455 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-07-03T15:34:47.455 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-07-03T15:34:47.455 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-07-03T15:34:47.455 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-07-03T15:34:47.456 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-07-03T15:34:47.456 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-07-03T15:34:47.456 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-07-03T15:34:47.456 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-07-03T15:34:47.456 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-07-03T15:34:47.456 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-07-03T15:34:47.456 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-07-03T15:34:47.456 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-07-03T15:34:47.456 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-07-03T15:34:47.456 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-07-03T15:34:47.456 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-07-03T15:34:47.456 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-07-03T15:34:47.456 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-07-03T15:34:47.456 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-07-03T15:34:47.457 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-07-03T15:34:47.457 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-07-03T15:34:47.457 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-07-03T15:34:47.457 DEBUG:teuthology.orchestra.run.smithi028:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-07-03T15:34:47.477 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = a85e4be6-3951-11ef-bcab-c7b262605968 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-07-03T15:34:47.477 DEBUG:teuthology.orchestra.run.smithi028:mon.smithi028> sudo journalctl -f -n 0 -u ceph-a85e4be6-3951-11ef-bcab-c7b262605968@mon.smithi028.service 2024-07-03T15:34:47.519 INFO:tasks.cephadm:Bootstrapping... 2024-07-03T15:34:47.520 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy -v bootstrap --fsid a85e4be6-3951-11ef-bcab-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.28 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-07-03T15:34:47.733 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-07-03T15:34:47.734 INFO:teuthology.orchestra.run.smithi028.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:quincy', '-v', 'bootstrap', '--fsid', 'a85e4be6-3951-11ef-bcab-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.28', '--skip-admin-label'] 2024-07-03T15:34:47.759 INFO:teuthology.orchestra.run.smithi028.stdout:/bin/podman: stdout 5.1.0 2024-07-03T15:34:47.759 INFO:teuthology.orchestra.run.smithi028.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-07-03T15:34:47.760 INFO:teuthology.orchestra.run.smithi028.stdout:Verifying podman|docker is present... 2024-07-03T15:34:47.785 INFO:teuthology.orchestra.run.smithi028.stdout:/bin/podman: stdout 5.1.0 2024-07-03T15:34:47.785 INFO:teuthology.orchestra.run.smithi028.stdout:Verifying lvm2 is present... 2024-07-03T15:34:47.785 INFO:teuthology.orchestra.run.smithi028.stdout:Verifying time synchronization is in place... 2024-07-03T15:34:47.793 INFO:teuthology.orchestra.run.smithi028.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-07-03T15:34:47.793 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-07-03T15:34:47.800 INFO:teuthology.orchestra.run.smithi028.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-07-03T15:34:47.800 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stdout inactive 2024-07-03T15:34:47.807 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stdout enabled 2024-07-03T15:34:47.814 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stdout active 2024-07-03T15:34:47.814 INFO:teuthology.orchestra.run.smithi028.stdout:Unit chronyd.service is enabled and running 2024-07-03T15:34:47.814 INFO:teuthology.orchestra.run.smithi028.stdout:Repeating the final host check... 2024-07-03T15:34:47.838 INFO:teuthology.orchestra.run.smithi028.stdout:/bin/podman: stdout 5.1.0 2024-07-03T15:34:47.838 INFO:teuthology.orchestra.run.smithi028.stdout:podman (/bin/podman) version 5.1.0 is present 2024-07-03T15:34:47.838 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl is present 2024-07-03T15:34:47.838 INFO:teuthology.orchestra.run.smithi028.stdout:lvcreate is present 2024-07-03T15:34:47.845 INFO:teuthology.orchestra.run.smithi028.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-07-03T15:34:47.845 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-07-03T15:34:47.852 INFO:teuthology.orchestra.run.smithi028.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-07-03T15:34:47.852 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stdout inactive 2024-07-03T15:34:47.859 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stdout enabled 2024-07-03T15:34:47.866 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stdout active 2024-07-03T15:34:47.866 INFO:teuthology.orchestra.run.smithi028.stdout:Unit chronyd.service is enabled and running 2024-07-03T15:34:47.867 INFO:teuthology.orchestra.run.smithi028.stdout:Host looks OK 2024-07-03T15:34:47.867 INFO:teuthology.orchestra.run.smithi028.stdout:Cluster fsid: a85e4be6-3951-11ef-bcab-c7b262605968 2024-07-03T15:34:47.867 INFO:teuthology.orchestra.run.smithi028.stdout:Acquiring lock 139766625989056 on /run/cephadm/a85e4be6-3951-11ef-bcab-c7b262605968.lock 2024-07-03T15:34:47.867 INFO:teuthology.orchestra.run.smithi028.stdout:Lock 139766625989056 acquired on /run/cephadm/a85e4be6-3951-11ef-bcab-c7b262605968.lock 2024-07-03T15:34:47.867 INFO:teuthology.orchestra.run.smithi028.stdout:Verifying IP 172.21.15.28 port 3300 ... 2024-07-03T15:34:47.867 INFO:teuthology.orchestra.run.smithi028.stdout:Verifying IP 172.21.15.28 port 6789 ... 2024-07-03T15:34:47.867 INFO:teuthology.orchestra.run.smithi028.stdout:Base mon IP(s) is [172.21.15.28:3300, 172.21.15.28:6789], mon addrv is [v2:172.21.15.28:3300,v1:172.21.15.28:6789] 2024-07-03T15:34:47.871 INFO:teuthology.orchestra.run.smithi028.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.28 metric 100 2024-07-03T15:34:47.871 INFO:teuthology.orchestra.run.smithi028.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.28 metric 100 2024-07-03T15:34:47.874 INFO:teuthology.orchestra.run.smithi028.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-07-03T15:34:47.874 INFO:teuthology.orchestra.run.smithi028.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-07-03T15:34:47.874 INFO:teuthology.orchestra.run.smithi028.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-07-03T15:34:47.876 INFO:teuthology.orchestra.run.smithi028.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-07-03T15:34:47.877 INFO:teuthology.orchestra.run.smithi028.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-07-03T15:34:47.877 INFO:teuthology.orchestra.run.smithi028.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-07-03T15:34:47.877 INFO:teuthology.orchestra.run.smithi028.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-07-03T15:34:47.877 INFO:teuthology.orchestra.run.smithi028.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fea5:ca3c/64 scope link noprefixroute 2024-07-03T15:34:47.877 INFO:teuthology.orchestra.run.smithi028.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-07-03T15:34:47.878 INFO:teuthology.orchestra.run.smithi028.stdout:Mon IP `172.21.15.28` is in CIDR network `172.21.0.0/20` 2024-07-03T15:34:47.878 INFO:teuthology.orchestra.run.smithi028.stdout:Mon IP `172.21.15.28` is in CIDR network `172.21.0.0/20` 2024-07-03T15:34:47.878 INFO:teuthology.orchestra.run.smithi028.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-07-03T15:34:47.878 INFO:teuthology.orchestra.run.smithi028.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-07-03T15:34:47.879 INFO:teuthology.orchestra.run.smithi028.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-07-03T15:34:48.347 INFO:teuthology.orchestra.run.smithi028.stdout:/bin/podman: stdout 4ca399d9e65db9fa94fcda22a8b06b0fd12b8ab90081173322ce426b15957323 2024-07-03T15:34:48.347 INFO:teuthology.orchestra.run.smithi028.stdout:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:quincy... 2024-07-03T15:34:48.347 INFO:teuthology.orchestra.run.smithi028.stdout:/bin/podman: stderr Getting image source signatures 2024-07-03T15:34:48.347 INFO:teuthology.orchestra.run.smithi028.stdout:/bin/podman: stderr Copying blob sha256:104d9b23aa7d21ba0992b42c04384da5add35544b7e39bfaf1a84e24163a6410 2024-07-03T15:34:48.347 INFO:teuthology.orchestra.run.smithi028.stdout:/bin/podman: stderr Copying blob sha256:0d9665cbb1e45ff458d07612450003bc3c523a9d941d88d73ef519caf59c67ab 2024-07-03T15:34:48.347 INFO:teuthology.orchestra.run.smithi028.stdout:/bin/podman: stderr Copying config sha256:4ca399d9e65db9fa94fcda22a8b06b0fd12b8ab90081173322ce426b15957323 2024-07-03T15:34:48.347 INFO:teuthology.orchestra.run.smithi028.stdout:/bin/podman: stderr Writing manifest to image destination 2024-07-03T15:34:49.218 INFO:teuthology.orchestra.run.smithi028.stdout:ceph: stdout ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable) 2024-07-03T15:34:49.219 INFO:teuthology.orchestra.run.smithi028.stdout:Ceph version: ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable) 2024-07-03T15:34:49.219 INFO:teuthology.orchestra.run.smithi028.stdout:Extracting ceph user uid/gid from container image... 2024-07-03T15:34:50.023 INFO:teuthology.orchestra.run.smithi028.stdout:stat: stdout 167 167 2024-07-03T15:34:50.023 INFO:teuthology.orchestra.run.smithi028.stdout:Creating initial keys... 2024-07-03T15:34:50.953 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph-authtool: stdout AQCab4VmD8yaFBAANDvh2ZhA3HxviaxaF+JR9g== 2024-07-03T15:34:51.891 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph-authtool: stdout AQCbb4VmXxzrEBAAJzPaL30F/lZCtMzEFxXKlw== 2024-07-03T15:34:52.821 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph-authtool: stdout AQCcb4VmXENJDBAAWfhQexx8xUJteJCJNL7mAA== 2024-07-03T15:34:52.822 INFO:teuthology.orchestra.run.smithi028.stdout:Creating initial monmap... 2024-07-03T15:34:53.752 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-07-03T15:34:53.753 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-07-03T15:34:53.753 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to a85e4be6-3951-11ef-bcab-c7b262605968 2024-07-03T15:34:53.753 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-07-03T15:34:53.753 INFO:teuthology.orchestra.run.smithi028.stdout:monmaptool for smithi028 [v2:172.21.15.28:3300,v1:172.21.15.28:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-07-03T15:34:53.753 INFO:teuthology.orchestra.run.smithi028.stdout:setting min_mon_release = octopus 2024-07-03T15:34:53.753 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/monmaptool: set fsid to a85e4be6-3951-11ef-bcab-c7b262605968 2024-07-03T15:34:53.753 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-07-03T15:34:53.753 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:34:53.753 INFO:teuthology.orchestra.run.smithi028.stdout:Creating mon... 2024-07-03T15:34:54.675 INFO:teuthology.orchestra.run.smithi028.stdout:create mon.smithi028 on 2024-07-03T15:34:54.882 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-07-03T15:34:55.037 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-07-03T15:34:55.194 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-a85e4be6-3951-11ef-bcab-c7b262605968.target → /etc/systemd/system/ceph-a85e4be6-3951-11ef-bcab-c7b262605968.target. 2024-07-03T15:34:55.194 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-a85e4be6-3951-11ef-bcab-c7b262605968.target → /etc/systemd/system/ceph-a85e4be6-3951-11ef-bcab-c7b262605968.target. 2024-07-03T15:34:55.365 INFO:teuthology.orchestra.run.smithi028.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a85e4be6-3951-11ef-bcab-c7b262605968@mon.smithi028 2024-07-03T15:34:55.365 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a85e4be6-3951-11ef-bcab-c7b262605968@mon.smithi028.service: Unit ceph-a85e4be6-3951-11ef-bcab-c7b262605968@mon.smithi028.service not loaded. 2024-07-03T15:34:55.533 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a85e4be6-3951-11ef-bcab-c7b262605968.target.wants/ceph-a85e4be6-3951-11ef-bcab-c7b262605968@mon.smithi028.service → /etc/systemd/system/ceph-a85e4be6-3951-11ef-bcab-c7b262605968@.service. 2024-07-03T15:34:56.030 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:34:55 smithi028 podman[28763]: 2024-07-03 15:34:55.9935714 +0000 UTC m=+0.333351532 container start 05faf912bff31f651a14b68ee54d2d802d93a7d733d94a488e35ef25815cd105 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-mon-smithi028, RELEASE=quincy-d4b063a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240625, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-03T15:34:56.113 INFO:teuthology.orchestra.run.smithi028.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-07-03T15:34:56.113 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stdout disabled 2024-07-03T15:34:56.121 INFO:teuthology.orchestra.run.smithi028.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-07-03T15:34:56.121 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stdout inactive 2024-07-03T15:34:56.121 INFO:teuthology.orchestra.run.smithi028.stdout:firewalld.service is not enabled 2024-07-03T15:34:56.121 INFO:teuthology.orchestra.run.smithi028.stdout:Not possible to enable service . firewalld.service is not available 2024-07-03T15:34:56.121 INFO:teuthology.orchestra.run.smithi028.stdout:Waiting for mon to start... 2024-07-03T15:34:56.122 INFO:teuthology.orchestra.run.smithi028.stdout:Waiting for mon... 2024-07-03T15:34:56.339 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:34:56 smithi028 ceph-mon[28782]: mkfs a85e4be6-3951-11ef-bcab-c7b262605968 2024-07-03T15:34:56.340 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:34:56 smithi028 ceph-mon[28782]: mon.smithi028 is new leader, mons smithi028 in quorum (ranks 0) 2024-07-03T15:34:56.340 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:34:56 smithi028 bash[28763]: 05faf912bff31f651a14b68ee54d2d802d93a7d733d94a488e35ef25815cd105 2024-07-03T15:34:56.340 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:34:56 smithi028 systemd[1]: Started Ceph mon.smithi028 for a85e4be6-3951-11ef-bcab-c7b262605968. 2024-07-03T15:34:57.139 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:34:57 smithi028 ceph-mon[28782]: mon.smithi028 is new leader, mons smithi028 in quorum (ranks 0) 2024-07-03T15:34:57.139 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:34:57 smithi028 ceph-mon[28782]: monmap e1: 1 mons at {smithi028=[v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-07-03T15:34:57.139 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:34:57 smithi028 ceph-mon[28782]: fsmap 2024-07-03T15:34:57.139 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:34:57 smithi028 ceph-mon[28782]: osdmap e1: 0 total, 0 up, 0 in 2024-07-03T15:34:57.139 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:34:57 smithi028 ceph-mon[28782]: mgrmap e1: no daemons active 2024-07-03T15:34:57.139 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:34:57 smithi028 ceph-mon[28782]: from='client.? 172.21.15.28:0/70251860' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-07-03T15:34:57.141 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout cluster: 2024-07-03T15:34:57.147 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout id: a85e4be6-3951-11ef-bcab-c7b262605968 2024-07-03T15:34:57.147 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-07-03T15:34:57.147 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout 2024-07-03T15:34:57.147 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout services: 2024-07-03T15:34:57.147 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi028 (age 0.566171s) 2024-07-03T15:34:57.147 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-07-03T15:34:57.147 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-07-03T15:34:57.148 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout 2024-07-03T15:34:57.148 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout data: 2024-07-03T15:34:57.148 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-07-03T15:34:57.148 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-07-03T15:34:57.148 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-07-03T15:34:57.148 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout pgs: 2024-07-03T15:34:57.148 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout 2024-07-03T15:34:57.148 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:56.519+0000 7fd850933640 1 Processor -- start 2024-07-03T15:34:57.148 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:56.519+0000 7fd850933640 1 -- start start 2024-07-03T15:34:57.148 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:56.520+0000 7fd850933640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd84c075b80 0x7fd84c104a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:34:57.148 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:56.520+0000 7fd850933640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84c075f90 con 0x7fd84c075b80 2024-07-03T15:34:57.148 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:56.520+0000 7fd84a59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd84c075b80 0x7fd84c104a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:34:57.149 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:56.520+0000 7fd84a59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd84c075b80 0x7fd84c104a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36092/0 (socket says 172.21.15.28:36092) 2024-07-03T15:34:57.149 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:56.520+0000 7fd84a59c640 1 -- 172.21.15.28:0/3961839855 learned_addr learned my addr 172.21.15.28:0/3961839855 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:34:57.149 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:56.520+0000 7fd84a59c640 1 -- 172.21.15.28:0/3961839855 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd84c104f90 con 0x7fd84c075b80 2024-07-03T15:34:57.149 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:56.522+0000 7fd84a59c640 1 --2- 172.21.15.28:0/3961839855 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd84c075b80 0x7fd84c104a50 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd83c009840 tx=0x7fd83c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fda2404f57eb36c server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:34:57.149 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:56.522+0000 7fd84959a640 1 -- 172.21.15.28:0/3961839855 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd83c032440 con 0x7fd84c075b80 2024-07-03T15:34:57.149 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:56.522+0000 7fd84959a640 1 -- 172.21.15.28:0/3961839855 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd83c0325a0 con 0x7fd84c075b80 2024-07-03T15:34:57.149 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:56.522+0000 7fd84959a640 1 -- 172.21.15.28:0/3961839855 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd83c0328d0 con 0x7fd84c075b80 2024-07-03T15:34:57.149 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:56.523+0000 7fd850933640 1 -- 172.21.15.28:0/3961839855 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd84c075b80 msgr2=0x7fd84c104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:34:57.149 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:56.523+0000 7fd850933640 1 --2- 172.21.15.28:0/3961839855 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd84c075b80 0x7fd84c104a50 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd83c009840 tx=0x7fd83c0091d0 comp rx=0 tx=0).stop 2024-07-03T15:34:57.149 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:56.523+0000 7fd850933640 1 -- 172.21.15.28:0/3961839855 shutdown_connections 2024-07-03T15:34:57.149 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:56.523+0000 7fd850933640 1 --2- 172.21.15.28:0/3961839855 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd84c075b80 0x7fd84c104a50 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:34:57.150 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:56.523+0000 7fd850933640 1 -- 172.21.15.28:0/3961839855 >> 172.21.15.28:0/3961839855 conn(0x7fd84c0fc520 msgr2=0x7fd84c0fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:34:57.150 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:56.523+0000 7fd850933640 1 -- 172.21.15.28:0/3961839855 shutdown_connections 2024-07-03T15:34:57.150 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:56.523+0000 7fd850933640 1 -- 172.21.15.28:0/3961839855 wait complete. 2024-07-03T15:34:57.150 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:56.523+0000 7fd850933640 1 Processor -- start 2024-07-03T15:34:57.150 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:56.523+0000 7fd850933640 1 -- start start 2024-07-03T15:34:57.150 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:56.523+0000 7fd850933640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd84c075b80 0x7fd84c102380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:34:57.150 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:56.523+0000 7fd850933640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84c1028c0 con 0x7fd84c075b80 2024-07-03T15:34:57.150 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:56.523+0000 7fd84a59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd84c075b80 0x7fd84c102380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:34:57.150 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:56.524+0000 7fd84a59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd84c075b80 0x7fd84c102380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36106/0 (socket says 172.21.15.28:36106) 2024-07-03T15:34:57.150 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:56.524+0000 7fd84a59c640 1 -- 172.21.15.28:0/70251860 learned_addr learned my addr 172.21.15.28:0/70251860 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:34:57.150 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:56.524+0000 7fd84a59c640 1 -- 172.21.15.28:0/70251860 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd83c009450 con 0x7fd84c075b80 2024-07-03T15:34:57.151 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:56.524+0000 7fd84a59c640 1 --2- 172.21.15.28:0/70251860 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd84c075b80 0x7fd84c102380 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd83c02a700 tx=0x7fd83c032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:34:57.151 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:56.524+0000 7fd83affd640 1 -- 172.21.15.28:0/70251860 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd83c031be0 con 0x7fd84c075b80 2024-07-03T15:34:57.151 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:56.524+0000 7fd83affd640 1 -- 172.21.15.28:0/70251860 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd83c031d40 con 0x7fd84c075b80 2024-07-03T15:34:57.151 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:56.524+0000 7fd850933640 1 -- 172.21.15.28:0/70251860 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd84c100950 con 0x7fd84c075b80 2024-07-03T15:34:57.151 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:56.524+0000 7fd83affd640 1 -- 172.21.15.28:0/70251860 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd83c03b5a0 con 0x7fd84c075b80 2024-07-03T15:34:57.151 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:56.524+0000 7fd850933640 1 -- 172.21.15.28:0/70251860 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd84c100d60 con 0x7fd84c075b80 2024-07-03T15:34:57.151 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:56.524+0000 7fd83affd640 1 -- 172.21.15.28:0/70251860 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fd83c039040 con 0x7fd84c075b80 2024-07-03T15:34:57.151 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:56.524+0000 7fd83affd640 1 -- 172.21.15.28:0/70251860 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fd83c041030 con 0x7fd84c075b80 2024-07-03T15:34:57.151 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:56.525+0000 7fd850933640 1 -- 172.21.15.28:0/70251860 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd84c05eb80 con 0x7fd84c075b80 2024-07-03T15:34:57.151 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:56.527+0000 7fd83affd640 1 -- 172.21.15.28:0/70251860 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7fd83c037070 con 0x7fd84c075b80 2024-07-03T15:34:57.151 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:56.591+0000 7fd850933640 1 -- 172.21.15.28:0/70251860 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fd84c109c70 con 0x7fd84c075b80 2024-07-03T15:34:57.151 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:56.591+0000 7fd83affd640 1 -- 172.21.15.28:0/70251860 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7fd83c03bb80 con 0x7fd84c075b80 2024-07-03T15:34:57.152 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:56.592+0000 7fd850933640 1 -- 172.21.15.28:0/70251860 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd84c075b80 msgr2=0x7fd84c102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:34:57.152 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:56.592+0000 7fd850933640 1 --2- 172.21.15.28:0/70251860 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd84c075b80 0x7fd84c102380 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd83c02a700 tx=0x7fd83c032000 comp rx=0 tx=0).stop 2024-07-03T15:34:57.152 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:56.592+0000 7fd850933640 1 -- 172.21.15.28:0/70251860 shutdown_connections 2024-07-03T15:34:57.152 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:56.592+0000 7fd850933640 1 --2- 172.21.15.28:0/70251860 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd84c075b80 0x7fd84c102380 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:34:57.152 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:56.592+0000 7fd850933640 1 -- 172.21.15.28:0/70251860 >> 172.21.15.28:0/70251860 conn(0x7fd84c0fc520 msgr2=0x7fd84c0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:34:57.152 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:56.592+0000 7fd850933640 1 -- 172.21.15.28:0/70251860 shutdown_connections 2024-07-03T15:34:57.152 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:56.592+0000 7fd850933640 1 -- 172.21.15.28:0/70251860 wait complete. 2024-07-03T15:34:57.152 INFO:teuthology.orchestra.run.smithi028.stdout:mon is available 2024-07-03T15:34:57.152 INFO:teuthology.orchestra.run.smithi028.stdout:Assimilating anything we can from ceph.conf... 2024-07-03T15:34:58.128 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:34:58 smithi028 ceph-mon[28782]: from='client.? 172.21.15.28:0/48649030' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-07-03T15:34:58.128 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:34:58 smithi028 ceph-mon[28782]: from='client.? 172.21.15.28:0/48649030' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-07-03T15:34:58.136 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout 2024-07-03T15:34:58.136 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout [global] 2024-07-03T15:34:58.136 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout fsid = a85e4be6-3951-11ef-bcab-c7b262605968 2024-07-03T15:34:58.136 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.28:3300,v1:172.21.15.28:6789] 2024-07-03T15:34:58.136 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-07-03T15:34:58.137 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-07-03T15:34:58.137 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-07-03T15:34:58.137 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-07-03T15:34:58.137 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout 2024-07-03T15:34:58.137 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout [mgr] 2024-07-03T15:34:58.137 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-07-03T15:34:58.137 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout 2024-07-03T15:34:58.137 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout [osd] 2024-07-03T15:34:58.137 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-07-03T15:34:58.137 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-07-03T15:34:58.137 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-07-03T15:34:58.137 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-07-03T15:34:58.137 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-07-03T15:34:58.138 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.527+0000 7f99c24d2640 1 Processor -- start 2024-07-03T15:34:58.138 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.527+0000 7f99c24d2640 1 -- start start 2024-07-03T15:34:58.138 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.527+0000 7f99c24d2640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f99bc101e00 0x7f99bc102210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:34:58.138 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.527+0000 7f99c24d2640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99bc102750 con 0x7f99bc101e00 2024-07-03T15:34:58.138 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.527+0000 7f99bbfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f99bc101e00 0x7f99bc102210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:34:58.138 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.527+0000 7f99bbfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f99bc101e00 0x7f99bc102210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36120/0 (socket says 172.21.15.28:36120) 2024-07-03T15:34:58.138 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.527+0000 7f99bbfff640 1 -- 172.21.15.28:0/4100674278 learned_addr learned my addr 172.21.15.28:0/4100674278 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:34:58.138 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.527+0000 7f99bbfff640 1 -- 172.21.15.28:0/4100674278 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99bc102890 con 0x7f99bc101e00 2024-07-03T15:34:58.138 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.528+0000 7f99bbfff640 1 --2- 172.21.15.28:0/4100674278 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f99bc101e00 0x7f99bc102210 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f99ac009840 tx=0x7f99ac0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=920daa643626ce43 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:34:58.138 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.528+0000 7f99baffd640 1 -- 172.21.15.28:0/4100674278 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99ac032440 con 0x7f99bc101e00 2024-07-03T15:34:58.138 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.528+0000 7f99baffd640 1 -- 172.21.15.28:0/4100674278 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99ac0325a0 con 0x7f99bc101e00 2024-07-03T15:34:58.139 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.528+0000 7f99baffd640 1 -- 172.21.15.28:0/4100674278 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99ac0328d0 con 0x7f99bc101e00 2024-07-03T15:34:58.139 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.529+0000 7f99c24d2640 1 -- 172.21.15.28:0/4100674278 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f99bc101e00 msgr2=0x7f99bc102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:34:58.139 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.529+0000 7f99c24d2640 1 --2- 172.21.15.28:0/4100674278 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f99bc101e00 0x7f99bc102210 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f99ac009840 tx=0x7f99ac0091d0 comp rx=0 tx=0).stop 2024-07-03T15:34:58.139 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.529+0000 7f99c24d2640 1 -- 172.21.15.28:0/4100674278 shutdown_connections 2024-07-03T15:34:58.139 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.529+0000 7f99c24d2640 1 --2- 172.21.15.28:0/4100674278 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f99bc101e00 0x7f99bc102210 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:34:58.139 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.529+0000 7f99c24d2640 1 -- 172.21.15.28:0/4100674278 >> 172.21.15.28:0/4100674278 conn(0x7f99bc0fd5a0 msgr2=0x7f99bc0ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:34:58.139 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.529+0000 7f99c24d2640 1 -- 172.21.15.28:0/4100674278 shutdown_connections 2024-07-03T15:34:58.139 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.529+0000 7f99c24d2640 1 -- 172.21.15.28:0/4100674278 wait complete. 2024-07-03T15:34:58.139 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.530+0000 7f99c24d2640 1 Processor -- start 2024-07-03T15:34:58.139 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.530+0000 7f99c24d2640 1 -- start start 2024-07-03T15:34:58.139 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.530+0000 7f99c24d2640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f99bc101e00 0x7f99bc077c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:34:58.139 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.530+0000 7f99c24d2640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99bc078160 con 0x7f99bc101e00 2024-07-03T15:34:58.139 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.530+0000 7f99bbfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f99bc101e00 0x7f99bc077c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:34:58.140 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.530+0000 7f99bbfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f99bc101e00 0x7f99bc077c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36136/0 (socket says 172.21.15.28:36136) 2024-07-03T15:34:58.140 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.530+0000 7f99bbfff640 1 -- 172.21.15.28:0/48649030 learned_addr learned my addr 172.21.15.28:0/48649030 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:34:58.140 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.530+0000 7f99bbfff640 1 -- 172.21.15.28:0/48649030 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99ac009450 con 0x7f99bc101e00 2024-07-03T15:34:58.140 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.531+0000 7f99bbfff640 1 --2- 172.21.15.28:0/48649030 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f99bc101e00 0x7f99bc077c20 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f99ac02a700 tx=0x7f99ac032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:34:58.140 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.531+0000 7f99b8ff9640 1 -- 172.21.15.28:0/48649030 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99ac031be0 con 0x7f99bc101e00 2024-07-03T15:34:58.140 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.531+0000 7f99b8ff9640 1 -- 172.21.15.28:0/48649030 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99ac031d40 con 0x7f99bc101e00 2024-07-03T15:34:58.140 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.531+0000 7f99c24d2640 1 -- 172.21.15.28:0/48649030 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99bc0761f0 con 0x7f99bc101e00 2024-07-03T15:34:58.140 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.531+0000 7f99b8ff9640 1 -- 172.21.15.28:0/48649030 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99ac03b5a0 con 0x7f99bc101e00 2024-07-03T15:34:58.140 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.531+0000 7f99c24d2640 1 -- 172.21.15.28:0/48649030 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99bc076600 con 0x7f99bc101e00 2024-07-03T15:34:58.140 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.531+0000 7f99b8ff9640 1 -- 172.21.15.28:0/48649030 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f99ac039040 con 0x7f99bc101e00 2024-07-03T15:34:58.140 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.531+0000 7f99b8ff9640 1 -- 172.21.15.28:0/48649030 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f99ac041030 con 0x7f99bc101e00 2024-07-03T15:34:58.140 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.532+0000 7f99c24d2640 1 -- 172.21.15.28:0/48649030 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99bc05eb80 con 0x7f99bc101e00 2024-07-03T15:34:58.140 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.534+0000 7f99b8ff9640 1 -- 172.21.15.28:0/48649030 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f99ac037070 con 0x7f99bc101e00 2024-07-03T15:34:58.141 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.595+0000 7f99c24d2640 1 -- 172.21.15.28:0/48649030 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f99bc103220 con 0x7f99bc101e00 2024-07-03T15:34:58.141 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.598+0000 7f99b8ff9640 1 -- 172.21.15.28:0/48649030 <== mon.0 v2:172.21.15.28:3300/0 7 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f99ac04e810 con 0x7f99bc101e00 2024-07-03T15:34:58.141 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.598+0000 7f99b8ff9640 1 -- 172.21.15.28:0/48649030 <== mon.0 v2:172.21.15.28:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f99ac054d60 con 0x7f99bc101e00 2024-07-03T15:34:58.141 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.599+0000 7f99c24d2640 1 -- 172.21.15.28:0/48649030 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f99bc101e00 msgr2=0x7f99bc077c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:34:58.141 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.599+0000 7f99c24d2640 1 --2- 172.21.15.28:0/48649030 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f99bc101e00 0x7f99bc077c20 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f99ac02a700 tx=0x7f99ac032000 comp rx=0 tx=0).stop 2024-07-03T15:34:58.141 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.599+0000 7f99c24d2640 1 -- 172.21.15.28:0/48649030 shutdown_connections 2024-07-03T15:34:58.141 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.599+0000 7f99c24d2640 1 --2- 172.21.15.28:0/48649030 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f99bc101e00 0x7f99bc077c20 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:34:58.141 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.599+0000 7f99c24d2640 1 -- 172.21.15.28:0/48649030 >> 172.21.15.28:0/48649030 conn(0x7f99bc0fd5a0 msgr2=0x7f99bc107240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:34:58.141 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.599+0000 7f99c24d2640 1 -- 172.21.15.28:0/48649030 shutdown_connections 2024-07-03T15:34:58.141 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:57.599+0000 7f99c24d2640 1 -- 172.21.15.28:0/48649030 wait complete. 2024-07-03T15:34:58.141 INFO:teuthology.orchestra.run.smithi028.stdout:Generating new minimal ceph.conf... 2024-07-03T15:34:59.165 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:58.531+0000 7f7e09291640 1 Processor -- start 2024-07-03T15:34:59.165 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:58.531+0000 7f7e09291640 1 -- start start 2024-07-03T15:34:59.165 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:58.531+0000 7f7e09291640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7e04100190 0x7f7e041005a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:34:59.165 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:58.531+0000 7f7e09291640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e04100ae0 con 0x7f7e04100190 2024-07-03T15:34:59.166 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:58.531+0000 7f7e02ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7e04100190 0x7f7e041005a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:34:59.166 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:58.531+0000 7f7e02ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7e04100190 0x7f7e041005a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36146/0 (socket says 172.21.15.28:36146) 2024-07-03T15:34:59.166 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:58.531+0000 7f7e02ffd640 1 -- 172.21.15.28:0/402185457 learned_addr learned my addr 172.21.15.28:0/402185457 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:34:59.166 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:58.531+0000 7f7e02ffd640 1 -- 172.21.15.28:0/402185457 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e04100c20 con 0x7f7e04100190 2024-07-03T15:34:59.166 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:58.532+0000 7f7e02ffd640 1 --2- 172.21.15.28:0/402185457 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7e04100190 0x7f7e041005a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7df4009840 tx=0x7f7df40091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e28738d3ad2e6222 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:34:59.166 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:58.532+0000 7f7e027fc640 1 -- 172.21.15.28:0/402185457 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7df4032440 con 0x7f7e04100190 2024-07-03T15:34:59.166 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:58.532+0000 7f7e027fc640 1 -- 172.21.15.28:0/402185457 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f7df4032a30 con 0x7f7e04100190 2024-07-03T15:34:59.166 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:58.532+0000 7f7e027fc640 1 -- 172.21.15.28:0/402185457 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7df4031ad0 con 0x7f7e04100190 2024-07-03T15:34:59.166 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:58.533+0000 7f7e09291640 1 -- 172.21.15.28:0/402185457 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7e04100190 msgr2=0x7f7e041005a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:34:59.166 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:58.533+0000 7f7e09291640 1 --2- 172.21.15.28:0/402185457 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7e04100190 0x7f7e041005a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7df4009840 tx=0x7f7df40091d0 comp rx=0 tx=0).stop 2024-07-03T15:34:59.166 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:58.533+0000 7f7e09291640 1 -- 172.21.15.28:0/402185457 shutdown_connections 2024-07-03T15:34:59.166 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:58.533+0000 7f7e09291640 1 --2- 172.21.15.28:0/402185457 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7e04100190 0x7f7e041005a0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:34:59.166 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:58.533+0000 7f7e09291640 1 -- 172.21.15.28:0/402185457 >> 172.21.15.28:0/402185457 conn(0x7f7e040fbcc0 msgr2=0x7f7e040fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:34:59.167 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:58.533+0000 7f7e09291640 1 -- 172.21.15.28:0/402185457 shutdown_connections 2024-07-03T15:34:59.167 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:58.533+0000 7f7e09291640 1 -- 172.21.15.28:0/402185457 wait complete. 2024-07-03T15:34:59.167 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:58.534+0000 7f7e09291640 1 Processor -- start 2024-07-03T15:34:59.167 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:58.534+0000 7f7e09291640 1 -- start start 2024-07-03T15:34:59.167 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:58.534+0000 7f7e09291640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7e04100190 0x7f7e04110a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:34:59.167 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:58.534+0000 7f7e09291640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e04110fd0 con 0x7f7e04100190 2024-07-03T15:34:59.167 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:58.534+0000 7f7e02ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7e04100190 0x7f7e04110a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:34:59.167 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:58.534+0000 7f7e02ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7e04100190 0x7f7e04110a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36150/0 (socket says 172.21.15.28:36150) 2024-07-03T15:34:59.167 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:58.534+0000 7f7e02ffd640 1 -- 172.21.15.28:0/612053784 learned_addr learned my addr 172.21.15.28:0/612053784 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:34:59.167 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:58.534+0000 7f7e02ffd640 1 -- 172.21.15.28:0/612053784 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df4009450 con 0x7f7e04100190 2024-07-03T15:34:59.167 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:58.534+0000 7f7e02ffd640 1 --2- 172.21.15.28:0/612053784 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7e04100190 0x7f7e04110a90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7df40317b0 tx=0x7f7df4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:34:59.167 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:58.535+0000 7f7dfbfff640 1 -- 172.21.15.28:0/612053784 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7df4031e90 con 0x7f7e04100190 2024-07-03T15:34:59.167 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:58.535+0000 7f7dfbfff640 1 -- 172.21.15.28:0/612053784 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f7df403daf0 con 0x7f7e04100190 2024-07-03T15:34:59.167 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:58.535+0000 7f7e09291640 1 -- 172.21.15.28:0/612053784 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e041111d0 con 0x7f7e04100190 2024-07-03T15:34:59.168 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:58.535+0000 7f7dfbfff640 1 -- 172.21.15.28:0/612053784 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7df403ba60 con 0x7f7e04100190 2024-07-03T15:34:59.168 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:58.535+0000 7f7e09291640 1 -- 172.21.15.28:0/612053784 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e04111640 con 0x7f7e04100190 2024-07-03T15:34:59.168 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:58.535+0000 7f7dfbfff640 1 -- 172.21.15.28:0/612053784 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f7df4039070 con 0x7f7e04100190 2024-07-03T15:34:59.168 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:58.535+0000 7f7dfbfff640 1 -- 172.21.15.28:0/612053784 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f7df4046940 con 0x7f7e04100190 2024-07-03T15:34:59.168 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:58.536+0000 7f7e09291640 1 -- 172.21.15.28:0/612053784 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e0404b530 con 0x7f7e04100190 2024-07-03T15:34:59.168 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:58.538+0000 7f7dfbfff640 1 -- 172.21.15.28:0/612053784 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f7e04111640 con 0x7f7e04100190 2024-07-03T15:34:59.168 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:58.600+0000 7f7e09291640 1 -- 172.21.15.28:0/612053784 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f7e04111d60 con 0x7f7e04100190 2024-07-03T15:34:59.168 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:58.600+0000 7f7dfbfff640 1 -- 172.21.15.28:0/612053784 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f7df4041020 con 0x7f7e04100190 2024-07-03T15:34:59.168 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:58.601+0000 7f7e09291640 1 -- 172.21.15.28:0/612053784 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7e04100190 msgr2=0x7f7e04110a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:34:59.168 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:58.601+0000 7f7e09291640 1 --2- 172.21.15.28:0/612053784 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7e04100190 0x7f7e04110a90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7df40317b0 tx=0x7f7df4008040 comp rx=0 tx=0).stop 2024-07-03T15:34:59.168 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:58.601+0000 7f7e09291640 1 -- 172.21.15.28:0/612053784 shutdown_connections 2024-07-03T15:34:59.168 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:58.601+0000 7f7e09291640 1 --2- 172.21.15.28:0/612053784 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7e04100190 0x7f7e04110a90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:34:59.169 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:58.601+0000 7f7e09291640 1 -- 172.21.15.28:0/612053784 >> 172.21.15.28:0/612053784 conn(0x7f7e040fbcc0 msgr2=0x7f7e040fc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:34:59.169 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:58.601+0000 7f7e09291640 1 -- 172.21.15.28:0/612053784 shutdown_connections 2024-07-03T15:34:59.169 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:34:58.601+0000 7f7e09291640 1 -- 172.21.15.28:0/612053784 wait complete. 2024-07-03T15:34:59.169 INFO:teuthology.orchestra.run.smithi028.stdout:Restarting the monitor... 2024-07-03T15:34:59.182 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:34:59 smithi028 ceph-mon[28782]: from='client.? 172.21.15.28:0/612053784' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:34:59.574 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:34:59 smithi028 systemd[1]: Stopping Ceph mon.smithi028 for a85e4be6-3951-11ef-bcab-c7b262605968... 2024-07-03T15:34:59.574 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:34:59 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-mon-smithi028[28778]: 2024-07-03T15:34:59.419+0000 7f2dffa9f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi028 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-07-03T15:34:59.574 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:34:59 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-mon-smithi028[28778]: 2024-07-03T15:34:59.419+0000 7f2dffa9f640 -1 mon.smithi028@0(leader) e1 *** Got Signal Terminated *** 2024-07-03T15:34:59.836 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:34:59 smithi028 podman[29025]: 2024-07-03 15:34:59.573161914 +0000 UTC m=+0.242682174 container died 05faf912bff31f651a14b68ee54d2d802d93a7d733d94a488e35ef25815cd105 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-mon-smithi028, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=quincy-d4b063a, org.label-schema.build-date=20240625, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.36.0) 2024-07-03T15:35:00.120 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:34:59 smithi028 podman[29025]: 2024-07-03 15:34:59.91052659 +0000 UTC m=+0.580046849 container cleanup 05faf912bff31f651a14b68ee54d2d802d93a7d733d94a488e35ef25815cd105 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-mon-smithi028, org.label-schema.license=GPLv2, RELEASE=quincy-d4b063a, ceph=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.build-date=20240625, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-07-03T15:35:00.120 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:34:59 smithi028 bash[29025]: ceph-a85e4be6-3951-11ef-bcab-c7b262605968-mon-smithi028 2024-07-03T15:35:00.120 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 podman[29038]: 2024-07-03 15:35:00.077804412 +0000 UTC m=+0.499727419 container remove 05faf912bff31f651a14b68ee54d2d802d93a7d733d94a488e35ef25815cd105 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-mon-smithi028, GIT_CLEAN=True, RELEASE=quincy-d4b063a, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240625, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-03T15:35:00.617 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 systemd[1]: ceph-a85e4be6-3951-11ef-bcab-c7b262605968@mon.smithi028.service: Deactivated successfully. 2024-07-03T15:35:00.618 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 systemd[1]: Stopped Ceph mon.smithi028 for a85e4be6-3951-11ef-bcab-c7b262605968. 2024-07-03T15:35:00.618 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 systemd[1]: Starting Ceph mon.smithi028 for a85e4be6-3951-11ef-bcab-c7b262605968... 2024-07-03T15:35:00.618 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 podman[29129]: 2024-07-03 15:35:00.503601579 +0000 UTC m=+0.106626095 container create 7461f9bef4e0a95623bbd52ef076537cfa9eb65ae7b952d69e006ee86776c794 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-mon-smithi028, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240625, maintainer=Guillaume Abrioux , RELEASE=quincy-d4b063a, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-03T15:35:00.618 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 podman[29129]: 2024-07-03 15:35:00.414352718 +0000 UTC m=+0.017377245 image pull 4ca399d9e65db9fa94fcda22a8b06b0fd12b8ab90081173322ce426b15957323 quay.ceph.io/ceph-ci/ceph:quincy 2024-07-03T15:35:00.802 INFO:teuthology.orchestra.run.smithi028.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-07-03T15:35:00.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 podman[29129]: 2024-07-03 15:35:00.722434391 +0000 UTC m=+0.325458918 container init 7461f9bef4e0a95623bbd52ef076537cfa9eb65ae7b952d69e006ee86776c794 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-mon-smithi028, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240625, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=quincy-d4b063a, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-07-03T15:35:00.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 podman[29129]: 2024-07-03 15:35:00.726317257 +0000 UTC m=+0.329341773 container start 7461f9bef4e0a95623bbd52ef076537cfa9eb65ae7b952d69e006ee86776c794 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-mon-smithi028, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240625, io.buildah.version=1.36.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=quincy-d4b063a, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-07-03T15:35:00.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: set uid:gid to 167:167 (ceph:ceph) 2024-07-03T15:35:00.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable), process ceph-mon, pid 2 2024-07-03T15:35:00.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: pidfile_write: ignore empty --pid-file 2024-07-03T15:35:00.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: load: jerasure load: lrc 2024-07-03T15:35:00.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: RocksDB version: 6.15.5 2024-07-03T15:35:00.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-07-03T15:35:00.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Compile date Jul 1 2024 2024-07-03T15:35:00.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: DB SUMMARY 2024-07-03T15:35:00.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: DB Session ID: CO3RYRPN8G5XYKAW7W0B 2024-07-03T15:35:00.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: CURRENT file: CURRENT 2024-07-03T15:35:00.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: IDENTITY file: IDENTITY 2024-07-03T15:35:00.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-07-03T15:35:00.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi028/store.db dir, Total Num: 1, files: 000008.sst 2024-07-03T15:35:00.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi028/store.db: 000010.log size: 98142 ; 2024-07-03T15:35:00.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.error_if_exists: 0 2024-07-03T15:35:00.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.create_if_missing: 0 2024-07-03T15:35:00.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.paranoid_checks: 1 2024-07-03T15:35:00.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-07-03T15:35:00.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.env: 0x55a6a5b88440 2024-07-03T15:35:00.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.fs: Posix File System 2024-07-03T15:35:00.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.info_log: 0x55a6a7c65120 2024-07-03T15:35:00.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.max_file_opening_threads: 16 2024-07-03T15:35:00.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.statistics: (nil) 2024-07-03T15:35:00.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.use_fsync: 0 2024-07-03T15:35:00.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.max_log_file_size: 0 2024-07-03T15:35:00.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-07-03T15:35:00.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.log_file_time_to_roll: 0 2024-07-03T15:35:00.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.keep_log_file_num: 1000 2024-07-03T15:35:00.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.recycle_log_file_num: 0 2024-07-03T15:35:00.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.allow_fallocate: 1 2024-07-03T15:35:00.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.allow_mmap_reads: 0 2024-07-03T15:35:00.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.allow_mmap_writes: 0 2024-07-03T15:35:00.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.use_direct_reads: 0 2024-07-03T15:35:00.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-07-03T15:35:00.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.create_missing_column_families: 0 2024-07-03T15:35:00.874 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.db_log_dir: 2024-07-03T15:35:00.874 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi028/store.db 2024-07-03T15:35:00.874 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.table_cache_numshardbits: 6 2024-07-03T15:35:00.874 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.WAL_ttl_seconds: 0 2024-07-03T15:35:00.874 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.WAL_size_limit_MB: 0 2024-07-03T15:35:00.874 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-07-03T15:35:00.874 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-07-03T15:35:00.874 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.is_fd_close_on_exec: 1 2024-07-03T15:35:00.874 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.advise_random_on_open: 1 2024-07-03T15:35:00.874 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.db_write_buffer_size: 0 2024-07-03T15:35:00.874 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.write_buffer_manager: 0x55a6a7d22840 2024-07-03T15:35:00.874 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-07-03T15:35:00.874 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-07-03T15:35:00.875 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-07-03T15:35:00.875 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.use_adaptive_mutex: 0 2024-07-03T15:35:00.875 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.rate_limiter: (nil) 2024-07-03T15:35:00.875 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-07-03T15:35:00.875 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.wal_recovery_mode: 2 2024-07-03T15:35:00.875 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.enable_thread_tracking: 0 2024-07-03T15:35:00.875 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.enable_pipelined_write: 0 2024-07-03T15:35:00.875 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.unordered_write: 0 2024-07-03T15:35:00.875 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-07-03T15:35:00.875 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-07-03T15:35:00.875 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-07-03T15:35:00.875 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-07-03T15:35:00.875 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.row_cache: None 2024-07-03T15:35:00.876 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.wal_filter: None 2024-07-03T15:35:00.876 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-07-03T15:35:00.876 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.allow_ingest_behind: 0 2024-07-03T15:35:00.877 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.preserve_deletes: 0 2024-07-03T15:35:00.877 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.two_write_queues: 0 2024-07-03T15:35:00.877 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.manual_wal_flush: 0 2024-07-03T15:35:00.877 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.atomic_flush: 0 2024-07-03T15:35:00.877 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-07-03T15:35:00.877 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.persist_stats_to_disk: 0 2024-07-03T15:35:00.877 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.write_dbid_to_manifest: 0 2024-07-03T15:35:00.878 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.log_readahead_size: 0 2024-07-03T15:35:00.878 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-07-03T15:35:00.878 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.best_efforts_recovery: 0 2024-07-03T15:35:00.878 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-07-03T15:35:00.878 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-07-03T15:35:00.878 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.allow_data_in_errors: 0 2024-07-03T15:35:00.878 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.db_host_id: __hostname__ 2024-07-03T15:35:00.878 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.max_background_jobs: 2 2024-07-03T15:35:00.878 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.max_background_compactions: -1 2024-07-03T15:35:00.878 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.max_subcompactions: 1 2024-07-03T15:35:00.878 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-07-03T15:35:00.879 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-07-03T15:35:00.879 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.delayed_write_rate : 16777216 2024-07-03T15:35:00.879 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.max_total_wal_size: 0 2024-07-03T15:35:00.879 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-07-03T15:35:00.879 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.stats_dump_period_sec: 600 2024-07-03T15:35:00.879 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.stats_persist_period_sec: 600 2024-07-03T15:35:00.879 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-07-03T15:35:00.879 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.max_open_files: -1 2024-07-03T15:35:00.879 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.bytes_per_sync: 0 2024-07-03T15:35:00.879 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.wal_bytes_per_sync: 0 2024-07-03T15:35:00.879 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.strict_bytes_per_sync: 0 2024-07-03T15:35:00.879 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.compaction_readahead_size: 0 2024-07-03T15:35:00.879 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.max_background_flushes: -1 2024-07-03T15:35:00.879 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Compression algorithms supported: 2024-07-03T15:35:00.880 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: kZSTD supported: 0 2024-07-03T15:35:00.880 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: kXpressCompression supported: 0 2024-07-03T15:35:00.880 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: kBZip2Compression supported: 0 2024-07-03T15:35:00.880 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-07-03T15:35:00.880 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: kLZ4Compression supported: 1 2024-07-03T15:35:00.880 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: kZlibCompression supported: 1 2024-07-03T15:35:00.880 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: kLZ4HCCompression supported: 1 2024-07-03T15:35:00.880 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: kSnappyCompression supported: 1 2024-07-03T15:35:00.880 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Fast CRC32 supported: Supported on x86 2024-07-03T15:35:00.880 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi028/store.db/MANIFEST-000009 2024-07-03T15:35:00.880 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-07-03T15:35:00.880 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-07-03T15:35:00.881 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.merge_operator: 2024-07-03T15:35:00.881 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.compaction_filter: None 2024-07-03T15:35:00.881 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.compaction_filter_factory: None 2024-07-03T15:35:00.881 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.sst_partitioner_factory: None 2024-07-03T15:35:00.881 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.memtable_factory: SkipListFactory 2024-07-03T15:35:00.881 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.table_factory: BlockBasedTable 2024-07-03T15:35:00.881 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55a6a7c0ba68) 2024-07-03T15:35:00.881 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: cache_index_and_filter_blocks: 1 2024-07-03T15:35:00.881 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-07-03T15:35:00.881 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-07-03T15:35:00.881 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: pin_top_level_index_and_filter: 1 2024-07-03T15:35:00.881 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: index_type: 0 2024-07-03T15:35:00.881 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: data_block_index_type: 0 2024-07-03T15:35:00.881 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: index_shortening: 1 2024-07-03T15:35:00.882 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: data_block_hash_table_util_ratio: 0.750000 2024-07-03T15:35:00.882 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: hash_index_allow_collision: 1 2024-07-03T15:35:00.882 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: checksum: 1 2024-07-03T15:35:00.882 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: no_block_cache: 0 2024-07-03T15:35:00.882 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: block_cache: 0x55a6a7c411f0 2024-07-03T15:35:00.882 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: block_cache_name: BinnedLRUCache 2024-07-03T15:35:00.882 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: block_cache_options: 2024-07-03T15:35:00.882 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: capacity : 536870912 2024-07-03T15:35:00.882 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: num_shard_bits : 4 2024-07-03T15:35:00.882 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: strict_capacity_limit : 0 2024-07-03T15:35:00.882 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: high_pri_pool_ratio: 0.000 2024-07-03T15:35:00.882 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: block_cache_compressed: (nil) 2024-07-03T15:35:00.882 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: persistent_cache: (nil) 2024-07-03T15:35:00.883 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: block_size: 4096 2024-07-03T15:35:00.883 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: block_size_deviation: 10 2024-07-03T15:35:00.883 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: block_restart_interval: 16 2024-07-03T15:35:00.883 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: index_block_restart_interval: 1 2024-07-03T15:35:00.883 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: metadata_block_size: 4096 2024-07-03T15:35:00.883 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: partition_filters: 0 2024-07-03T15:35:00.883 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: use_delta_encoding: 1 2024-07-03T15:35:00.883 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-07-03T15:35:00.883 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: whole_key_filtering: 1 2024-07-03T15:35:00.883 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: verify_compression: 0 2024-07-03T15:35:00.883 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: read_amp_bytes_per_bit: 0 2024-07-03T15:35:00.883 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: format_version: 4 2024-07-03T15:35:00.883 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: enable_index_compression: 1 2024-07-03T15:35:00.883 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: block_align: 0 2024-07-03T15:35:00.883 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.write_buffer_size: 33554432 2024-07-03T15:35:00.884 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.max_write_buffer_number: 2 2024-07-03T15:35:00.884 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.compression: NoCompression 2024-07-03T15:35:00.884 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.bottommost_compression: Disabled 2024-07-03T15:35:00.884 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.prefix_extractor: nullptr 2024-07-03T15:35:00.884 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-07-03T15:35:00.884 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.num_levels: 7 2024-07-03T15:35:00.884 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-07-03T15:35:00.884 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-07-03T15:35:00.884 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-07-03T15:35:00.884 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-07-03T15:35:00.884 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-07-03T15:35:00.884 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-07-03T15:35:00.884 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-07-03T15:35:00.885 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-07-03T15:35:00.885 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-07-03T15:35:00.885 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-07-03T15:35:00.885 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.compression_opts.window_bits: -14 2024-07-03T15:35:00.885 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.compression_opts.level: 32767 2024-07-03T15:35:00.885 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.compression_opts.strategy: 0 2024-07-03T15:35:00.885 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-07-03T15:35:00.885 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-07-03T15:35:00.885 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-07-03T15:35:00.885 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.compression_opts.enabled: false 2024-07-03T15:35:00.885 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-07-03T15:35:00.885 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-07-03T15:35:00.885 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-07-03T15:35:00.885 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.target_file_size_base: 67108864 2024-07-03T15:35:00.885 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.target_file_size_multiplier: 1 2024-07-03T15:35:00.886 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-07-03T15:35:00.886 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-07-03T15:35:00.886 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-07-03T15:35:00.886 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-07-03T15:35:00.886 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-07-03T15:35:00.886 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-07-03T15:35:00.886 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-07-03T15:35:00.886 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-07-03T15:35:00.886 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-07-03T15:35:00.886 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-07-03T15:35:00.886 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-07-03T15:35:00.886 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-07-03T15:35:00.886 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.arena_block_size: 4194304 2024-07-03T15:35:00.887 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-07-03T15:35:00.887 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-07-03T15:35:00.887 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-07-03T15:35:00.887 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.disable_auto_compactions: 0 2024-07-03T15:35:00.887 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-07-03T15:35:00.887 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-07-03T15:35:00.887 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-07-03T15:35:00.887 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-07-03T15:35:00.887 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-07-03T15:35:00.887 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-07-03T15:35:00.887 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-07-03T15:35:00.887 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-07-03T15:35:00.887 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-07-03T15:35:00.888 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-07-03T15:35:00.888 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.table_properties_collectors: 2024-07-03T15:35:00.888 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.inplace_update_support: 0 2024-07-03T15:35:00.888 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.inplace_update_num_locks: 10000 2024-07-03T15:35:00.888 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-07-03T15:35:00.888 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-07-03T15:35:00.888 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.memtable_huge_page_size: 0 2024-07-03T15:35:00.888 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.bloom_locality: 0 2024-07-03T15:35:00.888 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.max_successive_merges: 0 2024-07-03T15:35:00.888 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.optimize_filters_for_hits: 0 2024-07-03T15:35:00.888 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.paranoid_file_checks: 0 2024-07-03T15:35:00.888 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.force_consistency_checks: 1 2024-07-03T15:35:00.888 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.report_bg_io_stats: 0 2024-07-03T15:35:00.888 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.ttl: 2592000 2024-07-03T15:35:00.889 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.periodic_compaction_seconds: 0 2024-07-03T15:35:00.889 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.enable_blob_files: false 2024-07-03T15:35:00.889 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.min_blob_size: 0 2024-07-03T15:35:00.889 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.blob_file_size: 268435456 2024-07-03T15:35:00.889 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.blob_compression_type: NoCompression 2024-07-03T15:35:00.889 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.enable_blob_garbage_collection: false 2024-07-03T15:35:00.889 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-07-03T15:35:00.889 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi028/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-07-03T15:35:00.889 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-07-03T15:35:00.889 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-07-03T15:35:00.889 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: EVENT_LOG_v1 {"time_micros": 1720020900751193, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-07-03T15:35:00.889 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-07-03T15:35:00.889 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-07-03T15:35:00.890 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: EVENT_LOG_v1 {"time_micros": 1720020900752162, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 93348, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 91460, "index_size": 248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13576, "raw_average_key_size": 49, "raw_value_size": 84024, "raw_average_value_size": 307, "num_data_blocks": 11, "num_entries": 273, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1720020900, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "2f240546-3b3a-499f-a585-5ecb60da485f", "db_session_id": "CO3RYRPN8G5XYKAW7W0B"}} 2024-07-03T15:35:00.890 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-07-03T15:35:00.890 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: EVENT_LOG_v1 {"time_micros": 1720020900752589, "job": 1, "event": "recovery_finished"} 2024-07-03T15:35:00.890 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-smithi028/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-07-03T15:35:00.890 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x55a6a7c74700 2024-07-03T15:35:00.890 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: DB pointer 0x55a6a7c44000 2024-07-03T15:35:00.890 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-07-03T15:35:00.890 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-07-03T15:35:00.890 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: ** DB Stats ** 2024-07-03T15:35:00.890 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-07-03T15:35:00.890 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-07-03T15:35:00.890 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-07-03T15:35:00.891 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-07-03T15:35:00.891 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-07-03T15:35:00.891 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-07-03T15:35:00.891 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-07-03T15:35:00.891 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: 2024-07-03T15:35:00.891 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: ** Compaction Stats [default] ** 2024-07-03T15:35:00.891 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-07-03T15:35:00.891 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-07-03T15:35:00.891 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: L0 2/0 92.96 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 135.9 0.00 0.00 1 0.001 0 0 2024-07-03T15:35:00.891 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Sum 2/0 92.96 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 135.9 0.00 0.00 1 0.001 0 0 2024-07-03T15:35:00.891 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 135.9 0.00 0.00 1 0.001 0 0 2024-07-03T15:35:00.891 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: 2024-07-03T15:35:00.891 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: ** Compaction Stats [default] ** 2024-07-03T15:35:00.891 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-07-03T15:35:00.891 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-07-03T15:35:00.892 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 135.9 0.00 0.00 1 0.001 0 0 2024-07-03T15:35:00.892 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-07-03T15:35:00.892 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-07-03T15:35:00.892 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-07-03T15:35:00.892 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-07-03T15:35:00.892 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-07-03T15:35:00.892 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: AddFile(Keys): cumulative 0, interval 0 2024-07-03T15:35:00.892 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Cumulative compaction: 0.00 GB write, 26.02 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-07-03T15:35:00.892 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Interval compaction: 0.00 GB write, 26.02 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-07-03T15:35:00.892 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-07-03T15:35:00.892 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: 2024-07-03T15:35:00.892 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: ** File Read Latency Histogram By Level [default] ** 2024-07-03T15:35:00.892 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: 2024-07-03T15:35:00.893 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: ** Compaction Stats [default] ** 2024-07-03T15:35:00.893 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-07-03T15:35:00.893 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-07-03T15:35:00.893 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: L0 2/0 92.96 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 135.9 0.00 0.00 1 0.001 0 0 2024-07-03T15:35:00.893 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Sum 2/0 92.96 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 135.9 0.00 0.00 1 0.001 0 0 2024-07-03T15:35:00.893 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-07-03T15:35:00.893 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: 2024-07-03T15:35:00.893 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: ** Compaction Stats [default] ** 2024-07-03T15:35:00.893 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-07-03T15:35:00.893 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-07-03T15:35:00.893 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 135.9 0.00 0.00 1 0.001 0 0 2024-07-03T15:35:00.893 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-07-03T15:35:00.893 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-07-03T15:35:00.893 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-07-03T15:35:00.893 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-07-03T15:35:00.894 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-07-03T15:35:00.894 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: AddFile(Keys): cumulative 0, interval 0 2024-07-03T15:35:00.894 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Cumulative compaction: 0.00 GB write, 25.77 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-07-03T15:35:00.894 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-07-03T15:35:00.894 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-07-03T15:35:00.894 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: 2024-07-03T15:35:00.894 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: ** File Read Latency Histogram By Level [default] ** 2024-07-03T15:35:00.894 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: starting mon.smithi028 rank 0 at public addrs [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] at bind addrs [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi028 fsid a85e4be6-3951-11ef-bcab-c7b262605968 2024-07-03T15:35:00.894 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: mon.smithi028@-1(???) e1 preinit fsid a85e4be6-3951-11ef-bcab-c7b262605968 2024-07-03T15:35:00.894 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: mon.smithi028@-1(???).mds e1 new map 2024-07-03T15:35:00.894 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: mon.smithi028@-1(???).mds e1 print_map 2024-07-03T15:35:00.894 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: e1 2024-07-03T15:35:00.894 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-07-03T15:35:00.895 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:35:00.895 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: legacy client fscid: -1 2024-07-03T15:35:00.895 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: 2024-07-03T15:35:00.895 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: No filesystems configured 2024-07-03T15:35:00.895 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: mon.smithi028@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-07-03T15:35:00.895 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: mon.smithi028@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-07-03T15:35:00.895 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: mon.smithi028@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-07-03T15:35:00.895 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: mon.smithi028@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-07-03T15:35:00.895 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: mon.smithi028@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-07-03T15:35:00.895 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 bash[29129]: 7461f9bef4e0a95623bbd52ef076537cfa9eb65ae7b952d69e006ee86776c794 2024-07-03T15:35:00.895 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 systemd[1]: Started Ceph mon.smithi028 for a85e4be6-3951-11ef-bcab-c7b262605968. 2024-07-03T15:35:00.895 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: mon.smithi028 is new leader, mons smithi028 in quorum (ranks 0) 2024-07-03T15:35:00.895 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: monmap e1: 1 mons at {smithi028=[v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-07-03T15:35:00.895 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: fsmap 2024-07-03T15:35:00.896 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: osdmap e1: 0 total, 0 up, 0 in 2024-07-03T15:35:00.896 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:00 smithi028 ceph-mon[29148]: mgrmap e1: no daemons active 2024-07-03T15:35:01.861 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:01.323+0000 7f9915132640 1 Processor -- start 2024-07-03T15:35:01.861 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:01.323+0000 7f9915132640 1 -- start start 2024-07-03T15:35:01.861 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:01.323+0000 7f9915132640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9910101e00 0x7f9910102210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:01.861 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:01.323+0000 7f9915132640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9910102750 con 0x7f9910101e00 2024-07-03T15:35:01.861 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:01.323+0000 7f990ed9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9910101e00 0x7f9910102210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:01.862 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:01.323+0000 7f990ed9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9910101e00 0x7f9910102210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52646/0 (socket says 172.21.15.28:52646) 2024-07-03T15:35:01.862 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:01.323+0000 7f990ed9d640 1 -- 172.21.15.28:0/917876058 learned_addr learned my addr 172.21.15.28:0/917876058 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:01.862 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:01.324+0000 7f990ed9d640 1 -- 172.21.15.28:0/917876058 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9910102890 con 0x7f9910101e00 2024-07-03T15:35:01.862 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:01.325+0000 7f990ed9d640 1 --2- 172.21.15.28:0/917876058 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9910101e00 0x7f9910102210 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9900009840 tx=0x7f99000091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=904495a94c6b5f06 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:01.862 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:01.326+0000 7f990e59c640 1 -- 172.21.15.28:0/917876058 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9900032440 con 0x7f9910101e00 2024-07-03T15:35:01.862 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:01.326+0000 7f990e59c640 1 -- 172.21.15.28:0/917876058 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f9900032a30 con 0x7f9910101e00 2024-07-03T15:35:01.862 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:01.326+0000 7f990e59c640 1 -- 172.21.15.28:0/917876058 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9900031ad0 con 0x7f9910101e00 2024-07-03T15:35:01.862 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:01.326+0000 7f9915132640 1 -- 172.21.15.28:0/917876058 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9910101e00 msgr2=0x7f9910102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:01.862 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:01.326+0000 7f9915132640 1 --2- 172.21.15.28:0/917876058 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9910101e00 0x7f9910102210 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9900009840 tx=0x7f99000091d0 comp rx=0 tx=0).stop 2024-07-03T15:35:01.862 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:01.326+0000 7f9915132640 1 -- 172.21.15.28:0/917876058 shutdown_connections 2024-07-03T15:35:01.862 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:01.326+0000 7f9915132640 1 --2- 172.21.15.28:0/917876058 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9910101e00 0x7f9910102210 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:01.862 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:01.326+0000 7f9915132640 1 -- 172.21.15.28:0/917876058 >> 172.21.15.28:0/917876058 conn(0x7f99100fd5a0 msgr2=0x7f99100ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:01.862 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:01.326+0000 7f9915132640 1 -- 172.21.15.28:0/917876058 shutdown_connections 2024-07-03T15:35:01.862 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:01.326+0000 7f9915132640 1 -- 172.21.15.28:0/917876058 wait complete. 2024-07-03T15:35:01.863 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:01.327+0000 7f9915132640 1 Processor -- start 2024-07-03T15:35:01.863 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:01.327+0000 7f9915132640 1 -- start start 2024-07-03T15:35:01.863 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:01.327+0000 7f9915132640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9910101e00 0x7f9910110a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:01.863 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:01.327+0000 7f9915132640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9910110fd0 con 0x7f9910101e00 2024-07-03T15:35:01.863 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:01.327+0000 7f990ed9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9910101e00 0x7f9910110a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:01.863 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:01.327+0000 7f990ed9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9910101e00 0x7f9910110a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52648/0 (socket says 172.21.15.28:52648) 2024-07-03T15:35:01.863 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:01.327+0000 7f990ed9d640 1 -- 172.21.15.28:0/14098400 learned_addr learned my addr 172.21.15.28:0/14098400 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:01.863 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:01.328+0000 7f990ed9d640 1 -- 172.21.15.28:0/14098400 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9900009450 con 0x7f9910101e00 2024-07-03T15:35:01.863 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:01.328+0000 7f990ed9d640 1 --2- 172.21.15.28:0/14098400 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9910101e00 0x7f9910110a90 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9900009420 tx=0x7f9900008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:01.863 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:01.328+0000 7f99077fe640 1 -- 172.21.15.28:0/14098400 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f990003d5b0 con 0x7f9910101e00 2024-07-03T15:35:01.863 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:01.328+0000 7f9915132640 1 -- 172.21.15.28:0/14098400 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99101111d0 con 0x7f9910101e00 2024-07-03T15:35:01.863 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:01.328+0000 7f99077fe640 1 -- 172.21.15.28:0/14098400 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f990003dba0 con 0x7f9910101e00 2024-07-03T15:35:01.863 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:01.328+0000 7f99077fe640 1 -- 172.21.15.28:0/14098400 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f990003bb40 con 0x7f9910101e00 2024-07-03T15:35:01.863 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:01.328+0000 7f9915132640 1 -- 172.21.15.28:0/14098400 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9910111640 con 0x7f9910101e00 2024-07-03T15:35:01.864 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:01.328+0000 7f99077fe640 1 -- 172.21.15.28:0/14098400 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f9900039070 con 0x7f9910101e00 2024-07-03T15:35:01.864 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:01.328+0000 7f99077fe640 1 -- 172.21.15.28:0/14098400 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f9900046950 con 0x7f9910101e00 2024-07-03T15:35:01.864 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:01.329+0000 7f9915132640 1 -- 172.21.15.28:0/14098400 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f991004b530 con 0x7f9910101e00 2024-07-03T15:35:01.864 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:01.331+0000 7f99077fe640 1 -- 172.21.15.28:0/14098400 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f9910111640 con 0x7f9910101e00 2024-07-03T15:35:01.864 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:01.392+0000 7f9915132640 1 -- 172.21.15.28:0/14098400 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f9910103be0 con 0x7f9910101e00 2024-07-03T15:35:01.864 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:01.394+0000 7f99077fe640 1 -- 172.21.15.28:0/14098400 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f9900041020 con 0x7f9910101e00 2024-07-03T15:35:01.864 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:01.395+0000 7f9915132640 1 -- 172.21.15.28:0/14098400 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9910101e00 msgr2=0x7f9910110a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:01.864 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:01.395+0000 7f9915132640 1 --2- 172.21.15.28:0/14098400 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9910101e00 0x7f9910110a90 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9900009420 tx=0x7f9900008040 comp rx=0 tx=0).stop 2024-07-03T15:35:01.864 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:01.395+0000 7f9915132640 1 -- 172.21.15.28:0/14098400 shutdown_connections 2024-07-03T15:35:01.864 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:01.395+0000 7f9915132640 1 --2- 172.21.15.28:0/14098400 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9910101e00 0x7f9910110a90 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:01.864 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:01.395+0000 7f9915132640 1 -- 172.21.15.28:0/14098400 >> 172.21.15.28:0/14098400 conn(0x7f99100fd5a0 msgr2=0x7f99100fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:01.865 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:01.395+0000 7f9915132640 1 -- 172.21.15.28:0/14098400 shutdown_connections 2024-07-03T15:35:01.865 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:01.395+0000 7f9915132640 1 -- 172.21.15.28:0/14098400 wait complete. 2024-07-03T15:35:01.865 INFO:teuthology.orchestra.run.smithi028.stdout:Wrote config to /etc/ceph/ceph.conf 2024-07-03T15:35:01.865 INFO:teuthology.orchestra.run.smithi028.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-07-03T15:35:01.865 INFO:teuthology.orchestra.run.smithi028.stdout:Creating mgr... 2024-07-03T15:35:01.865 INFO:teuthology.orchestra.run.smithi028.stdout:Verifying port 9283 ... 2024-07-03T15:35:02.046 INFO:teuthology.orchestra.run.smithi028.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a85e4be6-3951-11ef-bcab-c7b262605968@mgr.smithi028.vtqrtz 2024-07-03T15:35:02.046 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a85e4be6-3951-11ef-bcab-c7b262605968@mgr.smithi028.vtqrtz.service: Unit ceph-a85e4be6-3951-11ef-bcab-c7b262605968@mgr.smithi028.vtqrtz.service not loaded. 2024-07-03T15:35:02.203 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a85e4be6-3951-11ef-bcab-c7b262605968.target.wants/ceph-a85e4be6-3951-11ef-bcab-c7b262605968@mgr.smithi028.vtqrtz.service → /etc/systemd/system/ceph-a85e4be6-3951-11ef-bcab-c7b262605968@.service. 2024-07-03T15:35:02.683 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:02 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/14098400' entity='client.admin' 2024-07-03T15:35:02.806 INFO:teuthology.orchestra.run.smithi028.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-07-03T15:35:02.806 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stdout disabled 2024-07-03T15:35:02.814 INFO:teuthology.orchestra.run.smithi028.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-07-03T15:35:02.814 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stdout inactive 2024-07-03T15:35:02.815 INFO:teuthology.orchestra.run.smithi028.stdout:firewalld.service is not enabled 2024-07-03T15:35:02.815 INFO:teuthology.orchestra.run.smithi028.stdout:Not possible to enable service . firewalld.service is not available 2024-07-03T15:35:02.824 INFO:teuthology.orchestra.run.smithi028.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-07-03T15:35:02.824 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stdout disabled 2024-07-03T15:35:02.832 INFO:teuthology.orchestra.run.smithi028.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-07-03T15:35:02.832 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stdout inactive 2024-07-03T15:35:02.832 INFO:teuthology.orchestra.run.smithi028.stdout:firewalld.service is not enabled 2024-07-03T15:35:02.832 INFO:teuthology.orchestra.run.smithi028.stdout:Not possible to open ports <[9283]>. firewalld.service is not available 2024-07-03T15:35:02.832 INFO:teuthology.orchestra.run.smithi028.stdout:Waiting for mgr to start... 2024-07-03T15:35:02.832 INFO:teuthology.orchestra.run.smithi028.stdout:Waiting for mgr... 2024-07-03T15:35:03.620 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:03 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/4097806270' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-07-03T15:35:03.869 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout 2024-07-03T15:35:03.869 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout { 2024-07-03T15:35:03.869 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "fsid": "a85e4be6-3951-11ef-bcab-c7b262605968", 2024-07-03T15:35:03.869 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "health": { 2024-07-03T15:35:03.869 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-07-03T15:35:03.869 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-07-03T15:35:03.869 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-07-03T15:35:03.869 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout }, 2024-07-03T15:35:03.869 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-07-03T15:35:03.869 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-07-03T15:35:03.870 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout 0 2024-07-03T15:35:03.870 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout ], 2024-07-03T15:35:03.870 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-07-03T15:35:03.870 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "smithi028" 2024-07-03T15:35:03.870 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout ], 2024-07-03T15:35:03.870 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-07-03T15:35:03.870 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "monmap": { 2024-07-03T15:35:03.870 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-03T15:35:03.870 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-07-03T15:35:03.870 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-07-03T15:35:03.870 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout }, 2024-07-03T15:35:03.870 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-07-03T15:35:03.870 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-03T15:35:03.871 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-07-03T15:35:03.871 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-07-03T15:35:03.871 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-07-03T15:35:03.878 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-07-03T15:35:03.878 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-07-03T15:35:03.878 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-07-03T15:35:03.878 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout }, 2024-07-03T15:35:03.878 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-07-03T15:35:03.878 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-07-03T15:35:03.879 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-07-03T15:35:03.879 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-07-03T15:35:03.879 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-07-03T15:35:03.879 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-07-03T15:35:03.879 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-07-03T15:35:03.879 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-07-03T15:35:03.879 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-07-03T15:35:03.879 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout }, 2024-07-03T15:35:03.879 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-07-03T15:35:03.879 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-03T15:35:03.879 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-07-03T15:35:03.879 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-07-03T15:35:03.879 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout }, 2024-07-03T15:35:03.879 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-07-03T15:35:03.879 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "available": false, 2024-07-03T15:35:03.880 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-07-03T15:35:03.880 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "modules": [ 2024-07-03T15:35:03.880 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "iostat", 2024-07-03T15:35:03.880 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "nfs", 2024-07-03T15:35:03.880 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "restful" 2024-07-03T15:35:03.880 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout ], 2024-07-03T15:35:03.880 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-03T15:35:03.880 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout }, 2024-07-03T15:35:03.880 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-07-03T15:35:03.880 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-03T15:35:03.880 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "modified": "2024-07-03T15:34:56.027426+0000", 2024-07-03T15:35:03.880 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-03T15:35:03.880 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout }, 2024-07-03T15:35:03.881 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-07-03T15:35:03.881 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout } 2024-07-03T15:35:03.881 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:03.229+0000 7fc3c90e7640 1 Processor -- start 2024-07-03T15:35:03.881 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:03.229+0000 7fc3c90e7640 1 -- start start 2024-07-03T15:35:03.881 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:03.229+0000 7fc3c90e7640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3c4101e00 0x7fc3c4102210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:03.881 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:03.229+0000 7fc3c90e7640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c4102750 con 0x7fc3c4101e00 2024-07-03T15:35:03.881 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:03.230+0000 7fc3c2d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3c4101e00 0x7fc3c4102210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:03.881 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:03.230+0000 7fc3c2d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3c4101e00 0x7fc3c4102210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52662/0 (socket says 172.21.15.28:52662) 2024-07-03T15:35:03.881 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:03.230+0000 7fc3c2d9d640 1 -- 172.21.15.28:0/246394867 learned_addr learned my addr 172.21.15.28:0/246394867 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:03.881 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:03.230+0000 7fc3c2d9d640 1 -- 172.21.15.28:0/246394867 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c4102890 con 0x7fc3c4101e00 2024-07-03T15:35:03.881 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:03.231+0000 7fc3c2d9d640 1 --2- 172.21.15.28:0/246394867 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3c4101e00 0x7fc3c4102210 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4009840 tx=0x7fc3b40091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2affc75b512e589 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:03.881 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:03.231+0000 7fc3c259c640 1 -- 172.21.15.28:0/246394867 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc3b4032440 con 0x7fc3c4101e00 2024-07-03T15:35:03.881 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:03.231+0000 7fc3c259c640 1 -- 172.21.15.28:0/246394867 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fc3b4032a30 con 0x7fc3c4101e00 2024-07-03T15:35:03.881 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:03.231+0000 7fc3c259c640 1 -- 172.21.15.28:0/246394867 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc3b4031ad0 con 0x7fc3c4101e00 2024-07-03T15:35:03.882 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:03.231+0000 7fc3c90e7640 1 -- 172.21.15.28:0/246394867 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3c4101e00 msgr2=0x7fc3c4102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:03.882 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:03.231+0000 7fc3c90e7640 1 --2- 172.21.15.28:0/246394867 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3c4101e00 0x7fc3c4102210 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4009840 tx=0x7fc3b40091d0 comp rx=0 tx=0).stop 2024-07-03T15:35:03.882 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:03.232+0000 7fc3c90e7640 1 -- 172.21.15.28:0/246394867 shutdown_connections 2024-07-03T15:35:03.882 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:03.232+0000 7fc3c90e7640 1 --2- 172.21.15.28:0/246394867 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3c4101e00 0x7fc3c4102210 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:03.882 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:03.232+0000 7fc3c90e7640 1 -- 172.21.15.28:0/246394867 >> 172.21.15.28:0/246394867 conn(0x7fc3c40fd5a0 msgr2=0x7fc3c40ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:03.882 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:03.232+0000 7fc3c90e7640 1 -- 172.21.15.28:0/246394867 shutdown_connections 2024-07-03T15:35:03.882 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:03.232+0000 7fc3c90e7640 1 -- 172.21.15.28:0/246394867 wait complete. 2024-07-03T15:35:03.882 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:03.232+0000 7fc3c90e7640 1 Processor -- start 2024-07-03T15:35:03.882 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:03.232+0000 7fc3c90e7640 1 -- start start 2024-07-03T15:35:03.882 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:03.232+0000 7fc3c90e7640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3c4101e00 0x7fc3c4077c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:03.882 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:03.232+0000 7fc3c90e7640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c4078160 con 0x7fc3c4101e00 2024-07-03T15:35:03.882 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:03.232+0000 7fc3c2d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3c4101e00 0x7fc3c4077c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:03.882 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:03.233+0000 7fc3c2d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3c4101e00 0x7fc3c4077c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52668/0 (socket says 172.21.15.28:52668) 2024-07-03T15:35:03.883 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:03.233+0000 7fc3c2d9d640 1 -- 172.21.15.28:0/4097806270 learned_addr learned my addr 172.21.15.28:0/4097806270 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:03.883 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:03.233+0000 7fc3c2d9d640 1 -- 172.21.15.28:0/4097806270 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b4009450 con 0x7fc3c4101e00 2024-07-03T15:35:03.883 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:03.233+0000 7fc3c2d9d640 1 --2- 172.21.15.28:0/4097806270 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3c4101e00 0x7fc3c4077c20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc3b40317b0 tx=0x7fc3b4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:03.883 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:03.233+0000 7fc3bb7fe640 1 -- 172.21.15.28:0/4097806270 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc3b4031e90 con 0x7fc3c4101e00 2024-07-03T15:35:03.883 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:03.233+0000 7fc3bb7fe640 1 -- 172.21.15.28:0/4097806270 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fc3b403daf0 con 0x7fc3c4101e00 2024-07-03T15:35:03.883 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:03.233+0000 7fc3c90e7640 1 -- 172.21.15.28:0/4097806270 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3c40761f0 con 0x7fc3c4101e00 2024-07-03T15:35:03.883 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:03.233+0000 7fc3bb7fe640 1 -- 172.21.15.28:0/4097806270 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc3b403baa0 con 0x7fc3c4101e00 2024-07-03T15:35:03.883 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:03.233+0000 7fc3c90e7640 1 -- 172.21.15.28:0/4097806270 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3c4076660 con 0x7fc3c4101e00 2024-07-03T15:35:03.883 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:03.233+0000 7fc3bb7fe640 1 -- 172.21.15.28:0/4097806270 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fc3b4039070 con 0x7fc3c4101e00 2024-07-03T15:35:03.883 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:03.233+0000 7fc3bb7fe640 1 -- 172.21.15.28:0/4097806270 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fc3b4046990 con 0x7fc3c4101e00 2024-07-03T15:35:03.883 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:03.234+0000 7fc3c90e7640 1 -- 172.21.15.28:0/4097806270 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3c404b530 con 0x7fc3c4101e00 2024-07-03T15:35:03.883 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:03.236+0000 7fc3bb7fe640 1 -- 172.21.15.28:0/4097806270 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7fc3b4041020 con 0x7fc3c4101e00 2024-07-03T15:35:03.883 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:03.301+0000 7fc3c90e7640 1 -- 172.21.15.28:0/4097806270 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fc3c4103be0 con 0x7fc3c4101e00 2024-07-03T15:35:03.883 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:03.301+0000 7fc3bb7fe640 1 -- 172.21.15.28:0/4097806270 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7fc3b4037070 con 0x7fc3c4101e00 2024-07-03T15:35:03.884 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:03.303+0000 7fc3c90e7640 1 -- 172.21.15.28:0/4097806270 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3c4101e00 msgr2=0x7fc3c4077c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:03.884 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:03.303+0000 7fc3c90e7640 1 --2- 172.21.15.28:0/4097806270 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3c4101e00 0x7fc3c4077c20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc3b40317b0 tx=0x7fc3b4008040 comp rx=0 tx=0).stop 2024-07-03T15:35:03.884 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:03.303+0000 7fc3c90e7640 1 -- 172.21.15.28:0/4097806270 shutdown_connections 2024-07-03T15:35:03.884 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:03.303+0000 7fc3c90e7640 1 --2- 172.21.15.28:0/4097806270 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3c4101e00 0x7fc3c4077c20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:03.884 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:03.303+0000 7fc3c90e7640 1 -- 172.21.15.28:0/4097806270 >> 172.21.15.28:0/4097806270 conn(0x7fc3c40fd5a0 msgr2=0x7fc3c40fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:03.884 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:03.303+0000 7fc3c90e7640 1 -- 172.21.15.28:0/4097806270 shutdown_connections 2024-07-03T15:35:03.884 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:03.303+0000 7fc3c90e7640 1 -- 172.21.15.28:0/4097806270 wait complete. 2024-07-03T15:35:03.884 INFO:teuthology.orchestra.run.smithi028.stdout:mgr not available, waiting (1/15)... 2024-07-03T15:35:06.620 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:06 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/3708894474' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-07-03T15:35:07.870 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:07 smithi028 ceph-mon[29148]: Activating manager daemon smithi028.vtqrtz 2024-07-03T15:35:07.870 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:07 smithi028 ceph-mon[29148]: mgrmap e2: smithi028.vtqrtz(active, starting, since 0.00282487s) 2024-07-03T15:35:07.870 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:07 smithi028 ceph-mon[29148]: from='mgr.14100 172.21.15.28:0/1785401248' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-03T15:35:07.870 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:07 smithi028 ceph-mon[29148]: from='mgr.14100 172.21.15.28:0/1785401248' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-03T15:35:07.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:07 smithi028 ceph-mon[29148]: from='mgr.14100 172.21.15.28:0/1785401248' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-03T15:35:07.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:07 smithi028 ceph-mon[29148]: from='mgr.14100 172.21.15.28:0/1785401248' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi028"}]: dispatch 2024-07-03T15:35:07.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:07 smithi028 ceph-mon[29148]: from='mgr.14100 172.21.15.28:0/1785401248' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mgr metadata", "who": "smithi028.vtqrtz", "id": "smithi028.vtqrtz"}]: dispatch 2024-07-03T15:35:07.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:07 smithi028 ceph-mon[29148]: Manager daemon smithi028.vtqrtz is now available 2024-07-03T15:35:07.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:07 smithi028 ceph-mon[29148]: from='mgr.14100 172.21.15.28:0/1785401248' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi028.vtqrtz/mirror_snapshot_schedule"}]: dispatch 2024-07-03T15:35:07.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:07 smithi028 ceph-mon[29148]: from='mgr.14100 172.21.15.28:0/1785401248' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi028.vtqrtz/trash_purge_schedule"}]: dispatch 2024-07-03T15:35:07.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:07 smithi028 ceph-mon[29148]: from='mgr.14100 172.21.15.28:0/1785401248' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:35:07.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:07 smithi028 ceph-mon[29148]: from='mgr.14100 172.21.15.28:0/1785401248' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:35:07.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:07 smithi028 ceph-mon[29148]: from='mgr.14100 172.21.15.28:0/1785401248' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:35:09.870 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:09 smithi028 ceph-mon[29148]: mgrmap e3: smithi028.vtqrtz(active, since 1.00732s) 2024-07-03T15:35:10.870 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:10 smithi028 ceph-mon[29148]: mgrmap e4: smithi028.vtqrtz(active, since 2s) 2024-07-03T15:35:12.730 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout 2024-07-03T15:35:12.730 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout { 2024-07-03T15:35:12.730 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "fsid": "a85e4be6-3951-11ef-bcab-c7b262605968", 2024-07-03T15:35:12.730 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "health": { 2024-07-03T15:35:12.730 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-07-03T15:35:12.730 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-07-03T15:35:12.730 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-07-03T15:35:12.731 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout }, 2024-07-03T15:35:12.731 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-07-03T15:35:12.731 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-07-03T15:35:12.731 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout 0 2024-07-03T15:35:12.731 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout ], 2024-07-03T15:35:12.731 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-07-03T15:35:12.731 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "smithi028" 2024-07-03T15:35:12.731 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout ], 2024-07-03T15:35:12.731 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-07-03T15:35:12.731 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "monmap": { 2024-07-03T15:35:12.731 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-03T15:35:12.731 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-07-03T15:35:12.731 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-07-03T15:35:12.731 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout }, 2024-07-03T15:35:12.732 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-07-03T15:35:12.732 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-03T15:35:12.732 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-07-03T15:35:12.732 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-07-03T15:35:12.732 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-07-03T15:35:12.739 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-07-03T15:35:12.739 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-07-03T15:35:12.739 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-07-03T15:35:12.739 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout }, 2024-07-03T15:35:12.739 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-07-03T15:35:12.739 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-07-03T15:35:12.740 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-07-03T15:35:12.740 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-07-03T15:35:12.740 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-07-03T15:35:12.740 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-07-03T15:35:12.740 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-07-03T15:35:12.740 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-07-03T15:35:12.740 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-07-03T15:35:12.740 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout }, 2024-07-03T15:35:12.740 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-07-03T15:35:12.740 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-03T15:35:12.740 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-07-03T15:35:12.740 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-07-03T15:35:12.740 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout }, 2024-07-03T15:35:12.741 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-07-03T15:35:12.741 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "available": false, 2024-07-03T15:35:12.741 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-07-03T15:35:12.741 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "modules": [ 2024-07-03T15:35:12.741 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "iostat", 2024-07-03T15:35:12.741 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "nfs", 2024-07-03T15:35:12.741 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "restful" 2024-07-03T15:35:12.741 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout ], 2024-07-03T15:35:12.741 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-03T15:35:12.741 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout }, 2024-07-03T15:35:12.741 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-07-03T15:35:12.741 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-03T15:35:12.741 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "modified": "2024-07-03T15:34:56.027426+0000", 2024-07-03T15:35:12.741 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-03T15:35:12.742 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout }, 2024-07-03T15:35:12.742 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-07-03T15:35:12.742 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout } 2024-07-03T15:35:12.742 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:06.400+0000 7efd03696640 1 Processor -- start 2024-07-03T15:35:12.742 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:06.400+0000 7efd03696640 1 -- start start 2024-07-03T15:35:12.742 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:06.400+0000 7efd03696640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcfc075b20 0x7efcfc104ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:12.742 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:06.400+0000 7efd03696640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc075f30 con 0x7efcfc075b20 2024-07-03T15:35:12.742 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:06.400+0000 7efd01432640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcfc075b20 0x7efcfc104ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:12.742 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:06.400+0000 7efd01432640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcfc075b20 0x7efcfc104ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52678/0 (socket says 172.21.15.28:52678) 2024-07-03T15:35:12.742 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:06.400+0000 7efd01432640 1 -- 172.21.15.28:0/89111333 learned_addr learned my addr 172.21.15.28:0/89111333 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:12.742 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:06.401+0000 7efd01432640 1 -- 172.21.15.28:0/89111333 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcfc104ff0 con 0x7efcfc075b20 2024-07-03T15:35:12.742 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:06.401+0000 7efd01432640 1 --2- 172.21.15.28:0/89111333 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcfc075b20 0x7efcfc104ab0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efcf8009840 tx=0x7efcf80091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c24dda01811320bb server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:12.742 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:06.402+0000 7efcf3fff640 1 -- 172.21.15.28:0/89111333 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efcf8032440 con 0x7efcfc075b20 2024-07-03T15:35:12.743 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:06.402+0000 7efcf3fff640 1 -- 172.21.15.28:0/89111333 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7efcf8032a30 con 0x7efcfc075b20 2024-07-03T15:35:12.743 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:06.402+0000 7efcf3fff640 1 -- 172.21.15.28:0/89111333 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efcf8031ad0 con 0x7efcfc075b20 2024-07-03T15:35:12.743 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:06.402+0000 7efd03696640 1 -- 172.21.15.28:0/89111333 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcfc075b20 msgr2=0x7efcfc104ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:12.743 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:06.402+0000 7efd03696640 1 --2- 172.21.15.28:0/89111333 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcfc075b20 0x7efcfc104ab0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efcf8009840 tx=0x7efcf80091d0 comp rx=0 tx=0).stop 2024-07-03T15:35:12.743 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:06.402+0000 7efd03696640 1 -- 172.21.15.28:0/89111333 shutdown_connections 2024-07-03T15:35:12.743 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:06.402+0000 7efd03696640 1 --2- 172.21.15.28:0/89111333 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcfc075b20 0x7efcfc104ab0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:12.743 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:06.402+0000 7efd03696640 1 -- 172.21.15.28:0/89111333 >> 172.21.15.28:0/89111333 conn(0x7efcfc0fc520 msgr2=0x7efcfc0fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:12.743 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:06.402+0000 7efd03696640 1 -- 172.21.15.28:0/89111333 shutdown_connections 2024-07-03T15:35:12.743 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:06.403+0000 7efd03696640 1 -- 172.21.15.28:0/89111333 wait complete. 2024-07-03T15:35:12.743 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:06.403+0000 7efd03696640 1 Processor -- start 2024-07-03T15:35:12.743 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:06.403+0000 7efd03696640 1 -- start start 2024-07-03T15:35:12.743 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:06.403+0000 7efd03696640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcfc075b20 0x7efcfc1023e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:12.743 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:06.403+0000 7efd03696640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc102920 con 0x7efcfc075b20 2024-07-03T15:35:12.744 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:06.403+0000 7efd01432640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcfc075b20 0x7efcfc1023e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:12.744 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:06.403+0000 7efd01432640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcfc075b20 0x7efcfc1023e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52692/0 (socket says 172.21.15.28:52692) 2024-07-03T15:35:12.744 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:06.403+0000 7efd01432640 1 -- 172.21.15.28:0/3708894474 learned_addr learned my addr 172.21.15.28:0/3708894474 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:12.744 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:06.404+0000 7efd01432640 1 -- 172.21.15.28:0/3708894474 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf8009450 con 0x7efcfc075b20 2024-07-03T15:35:12.744 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:06.404+0000 7efd01432640 1 --2- 172.21.15.28:0/3708894474 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcfc075b20 0x7efcfc1023e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efcf8006f00 tx=0x7efcf8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:12.744 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:06.404+0000 7efcf1ffb640 1 -- 172.21.15.28:0/3708894474 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efcf803d450 con 0x7efcfc075b20 2024-07-03T15:35:12.744 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:06.404+0000 7efd03696640 1 -- 172.21.15.28:0/3708894474 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcfc1009b0 con 0x7efcfc075b20 2024-07-03T15:35:12.744 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:06.404+0000 7efcf1ffb640 1 -- 172.21.15.28:0/3708894474 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7efcf803da40 con 0x7efcfc075b20 2024-07-03T15:35:12.744 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:06.404+0000 7efcf1ffb640 1 -- 172.21.15.28:0/3708894474 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efcf803bb10 con 0x7efcfc075b20 2024-07-03T15:35:12.744 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:06.404+0000 7efd03696640 1 -- 172.21.15.28:0/3708894474 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcfc100e20 con 0x7efcfc075b20 2024-07-03T15:35:12.744 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:06.404+0000 7efcf1ffb640 1 -- 172.21.15.28:0/3708894474 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7efcf8039070 con 0x7efcfc075b20 2024-07-03T15:35:12.744 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:06.405+0000 7efcf1ffb640 1 -- 172.21.15.28:0/3708894474 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7efcf8046930 con 0x7efcfc075b20 2024-07-03T15:35:12.744 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:06.405+0000 7efd03696640 1 -- 172.21.15.28:0/3708894474 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcfc04b530 con 0x7efcfc075b20 2024-07-03T15:35:12.745 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:06.409+0000 7efcf1ffb640 1 -- 172.21.15.28:0/3708894474 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7efcf8041020 con 0x7efcfc075b20 2024-07-03T15:35:12.745 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:06.473+0000 7efd03696640 1 -- 172.21.15.28:0/3708894474 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7efcfc105da0 con 0x7efcfc075b20 2024-07-03T15:35:12.745 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:06.473+0000 7efcf1ffb640 1 -- 172.21.15.28:0/3708894474 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7efcf8037070 con 0x7efcfc075b20 2024-07-03T15:35:12.745 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:06.474+0000 7efd03696640 1 -- 172.21.15.28:0/3708894474 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcfc075b20 msgr2=0x7efcfc1023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:12.745 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:06.474+0000 7efd03696640 1 --2- 172.21.15.28:0/3708894474 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcfc075b20 0x7efcfc1023e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efcf8006f00 tx=0x7efcf8008040 comp rx=0 tx=0).stop 2024-07-03T15:35:12.745 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:06.474+0000 7efd03696640 1 -- 172.21.15.28:0/3708894474 shutdown_connections 2024-07-03T15:35:12.745 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:06.474+0000 7efd03696640 1 --2- 172.21.15.28:0/3708894474 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcfc075b20 0x7efcfc1023e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:12.745 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:06.474+0000 7efd03696640 1 -- 172.21.15.28:0/3708894474 >> 172.21.15.28:0/3708894474 conn(0x7efcfc0fc520 msgr2=0x7efcfc0fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:12.745 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:06.474+0000 7efd03696640 1 -- 172.21.15.28:0/3708894474 shutdown_connections 2024-07-03T15:35:12.745 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:06.474+0000 7efd03696640 1 -- 172.21.15.28:0/3708894474 wait complete. 2024-07-03T15:35:12.745 INFO:teuthology.orchestra.run.smithi028.stdout:mgr not available, waiting (2/15)... 2024-07-03T15:35:15.706 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:15 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/1548356589' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-07-03T15:35:15.962 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout 2024-07-03T15:35:15.962 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout { 2024-07-03T15:35:15.962 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "fsid": "a85e4be6-3951-11ef-bcab-c7b262605968", 2024-07-03T15:35:15.962 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "health": { 2024-07-03T15:35:15.962 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-07-03T15:35:15.962 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-07-03T15:35:15.962 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-07-03T15:35:15.962 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout }, 2024-07-03T15:35:15.962 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-07-03T15:35:15.963 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-07-03T15:35:15.963 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout 0 2024-07-03T15:35:15.963 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout ], 2024-07-03T15:35:15.963 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-07-03T15:35:15.963 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "smithi028" 2024-07-03T15:35:15.963 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout ], 2024-07-03T15:35:15.963 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-07-03T15:35:15.963 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "monmap": { 2024-07-03T15:35:15.963 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-03T15:35:15.963 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-07-03T15:35:15.963 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-07-03T15:35:15.963 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout }, 2024-07-03T15:35:15.963 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-07-03T15:35:15.963 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-03T15:35:15.964 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-07-03T15:35:15.964 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-07-03T15:35:15.964 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-07-03T15:35:15.972 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-07-03T15:35:15.972 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-07-03T15:35:15.972 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-07-03T15:35:15.972 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout }, 2024-07-03T15:35:15.972 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-07-03T15:35:15.972 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-07-03T15:35:15.972 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-07-03T15:35:15.972 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-07-03T15:35:15.973 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-07-03T15:35:15.973 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-07-03T15:35:15.973 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-07-03T15:35:15.973 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-07-03T15:35:15.973 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-07-03T15:35:15.973 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout }, 2024-07-03T15:35:15.973 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-07-03T15:35:15.973 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-03T15:35:15.973 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-07-03T15:35:15.973 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-07-03T15:35:15.973 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout }, 2024-07-03T15:35:15.973 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-07-03T15:35:15.973 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "available": true, 2024-07-03T15:35:15.974 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-07-03T15:35:15.974 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "modules": [ 2024-07-03T15:35:15.974 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "iostat", 2024-07-03T15:35:15.974 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "nfs", 2024-07-03T15:35:15.974 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "restful" 2024-07-03T15:35:15.974 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout ], 2024-07-03T15:35:15.974 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-03T15:35:15.974 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout }, 2024-07-03T15:35:15.974 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-07-03T15:35:15.974 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-03T15:35:15.974 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "modified": "2024-07-03T15:34:56.027426+0000", 2024-07-03T15:35:15.974 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-03T15:35:15.974 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout }, 2024-07-03T15:35:15.975 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-07-03T15:35:15.975 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout } 2024-07-03T15:35:15.975 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.147+0000 7f71a3792640 1 Processor -- start 2024-07-03T15:35:15.975 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.148+0000 7f71a3792640 1 -- start start 2024-07-03T15:35:15.975 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.148+0000 7f71a3792640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f719c075c00 0x7f719c1049d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:15.975 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.148+0000 7f71a3792640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719c104f10 con 0x7f719c075c00 2024-07-03T15:35:15.975 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.148+0000 7f71a152e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f719c075c00 0x7f719c1049d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:15.975 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.148+0000 7f71a152e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f719c075c00 0x7f719c1049d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51806/0 (socket says 172.21.15.28:51806) 2024-07-03T15:35:15.975 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.148+0000 7f71a152e640 1 -- 172.21.15.28:0/797171888 learned_addr learned my addr 172.21.15.28:0/797171888 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:15.975 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.148+0000 7f71a152e640 1 -- 172.21.15.28:0/797171888 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f719c105050 con 0x7f719c075c00 2024-07-03T15:35:15.975 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.149+0000 7f71a152e640 1 --2- 172.21.15.28:0/797171888 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f719c075c00 0x7f719c1049d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7198009840 tx=0x7f71980091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d03d6f4da1ba3afe server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:15.976 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.149+0000 7f7193fff640 1 -- 172.21.15.28:0/797171888 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7198032440 con 0x7f719c075c00 2024-07-03T15:35:15.976 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.149+0000 7f7193fff640 1 -- 172.21.15.28:0/797171888 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f7198032a30 con 0x7f719c075c00 2024-07-03T15:35:15.976 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.149+0000 7f7193fff640 1 -- 172.21.15.28:0/797171888 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7198031ad0 con 0x7f719c075c00 2024-07-03T15:35:15.976 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.150+0000 7f71a3792640 1 -- 172.21.15.28:0/797171888 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f719c075c00 msgr2=0x7f719c1049d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:15.976 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.150+0000 7f71a3792640 1 --2- 172.21.15.28:0/797171888 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f719c075c00 0x7f719c1049d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7198009840 tx=0x7f71980091d0 comp rx=0 tx=0).stop 2024-07-03T15:35:15.976 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.150+0000 7f71a3792640 1 -- 172.21.15.28:0/797171888 shutdown_connections 2024-07-03T15:35:15.976 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.150+0000 7f71a3792640 1 --2- 172.21.15.28:0/797171888 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f719c075c00 0x7f719c1049d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:15.976 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.150+0000 7f71a3792640 1 -- 172.21.15.28:0/797171888 >> 172.21.15.28:0/797171888 conn(0x7f719c0fc520 msgr2=0x7f719c0fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:15.976 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.150+0000 7f71a3792640 1 -- 172.21.15.28:0/797171888 shutdown_connections 2024-07-03T15:35:15.976 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.150+0000 7f71a3792640 1 -- 172.21.15.28:0/797171888 wait complete. 2024-07-03T15:35:15.976 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.151+0000 7f71a3792640 1 Processor -- start 2024-07-03T15:35:15.976 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.151+0000 7f71a3792640 1 -- start start 2024-07-03T15:35:15.976 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.151+0000 7f71a3792640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f719c075c00 0x7f719c102300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:15.977 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.151+0000 7f71a3792640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719c102840 con 0x7f719c075c00 2024-07-03T15:35:15.977 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.151+0000 7f71a152e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f719c075c00 0x7f719c102300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:15.977 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.151+0000 7f71a152e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f719c075c00 0x7f719c102300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51822/0 (socket says 172.21.15.28:51822) 2024-07-03T15:35:15.977 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.151+0000 7f71a152e640 1 -- 172.21.15.28:0/1548356589 learned_addr learned my addr 172.21.15.28:0/1548356589 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:15.977 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.151+0000 7f71a152e640 1 -- 172.21.15.28:0/1548356589 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7198009450 con 0x7f719c075c00 2024-07-03T15:35:15.977 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.151+0000 7f71a152e640 1 --2- 172.21.15.28:0/1548356589 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f719c075c00 0x7f719c102300 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7198031b90 tx=0x7f7198008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:15.977 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.152+0000 7f7191ffb640 1 -- 172.21.15.28:0/1548356589 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f71980328b0 con 0x7f719c075c00 2024-07-03T15:35:15.977 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.152+0000 7f71a3792640 1 -- 172.21.15.28:0/1548356589 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f719c1008d0 con 0x7f719c075c00 2024-07-03T15:35:15.977 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.152+0000 7f7191ffb640 1 -- 172.21.15.28:0/1548356589 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f719803d8e0 con 0x7f719c075c00 2024-07-03T15:35:15.977 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.152+0000 7f7191ffb640 1 -- 172.21.15.28:0/1548356589 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f719803ba20 con 0x7f719c075c00 2024-07-03T15:35:15.977 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.152+0000 7f71a3792640 1 -- 172.21.15.28:0/1548356589 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f719c100d40 con 0x7f719c075c00 2024-07-03T15:35:15.977 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.153+0000 7f7191ffb640 1 -- 172.21.15.28:0/1548356589 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 4) v1 ==== 42625+0+0 (secure 0 0 0) 0x7f7198039070 con 0x7f719c075c00 2024-07-03T15:35:15.977 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.153+0000 7f7191ffb640 1 --2- 172.21.15.28:0/1548356589 >> [v2:172.21.15.28:6800/1358048352,v1:172.21.15.28:6801/1358048352] conn(0x7f7188035730 0x7f7188037be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:15.978 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.153+0000 7f71a3792640 1 -- 172.21.15.28:0/1548356589 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f719c04b530 con 0x7f719c075c00 2024-07-03T15:35:15.978 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.153+0000 7f7191ffb640 1 -- 172.21.15.28:0/1548356589 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f71980685e0 con 0x7f719c075c00 2024-07-03T15:35:15.978 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.153+0000 7f71a0d2d640 1 --2- 172.21.15.28:0/1548356589 >> [v2:172.21.15.28:6800/1358048352,v1:172.21.15.28:6801/1358048352] conn(0x7f7188035730 0x7f7188037be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:15.978 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.154+0000 7f71a0d2d640 1 --2- 172.21.15.28:0/1548356589 >> [v2:172.21.15.28:6800/1358048352,v1:172.21.15.28:6801/1358048352] conn(0x7f7188035730 0x7f7188037be0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f718c006fd0 tx=0x7f718c006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:15.978 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.158+0000 7f7191ffb640 1 -- 172.21.15.28:0/1548356589 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7198046400 con 0x7f719c075c00 2024-07-03T15:35:15.978 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.398+0000 7f71a3792640 1 -- 172.21.15.28:0/1548356589 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f719c105dd0 con 0x7f719c075c00 2024-07-03T15:35:15.978 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.398+0000 7f7191ffb640 1 -- 172.21.15.28:0/1548356589 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f7198041030 con 0x7f719c075c00 2024-07-03T15:35:15.978 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.401+0000 7f71a3792640 1 -- 172.21.15.28:0/1548356589 >> [v2:172.21.15.28:6800/1358048352,v1:172.21.15.28:6801/1358048352] conn(0x7f7188035730 msgr2=0x7f7188037be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:15.978 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.401+0000 7f71a3792640 1 --2- 172.21.15.28:0/1548356589 >> [v2:172.21.15.28:6800/1358048352,v1:172.21.15.28:6801/1358048352] conn(0x7f7188035730 0x7f7188037be0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f718c006fd0 tx=0x7f718c006db0 comp rx=0 tx=0).stop 2024-07-03T15:35:15.978 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.401+0000 7f71a3792640 1 -- 172.21.15.28:0/1548356589 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f719c075c00 msgr2=0x7f719c102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:15.978 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.401+0000 7f71a3792640 1 --2- 172.21.15.28:0/1548356589 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f719c075c00 0x7f719c102300 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7198031b90 tx=0x7f7198008040 comp rx=0 tx=0).stop 2024-07-03T15:35:15.978 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.401+0000 7f71a3792640 1 -- 172.21.15.28:0/1548356589 shutdown_connections 2024-07-03T15:35:15.979 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.401+0000 7f71a3792640 1 --2- 172.21.15.28:0/1548356589 >> [v2:172.21.15.28:6800/1358048352,v1:172.21.15.28:6801/1358048352] conn(0x7f7188035730 0x7f7188037be0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:15.979 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.401+0000 7f71a3792640 1 --2- 172.21.15.28:0/1548356589 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f719c075c00 0x7f719c102300 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:15.979 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.401+0000 7f71a3792640 1 -- 172.21.15.28:0/1548356589 >> 172.21.15.28:0/1548356589 conn(0x7f719c0fc520 msgr2=0x7f719c0fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:15.979 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.401+0000 7f71a3792640 1 -- 172.21.15.28:0/1548356589 shutdown_connections 2024-07-03T15:35:15.979 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:15.401+0000 7f71a3792640 1 -- 172.21.15.28:0/1548356589 wait complete. 2024-07-03T15:35:15.979 INFO:teuthology.orchestra.run.smithi028.stdout:mgr is available 2024-07-03T15:35:16.870 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:16 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/4231324912' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-07-03T15:35:17.189 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout 2024-07-03T15:35:17.189 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout [global] 2024-07-03T15:35:17.189 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout fsid = a85e4be6-3951-11ef-bcab-c7b262605968 2024-07-03T15:35:17.189 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-07-03T15:35:17.189 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-07-03T15:35:17.189 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-07-03T15:35:17.189 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-07-03T15:35:17.190 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout 2024-07-03T15:35:17.190 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout [mgr] 2024-07-03T15:35:17.190 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-07-03T15:35:17.190 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout 2024-07-03T15:35:17.190 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout [osd] 2024-07-03T15:35:17.190 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-07-03T15:35:17.190 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-07-03T15:35:17.190 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-07-03T15:35:17.190 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-07-03T15:35:17.190 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-07-03T15:35:17.190 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.338+0000 7f42b5812640 1 Processor -- start 2024-07-03T15:35:17.190 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.338+0000 7f42b5812640 1 -- start start 2024-07-03T15:35:17.190 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.338+0000 7f42b5812640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42b0101e00 0x7f42b0102210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:17.190 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.338+0000 7f42b5812640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b0102750 con 0x7f42b0101e00 2024-07-03T15:35:17.191 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.338+0000 7f42aeffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42b0101e00 0x7f42b0102210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:17.191 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.338+0000 7f42aeffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42b0101e00 0x7f42b0102210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51824/0 (socket says 172.21.15.28:51824) 2024-07-03T15:35:17.191 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.338+0000 7f42aeffd640 1 -- 172.21.15.28:0/3883508571 learned_addr learned my addr 172.21.15.28:0/3883508571 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:17.191 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.338+0000 7f42aeffd640 1 -- 172.21.15.28:0/3883508571 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42b0102890 con 0x7f42b0101e00 2024-07-03T15:35:17.191 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.339+0000 7f42aeffd640 1 --2- 172.21.15.28:0/3883508571 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42b0101e00 0x7f42b0102210 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f42a0009840 tx=0x7f42a00091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da37c0419b834cc server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:17.191 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.339+0000 7f42ae7fc640 1 -- 172.21.15.28:0/3883508571 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f42a0032440 con 0x7f42b0101e00 2024-07-03T15:35:17.191 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.339+0000 7f42ae7fc640 1 -- 172.21.15.28:0/3883508571 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f42a0032a30 con 0x7f42b0101e00 2024-07-03T15:35:17.191 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.339+0000 7f42ae7fc640 1 -- 172.21.15.28:0/3883508571 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f42a0031ad0 con 0x7f42b0101e00 2024-07-03T15:35:17.191 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.340+0000 7f42b5812640 1 -- 172.21.15.28:0/3883508571 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42b0101e00 msgr2=0x7f42b0102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:17.191 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.340+0000 7f42b5812640 1 --2- 172.21.15.28:0/3883508571 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42b0101e00 0x7f42b0102210 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f42a0009840 tx=0x7f42a00091d0 comp rx=0 tx=0).stop 2024-07-03T15:35:17.191 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.340+0000 7f42b5812640 1 -- 172.21.15.28:0/3883508571 shutdown_connections 2024-07-03T15:35:17.191 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.340+0000 7f42b5812640 1 --2- 172.21.15.28:0/3883508571 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42b0101e00 0x7f42b0102210 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:17.191 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.340+0000 7f42b5812640 1 -- 172.21.15.28:0/3883508571 >> 172.21.15.28:0/3883508571 conn(0x7f42b00fd5a0 msgr2=0x7f42b00ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:17.191 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.340+0000 7f42b5812640 1 -- 172.21.15.28:0/3883508571 shutdown_connections 2024-07-03T15:35:17.192 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.340+0000 7f42b5812640 1 -- 172.21.15.28:0/3883508571 wait complete. 2024-07-03T15:35:17.192 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.340+0000 7f42b5812640 1 Processor -- start 2024-07-03T15:35:17.192 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.340+0000 7f42b5812640 1 -- start start 2024-07-03T15:35:17.192 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.341+0000 7f42b5812640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42b0101e00 0x7f42b0077c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:17.192 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.341+0000 7f42b5812640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b0078160 con 0x7f42b0101e00 2024-07-03T15:35:17.192 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.341+0000 7f42aeffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42b0101e00 0x7f42b0077c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:17.192 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.341+0000 7f42aeffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42b0101e00 0x7f42b0077c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51840/0 (socket says 172.21.15.28:51840) 2024-07-03T15:35:17.192 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.341+0000 7f42aeffd640 1 -- 172.21.15.28:0/4231324912 learned_addr learned my addr 172.21.15.28:0/4231324912 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:17.192 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.341+0000 7f42aeffd640 1 -- 172.21.15.28:0/4231324912 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a0009450 con 0x7f42b0101e00 2024-07-03T15:35:17.192 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.341+0000 7f42aeffd640 1 --2- 172.21.15.28:0/4231324912 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42b0101e00 0x7f42b0077c20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f42a00317b0 tx=0x7f42a0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:17.192 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.341+0000 7f42a77fe640 1 -- 172.21.15.28:0/4231324912 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f42a0031e90 con 0x7f42b0101e00 2024-07-03T15:35:17.192 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.341+0000 7f42a77fe640 1 -- 172.21.15.28:0/4231324912 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f42a003daf0 con 0x7f42b0101e00 2024-07-03T15:35:17.192 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.341+0000 7f42b5812640 1 -- 172.21.15.28:0/4231324912 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42b00761f0 con 0x7f42b0101e00 2024-07-03T15:35:17.192 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.341+0000 7f42a77fe640 1 -- 172.21.15.28:0/4231324912 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f42a003baa0 con 0x7f42b0101e00 2024-07-03T15:35:17.192 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.342+0000 7f42b5812640 1 -- 172.21.15.28:0/4231324912 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42b0076660 con 0x7f42b0101e00 2024-07-03T15:35:17.193 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.342+0000 7f42a77fe640 1 -- 172.21.15.28:0/4231324912 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 4) v1 ==== 42625+0+0 (secure 0 0 0) 0x7f42a0039070 con 0x7f42b0101e00 2024-07-03T15:35:17.193 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.343+0000 7f42a77fe640 1 --2- 172.21.15.28:0/4231324912 >> [v2:172.21.15.28:6800/1358048352,v1:172.21.15.28:6801/1358048352] conn(0x7f429c035730 0x7f429c037be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:17.193 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.343+0000 7f42b5812640 1 -- 172.21.15.28:0/4231324912 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42b004b530 con 0x7f42b0101e00 2024-07-03T15:35:17.193 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.343+0000 7f42a77fe640 1 -- 172.21.15.28:0/4231324912 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f42a0068950 con 0x7f42b0101e00 2024-07-03T15:35:17.193 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.343+0000 7f42a7fff640 1 --2- 172.21.15.28:0/4231324912 >> [v2:172.21.15.28:6800/1358048352,v1:172.21.15.28:6801/1358048352] conn(0x7f429c035730 0x7f429c037be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:17.193 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.343+0000 7f42a7fff640 1 --2- 172.21.15.28:0/4231324912 >> [v2:172.21.15.28:6800/1358048352,v1:172.21.15.28:6801/1358048352] conn(0x7f429c035730 0x7f429c037be0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4298006fd0 tx=0x7f4298006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:17.193 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.347+0000 7f42a77fe640 1 -- 172.21.15.28:0/4231324912 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f42a0039320 con 0x7f42b0101e00 2024-07-03T15:35:17.193 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.523+0000 7f42b5812640 1 -- 172.21.15.28:0/4231324912 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f42b0076960 con 0x7f42b0101e00 2024-07-03T15:35:17.193 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.524+0000 7f42a77fe640 1 -- 172.21.15.28:0/4231324912 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f42a0041030 con 0x7f42b0101e00 2024-07-03T15:35:17.193 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.525+0000 7f42b5812640 1 -- 172.21.15.28:0/4231324912 >> [v2:172.21.15.28:6800/1358048352,v1:172.21.15.28:6801/1358048352] conn(0x7f429c035730 msgr2=0x7f429c037be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:17.193 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.525+0000 7f42b5812640 1 --2- 172.21.15.28:0/4231324912 >> [v2:172.21.15.28:6800/1358048352,v1:172.21.15.28:6801/1358048352] conn(0x7f429c035730 0x7f429c037be0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4298006fd0 tx=0x7f4298006db0 comp rx=0 tx=0).stop 2024-07-03T15:35:17.193 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.525+0000 7f42b5812640 1 -- 172.21.15.28:0/4231324912 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42b0101e00 msgr2=0x7f42b0077c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:17.193 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.525+0000 7f42b5812640 1 --2- 172.21.15.28:0/4231324912 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42b0101e00 0x7f42b0077c20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f42a00317b0 tx=0x7f42a0008040 comp rx=0 tx=0).stop 2024-07-03T15:35:17.193 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.526+0000 7f42b5812640 1 -- 172.21.15.28:0/4231324912 shutdown_connections 2024-07-03T15:35:17.194 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.526+0000 7f42b5812640 1 --2- 172.21.15.28:0/4231324912 >> [v2:172.21.15.28:6800/1358048352,v1:172.21.15.28:6801/1358048352] conn(0x7f429c035730 0x7f429c037be0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:17.194 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.526+0000 7f42b5812640 1 --2- 172.21.15.28:0/4231324912 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42b0101e00 0x7f42b0077c20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:17.194 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.526+0000 7f42b5812640 1 -- 172.21.15.28:0/4231324912 >> 172.21.15.28:0/4231324912 conn(0x7f42b00fd5a0 msgr2=0x7f42b00fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:17.194 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.526+0000 7f42b5812640 1 -- 172.21.15.28:0/4231324912 shutdown_connections 2024-07-03T15:35:17.194 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:16.526+0000 7f42b5812640 1 -- 172.21.15.28:0/4231324912 wait complete. 2024-07-03T15:35:17.194 INFO:teuthology.orchestra.run.smithi028.stdout:Enabling cephadm module... 2024-07-03T15:35:18.212 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:17 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/2174994004' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-07-03T15:35:19.370 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:18 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/2174994004' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-07-03T15:35:19.370 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:18 smithi028 ceph-mon[29148]: mgrmap e5: smithi028.vtqrtz(active, since 10s) 2024-07-03T15:35:21.554 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.618+0000 7f3453c2b640 1 Processor -- start 2024-07-03T15:35:21.554 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.618+0000 7f3453c2b640 1 -- start start 2024-07-03T15:35:21.554 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.619+0000 7f3453c2b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f344c075ba0 0x7f344c106e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:21.554 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.619+0000 7f3453c2b640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344c102820 con 0x7f344c075ba0 2024-07-03T15:35:21.554 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.619+0000 7f34519c7640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f344c075ba0 0x7f344c106e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:21.554 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.619+0000 7f34519c7640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f344c075ba0 0x7f344c106e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51844/0 (socket says 172.21.15.28:51844) 2024-07-03T15:35:21.554 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.619+0000 7f34519c7640 1 -- 172.21.15.28:0/833286715 learned_addr learned my addr 172.21.15.28:0/833286715 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:21.554 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.619+0000 7f34519c7640 1 -- 172.21.15.28:0/833286715 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f344c107370 con 0x7f344c075ba0 2024-07-03T15:35:21.554 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.620+0000 7f34519c7640 1 --2- 172.21.15.28:0/833286715 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f344c075ba0 0x7f344c106e30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3448009840 tx=0x7f34480091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=625c57ba2f9736e7 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:21.554 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.620+0000 7f34509c5640 1 -- 172.21.15.28:0/833286715 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3448032440 con 0x7f344c075ba0 2024-07-03T15:35:21.555 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.620+0000 7f34509c5640 1 -- 172.21.15.28:0/833286715 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f3448032a30 con 0x7f344c075ba0 2024-07-03T15:35:21.555 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.620+0000 7f34509c5640 1 -- 172.21.15.28:0/833286715 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3448031ad0 con 0x7f344c075ba0 2024-07-03T15:35:21.555 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.621+0000 7f3453c2b640 1 -- 172.21.15.28:0/833286715 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f344c075ba0 msgr2=0x7f344c106e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:21.555 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.621+0000 7f3453c2b640 1 --2- 172.21.15.28:0/833286715 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f344c075ba0 0x7f344c106e30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3448009840 tx=0x7f34480091d0 comp rx=0 tx=0).stop 2024-07-03T15:35:21.555 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.621+0000 7f3453c2b640 1 -- 172.21.15.28:0/833286715 shutdown_connections 2024-07-03T15:35:21.555 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.621+0000 7f3453c2b640 1 --2- 172.21.15.28:0/833286715 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f344c075ba0 0x7f344c106e30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:21.555 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.621+0000 7f3453c2b640 1 -- 172.21.15.28:0/833286715 >> 172.21.15.28:0/833286715 conn(0x7f344c0fc520 msgr2=0x7f344c0fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:21.555 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.621+0000 7f3453c2b640 1 -- 172.21.15.28:0/833286715 shutdown_connections 2024-07-03T15:35:21.555 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.621+0000 7f3453c2b640 1 -- 172.21.15.28:0/833286715 wait complete. 2024-07-03T15:35:21.555 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.622+0000 7f3453c2b640 1 Processor -- start 2024-07-03T15:35:21.555 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.622+0000 7f3453c2b640 1 -- start start 2024-07-03T15:35:21.555 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.622+0000 7f3453c2b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f344c075ba0 0x7f344c112c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:21.555 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.622+0000 7f3453c2b640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344c1131b0 con 0x7f344c075ba0 2024-07-03T15:35:21.555 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.622+0000 7f34519c7640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f344c075ba0 0x7f344c112c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:21.556 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.622+0000 7f34519c7640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f344c075ba0 0x7f344c112c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51852/0 (socket says 172.21.15.28:51852) 2024-07-03T15:35:21.556 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.622+0000 7f34519c7640 1 -- 172.21.15.28:0/2174994004 learned_addr learned my addr 172.21.15.28:0/2174994004 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:21.556 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.622+0000 7f34519c7640 1 -- 172.21.15.28:0/2174994004 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3448009450 con 0x7f344c075ba0 2024-07-03T15:35:21.556 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.623+0000 7f34519c7640 1 --2- 172.21.15.28:0/2174994004 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f344c075ba0 0x7f344c112c70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3448031b90 tx=0x7f3448008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:21.556 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.623+0000 7f343e7fc640 1 -- 172.21.15.28:0/2174994004 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f34480328b0 con 0x7f344c075ba0 2024-07-03T15:35:21.556 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.623+0000 7f343e7fc640 1 -- 172.21.15.28:0/2174994004 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f344803d8e0 con 0x7f344c075ba0 2024-07-03T15:35:21.556 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.623+0000 7f3453c2b640 1 -- 172.21.15.28:0/2174994004 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f344c1133b0 con 0x7f344c075ba0 2024-07-03T15:35:21.556 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.623+0000 7f343e7fc640 1 -- 172.21.15.28:0/2174994004 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f344803ba20 con 0x7f344c075ba0 2024-07-03T15:35:21.556 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.623+0000 7f3453c2b640 1 -- 172.21.15.28:0/2174994004 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f344c113820 con 0x7f344c075ba0 2024-07-03T15:35:21.556 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.624+0000 7f343e7fc640 1 -- 172.21.15.28:0/2174994004 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 4) v1 ==== 42625+0+0 (secure 0 0 0) 0x7f3448039070 con 0x7f344c075ba0 2024-07-03T15:35:21.556 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.624+0000 7f3453c2b640 1 -- 172.21.15.28:0/2174994004 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f344c04b530 con 0x7f344c075ba0 2024-07-03T15:35:21.556 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.624+0000 7f343e7fc640 1 --2- 172.21.15.28:0/2174994004 >> [v2:172.21.15.28:6800/1358048352,v1:172.21.15.28:6801/1358048352] conn(0x7f3438035780 0x7f3438037c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:21.556 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.624+0000 7f343e7fc640 1 -- 172.21.15.28:0/2174994004 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f3448068440 con 0x7f344c075ba0 2024-07-03T15:35:21.557 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.624+0000 7f34511c6640 1 --2- 172.21.15.28:0/2174994004 >> [v2:172.21.15.28:6800/1358048352,v1:172.21.15.28:6801/1358048352] conn(0x7f3438035780 0x7f3438037c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:21.557 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.625+0000 7f34511c6640 1 --2- 172.21.15.28:0/2174994004 >> [v2:172.21.15.28:6800/1358048352,v1:172.21.15.28:6801/1358048352] conn(0x7f3438035780 0x7f3438037c30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3440006fd0 tx=0x7f3440006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:21.557 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.629+0000 7f343e7fc640 1 -- 172.21.15.28:0/2174994004 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f344804e940 con 0x7f344c075ba0 2024-07-03T15:35:21.557 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.849+0000 7f3453c2b640 1 -- 172.21.15.28:0/2174994004 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f344c107fa0 con 0x7f344c075ba0 2024-07-03T15:35:21.557 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.902+0000 7f343e7fc640 1 -- 172.21.15.28:0/2174994004 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mgrmap(e 5) v1 ==== 42636+0+0 (secure 0 0 0) 0x7f34480464f0 con 0x7f344c075ba0 2024-07-03T15:35:21.557 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.903+0000 7f343e7fc640 1 -- 172.21.15.28:0/2174994004 <== mon.0 v2:172.21.15.28:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f34480681a0 con 0x7f344c075ba0 2024-07-03T15:35:21.557 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.906+0000 7f3453c2b640 1 -- 172.21.15.28:0/2174994004 >> [v2:172.21.15.28:6800/1358048352,v1:172.21.15.28:6801/1358048352] conn(0x7f3438035780 msgr2=0x7f3438037c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:21.557 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.906+0000 7f3453c2b640 1 --2- 172.21.15.28:0/2174994004 >> [v2:172.21.15.28:6800/1358048352,v1:172.21.15.28:6801/1358048352] conn(0x7f3438035780 0x7f3438037c30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3440006fd0 tx=0x7f3440006db0 comp rx=0 tx=0).stop 2024-07-03T15:35:21.557 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.907+0000 7f3453c2b640 1 -- 172.21.15.28:0/2174994004 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f344c075ba0 msgr2=0x7f344c112c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:21.557 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.907+0000 7f3453c2b640 1 --2- 172.21.15.28:0/2174994004 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f344c075ba0 0x7f344c112c70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3448031b90 tx=0x7f3448008040 comp rx=0 tx=0).stop 2024-07-03T15:35:21.557 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.907+0000 7f3453c2b640 1 -- 172.21.15.28:0/2174994004 shutdown_connections 2024-07-03T15:35:21.557 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.907+0000 7f3453c2b640 1 --2- 172.21.15.28:0/2174994004 >> [v2:172.21.15.28:6800/1358048352,v1:172.21.15.28:6801/1358048352] conn(0x7f3438035780 0x7f3438037c30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:21.557 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.907+0000 7f3453c2b640 1 --2- 172.21.15.28:0/2174994004 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f344c075ba0 0x7f344c112c70 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:21.557 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.907+0000 7f3453c2b640 1 -- 172.21.15.28:0/2174994004 >> 172.21.15.28:0/2174994004 conn(0x7f344c0fc520 msgr2=0x7f344c0fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:21.558 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.907+0000 7f3453c2b640 1 -- 172.21.15.28:0/2174994004 shutdown_connections 2024-07-03T15:35:21.558 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:17.907+0000 7f3453c2b640 1 -- 172.21.15.28:0/2174994004 wait complete. 2024-07-03T15:35:22.457 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:22 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/2639297760' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-07-03T15:35:22.777 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout { 2024-07-03T15:35:22.777 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-07-03T15:35:22.778 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "available": true, 2024-07-03T15:35:22.778 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "active_name": "smithi028.vtqrtz", 2024-07-03T15:35:22.778 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-07-03T15:35:22.778 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout } 2024-07-03T15:35:22.778 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:21.959+0000 7f63d904c640 1 Processor -- start 2024-07-03T15:35:22.778 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:21.960+0000 7f63d904c640 1 -- start start 2024-07-03T15:35:22.778 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:21.960+0000 7f63d904c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f63d4101e00 0x7f63d4102210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:22.778 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:21.960+0000 7f63d904c640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d4102750 con 0x7f63d4101e00 2024-07-03T15:35:22.778 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:21.960+0000 7f63d2d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f63d4101e00 0x7f63d4102210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:22.778 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:21.960+0000 7f63d2d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f63d4101e00 0x7f63d4102210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48610/0 (socket says 172.21.15.28:48610) 2024-07-03T15:35:22.778 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:21.960+0000 7f63d2d9d640 1 -- 172.21.15.28:0/2555257176 learned_addr learned my addr 172.21.15.28:0/2555257176 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:22.778 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:21.961+0000 7f63d2d9d640 1 -- 172.21.15.28:0/2555257176 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d4102890 con 0x7f63d4101e00 2024-07-03T15:35:22.779 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:21.961+0000 7f63d2d9d640 1 --2- 172.21.15.28:0/2555257176 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f63d4101e00 0x7f63d4102210 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f63c4009840 tx=0x7f63c40091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba7cdec9ce1c90df server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:22.779 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:21.961+0000 7f63d259c640 1 -- 172.21.15.28:0/2555257176 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f63c4032440 con 0x7f63d4101e00 2024-07-03T15:35:22.779 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:21.961+0000 7f63d259c640 1 -- 172.21.15.28:0/2555257176 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f63c4032a30 con 0x7f63d4101e00 2024-07-03T15:35:22.779 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:21.961+0000 7f63d259c640 1 -- 172.21.15.28:0/2555257176 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f63c4031ad0 con 0x7f63d4101e00 2024-07-03T15:35:22.779 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:21.962+0000 7f63d904c640 1 -- 172.21.15.28:0/2555257176 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f63d4101e00 msgr2=0x7f63d4102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:22.779 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:21.962+0000 7f63d904c640 1 --2- 172.21.15.28:0/2555257176 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f63d4101e00 0x7f63d4102210 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f63c4009840 tx=0x7f63c40091d0 comp rx=0 tx=0).stop 2024-07-03T15:35:22.779 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:21.962+0000 7f63d904c640 1 -- 172.21.15.28:0/2555257176 shutdown_connections 2024-07-03T15:35:22.779 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:21.962+0000 7f63d904c640 1 --2- 172.21.15.28:0/2555257176 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f63d4101e00 0x7f63d4102210 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:22.779 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:21.962+0000 7f63d904c640 1 -- 172.21.15.28:0/2555257176 >> 172.21.15.28:0/2555257176 conn(0x7f63d40fd5a0 msgr2=0x7f63d40ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:22.779 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:21.962+0000 7f63d904c640 1 -- 172.21.15.28:0/2555257176 shutdown_connections 2024-07-03T15:35:22.779 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:21.962+0000 7f63d904c640 1 -- 172.21.15.28:0/2555257176 wait complete. 2024-07-03T15:35:22.779 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:21.963+0000 7f63d904c640 1 Processor -- start 2024-07-03T15:35:22.779 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:21.963+0000 7f63d904c640 1 -- start start 2024-07-03T15:35:22.779 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:21.963+0000 7f63d904c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f63d4101e00 0x7f63d4077c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:22.780 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:21.963+0000 7f63d904c640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d4078160 con 0x7f63d4101e00 2024-07-03T15:35:22.780 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:21.963+0000 7f63d2d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f63d4101e00 0x7f63d4077c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:22.780 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:21.963+0000 7f63d2d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f63d4101e00 0x7f63d4077c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48624/0 (socket says 172.21.15.28:48624) 2024-07-03T15:35:22.780 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:21.963+0000 7f63d2d9d640 1 -- 172.21.15.28:0/2639297760 learned_addr learned my addr 172.21.15.28:0/2639297760 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:22.780 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:21.963+0000 7f63d2d9d640 1 -- 172.21.15.28:0/2639297760 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c4009450 con 0x7f63d4101e00 2024-07-03T15:35:22.780 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:21.964+0000 7f63d2d9d640 1 --2- 172.21.15.28:0/2639297760 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f63d4101e00 0x7f63d4077c20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f63c4009420 tx=0x7f63c4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:22.780 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:21.964+0000 7f63cb7fe640 1 -- 172.21.15.28:0/2639297760 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f63c403d5b0 con 0x7f63d4101e00 2024-07-03T15:35:22.780 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:21.964+0000 7f63cb7fe640 1 -- 172.21.15.28:0/2639297760 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f63c403dba0 con 0x7f63d4101e00 2024-07-03T15:35:22.780 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:21.964+0000 7f63d904c640 1 -- 172.21.15.28:0/2639297760 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63d40761f0 con 0x7f63d4101e00 2024-07-03T15:35:22.780 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:21.964+0000 7f63cb7fe640 1 -- 172.21.15.28:0/2639297760 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f63c403bb40 con 0x7f63d4101e00 2024-07-03T15:35:22.780 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:21.964+0000 7f63d904c640 1 -- 172.21.15.28:0/2639297760 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d4076660 con 0x7f63d4101e00 2024-07-03T15:35:22.780 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:21.965+0000 7f63cb7fe640 1 -- 172.21.15.28:0/2639297760 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 42636+0+0 (secure 0 0 0) 0x7f63c4039070 con 0x7f63d4101e00 2024-07-03T15:35:22.780 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:21.965+0000 7f63cb7fe640 1 --2- 172.21.15.28:0/2639297760 >> [v2:172.21.15.28:6800/1358048352,v1:172.21.15.28:6801/1358048352] conn(0x7f63c0035780 0x7f63c0037c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:22.780 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:21.965+0000 7f63cb7fe640 1 -- 172.21.15.28:0/2639297760 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f63c4068760 con 0x7f63d4101e00 2024-07-03T15:35:22.781 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:21.965+0000 7f63cbfff640 1 -- 172.21.15.28:0/2639297760 >> [v2:172.21.15.28:6800/1358048352,v1:172.21.15.28:6801/1358048352] conn(0x7f63c0035780 msgr2=0x7f63c0037c30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.28:6800/1358048352 2024-07-03T15:35:22.781 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:21.965+0000 7f63cbfff640 1 --2- 172.21.15.28:0/2639297760 >> [v2:172.21.15.28:6800/1358048352,v1:172.21.15.28:6801/1358048352] conn(0x7f63c0035780 0x7f63c0037c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-03T15:35:22.781 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:21.965+0000 7f63d904c640 1 -- 172.21.15.28:0/2639297760 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63d404b530 con 0x7f63d4101e00 2024-07-03T15:35:22.781 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:21.970+0000 7f63cb7fe640 1 -- 172.21.15.28:0/2639297760 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f63c4037020 con 0x7f63d4101e00 2024-07-03T15:35:22.781 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:22.166+0000 7f63cbfff640 1 -- 172.21.15.28:0/2639297760 >> [v2:172.21.15.28:6800/1358048352,v1:172.21.15.28:6801/1358048352] conn(0x7f63c0035780 msgr2=0x7f63c0037c30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.28:6800/1358048352 2024-07-03T15:35:22.781 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:22.166+0000 7f63cbfff640 1 --2- 172.21.15.28:0/2639297760 >> [v2:172.21.15.28:6800/1358048352,v1:172.21.15.28:6801/1358048352] conn(0x7f63c0035780 0x7f63c0037c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-03T15:35:22.781 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:22.190+0000 7f63d904c640 1 -- 172.21.15.28:0/2639297760 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f63d4076bd0 con 0x7f63d4101e00 2024-07-03T15:35:22.781 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:22.190+0000 7f63cb7fe640 1 -- 172.21.15.28:0/2639297760 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7f63c4041070 con 0x7f63d4101e00 2024-07-03T15:35:22.781 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:22.192+0000 7f63d904c640 1 -- 172.21.15.28:0/2639297760 >> [v2:172.21.15.28:6800/1358048352,v1:172.21.15.28:6801/1358048352] conn(0x7f63c0035780 msgr2=0x7f63c0037c30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-03T15:35:22.781 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:22.192+0000 7f63d904c640 1 --2- 172.21.15.28:0/2639297760 >> [v2:172.21.15.28:6800/1358048352,v1:172.21.15.28:6801/1358048352] conn(0x7f63c0035780 0x7f63c0037c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:22.781 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:22.192+0000 7f63d904c640 1 -- 172.21.15.28:0/2639297760 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f63d4101e00 msgr2=0x7f63d4077c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:22.781 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:22.192+0000 7f63d904c640 1 --2- 172.21.15.28:0/2639297760 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f63d4101e00 0x7f63d4077c20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f63c4009420 tx=0x7f63c4008040 comp rx=0 tx=0).stop 2024-07-03T15:35:22.781 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:22.192+0000 7f63d904c640 1 -- 172.21.15.28:0/2639297760 shutdown_connections 2024-07-03T15:35:22.782 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:22.192+0000 7f63d904c640 1 --2- 172.21.15.28:0/2639297760 >> [v2:172.21.15.28:6800/1358048352,v1:172.21.15.28:6801/1358048352] conn(0x7f63c0035780 0x7f63c0037c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:22.782 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:22.192+0000 7f63d904c640 1 --2- 172.21.15.28:0/2639297760 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f63d4101e00 0x7f63d4077c20 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:22.782 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:22.192+0000 7f63d904c640 1 -- 172.21.15.28:0/2639297760 >> 172.21.15.28:0/2639297760 conn(0x7f63d40fd5a0 msgr2=0x7f63d40fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:22.782 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:22.193+0000 7f63d904c640 1 -- 172.21.15.28:0/2639297760 shutdown_connections 2024-07-03T15:35:22.782 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:22.193+0000 7f63d904c640 1 -- 172.21.15.28:0/2639297760 wait complete. 2024-07-03T15:35:22.782 INFO:teuthology.orchestra.run.smithi028.stdout:Waiting for the mgr to restart... 2024-07-03T15:35:22.782 INFO:teuthology.orchestra.run.smithi028.stdout:Waiting for mgr epoch 5... 2024-07-03T15:35:23.370 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:23 smithi028 ceph-mon[29148]: Active manager daemon smithi028.vtqrtz restarted 2024-07-03T15:35:23.370 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:23 smithi028 ceph-mon[29148]: Activating manager daemon smithi028.vtqrtz 2024-07-03T15:35:23.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:23 smithi028 ceph-mon[29148]: osdmap e2: 0 total, 0 up, 0 in 2024-07-03T15:35:23.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:23 smithi028 ceph-mon[29148]: mgrmap e6: smithi028.vtqrtz(active, starting, since 0.00279545s) 2024-07-03T15:35:23.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:23 smithi028 ceph-mon[29148]: from='mgr.14118 172.21.15.28:0/1044821505' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi028"}]: dispatch 2024-07-03T15:35:23.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:23 smithi028 ceph-mon[29148]: from='mgr.14118 172.21.15.28:0/1044821505' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mgr metadata", "who": "smithi028.vtqrtz", "id": "smithi028.vtqrtz"}]: dispatch 2024-07-03T15:35:23.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:23 smithi028 ceph-mon[29148]: from='mgr.14118 172.21.15.28:0/1044821505' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-03T15:35:23.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:23 smithi028 ceph-mon[29148]: from='mgr.14118 172.21.15.28:0/1044821505' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-03T15:35:23.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:23 smithi028 ceph-mon[29148]: from='mgr.14118 172.21.15.28:0/1044821505' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-03T15:35:23.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:23 smithi028 ceph-mon[29148]: Manager daemon smithi028.vtqrtz is now available 2024-07-03T15:35:23.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:23 smithi028 ceph-mon[29148]: Found migration_current of "None". Setting to last migration. 2024-07-03T15:35:23.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:23 smithi028 ceph-mon[29148]: from='mgr.14118 172.21.15.28:0/1044821505' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:35:23.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:23 smithi028 ceph-mon[29148]: from='mgr.14118 172.21.15.28:0/1044821505' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:35:23.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:23 smithi028 ceph-mon[29148]: from='mgr.14118 172.21.15.28:0/1044821505' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:35:23.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:23 smithi028 ceph-mon[29148]: from='mgr.14118 172.21.15.28:0/1044821505' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:35:23.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:23 smithi028 ceph-mon[29148]: from='mgr.14118 172.21.15.28:0/1044821505' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:35:23.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:23 smithi028 ceph-mon[29148]: from='mgr.14118 172.21.15.28:0/1044821505' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi028.vtqrtz/mirror_snapshot_schedule"}]: dispatch 2024-07-03T15:35:23.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:23 smithi028 ceph-mon[29148]: from='mgr.14118 172.21.15.28:0/1044821505' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi028.vtqrtz/trash_purge_schedule"}]: dispatch 2024-07-03T15:35:24.200 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout { 2024-07-03T15:35:24.200 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-07-03T15:35:24.200 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "initialized": true 2024-07-03T15:35:24.201 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout } 2024-07-03T15:35:24.201 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.140+0000 7fb663a17640 1 Processor -- start 2024-07-03T15:35:24.201 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.140+0000 7fb663a17640 1 -- start start 2024-07-03T15:35:24.201 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.140+0000 7fb663a17640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb65c075c00 0x7fb65c1049d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:24.201 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.140+0000 7fb663a17640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c104f10 con 0x7fb65c075c00 2024-07-03T15:35:24.201 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.141+0000 7fb6617b3640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb65c075c00 0x7fb65c1049d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:24.201 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.141+0000 7fb6617b3640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb65c075c00 0x7fb65c1049d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48678/0 (socket says 172.21.15.28:48678) 2024-07-03T15:35:24.201 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.141+0000 7fb6617b3640 1 -- 172.21.15.28:0/3057184128 learned_addr learned my addr 172.21.15.28:0/3057184128 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:24.201 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.141+0000 7fb6617b3640 1 -- 172.21.15.28:0/3057184128 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb65c105050 con 0x7fb65c075c00 2024-07-03T15:35:24.201 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.142+0000 7fb6617b3640 1 --2- 172.21.15.28:0/3057184128 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb65c075c00 0x7fb65c1049d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb650009840 tx=0x7fb6500091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45e786512ea994b7 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:24.201 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.142+0000 7fb64ffff640 1 -- 172.21.15.28:0/3057184128 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb650032440 con 0x7fb65c075c00 2024-07-03T15:35:24.201 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.142+0000 7fb64ffff640 1 -- 172.21.15.28:0/3057184128 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fb650032a30 con 0x7fb65c075c00 2024-07-03T15:35:24.201 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.142+0000 7fb64ffff640 1 -- 172.21.15.28:0/3057184128 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb650031ad0 con 0x7fb65c075c00 2024-07-03T15:35:24.202 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.142+0000 7fb663a17640 1 -- 172.21.15.28:0/3057184128 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb65c075c00 msgr2=0x7fb65c1049d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:24.202 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.142+0000 7fb663a17640 1 --2- 172.21.15.28:0/3057184128 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb65c075c00 0x7fb65c1049d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb650009840 tx=0x7fb6500091d0 comp rx=0 tx=0).stop 2024-07-03T15:35:24.202 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.142+0000 7fb663a17640 1 -- 172.21.15.28:0/3057184128 shutdown_connections 2024-07-03T15:35:24.202 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.142+0000 7fb663a17640 1 --2- 172.21.15.28:0/3057184128 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb65c075c00 0x7fb65c1049d0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:24.202 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.142+0000 7fb663a17640 1 -- 172.21.15.28:0/3057184128 >> 172.21.15.28:0/3057184128 conn(0x7fb65c0fc4c0 msgr2=0x7fb65c0fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:24.202 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.143+0000 7fb663a17640 1 -- 172.21.15.28:0/3057184128 shutdown_connections 2024-07-03T15:35:24.202 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.143+0000 7fb663a17640 1 -- 172.21.15.28:0/3057184128 wait complete. 2024-07-03T15:35:24.202 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.143+0000 7fb663a17640 1 Processor -- start 2024-07-03T15:35:24.202 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.143+0000 7fb663a17640 1 -- start start 2024-07-03T15:35:24.202 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.143+0000 7fb663a17640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb65c075c00 0x7fb65c110aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:24.202 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.143+0000 7fb663a17640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c110fe0 con 0x7fb65c075c00 2024-07-03T15:35:24.202 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.143+0000 7fb6617b3640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb65c075c00 0x7fb65c110aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:24.202 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.143+0000 7fb6617b3640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb65c075c00 0x7fb65c110aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48684/0 (socket says 172.21.15.28:48684) 2024-07-03T15:35:24.202 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.143+0000 7fb6617b3640 1 -- 172.21.15.28:0/870849446 learned_addr learned my addr 172.21.15.28:0/870849446 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:24.203 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.144+0000 7fb6617b3640 1 -- 172.21.15.28:0/870849446 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb650009450 con 0x7fb65c075c00 2024-07-03T15:35:24.203 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.144+0000 7fb6617b3640 1 --2- 172.21.15.28:0/870849446 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb65c075c00 0x7fb65c110aa0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb650031b90 tx=0x7fb650008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:24.203 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.144+0000 7fb64dffb640 1 -- 172.21.15.28:0/870849446 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb6500328b0 con 0x7fb65c075c00 2024-07-03T15:35:24.203 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.144+0000 7fb663a17640 1 -- 172.21.15.28:0/870849446 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb65c1111e0 con 0x7fb65c075c00 2024-07-03T15:35:24.203 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.144+0000 7fb64dffb640 1 -- 172.21.15.28:0/870849446 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fb65003d8e0 con 0x7fb65c075c00 2024-07-03T15:35:24.203 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.144+0000 7fb64dffb640 1 -- 172.21.15.28:0/870849446 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb65003ba20 con 0x7fb65c075c00 2024-07-03T15:35:24.203 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.144+0000 7fb663a17640 1 -- 172.21.15.28:0/870849446 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb65c111650 con 0x7fb65c075c00 2024-07-03T15:35:24.203 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.145+0000 7fb64dffb640 1 -- 172.21.15.28:0/870849446 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 6) v1 ==== 42403+0+0 (secure 0 0 0) 0x7fb650039070 con 0x7fb65c075c00 2024-07-03T15:35:24.203 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.145+0000 7fb64dffb640 1 -- 172.21.15.28:0/870849446 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fb650068480 con 0x7fb65c075c00 2024-07-03T15:35:24.203 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.606+0000 7fb64dffb640 1 -- 172.21.15.28:0/870849446 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mgrmap(e 7) v1 ==== 42530+0+0 (secure 0 0 0) 0x7fb650067890 con 0x7fb65c075c00 2024-07-03T15:35:24.203 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.606+0000 7fb64dffb640 1 --2- 172.21.15.28:0/870849446 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7fb64803a710 0x7fb64803caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:24.203 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.606+0000 7fb64dffb640 1 -- 172.21.15.28:0/870849446 --> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb64803d030 con 0x7fb64803a710 2024-07-03T15:35:24.204 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.607+0000 7fb660fb2640 1 --2- 172.21.15.28:0/870849446 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7fb64803a710 0x7fb64803caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:24.204 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.607+0000 7fb660fb2640 1 --2- 172.21.15.28:0/870849446 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7fb64803a710 0x7fb64803caf0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb658006fd0 tx=0x7fb658006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:24.204 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.608+0000 7fb64dffb640 1 -- 172.21.15.28:0/870849446 <== mgr.14118 v2:172.21.15.28:6800/1434683766 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7fb64803d030 con 0x7fb64803a710 2024-07-03T15:35:24.204 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.617+0000 7fb663a17640 1 -- 172.21.15.28:0/870849446 --> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fb65c05eb80 con 0x7fb64803a710 2024-07-03T15:35:24.204 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.618+0000 7fb64dffb640 1 -- 172.21.15.28:0/870849446 <== mgr.14118 v2:172.21.15.28:6800/1434683766 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fb65c05eb80 con 0x7fb64803a710 2024-07-03T15:35:24.204 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.618+0000 7fb663a17640 1 -- 172.21.15.28:0/870849446 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7fb64803a710 msgr2=0x7fb64803caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:24.204 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.618+0000 7fb663a17640 1 --2- 172.21.15.28:0/870849446 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7fb64803a710 0x7fb64803caf0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb658006fd0 tx=0x7fb658006db0 comp rx=0 tx=0).stop 2024-07-03T15:35:24.204 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.618+0000 7fb663a17640 1 -- 172.21.15.28:0/870849446 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb65c075c00 msgr2=0x7fb65c110aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:24.204 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.618+0000 7fb663a17640 1 --2- 172.21.15.28:0/870849446 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb65c075c00 0x7fb65c110aa0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb650031b90 tx=0x7fb650008040 comp rx=0 tx=0).stop 2024-07-03T15:35:24.204 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.618+0000 7fb663a17640 1 -- 172.21.15.28:0/870849446 shutdown_connections 2024-07-03T15:35:24.204 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.618+0000 7fb663a17640 1 --2- 172.21.15.28:0/870849446 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7fb64803a710 0x7fb64803caf0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:24.204 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.618+0000 7fb663a17640 1 --2- 172.21.15.28:0/870849446 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb65c075c00 0x7fb65c110aa0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:24.204 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.618+0000 7fb663a17640 1 -- 172.21.15.28:0/870849446 >> 172.21.15.28:0/870849446 conn(0x7fb65c0fc4c0 msgr2=0x7fb65c0fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:24.204 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.618+0000 7fb663a17640 1 -- 172.21.15.28:0/870849446 shutdown_connections 2024-07-03T15:35:24.205 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:23.618+0000 7fb663a17640 1 -- 172.21.15.28:0/870849446 wait complete. 2024-07-03T15:35:24.205 INFO:teuthology.orchestra.run.smithi028.stdout:mgr epoch 5 is available 2024-07-03T15:35:24.205 INFO:teuthology.orchestra.run.smithi028.stdout:Setting orchestrator backend to cephadm... 2024-07-03T15:35:24.808 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:24 smithi028 ceph-mon[29148]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-07-03T15:35:24.808 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:24 smithi028 ceph-mon[29148]: mgrmap e7: smithi028.vtqrtz(active, since 1.00514s) 2024-07-03T15:35:24.808 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:24 smithi028 ceph-mon[29148]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-07-03T15:35:25.390 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.607+0000 7f3706795640 1 Processor -- start 2024-07-03T15:35:25.391 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.607+0000 7f3706795640 1 -- start start 2024-07-03T15:35:25.391 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.607+0000 7f3706795640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37001012d0 0x7f37001016e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:25.391 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.607+0000 7f3706795640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3700101c20 con 0x7f37001012d0 2024-07-03T15:35:25.391 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.607+0000 7f36fffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37001012d0 0x7f37001016e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:25.391 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.607+0000 7f36fffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37001012d0 0x7f37001016e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48696/0 (socket says 172.21.15.28:48696) 2024-07-03T15:35:25.391 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.607+0000 7f36fffff640 1 -- 172.21.15.28:0/573422983 learned_addr learned my addr 172.21.15.28:0/573422983 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:25.391 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.608+0000 7f36fffff640 1 -- 172.21.15.28:0/573422983 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3700101d60 con 0x7f37001012d0 2024-07-03T15:35:25.391 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.609+0000 7f36fffff640 1 --2- 172.21.15.28:0/573422983 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37001012d0 0x7f37001016e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f36f0009840 tx=0x7f36f00091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=badbac49ecac7840 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:25.391 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.610+0000 7f36feffd640 1 -- 172.21.15.28:0/573422983 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f36f0032440 con 0x7f37001012d0 2024-07-03T15:35:25.391 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.610+0000 7f36feffd640 1 -- 172.21.15.28:0/573422983 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f36f0032a30 con 0x7f37001012d0 2024-07-03T15:35:25.392 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.610+0000 7f36feffd640 1 -- 172.21.15.28:0/573422983 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f36f0031ad0 con 0x7f37001012d0 2024-07-03T15:35:25.392 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.611+0000 7f3706795640 1 -- 172.21.15.28:0/573422983 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37001012d0 msgr2=0x7f37001016e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:25.392 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.611+0000 7f3706795640 1 --2- 172.21.15.28:0/573422983 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37001012d0 0x7f37001016e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f36f0009840 tx=0x7f36f00091d0 comp rx=0 tx=0).stop 2024-07-03T15:35:25.392 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.611+0000 7f3706795640 1 -- 172.21.15.28:0/573422983 shutdown_connections 2024-07-03T15:35:25.392 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.611+0000 7f3706795640 1 --2- 172.21.15.28:0/573422983 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37001012d0 0x7f37001016e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:25.392 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.611+0000 7f3706795640 1 -- 172.21.15.28:0/573422983 >> 172.21.15.28:0/573422983 conn(0x7f37000fbcc0 msgr2=0x7f37000fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:25.392 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.611+0000 7f3706795640 1 -- 172.21.15.28:0/573422983 shutdown_connections 2024-07-03T15:35:25.392 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.611+0000 7f3706795640 1 -- 172.21.15.28:0/573422983 wait complete. 2024-07-03T15:35:25.392 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.612+0000 7f3706795640 1 Processor -- start 2024-07-03T15:35:25.392 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.612+0000 7f3706795640 1 -- start start 2024-07-03T15:35:25.392 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.612+0000 7f3706795640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37001012d0 0x7f370010f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:25.392 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.612+0000 7f3706795640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370010fec0 con 0x7f37001012d0 2024-07-03T15:35:25.393 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.612+0000 7f36fffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37001012d0 0x7f370010f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:25.393 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.612+0000 7f36fffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37001012d0 0x7f370010f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48704/0 (socket says 172.21.15.28:48704) 2024-07-03T15:35:25.393 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.612+0000 7f36fffff640 1 -- 172.21.15.28:0/2818661231 learned_addr learned my addr 172.21.15.28:0/2818661231 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:25.393 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.613+0000 7f36fffff640 1 -- 172.21.15.28:0/2818661231 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36f0009450 con 0x7f37001012d0 2024-07-03T15:35:25.393 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.613+0000 7f36fffff640 1 --2- 172.21.15.28:0/2818661231 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37001012d0 0x7f370010f980 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f36f00317b0 tx=0x7f36f0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:25.393 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.613+0000 7f36fcff9640 1 -- 172.21.15.28:0/2818661231 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f36f0031e90 con 0x7f37001012d0 2024-07-03T15:35:25.393 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.613+0000 7f3706795640 1 -- 172.21.15.28:0/2818661231 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37001100c0 con 0x7f37001012d0 2024-07-03T15:35:25.393 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.613+0000 7f36fcff9640 1 -- 172.21.15.28:0/2818661231 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f36f003daf0 con 0x7f37001012d0 2024-07-03T15:35:25.393 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.613+0000 7f36fcff9640 1 -- 172.21.15.28:0/2818661231 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f36f003baa0 con 0x7f37001012d0 2024-07-03T15:35:25.393 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.613+0000 7f3706795640 1 -- 172.21.15.28:0/2818661231 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3700110530 con 0x7f37001012d0 2024-07-03T15:35:25.393 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.614+0000 7f36fcff9640 1 -- 172.21.15.28:0/2818661231 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 8) v1 ==== 42636+0+0 (secure 0 0 0) 0x7f36f0039070 con 0x7f37001012d0 2024-07-03T15:35:25.394 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.614+0000 7f3706795640 1 -- 172.21.15.28:0/2818661231 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f370004b530 con 0x7f37001012d0 2024-07-03T15:35:25.394 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.614+0000 7f36fcff9640 1 --2- 172.21.15.28:0/2818661231 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f36e8035780 0x7f36e8037c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:25.394 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.615+0000 7f36fcff9640 1 -- 172.21.15.28:0/2818661231 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f36f00688c0 con 0x7f37001012d0 2024-07-03T15:35:25.394 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.615+0000 7f36ff7fe640 1 --2- 172.21.15.28:0/2818661231 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f36e8035780 0x7f36e8037c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:25.394 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.615+0000 7f36ff7fe640 1 --2- 172.21.15.28:0/2818661231 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f36e8035780 0x7f36e8037c30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f36f4006fd0 tx=0x7f36f4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:25.394 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.619+0000 7f36fcff9640 1 -- 172.21.15.28:0/2818661231 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f36f003be00 con 0x7f37001012d0 2024-07-03T15:35:25.394 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.806+0000 7f3706795640 1 -- 172.21.15.28:0/2818661231 --> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f3700110530 con 0x7f36e8035780 2024-07-03T15:35:25.394 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.814+0000 7f36fcff9640 1 -- 172.21.15.28:0/2818661231 <== mgr.14118 v2:172.21.15.28:6800/1434683766 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3700110530 con 0x7f36e8035780 2024-07-03T15:35:25.394 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.816+0000 7f3706795640 1 -- 172.21.15.28:0/2818661231 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f36e8035780 msgr2=0x7f36e8037c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:25.394 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.816+0000 7f3706795640 1 --2- 172.21.15.28:0/2818661231 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f36e8035780 0x7f36e8037c30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f36f4006fd0 tx=0x7f36f4006db0 comp rx=0 tx=0).stop 2024-07-03T15:35:25.394 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.816+0000 7f3706795640 1 -- 172.21.15.28:0/2818661231 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37001012d0 msgr2=0x7f370010f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:25.395 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.816+0000 7f3706795640 1 --2- 172.21.15.28:0/2818661231 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37001012d0 0x7f370010f980 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f36f00317b0 tx=0x7f36f0008040 comp rx=0 tx=0).stop 2024-07-03T15:35:25.395 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.816+0000 7f3706795640 1 -- 172.21.15.28:0/2818661231 shutdown_connections 2024-07-03T15:35:25.395 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.816+0000 7f3706795640 1 --2- 172.21.15.28:0/2818661231 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f36e8035780 0x7f36e8037c30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:25.395 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.816+0000 7f3706795640 1 --2- 172.21.15.28:0/2818661231 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37001012d0 0x7f370010f980 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:25.395 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.816+0000 7f3706795640 1 -- 172.21.15.28:0/2818661231 >> 172.21.15.28:0/2818661231 conn(0x7f37000fbcc0 msgr2=0x7f37000fc690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:25.395 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.816+0000 7f3706795640 1 -- 172.21.15.28:0/2818661231 shutdown_connections 2024-07-03T15:35:25.395 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:24.816+0000 7f3706795640 1 -- 172.21.15.28:0/2818661231 wait complete. 2024-07-03T15:35:25.779 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:25 smithi028 ceph-mon[29148]: mgrmap e8: smithi028.vtqrtz(active, since 2s) 2024-07-03T15:35:25.779 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:25 smithi028 ceph-mon[29148]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:35:25.779 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:25 smithi028 ceph-mon[29148]: from='mgr.14118 172.21.15.28:0/1044821505' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:35:25.779 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:25 smithi028 ceph-mon[29148]: from='mgr.14118 172.21.15.28:0/1044821505' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:35:25.779 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:25 smithi028 ceph-mon[29148]: from='mgr.14118 172.21.15.28:0/1044821505' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:35:25.780 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:25 smithi028 ceph-mon[29148]: from='mgr.14118 172.21.15.28:0/1044821505' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:35:26.512 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout value unchanged 2024-07-03T15:35:26.512 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.777+0000 7f95b873b640 1 Processor -- start 2024-07-03T15:35:26.512 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.777+0000 7f95b873b640 1 -- start start 2024-07-03T15:35:26.512 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.777+0000 7f95b873b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f95b0075bc0 0x7f95b0104a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:26.513 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.777+0000 7f95b873b640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b0104f50 con 0x7f95b0075bc0 2024-07-03T15:35:26.513 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.777+0000 7f95b64d7640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f95b0075bc0 0x7f95b0104a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:26.513 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.778+0000 7f95b64d7640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f95b0075bc0 0x7f95b0104a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48712/0 (socket says 172.21.15.28:48712) 2024-07-03T15:35:26.513 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.778+0000 7f95b64d7640 1 -- 172.21.15.28:0/211598213 learned_addr learned my addr 172.21.15.28:0/211598213 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:26.513 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.778+0000 7f95b64d7640 1 -- 172.21.15.28:0/211598213 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b0105090 con 0x7f95b0075bc0 2024-07-03T15:35:26.513 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.779+0000 7f95b64d7640 1 --2- 172.21.15.28:0/211598213 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f95b0075bc0 0x7f95b0104a10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f95ac009840 tx=0x7f95ac0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5668797e70ace7 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:26.513 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.779+0000 7f95b54d5640 1 -- 172.21.15.28:0/211598213 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f95ac032440 con 0x7f95b0075bc0 2024-07-03T15:35:26.513 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.779+0000 7f95b54d5640 1 -- 172.21.15.28:0/211598213 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f95ac032a30 con 0x7f95b0075bc0 2024-07-03T15:35:26.513 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.779+0000 7f95b54d5640 1 -- 172.21.15.28:0/211598213 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f95ac031ad0 con 0x7f95b0075bc0 2024-07-03T15:35:26.513 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.779+0000 7f95b873b640 1 -- 172.21.15.28:0/211598213 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f95b0075bc0 msgr2=0x7f95b0104a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:26.513 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.779+0000 7f95b873b640 1 --2- 172.21.15.28:0/211598213 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f95b0075bc0 0x7f95b0104a10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f95ac009840 tx=0x7f95ac0091d0 comp rx=0 tx=0).stop 2024-07-03T15:35:26.513 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.780+0000 7f95b873b640 1 -- 172.21.15.28:0/211598213 shutdown_connections 2024-07-03T15:35:26.514 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.780+0000 7f95b873b640 1 --2- 172.21.15.28:0/211598213 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f95b0075bc0 0x7f95b0104a10 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:26.514 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.780+0000 7f95b873b640 1 -- 172.21.15.28:0/211598213 >> 172.21.15.28:0/211598213 conn(0x7f95b00fc4c0 msgr2=0x7f95b00fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:26.514 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.780+0000 7f95b873b640 1 -- 172.21.15.28:0/211598213 shutdown_connections 2024-07-03T15:35:26.514 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.780+0000 7f95b873b640 1 -- 172.21.15.28:0/211598213 wait complete. 2024-07-03T15:35:26.514 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.780+0000 7f95b873b640 1 Processor -- start 2024-07-03T15:35:26.514 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.780+0000 7f95b873b640 1 -- start start 2024-07-03T15:35:26.514 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.780+0000 7f95b873b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f95b0075bc0 0x7f95b0110aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:26.514 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.780+0000 7f95b873b640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b0110fe0 con 0x7f95b0075bc0 2024-07-03T15:35:26.514 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.780+0000 7f95b64d7640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f95b0075bc0 0x7f95b0110aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:26.514 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.781+0000 7f95b64d7640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f95b0075bc0 0x7f95b0110aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48720/0 (socket says 172.21.15.28:48720) 2024-07-03T15:35:26.514 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.781+0000 7f95b64d7640 1 -- 172.21.15.28:0/1822342845 learned_addr learned my addr 172.21.15.28:0/1822342845 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:26.514 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.781+0000 7f95b64d7640 1 -- 172.21.15.28:0/1822342845 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ac009450 con 0x7f95b0075bc0 2024-07-03T15:35:26.514 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.781+0000 7f95b64d7640 1 --2- 172.21.15.28:0/1822342845 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f95b0075bc0 0x7f95b0110aa0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f95ac031b90 tx=0x7f95ac008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:26.515 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.781+0000 7f95a6ffd640 1 -- 172.21.15.28:0/1822342845 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f95ac0328b0 con 0x7f95b0075bc0 2024-07-03T15:35:26.515 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.781+0000 7f95a6ffd640 1 -- 172.21.15.28:0/1822342845 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f95ac03d8e0 con 0x7f95b0075bc0 2024-07-03T15:35:26.515 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.781+0000 7f95b873b640 1 -- 172.21.15.28:0/1822342845 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95b01111e0 con 0x7f95b0075bc0 2024-07-03T15:35:26.515 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.781+0000 7f95a6ffd640 1 -- 172.21.15.28:0/1822342845 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f95ac03ba20 con 0x7f95b0075bc0 2024-07-03T15:35:26.515 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.781+0000 7f95b873b640 1 -- 172.21.15.28:0/1822342845 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95b0111650 con 0x7f95b0075bc0 2024-07-03T15:35:26.515 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.782+0000 7f95a6ffd640 1 -- 172.21.15.28:0/1822342845 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 8) v1 ==== 42636+0+0 (secure 0 0 0) 0x7f95ac039070 con 0x7f95b0075bc0 2024-07-03T15:35:26.515 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.782+0000 7f95a6ffd640 1 --2- 172.21.15.28:0/1822342845 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f959c035780 0x7f959c037c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:26.515 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.782+0000 7f95b873b640 1 -- 172.21.15.28:0/1822342845 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95b004b530 con 0x7f95b0075bc0 2024-07-03T15:35:26.515 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.782+0000 7f95a6ffd640 1 -- 172.21.15.28:0/1822342845 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f95ac068880 con 0x7f95b0075bc0 2024-07-03T15:35:26.515 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.782+0000 7f95b5cd6640 1 --2- 172.21.15.28:0/1822342845 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f959c035780 0x7f959c037c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:26.515 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.783+0000 7f95b5cd6640 1 --2- 172.21.15.28:0/1822342845 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f959c035780 0x7f959c037c30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f95a0006fd0 tx=0x7f95a0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:26.515 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.787+0000 7f95a6ffd640 1 -- 172.21.15.28:0/1822342845 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f95ac03ca00 con 0x7f95b0075bc0 2024-07-03T15:35:26.515 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.961+0000 7f95b873b640 1 -- 172.21.15.28:0/1822342845 --> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f95b0111650 con 0x7f959c035780 2024-07-03T15:35:26.516 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.962+0000 7f95a6ffd640 1 -- 172.21.15.28:0/1822342845 <== mgr.14118 v2:172.21.15.28:6800/1434683766 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f95b0111650 con 0x7f959c035780 2024-07-03T15:35:26.516 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.964+0000 7f95b873b640 1 -- 172.21.15.28:0/1822342845 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f959c035780 msgr2=0x7f959c037c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:26.516 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.964+0000 7f95b873b640 1 --2- 172.21.15.28:0/1822342845 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f959c035780 0x7f959c037c30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f95a0006fd0 tx=0x7f95a0006db0 comp rx=0 tx=0).stop 2024-07-03T15:35:26.516 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.964+0000 7f95b873b640 1 -- 172.21.15.28:0/1822342845 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f95b0075bc0 msgr2=0x7f95b0110aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:26.516 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.964+0000 7f95b873b640 1 --2- 172.21.15.28:0/1822342845 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f95b0075bc0 0x7f95b0110aa0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f95ac031b90 tx=0x7f95ac008040 comp rx=0 tx=0).stop 2024-07-03T15:35:26.516 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.964+0000 7f95b873b640 1 -- 172.21.15.28:0/1822342845 shutdown_connections 2024-07-03T15:35:26.516 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.964+0000 7f95b873b640 1 --2- 172.21.15.28:0/1822342845 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f959c035780 0x7f959c037c30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:26.516 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.964+0000 7f95b873b640 1 --2- 172.21.15.28:0/1822342845 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f95b0075bc0 0x7f95b0110aa0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:26.516 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.964+0000 7f95b873b640 1 -- 172.21.15.28:0/1822342845 >> 172.21.15.28:0/1822342845 conn(0x7f95b00fc4c0 msgr2=0x7f95b00fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:26.516 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.964+0000 7f95b873b640 1 -- 172.21.15.28:0/1822342845 shutdown_connections 2024-07-03T15:35:26.516 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:25.964+0000 7f95b873b640 1 -- 172.21.15.28:0/1822342845 wait complete. 2024-07-03T15:35:26.516 INFO:teuthology.orchestra.run.smithi028.stdout:Generating ssh key... 2024-07-03T15:35:26.870 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:26 smithi028 ceph-mon[29148]: [03/Jul/2024:15:35:25] ENGINE Bus STARTING 2024-07-03T15:35:26.870 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:26 smithi028 ceph-mon[29148]: [03/Jul/2024:15:35:25] ENGINE Serving on https://172.21.15.28:7150 2024-07-03T15:35:26.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:26 smithi028 ceph-mon[29148]: [03/Jul/2024:15:35:25] ENGINE Bus STARTED 2024-07-03T15:35:26.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:26 smithi028 ceph-mon[29148]: [03/Jul/2024:15:35:25] ENGINE Error in HTTPServer.serve 2024-07-03T15:35:26.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Traceback (most recent call last): 2024-07-03T15:35:26.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-03T15:35:26.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: self._connections.run(self.expiration_interval) 2024-07-03T15:35:26.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-03T15:35:26.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: self._run(expiration_interval) 2024-07-03T15:35:26.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-03T15:35:26.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-07-03T15:35:26.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-03T15:35:26.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-03T15:35:26.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-03T15:35:26.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: s = self.context.wrap_socket( 2024-07-03T15:35:26.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-03T15:35:26.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: return self.sslsocket_class._create( 2024-07-03T15:35:26.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-03T15:35:26.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: self.do_handshake() 2024-07-03T15:35:26.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-03T15:35:26.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: self._sslobj.do_handshake() 2024-07-03T15:35:26.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-03T15:35:27.827 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:27 smithi028 ceph-mon[29148]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:35:27.827 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:27 smithi028 ceph-mon[29148]: from='mgr.14118 172.21.15.28:0/1044821505' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:35:27.828 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:27 smithi028 ceph-mon[29148]: from='mgr.14118 172.21.15.28:0/1044821505' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:35:27.835 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:26.886+0000 7f8cbfc69640 1 Processor -- start 2024-07-03T15:35:27.835 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:26.886+0000 7f8cbfc69640 1 -- start start 2024-07-03T15:35:27.835 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:26.887+0000 7f8cbfc69640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8cb8101e00 0x7f8cb8102210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:27.835 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:26.887+0000 7f8cbfc69640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb8102750 con 0x7f8cb8101e00 2024-07-03T15:35:27.835 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:26.887+0000 7f8cbda05640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8cb8101e00 0x7f8cb8102210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:27.835 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:26.887+0000 7f8cbda05640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8cb8101e00 0x7f8cb8102210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48728/0 (socket says 172.21.15.28:48728) 2024-07-03T15:35:27.836 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:26.887+0000 7f8cbda05640 1 -- 172.21.15.28:0/679522723 learned_addr learned my addr 172.21.15.28:0/679522723 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:27.836 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:26.887+0000 7f8cbda05640 1 -- 172.21.15.28:0/679522723 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb8102890 con 0x7f8cb8101e00 2024-07-03T15:35:27.836 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:26.888+0000 7f8cbda05640 1 --2- 172.21.15.28:0/679522723 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8cb8101e00 0x7f8cb8102210 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8009e50 tx=0x7f8ca8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a00612735d4aed6 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:27.836 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:26.888+0000 7f8cbca03640 1 -- 172.21.15.28:0/679522723 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8ca8031560 con 0x7f8cb8101e00 2024-07-03T15:35:27.836 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:26.888+0000 7f8cbca03640 1 -- 172.21.15.28:0/679522723 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f8ca802d050 con 0x7f8cb8101e00 2024-07-03T15:35:27.836 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:26.888+0000 7f8cbca03640 1 -- 172.21.15.28:0/679522723 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8ca8007a50 con 0x7f8cb8101e00 2024-07-03T15:35:27.836 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:26.889+0000 7f8cbfc69640 1 -- 172.21.15.28:0/679522723 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8cb8101e00 msgr2=0x7f8cb8102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:27.836 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:26.889+0000 7f8cbfc69640 1 --2- 172.21.15.28:0/679522723 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8cb8101e00 0x7f8cb8102210 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8009e50 tx=0x7f8ca8009840 comp rx=0 tx=0).stop 2024-07-03T15:35:27.836 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:26.889+0000 7f8cbfc69640 1 -- 172.21.15.28:0/679522723 shutdown_connections 2024-07-03T15:35:27.836 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:26.889+0000 7f8cbfc69640 1 --2- 172.21.15.28:0/679522723 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8cb8101e00 0x7f8cb8102210 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:27.836 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:26.889+0000 7f8cbfc69640 1 -- 172.21.15.28:0/679522723 >> 172.21.15.28:0/679522723 conn(0x7f8cb80fd5a0 msgr2=0x7f8cb80ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:27.836 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:26.889+0000 7f8cbfc69640 1 -- 172.21.15.28:0/679522723 shutdown_connections 2024-07-03T15:35:27.836 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:26.889+0000 7f8cbfc69640 1 -- 172.21.15.28:0/679522723 wait complete. 2024-07-03T15:35:27.837 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:26.889+0000 7f8cbfc69640 1 Processor -- start 2024-07-03T15:35:27.837 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:26.889+0000 7f8cbfc69640 1 -- start start 2024-07-03T15:35:27.837 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:26.890+0000 7f8cbfc69640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8cb8101e00 0x7f8cb8110b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:27.837 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:26.890+0000 7f8cbfc69640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb8111050 con 0x7f8cb8101e00 2024-07-03T15:35:27.837 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:26.890+0000 7f8cbda05640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8cb8101e00 0x7f8cb8110b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:27.837 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:26.890+0000 7f8cbda05640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8cb8101e00 0x7f8cb8110b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48730/0 (socket says 172.21.15.28:48730) 2024-07-03T15:35:27.837 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:26.890+0000 7f8cbda05640 1 -- 172.21.15.28:0/3210934936 learned_addr learned my addr 172.21.15.28:0/3210934936 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:27.837 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:26.890+0000 7f8cbda05640 1 -- 172.21.15.28:0/3210934936 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca8009ae0 con 0x7f8cb8101e00 2024-07-03T15:35:27.837 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:26.890+0000 7f8cbda05640 1 --2- 172.21.15.28:0/3210934936 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8cb8101e00 0x7f8cb8110b10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80319b0 tx=0x7f8ca8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:27.837 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:26.890+0000 7f8cb67fc640 1 -- 172.21.15.28:0/3210934936 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8ca8031df0 con 0x7f8cb8101e00 2024-07-03T15:35:27.837 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:26.890+0000 7f8cb67fc640 1 -- 172.21.15.28:0/3210934936 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f8ca80458c0 con 0x7f8cb8101e00 2024-07-03T15:35:27.837 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:26.890+0000 7f8cbfc69640 1 -- 172.21.15.28:0/3210934936 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cb8111250 con 0x7f8cb8101e00 2024-07-03T15:35:27.837 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:26.890+0000 7f8cb67fc640 1 -- 172.21.15.28:0/3210934936 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8ca803a8c0 con 0x7f8cb8101e00 2024-07-03T15:35:27.837 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:26.890+0000 7f8cbfc69640 1 -- 172.21.15.28:0/3210934936 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cb81116c0 con 0x7f8cb8101e00 2024-07-03T15:35:27.837 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:26.891+0000 7f8cb67fc640 1 -- 172.21.15.28:0/3210934936 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 8) v1 ==== 42636+0+0 (secure 0 0 0) 0x7f8ca8038040 con 0x7f8cb8101e00 2024-07-03T15:35:27.838 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:26.892+0000 7f8cb67fc640 1 --2- 172.21.15.28:0/3210934936 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f8ca00357d0 0x7f8ca0037c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:27.838 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:26.892+0000 7f8cbfc69640 1 -- 172.21.15.28:0/3210934936 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cb804b530 con 0x7f8cb8101e00 2024-07-03T15:35:27.838 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:26.892+0000 7f8cb67fc640 1 -- 172.21.15.28:0/3210934936 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f8ca8040030 con 0x7f8cb8101e00 2024-07-03T15:35:27.838 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:26.892+0000 7f8cbd204640 1 --2- 172.21.15.28:0/3210934936 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f8ca00357d0 0x7f8ca0037c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:27.838 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:26.892+0000 7f8cbd204640 1 --2- 172.21.15.28:0/3210934936 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f8ca00357d0 0x7f8ca0037c80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8cac006fd0 tx=0x7f8cac006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:27.838 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:26.896+0000 7f8cb67fc640 1 -- 172.21.15.28:0/3210934936 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8ca8036070 con 0x7f8cb8101e00 2024-07-03T15:35:27.838 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:27.072+0000 7f8cbfc69640 1 -- 172.21.15.28:0/3210934936 --> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f8cb80fe790 con 0x7f8ca00357d0 2024-07-03T15:35:27.838 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:27.208+0000 7f8cb67fc640 1 -- 172.21.15.28:0/3210934936 <== mgr.14118 v2:172.21.15.28:6800/1434683766 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8cb80fe790 con 0x7f8ca00357d0 2024-07-03T15:35:27.838 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:27.210+0000 7f8cbfc69640 1 -- 172.21.15.28:0/3210934936 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f8ca00357d0 msgr2=0x7f8ca0037c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:27.838 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:27.210+0000 7f8cbfc69640 1 --2- 172.21.15.28:0/3210934936 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f8ca00357d0 0x7f8ca0037c80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8cac006fd0 tx=0x7f8cac006db0 comp rx=0 tx=0).stop 2024-07-03T15:35:27.838 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:27.210+0000 7f8cbfc69640 1 -- 172.21.15.28:0/3210934936 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8cb8101e00 msgr2=0x7f8cb8110b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:27.838 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:27.210+0000 7f8cbfc69640 1 --2- 172.21.15.28:0/3210934936 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8cb8101e00 0x7f8cb8110b10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80319b0 tx=0x7f8ca8009840 comp rx=0 tx=0).stop 2024-07-03T15:35:27.839 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:27.210+0000 7f8cbfc69640 1 -- 172.21.15.28:0/3210934936 shutdown_connections 2024-07-03T15:35:27.839 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:27.210+0000 7f8cbfc69640 1 --2- 172.21.15.28:0/3210934936 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f8ca00357d0 0x7f8ca0037c80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:27.839 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:27.210+0000 7f8cbfc69640 1 --2- 172.21.15.28:0/3210934936 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8cb8101e00 0x7f8cb8110b10 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:27.839 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:27.210+0000 7f8cbfc69640 1 -- 172.21.15.28:0/3210934936 >> 172.21.15.28:0/3210934936 conn(0x7f8cb80fd5a0 msgr2=0x7f8cb80fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:27.839 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:27.210+0000 7f8cbfc69640 1 -- 172.21.15.28:0/3210934936 shutdown_connections 2024-07-03T15:35:27.839 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:27.210+0000 7f8cbfc69640 1 -- 172.21.15.28:0/3210934936 wait complete. 2024-07-03T15:35:28.725 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:28 smithi028 ceph-mon[29148]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:35:28.725 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:28 smithi028 ceph-mon[29148]: Generating ssh key... 2024-07-03T15:35:28.960 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC24c+XU+Lshmz+j874Ll8PTloFWBz8f0ABEbHIdykRz/iEUS4PW+lmBvDfVTjpORnMQtrvUURCCXuMG1isTWUX/0oZpHFq9Zw/W60bUSAWT4x0Jn/rGaHpToLFVpPnVkFgM2CtlTnYAKAlzQf4TKyRUX2fpxewU21sh12iizjpZ26ccGsUOvy4WbldyfuLo67Fghs04dA3bd4AwrcDlgWzzRxrnxuzpQTkTAa5Bi4jKVJrMgK7SZFD/AoAWgyMCYPHUpGHJ1nwTbNSuBoxPg+z4fH2qnRk+N6mWNykT1VK+NU1rGOqqmB8hHHhWazIHrTrhDu06ApWgrHfzVfv+YFF+cLD37L9Twx9T4wjFncJSiXTnPLnFYN3Wx7ejwsDqf+CBFqsknQIfiBlOYPDt2Vum7UzL+sCUjUIC629QxVzbGr7j8vzBw3Zl9oxZ3TJaWjinK5H+w5vufhIJ1/puk/bDN/MQdN/phfcrsRXmdpYLRqnaGywBQytVe39diTldAE= ceph-a85e4be6-3951-11ef-bcab-c7b262605968 2024-07-03T15:35:28.961 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.224+0000 7f9ba4186640 1 Processor -- start 2024-07-03T15:35:28.961 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.224+0000 7f9ba4186640 1 -- start start 2024-07-03T15:35:28.961 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.224+0000 7f9ba4186640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b9c101e00 0x7f9b9c102210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:28.961 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.224+0000 7f9ba4186640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c102750 con 0x7f9b9c101e00 2024-07-03T15:35:28.961 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.225+0000 7f9ba1f22640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b9c101e00 0x7f9b9c102210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:28.961 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.225+0000 7f9ba1f22640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b9c101e00 0x7f9b9c102210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48744/0 (socket says 172.21.15.28:48744) 2024-07-03T15:35:28.961 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.225+0000 7f9ba1f22640 1 -- 172.21.15.28:0/314056914 learned_addr learned my addr 172.21.15.28:0/314056914 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:28.961 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.225+0000 7f9ba1f22640 1 -- 172.21.15.28:0/314056914 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b9c102890 con 0x7f9b9c101e00 2024-07-03T15:35:28.961 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.226+0000 7f9ba1f22640 1 --2- 172.21.15.28:0/314056914 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b9c101e00 0x7f9b9c102210 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9b94009840 tx=0x7f9b940091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86afb8ffda788e0d server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:28.961 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.226+0000 7f9ba0f20640 1 -- 172.21.15.28:0/314056914 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9b94032440 con 0x7f9b9c101e00 2024-07-03T15:35:28.961 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.226+0000 7f9ba0f20640 1 -- 172.21.15.28:0/314056914 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f9b94032a30 con 0x7f9b9c101e00 2024-07-03T15:35:28.961 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.226+0000 7f9ba0f20640 1 -- 172.21.15.28:0/314056914 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9b94031ad0 con 0x7f9b9c101e00 2024-07-03T15:35:28.961 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.226+0000 7f9ba4186640 1 -- 172.21.15.28:0/314056914 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b9c101e00 msgr2=0x7f9b9c102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:28.962 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.226+0000 7f9ba4186640 1 --2- 172.21.15.28:0/314056914 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b9c101e00 0x7f9b9c102210 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9b94009840 tx=0x7f9b940091d0 comp rx=0 tx=0).stop 2024-07-03T15:35:28.962 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.226+0000 7f9ba4186640 1 -- 172.21.15.28:0/314056914 shutdown_connections 2024-07-03T15:35:28.962 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.226+0000 7f9ba4186640 1 --2- 172.21.15.28:0/314056914 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b9c101e00 0x7f9b9c102210 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:28.962 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.226+0000 7f9ba4186640 1 -- 172.21.15.28:0/314056914 >> 172.21.15.28:0/314056914 conn(0x7f9b9c0fd5a0 msgr2=0x7f9b9c0ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:28.962 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.227+0000 7f9ba4186640 1 -- 172.21.15.28:0/314056914 shutdown_connections 2024-07-03T15:35:28.962 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.227+0000 7f9ba4186640 1 -- 172.21.15.28:0/314056914 wait complete. 2024-07-03T15:35:28.962 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.227+0000 7f9ba4186640 1 Processor -- start 2024-07-03T15:35:28.962 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.227+0000 7f9ba4186640 1 -- start start 2024-07-03T15:35:28.962 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.227+0000 7f9ba4186640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b9c101e00 0x7f9b9c110a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:28.962 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.227+0000 7f9ba4186640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c110fd0 con 0x7f9b9c101e00 2024-07-03T15:35:28.962 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.227+0000 7f9ba1f22640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b9c101e00 0x7f9b9c110a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:28.962 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.227+0000 7f9ba1f22640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b9c101e00 0x7f9b9c110a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48758/0 (socket says 172.21.15.28:48758) 2024-07-03T15:35:28.962 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.227+0000 7f9ba1f22640 1 -- 172.21.15.28:0/1911475075 learned_addr learned my addr 172.21.15.28:0/1911475075 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:28.963 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.228+0000 7f9ba1f22640 1 -- 172.21.15.28:0/1911475075 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b94009450 con 0x7f9b9c101e00 2024-07-03T15:35:28.963 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.228+0000 7f9ba1f22640 1 --2- 172.21.15.28:0/1911475075 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b9c101e00 0x7f9b9c110a90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9b94009420 tx=0x7f9b94008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:28.963 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.228+0000 7f9b9a7fc640 1 -- 172.21.15.28:0/1911475075 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9b9403d5b0 con 0x7f9b9c101e00 2024-07-03T15:35:28.963 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.228+0000 7f9b9a7fc640 1 -- 172.21.15.28:0/1911475075 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f9b9403dba0 con 0x7f9b9c101e00 2024-07-03T15:35:28.963 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.228+0000 7f9ba4186640 1 -- 172.21.15.28:0/1911475075 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b9c1111d0 con 0x7f9b9c101e00 2024-07-03T15:35:28.963 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.228+0000 7f9b9a7fc640 1 -- 172.21.15.28:0/1911475075 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9b9403bb40 con 0x7f9b9c101e00 2024-07-03T15:35:28.963 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.228+0000 7f9ba4186640 1 -- 172.21.15.28:0/1911475075 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b9c111640 con 0x7f9b9c101e00 2024-07-03T15:35:28.963 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.229+0000 7f9b9a7fc640 1 -- 172.21.15.28:0/1911475075 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 8) v1 ==== 42636+0+0 (secure 0 0 0) 0x7f9b94039070 con 0x7f9b9c101e00 2024-07-03T15:35:28.963 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.229+0000 7f9ba4186640 1 -- 172.21.15.28:0/1911475075 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b9c04b530 con 0x7f9b9c101e00 2024-07-03T15:35:28.963 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.229+0000 7f9b9a7fc640 1 --2- 172.21.15.28:0/1911475075 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f9b84035780 0x7f9b84037c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:28.963 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.229+0000 7f9b9a7fc640 1 -- 172.21.15.28:0/1911475075 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f9b940687e0 con 0x7f9b9c101e00 2024-07-03T15:35:28.963 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.229+0000 7f9ba1721640 1 --2- 172.21.15.28:0/1911475075 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f9b84035780 0x7f9b84037c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:28.963 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.230+0000 7f9ba1721640 1 --2- 172.21.15.28:0/1911475075 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f9b84035780 0x7f9b84037c30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9b90006fd0 tx=0x7f9b90006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:28.964 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.234+0000 7f9b9a7fc640 1 -- 172.21.15.28:0/1911475075 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9b9403cdf0 con 0x7f9b9c101e00 2024-07-03T15:35:28.964 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.409+0000 7f9ba4186640 1 -- 172.21.15.28:0/1911475075 --> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b9c111640 con 0x7f9b84035780 2024-07-03T15:35:28.964 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.410+0000 7f9b9a7fc640 1 -- 172.21.15.28:0/1911475075 <== mgr.14118 v2:172.21.15.28:6800/1434683766 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f9b9c111640 con 0x7f9b84035780 2024-07-03T15:35:28.964 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.412+0000 7f9ba4186640 1 -- 172.21.15.28:0/1911475075 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f9b84035780 msgr2=0x7f9b84037c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:28.964 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.412+0000 7f9ba4186640 1 --2- 172.21.15.28:0/1911475075 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f9b84035780 0x7f9b84037c30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9b90006fd0 tx=0x7f9b90006db0 comp rx=0 tx=0).stop 2024-07-03T15:35:28.964 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.412+0000 7f9ba4186640 1 -- 172.21.15.28:0/1911475075 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b9c101e00 msgr2=0x7f9b9c110a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:28.964 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.412+0000 7f9ba4186640 1 --2- 172.21.15.28:0/1911475075 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b9c101e00 0x7f9b9c110a90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9b94009420 tx=0x7f9b94008040 comp rx=0 tx=0).stop 2024-07-03T15:35:28.964 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.412+0000 7f9ba4186640 1 -- 172.21.15.28:0/1911475075 shutdown_connections 2024-07-03T15:35:28.964 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.412+0000 7f9ba4186640 1 --2- 172.21.15.28:0/1911475075 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f9b84035780 0x7f9b84037c30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:28.964 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.412+0000 7f9ba4186640 1 --2- 172.21.15.28:0/1911475075 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b9c101e00 0x7f9b9c110a90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:28.964 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.412+0000 7f9ba4186640 1 -- 172.21.15.28:0/1911475075 >> 172.21.15.28:0/1911475075 conn(0x7f9b9c0fd5a0 msgr2=0x7f9b9c0fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:28.964 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.412+0000 7f9ba4186640 1 -- 172.21.15.28:0/1911475075 shutdown_connections 2024-07-03T15:35:28.964 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:28.412+0000 7f9ba4186640 1 -- 172.21.15.28:0/1911475075 wait complete. 2024-07-03T15:35:28.964 INFO:teuthology.orchestra.run.smithi028.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-07-03T15:35:28.965 INFO:teuthology.orchestra.run.smithi028.stdout:Adding key to root@localhost authorized_keys... 2024-07-03T15:35:28.965 INFO:teuthology.orchestra.run.smithi028.stdout:Adding host smithi028... 2024-07-03T15:35:29.620 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:29 smithi028 ceph-mon[29148]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:35:30.870 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:30 smithi028 ceph-mon[29148]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi028", "addr": "172.21.15.28", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:35:31.822 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:31 smithi028 ceph-mon[29148]: Deploying cephadm binary to smithi028 2024-07-03T15:35:33.600 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout Added host 'smithi028' with addr '172.21.15.28' 2024-07-03T15:35:33.600 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:29.361+0000 7ff740e46640 1 Processor -- start 2024-07-03T15:35:33.601 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:29.361+0000 7ff740e46640 1 -- start start 2024-07-03T15:35:33.601 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:29.361+0000 7ff740e46640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff73c101e00 0x7ff73c102210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:33.601 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:29.361+0000 7ff740e46640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73c102750 con 0x7ff73c101e00 2024-07-03T15:35:33.601 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:29.362+0000 7ff73a59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff73c101e00 0x7ff73c102210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:33.601 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:29.362+0000 7ff73a59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff73c101e00 0x7ff73c102210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48774/0 (socket says 172.21.15.28:48774) 2024-07-03T15:35:33.601 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:29.362+0000 7ff73a59c640 1 -- 172.21.15.28:0/1391043543 learned_addr learned my addr 172.21.15.28:0/1391043543 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:33.601 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:29.362+0000 7ff73a59c640 1 -- 172.21.15.28:0/1391043543 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff73c102890 con 0x7ff73c101e00 2024-07-03T15:35:33.601 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:29.363+0000 7ff73a59c640 1 --2- 172.21.15.28:0/1391043543 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff73c101e00 0x7ff73c102210 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff72c009840 tx=0x7ff72c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adaeabd89670356d server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:33.601 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:29.363+0000 7ff739d9b640 1 -- 172.21.15.28:0/1391043543 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff72c032440 con 0x7ff73c101e00 2024-07-03T15:35:33.601 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:29.363+0000 7ff739d9b640 1 -- 172.21.15.28:0/1391043543 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7ff72c032a30 con 0x7ff73c101e00 2024-07-03T15:35:33.601 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:29.363+0000 7ff739d9b640 1 -- 172.21.15.28:0/1391043543 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff72c031ad0 con 0x7ff73c101e00 2024-07-03T15:35:33.601 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:29.364+0000 7ff740e46640 1 -- 172.21.15.28:0/1391043543 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff73c101e00 msgr2=0x7ff73c102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:33.601 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:29.364+0000 7ff740e46640 1 --2- 172.21.15.28:0/1391043543 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff73c101e00 0x7ff73c102210 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff72c009840 tx=0x7ff72c0091d0 comp rx=0 tx=0).stop 2024-07-03T15:35:33.601 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:29.364+0000 7ff740e46640 1 -- 172.21.15.28:0/1391043543 shutdown_connections 2024-07-03T15:35:33.602 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:29.364+0000 7ff740e46640 1 --2- 172.21.15.28:0/1391043543 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff73c101e00 0x7ff73c102210 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:33.602 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:29.364+0000 7ff740e46640 1 -- 172.21.15.28:0/1391043543 >> 172.21.15.28:0/1391043543 conn(0x7ff73c0fd5a0 msgr2=0x7ff73c0ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:33.602 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:29.364+0000 7ff740e46640 1 -- 172.21.15.28:0/1391043543 shutdown_connections 2024-07-03T15:35:33.602 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:29.364+0000 7ff740e46640 1 -- 172.21.15.28:0/1391043543 wait complete. 2024-07-03T15:35:33.602 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:29.364+0000 7ff740e46640 1 Processor -- start 2024-07-03T15:35:33.602 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:29.364+0000 7ff740e46640 1 -- start start 2024-07-03T15:35:33.602 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:29.365+0000 7ff740e46640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff73c101e00 0x7ff73c110a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:33.602 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:29.365+0000 7ff740e46640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73c110fd0 con 0x7ff73c101e00 2024-07-03T15:35:33.602 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:29.365+0000 7ff73a59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff73c101e00 0x7ff73c110a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:33.603 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:29.365+0000 7ff73a59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff73c101e00 0x7ff73c110a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48788/0 (socket says 172.21.15.28:48788) 2024-07-03T15:35:33.603 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:29.365+0000 7ff73a59c640 1 -- 172.21.15.28:0/4117501501 learned_addr learned my addr 172.21.15.28:0/4117501501 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:33.603 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:29.365+0000 7ff73a59c640 1 -- 172.21.15.28:0/4117501501 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff72c009450 con 0x7ff73c101e00 2024-07-03T15:35:33.603 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:29.365+0000 7ff73a59c640 1 --2- 172.21.15.28:0/4117501501 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff73c101e00 0x7ff73c110a90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff72c0317b0 tx=0x7ff72c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:33.603 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:29.365+0000 7ff732ffd640 1 -- 172.21.15.28:0/4117501501 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff72c031e90 con 0x7ff73c101e00 2024-07-03T15:35:33.603 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:29.365+0000 7ff732ffd640 1 -- 172.21.15.28:0/4117501501 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7ff72c03daf0 con 0x7ff73c101e00 2024-07-03T15:35:33.603 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:29.365+0000 7ff740e46640 1 -- 172.21.15.28:0/4117501501 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff73c1111d0 con 0x7ff73c101e00 2024-07-03T15:35:33.603 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:29.365+0000 7ff732ffd640 1 -- 172.21.15.28:0/4117501501 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff72c03baa0 con 0x7ff73c101e00 2024-07-03T15:35:33.603 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:29.365+0000 7ff740e46640 1 -- 172.21.15.28:0/4117501501 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff73c111640 con 0x7ff73c101e00 2024-07-03T15:35:33.603 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:29.366+0000 7ff732ffd640 1 -- 172.21.15.28:0/4117501501 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 8) v1 ==== 42636+0+0 (secure 0 0 0) 0x7ff72c039070 con 0x7ff73c101e00 2024-07-03T15:35:33.603 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:29.366+0000 7ff732ffd640 1 --2- 172.21.15.28:0/4117501501 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7ff7280357d0 0x7ff728037c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:33.603 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:29.366+0000 7ff740e46640 1 -- 172.21.15.28:0/4117501501 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff73c04b530 con 0x7ff73c101e00 2024-07-03T15:35:33.603 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:29.366+0000 7ff732ffd640 1 -- 172.21.15.28:0/4117501501 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7ff72c0689b0 con 0x7ff73c101e00 2024-07-03T15:35:33.604 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:29.366+0000 7ff733fff640 1 --2- 172.21.15.28:0/4117501501 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7ff7280357d0 0x7ff728037c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:33.604 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:29.367+0000 7ff733fff640 1 --2- 172.21.15.28:0/4117501501 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7ff7280357d0 0x7ff728037c80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff724006fd0 tx=0x7ff724006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:33.604 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:29.371+0000 7ff732ffd640 1 -- 172.21.15.28:0/4117501501 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff72c04e940 con 0x7ff73c101e00 2024-07-03T15:35:33.604 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:29.553+0000 7ff740e46640 1 -- 172.21.15.28:0/4117501501 --> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi028", "addr": "172.21.15.28", "target": ["mon-mgr", ""]}) v1 -- 0x7ff73c0fe6f0 con 0x7ff7280357d0 2024-07-03T15:35:33.604 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:32.982+0000 7ff732ffd640 1 -- 172.21.15.28:0/4117501501 <== mgr.14118 v2:172.21.15.28:6800/1434683766 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7ff73c0fe6f0 con 0x7ff7280357d0 2024-07-03T15:35:33.604 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:32.985+0000 7ff740e46640 1 -- 172.21.15.28:0/4117501501 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7ff7280357d0 msgr2=0x7ff728037c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:33.604 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:32.985+0000 7ff740e46640 1 --2- 172.21.15.28:0/4117501501 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7ff7280357d0 0x7ff728037c80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff724006fd0 tx=0x7ff724006db0 comp rx=0 tx=0).stop 2024-07-03T15:35:33.604 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:32.985+0000 7ff740e46640 1 -- 172.21.15.28:0/4117501501 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff73c101e00 msgr2=0x7ff73c110a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:33.604 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:32.985+0000 7ff740e46640 1 --2- 172.21.15.28:0/4117501501 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff73c101e00 0x7ff73c110a90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff72c0317b0 tx=0x7ff72c008040 comp rx=0 tx=0).stop 2024-07-03T15:35:33.604 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:32.985+0000 7ff740e46640 1 -- 172.21.15.28:0/4117501501 shutdown_connections 2024-07-03T15:35:33.604 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:32.985+0000 7ff740e46640 1 --2- 172.21.15.28:0/4117501501 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7ff7280357d0 0x7ff728037c80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:33.604 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:32.985+0000 7ff740e46640 1 --2- 172.21.15.28:0/4117501501 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff73c101e00 0x7ff73c110a90 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:33.605 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:32.985+0000 7ff740e46640 1 -- 172.21.15.28:0/4117501501 >> 172.21.15.28:0/4117501501 conn(0x7ff73c0fd5a0 msgr2=0x7ff73c0fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:33.605 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:32.986+0000 7ff740e46640 1 -- 172.21.15.28:0/4117501501 shutdown_connections 2024-07-03T15:35:33.605 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:32.986+0000 7ff740e46640 1 -- 172.21.15.28:0/4117501501 wait complete. 2024-07-03T15:35:33.605 INFO:teuthology.orchestra.run.smithi028.stdout:Deploying mon service with default placement... 2024-07-03T15:35:34.203 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:33 smithi028 ceph-mon[29148]: from='mgr.14118 172.21.15.28:0/1044821505' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:35:34.203 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:33 smithi028 ceph-mon[29148]: Added host smithi028 2024-07-03T15:35:34.203 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:33 smithi028 ceph-mon[29148]: from='mgr.14118 172.21.15.28:0/1044821505' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:35:34.967 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-07-03T15:35:34.967 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:33.985+0000 7f4d06c9c640 1 Processor -- start 2024-07-03T15:35:34.968 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:33.985+0000 7f4d06c9c640 1 -- start start 2024-07-03T15:35:34.968 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:33.985+0000 7f4d06c9c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d00075b80 0x7f4d00104a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:34.968 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:33.985+0000 7f4d06c9c640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d00075f90 con 0x7f4d00075b80 2024-07-03T15:35:34.968 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:33.985+0000 7f4d04a38640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d00075b80 0x7f4d00104a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:34.968 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:33.985+0000 7f4d04a38640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d00075b80 0x7f4d00104a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35030/0 (socket says 172.21.15.28:35030) 2024-07-03T15:35:34.968 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:33.985+0000 7f4d04a38640 1 -- 172.21.15.28:0/4118366498 learned_addr learned my addr 172.21.15.28:0/4118366498 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:34.968 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:33.986+0000 7f4d04a38640 1 -- 172.21.15.28:0/4118366498 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d00104f90 con 0x7f4d00075b80 2024-07-03T15:35:34.968 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:33.986+0000 7f4d04a38640 1 --2- 172.21.15.28:0/4118366498 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d00075b80 0x7f4d00104a50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc009840 tx=0x7f4cfc0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33aa4b9293e68824 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:34.968 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:33.987+0000 7f4cf77fe640 1 -- 172.21.15.28:0/4118366498 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4cfc032440 con 0x7f4d00075b80 2024-07-03T15:35:34.968 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:33.987+0000 7f4cf77fe640 1 -- 172.21.15.28:0/4118366498 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f4cfc032a30 con 0x7f4d00075b80 2024-07-03T15:35:34.968 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:33.987+0000 7f4cf77fe640 1 -- 172.21.15.28:0/4118366498 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4cfc031ad0 con 0x7f4d00075b80 2024-07-03T15:35:34.968 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:33.987+0000 7f4d06c9c640 1 -- 172.21.15.28:0/4118366498 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d00075b80 msgr2=0x7f4d00104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:34.968 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:33.987+0000 7f4d06c9c640 1 --2- 172.21.15.28:0/4118366498 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d00075b80 0x7f4d00104a50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc009840 tx=0x7f4cfc0091d0 comp rx=0 tx=0).stop 2024-07-03T15:35:34.969 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:33.987+0000 7f4d06c9c640 1 -- 172.21.15.28:0/4118366498 shutdown_connections 2024-07-03T15:35:34.969 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:33.987+0000 7f4d06c9c640 1 --2- 172.21.15.28:0/4118366498 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d00075b80 0x7f4d00104a50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:34.969 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:33.987+0000 7f4d06c9c640 1 -- 172.21.15.28:0/4118366498 >> 172.21.15.28:0/4118366498 conn(0x7f4d000fc4c0 msgr2=0x7f4d000fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:34.969 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:33.987+0000 7f4d06c9c640 1 -- 172.21.15.28:0/4118366498 shutdown_connections 2024-07-03T15:35:34.969 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:33.987+0000 7f4d06c9c640 1 -- 172.21.15.28:0/4118366498 wait complete. 2024-07-03T15:35:34.969 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:33.988+0000 7f4d06c9c640 1 Processor -- start 2024-07-03T15:35:34.969 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:33.988+0000 7f4d06c9c640 1 -- start start 2024-07-03T15:35:34.969 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:33.988+0000 7f4d06c9c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d00075b80 0x7f4d00110aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:34.969 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:33.988+0000 7f4d06c9c640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d00110fe0 con 0x7f4d00075b80 2024-07-03T15:35:34.969 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:33.988+0000 7f4d04a38640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d00075b80 0x7f4d00110aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:34.969 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:33.988+0000 7f4d04a38640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d00075b80 0x7f4d00110aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35036/0 (socket says 172.21.15.28:35036) 2024-07-03T15:35:34.969 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:33.988+0000 7f4d04a38640 1 -- 172.21.15.28:0/2897564059 learned_addr learned my addr 172.21.15.28:0/2897564059 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:34.970 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:33.988+0000 7f4d04a38640 1 -- 172.21.15.28:0/2897564059 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cfc009450 con 0x7f4d00075b80 2024-07-03T15:35:34.970 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:33.989+0000 7f4d04a38640 1 --2- 172.21.15.28:0/2897564059 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d00075b80 0x7f4d00110aa0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc006f00 tx=0x7f4cfc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:34.970 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:33.989+0000 7f4cf57fa640 1 -- 172.21.15.28:0/2897564059 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4cfc03d450 con 0x7f4d00075b80 2024-07-03T15:35:34.970 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:33.989+0000 7f4d06c9c640 1 -- 172.21.15.28:0/2897564059 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d001111e0 con 0x7f4d00075b80 2024-07-03T15:35:34.970 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:33.989+0000 7f4cf57fa640 1 -- 172.21.15.28:0/2897564059 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f4cfc03da40 con 0x7f4d00075b80 2024-07-03T15:35:34.970 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:33.989+0000 7f4cf57fa640 1 -- 172.21.15.28:0/2897564059 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4cfc03bb10 con 0x7f4d00075b80 2024-07-03T15:35:34.970 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:33.989+0000 7f4d06c9c640 1 -- 172.21.15.28:0/2897564059 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d00111650 con 0x7f4d00075b80 2024-07-03T15:35:34.970 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:33.990+0000 7f4cf57fa640 1 -- 172.21.15.28:0/2897564059 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 8) v1 ==== 42636+0+0 (secure 0 0 0) 0x7f4cfc039070 con 0x7f4d00075b80 2024-07-03T15:35:34.970 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:33.990+0000 7f4cf57fa640 1 --2- 172.21.15.28:0/2897564059 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f4ce80357d0 0x7f4ce8037c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:34.970 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:33.990+0000 7f4cf57fa640 1 -- 172.21.15.28:0/2897564059 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f4cfc0688c0 con 0x7f4d00075b80 2024-07-03T15:35:34.970 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:33.990+0000 7f4d06c9c640 1 -- 172.21.15.28:0/2897564059 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d0005eb80 con 0x7f4d00075b80 2024-07-03T15:35:34.971 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:33.990+0000 7f4cf7fff640 1 --2- 172.21.15.28:0/2897564059 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f4ce80357d0 0x7f4ce8037c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:34.971 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:33.991+0000 7f4cf7fff640 1 --2- 172.21.15.28:0/2897564059 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f4ce80357d0 0x7f4ce8037c80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0006fd0 tx=0x7f4cf0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:34.971 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:33.995+0000 7f4cf57fa640 1 -- 172.21.15.28:0/2897564059 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4cfc041020 con 0x7f4d00075b80 2024-07-03T15:35:34.971 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:34.201+0000 7f4d06c9c640 1 -- 172.21.15.28:0/2897564059 --> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7f4d000fd600 con 0x7f4ce80357d0 2024-07-03T15:35:34.971 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:34.206+0000 7f4cf57fa640 1 -- 172.21.15.28:0/2897564059 <== mgr.14118 v2:172.21.15.28:6800/1434683766 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f4d000fd600 con 0x7f4ce80357d0 2024-07-03T15:35:34.971 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:34.208+0000 7f4d06c9c640 1 -- 172.21.15.28:0/2897564059 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f4ce80357d0 msgr2=0x7f4ce8037c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:34.971 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:34.208+0000 7f4d06c9c640 1 --2- 172.21.15.28:0/2897564059 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f4ce80357d0 0x7f4ce8037c80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0006fd0 tx=0x7f4cf0006db0 comp rx=0 tx=0).stop 2024-07-03T15:35:34.971 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:34.208+0000 7f4d06c9c640 1 -- 172.21.15.28:0/2897564059 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d00075b80 msgr2=0x7f4d00110aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:34.971 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:34.208+0000 7f4d06c9c640 1 --2- 172.21.15.28:0/2897564059 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d00075b80 0x7f4d00110aa0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc006f00 tx=0x7f4cfc008040 comp rx=0 tx=0).stop 2024-07-03T15:35:34.971 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:34.208+0000 7f4d06c9c640 1 -- 172.21.15.28:0/2897564059 shutdown_connections 2024-07-03T15:35:34.971 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:34.208+0000 7f4d06c9c640 1 --2- 172.21.15.28:0/2897564059 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f4ce80357d0 0x7f4ce8037c80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:34.971 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:34.208+0000 7f4d06c9c640 1 --2- 172.21.15.28:0/2897564059 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d00075b80 0x7f4d00110aa0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:34.972 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:34.208+0000 7f4d06c9c640 1 -- 172.21.15.28:0/2897564059 >> 172.21.15.28:0/2897564059 conn(0x7f4d000fc4c0 msgr2=0x7f4d000fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:34.972 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:34.208+0000 7f4d06c9c640 1 -- 172.21.15.28:0/2897564059 shutdown_connections 2024-07-03T15:35:34.972 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:34.208+0000 7f4d06c9c640 1 -- 172.21.15.28:0/2897564059 wait complete. 2024-07-03T15:35:34.972 INFO:teuthology.orchestra.run.smithi028.stdout:Deploying mgr service with default placement... 2024-07-03T15:35:35.296 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:35 smithi028 ceph-mon[29148]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:35:35.296 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:35 smithi028 ceph-mon[29148]: Saving service mon spec with placement count:5 2024-07-03T15:35:35.296 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:35 smithi028 ceph-mon[29148]: from='mgr.14118 172.21.15.28:0/1044821505' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:35:36.870 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:36 smithi028 ceph-mon[29148]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:35:36.870 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:36 smithi028 ceph-mon[29148]: Saving service mgr spec with placement count:2 2024-07-03T15:35:36.870 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:36 smithi028 ceph-mon[29148]: from='mgr.14118 172.21.15.28:0/1044821505' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:35:36.870 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:36 smithi028 ceph-mon[29148]: from='mgr.14118 172.21.15.28:0/1044821505' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:35:36.904 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-07-03T15:35:36.904 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.375+0000 7f782823b640 1 Processor -- start 2024-07-03T15:35:36.905 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.375+0000 7f782823b640 1 -- start start 2024-07-03T15:35:36.905 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.375+0000 7f782823b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78201021a0 0x7f78201025b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:36.905 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.375+0000 7f782823b640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7820102b80 con 0x7f78201021a0 2024-07-03T15:35:36.905 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.376+0000 7f7825fd7640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78201021a0 0x7f78201025b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:36.905 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.376+0000 7f7825fd7640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78201021a0 0x7f78201025b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35046/0 (socket says 172.21.15.28:35046) 2024-07-03T15:35:36.905 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.376+0000 7f7825fd7640 1 -- 172.21.15.28:0/757469183 learned_addr learned my addr 172.21.15.28:0/757469183 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:36.905 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.376+0000 7f7825fd7640 1 -- 172.21.15.28:0/757469183 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7820102cc0 con 0x7f78201021a0 2024-07-03T15:35:36.905 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.377+0000 7f7825fd7640 1 --2- 172.21.15.28:0/757469183 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78201021a0 0x7f78201025b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f781c009840 tx=0x7f781c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98c91bad134def45 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:36.905 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.377+0000 7f7824fd5640 1 -- 172.21.15.28:0/757469183 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f781c032440 con 0x7f78201021a0 2024-07-03T15:35:36.905 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.377+0000 7f7824fd5640 1 -- 172.21.15.28:0/757469183 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f781c032a30 con 0x7f78201021a0 2024-07-03T15:35:36.905 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.377+0000 7f7824fd5640 1 -- 172.21.15.28:0/757469183 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f781c031ad0 con 0x7f78201021a0 2024-07-03T15:35:36.905 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.377+0000 7f782823b640 1 -- 172.21.15.28:0/757469183 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78201021a0 msgr2=0x7f78201025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:36.906 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.377+0000 7f782823b640 1 --2- 172.21.15.28:0/757469183 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78201021a0 0x7f78201025b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f781c009840 tx=0x7f781c0091d0 comp rx=0 tx=0).stop 2024-07-03T15:35:36.906 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.378+0000 7f782823b640 1 -- 172.21.15.28:0/757469183 shutdown_connections 2024-07-03T15:35:36.906 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.378+0000 7f782823b640 1 --2- 172.21.15.28:0/757469183 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78201021a0 0x7f78201025b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:36.906 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.378+0000 7f782823b640 1 -- 172.21.15.28:0/757469183 >> 172.21.15.28:0/757469183 conn(0x7f78200fbc20 msgr2=0x7f78200fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:36.906 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.378+0000 7f782823b640 1 -- 172.21.15.28:0/757469183 shutdown_connections 2024-07-03T15:35:36.906 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.378+0000 7f782823b640 1 -- 172.21.15.28:0/757469183 wait complete. 2024-07-03T15:35:36.906 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.378+0000 7f782823b640 1 Processor -- start 2024-07-03T15:35:36.906 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.378+0000 7f782823b640 1 -- start start 2024-07-03T15:35:36.906 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.378+0000 7f782823b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78201021a0 0x7f7820112c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:36.906 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.378+0000 7f782823b640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78201131a0 con 0x7f78201021a0 2024-07-03T15:35:36.906 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.379+0000 7f7825fd7640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78201021a0 0x7f7820112c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:36.906 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.379+0000 7f7825fd7640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78201021a0 0x7f7820112c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35052/0 (socket says 172.21.15.28:35052) 2024-07-03T15:35:36.906 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.379+0000 7f7825fd7640 1 -- 172.21.15.28:0/3831725730 learned_addr learned my addr 172.21.15.28:0/3831725730 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:36.907 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.379+0000 7f7825fd7640 1 -- 172.21.15.28:0/3831725730 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f781c009450 con 0x7f78201021a0 2024-07-03T15:35:36.907 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.379+0000 7f7825fd7640 1 --2- 172.21.15.28:0/3831725730 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78201021a0 0x7f7820112c60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f781c031b90 tx=0x7f781c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:36.907 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.379+0000 7f78167fc640 1 -- 172.21.15.28:0/3831725730 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f781c0328b0 con 0x7f78201021a0 2024-07-03T15:35:36.907 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.379+0000 7f782823b640 1 -- 172.21.15.28:0/3831725730 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78201133a0 con 0x7f78201021a0 2024-07-03T15:35:36.907 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.379+0000 7f78167fc640 1 -- 172.21.15.28:0/3831725730 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f781c03d8e0 con 0x7f78201021a0 2024-07-03T15:35:36.907 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.379+0000 7f78167fc640 1 -- 172.21.15.28:0/3831725730 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f781c03ba20 con 0x7f78201021a0 2024-07-03T15:35:36.907 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.379+0000 7f782823b640 1 -- 172.21.15.28:0/3831725730 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7820113810 con 0x7f78201021a0 2024-07-03T15:35:36.907 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.380+0000 7f78167fc640 1 -- 172.21.15.28:0/3831725730 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 8) v1 ==== 42636+0+0 (secure 0 0 0) 0x7f781c039070 con 0x7f78201021a0 2024-07-03T15:35:36.907 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.380+0000 7f78167fc640 1 --2- 172.21.15.28:0/3831725730 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f780c035780 0x7f780c037c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:36.907 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.380+0000 7f782823b640 1 -- 172.21.15.28:0/3831725730 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f782004b530 con 0x7f78201021a0 2024-07-03T15:35:36.907 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.380+0000 7f78167fc640 1 -- 172.21.15.28:0/3831725730 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f781c068970 con 0x7f78201021a0 2024-07-03T15:35:36.907 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.380+0000 7f78257d6640 1 --2- 172.21.15.28:0/3831725730 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f780c035780 0x7f780c037c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:36.907 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.381+0000 7f78257d6640 1 --2- 172.21.15.28:0/3831725730 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f780c035780 0x7f780c037c30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7810006fd0 tx=0x7f7810006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:36.908 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.385+0000 7f78167fc640 1 -- 172.21.15.28:0/3831725730 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f781c041020 con 0x7f78201021a0 2024-07-03T15:35:36.908 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.572+0000 7f782823b640 1 -- 172.21.15.28:0/3831725730 --> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f78200fcdd0 con 0x7f780c035780 2024-07-03T15:35:36.908 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.576+0000 7f78167fc640 1 -- 172.21.15.28:0/3831725730 <== mgr.14118 v2:172.21.15.28:6800/1434683766 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f78200fcdd0 con 0x7f780c035780 2024-07-03T15:35:36.908 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.578+0000 7f782823b640 1 -- 172.21.15.28:0/3831725730 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f780c035780 msgr2=0x7f780c037c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:36.908 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.578+0000 7f782823b640 1 --2- 172.21.15.28:0/3831725730 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f780c035780 0x7f780c037c30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7810006fd0 tx=0x7f7810006db0 comp rx=0 tx=0).stop 2024-07-03T15:35:36.908 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.578+0000 7f782823b640 1 -- 172.21.15.28:0/3831725730 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78201021a0 msgr2=0x7f7820112c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:36.908 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.578+0000 7f782823b640 1 --2- 172.21.15.28:0/3831725730 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78201021a0 0x7f7820112c60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f781c031b90 tx=0x7f781c008040 comp rx=0 tx=0).stop 2024-07-03T15:35:36.908 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.578+0000 7f782823b640 1 -- 172.21.15.28:0/3831725730 shutdown_connections 2024-07-03T15:35:36.908 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.578+0000 7f782823b640 1 --2- 172.21.15.28:0/3831725730 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f780c035780 0x7f780c037c30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:36.908 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.578+0000 7f782823b640 1 --2- 172.21.15.28:0/3831725730 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78201021a0 0x7f7820112c60 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:36.908 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.578+0000 7f782823b640 1 -- 172.21.15.28:0/3831725730 >> 172.21.15.28:0/3831725730 conn(0x7f78200fbc20 msgr2=0x7f78200fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:36.908 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.578+0000 7f782823b640 1 -- 172.21.15.28:0/3831725730 shutdown_connections 2024-07-03T15:35:36.909 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:35.578+0000 7f782823b640 1 -- 172.21.15.28:0/3831725730 wait complete. 2024-07-03T15:35:36.909 INFO:teuthology.orchestra.run.smithi028.stdout:Deploying crash service with default placement... 2024-07-03T15:35:37.773 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:37 smithi028 ceph-mon[29148]: from='mgr.14118 172.21.15.28:0/1044821505' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:35:37.774 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:37 smithi028 ceph-mon[29148]: from='mgr.14118 172.21.15.28:0/1044821505' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:35:38.613 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-07-03T15:35:38.613 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.282+0000 7f811f264640 1 Processor -- start 2024-07-03T15:35:38.613 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.282+0000 7f811f264640 1 -- start start 2024-07-03T15:35:38.613 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.283+0000 7f811f264640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8118075ba0 0x7f8118106e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:38.613 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.283+0000 7f811f264640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8118102820 con 0x7f8118075ba0 2024-07-03T15:35:38.613 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.283+0000 7f811d000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8118075ba0 0x7f8118106e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:38.613 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.283+0000 7f811d000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8118075ba0 0x7f8118106e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35054/0 (socket says 172.21.15.28:35054) 2024-07-03T15:35:38.613 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.283+0000 7f811d000640 1 -- 172.21.15.28:0/2895879824 learned_addr learned my addr 172.21.15.28:0/2895879824 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:38.614 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.283+0000 7f811d000640 1 -- 172.21.15.28:0/2895879824 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8118107370 con 0x7f8118075ba0 2024-07-03T15:35:38.614 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.284+0000 7f811d000640 1 --2- 172.21.15.28:0/2895879824 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8118075ba0 0x7f8118106e30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8110009e50 tx=0x7f8110009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395eb80a8d8f50ae server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:38.614 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.284+0000 7f81177fe640 1 -- 172.21.15.28:0/2895879824 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8110031560 con 0x7f8118075ba0 2024-07-03T15:35:38.614 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.284+0000 7f81177fe640 1 -- 172.21.15.28:0/2895879824 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f811002d050 con 0x7f8118075ba0 2024-07-03T15:35:38.614 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.284+0000 7f81177fe640 1 -- 172.21.15.28:0/2895879824 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8110007ad0 con 0x7f8118075ba0 2024-07-03T15:35:38.614 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.285+0000 7f811f264640 1 -- 172.21.15.28:0/2895879824 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8118075ba0 msgr2=0x7f8118106e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:38.614 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.285+0000 7f811f264640 1 --2- 172.21.15.28:0/2895879824 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8118075ba0 0x7f8118106e30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8110009e50 tx=0x7f8110009840 comp rx=0 tx=0).stop 2024-07-03T15:35:38.614 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.285+0000 7f811f264640 1 -- 172.21.15.28:0/2895879824 shutdown_connections 2024-07-03T15:35:38.614 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.285+0000 7f811f264640 1 --2- 172.21.15.28:0/2895879824 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8118075ba0 0x7f8118106e30 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:38.614 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.285+0000 7f811f264640 1 -- 172.21.15.28:0/2895879824 >> 172.21.15.28:0/2895879824 conn(0x7f81180fc4c0 msgr2=0x7f81180fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:38.614 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.285+0000 7f811f264640 1 -- 172.21.15.28:0/2895879824 shutdown_connections 2024-07-03T15:35:38.614 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.285+0000 7f811f264640 1 -- 172.21.15.28:0/2895879824 wait complete. 2024-07-03T15:35:38.615 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.285+0000 7f811f264640 1 Processor -- start 2024-07-03T15:35:38.615 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.286+0000 7f811f264640 1 -- start start 2024-07-03T15:35:38.615 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.286+0000 7f811f264640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8118075ba0 0x7f8118112e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:38.615 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.286+0000 7f811f264640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8118113340 con 0x7f8118075ba0 2024-07-03T15:35:38.615 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.286+0000 7f811d000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8118075ba0 0x7f8118112e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:38.615 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.286+0000 7f811d000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8118075ba0 0x7f8118112e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35064/0 (socket says 172.21.15.28:35064) 2024-07-03T15:35:38.615 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.286+0000 7f811d000640 1 -- 172.21.15.28:0/4022540968 learned_addr learned my addr 172.21.15.28:0/4022540968 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:38.615 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.286+0000 7f811d000640 1 -- 172.21.15.28:0/4022540968 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8110009ae0 con 0x7f8118075ba0 2024-07-03T15:35:38.615 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.286+0000 7f811d000640 1 --2- 172.21.15.28:0/4022540968 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8118075ba0 0x7f8118112e00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8110009e20 tx=0x7f8110009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:38.615 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.287+0000 7f81157fa640 1 -- 172.21.15.28:0/4022540968 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f811003d450 con 0x7f8118075ba0 2024-07-03T15:35:38.615 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.287+0000 7f81157fa640 1 -- 172.21.15.28:0/4022540968 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f811003da80 con 0x7f8118075ba0 2024-07-03T15:35:38.615 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.287+0000 7f811f264640 1 -- 172.21.15.28:0/4022540968 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8118113540 con 0x7f8118075ba0 2024-07-03T15:35:38.615 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.287+0000 7f81157fa640 1 -- 172.21.15.28:0/4022540968 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f811003b920 con 0x7f8118075ba0 2024-07-03T15:35:38.616 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.287+0000 7f811f264640 1 -- 172.21.15.28:0/4022540968 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8118113f90 con 0x7f8118075ba0 2024-07-03T15:35:38.616 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.287+0000 7f81157fa640 1 -- 172.21.15.28:0/4022540968 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 8) v1 ==== 42636+0+0 (secure 0 0 0) 0x7f811003baa0 con 0x7f8118075ba0 2024-07-03T15:35:38.616 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.288+0000 7f811f264640 1 -- 172.21.15.28:0/4022540968 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f811805eb80 con 0x7f8118075ba0 2024-07-03T15:35:38.616 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.288+0000 7f81157fa640 1 --2- 172.21.15.28:0/4022540968 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f81000357d0 0x7f8100037c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:38.616 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.288+0000 7f81157fa640 1 -- 172.21.15.28:0/4022540968 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f8110068d80 con 0x7f8118075ba0 2024-07-03T15:35:38.616 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.288+0000 7f8117fff640 1 --2- 172.21.15.28:0/4022540968 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f81000357d0 0x7f8100037c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:38.616 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.288+0000 7f8117fff640 1 --2- 172.21.15.28:0/4022540968 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f81000357d0 0x7f8100037c80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f810c006fd0 tx=0x7f810c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:38.616 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.292+0000 7f81157fa640 1 -- 172.21.15.28:0/4022540968 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f811003d790 con 0x7f8118075ba0 2024-07-03T15:35:38.616 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.487+0000 7f811f264640 1 -- 172.21.15.28:0/4022540968 --> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7f81180fd6c0 con 0x7f81000357d0 2024-07-03T15:35:38.616 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.492+0000 7f81157fa640 1 -- 172.21.15.28:0/4022540968 <== mgr.14118 v2:172.21.15.28:6800/1434683766 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f81180fd6c0 con 0x7f81000357d0 2024-07-03T15:35:38.616 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.494+0000 7f811f264640 1 -- 172.21.15.28:0/4022540968 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f81000357d0 msgr2=0x7f8100037c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:38.616 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.494+0000 7f811f264640 1 --2- 172.21.15.28:0/4022540968 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f81000357d0 0x7f8100037c80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f810c006fd0 tx=0x7f810c006db0 comp rx=0 tx=0).stop 2024-07-03T15:35:38.617 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.494+0000 7f811f264640 1 -- 172.21.15.28:0/4022540968 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8118075ba0 msgr2=0x7f8118112e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:38.617 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.494+0000 7f811f264640 1 --2- 172.21.15.28:0/4022540968 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8118075ba0 0x7f8118112e00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8110009e20 tx=0x7f8110009840 comp rx=0 tx=0).stop 2024-07-03T15:35:38.617 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.494+0000 7f811f264640 1 -- 172.21.15.28:0/4022540968 shutdown_connections 2024-07-03T15:35:38.617 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.494+0000 7f811f264640 1 --2- 172.21.15.28:0/4022540968 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f81000357d0 0x7f8100037c80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:38.617 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.494+0000 7f811f264640 1 --2- 172.21.15.28:0/4022540968 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8118075ba0 0x7f8118112e00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:38.617 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.494+0000 7f811f264640 1 -- 172.21.15.28:0/4022540968 >> 172.21.15.28:0/4022540968 conn(0x7f81180fc4c0 msgr2=0x7f81180fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:38.617 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.494+0000 7f811f264640 1 -- 172.21.15.28:0/4022540968 shutdown_connections 2024-07-03T15:35:38.617 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:37.494+0000 7f811f264640 1 -- 172.21.15.28:0/4022540968 wait complete. 2024-07-03T15:35:38.617 INFO:teuthology.orchestra.run.smithi028.stdout:Deploying prometheus service with default placement... 2024-07-03T15:35:38.870 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:38 smithi028 ceph-mon[29148]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:35:38.870 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:38 smithi028 ceph-mon[29148]: Saving service crash spec with placement * 2024-07-03T15:35:38.870 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:38 smithi028 ceph-mon[29148]: from='mgr.14118 172.21.15.28:0/1044821505' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:35:39.928 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-07-03T15:35:39.928 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:38.984+0000 7f014af8b640 1 Processor -- start 2024-07-03T15:35:39.928 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:38.984+0000 7f014af8b640 1 -- start start 2024-07-03T15:35:39.928 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:38.984+0000 7f014af8b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0144075ba0 0x7f0144104a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:39.928 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:38.985+0000 7f014af8b640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0144104f70 con 0x7f0144075ba0 2024-07-03T15:35:39.928 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:38.985+0000 7f0148d27640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0144075ba0 0x7f0144104a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:39.928 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:38.985+0000 7f0148d27640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0144075ba0 0x7f0144104a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35066/0 (socket says 172.21.15.28:35066) 2024-07-03T15:35:39.928 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:38.985+0000 7f0148d27640 1 -- 172.21.15.28:0/3392674988 learned_addr learned my addr 172.21.15.28:0/3392674988 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:39.928 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:38.985+0000 7f0148d27640 1 -- 172.21.15.28:0/3392674988 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01441050b0 con 0x7f0144075ba0 2024-07-03T15:35:39.929 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:38.986+0000 7f0148d27640 1 --2- 172.21.15.28:0/3392674988 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0144075ba0 0x7f0144104a30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0138009840 tx=0x7f01380091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af31cbb29fa3f9fe server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:39.929 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:38.986+0000 7f01437fe640 1 -- 172.21.15.28:0/3392674988 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0138032440 con 0x7f0144075ba0 2024-07-03T15:35:39.929 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:38.986+0000 7f01437fe640 1 -- 172.21.15.28:0/3392674988 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0138032a70 con 0x7f0144075ba0 2024-07-03T15:35:39.929 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:38.986+0000 7f01437fe640 1 -- 172.21.15.28:0/3392674988 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0138031ad0 con 0x7f0144075ba0 2024-07-03T15:35:39.929 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:38.986+0000 7f014af8b640 1 -- 172.21.15.28:0/3392674988 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0144075ba0 msgr2=0x7f0144104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:39.929 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:38.986+0000 7f014af8b640 1 --2- 172.21.15.28:0/3392674988 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0144075ba0 0x7f0144104a30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0138009840 tx=0x7f01380091d0 comp rx=0 tx=0).stop 2024-07-03T15:35:39.929 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:38.986+0000 7f014af8b640 1 -- 172.21.15.28:0/3392674988 shutdown_connections 2024-07-03T15:35:39.929 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:38.986+0000 7f014af8b640 1 --2- 172.21.15.28:0/3392674988 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0144075ba0 0x7f0144104a30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:39.929 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:38.986+0000 7f014af8b640 1 -- 172.21.15.28:0/3392674988 >> 172.21.15.28:0/3392674988 conn(0x7f01440fc4c0 msgr2=0x7f01440fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:39.929 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:38.986+0000 7f014af8b640 1 -- 172.21.15.28:0/3392674988 shutdown_connections 2024-07-03T15:35:39.929 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:38.986+0000 7f014af8b640 1 -- 172.21.15.28:0/3392674988 wait complete. 2024-07-03T15:35:39.929 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:38.987+0000 7f014af8b640 1 Processor -- start 2024-07-03T15:35:39.930 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:38.987+0000 7f014af8b640 1 -- start start 2024-07-03T15:35:39.930 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:38.987+0000 7f014af8b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0144075ba0 0x7f0144102360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:39.930 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:38.987+0000 7f014af8b640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01441028a0 con 0x7f0144075ba0 2024-07-03T15:35:39.930 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:38.987+0000 7f0148d27640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0144075ba0 0x7f0144102360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:39.930 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:38.987+0000 7f0148d27640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0144075ba0 0x7f0144102360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35074/0 (socket says 172.21.15.28:35074) 2024-07-03T15:35:39.930 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:38.987+0000 7f0148d27640 1 -- 172.21.15.28:0/2312175585 learned_addr learned my addr 172.21.15.28:0/2312175585 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:39.930 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:38.988+0000 7f0148d27640 1 -- 172.21.15.28:0/2312175585 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0138009450 con 0x7f0144075ba0 2024-07-03T15:35:39.930 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:38.988+0000 7f0148d27640 1 --2- 172.21.15.28:0/2312175585 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0144075ba0 0x7f0144102360 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f01380317f0 tx=0x7f013802c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:39.930 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:38.988+0000 7f01417fa640 1 -- 172.21.15.28:0/2312175585 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0138032900 con 0x7f0144075ba0 2024-07-03T15:35:39.930 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:38.988+0000 7f01417fa640 1 -- 172.21.15.28:0/2312175585 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f013803b910 con 0x7f0144075ba0 2024-07-03T15:35:39.930 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:38.988+0000 7f014af8b640 1 -- 172.21.15.28:0/2312175585 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0144100930 con 0x7f0144075ba0 2024-07-03T15:35:39.930 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:38.988+0000 7f01417fa640 1 -- 172.21.15.28:0/2312175585 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f013803a960 con 0x7f0144075ba0 2024-07-03T15:35:39.930 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:38.988+0000 7f014af8b640 1 -- 172.21.15.28:0/2312175585 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0144100da0 con 0x7f0144075ba0 2024-07-03T15:35:39.931 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:38.989+0000 7f01417fa640 1 -- 172.21.15.28:0/2312175585 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 8) v1 ==== 42636+0+0 (secure 0 0 0) 0x7f0138039070 con 0x7f0144075ba0 2024-07-03T15:35:39.931 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:38.989+0000 7f01417fa640 1 --2- 172.21.15.28:0/2312175585 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f0134035780 0x7f0134037c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:39.931 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:38.989+0000 7f01417fa640 1 -- 172.21.15.28:0/2312175585 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f0138068060 con 0x7f0144075ba0 2024-07-03T15:35:39.931 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:38.989+0000 7f0143fff640 1 --2- 172.21.15.28:0/2312175585 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f0134035780 0x7f0134037c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:39.931 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:38.989+0000 7f014af8b640 1 -- 172.21.15.28:0/2312175585 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f014404b530 con 0x7f0144075ba0 2024-07-03T15:35:39.931 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:38.989+0000 7f0143fff640 1 --2- 172.21.15.28:0/2312175585 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f0134035780 0x7f0134037c30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0130006fd0 tx=0x7f0130006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:39.931 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:38.994+0000 7f01417fa640 1 -- 172.21.15.28:0/2312175585 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f013803f080 con 0x7f0144075ba0 2024-07-03T15:35:39.931 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:39.179+0000 7f014af8b640 1 -- 172.21.15.28:0/2312175585 --> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7f0144101320 con 0x7f0134035780 2024-07-03T15:35:39.931 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:39.183+0000 7f01417fa640 1 -- 172.21.15.28:0/2312175585 <== mgr.14118 v2:172.21.15.28:6800/1434683766 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7f0144101320 con 0x7f0134035780 2024-07-03T15:35:39.931 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:39.185+0000 7f014af8b640 1 -- 172.21.15.28:0/2312175585 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f0134035780 msgr2=0x7f0134037c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:39.931 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:39.185+0000 7f014af8b640 1 --2- 172.21.15.28:0/2312175585 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f0134035780 0x7f0134037c30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0130006fd0 tx=0x7f0130006db0 comp rx=0 tx=0).stop 2024-07-03T15:35:39.931 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:39.185+0000 7f014af8b640 1 -- 172.21.15.28:0/2312175585 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0144075ba0 msgr2=0x7f0144102360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:39.931 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:39.185+0000 7f014af8b640 1 --2- 172.21.15.28:0/2312175585 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0144075ba0 0x7f0144102360 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f01380317f0 tx=0x7f013802c040 comp rx=0 tx=0).stop 2024-07-03T15:35:39.932 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:39.186+0000 7f014af8b640 1 -- 172.21.15.28:0/2312175585 shutdown_connections 2024-07-03T15:35:39.932 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:39.186+0000 7f014af8b640 1 --2- 172.21.15.28:0/2312175585 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f0134035780 0x7f0134037c30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:39.932 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:39.186+0000 7f014af8b640 1 --2- 172.21.15.28:0/2312175585 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0144075ba0 0x7f0144102360 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:39.932 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:39.186+0000 7f014af8b640 1 -- 172.21.15.28:0/2312175585 >> 172.21.15.28:0/2312175585 conn(0x7f01440fc4c0 msgr2=0x7f01440fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:39.932 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:39.186+0000 7f014af8b640 1 -- 172.21.15.28:0/2312175585 shutdown_connections 2024-07-03T15:35:39.932 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:39.186+0000 7f014af8b640 1 -- 172.21.15.28:0/2312175585 wait complete. 2024-07-03T15:35:39.932 INFO:teuthology.orchestra.run.smithi028.stdout:Deploying grafana service with default placement... 2024-07-03T15:35:40.560 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:40 smithi028 ceph-mon[29148]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:35:40.560 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:40 smithi028 ceph-mon[29148]: Saving service prometheus spec with placement count:1 2024-07-03T15:35:40.560 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:40 smithi028 ceph-mon[29148]: from='mgr.14118 172.21.15.28:0/1044821505' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:35:40.560 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:40 smithi028 ceph-mon[29148]: from='mgr.14118 172.21.15.28:0/1044821505' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:35:41.102 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-07-03T15:35:41.102 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.359+0000 7f4db86ac640 1 Processor -- start 2024-07-03T15:35:41.102 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.359+0000 7f4db86ac640 1 -- start start 2024-07-03T15:35:41.102 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.359+0000 7f4db86ac640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4db0102210 0x7f4db0102620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:41.102 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.359+0000 7f4db86ac640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db0102bf0 con 0x7f4db0102210 2024-07-03T15:35:41.102 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.359+0000 7f4db6448640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4db0102210 0x7f4db0102620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:41.102 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.359+0000 7f4db6448640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4db0102210 0x7f4db0102620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37936/0 (socket says 172.21.15.28:37936) 2024-07-03T15:35:41.102 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.359+0000 7f4db6448640 1 -- 172.21.15.28:0/908171971 learned_addr learned my addr 172.21.15.28:0/908171971 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:41.102 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.360+0000 7f4db6448640 1 -- 172.21.15.28:0/908171971 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db0102d30 con 0x7f4db0102210 2024-07-03T15:35:41.103 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.360+0000 7f4db6448640 1 --2- 172.21.15.28:0/908171971 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4db0102210 0x7f4db0102620 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4dac009840 tx=0x7f4dac0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=496300b25ffadcf9 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:41.103 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.360+0000 7f4db5446640 1 -- 172.21.15.28:0/908171971 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4dac032440 con 0x7f4db0102210 2024-07-03T15:35:41.103 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.361+0000 7f4db5446640 1 -- 172.21.15.28:0/908171971 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4dac032a70 con 0x7f4db0102210 2024-07-03T15:35:41.103 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.361+0000 7f4db5446640 1 -- 172.21.15.28:0/908171971 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4dac031ad0 con 0x7f4db0102210 2024-07-03T15:35:41.103 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.361+0000 7f4db86ac640 1 -- 172.21.15.28:0/908171971 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4db0102210 msgr2=0x7f4db0102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:41.103 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.361+0000 7f4db86ac640 1 --2- 172.21.15.28:0/908171971 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4db0102210 0x7f4db0102620 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4dac009840 tx=0x7f4dac0091d0 comp rx=0 tx=0).stop 2024-07-03T15:35:41.103 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.361+0000 7f4db86ac640 1 -- 172.21.15.28:0/908171971 shutdown_connections 2024-07-03T15:35:41.103 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.361+0000 7f4db86ac640 1 --2- 172.21.15.28:0/908171971 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4db0102210 0x7f4db0102620 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:41.103 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.361+0000 7f4db86ac640 1 -- 172.21.15.28:0/908171971 >> 172.21.15.28:0/908171971 conn(0x7f4db00fbc50 msgr2=0x7f4db00fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:41.103 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.361+0000 7f4db86ac640 1 -- 172.21.15.28:0/908171971 shutdown_connections 2024-07-03T15:35:41.103 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.361+0000 7f4db86ac640 1 -- 172.21.15.28:0/908171971 wait complete. 2024-07-03T15:35:41.103 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.362+0000 7f4db86ac640 1 Processor -- start 2024-07-03T15:35:41.104 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.362+0000 7f4db86ac640 1 -- start start 2024-07-03T15:35:41.104 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.362+0000 7f4db86ac640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4db0102210 0x7f4db0112e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:41.104 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.362+0000 7f4db86ac640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db0113360 con 0x7f4db0102210 2024-07-03T15:35:41.104 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.362+0000 7f4db6448640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4db0102210 0x7f4db0112e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:41.104 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.362+0000 7f4db6448640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4db0102210 0x7f4db0112e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37944/0 (socket says 172.21.15.28:37944) 2024-07-03T15:35:41.104 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.362+0000 7f4db6448640 1 -- 172.21.15.28:0/373283766 learned_addr learned my addr 172.21.15.28:0/373283766 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:41.104 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.362+0000 7f4db6448640 1 -- 172.21.15.28:0/373283766 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dac009450 con 0x7f4db0102210 2024-07-03T15:35:41.104 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.362+0000 7f4db6448640 1 --2- 172.21.15.28:0/373283766 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4db0102210 0x7f4db0112e20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4dac0098a0 tx=0x7f4dac02c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:41.104 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.363+0000 7f4da6ffd640 1 -- 172.21.15.28:0/373283766 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4dac031e90 con 0x7f4db0102210 2024-07-03T15:35:41.104 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.363+0000 7f4da6ffd640 1 -- 172.21.15.28:0/373283766 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4dac03b910 con 0x7f4db0102210 2024-07-03T15:35:41.104 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.363+0000 7f4db86ac640 1 -- 172.21.15.28:0/373283766 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4db0113560 con 0x7f4db0102210 2024-07-03T15:35:41.104 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.363+0000 7f4da6ffd640 1 -- 172.21.15.28:0/373283766 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4dac03a9b0 con 0x7f4db0102210 2024-07-03T15:35:41.105 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.363+0000 7f4db86ac640 1 -- 172.21.15.28:0/373283766 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4db0113fb0 con 0x7f4db0102210 2024-07-03T15:35:41.105 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.363+0000 7f4da6ffd640 1 -- 172.21.15.28:0/373283766 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 8) v1 ==== 42636+0+0 (secure 0 0 0) 0x7f4dac039070 con 0x7f4db0102210 2024-07-03T15:35:41.105 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.364+0000 7f4da6ffd640 1 --2- 172.21.15.28:0/373283766 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f4d9c035780 0x7f4d9c037c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:41.105 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.364+0000 7f4da6ffd640 1 -- 172.21.15.28:0/373283766 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f4dac03a3a0 con 0x7f4db0102210 2024-07-03T15:35:41.105 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.364+0000 7f4db5c47640 1 --2- 172.21.15.28:0/373283766 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f4d9c035780 0x7f4d9c037c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:41.105 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.364+0000 7f4db86ac640 1 -- 172.21.15.28:0/373283766 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4db005eb80 con 0x7f4db0102210 2024-07-03T15:35:41.105 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.364+0000 7f4db5c47640 1 --2- 172.21.15.28:0/373283766 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f4d9c035780 0x7f4d9c037c30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4da0006fd0 tx=0x7f4da0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:41.105 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.368+0000 7f4da6ffd640 1 -- 172.21.15.28:0/373283766 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4dac03f080 con 0x7f4db0102210 2024-07-03T15:35:41.105 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.558+0000 7f4db86ac640 1 -- 172.21.15.28:0/373283766 --> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7f4db00fce90 con 0x7f4d9c035780 2024-07-03T15:35:41.105 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.562+0000 7f4da6ffd640 1 -- 172.21.15.28:0/373283766 <== mgr.14118 v2:172.21.15.28:6800/1434683766 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f4db00fce90 con 0x7f4d9c035780 2024-07-03T15:35:41.105 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.563+0000 7f4db86ac640 1 -- 172.21.15.28:0/373283766 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f4d9c035780 msgr2=0x7f4d9c037c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:41.106 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.563+0000 7f4db86ac640 1 --2- 172.21.15.28:0/373283766 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f4d9c035780 0x7f4d9c037c30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4da0006fd0 tx=0x7f4da0006db0 comp rx=0 tx=0).stop 2024-07-03T15:35:41.106 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.563+0000 7f4db86ac640 1 -- 172.21.15.28:0/373283766 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4db0102210 msgr2=0x7f4db0112e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:41.106 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.563+0000 7f4db86ac640 1 --2- 172.21.15.28:0/373283766 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4db0102210 0x7f4db0112e20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4dac0098a0 tx=0x7f4dac02c040 comp rx=0 tx=0).stop 2024-07-03T15:35:41.106 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.564+0000 7f4db86ac640 1 -- 172.21.15.28:0/373283766 shutdown_connections 2024-07-03T15:35:41.106 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.564+0000 7f4db86ac640 1 --2- 172.21.15.28:0/373283766 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f4d9c035780 0x7f4d9c037c30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:41.106 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.564+0000 7f4db86ac640 1 --2- 172.21.15.28:0/373283766 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4db0102210 0x7f4db0112e20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:41.106 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.564+0000 7f4db86ac640 1 -- 172.21.15.28:0/373283766 >> 172.21.15.28:0/373283766 conn(0x7f4db00fbc50 msgr2=0x7f4db00fc780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:41.106 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.564+0000 7f4db86ac640 1 -- 172.21.15.28:0/373283766 shutdown_connections 2024-07-03T15:35:41.106 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:40.564+0000 7f4db86ac640 1 -- 172.21.15.28:0/373283766 wait complete. 2024-07-03T15:35:41.106 INFO:teuthology.orchestra.run.smithi028.stdout:Deploying node-exporter service with default placement... 2024-07-03T15:35:41.870 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:41 smithi028 ceph-mon[29148]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:35:41.870 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:41 smithi028 ceph-mon[29148]: Saving service grafana spec with placement count:1 2024-07-03T15:35:41.870 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:41 smithi028 ceph-mon[29148]: from='mgr.14118 172.21.15.28:0/1044821505' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:35:42.997 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-07-03T15:35:42.998 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.009+0000 7f333ca0d640 1 Processor -- start 2024-07-03T15:35:42.998 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.009+0000 7f333ca0d640 1 -- start start 2024-07-03T15:35:42.998 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.010+0000 7f333ca0d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3338075b80 0x7f3338104a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:42.998 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.010+0000 7f333ca0d640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3338075f90 con 0x7f3338075b80 2024-07-03T15:35:42.998 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.010+0000 7f333659c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3338075b80 0x7f3338104a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:42.998 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.010+0000 7f333659c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3338075b80 0x7f3338104a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37960/0 (socket says 172.21.15.28:37960) 2024-07-03T15:35:42.998 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.010+0000 7f333659c640 1 -- 172.21.15.28:0/2548167106 learned_addr learned my addr 172.21.15.28:0/2548167106 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:42.998 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.010+0000 7f333659c640 1 -- 172.21.15.28:0/2548167106 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3338104f90 con 0x7f3338075b80 2024-07-03T15:35:42.998 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.011+0000 7f333659c640 1 --2- 172.21.15.28:0/2548167106 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3338075b80 0x7f3338104a50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3330009840 tx=0x7f33300091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33df783bd9202dda server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:42.998 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.011+0000 7f333559a640 1 -- 172.21.15.28:0/2548167106 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3330032440 con 0x7f3338075b80 2024-07-03T15:35:42.999 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.011+0000 7f333559a640 1 -- 172.21.15.28:0/2548167106 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3330032a70 con 0x7f3338075b80 2024-07-03T15:35:42.999 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.011+0000 7f333559a640 1 -- 172.21.15.28:0/2548167106 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3330031ad0 con 0x7f3338075b80 2024-07-03T15:35:42.999 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.012+0000 7f333ca0d640 1 -- 172.21.15.28:0/2548167106 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3338075b80 msgr2=0x7f3338104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:42.999 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.012+0000 7f333ca0d640 1 --2- 172.21.15.28:0/2548167106 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3338075b80 0x7f3338104a50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3330009840 tx=0x7f33300091d0 comp rx=0 tx=0).stop 2024-07-03T15:35:42.999 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.012+0000 7f333ca0d640 1 -- 172.21.15.28:0/2548167106 shutdown_connections 2024-07-03T15:35:42.999 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.012+0000 7f333ca0d640 1 --2- 172.21.15.28:0/2548167106 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3338075b80 0x7f3338104a50 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:42.999 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.012+0000 7f333ca0d640 1 -- 172.21.15.28:0/2548167106 >> 172.21.15.28:0/2548167106 conn(0x7f33380fc4c0 msgr2=0x7f33380fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:42.999 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.012+0000 7f333ca0d640 1 -- 172.21.15.28:0/2548167106 shutdown_connections 2024-07-03T15:35:42.999 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.012+0000 7f333ca0d640 1 -- 172.21.15.28:0/2548167106 wait complete. 2024-07-03T15:35:42.999 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.013+0000 7f333ca0d640 1 Processor -- start 2024-07-03T15:35:42.999 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.013+0000 7f333ca0d640 1 -- start start 2024-07-03T15:35:42.999 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.013+0000 7f333ca0d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3338075b80 0x7f3338102380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:42.999 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.013+0000 7f333ca0d640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33381028c0 con 0x7f3338075b80 2024-07-03T15:35:42.999 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.013+0000 7f333659c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3338075b80 0x7f3338102380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:43.000 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.013+0000 7f333659c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3338075b80 0x7f3338102380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37976/0 (socket says 172.21.15.28:37976) 2024-07-03T15:35:43.000 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.013+0000 7f333659c640 1 -- 172.21.15.28:0/1293393875 learned_addr learned my addr 172.21.15.28:0/1293393875 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:43.000 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.013+0000 7f333659c640 1 -- 172.21.15.28:0/1293393875 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3330009450 con 0x7f3338075b80 2024-07-03T15:35:43.000 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.013+0000 7f333659c640 1 --2- 172.21.15.28:0/1293393875 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3338075b80 0x7f3338102380 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f33300098a0 tx=0x7f333002c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:43.000 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.014+0000 7f332effd640 1 -- 172.21.15.28:0/1293393875 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3330031e90 con 0x7f3338075b80 2024-07-03T15:35:43.000 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.014+0000 7f332effd640 1 -- 172.21.15.28:0/1293393875 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f333003b910 con 0x7f3338075b80 2024-07-03T15:35:43.000 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.014+0000 7f333ca0d640 1 -- 172.21.15.28:0/1293393875 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3338100950 con 0x7f3338075b80 2024-07-03T15:35:43.000 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.014+0000 7f332effd640 1 -- 172.21.15.28:0/1293393875 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f333003a9b0 con 0x7f3338075b80 2024-07-03T15:35:43.000 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.014+0000 7f333ca0d640 1 -- 172.21.15.28:0/1293393875 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3338100dc0 con 0x7f3338075b80 2024-07-03T15:35:43.000 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.015+0000 7f332effd640 1 -- 172.21.15.28:0/1293393875 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 8) v1 ==== 42636+0+0 (secure 0 0 0) 0x7f3330039070 con 0x7f3338075b80 2024-07-03T15:35:43.000 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.015+0000 7f333ca0d640 1 -- 172.21.15.28:0/1293393875 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f333804b530 con 0x7f3338075b80 2024-07-03T15:35:43.000 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.015+0000 7f332effd640 1 --2- 172.21.15.28:0/1293393875 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f3324035780 0x7f3324037c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:43.000 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.015+0000 7f332effd640 1 -- 172.21.15.28:0/1293393875 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f3330068080 con 0x7f3338075b80 2024-07-03T15:35:43.000 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.015+0000 7f3335d9b640 1 --2- 172.21.15.28:0/1293393875 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f3324035780 0x7f3324037c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:43.001 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.016+0000 7f3335d9b640 1 --2- 172.21.15.28:0/1293393875 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f3324035780 0x7f3324037c30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3320006fd0 tx=0x7f3320006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:43.001 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.020+0000 7f332effd640 1 -- 172.21.15.28:0/1293393875 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f333009a250 con 0x7f3338075b80 2024-07-03T15:35:43.001 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.206+0000 7f333ca0d640 1 -- 172.21.15.28:0/1293393875 --> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f3338101570 con 0x7f3324035780 2024-07-03T15:35:43.001 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.210+0000 7f332effd640 1 -- 172.21.15.28:0/1293393875 <== mgr.14118 v2:172.21.15.28:6800/1434683766 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f3338101570 con 0x7f3324035780 2024-07-03T15:35:43.001 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.212+0000 7f333ca0d640 1 -- 172.21.15.28:0/1293393875 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f3324035780 msgr2=0x7f3324037c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:43.001 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.212+0000 7f333ca0d640 1 --2- 172.21.15.28:0/1293393875 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f3324035780 0x7f3324037c30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3320006fd0 tx=0x7f3320006db0 comp rx=0 tx=0).stop 2024-07-03T15:35:43.001 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.212+0000 7f333ca0d640 1 -- 172.21.15.28:0/1293393875 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3338075b80 msgr2=0x7f3338102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:43.001 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.212+0000 7f333ca0d640 1 --2- 172.21.15.28:0/1293393875 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3338075b80 0x7f3338102380 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f33300098a0 tx=0x7f333002c040 comp rx=0 tx=0).stop 2024-07-03T15:35:43.001 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.212+0000 7f333ca0d640 1 -- 172.21.15.28:0/1293393875 shutdown_connections 2024-07-03T15:35:43.001 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.212+0000 7f333ca0d640 1 --2- 172.21.15.28:0/1293393875 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f3324035780 0x7f3324037c30 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:43.001 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.212+0000 7f333ca0d640 1 --2- 172.21.15.28:0/1293393875 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3338075b80 0x7f3338102380 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:43.001 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.212+0000 7f333ca0d640 1 -- 172.21.15.28:0/1293393875 >> 172.21.15.28:0/1293393875 conn(0x7f33380fc4c0 msgr2=0x7f33380fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:43.001 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.212+0000 7f333ca0d640 1 -- 172.21.15.28:0/1293393875 shutdown_connections 2024-07-03T15:35:43.001 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:42.212+0000 7f333ca0d640 1 -- 172.21.15.28:0/1293393875 wait complete. 2024-07-03T15:35:43.002 INFO:teuthology.orchestra.run.smithi028.stdout:Deploying alertmanager service with default placement... 2024-07-03T15:35:43.449 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:43 smithi028 ceph-mon[29148]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:35:43.449 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:43 smithi028 ceph-mon[29148]: Saving service node-exporter spec with placement * 2024-07-03T15:35:43.449 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:43 smithi028 ceph-mon[29148]: from='mgr.14118 172.21.15.28:0/1044821505' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:35:43.449 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:43 smithi028 ceph-mon[29148]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:35:44.235 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-07-03T15:35:44.236 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.451+0000 7fae91897640 1 Processor -- start 2024-07-03T15:35:44.236 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.452+0000 7fae91897640 1 -- start start 2024-07-03T15:35:44.236 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.452+0000 7fae91897640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae8c100d40 0x7fae8c101150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:44.236 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.452+0000 7fae91897640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8c101690 con 0x7fae8c100d40 2024-07-03T15:35:44.236 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.452+0000 7fae8affd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae8c100d40 0x7fae8c101150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:44.236 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.452+0000 7fae8affd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae8c100d40 0x7fae8c101150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37992/0 (socket says 172.21.15.28:37992) 2024-07-03T15:35:44.236 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.452+0000 7fae8affd640 1 -- 172.21.15.28:0/169144009 learned_addr learned my addr 172.21.15.28:0/169144009 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:44.236 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.452+0000 7fae8affd640 1 -- 172.21.15.28:0/169144009 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae8c1017d0 con 0x7fae8c100d40 2024-07-03T15:35:44.236 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.453+0000 7fae8affd640 1 --2- 172.21.15.28:0/169144009 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae8c100d40 0x7fae8c101150 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fae84009840 tx=0x7fae840091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eabb903bd52f575 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:44.236 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.453+0000 7fae89ffb640 1 -- 172.21.15.28:0/169144009 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fae84032440 con 0x7fae8c100d40 2024-07-03T15:35:44.236 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.453+0000 7fae89ffb640 1 -- 172.21.15.28:0/169144009 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fae84032a70 con 0x7fae8c100d40 2024-07-03T15:35:44.237 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.453+0000 7fae89ffb640 1 -- 172.21.15.28:0/169144009 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fae84031ad0 con 0x7fae8c100d40 2024-07-03T15:35:44.237 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.454+0000 7fae91897640 1 -- 172.21.15.28:0/169144009 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae8c100d40 msgr2=0x7fae8c101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:44.237 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.454+0000 7fae91897640 1 --2- 172.21.15.28:0/169144009 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae8c100d40 0x7fae8c101150 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fae84009840 tx=0x7fae840091d0 comp rx=0 tx=0).stop 2024-07-03T15:35:44.237 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.454+0000 7fae91897640 1 -- 172.21.15.28:0/169144009 shutdown_connections 2024-07-03T15:35:44.237 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.454+0000 7fae91897640 1 --2- 172.21.15.28:0/169144009 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae8c100d40 0x7fae8c101150 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:44.237 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.454+0000 7fae91897640 1 -- 172.21.15.28:0/169144009 >> 172.21.15.28:0/169144009 conn(0x7fae8c0fc4e0 msgr2=0x7fae8c0fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:44.237 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.454+0000 7fae91897640 1 -- 172.21.15.28:0/169144009 shutdown_connections 2024-07-03T15:35:44.237 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.454+0000 7fae91897640 1 -- 172.21.15.28:0/169144009 wait complete. 2024-07-03T15:35:44.237 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.455+0000 7fae91897640 1 Processor -- start 2024-07-03T15:35:44.237 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.455+0000 7fae91897640 1 -- start start 2024-07-03T15:35:44.237 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.455+0000 7fae91897640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae8c100d40 0x7fae8c075b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:44.237 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.455+0000 7fae91897640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8c1116d0 con 0x7fae8c100d40 2024-07-03T15:35:44.238 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.455+0000 7fae8affd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae8c100d40 0x7fae8c075b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:44.238 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.455+0000 7fae8affd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae8c100d40 0x7fae8c075b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37996/0 (socket says 172.21.15.28:37996) 2024-07-03T15:35:44.238 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.455+0000 7fae8affd640 1 -- 172.21.15.28:0/1946895697 learned_addr learned my addr 172.21.15.28:0/1946895697 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:44.238 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.455+0000 7fae8affd640 1 -- 172.21.15.28:0/1946895697 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae84009450 con 0x7fae8c100d40 2024-07-03T15:35:44.238 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.456+0000 7fae8affd640 1 --2- 172.21.15.28:0/1946895697 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae8c100d40 0x7fae8c075b00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fae84006a70 tx=0x7fae8402c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:44.238 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.456+0000 7fae837fe640 1 -- 172.21.15.28:0/1946895697 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fae84031e90 con 0x7fae8c100d40 2024-07-03T15:35:44.238 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.456+0000 7fae837fe640 1 -- 172.21.15.28:0/1946895697 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fae8403b910 con 0x7fae8c100d40 2024-07-03T15:35:44.238 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.456+0000 7fae91897640 1 -- 172.21.15.28:0/1946895697 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae8c115ce0 con 0x7fae8c100d40 2024-07-03T15:35:44.238 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.456+0000 7fae837fe640 1 -- 172.21.15.28:0/1946895697 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fae8403a940 con 0x7fae8c100d40 2024-07-03T15:35:44.238 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.456+0000 7fae91897640 1 -- 172.21.15.28:0/1946895697 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae8c116010 con 0x7fae8c100d40 2024-07-03T15:35:44.238 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.457+0000 7fae837fe640 1 -- 172.21.15.28:0/1946895697 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 8) v1 ==== 42636+0+0 (secure 0 0 0) 0x7fae84039070 con 0x7fae8c100d40 2024-07-03T15:35:44.238 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.457+0000 7fae837fe640 1 --2- 172.21.15.28:0/1946895697 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7fae78035780 0x7fae78037c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:44.239 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.457+0000 7fae837fe640 1 -- 172.21.15.28:0/1946895697 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fae84068420 con 0x7fae8c100d40 2024-07-03T15:35:44.239 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.457+0000 7fae8a7fc640 1 --2- 172.21.15.28:0/1946895697 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7fae78035780 0x7fae78037c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:44.239 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.457+0000 7fae91897640 1 -- 172.21.15.28:0/1946895697 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae8c04b530 con 0x7fae8c100d40 2024-07-03T15:35:44.239 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.457+0000 7fae8a7fc640 1 --2- 172.21.15.28:0/1946895697 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7fae78035780 0x7fae78037c30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fae74006fd0 tx=0x7fae74006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:44.239 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.462+0000 7fae837fe640 1 -- 172.21.15.28:0/1946895697 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fae84037030 con 0x7fae8c100d40 2024-07-03T15:35:44.239 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.647+0000 7fae91897640 1 -- 172.21.15.28:0/1946895697 --> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7fae8c0fd790 con 0x7fae78035780 2024-07-03T15:35:44.239 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.651+0000 7fae837fe640 1 -- 172.21.15.28:0/1946895697 <== mgr.14118 v2:172.21.15.28:6800/1434683766 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7fae8c0fd790 con 0x7fae78035780 2024-07-03T15:35:44.239 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.653+0000 7fae91897640 1 -- 172.21.15.28:0/1946895697 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7fae78035780 msgr2=0x7fae78037c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:44.239 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.653+0000 7fae91897640 1 --2- 172.21.15.28:0/1946895697 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7fae78035780 0x7fae78037c30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fae74006fd0 tx=0x7fae74006db0 comp rx=0 tx=0).stop 2024-07-03T15:35:44.239 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.653+0000 7fae91897640 1 -- 172.21.15.28:0/1946895697 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae8c100d40 msgr2=0x7fae8c075b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:44.239 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.653+0000 7fae91897640 1 --2- 172.21.15.28:0/1946895697 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae8c100d40 0x7fae8c075b00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fae84006a70 tx=0x7fae8402c040 comp rx=0 tx=0).stop 2024-07-03T15:35:44.239 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.653+0000 7fae91897640 1 -- 172.21.15.28:0/1946895697 shutdown_connections 2024-07-03T15:35:44.239 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.653+0000 7fae91897640 1 --2- 172.21.15.28:0/1946895697 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7fae78035780 0x7fae78037c30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:44.239 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.653+0000 7fae91897640 1 --2- 172.21.15.28:0/1946895697 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae8c100d40 0x7fae8c075b00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:44.240 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.653+0000 7fae91897640 1 -- 172.21.15.28:0/1946895697 >> 172.21.15.28:0/1946895697 conn(0x7fae8c0fc4e0 msgr2=0x7fae8c0fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:44.240 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.653+0000 7fae91897640 1 -- 172.21.15.28:0/1946895697 shutdown_connections 2024-07-03T15:35:44.240 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:43.653+0000 7fae91897640 1 -- 172.21.15.28:0/1946895697 wait complete. 2024-07-03T15:35:44.814 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:44 smithi028 ceph-mon[29148]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:35:44.814 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:44 smithi028 ceph-mon[29148]: Saving service alertmanager spec with placement count:1 2024-07-03T15:35:44.814 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:44 smithi028 ceph-mon[29148]: from='mgr.14118 172.21.15.28:0/1044821505' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:35:45.382 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.624+0000 7f7ccc043640 1 Processor -- start 2024-07-03T15:35:45.383 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.625+0000 7f7ccc043640 1 -- start start 2024-07-03T15:35:45.383 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.625+0000 7f7ccc043640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7cc4101e00 0x7f7cc4102210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:45.383 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.625+0000 7f7ccc043640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc4102750 con 0x7f7cc4101e00 2024-07-03T15:35:45.383 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.625+0000 7f7cc9ddf640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7cc4101e00 0x7f7cc4102210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:45.383 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.625+0000 7f7cc9ddf640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7cc4101e00 0x7f7cc4102210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38004/0 (socket says 172.21.15.28:38004) 2024-07-03T15:35:45.383 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.625+0000 7f7cc9ddf640 1 -- 172.21.15.28:0/1077058091 learned_addr learned my addr 172.21.15.28:0/1077058091 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:45.383 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.625+0000 7f7cc9ddf640 1 -- 172.21.15.28:0/1077058091 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc4102890 con 0x7f7cc4101e00 2024-07-03T15:35:45.383 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.626+0000 7f7cc9ddf640 1 --2- 172.21.15.28:0/1077058091 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7cc4101e00 0x7f7cc4102210 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0009840 tx=0x7f7cc00091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17da3998908e09b3 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:45.383 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.626+0000 7f7cc8ddd640 1 -- 172.21.15.28:0/1077058091 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7cc0032440 con 0x7f7cc4101e00 2024-07-03T15:35:45.383 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.626+0000 7f7cc8ddd640 1 -- 172.21.15.28:0/1077058091 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7cc0032a70 con 0x7f7cc4101e00 2024-07-03T15:35:45.383 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.626+0000 7f7cc8ddd640 1 -- 172.21.15.28:0/1077058091 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7cc0031ad0 con 0x7f7cc4101e00 2024-07-03T15:35:45.383 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.627+0000 7f7ccc043640 1 -- 172.21.15.28:0/1077058091 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7cc4101e00 msgr2=0x7f7cc4102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:45.384 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.627+0000 7f7ccc043640 1 --2- 172.21.15.28:0/1077058091 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7cc4101e00 0x7f7cc4102210 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0009840 tx=0x7f7cc00091d0 comp rx=0 tx=0).stop 2024-07-03T15:35:45.384 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.627+0000 7f7ccc043640 1 -- 172.21.15.28:0/1077058091 shutdown_connections 2024-07-03T15:35:45.384 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.627+0000 7f7ccc043640 1 --2- 172.21.15.28:0/1077058091 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7cc4101e00 0x7f7cc4102210 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:45.384 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.627+0000 7f7ccc043640 1 -- 172.21.15.28:0/1077058091 >> 172.21.15.28:0/1077058091 conn(0x7f7cc40fd5a0 msgr2=0x7f7cc40ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:45.384 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.627+0000 7f7ccc043640 1 -- 172.21.15.28:0/1077058091 shutdown_connections 2024-07-03T15:35:45.384 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.627+0000 7f7ccc043640 1 -- 172.21.15.28:0/1077058091 wait complete. 2024-07-03T15:35:45.384 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.628+0000 7f7ccc043640 1 Processor -- start 2024-07-03T15:35:45.384 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.628+0000 7f7ccc043640 1 -- start start 2024-07-03T15:35:45.384 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.628+0000 7f7ccc043640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7cc4101e00 0x7f7cc4110b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:45.384 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.628+0000 7f7ccc043640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc4111050 con 0x7f7cc4101e00 2024-07-03T15:35:45.384 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.628+0000 7f7cc9ddf640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7cc4101e00 0x7f7cc4110b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:45.384 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.628+0000 7f7cc9ddf640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7cc4101e00 0x7f7cc4110b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38016/0 (socket says 172.21.15.28:38016) 2024-07-03T15:35:45.384 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.628+0000 7f7cc9ddf640 1 -- 172.21.15.28:0/2451841196 learned_addr learned my addr 172.21.15.28:0/2451841196 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:45.384 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.628+0000 7f7cc9ddf640 1 -- 172.21.15.28:0/2451841196 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc0009450 con 0x7f7cc4101e00 2024-07-03T15:35:45.385 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.629+0000 7f7cc9ddf640 1 --2- 172.21.15.28:0/2451841196 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7cc4101e00 0x7f7cc4110b10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0009810 tx=0x7f7cc002c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:45.385 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.629+0000 7f7cb67fc640 1 -- 172.21.15.28:0/2451841196 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7cc00328b0 con 0x7f7cc4101e00 2024-07-03T15:35:45.385 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.629+0000 7f7cb67fc640 1 -- 172.21.15.28:0/2451841196 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7cc003b910 con 0x7f7cc4101e00 2024-07-03T15:35:45.385 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.629+0000 7f7ccc043640 1 -- 172.21.15.28:0/2451841196 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cc4111250 con 0x7f7cc4101e00 2024-07-03T15:35:45.385 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.629+0000 7f7cb67fc640 1 -- 172.21.15.28:0/2451841196 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7cc003a980 con 0x7f7cc4101e00 2024-07-03T15:35:45.385 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.629+0000 7f7ccc043640 1 -- 172.21.15.28:0/2451841196 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cc41116c0 con 0x7f7cc4101e00 2024-07-03T15:35:45.385 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.630+0000 7f7cb67fc640 1 -- 172.21.15.28:0/2451841196 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 8) v1 ==== 42636+0+0 (secure 0 0 0) 0x7f7cc0039070 con 0x7f7cc4101e00 2024-07-03T15:35:45.385 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.630+0000 7f7ccc043640 1 -- 172.21.15.28:0/2451841196 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cc404b530 con 0x7f7cc4101e00 2024-07-03T15:35:45.385 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.630+0000 7f7cb67fc640 1 --2- 172.21.15.28:0/2451841196 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f7cb00357d0 0x7f7cb0037c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:45.385 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.630+0000 7f7cb67fc640 1 -- 172.21.15.28:0/2451841196 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f7cc0067f10 con 0x7f7cc4101e00 2024-07-03T15:35:45.385 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.630+0000 7f7cc95de640 1 --2- 172.21.15.28:0/2451841196 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f7cb00357d0 0x7f7cb0037c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:45.385 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.631+0000 7f7cc95de640 1 --2- 172.21.15.28:0/2451841196 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f7cb00357d0 0x7f7cb0037c80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8006fd0 tx=0x7f7cb8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:45.385 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.635+0000 7f7cb67fc640 1 -- 172.21.15.28:0/2451841196 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7cc0030750 con 0x7f7cc4101e00 2024-07-03T15:35:45.385 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.811+0000 7f7ccc043640 1 -- 172.21.15.28:0/2451841196 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f7cc41116c0 con 0x7f7cc4101e00 2024-07-03T15:35:45.385 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.815+0000 7f7cb67fc640 1 -- 172.21.15.28:0/2451841196 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f7cc0037020 con 0x7f7cc4101e00 2024-07-03T15:35:45.386 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.817+0000 7f7ccc043640 1 -- 172.21.15.28:0/2451841196 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f7cb00357d0 msgr2=0x7f7cb0037c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:45.386 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.817+0000 7f7ccc043640 1 --2- 172.21.15.28:0/2451841196 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f7cb00357d0 0x7f7cb0037c80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8006fd0 tx=0x7f7cb8006db0 comp rx=0 tx=0).stop 2024-07-03T15:35:45.386 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.817+0000 7f7ccc043640 1 -- 172.21.15.28:0/2451841196 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7cc4101e00 msgr2=0x7f7cc4110b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:45.386 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.817+0000 7f7ccc043640 1 --2- 172.21.15.28:0/2451841196 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7cc4101e00 0x7f7cc4110b10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0009810 tx=0x7f7cc002c040 comp rx=0 tx=0).stop 2024-07-03T15:35:45.386 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.817+0000 7f7ccc043640 1 -- 172.21.15.28:0/2451841196 shutdown_connections 2024-07-03T15:35:45.386 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.817+0000 7f7ccc043640 1 --2- 172.21.15.28:0/2451841196 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f7cb00357d0 0x7f7cb0037c80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:45.386 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.817+0000 7f7ccc043640 1 --2- 172.21.15.28:0/2451841196 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7cc4101e00 0x7f7cc4110b10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:45.386 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.817+0000 7f7ccc043640 1 -- 172.21.15.28:0/2451841196 >> 172.21.15.28:0/2451841196 conn(0x7f7cc40fd5a0 msgr2=0x7f7cc40fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:45.386 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.817+0000 7f7ccc043640 1 -- 172.21.15.28:0/2451841196 shutdown_connections 2024-07-03T15:35:45.386 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:44.818+0000 7f7ccc043640 1 -- 172.21.15.28:0/2451841196 wait complete. 2024-07-03T15:35:45.977 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:45 smithi028 ceph-mon[29148]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:35:45.978 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:45 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/2451841196' entity='client.admin' 2024-07-03T15:35:46.580 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.790+0000 7f63d9209640 1 Processor -- start 2024-07-03T15:35:46.580 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.790+0000 7f63d9209640 1 -- start start 2024-07-03T15:35:46.580 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.790+0000 7f63d9209640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f63d4100ce0 0x7f63d41010f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:46.581 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.790+0000 7f63d9209640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d4101630 con 0x7f63d4100ce0 2024-07-03T15:35:46.581 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.790+0000 7f63d2d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f63d4100ce0 0x7f63d41010f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:46.581 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.790+0000 7f63d2d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f63d4100ce0 0x7f63d41010f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38020/0 (socket says 172.21.15.28:38020) 2024-07-03T15:35:46.581 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.790+0000 7f63d2d9d640 1 -- 172.21.15.28:0/4212295325 learned_addr learned my addr 172.21.15.28:0/4212295325 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:46.581 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.791+0000 7f63d2d9d640 1 -- 172.21.15.28:0/4212295325 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d4101770 con 0x7f63d4100ce0 2024-07-03T15:35:46.581 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.791+0000 7f63d2d9d640 1 --2- 172.21.15.28:0/4212295325 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f63d4100ce0 0x7f63d41010f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f63c400baf0 tx=0x7f63c400b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d01f82e44c3d97 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:46.581 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.792+0000 7f63d1d9b640 1 -- 172.21.15.28:0/4212295325 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f63c40327a0 con 0x7f63d4100ce0 2024-07-03T15:35:46.581 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.792+0000 7f63d1d9b640 1 -- 172.21.15.28:0/4212295325 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f63c4032dd0 con 0x7f63d4100ce0 2024-07-03T15:35:46.581 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.792+0000 7f63d1d9b640 1 -- 172.21.15.28:0/4212295325 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f63c4039310 con 0x7f63d4100ce0 2024-07-03T15:35:46.581 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.792+0000 7f63d9209640 1 -- 172.21.15.28:0/4212295325 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f63d4100ce0 msgr2=0x7f63d41010f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:46.581 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.792+0000 7f63d9209640 1 --2- 172.21.15.28:0/4212295325 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f63d4100ce0 0x7f63d41010f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f63c400baf0 tx=0x7f63c400b460 comp rx=0 tx=0).stop 2024-07-03T15:35:46.581 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.792+0000 7f63d9209640 1 -- 172.21.15.28:0/4212295325 shutdown_connections 2024-07-03T15:35:46.581 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.792+0000 7f63d9209640 1 --2- 172.21.15.28:0/4212295325 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f63d4100ce0 0x7f63d41010f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:46.581 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.792+0000 7f63d9209640 1 -- 172.21.15.28:0/4212295325 >> 172.21.15.28:0/4212295325 conn(0x7f63d40fc520 msgr2=0x7f63d40fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:46.582 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.792+0000 7f63d9209640 1 -- 172.21.15.28:0/4212295325 shutdown_connections 2024-07-03T15:35:46.582 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.792+0000 7f63d9209640 1 -- 172.21.15.28:0/4212295325 wait complete. 2024-07-03T15:35:46.582 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.793+0000 7f63d9209640 1 Processor -- start 2024-07-03T15:35:46.582 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.793+0000 7f63d9209640 1 -- start start 2024-07-03T15:35:46.582 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.793+0000 7f63d9209640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f63d4100ce0 0x7f63d410e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:46.582 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.793+0000 7f63d9209640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d410ee80 con 0x7f63d4100ce0 2024-07-03T15:35:46.582 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.793+0000 7f63d2d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f63d4100ce0 0x7f63d410e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:46.582 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.793+0000 7f63d2d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f63d4100ce0 0x7f63d410e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38036/0 (socket says 172.21.15.28:38036) 2024-07-03T15:35:46.582 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.793+0000 7f63d2d9d640 1 -- 172.21.15.28:0/517607698 learned_addr learned my addr 172.21.15.28:0/517607698 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:46.582 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.793+0000 7f63d2d9d640 1 -- 172.21.15.28:0/517607698 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c400b700 con 0x7f63d4100ce0 2024-07-03T15:35:46.582 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.794+0000 7f63d2d9d640 1 --2- 172.21.15.28:0/517607698 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f63d4100ce0 0x7f63d410e940 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f63c40315c0 tx=0x7f63c400b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:46.582 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.794+0000 7f63bb7fe640 1 -- 172.21.15.28:0/517607698 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f63c40329b0 con 0x7f63d4100ce0 2024-07-03T15:35:46.582 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.794+0000 7f63bb7fe640 1 -- 172.21.15.28:0/517607698 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f63c4032b10 con 0x7f63d4100ce0 2024-07-03T15:35:46.582 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.794+0000 7f63d9209640 1 -- 172.21.15.28:0/517607698 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63d410f080 con 0x7f63d4100ce0 2024-07-03T15:35:46.583 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.794+0000 7f63bb7fe640 1 -- 172.21.15.28:0/517607698 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f63c403b740 con 0x7f63d4100ce0 2024-07-03T15:35:46.583 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.794+0000 7f63d9209640 1 -- 172.21.15.28:0/517607698 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d410f490 con 0x7f63d4100ce0 2024-07-03T15:35:46.583 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.795+0000 7f63bb7fe640 1 -- 172.21.15.28:0/517607698 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 8) v1 ==== 42636+0+0 (secure 0 0 0) 0x7f63c403b8a0 con 0x7f63d4100ce0 2024-07-03T15:35:46.583 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.795+0000 7f63bb7fe640 1 --2- 172.21.15.28:0/517607698 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f63bc0357d0 0x7f63bc037c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:46.583 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.795+0000 7f63bb7fe640 1 -- 172.21.15.28:0/517607698 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f63c406ba80 con 0x7f63d4100ce0 2024-07-03T15:35:46.583 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.795+0000 7f63d259c640 1 --2- 172.21.15.28:0/517607698 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f63bc0357d0 0x7f63bc037c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:46.583 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.795+0000 7f63d9209640 1 -- 172.21.15.28:0/517607698 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63d404b530 con 0x7f63d4100ce0 2024-07-03T15:35:46.583 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.796+0000 7f63d259c640 1 --2- 172.21.15.28:0/517607698 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f63bc0357d0 0x7f63bc037c80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f63c8006fd0 tx=0x7f63c8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:46.583 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.800+0000 7f63bb7fe640 1 -- 172.21.15.28:0/517607698 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f63c4036070 con 0x7f63d4100ce0 2024-07-03T15:35:46.583 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.974+0000 7f63d9209640 1 -- 172.21.15.28:0/517607698 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f63d4102b10 con 0x7f63d4100ce0 2024-07-03T15:35:46.583 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.978+0000 7f63bb7fe640 1 -- 172.21.15.28:0/517607698 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f63c404d900 con 0x7f63d4100ce0 2024-07-03T15:35:46.583 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.980+0000 7f63d9209640 1 -- 172.21.15.28:0/517607698 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f63bc0357d0 msgr2=0x7f63bc037c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:46.583 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.980+0000 7f63d9209640 1 --2- 172.21.15.28:0/517607698 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f63bc0357d0 0x7f63bc037c80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f63c8006fd0 tx=0x7f63c8006db0 comp rx=0 tx=0).stop 2024-07-03T15:35:46.584 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.980+0000 7f63d9209640 1 -- 172.21.15.28:0/517607698 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f63d4100ce0 msgr2=0x7f63d410e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:46.584 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.980+0000 7f63d9209640 1 --2- 172.21.15.28:0/517607698 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f63d4100ce0 0x7f63d410e940 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f63c40315c0 tx=0x7f63c400b460 comp rx=0 tx=0).stop 2024-07-03T15:35:46.584 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.980+0000 7f63d9209640 1 -- 172.21.15.28:0/517607698 shutdown_connections 2024-07-03T15:35:46.584 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.980+0000 7f63d9209640 1 --2- 172.21.15.28:0/517607698 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f63bc0357d0 0x7f63bc037c80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:46.584 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.980+0000 7f63d9209640 1 --2- 172.21.15.28:0/517607698 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f63d4100ce0 0x7f63d410e940 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:46.584 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.980+0000 7f63d9209640 1 -- 172.21.15.28:0/517607698 >> 172.21.15.28:0/517607698 conn(0x7f63d40fc520 msgr2=0x7f63d4105240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:46.584 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.980+0000 7f63d9209640 1 -- 172.21.15.28:0/517607698 shutdown_connections 2024-07-03T15:35:46.584 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:45.980+0000 7f63d9209640 1 -- 172.21.15.28:0/517607698 wait complete. 2024-07-03T15:35:46.584 INFO:teuthology.orchestra.run.smithi028.stdout:Enabling the dashboard module... 2024-07-03T15:35:47.370 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:46 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/517607698' entity='client.admin' 2024-07-03T15:35:47.370 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:46 smithi028 ceph-mon[29148]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:35:48.120 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:47 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/3467097823' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-07-03T15:35:48.596 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:46.986+0000 7ff6156fd640 1 Processor -- start 2024-07-03T15:35:48.596 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:46.986+0000 7ff6156fd640 1 -- start start 2024-07-03T15:35:48.596 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:46.987+0000 7ff6156fd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff610075c00 0x7ff6101049d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:48.596 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:46.987+0000 7ff6156fd640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff610104f10 con 0x7ff610075c00 2024-07-03T15:35:48.596 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:46.987+0000 7ff60effd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff610075c00 0x7ff6101049d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:48.596 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:46.987+0000 7ff60effd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff610075c00 0x7ff6101049d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38046/0 (socket says 172.21.15.28:38046) 2024-07-03T15:35:48.596 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:46.987+0000 7ff60effd640 1 -- 172.21.15.28:0/3338309784 learned_addr learned my addr 172.21.15.28:0/3338309784 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:48.596 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:46.987+0000 7ff60effd640 1 -- 172.21.15.28:0/3338309784 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff610105050 con 0x7ff610075c00 2024-07-03T15:35:48.596 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:46.988+0000 7ff60effd640 1 --2- 172.21.15.28:0/3338309784 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff610075c00 0x7ff6101049d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff600009840 tx=0x7ff6000091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9653f08be85d9236 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:48.596 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:46.988+0000 7ff60dffb640 1 -- 172.21.15.28:0/3338309784 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff600032440 con 0x7ff610075c00 2024-07-03T15:35:48.597 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:46.988+0000 7ff60dffb640 1 -- 172.21.15.28:0/3338309784 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff600032a70 con 0x7ff610075c00 2024-07-03T15:35:48.597 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:46.988+0000 7ff60dffb640 1 -- 172.21.15.28:0/3338309784 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff600031ad0 con 0x7ff610075c00 2024-07-03T15:35:48.597 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:46.989+0000 7ff6156fd640 1 -- 172.21.15.28:0/3338309784 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff610075c00 msgr2=0x7ff6101049d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:48.597 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:46.989+0000 7ff6156fd640 1 --2- 172.21.15.28:0/3338309784 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff610075c00 0x7ff6101049d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff600009840 tx=0x7ff6000091d0 comp rx=0 tx=0).stop 2024-07-03T15:35:48.597 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:46.989+0000 7ff6156fd640 1 -- 172.21.15.28:0/3338309784 shutdown_connections 2024-07-03T15:35:48.597 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:46.989+0000 7ff6156fd640 1 --2- 172.21.15.28:0/3338309784 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff610075c00 0x7ff6101049d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:48.597 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:46.989+0000 7ff6156fd640 1 -- 172.21.15.28:0/3338309784 >> 172.21.15.28:0/3338309784 conn(0x7ff6100fc520 msgr2=0x7ff6100fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:48.597 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:46.989+0000 7ff6156fd640 1 -- 172.21.15.28:0/3338309784 shutdown_connections 2024-07-03T15:35:48.597 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:46.989+0000 7ff6156fd640 1 -- 172.21.15.28:0/3338309784 wait complete. 2024-07-03T15:35:48.597 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:46.990+0000 7ff6156fd640 1 Processor -- start 2024-07-03T15:35:48.597 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:46.990+0000 7ff6156fd640 1 -- start start 2024-07-03T15:35:48.597 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:46.990+0000 7ff6156fd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff610075c00 0x7ff610110b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:48.597 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:46.990+0000 7ff6156fd640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6101110c0 con 0x7ff610075c00 2024-07-03T15:35:48.598 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:46.990+0000 7ff60effd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff610075c00 0x7ff610110b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:48.598 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:46.990+0000 7ff60effd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff610075c00 0x7ff610110b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38056/0 (socket says 172.21.15.28:38056) 2024-07-03T15:35:48.598 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:46.990+0000 7ff60effd640 1 -- 172.21.15.28:0/3467097823 learned_addr learned my addr 172.21.15.28:0/3467097823 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:48.598 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:46.990+0000 7ff60effd640 1 -- 172.21.15.28:0/3467097823 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff600009450 con 0x7ff610075c00 2024-07-03T15:35:48.598 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:46.991+0000 7ff60effd640 1 --2- 172.21.15.28:0/3467097823 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff610075c00 0x7ff610110b80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff6000098a0 tx=0x7ff60002c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:48.598 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:46.991+0000 7ff5f77fe640 1 -- 172.21.15.28:0/3467097823 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff600031e10 con 0x7ff610075c00 2024-07-03T15:35:48.598 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:46.991+0000 7ff5f77fe640 1 -- 172.21.15.28:0/3467097823 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff60003b910 con 0x7ff610075c00 2024-07-03T15:35:48.598 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:46.991+0000 7ff6156fd640 1 -- 172.21.15.28:0/3467097823 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6101112c0 con 0x7ff610075c00 2024-07-03T15:35:48.598 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:46.991+0000 7ff5f77fe640 1 -- 172.21.15.28:0/3467097823 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff60003a880 con 0x7ff610075c00 2024-07-03T15:35:48.598 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:46.991+0000 7ff6156fd640 1 -- 172.21.15.28:0/3467097823 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff610111730 con 0x7ff610075c00 2024-07-03T15:35:48.598 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:46.992+0000 7ff5f77fe640 1 -- 172.21.15.28:0/3467097823 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 8) v1 ==== 42636+0+0 (secure 0 0 0) 0x7ff600039070 con 0x7ff610075c00 2024-07-03T15:35:48.598 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:46.992+0000 7ff5f77fe640 1 --2- 172.21.15.28:0/3467097823 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7ff5f80357d0 0x7ff5f8037c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:48.599 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:46.992+0000 7ff5f77fe640 1 -- 172.21.15.28:0/3467097823 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7ff600067d90 con 0x7ff610075c00 2024-07-03T15:35:48.599 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:46.992+0000 7ff6156fd640 1 -- 172.21.15.28:0/3467097823 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff61005eb80 con 0x7ff610075c00 2024-07-03T15:35:48.599 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:46.992+0000 7ff60e7fc640 1 --2- 172.21.15.28:0/3467097823 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7ff5f80357d0 0x7ff5f8037c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:48.599 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:46.993+0000 7ff60e7fc640 1 --2- 172.21.15.28:0/3467097823 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7ff5f80357d0 0x7ff5f8037c80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff604006fd0 tx=0x7ff604006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:48.599 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:46.997+0000 7ff5f77fe640 1 -- 172.21.15.28:0/3467097823 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff60003f080 con 0x7ff610075c00 2024-07-03T15:35:48.599 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:47.216+0000 7ff6156fd640 1 -- 172.21.15.28:0/3467097823 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7ff610111ef0 con 0x7ff610075c00 2024-07-03T15:35:48.599 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:47.983+0000 7ff5f77fe640 1 -- 172.21.15.28:0/3467097823 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mgrmap(e 9) v1 ==== 42649+0+0 (secure 0 0 0) 0x7ff600041da0 con 0x7ff610075c00 2024-07-03T15:35:48.599 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:47.984+0000 7ff5f77fe640 1 -- 172.21.15.28:0/3467097823 <== mon.0 v2:172.21.15.28:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7ff600041b10 con 0x7ff610075c00 2024-07-03T15:35:48.599 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:47.986+0000 7ff6156fd640 1 -- 172.21.15.28:0/3467097823 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7ff5f80357d0 msgr2=0x7ff5f8037c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:48.599 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:47.986+0000 7ff6156fd640 1 --2- 172.21.15.28:0/3467097823 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7ff5f80357d0 0x7ff5f8037c80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff604006fd0 tx=0x7ff604006db0 comp rx=0 tx=0).stop 2024-07-03T15:35:48.599 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:47.986+0000 7ff6156fd640 1 -- 172.21.15.28:0/3467097823 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff610075c00 msgr2=0x7ff610110b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:48.599 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:47.986+0000 7ff6156fd640 1 --2- 172.21.15.28:0/3467097823 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff610075c00 0x7ff610110b80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff6000098a0 tx=0x7ff60002c040 comp rx=0 tx=0).stop 2024-07-03T15:35:48.599 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:47.986+0000 7ff6156fd640 1 -- 172.21.15.28:0/3467097823 shutdown_connections 2024-07-03T15:35:48.600 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:47.986+0000 7ff6156fd640 1 --2- 172.21.15.28:0/3467097823 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7ff5f80357d0 0x7ff5f8037c80 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:48.600 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:47.986+0000 7ff6156fd640 1 --2- 172.21.15.28:0/3467097823 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff610075c00 0x7ff610110b80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:48.600 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:47.986+0000 7ff6156fd640 1 -- 172.21.15.28:0/3467097823 >> 172.21.15.28:0/3467097823 conn(0x7ff6100fc520 msgr2=0x7ff6100fcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:48.600 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:47.986+0000 7ff6156fd640 1 -- 172.21.15.28:0/3467097823 shutdown_connections 2024-07-03T15:35:48.600 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:47.986+0000 7ff6156fd640 1 -- 172.21.15.28:0/3467097823 wait complete. 2024-07-03T15:35:49.212 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:48 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/3467097823' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-07-03T15:35:49.212 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:48 smithi028 ceph-mon[29148]: mgrmap e9: smithi028.vtqrtz(active, since 25s) 2024-07-03T15:35:49.878 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout { 2024-07-03T15:35:49.878 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-07-03T15:35:49.878 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "available": true, 2024-07-03T15:35:49.878 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "active_name": "smithi028.vtqrtz", 2024-07-03T15:35:49.879 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-07-03T15:35:49.879 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout } 2024-07-03T15:35:49.879 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.003+0000 7fb0ec451640 1 Processor -- start 2024-07-03T15:35:49.879 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.003+0000 7fb0ec451640 1 -- start start 2024-07-03T15:35:49.879 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.003+0000 7fb0ec451640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0e4100ce0 0x7fb0e41010f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:49.879 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.003+0000 7fb0ec451640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e4101630 con 0x7fb0e4100ce0 2024-07-03T15:35:49.879 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.004+0000 7fb0ea1ed640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0e4100ce0 0x7fb0e41010f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:49.879 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.004+0000 7fb0ea1ed640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0e4100ce0 0x7fb0e41010f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38068/0 (socket says 172.21.15.28:38068) 2024-07-03T15:35:49.879 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.004+0000 7fb0ea1ed640 1 -- 172.21.15.28:0/2752341410 learned_addr learned my addr 172.21.15.28:0/2752341410 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:49.879 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.004+0000 7fb0ea1ed640 1 -- 172.21.15.28:0/2752341410 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e4101770 con 0x7fb0e4100ce0 2024-07-03T15:35:49.879 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.005+0000 7fb0ea1ed640 1 --2- 172.21.15.28:0/2752341410 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0e4100ce0 0x7fb0e41010f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0009840 tx=0x7fb0e00091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d91de869145fb0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:49.879 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.005+0000 7fb0e91eb640 1 -- 172.21.15.28:0/2752341410 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb0e0032440 con 0x7fb0e4100ce0 2024-07-03T15:35:49.880 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.005+0000 7fb0e91eb640 1 -- 172.21.15.28:0/2752341410 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb0e0032a70 con 0x7fb0e4100ce0 2024-07-03T15:35:49.880 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.005+0000 7fb0e91eb640 1 -- 172.21.15.28:0/2752341410 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb0e0031ad0 con 0x7fb0e4100ce0 2024-07-03T15:35:49.880 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.006+0000 7fb0ec451640 1 -- 172.21.15.28:0/2752341410 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0e4100ce0 msgr2=0x7fb0e41010f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:49.880 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.006+0000 7fb0ec451640 1 --2- 172.21.15.28:0/2752341410 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0e4100ce0 0x7fb0e41010f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0009840 tx=0x7fb0e00091d0 comp rx=0 tx=0).stop 2024-07-03T15:35:49.880 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.006+0000 7fb0ec451640 1 -- 172.21.15.28:0/2752341410 shutdown_connections 2024-07-03T15:35:49.880 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.006+0000 7fb0ec451640 1 --2- 172.21.15.28:0/2752341410 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0e4100ce0 0x7fb0e41010f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:49.880 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.006+0000 7fb0ec451640 1 -- 172.21.15.28:0/2752341410 >> 172.21.15.28:0/2752341410 conn(0x7fb0e40fc520 msgr2=0x7fb0e40fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:49.880 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.006+0000 7fb0ec451640 1 -- 172.21.15.28:0/2752341410 shutdown_connections 2024-07-03T15:35:49.880 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.006+0000 7fb0ec451640 1 -- 172.21.15.28:0/2752341410 wait complete. 2024-07-03T15:35:49.880 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.007+0000 7fb0ec451640 1 Processor -- start 2024-07-03T15:35:49.880 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.007+0000 7fb0ec451640 1 -- start start 2024-07-03T15:35:49.880 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.007+0000 7fb0ec451640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0e4100ce0 0x7fb0e4112e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:49.880 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.007+0000 7fb0ec451640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e4113350 con 0x7fb0e4100ce0 2024-07-03T15:35:49.880 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.007+0000 7fb0ea1ed640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0e4100ce0 0x7fb0e4112e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:49.880 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.007+0000 7fb0ea1ed640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0e4100ce0 0x7fb0e4112e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38070/0 (socket says 172.21.15.28:38070) 2024-07-03T15:35:49.881 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.007+0000 7fb0ea1ed640 1 -- 172.21.15.28:0/3493883065 learned_addr learned my addr 172.21.15.28:0/3493883065 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:49.881 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.007+0000 7fb0ea1ed640 1 -- 172.21.15.28:0/3493883065 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e0009450 con 0x7fb0e4100ce0 2024-07-03T15:35:49.881 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.008+0000 7fb0ea1ed640 1 --2- 172.21.15.28:0/3493883065 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0e4100ce0 0x7fb0e4112e10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0009810 tx=0x7fb0e002c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:49.881 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.008+0000 7fb0daffd640 1 -- 172.21.15.28:0/3493883065 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb0e00328b0 con 0x7fb0e4100ce0 2024-07-03T15:35:49.881 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.008+0000 7fb0ec451640 1 -- 172.21.15.28:0/3493883065 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0e4113550 con 0x7fb0e4100ce0 2024-07-03T15:35:49.881 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.008+0000 7fb0daffd640 1 -- 172.21.15.28:0/3493883065 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb0e003b910 con 0x7fb0e4100ce0 2024-07-03T15:35:49.881 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.008+0000 7fb0daffd640 1 -- 172.21.15.28:0/3493883065 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb0e003a980 con 0x7fb0e4100ce0 2024-07-03T15:35:49.881 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.008+0000 7fb0ec451640 1 -- 172.21.15.28:0/3493883065 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0e4113fa0 con 0x7fb0e4100ce0 2024-07-03T15:35:49.881 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.009+0000 7fb0daffd640 1 -- 172.21.15.28:0/3493883065 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 9) v1 ==== 42649+0+0 (secure 0 0 0) 0x7fb0e0039070 con 0x7fb0e4100ce0 2024-07-03T15:35:49.881 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.009+0000 7fb0daffd640 1 --2- 172.21.15.28:0/3493883065 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7fb0d0035820 0x7fb0d0037cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:49.881 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.009+0000 7fb0daffd640 1 -- 172.21.15.28:0/3493883065 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fb0e0067fa0 con 0x7fb0e4100ce0 2024-07-03T15:35:49.881 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.009+0000 7fb0e99ec640 1 -- 172.21.15.28:0/3493883065 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7fb0d0035820 msgr2=0x7fb0d0037cd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.28:6800/1434683766 2024-07-03T15:35:49.881 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.009+0000 7fb0e99ec640 1 --2- 172.21.15.28:0/3493883065 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7fb0d0035820 0x7fb0d0037cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-03T15:35:49.881 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.009+0000 7fb0ec451640 1 -- 172.21.15.28:0/3493883065 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0e405eb80 con 0x7fb0e4100ce0 2024-07-03T15:35:49.882 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.018+0000 7fb0daffd640 1 -- 172.21.15.28:0/3493883065 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb0e003f080 con 0x7fb0e4100ce0 2024-07-03T15:35:49.882 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.210+0000 7fb0e99ec640 1 -- 172.21.15.28:0/3493883065 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7fb0d0035820 msgr2=0x7fb0d0037cd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.28:6800/1434683766 2024-07-03T15:35:49.882 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.210+0000 7fb0e99ec640 1 --2- 172.21.15.28:0/3493883065 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7fb0d0035820 0x7fb0d0037cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-03T15:35:49.882 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.240+0000 7fb0ec451640 1 -- 172.21.15.28:0/3493883065 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fb0e4102b10 con 0x7fb0e4100ce0 2024-07-03T15:35:49.882 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.241+0000 7fb0daffd640 1 -- 172.21.15.28:0/3493883065 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7fb0e0066070 con 0x7fb0e4100ce0 2024-07-03T15:35:49.882 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.243+0000 7fb0ec451640 1 -- 172.21.15.28:0/3493883065 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7fb0d0035820 msgr2=0x7fb0d0037cd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-03T15:35:49.882 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.243+0000 7fb0ec451640 1 --2- 172.21.15.28:0/3493883065 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7fb0d0035820 0x7fb0d0037cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:49.882 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.243+0000 7fb0ec451640 1 -- 172.21.15.28:0/3493883065 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0e4100ce0 msgr2=0x7fb0e4112e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:49.882 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.243+0000 7fb0ec451640 1 --2- 172.21.15.28:0/3493883065 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0e4100ce0 0x7fb0e4112e10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0009810 tx=0x7fb0e002c040 comp rx=0 tx=0).stop 2024-07-03T15:35:49.882 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.243+0000 7fb0ec451640 1 -- 172.21.15.28:0/3493883065 shutdown_connections 2024-07-03T15:35:49.882 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.243+0000 7fb0ec451640 1 --2- 172.21.15.28:0/3493883065 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7fb0d0035820 0x7fb0d0037cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:49.882 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.243+0000 7fb0ec451640 1 --2- 172.21.15.28:0/3493883065 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0e4100ce0 0x7fb0e4112e10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:49.882 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.243+0000 7fb0ec451640 1 -- 172.21.15.28:0/3493883065 >> 172.21.15.28:0/3493883065 conn(0x7fb0e40fc520 msgr2=0x7fb0e40fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:49.883 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.243+0000 7fb0ec451640 1 -- 172.21.15.28:0/3493883065 shutdown_connections 2024-07-03T15:35:49.883 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:49.243+0000 7fb0ec451640 1 -- 172.21.15.28:0/3493883065 wait complete. 2024-07-03T15:35:49.883 INFO:teuthology.orchestra.run.smithi028.stdout:Waiting for the mgr to restart... 2024-07-03T15:35:49.883 INFO:teuthology.orchestra.run.smithi028.stdout:Waiting for mgr epoch 9... 2024-07-03T15:35:50.117 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:49 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/3493883065' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-07-03T15:35:52.870 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:52 smithi028 ceph-mon[29148]: Active manager daemon smithi028.vtqrtz restarted 2024-07-03T15:35:52.870 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:52 smithi028 ceph-mon[29148]: Activating manager daemon smithi028.vtqrtz 2024-07-03T15:35:52.870 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:52 smithi028 ceph-mon[29148]: osdmap e3: 0 total, 0 up, 0 in 2024-07-03T15:35:52.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:52 smithi028 ceph-mon[29148]: mgrmap e10: smithi028.vtqrtz(active, starting, since 0.00320686s) 2024-07-03T15:35:52.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:52 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi028"}]: dispatch 2024-07-03T15:35:52.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:52 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mgr metadata", "who": "smithi028.vtqrtz", "id": "smithi028.vtqrtz"}]: dispatch 2024-07-03T15:35:52.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:52 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-03T15:35:52.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:52 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-03T15:35:52.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:52 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-03T15:35:52.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:52 smithi028 ceph-mon[29148]: Manager daemon smithi028.vtqrtz is now available 2024-07-03T15:35:52.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:52 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:35:52.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:52 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:35:52.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:52 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi028.vtqrtz/mirror_snapshot_schedule"}]: dispatch 2024-07-03T15:35:52.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:52 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi028.vtqrtz/trash_purge_schedule"}]: dispatch 2024-07-03T15:35:54.505 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout { 2024-07-03T15:35:54.505 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-07-03T15:35:54.505 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "initialized": true 2024-07-03T15:35:54.505 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout } 2024-07-03T15:35:54.505 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:50.315+0000 7f5a7e915640 1 Processor -- start 2024-07-03T15:35:54.506 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:50.315+0000 7f5a7e915640 1 -- start start 2024-07-03T15:35:54.506 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:50.315+0000 7f5a7e915640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5a78101dc0 0x7f5a781021d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:54.506 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:50.315+0000 7f5a7e915640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a78102710 con 0x7f5a78101dc0 2024-07-03T15:35:54.506 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:50.316+0000 7f5a77fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5a78101dc0 0x7f5a781021d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:54.506 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:50.316+0000 7f5a77fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5a78101dc0 0x7f5a781021d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41318/0 (socket says 172.21.15.28:41318) 2024-07-03T15:35:54.506 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:50.316+0000 7f5a77fff640 1 -- 172.21.15.28:0/2816940247 learned_addr learned my addr 172.21.15.28:0/2816940247 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:54.506 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:50.316+0000 7f5a77fff640 1 -- 172.21.15.28:0/2816940247 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a78102850 con 0x7f5a78101dc0 2024-07-03T15:35:54.506 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:50.317+0000 7f5a77fff640 1 --2- 172.21.15.28:0/2816940247 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5a78101dc0 0x7f5a781021d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5a68009840 tx=0x7f5a680091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d259391d76080348 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:54.506 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:50.317+0000 7f5a777fe640 1 -- 172.21.15.28:0/2816940247 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5a68032440 con 0x7f5a78101dc0 2024-07-03T15:35:54.506 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:50.317+0000 7f5a777fe640 1 -- 172.21.15.28:0/2816940247 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5a68032a70 con 0x7f5a78101dc0 2024-07-03T15:35:54.506 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:50.317+0000 7f5a777fe640 1 -- 172.21.15.28:0/2816940247 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5a68031ad0 con 0x7f5a78101dc0 2024-07-03T15:35:54.506 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:50.318+0000 7f5a7e915640 1 -- 172.21.15.28:0/2816940247 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5a78101dc0 msgr2=0x7f5a781021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:54.507 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:50.318+0000 7f5a7e915640 1 --2- 172.21.15.28:0/2816940247 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5a78101dc0 0x7f5a781021d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5a68009840 tx=0x7f5a680091d0 comp rx=0 tx=0).stop 2024-07-03T15:35:54.507 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:50.318+0000 7f5a7e915640 1 -- 172.21.15.28:0/2816940247 shutdown_connections 2024-07-03T15:35:54.507 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:50.318+0000 7f5a7e915640 1 --2- 172.21.15.28:0/2816940247 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5a78101dc0 0x7f5a781021d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:54.507 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:50.318+0000 7f5a7e915640 1 -- 172.21.15.28:0/2816940247 >> 172.21.15.28:0/2816940247 conn(0x7f5a780fd5e0 msgr2=0x7f5a780ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:54.507 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:50.318+0000 7f5a7e915640 1 -- 172.21.15.28:0/2816940247 shutdown_connections 2024-07-03T15:35:54.507 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:50.318+0000 7f5a7e915640 1 -- 172.21.15.28:0/2816940247 wait complete. 2024-07-03T15:35:54.507 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:50.318+0000 7f5a7e915640 1 Processor -- start 2024-07-03T15:35:54.507 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:50.319+0000 7f5a7e915640 1 -- start start 2024-07-03T15:35:54.507 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:50.319+0000 7f5a7e915640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5a78101dc0 0x7f5a78110b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:54.507 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:50.319+0000 7f5a7e915640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a78111080 con 0x7f5a78101dc0 2024-07-03T15:35:54.507 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:50.319+0000 7f5a77fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5a78101dc0 0x7f5a78110b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:54.507 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:50.319+0000 7f5a77fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5a78101dc0 0x7f5a78110b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41334/0 (socket says 172.21.15.28:41334) 2024-07-03T15:35:54.507 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:50.319+0000 7f5a77fff640 1 -- 172.21.15.28:0/1919311165 learned_addr learned my addr 172.21.15.28:0/1919311165 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:54.508 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:50.319+0000 7f5a77fff640 1 -- 172.21.15.28:0/1919311165 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a68009450 con 0x7f5a78101dc0 2024-07-03T15:35:54.508 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:50.319+0000 7f5a77fff640 1 --2- 172.21.15.28:0/1919311165 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5a78101dc0 0x7f5a78110b40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5a680098a0 tx=0x7f5a6802c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:54.508 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:50.319+0000 7f5a757fa640 1 -- 172.21.15.28:0/1919311165 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5a68031e90 con 0x7f5a78101dc0 2024-07-03T15:35:54.508 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:50.319+0000 7f5a757fa640 1 -- 172.21.15.28:0/1919311165 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5a6803b910 con 0x7f5a78101dc0 2024-07-03T15:35:54.508 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:50.319+0000 7f5a7e915640 1 -- 172.21.15.28:0/1919311165 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a78111280 con 0x7f5a78101dc0 2024-07-03T15:35:54.508 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:50.319+0000 7f5a757fa640 1 -- 172.21.15.28:0/1919311165 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5a6803a8d0 con 0x7f5a78101dc0 2024-07-03T15:35:54.508 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:50.320+0000 7f5a7e915640 1 -- 172.21.15.28:0/1919311165 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a781116f0 con 0x7f5a78101dc0 2024-07-03T15:35:54.508 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:50.321+0000 7f5a757fa640 1 -- 172.21.15.28:0/1919311165 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 9) v1 ==== 42649+0+0 (secure 0 0 0) 0x7f5a68039070 con 0x7f5a78101dc0 2024-07-03T15:35:54.508 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:50.321+0000 7f5a757fa640 1 --2- 172.21.15.28:0/1919311165 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f5a64035820 0x7f5a64037cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:54.508 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:50.321+0000 7f5a757fa640 1 -- 172.21.15.28:0/1919311165 --> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5a640383e0 con 0x7f5a64035820 2024-07-03T15:35:54.508 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:50.321+0000 7f5a757fa640 1 -- 172.21.15.28:0/1919311165 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f5a68067da0 con 0x7f5a78101dc0 2024-07-03T15:35:54.508 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:50.321+0000 7f5a6ffff640 1 -- 172.21.15.28:0/1919311165 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f5a64035820 msgr2=0x7f5a64037cd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.28:6800/1434683766 2024-07-03T15:35:54.508 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:50.321+0000 7f5a6ffff640 1 --2- 172.21.15.28:0/1919311165 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f5a64035820 0x7f5a64037cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-03T15:35:54.509 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:50.522+0000 7f5a6ffff640 1 -- 172.21.15.28:0/1919311165 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f5a64035820 msgr2=0x7f5a64037cd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.28:6800/1434683766 2024-07-03T15:35:54.509 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:50.522+0000 7f5a6ffff640 1 --2- 172.21.15.28:0/1919311165 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f5a64035820 0x7f5a64037cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-03T15:35:54.509 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:50.923+0000 7f5a6ffff640 1 -- 172.21.15.28:0/1919311165 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f5a64035820 msgr2=0x7f5a64037cd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.28:6800/1434683766 2024-07-03T15:35:54.509 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:50.923+0000 7f5a6ffff640 1 --2- 172.21.15.28:0/1919311165 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f5a64035820 0x7f5a64037cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-07-03T15:35:54.509 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:51.724+0000 7f5a6ffff640 1 -- 172.21.15.28:0/1919311165 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f5a64035820 msgr2=0x7f5a64037cd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.28:6800/1434683766 2024-07-03T15:35:54.509 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:51.724+0000 7f5a6ffff640 1 --2- 172.21.15.28:0/1919311165 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f5a64035820 0x7f5a64037cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-07-03T15:35:54.509 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:52.742+0000 7f5a757fa640 1 -- 172.21.15.28:0/1919311165 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mgrmap(e 10) v1 ==== 42416+0+0 (secure 0 0 0) 0x7f5a68066f40 con 0x7f5a78101dc0 2024-07-03T15:35:54.509 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:52.742+0000 7f5a757fa640 1 -- 172.21.15.28:0/1919311165 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f5a64035820 msgr2=0x7f5a64037cd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-03T15:35:54.509 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:52.742+0000 7f5a757fa640 1 --2- 172.21.15.28:0/1919311165 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f5a64035820 0x7f5a64037cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:54.509 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:53.744+0000 7f5a757fa640 1 -- 172.21.15.28:0/1919311165 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mgrmap(e 11) v1 ==== 42543+0+0 (secure 0 0 0) 0x7f5a68041750 con 0x7f5a78101dc0 2024-07-03T15:35:54.509 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:53.744+0000 7f5a757fa640 1 --2- 172.21.15.28:0/1919311165 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f5a64038fc0 0x7f5a6403b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:54.509 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:53.744+0000 7f5a757fa640 1 -- 172.21.15.28:0/1919311165 --> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5a640383e0 con 0x7f5a64038fc0 2024-07-03T15:35:54.509 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:53.745+0000 7f5a6ffff640 1 --2- 172.21.15.28:0/1919311165 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f5a64038fc0 0x7f5a6403b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:54.509 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:53.745+0000 7f5a6ffff640 1 --2- 172.21.15.28:0/1919311165 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f5a64038fc0 0x7f5a6403b3a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5a60003940 tx=0x7f5a60009350 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:54.510 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:53.746+0000 7f5a757fa640 1 -- 172.21.15.28:0/1919311165 <== mgr.14160 v2:172.21.15.28:6800/577887491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7f5a640383e0 con 0x7f5a64038fc0 2024-07-03T15:35:54.510 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:53.751+0000 7f5a7e915640 1 -- 172.21.15.28:0/1919311165 --> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f5a781116f0 con 0x7f5a64038fc0 2024-07-03T15:35:54.510 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:53.751+0000 7f5a757fa640 1 -- 172.21.15.28:0/1919311165 <== mgr.14160 v2:172.21.15.28:6800/577887491 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f5a781116f0 con 0x7f5a64038fc0 2024-07-03T15:35:54.510 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:53.752+0000 7f5a7e915640 1 -- 172.21.15.28:0/1919311165 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f5a64038fc0 msgr2=0x7f5a6403b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:54.510 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:53.752+0000 7f5a7e915640 1 --2- 172.21.15.28:0/1919311165 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f5a64038fc0 0x7f5a6403b3a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5a60003940 tx=0x7f5a60009350 comp rx=0 tx=0).stop 2024-07-03T15:35:54.510 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:53.752+0000 7f5a7e915640 1 -- 172.21.15.28:0/1919311165 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5a78101dc0 msgr2=0x7f5a78110b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:54.510 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:53.752+0000 7f5a7e915640 1 --2- 172.21.15.28:0/1919311165 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5a78101dc0 0x7f5a78110b40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5a680098a0 tx=0x7f5a6802c040 comp rx=0 tx=0).stop 2024-07-03T15:35:54.510 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:53.752+0000 7f5a7e915640 1 -- 172.21.15.28:0/1919311165 shutdown_connections 2024-07-03T15:35:54.510 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:53.752+0000 7f5a7e915640 1 --2- 172.21.15.28:0/1919311165 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f5a64038fc0 0x7f5a6403b3a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:54.510 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:53.752+0000 7f5a7e915640 1 --2- 172.21.15.28:0/1919311165 >> [v2:172.21.15.28:6800/1434683766,v1:172.21.15.28:6801/1434683766] conn(0x7f5a64035820 0x7f5a64037cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:54.510 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:53.752+0000 7f5a7e915640 1 --2- 172.21.15.28:0/1919311165 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5a78101dc0 0x7f5a78110b40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:54.510 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:53.752+0000 7f5a7e915640 1 -- 172.21.15.28:0/1919311165 >> 172.21.15.28:0/1919311165 conn(0x7f5a780fd5e0 msgr2=0x7f5a780fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:54.510 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:53.752+0000 7f5a7e915640 1 -- 172.21.15.28:0/1919311165 shutdown_connections 2024-07-03T15:35:54.511 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:53.752+0000 7f5a7e915640 1 -- 172.21.15.28:0/1919311165 wait complete. 2024-07-03T15:35:54.511 INFO:teuthology.orchestra.run.smithi028.stdout:mgr epoch 9 is available 2024-07-03T15:35:54.511 INFO:teuthology.orchestra.run.smithi028.stdout:Generating a dashboard self-signed certificate... 2024-07-03T15:35:54.881 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:54 smithi028 ceph-mon[29148]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-07-03T15:35:54.881 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:54 smithi028 ceph-mon[29148]: mgrmap e11: smithi028.vtqrtz(active, since 1.00551s) 2024-07-03T15:35:54.881 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:54 smithi028 ceph-mon[29148]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-07-03T15:35:54.881 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:54 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:35:56.010 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-07-03T15:35:56.011 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.092+0000 7f0038d4d640 1 Processor -- start 2024-07-03T15:35:56.011 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.092+0000 7f0038d4d640 1 -- start start 2024-07-03T15:35:56.011 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.092+0000 7f0038d4d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0034101e00 0x7f0034102210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:56.011 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.092+0000 7f0038d4d640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0034102750 con 0x7f0034101e00 2024-07-03T15:35:56.011 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.093+0000 7f003259c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0034101e00 0x7f0034102210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:56.011 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.093+0000 7f003259c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0034101e00 0x7f0034102210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41432/0 (socket says 172.21.15.28:41432) 2024-07-03T15:35:56.011 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.093+0000 7f003259c640 1 -- 172.21.15.28:0/779217798 learned_addr learned my addr 172.21.15.28:0/779217798 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:56.011 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.093+0000 7f003259c640 1 -- 172.21.15.28:0/779217798 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0034102890 con 0x7f0034101e00 2024-07-03T15:35:56.011 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.094+0000 7f003259c640 1 --2- 172.21.15.28:0/779217798 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0034101e00 0x7f0034102210 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0024009840 tx=0x7f00240091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7bb1c2a3b8361f9 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:56.011 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.094+0000 7f0031d9b640 1 -- 172.21.15.28:0/779217798 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0024032440 con 0x7f0034101e00 2024-07-03T15:35:56.011 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.094+0000 7f0031d9b640 1 -- 172.21.15.28:0/779217798 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0024032a70 con 0x7f0034101e00 2024-07-03T15:35:56.011 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.094+0000 7f0031d9b640 1 -- 172.21.15.28:0/779217798 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0024031ad0 con 0x7f0034101e00 2024-07-03T15:35:56.012 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.095+0000 7f0038d4d640 1 -- 172.21.15.28:0/779217798 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0034101e00 msgr2=0x7f0034102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:56.012 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.095+0000 7f0038d4d640 1 --2- 172.21.15.28:0/779217798 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0034101e00 0x7f0034102210 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0024009840 tx=0x7f00240091d0 comp rx=0 tx=0).stop 2024-07-03T15:35:56.012 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.095+0000 7f0038d4d640 1 -- 172.21.15.28:0/779217798 shutdown_connections 2024-07-03T15:35:56.012 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.095+0000 7f0038d4d640 1 --2- 172.21.15.28:0/779217798 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0034101e00 0x7f0034102210 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:56.012 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.095+0000 7f0038d4d640 1 -- 172.21.15.28:0/779217798 >> 172.21.15.28:0/779217798 conn(0x7f00340fd5a0 msgr2=0x7f00340ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:56.012 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.095+0000 7f0038d4d640 1 -- 172.21.15.28:0/779217798 shutdown_connections 2024-07-03T15:35:56.012 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.095+0000 7f0038d4d640 1 -- 172.21.15.28:0/779217798 wait complete. 2024-07-03T15:35:56.012 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.095+0000 7f0038d4d640 1 Processor -- start 2024-07-03T15:35:56.012 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.095+0000 7f0038d4d640 1 -- start start 2024-07-03T15:35:56.012 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.096+0000 7f0038d4d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0034101e00 0x7f0034110b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:56.012 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.096+0000 7f0038d4d640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0034111090 con 0x7f0034101e00 2024-07-03T15:35:56.012 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.096+0000 7f003259c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0034101e00 0x7f0034110b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:56.013 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.096+0000 7f003259c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0034101e00 0x7f0034110b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41440/0 (socket says 172.21.15.28:41440) 2024-07-03T15:35:56.013 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.096+0000 7f003259c640 1 -- 172.21.15.28:0/1159357081 learned_addr learned my addr 172.21.15.28:0/1159357081 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:56.013 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.096+0000 7f003259c640 1 -- 172.21.15.28:0/1159357081 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0024009450 con 0x7f0034101e00 2024-07-03T15:35:56.013 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.096+0000 7f003259c640 1 --2- 172.21.15.28:0/1159357081 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0034101e00 0x7f0034110b50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0024009810 tx=0x7f002402c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:56.013 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.096+0000 7f002affd640 1 -- 172.21.15.28:0/1159357081 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f00240328b0 con 0x7f0034101e00 2024-07-03T15:35:56.013 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.097+0000 7f002affd640 1 -- 172.21.15.28:0/1159357081 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f002403b910 con 0x7f0034101e00 2024-07-03T15:35:56.013 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.097+0000 7f0038d4d640 1 -- 172.21.15.28:0/1159357081 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0034111290 con 0x7f0034101e00 2024-07-03T15:35:56.013 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.097+0000 7f002affd640 1 -- 172.21.15.28:0/1159357081 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f002403a980 con 0x7f0034101e00 2024-07-03T15:35:56.013 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.097+0000 7f0038d4d640 1 -- 172.21.15.28:0/1159357081 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0034111700 con 0x7f0034101e00 2024-07-03T15:35:56.013 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.097+0000 7f002affd640 1 -- 172.21.15.28:0/1159357081 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 11) v1 ==== 42543+0+0 (secure 0 0 0) 0x7f0024039070 con 0x7f0034101e00 2024-07-03T15:35:56.013 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.098+0000 7f0038d4d640 1 -- 172.21.15.28:0/1159357081 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f003404b530 con 0x7f0034101e00 2024-07-03T15:35:56.013 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.098+0000 7f002affd640 1 --2- 172.21.15.28:0/1159357081 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f00200356a0 0x7f0020037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:56.013 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.098+0000 7f002affd640 1 -- 172.21.15.28:0/1159357081 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f0024067d00 con 0x7f0034101e00 2024-07-03T15:35:56.014 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.098+0000 7f002bfff640 1 --2- 172.21.15.28:0/1159357081 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f00200356a0 0x7f0020037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:56.014 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.098+0000 7f002bfff640 1 --2- 172.21.15.28:0/1159357081 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f00200356a0 0x7f0020037b50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f001c006fd0 tx=0x7f001c006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:56.014 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.106+0000 7f002affd640 1 -- 172.21.15.28:0/1159357081 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0024039500 con 0x7f0034101e00 2024-07-03T15:35:56.014 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.288+0000 7f0038d4d640 1 -- 172.21.15.28:0/1159357081 --> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f0034111700 con 0x7f00200356a0 2024-07-03T15:35:56.014 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.353+0000 7f002affd640 1 -- 172.21.15.28:0/1159357081 <== mgr.14160 v2:172.21.15.28:6800/577887491 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f0034111700 con 0x7f00200356a0 2024-07-03T15:35:56.014 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.355+0000 7f0038d4d640 1 -- 172.21.15.28:0/1159357081 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f00200356a0 msgr2=0x7f0020037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:56.014 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.355+0000 7f0038d4d640 1 --2- 172.21.15.28:0/1159357081 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f00200356a0 0x7f0020037b50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f001c006fd0 tx=0x7f001c006db0 comp rx=0 tx=0).stop 2024-07-03T15:35:56.014 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.355+0000 7f0038d4d640 1 -- 172.21.15.28:0/1159357081 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0034101e00 msgr2=0x7f0034110b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:56.014 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.355+0000 7f0038d4d640 1 --2- 172.21.15.28:0/1159357081 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0034101e00 0x7f0034110b50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0024009810 tx=0x7f002402c040 comp rx=0 tx=0).stop 2024-07-03T15:35:56.014 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.355+0000 7f0038d4d640 1 -- 172.21.15.28:0/1159357081 shutdown_connections 2024-07-03T15:35:56.014 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.355+0000 7f0038d4d640 1 --2- 172.21.15.28:0/1159357081 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f00200356a0 0x7f0020037b50 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:56.015 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.355+0000 7f0038d4d640 1 --2- 172.21.15.28:0/1159357081 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0034101e00 0x7f0034110b50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:56.015 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.355+0000 7f0038d4d640 1 -- 172.21.15.28:0/1159357081 >> 172.21.15.28:0/1159357081 conn(0x7f00340fd5a0 msgr2=0x7f00340fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:56.015 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.355+0000 7f0038d4d640 1 -- 172.21.15.28:0/1159357081 shutdown_connections 2024-07-03T15:35:56.015 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:55.355+0000 7f0038d4d640 1 -- 172.21.15.28:0/1159357081 wait complete. 2024-07-03T15:35:56.015 INFO:teuthology.orchestra.run.smithi028.stdout:Creating initial admin user... 2024-07-03T15:35:56.120 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:55 smithi028 ceph-mon[29148]: [03/Jul/2024:15:35:54] ENGINE Bus STARTING 2024-07-03T15:35:56.120 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:55 smithi028 ceph-mon[29148]: [03/Jul/2024:15:35:54] ENGINE Serving on https://172.21.15.28:7150 2024-07-03T15:35:56.120 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:55 smithi028 ceph-mon[29148]: [03/Jul/2024:15:35:54] ENGINE Bus STARTED 2024-07-03T15:35:56.120 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:55 smithi028 ceph-mon[29148]: [03/Jul/2024:15:35:54] ENGINE Error in HTTPServer.serve 2024-07-03T15:35:56.120 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Traceback (most recent call last): 2024-07-03T15:35:56.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-03T15:35:56.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: self._connections.run(self.expiration_interval) 2024-07-03T15:35:56.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-03T15:35:56.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: self._run(expiration_interval) 2024-07-03T15:35:56.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-03T15:35:56.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-07-03T15:35:56.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-03T15:35:56.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-03T15:35:56.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-03T15:35:56.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: s = self.context.wrap_socket( 2024-07-03T15:35:56.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-03T15:35:56.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: return self.sslsocket_class._create( 2024-07-03T15:35:56.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-03T15:35:56.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: self.do_handshake() 2024-07-03T15:35:56.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-03T15:35:56.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: self._sslobj.do_handshake() 2024-07-03T15:35:56.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-03T15:35:56.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:55 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:35:56.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:55 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:35:56.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:55 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:35:56.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:55 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:35:56.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:55 smithi028 ceph-mon[29148]: mgrmap e12: smithi028.vtqrtz(active, since 2s) 2024-07-03T15:35:57.120 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:56 smithi028 ceph-mon[29148]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:35:57.120 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:56 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:35:57.870 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$1wjd/DDlYbqn/N1mjqdyXeqaYBWneuqpsI0Mh81wMcDNtbNcXy/pi", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1720020956, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-07-03T15:35:57.870 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.426+0000 7f74d5d0f640 1 Processor -- start 2024-07-03T15:35:57.870 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.426+0000 7f74d5d0f640 1 -- start start 2024-07-03T15:35:57.870 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.426+0000 7f74d5d0f640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f74d0100ce0 0x7f74d01010f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:57.870 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.426+0000 7f74d5d0f640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d0101630 con 0x7f74d0100ce0 2024-07-03T15:35:57.871 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.426+0000 7f74cf7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f74d0100ce0 0x7f74d01010f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:57.871 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.426+0000 7f74cf7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f74d0100ce0 0x7f74d01010f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41452/0 (socket says 172.21.15.28:41452) 2024-07-03T15:35:57.871 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.426+0000 7f74cf7fe640 1 -- 172.21.15.28:0/667172832 learned_addr learned my addr 172.21.15.28:0/667172832 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:57.871 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.427+0000 7f74cf7fe640 1 -- 172.21.15.28:0/667172832 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74d0101770 con 0x7f74d0100ce0 2024-07-03T15:35:57.871 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.428+0000 7f74cf7fe640 1 --2- 172.21.15.28:0/667172832 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f74d0100ce0 0x7f74d01010f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f74c0009840 tx=0x7f74c00091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2e4a3fdd605782d server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:57.871 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.428+0000 7f74ce7fc640 1 -- 172.21.15.28:0/667172832 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f74c0032440 con 0x7f74d0100ce0 2024-07-03T15:35:57.871 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.428+0000 7f74ce7fc640 1 -- 172.21.15.28:0/667172832 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f74c0032a70 con 0x7f74d0100ce0 2024-07-03T15:35:57.871 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.428+0000 7f74ce7fc640 1 -- 172.21.15.28:0/667172832 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f74c0031ad0 con 0x7f74d0100ce0 2024-07-03T15:35:57.871 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.428+0000 7f74d5d0f640 1 -- 172.21.15.28:0/667172832 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f74d0100ce0 msgr2=0x7f74d01010f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:57.871 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.428+0000 7f74d5d0f640 1 --2- 172.21.15.28:0/667172832 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f74d0100ce0 0x7f74d01010f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f74c0009840 tx=0x7f74c00091d0 comp rx=0 tx=0).stop 2024-07-03T15:35:57.871 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.428+0000 7f74d5d0f640 1 -- 172.21.15.28:0/667172832 shutdown_connections 2024-07-03T15:35:57.871 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.428+0000 7f74d5d0f640 1 --2- 172.21.15.28:0/667172832 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f74d0100ce0 0x7f74d01010f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:57.871 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.428+0000 7f74d5d0f640 1 -- 172.21.15.28:0/667172832 >> 172.21.15.28:0/667172832 conn(0x7f74d00fc520 msgr2=0x7f74d00fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:57.871 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.429+0000 7f74d5d0f640 1 -- 172.21.15.28:0/667172832 shutdown_connections 2024-07-03T15:35:57.872 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.429+0000 7f74d5d0f640 1 -- 172.21.15.28:0/667172832 wait complete. 2024-07-03T15:35:57.872 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.429+0000 7f74d5d0f640 1 Processor -- start 2024-07-03T15:35:57.872 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.429+0000 7f74d5d0f640 1 -- start start 2024-07-03T15:35:57.872 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.429+0000 7f74d5d0f640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f74d0100ce0 0x7f74d0075b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:57.872 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.429+0000 7f74d5d0f640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d0113840 con 0x7f74d0100ce0 2024-07-03T15:35:57.872 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.430+0000 7f74cf7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f74d0100ce0 0x7f74d0075b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:57.872 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.430+0000 7f74cf7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f74d0100ce0 0x7f74d0075b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41458/0 (socket says 172.21.15.28:41458) 2024-07-03T15:35:57.872 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.430+0000 7f74cf7fe640 1 -- 172.21.15.28:0/2516683625 learned_addr learned my addr 172.21.15.28:0/2516683625 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:57.872 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.430+0000 7f74cf7fe640 1 -- 172.21.15.28:0/2516683625 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c0009450 con 0x7f74d0100ce0 2024-07-03T15:35:57.872 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.430+0000 7f74cf7fe640 1 --2- 172.21.15.28:0/2516683625 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f74d0100ce0 0x7f74d0075b00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f74c0006a70 tx=0x7f74c002c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:57.872 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.430+0000 7f74b7fff640 1 -- 172.21.15.28:0/2516683625 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f74c0031e90 con 0x7f74d0100ce0 2024-07-03T15:35:57.872 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.430+0000 7f74b7fff640 1 -- 172.21.15.28:0/2516683625 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f74c003b910 con 0x7f74d0100ce0 2024-07-03T15:35:57.872 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.430+0000 7f74d5d0f640 1 -- 172.21.15.28:0/2516683625 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74d0113980 con 0x7f74d0100ce0 2024-07-03T15:35:57.873 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.430+0000 7f74b7fff640 1 -- 172.21.15.28:0/2516683625 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f74c003a940 con 0x7f74d0100ce0 2024-07-03T15:35:57.873 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.430+0000 7f74d5d0f640 1 -- 172.21.15.28:0/2516683625 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74d0113d90 con 0x7f74d0100ce0 2024-07-03T15:35:57.873 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.431+0000 7f74b7fff640 1 -- 172.21.15.28:0/2516683625 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 12) v1 ==== 42649+0+0 (secure 0 0 0) 0x7f74c0039070 con 0x7f74d0100ce0 2024-07-03T15:35:57.873 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.431+0000 7f74b7fff640 1 --2- 172.21.15.28:0/2516683625 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f74b8035820 0x7f74b8037cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:57.873 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.431+0000 7f74b7fff640 1 -- 172.21.15.28:0/2516683625 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f74c00681f0 con 0x7f74d0100ce0 2024-07-03T15:35:57.873 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.431+0000 7f74d5d0f640 1 -- 172.21.15.28:0/2516683625 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74d004b530 con 0x7f74d0100ce0 2024-07-03T15:35:57.873 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.431+0000 7f74ceffd640 1 --2- 172.21.15.28:0/2516683625 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f74b8035820 0x7f74b8037cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:57.873 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.432+0000 7f74ceffd640 1 --2- 172.21.15.28:0/2516683625 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f74b8035820 0x7f74b8037cd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f74c4006fd0 tx=0x7f74c4006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:57.873 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.436+0000 7f74b7fff640 1 -- 172.21.15.28:0/2516683625 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f74c003f080 con 0x7f74d0100ce0 2024-07-03T15:35:57.873 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.636+0000 7f74d5d0f640 1 -- 172.21.15.28:0/2516683625 --> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f74d0113fd0 con 0x7f74b8035820 2024-07-03T15:35:57.873 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.885+0000 7f74b7fff640 1 -- 172.21.15.28:0/2516683625 <== mgr.14160 v2:172.21.15.28:6800/577887491 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f74d0113fd0 con 0x7f74b8035820 2024-07-03T15:35:57.873 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.887+0000 7f74d5d0f640 1 -- 172.21.15.28:0/2516683625 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f74b8035820 msgr2=0x7f74b8037cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:57.873 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.887+0000 7f74d5d0f640 1 --2- 172.21.15.28:0/2516683625 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f74b8035820 0x7f74b8037cd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f74c4006fd0 tx=0x7f74c4006db0 comp rx=0 tx=0).stop 2024-07-03T15:35:57.874 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.887+0000 7f74d5d0f640 1 -- 172.21.15.28:0/2516683625 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f74d0100ce0 msgr2=0x7f74d0075b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:57.874 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.887+0000 7f74d5d0f640 1 --2- 172.21.15.28:0/2516683625 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f74d0100ce0 0x7f74d0075b00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f74c0006a70 tx=0x7f74c002c040 comp rx=0 tx=0).stop 2024-07-03T15:35:57.874 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.887+0000 7f74d5d0f640 1 -- 172.21.15.28:0/2516683625 shutdown_connections 2024-07-03T15:35:57.874 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.887+0000 7f74d5d0f640 1 --2- 172.21.15.28:0/2516683625 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f74b8035820 0x7f74b8037cd0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:57.874 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.887+0000 7f74d5d0f640 1 --2- 172.21.15.28:0/2516683625 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f74d0100ce0 0x7f74d0075b00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:57.874 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.887+0000 7f74d5d0f640 1 -- 172.21.15.28:0/2516683625 >> 172.21.15.28:0/2516683625 conn(0x7f74d00fc520 msgr2=0x7f74d00fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:57.874 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.887+0000 7f74d5d0f640 1 -- 172.21.15.28:0/2516683625 shutdown_connections 2024-07-03T15:35:57.874 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:56.887+0000 7f74d5d0f640 1 -- 172.21.15.28:0/2516683625 wait complete. 2024-07-03T15:35:57.874 INFO:teuthology.orchestra.run.smithi028.stdout:Fetching dashboard port number... 2024-07-03T15:35:58.106 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:57 smithi028 ceph-mon[29148]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:35:59.096 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:58 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/2255446496' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-07-03T15:35:59.378 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout 8443 2024-07-03T15:35:59.379 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.275+0000 7fd057ffe640 1 Processor -- start 2024-07-03T15:35:59.379 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.275+0000 7fd057ffe640 1 -- start start 2024-07-03T15:35:59.379 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.275+0000 7fd057ffe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd050101e00 0x7fd050102210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:59.379 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.275+0000 7fd057ffe640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd050102750 con 0x7fd050101e00 2024-07-03T15:35:59.379 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.276+0000 7fd055d9a640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd050101e00 0x7fd050102210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:59.379 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.276+0000 7fd055d9a640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd050101e00 0x7fd050102210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41460/0 (socket says 172.21.15.28:41460) 2024-07-03T15:35:59.379 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.276+0000 7fd055d9a640 1 -- 172.21.15.28:0/3858200045 learned_addr learned my addr 172.21.15.28:0/3858200045 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:59.379 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.276+0000 7fd055d9a640 1 -- 172.21.15.28:0/3858200045 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd050102890 con 0x7fd050101e00 2024-07-03T15:35:59.379 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.277+0000 7fd055d9a640 1 --2- 172.21.15.28:0/3858200045 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd050101e00 0x7fd050102210 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd04c009840 tx=0x7fd04c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9c65c78b4ad3d5 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:59.379 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.277+0000 7fd054d98640 1 -- 172.21.15.28:0/3858200045 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd04c032440 con 0x7fd050101e00 2024-07-03T15:35:59.379 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.277+0000 7fd054d98640 1 -- 172.21.15.28:0/3858200045 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd04c032a70 con 0x7fd050101e00 2024-07-03T15:35:59.380 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.277+0000 7fd054d98640 1 -- 172.21.15.28:0/3858200045 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd04c031ad0 con 0x7fd050101e00 2024-07-03T15:35:59.380 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.277+0000 7fd057ffe640 1 -- 172.21.15.28:0/3858200045 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd050101e00 msgr2=0x7fd050102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:59.380 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.277+0000 7fd057ffe640 1 --2- 172.21.15.28:0/3858200045 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd050101e00 0x7fd050102210 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd04c009840 tx=0x7fd04c0091d0 comp rx=0 tx=0).stop 2024-07-03T15:35:59.380 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.277+0000 7fd057ffe640 1 -- 172.21.15.28:0/3858200045 shutdown_connections 2024-07-03T15:35:59.380 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.277+0000 7fd057ffe640 1 --2- 172.21.15.28:0/3858200045 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd050101e00 0x7fd050102210 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:59.380 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.277+0000 7fd057ffe640 1 -- 172.21.15.28:0/3858200045 >> 172.21.15.28:0/3858200045 conn(0x7fd0500fd5a0 msgr2=0x7fd0500ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:59.380 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.278+0000 7fd057ffe640 1 -- 172.21.15.28:0/3858200045 shutdown_connections 2024-07-03T15:35:59.380 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.278+0000 7fd057ffe640 1 -- 172.21.15.28:0/3858200045 wait complete. 2024-07-03T15:35:59.380 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.278+0000 7fd057ffe640 1 Processor -- start 2024-07-03T15:35:59.380 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.278+0000 7fd057ffe640 1 -- start start 2024-07-03T15:35:59.380 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.278+0000 7fd057ffe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd050101e00 0x7fd050077c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:59.380 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.278+0000 7fd057ffe640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd050078160 con 0x7fd050101e00 2024-07-03T15:35:59.380 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.278+0000 7fd055d9a640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd050101e00 0x7fd050077c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:59.380 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.278+0000 7fd055d9a640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd050101e00 0x7fd050077c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41466/0 (socket says 172.21.15.28:41466) 2024-07-03T15:35:59.381 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.278+0000 7fd055d9a640 1 -- 172.21.15.28:0/2255446496 learned_addr learned my addr 172.21.15.28:0/2255446496 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:35:59.381 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.279+0000 7fd055d9a640 1 -- 172.21.15.28:0/2255446496 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd04c009450 con 0x7fd050101e00 2024-07-03T15:35:59.381 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.279+0000 7fd055d9a640 1 --2- 172.21.15.28:0/2255446496 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd050101e00 0x7fd050077c20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd04c009810 tx=0x7fd04c02c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:59.381 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.279+0000 7fd0467fc640 1 -- 172.21.15.28:0/2255446496 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd04c0328b0 con 0x7fd050101e00 2024-07-03T15:35:59.381 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.279+0000 7fd0467fc640 1 -- 172.21.15.28:0/2255446496 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd04c03b910 con 0x7fd050101e00 2024-07-03T15:35:59.381 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.279+0000 7fd057ffe640 1 -- 172.21.15.28:0/2255446496 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0500761f0 con 0x7fd050101e00 2024-07-03T15:35:59.381 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.279+0000 7fd0467fc640 1 -- 172.21.15.28:0/2255446496 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd04c03a980 con 0x7fd050101e00 2024-07-03T15:35:59.381 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.279+0000 7fd057ffe640 1 -- 172.21.15.28:0/2255446496 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd050076660 con 0x7fd050101e00 2024-07-03T15:35:59.381 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.280+0000 7fd0467fc640 1 -- 172.21.15.28:0/2255446496 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 12) v1 ==== 42649+0+0 (secure 0 0 0) 0x7fd04c039070 con 0x7fd050101e00 2024-07-03T15:35:59.381 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.280+0000 7fd057ffe640 1 -- 172.21.15.28:0/2255446496 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd05004b530 con 0x7fd050101e00 2024-07-03T15:35:59.381 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.280+0000 7fd0467fc640 1 --2- 172.21.15.28:0/2255446496 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7fd03c035820 0x7fd03c037cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:35:59.381 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.280+0000 7fd0467fc640 1 -- 172.21.15.28:0/2255446496 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fd04c068070 con 0x7fd050101e00 2024-07-03T15:35:59.381 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.280+0000 7fd055599640 1 --2- 172.21.15.28:0/2255446496 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7fd03c035820 0x7fd03c037cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:35:59.382 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.281+0000 7fd055599640 1 --2- 172.21.15.28:0/2255446496 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7fd03c035820 0x7fd03c037cd0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd040006fd0 tx=0x7fd040006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:35:59.382 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.285+0000 7fd0467fc640 1 -- 172.21.15.28:0/2255446496 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd04c039500 con 0x7fd050101e00 2024-07-03T15:35:59.382 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.461+0000 7fd057ffe640 1 -- 172.21.15.28:0/2255446496 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fd050103c50 con 0x7fd050101e00 2024-07-03T15:35:59.382 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.461+0000 7fd0467fc640 1 -- 172.21.15.28:0/2255446496 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fd04c03f070 con 0x7fd050101e00 2024-07-03T15:35:59.382 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.463+0000 7fd057ffe640 1 -- 172.21.15.28:0/2255446496 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7fd03c035820 msgr2=0x7fd03c037cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:59.382 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.463+0000 7fd057ffe640 1 --2- 172.21.15.28:0/2255446496 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7fd03c035820 0x7fd03c037cd0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd040006fd0 tx=0x7fd040006db0 comp rx=0 tx=0).stop 2024-07-03T15:35:59.382 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.463+0000 7fd057ffe640 1 -- 172.21.15.28:0/2255446496 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd050101e00 msgr2=0x7fd050077c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:35:59.382 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.463+0000 7fd057ffe640 1 --2- 172.21.15.28:0/2255446496 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd050101e00 0x7fd050077c20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd04c009810 tx=0x7fd04c02c040 comp rx=0 tx=0).stop 2024-07-03T15:35:59.382 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.464+0000 7fd057ffe640 1 -- 172.21.15.28:0/2255446496 shutdown_connections 2024-07-03T15:35:59.382 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.464+0000 7fd057ffe640 1 --2- 172.21.15.28:0/2255446496 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7fd03c035820 0x7fd03c037cd0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:59.382 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.464+0000 7fd057ffe640 1 --2- 172.21.15.28:0/2255446496 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd050101e00 0x7fd050077c20 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:35:59.382 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.464+0000 7fd057ffe640 1 -- 172.21.15.28:0/2255446496 >> 172.21.15.28:0/2255446496 conn(0x7fd0500fd5a0 msgr2=0x7fd0500fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:35:59.383 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.464+0000 7fd057ffe640 1 -- 172.21.15.28:0/2255446496 shutdown_connections 2024-07-03T15:35:59.383 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:58.464+0000 7fd057ffe640 1 -- 172.21.15.28:0/2255446496 wait complete. 2024-07-03T15:35:59.387 INFO:teuthology.orchestra.run.smithi028.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-07-03T15:35:59.387 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stdout disabled 2024-07-03T15:35:59.396 INFO:teuthology.orchestra.run.smithi028.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-07-03T15:35:59.396 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stdout inactive 2024-07-03T15:35:59.396 INFO:teuthology.orchestra.run.smithi028.stdout:firewalld.service is not enabled 2024-07-03T15:35:59.396 INFO:teuthology.orchestra.run.smithi028.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-07-03T15:35:59.398 INFO:teuthology.orchestra.run.smithi028.stdout:Ceph Dashboard is now available at: 2024-07-03T15:35:59.398 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:35:59.398 INFO:teuthology.orchestra.run.smithi028.stdout: URL: https://smithi028.front.sepia.ceph.com:8443/ 2024-07-03T15:35:59.398 INFO:teuthology.orchestra.run.smithi028.stdout: User: admin 2024-07-03T15:35:59.398 INFO:teuthology.orchestra.run.smithi028.stdout: Password: u1nbwwadal 2024-07-03T15:35:59.398 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:35:59.399 INFO:teuthology.orchestra.run.smithi028.stdout:Saving cluster configuration to /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/config directory 2024-07-03T15:35:59.399 INFO:teuthology.orchestra.run.smithi028.stdout:Enabling autotune for osd_memory_target 2024-07-03T15:36:00.276 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:35:59 smithi028 ceph-mon[29148]: mgrmap e13: smithi028.vtqrtz(active, since 6s) 2024-07-03T15:36:00.568 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:59.822+0000 7fb5fb980640 1 Processor -- start 2024-07-03T15:36:00.568 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:59.823+0000 7fb5fb980640 1 -- start start 2024-07-03T15:36:00.568 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:59.823+0000 7fb5fb980640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5f4101e00 0x7fb5f4102210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:36:00.568 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:59.823+0000 7fb5fb980640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f4102750 con 0x7fb5f4101e00 2024-07-03T15:36:00.568 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:59.823+0000 7fb5f971c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5f4101e00 0x7fb5f4102210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:36:00.568 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:59.823+0000 7fb5f971c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5f4101e00 0x7fb5f4102210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45696/0 (socket says 172.21.15.28:45696) 2024-07-03T15:36:00.568 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:59.823+0000 7fb5f971c640 1 -- 172.21.15.28:0/844819146 learned_addr learned my addr 172.21.15.28:0/844819146 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:36:00.568 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:59.823+0000 7fb5f971c640 1 -- 172.21.15.28:0/844819146 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f4102890 con 0x7fb5f4101e00 2024-07-03T15:36:00.568 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:59.824+0000 7fb5f971c640 1 --2- 172.21.15.28:0/844819146 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5f4101e00 0x7fb5f4102210 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0009840 tx=0x7fb5f00091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff877630aea71569 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:36:00.568 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:59.824+0000 7fb5ebfff640 1 -- 172.21.15.28:0/844819146 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb5f0032440 con 0x7fb5f4101e00 2024-07-03T15:36:00.569 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:59.824+0000 7fb5ebfff640 1 -- 172.21.15.28:0/844819146 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb5f0032a70 con 0x7fb5f4101e00 2024-07-03T15:36:00.569 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:59.824+0000 7fb5ebfff640 1 -- 172.21.15.28:0/844819146 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb5f0031ad0 con 0x7fb5f4101e00 2024-07-03T15:36:00.569 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:59.825+0000 7fb5fb980640 1 -- 172.21.15.28:0/844819146 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5f4101e00 msgr2=0x7fb5f4102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:00.569 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:59.825+0000 7fb5fb980640 1 --2- 172.21.15.28:0/844819146 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5f4101e00 0x7fb5f4102210 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0009840 tx=0x7fb5f00091d0 comp rx=0 tx=0).stop 2024-07-03T15:36:00.569 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:59.825+0000 7fb5fb980640 1 -- 172.21.15.28:0/844819146 shutdown_connections 2024-07-03T15:36:00.569 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:59.825+0000 7fb5fb980640 1 --2- 172.21.15.28:0/844819146 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5f4101e00 0x7fb5f4102210 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:00.569 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:59.825+0000 7fb5fb980640 1 -- 172.21.15.28:0/844819146 >> 172.21.15.28:0/844819146 conn(0x7fb5f40fd5a0 msgr2=0x7fb5f40ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:36:00.569 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:59.825+0000 7fb5fb980640 1 -- 172.21.15.28:0/844819146 shutdown_connections 2024-07-03T15:36:00.569 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:59.825+0000 7fb5fb980640 1 -- 172.21.15.28:0/844819146 wait complete. 2024-07-03T15:36:00.569 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:59.825+0000 7fb5fb980640 1 Processor -- start 2024-07-03T15:36:00.569 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:59.825+0000 7fb5fb980640 1 -- start start 2024-07-03T15:36:00.569 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:59.826+0000 7fb5fb980640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5f4101e00 0x7fb5f4110b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:36:00.569 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:59.826+0000 7fb5fb980640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f4111090 con 0x7fb5f4101e00 2024-07-03T15:36:00.569 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:59.826+0000 7fb5f971c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5f4101e00 0x7fb5f4110b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:36:00.570 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:59.826+0000 7fb5f971c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5f4101e00 0x7fb5f4110b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45700/0 (socket says 172.21.15.28:45700) 2024-07-03T15:36:00.570 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:59.826+0000 7fb5f971c640 1 -- 172.21.15.28:0/3598716567 learned_addr learned my addr 172.21.15.28:0/3598716567 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:36:00.570 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:59.826+0000 7fb5f971c640 1 -- 172.21.15.28:0/3598716567 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f0009450 con 0x7fb5f4101e00 2024-07-03T15:36:00.570 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:59.826+0000 7fb5f971c640 1 --2- 172.21.15.28:0/3598716567 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5f4101e00 0x7fb5f4110b50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0009810 tx=0x7fb5f002c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:36:00.570 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:59.826+0000 7fb5e9ffb640 1 -- 172.21.15.28:0/3598716567 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb5f00328b0 con 0x7fb5f4101e00 2024-07-03T15:36:00.570 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:59.826+0000 7fb5e9ffb640 1 -- 172.21.15.28:0/3598716567 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb5f003b910 con 0x7fb5f4101e00 2024-07-03T15:36:00.570 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:59.826+0000 7fb5e9ffb640 1 -- 172.21.15.28:0/3598716567 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb5f003a980 con 0x7fb5f4101e00 2024-07-03T15:36:00.570 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:59.826+0000 7fb5fb980640 1 -- 172.21.15.28:0/3598716567 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5f4111290 con 0x7fb5f4101e00 2024-07-03T15:36:00.570 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:59.827+0000 7fb5fb980640 1 -- 172.21.15.28:0/3598716567 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5f4111700 con 0x7fb5f4101e00 2024-07-03T15:36:00.570 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:59.827+0000 7fb5e9ffb640 1 -- 172.21.15.28:0/3598716567 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 13) v1 ==== 42692+0+0 (secure 0 0 0) 0x7fb5f0039070 con 0x7fb5f4101e00 2024-07-03T15:36:00.570 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:59.828+0000 7fb5e9ffb640 1 --2- 172.21.15.28:0/3598716567 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7fb5e0035870 0x7fb5e0037d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:36:00.570 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:59.828+0000 7fb5e9ffb640 1 -- 172.21.15.28:0/3598716567 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fb5f0068030 con 0x7fb5f4101e00 2024-07-03T15:36:00.570 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:59.828+0000 7fb5f8f1b640 1 --2- 172.21.15.28:0/3598716567 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7fb5e0035870 0x7fb5e0037d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:36:00.571 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:59.828+0000 7fb5fb980640 1 -- 172.21.15.28:0/3598716567 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5f405eb80 con 0x7fb5f4101e00 2024-07-03T15:36:00.571 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:59.828+0000 7fb5f8f1b640 1 --2- 172.21.15.28:0/3598716567 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7fb5e0035870 0x7fb5e0037d20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4006fd0 tx=0x7fb5e4006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:36:00.571 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:35:59.832+0000 7fb5e9ffb640 1 -- 172.21.15.28:0/3598716567 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb5f0037070 con 0x7fb5f4101e00 2024-07-03T15:36:00.571 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.008+0000 7fb5fb980640 1 -- 172.21.15.28:0/3598716567 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fb5f4103c50 con 0x7fb5f4101e00 2024-07-03T15:36:00.571 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.008+0000 7fb5e9ffb640 1 -- 172.21.15.28:0/3598716567 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7fb5f00530c0 con 0x7fb5f4101e00 2024-07-03T15:36:00.571 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.010+0000 7fb5fb980640 1 -- 172.21.15.28:0/3598716567 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7fb5e0035870 msgr2=0x7fb5e0037d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:00.571 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.010+0000 7fb5fb980640 1 --2- 172.21.15.28:0/3598716567 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7fb5e0035870 0x7fb5e0037d20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4006fd0 tx=0x7fb5e4006db0 comp rx=0 tx=0).stop 2024-07-03T15:36:00.571 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.010+0000 7fb5fb980640 1 -- 172.21.15.28:0/3598716567 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5f4101e00 msgr2=0x7fb5f4110b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:00.571 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.010+0000 7fb5fb980640 1 --2- 172.21.15.28:0/3598716567 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5f4101e00 0x7fb5f4110b50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0009810 tx=0x7fb5f002c040 comp rx=0 tx=0).stop 2024-07-03T15:36:00.571 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.010+0000 7fb5fb980640 1 -- 172.21.15.28:0/3598716567 shutdown_connections 2024-07-03T15:36:00.571 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.010+0000 7fb5fb980640 1 --2- 172.21.15.28:0/3598716567 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7fb5e0035870 0x7fb5e0037d20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:00.571 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.010+0000 7fb5fb980640 1 --2- 172.21.15.28:0/3598716567 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5f4101e00 0x7fb5f4110b50 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:00.571 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.010+0000 7fb5fb980640 1 -- 172.21.15.28:0/3598716567 >> 172.21.15.28:0/3598716567 conn(0x7fb5f40fd5a0 msgr2=0x7fb5f40fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:36:00.571 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.010+0000 7fb5fb980640 1 -- 172.21.15.28:0/3598716567 shutdown_connections 2024-07-03T15:36:00.572 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.010+0000 7fb5fb980640 1 -- 172.21.15.28:0/3598716567 wait complete. 2024-07-03T15:36:01.800 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.935+0000 7feb5a81a640 1 Processor -- start 2024-07-03T15:36:01.800 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.935+0000 7feb5a81a640 1 -- start start 2024-07-03T15:36:01.800 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.936+0000 7feb5a81a640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feb54075bc0 0x7feb54104a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:36:01.800 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.936+0000 7feb5a81a640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb54104f70 con 0x7feb54075bc0 2024-07-03T15:36:01.800 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.936+0000 7feb53fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feb54075bc0 0x7feb54104a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:36:01.800 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.936+0000 7feb53fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feb54075bc0 0x7feb54104a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45712/0 (socket says 172.21.15.28:45712) 2024-07-03T15:36:01.800 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.936+0000 7feb53fff640 1 -- 172.21.15.28:0/1073668868 learned_addr learned my addr 172.21.15.28:0/1073668868 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:36:01.800 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.936+0000 7feb53fff640 1 -- 172.21.15.28:0/1073668868 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb541050b0 con 0x7feb54075bc0 2024-07-03T15:36:01.800 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.937+0000 7feb53fff640 1 --2- 172.21.15.28:0/1073668868 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feb54075bc0 0x7feb54104a30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7feb44009840 tx=0x7feb440091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e369a540cc1d818c server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:36:01.800 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.937+0000 7feb52ffd640 1 -- 172.21.15.28:0/1073668868 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feb44032440 con 0x7feb54075bc0 2024-07-03T15:36:01.801 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.937+0000 7feb52ffd640 1 -- 172.21.15.28:0/1073668868 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feb44032a70 con 0x7feb54075bc0 2024-07-03T15:36:01.801 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.937+0000 7feb52ffd640 1 -- 172.21.15.28:0/1073668868 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feb44031ad0 con 0x7feb54075bc0 2024-07-03T15:36:01.801 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.938+0000 7feb5a81a640 1 -- 172.21.15.28:0/1073668868 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feb54075bc0 msgr2=0x7feb54104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:01.801 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.938+0000 7feb5a81a640 1 --2- 172.21.15.28:0/1073668868 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feb54075bc0 0x7feb54104a30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7feb44009840 tx=0x7feb440091d0 comp rx=0 tx=0).stop 2024-07-03T15:36:01.801 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.938+0000 7feb5a81a640 1 -- 172.21.15.28:0/1073668868 shutdown_connections 2024-07-03T15:36:01.801 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.938+0000 7feb5a81a640 1 --2- 172.21.15.28:0/1073668868 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feb54075bc0 0x7feb54104a30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:01.801 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.938+0000 7feb5a81a640 1 -- 172.21.15.28:0/1073668868 >> 172.21.15.28:0/1073668868 conn(0x7feb540fc520 msgr2=0x7feb540fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:36:01.801 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.938+0000 7feb5a81a640 1 -- 172.21.15.28:0/1073668868 shutdown_connections 2024-07-03T15:36:01.801 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.938+0000 7feb5a81a640 1 -- 172.21.15.28:0/1073668868 wait complete. 2024-07-03T15:36:01.801 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.938+0000 7feb5a81a640 1 Processor -- start 2024-07-03T15:36:01.801 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.938+0000 7feb5a81a640 1 -- start start 2024-07-03T15:36:01.801 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.939+0000 7feb5a81a640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feb54075bc0 0x7feb54102360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:36:01.802 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.939+0000 7feb5a81a640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb541028a0 con 0x7feb54075bc0 2024-07-03T15:36:01.802 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.939+0000 7feb53fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feb54075bc0 0x7feb54102360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:36:01.802 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.939+0000 7feb53fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feb54075bc0 0x7feb54102360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45722/0 (socket says 172.21.15.28:45722) 2024-07-03T15:36:01.802 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.939+0000 7feb53fff640 1 -- 172.21.15.28:0/3509582069 learned_addr learned my addr 172.21.15.28:0/3509582069 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:36:01.802 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.939+0000 7feb53fff640 1 -- 172.21.15.28:0/3509582069 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb44009450 con 0x7feb54075bc0 2024-07-03T15:36:01.802 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.939+0000 7feb53fff640 1 --2- 172.21.15.28:0/3509582069 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feb54075bc0 0x7feb54102360 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7feb440317f0 tx=0x7feb4402c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:36:01.802 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.939+0000 7feb50ff9640 1 -- 172.21.15.28:0/3509582069 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feb44032900 con 0x7feb54075bc0 2024-07-03T15:36:01.802 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.939+0000 7feb50ff9640 1 -- 172.21.15.28:0/3509582069 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feb4403b910 con 0x7feb54075bc0 2024-07-03T15:36:01.802 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.939+0000 7feb5a81a640 1 -- 172.21.15.28:0/3509582069 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb54100930 con 0x7feb54075bc0 2024-07-03T15:36:01.803 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.939+0000 7feb50ff9640 1 -- 172.21.15.28:0/3509582069 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feb4403a960 con 0x7feb54075bc0 2024-07-03T15:36:01.803 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.939+0000 7feb5a81a640 1 -- 172.21.15.28:0/3509582069 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb54100da0 con 0x7feb54075bc0 2024-07-03T15:36:01.803 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.940+0000 7feb50ff9640 1 -- 172.21.15.28:0/3509582069 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 13) v1 ==== 42692+0+0 (secure 0 0 0) 0x7feb44039070 con 0x7feb54075bc0 2024-07-03T15:36:01.803 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.941+0000 7feb50ff9640 1 --2- 172.21.15.28:0/3509582069 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7feb3c035870 0x7feb3c037d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:36:01.803 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.941+0000 7feb50ff9640 1 -- 172.21.15.28:0/3509582069 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7feb440685c0 con 0x7feb54075bc0 2024-07-03T15:36:01.803 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.941+0000 7feb5a81a640 1 -- 172.21.15.28:0/3509582069 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb5404b530 con 0x7feb54075bc0 2024-07-03T15:36:01.803 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.941+0000 7feb537fe640 1 --2- 172.21.15.28:0/3509582069 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7feb3c035870 0x7feb3c037d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:36:01.803 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.941+0000 7feb537fe640 1 --2- 172.21.15.28:0/3509582069 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7feb3c035870 0x7feb3c037d20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7feb48006fd0 tx=0x7feb48006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:36:01.803 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:00.945+0000 7feb50ff9640 1 -- 172.21.15.28:0/3509582069 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7feb4403f020 con 0x7feb54075bc0 2024-07-03T15:36:01.803 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:01.212+0000 7feb5a81a640 1 -- 172.21.15.28:0/3509582069 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7feb54105e40 con 0x7feb54075bc0 2024-07-03T15:36:01.803 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:01.214+0000 7feb50ff9640 1 -- 172.21.15.28:0/3509582069 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v33)=0 set mgr/dashboard/cluster/status v33) v1 ==== 153+0+0 (secure 0 0 0) 0x7feb54105e40 con 0x7feb54075bc0 2024-07-03T15:36:01.803 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-07-03T15:36:01.804 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:01.216+0000 7feb5a81a640 1 -- 172.21.15.28:0/3509582069 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7feb3c035870 msgr2=0x7feb3c037d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:01.804 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:01.216+0000 7feb5a81a640 1 --2- 172.21.15.28:0/3509582069 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7feb3c035870 0x7feb3c037d20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7feb48006fd0 tx=0x7feb48006db0 comp rx=0 tx=0).stop 2024-07-03T15:36:01.804 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:01.216+0000 7feb5a81a640 1 -- 172.21.15.28:0/3509582069 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feb54075bc0 msgr2=0x7feb54102360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:01.804 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:01.216+0000 7feb5a81a640 1 --2- 172.21.15.28:0/3509582069 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feb54075bc0 0x7feb54102360 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7feb440317f0 tx=0x7feb4402c040 comp rx=0 tx=0).stop 2024-07-03T15:36:01.804 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:01.216+0000 7feb5a81a640 1 -- 172.21.15.28:0/3509582069 shutdown_connections 2024-07-03T15:36:01.804 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:01.216+0000 7feb5a81a640 1 --2- 172.21.15.28:0/3509582069 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7feb3c035870 0x7feb3c037d20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:01.804 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:01.216+0000 7feb5a81a640 1 --2- 172.21.15.28:0/3509582069 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feb54075bc0 0x7feb54102360 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:01.804 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:01.216+0000 7feb5a81a640 1 -- 172.21.15.28:0/3509582069 >> 172.21.15.28:0/3509582069 conn(0x7feb540fc520 msgr2=0x7feb540fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:36:01.804 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:01.216+0000 7feb5a81a640 1 -- 172.21.15.28:0/3509582069 shutdown_connections 2024-07-03T15:36:01.804 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-07-03T15:36:01.216+0000 7feb5a81a640 1 -- 172.21.15.28:0/3509582069 wait complete. 2024-07-03T15:36:01.804 INFO:teuthology.orchestra.run.smithi028.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-07-03T15:36:01.804 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:36:01.804 INFO:teuthology.orchestra.run.smithi028.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-07-03T15:36:01.804 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:36:01.805 INFO:teuthology.orchestra.run.smithi028.stdout:Or, if you are only running a single cluster on this host: 2024-07-03T15:36:01.805 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:36:01.805 INFO:teuthology.orchestra.run.smithi028.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-07-03T15:36:01.805 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:36:01.805 INFO:teuthology.orchestra.run.smithi028.stdout:Please consider enabling telemetry to help improve Ceph: 2024-07-03T15:36:01.805 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:36:01.805 INFO:teuthology.orchestra.run.smithi028.stdout: ceph telemetry on 2024-07-03T15:36:01.805 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:36:01.805 INFO:teuthology.orchestra.run.smithi028.stdout:For more information see: 2024-07-03T15:36:01.805 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:36:01.805 INFO:teuthology.orchestra.run.smithi028.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-07-03T15:36:01.805 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:36:01.805 INFO:teuthology.orchestra.run.smithi028.stdout:Bootstrap complete. 2024-07-03T15:36:01.838 INFO:tasks.cephadm:Fetching config... 2024-07-03T15:36:01.839 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-07-03T15:36:01.839 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-07-03T15:36:01.856 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-07-03T15:36:01.856 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-07-03T15:36:01.856 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-07-03T15:36:01.911 INFO:tasks.cephadm:Fetching mon keyring... 2024-07-03T15:36:01.912 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-07-03T15:36:01.912 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/keyring of=/dev/stdout 2024-07-03T15:36:01.978 INFO:tasks.cephadm:Fetching pub ssh key... 2024-07-03T15:36:01.978 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-07-03T15:36:01.978 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-07-03T15:36:02.035 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-07-03T15:36:02.035 DEBUG:teuthology.orchestra.run.smithi028:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC24c+XU+Lshmz+j874Ll8PTloFWBz8f0ABEbHIdykRz/iEUS4PW+lmBvDfVTjpORnMQtrvUURCCXuMG1isTWUX/0oZpHFq9Zw/W60bUSAWT4x0Jn/rGaHpToLFVpPnVkFgM2CtlTnYAKAlzQf4TKyRUX2fpxewU21sh12iizjpZ26ccGsUOvy4WbldyfuLo67Fghs04dA3bd4AwrcDlgWzzRxrnxuzpQTkTAa5Bi4jKVJrMgK7SZFD/AoAWgyMCYPHUpGHJ1nwTbNSuBoxPg+z4fH2qnRk+N6mWNykT1VK+NU1rGOqqmB8hHHhWazIHrTrhDu06ApWgrHfzVfv+YFF+cLD37L9Twx9T4wjFncJSiXTnPLnFYN3Wx7ejwsDqf+CBFqsknQIfiBlOYPDt2Vum7UzL+sCUjUIC629QxVzbGr7j8vzBw3Zl9oxZ3TJaWjinK5H+w5vufhIJ1/puk/bDN/MQdN/phfcrsRXmdpYLRqnaGywBQytVe39diTldAE= ceph-a85e4be6-3951-11ef-bcab-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-07-03T15:36:02.124 INFO:teuthology.orchestra.run.smithi028.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC24c+XU+Lshmz+j874Ll8PTloFWBz8f0ABEbHIdykRz/iEUS4PW+lmBvDfVTjpORnMQtrvUURCCXuMG1isTWUX/0oZpHFq9Zw/W60bUSAWT4x0Jn/rGaHpToLFVpPnVkFgM2CtlTnYAKAlzQf4TKyRUX2fpxewU21sh12iizjpZ26ccGsUOvy4WbldyfuLo67Fghs04dA3bd4AwrcDlgWzzRxrnxuzpQTkTAa5Bi4jKVJrMgK7SZFD/AoAWgyMCYPHUpGHJ1nwTbNSuBoxPg+z4fH2qnRk+N6mWNykT1VK+NU1rGOqqmB8hHHhWazIHrTrhDu06ApWgrHfzVfv+YFF+cLD37L9Twx9T4wjFncJSiXTnPLnFYN3Wx7ejwsDqf+CBFqsknQIfiBlOYPDt2Vum7UzL+sCUjUIC629QxVzbGr7j8vzBw3Zl9oxZ3TJaWjinK5H+w5vufhIJ1/puk/bDN/MQdN/phfcrsRXmdpYLRqnaGywBQytVe39diTldAE= ceph-a85e4be6-3951-11ef-bcab-c7b262605968 2024-07-03T15:36:02.137 DEBUG:teuthology.orchestra.run.smithi148:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC24c+XU+Lshmz+j874Ll8PTloFWBz8f0ABEbHIdykRz/iEUS4PW+lmBvDfVTjpORnMQtrvUURCCXuMG1isTWUX/0oZpHFq9Zw/W60bUSAWT4x0Jn/rGaHpToLFVpPnVkFgM2CtlTnYAKAlzQf4TKyRUX2fpxewU21sh12iizjpZ26ccGsUOvy4WbldyfuLo67Fghs04dA3bd4AwrcDlgWzzRxrnxuzpQTkTAa5Bi4jKVJrMgK7SZFD/AoAWgyMCYPHUpGHJ1nwTbNSuBoxPg+z4fH2qnRk+N6mWNykT1VK+NU1rGOqqmB8hHHhWazIHrTrhDu06ApWgrHfzVfv+YFF+cLD37L9Twx9T4wjFncJSiXTnPLnFYN3Wx7ejwsDqf+CBFqsknQIfiBlOYPDt2Vum7UzL+sCUjUIC629QxVzbGr7j8vzBw3Zl9oxZ3TJaWjinK5H+w5vufhIJ1/puk/bDN/MQdN/phfcrsRXmdpYLRqnaGywBQytVe39diTldAE= ceph-a85e4be6-3951-11ef-bcab-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-07-03T15:36:02.178 INFO:teuthology.orchestra.run.smithi148.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC24c+XU+Lshmz+j874Ll8PTloFWBz8f0ABEbHIdykRz/iEUS4PW+lmBvDfVTjpORnMQtrvUURCCXuMG1isTWUX/0oZpHFq9Zw/W60bUSAWT4x0Jn/rGaHpToLFVpPnVkFgM2CtlTnYAKAlzQf4TKyRUX2fpxewU21sh12iizjpZ26ccGsUOvy4WbldyfuLo67Fghs04dA3bd4AwrcDlgWzzRxrnxuzpQTkTAa5Bi4jKVJrMgK7SZFD/AoAWgyMCYPHUpGHJ1nwTbNSuBoxPg+z4fH2qnRk+N6mWNykT1VK+NU1rGOqqmB8hHHhWazIHrTrhDu06ApWgrHfzVfv+YFF+cLD37L9Twx9T4wjFncJSiXTnPLnFYN3Wx7ejwsDqf+CBFqsknQIfiBlOYPDt2Vum7UzL+sCUjUIC629QxVzbGr7j8vzBw3Zl9oxZ3TJaWjinK5H+w5vufhIJ1/puk/bDN/MQdN/phfcrsRXmdpYLRqnaGywBQytVe39diTldAE= ceph-a85e4be6-3951-11ef-bcab-c7b262605968 2024-07-03T15:36:02.191 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-07-03T15:36:02.370 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:02 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/3509582069' entity='client.admin' 2024-07-03T15:36:02.452 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:36:04.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:04.087+0000 7f105b24c640 1 -- 172.21.15.28:0/105462422 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f10540fb0f0 msgr2=0x7f10540fb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:04.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:04.087+0000 7f105b24c640 1 --2- 172.21.15.28:0/105462422 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f10540fb0f0 0x7f10540fb500 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1048004850 tx=0x7f1048009da0 comp rx=0 tx=0).stop 2024-07-03T15:36:04.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:04.087+0000 7f105b24c640 1 -- 172.21.15.28:0/105462422 shutdown_connections 2024-07-03T15:36:04.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:04.087+0000 7f105b24c640 1 --2- 172.21.15.28:0/105462422 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f10540fb0f0 0x7f10540fb500 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:04.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:04.087+0000 7f105b24c640 1 -- 172.21.15.28:0/105462422 >> 172.21.15.28:0/105462422 conn(0x7f10540f68d0 msgr2=0x7f10540f8ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:36:04.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:04.087+0000 7f105b24c640 1 -- 172.21.15.28:0/105462422 shutdown_connections 2024-07-03T15:36:04.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:04.087+0000 7f105b24c640 1 -- 172.21.15.28:0/105462422 wait complete. 2024-07-03T15:36:04.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:04.087+0000 7f105b24c640 1 Processor -- start 2024-07-03T15:36:04.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:04.087+0000 7f105b24c640 1 -- start start 2024-07-03T15:36:04.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:04.088+0000 7f105b24c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f10540fb0f0 0x7f1054112e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:36:04.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:04.088+0000 7f105b24c640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1054113390 con 0x7f10540fb0f0 2024-07-03T15:36:04.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:04.088+0000 7f1058fe8640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f10540fb0f0 0x7f1054112e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:36:04.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:04.088+0000 7f1058fe8640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f10540fb0f0 0x7f1054112e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45742/0 (socket says 172.21.15.28:45742) 2024-07-03T15:36:04.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:04.088+0000 7f1058fe8640 1 -- 172.21.15.28:0/3434198382 learned_addr learned my addr 172.21.15.28:0/3434198382 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:36:04.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:04.088+0000 7f1058fe8640 1 -- 172.21.15.28:0/3434198382 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1048005040 con 0x7f10540fb0f0 2024-07-03T15:36:04.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:04.088+0000 7f1058fe8640 1 --2- 172.21.15.28:0/3434198382 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f10540fb0f0 0x7f1054112e50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1048004820 tx=0x7f1048009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:36:04.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:04.089+0000 7f10517fa640 1 -- 172.21.15.28:0/3434198382 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1048032650 con 0x7f10540fb0f0 2024-07-03T15:36:04.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:04.089+0000 7f10517fa640 1 -- 172.21.15.28:0/3434198382 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1048032c80 con 0x7f10540fb0f0 2024-07-03T15:36:04.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:04.089+0000 7f105b24c640 1 -- 172.21.15.28:0/3434198382 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1054113590 con 0x7f10540fb0f0 2024-07-03T15:36:04.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:04.089+0000 7f10517fa640 1 -- 172.21.15.28:0/3434198382 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f104803eda0 con 0x7f10540fb0f0 2024-07-03T15:36:04.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:04.089+0000 7f105b24c640 1 -- 172.21.15.28:0/3434198382 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1054114070 con 0x7f10540fb0f0 2024-07-03T15:36:04.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:04.089+0000 7f10517fa640 1 -- 172.21.15.28:0/3434198382 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 13) v1 ==== 42692+0+0 (secure 0 0 0) 0x7f1048030020 con 0x7f10540fb0f0 2024-07-03T15:36:04.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:04.090+0000 7f10517fa640 1 --2- 172.21.15.28:0/3434198382 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f10440357b0 0x7f1044037c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:36:04.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:04.090+0000 7f10517fa640 1 -- 172.21.15.28:0/3434198382 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f1048068b50 con 0x7f10540fb0f0 2024-07-03T15:36:04.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:04.090+0000 7f105b24c640 1 -- 172.21.15.28:0/3434198382 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1054100550 con 0x7f10540fb0f0 2024-07-03T15:36:04.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:04.090+0000 7f1053fff640 1 --2- 172.21.15.28:0/3434198382 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f10440357b0 0x7f1044037c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:36:04.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:04.090+0000 7f1053fff640 1 --2- 172.21.15.28:0/3434198382 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f10440357b0 0x7f1044037c60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1040006fd0 tx=0x7f1040006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:36:04.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:04.094+0000 7f10517fa640 1 -- 172.21.15.28:0/3434198382 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f1048068e60 con 0x7f10540fb0f0 2024-07-03T15:36:04.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:04.274+0000 7f105b24c640 1 -- 172.21.15.28:0/3434198382 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f105405eb80 con 0x7f10540fb0f0 2024-07-03T15:36:04.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:04.277+0000 7f10517fa640 1 -- 172.21.15.28:0/3434198382 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f1048033420 con 0x7f10540fb0f0 2024-07-03T15:36:04.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:04.280+0000 7f105b24c640 1 -- 172.21.15.28:0/3434198382 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f10440357b0 msgr2=0x7f1044037c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:04.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:04.280+0000 7f105b24c640 1 --2- 172.21.15.28:0/3434198382 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f10440357b0 0x7f1044037c60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1040006fd0 tx=0x7f1040006db0 comp rx=0 tx=0).stop 2024-07-03T15:36:04.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:04.280+0000 7f105b24c640 1 -- 172.21.15.28:0/3434198382 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f10540fb0f0 msgr2=0x7f1054112e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:04.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:04.280+0000 7f105b24c640 1 --2- 172.21.15.28:0/3434198382 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f10540fb0f0 0x7f1054112e50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1048004820 tx=0x7f1048009da0 comp rx=0 tx=0).stop 2024-07-03T15:36:04.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:04.280+0000 7f105b24c640 1 -- 172.21.15.28:0/3434198382 shutdown_connections 2024-07-03T15:36:04.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:04.280+0000 7f105b24c640 1 --2- 172.21.15.28:0/3434198382 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f10440357b0 0x7f1044037c60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:04.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:04.280+0000 7f105b24c640 1 --2- 172.21.15.28:0/3434198382 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f10540fb0f0 0x7f1054112e50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:04.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:04.280+0000 7f105b24c640 1 -- 172.21.15.28:0/3434198382 >> 172.21.15.28:0/3434198382 conn(0x7f10540f68d0 msgr2=0x7f1054101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:36:04.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:04.280+0000 7f105b24c640 1 -- 172.21.15.28:0/3434198382 shutdown_connections 2024-07-03T15:36:04.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:04.280+0000 7f105b24c640 1 -- 172.21.15.28:0/3434198382 wait complete. 2024-07-03T15:36:05.581 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:05 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/3434198382' entity='client.admin' 2024-07-03T15:36:05.581 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:05 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:36:05.581 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:05 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:36:05.581 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:05 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "who": "osd/host:smithi028", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:36:05.581 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:05 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:36:05.581 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:05 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi028", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-07-03T15:36:05.581 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:05 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi028", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-07-03T15:36:05.581 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:05 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:36:05.581 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:05 smithi028 ceph-mon[29148]: Deploying daemon crash.smithi028 on smithi028 2024-07-03T15:36:05.750 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-07-03T15:36:05.750 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-07-03T15:36:06.021 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:36:08.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:08.079+0000 7f2ef994c640 1 -- 172.21.15.28:0/2581090492 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ef4065f10 msgr2=0x7f2ef40fec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:08.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:08.079+0000 7f2ef994c640 1 --2- 172.21.15.28:0/2581090492 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ef4065f10 0x7f2ef40fec90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4004850 tx=0x7f2ee4009da0 comp rx=0 tx=0).stop 2024-07-03T15:36:08.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:08.079+0000 7f2ef994c640 1 -- 172.21.15.28:0/2581090492 shutdown_connections 2024-07-03T15:36:08.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:08.079+0000 7f2ef994c640 1 --2- 172.21.15.28:0/2581090492 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ef4065f10 0x7f2ef40fec90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:08.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:08.079+0000 7f2ef994c640 1 -- 172.21.15.28:0/2581090492 >> 172.21.15.28:0/2581090492 conn(0x7f2ef40f66a0 msgr2=0x7f2ef40f8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:36:08.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:08.079+0000 7f2ef994c640 1 -- 172.21.15.28:0/2581090492 shutdown_connections 2024-07-03T15:36:08.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:08.079+0000 7f2ef994c640 1 -- 172.21.15.28:0/2581090492 wait complete. 2024-07-03T15:36:08.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:08.080+0000 7f2ef994c640 1 Processor -- start 2024-07-03T15:36:08.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:08.080+0000 7f2ef994c640 1 -- start start 2024-07-03T15:36:08.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:08.080+0000 7f2ef994c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ef4065f10 0x7f2ef410c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:36:08.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:08.080+0000 7f2ef994c640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef410cb40 con 0x7f2ef4065f10 2024-07-03T15:36:08.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:08.080+0000 7f2ef2ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ef4065f10 0x7f2ef410c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:36:08.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:08.080+0000 7f2ef2ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ef4065f10 0x7f2ef410c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45762/0 (socket says 172.21.15.28:45762) 2024-07-03T15:36:08.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:08.080+0000 7f2ef2ffd640 1 -- 172.21.15.28:0/2586184494 learned_addr learned my addr 172.21.15.28:0/2586184494 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:36:08.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:08.081+0000 7f2ef2ffd640 1 -- 172.21.15.28:0/2586184494 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee4005040 con 0x7f2ef4065f10 2024-07-03T15:36:08.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:08.081+0000 7f2ef2ffd640 1 --2- 172.21.15.28:0/2586184494 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ef4065f10 0x7f2ef410c600 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4004820 tx=0x7f2ee4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:36:08.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:08.081+0000 7f2eebfff640 1 -- 172.21.15.28:0/2586184494 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2ee4032650 con 0x7f2ef4065f10 2024-07-03T15:36:08.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:08.081+0000 7f2eebfff640 1 -- 172.21.15.28:0/2586184494 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2ee4032c80 con 0x7f2ef4065f10 2024-07-03T15:36:08.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:08.081+0000 7f2eebfff640 1 -- 172.21.15.28:0/2586184494 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2ee4033d60 con 0x7f2ef4065f10 2024-07-03T15:36:08.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:08.081+0000 7f2ef994c640 1 -- 172.21.15.28:0/2586184494 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ef410cd40 con 0x7f2ef4065f10 2024-07-03T15:36:08.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:08.081+0000 7f2ef994c640 1 -- 172.21.15.28:0/2586184494 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ef410d8d0 con 0x7f2ef4065f10 2024-07-03T15:36:08.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:08.082+0000 7f2eebfff640 1 -- 172.21.15.28:0/2586184494 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 13) v1 ==== 42692+0+0 (secure 0 0 0) 0x7f2ee4030020 con 0x7f2ef4065f10 2024-07-03T15:36:08.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:08.082+0000 7f2eebfff640 1 --2- 172.21.15.28:0/2586184494 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f2ee00357b0 0x7f2ee0037c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:36:08.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:08.082+0000 7f2eebfff640 1 -- 172.21.15.28:0/2586184494 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f2ee40678e0 con 0x7f2ef4065f10 2024-07-03T15:36:08.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:08.082+0000 7f2ef994c640 1 -- 172.21.15.28:0/2586184494 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ef4102550 con 0x7f2ef4065f10 2024-07-03T15:36:08.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:08.082+0000 7f2eea7fc640 1 --2- 172.21.15.28:0/2586184494 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f2ee00357b0 0x7f2ee0037c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:36:08.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:08.083+0000 7f2eea7fc640 1 --2- 172.21.15.28:0/2586184494 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f2ee00357b0 0x7f2ee0037c60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2edc006fd0 tx=0x7f2edc006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:36:08.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:08.087+0000 7f2eebfff640 1 -- 172.21.15.28:0/2586184494 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2ee403c380 con 0x7f2ef4065f10 2024-07-03T15:36:08.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:08.282+0000 7f2ef994c640 1 -- 172.21.15.28:0/2586184494 --> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ef4101f30 con 0x7f2ee00357b0 2024-07-03T15:36:08.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:08.285+0000 7f2eebfff640 1 -- 172.21.15.28:0/2586184494 <== mgr.14160 v2:172.21.15.28:6800/577887491 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f2edc009510 con 0x7f2ee00357b0 2024-07-03T15:36:08.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:08.287+0000 7f2ef994c640 1 -- 172.21.15.28:0/2586184494 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f2ee00357b0 msgr2=0x7f2ee0037c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:08.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:08.287+0000 7f2ef994c640 1 --2- 172.21.15.28:0/2586184494 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f2ee00357b0 0x7f2ee0037c60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2edc006fd0 tx=0x7f2edc006db0 comp rx=0 tx=0).stop 2024-07-03T15:36:08.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:08.287+0000 7f2ef994c640 1 -- 172.21.15.28:0/2586184494 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ef4065f10 msgr2=0x7f2ef410c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:08.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:08.287+0000 7f2ef994c640 1 --2- 172.21.15.28:0/2586184494 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ef4065f10 0x7f2ef410c600 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4004820 tx=0x7f2ee4009da0 comp rx=0 tx=0).stop 2024-07-03T15:36:08.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:08.287+0000 7f2ef994c640 1 -- 172.21.15.28:0/2586184494 shutdown_connections 2024-07-03T15:36:08.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:08.287+0000 7f2ef994c640 1 --2- 172.21.15.28:0/2586184494 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f2ee00357b0 0x7f2ee0037c60 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:08.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:08.287+0000 7f2ef994c640 1 --2- 172.21.15.28:0/2586184494 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ef4065f10 0x7f2ef410c600 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:08.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:08.287+0000 7f2ef994c640 1 -- 172.21.15.28:0/2586184494 >> 172.21.15.28:0/2586184494 conn(0x7f2ef40f66a0 msgr2=0x7f2ef4103180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:36:08.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:08.288+0000 7f2ef994c640 1 -- 172.21.15.28:0/2586184494 shutdown_connections 2024-07-03T15:36:08.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:08.288+0000 7f2ef994c640 1 -- 172.21.15.28:0/2586184494 wait complete. 2024-07-03T15:36:09.314 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi148 2024-07-03T15:36:09.315 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-07-03T15:36:09.315 DEBUG:teuthology.orchestra.run.smithi148:> dd of=/etc/ceph/ceph.conf 2024-07-03T15:36:09.333 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-07-03T15:36:09.334 DEBUG:teuthology.orchestra.run.smithi148:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-07-03T15:36:09.391 INFO:tasks.cephadm:Adding host smithi148 to orchestrator... 2024-07-03T15:36:09.392 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph orch host add smithi148 2024-07-03T15:36:09.620 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:09 smithi028 ceph-mon[29148]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:36:09.620 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:09 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:36:09.756 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:36:11.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:11.385+0000 7f17f22ad640 1 -- 172.21.15.28:0/2113186124 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f17ec0fbfe0 msgr2=0x7f17ec0fc3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:11.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:11.385+0000 7f17f22ad640 1 --2- 172.21.15.28:0/2113186124 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f17ec0fbfe0 0x7f17ec0fc3f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f17dc004850 tx=0x7f17dc009da0 comp rx=0 tx=0).stop 2024-07-03T15:36:11.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:11.385+0000 7f17f22ad640 1 -- 172.21.15.28:0/2113186124 shutdown_connections 2024-07-03T15:36:11.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:11.385+0000 7f17f22ad640 1 --2- 172.21.15.28:0/2113186124 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f17ec0fbfe0 0x7f17ec0fc3f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:11.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:11.385+0000 7f17f22ad640 1 -- 172.21.15.28:0/2113186124 >> 172.21.15.28:0/2113186124 conn(0x7f17ec0f77c0 msgr2=0x7f17ec0f9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:36:11.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:11.385+0000 7f17f22ad640 1 -- 172.21.15.28:0/2113186124 shutdown_connections 2024-07-03T15:36:11.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:11.385+0000 7f17f22ad640 1 -- 172.21.15.28:0/2113186124 wait complete. 2024-07-03T15:36:11.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:11.385+0000 7f17f22ad640 1 Processor -- start 2024-07-03T15:36:11.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:11.386+0000 7f17f22ad640 1 -- start start 2024-07-03T15:36:11.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:11.386+0000 7f17f22ad640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f17ec0fbfe0 0x7f17ec10c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:36:11.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:11.386+0000 7f17f22ad640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17ec10cae0 con 0x7f17ec0fbfe0 2024-07-03T15:36:11.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:11.386+0000 7f17ebfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f17ec0fbfe0 0x7f17ec10c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:36:11.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:11.386+0000 7f17ebfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f17ec0fbfe0 0x7f17ec10c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59490/0 (socket says 172.21.15.28:59490) 2024-07-03T15:36:11.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:11.386+0000 7f17ebfff640 1 -- 172.21.15.28:0/1608494473 learned_addr learned my addr 172.21.15.28:0/1608494473 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:36:11.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:11.386+0000 7f17ebfff640 1 -- 172.21.15.28:0/1608494473 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17dc005040 con 0x7f17ec0fbfe0 2024-07-03T15:36:11.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:11.386+0000 7f17ebfff640 1 --2- 172.21.15.28:0/1608494473 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f17ec0fbfe0 0x7f17ec10c5a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f17dc004820 tx=0x7f17dc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:36:11.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:11.387+0000 7f17e8ff9640 1 -- 172.21.15.28:0/1608494473 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f17dc032650 con 0x7f17ec0fbfe0 2024-07-03T15:36:11.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:11.387+0000 7f17f22ad640 1 -- 172.21.15.28:0/1608494473 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17ec10cce0 con 0x7f17ec0fbfe0 2024-07-03T15:36:11.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:11.387+0000 7f17e8ff9640 1 -- 172.21.15.28:0/1608494473 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f17dc032c80 con 0x7f17ec0fbfe0 2024-07-03T15:36:11.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:11.387+0000 7f17e8ff9640 1 -- 172.21.15.28:0/1608494473 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f17dc03eda0 con 0x7f17ec0fbfe0 2024-07-03T15:36:11.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:11.387+0000 7f17f22ad640 1 -- 172.21.15.28:0/1608494473 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17ec10d870 con 0x7f17ec0fbfe0 2024-07-03T15:36:11.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:11.388+0000 7f17e8ff9640 1 -- 172.21.15.28:0/1608494473 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 13) v1 ==== 42692+0+0 (secure 0 0 0) 0x7f17dc030020 con 0x7f17ec0fbfe0 2024-07-03T15:36:11.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:11.388+0000 7f17f22ad640 1 -- 172.21.15.28:0/1608494473 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17ec10d1b0 con 0x7f17ec0fbfe0 2024-07-03T15:36:11.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:11.388+0000 7f17e8ff9640 1 --2- 172.21.15.28:0/1608494473 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f17d40357b0 0x7f17d4037c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:36:11.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:11.388+0000 7f17e8ff9640 1 -- 172.21.15.28:0/1608494473 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f17dc067c90 con 0x7f17ec0fbfe0 2024-07-03T15:36:11.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:11.388+0000 7f17eb7fe640 1 --2- 172.21.15.28:0/1608494473 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f17d40357b0 0x7f17d4037c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:36:11.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:11.388+0000 7f17eb7fe640 1 --2- 172.21.15.28:0/1608494473 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f17d40357b0 0x7f17d4037c60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f17e0006fd0 tx=0x7f17e0006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:36:11.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:11.392+0000 7f17e8ff9640 1 -- 172.21.15.28:0/1608494473 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f17ec10d1b0 con 0x7f17ec0fbfe0 2024-07-03T15:36:11.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:11.575+0000 7f17f22ad640 1 -- 172.21.15.28:0/1608494473 --> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi148", "target": ["mon-mgr", ""]}) v1 -- 0x7f17ec10d1b0 con 0x7f17d40357b0 2024-07-03T15:36:13.370 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:12 smithi028 ceph-mon[29148]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi148", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:36:13.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:12 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:36:13.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:12 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:36:13.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:12 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:36:13.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:12 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:36:13.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:12 smithi028 ceph-mon[29148]: Deploying daemon node-exporter.smithi028 on smithi028 2024-07-03T15:36:13.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:12 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:36:14.370 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:13 smithi028 ceph-mon[29148]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:36:14.370 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:13 smithi028 ceph-mon[29148]: Deploying cephadm binary to smithi148 2024-07-03T15:36:15.040 INFO:teuthology.orchestra.run.smithi028.stdout:Added host 'smithi148' with addr '172.21.15.148' 2024-07-03T15:36:15.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:15.037+0000 7f17e8ff9640 1 -- 172.21.15.28:0/1608494473 <== mgr.14160 v2:172.21.15.28:6800/577887491 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f17ec10d1b0 con 0x7f17d40357b0 2024-07-03T15:36:15.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:15.039+0000 7f17f22ad640 1 -- 172.21.15.28:0/1608494473 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f17d40357b0 msgr2=0x7f17d4037c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:15.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:15.039+0000 7f17f22ad640 1 --2- 172.21.15.28:0/1608494473 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f17d40357b0 0x7f17d4037c60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f17e0006fd0 tx=0x7f17e0006db0 comp rx=0 tx=0).stop 2024-07-03T15:36:15.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:15.039+0000 7f17f22ad640 1 -- 172.21.15.28:0/1608494473 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f17ec0fbfe0 msgr2=0x7f17ec10c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:15.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:15.039+0000 7f17f22ad640 1 --2- 172.21.15.28:0/1608494473 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f17ec0fbfe0 0x7f17ec10c5a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f17dc004820 tx=0x7f17dc009da0 comp rx=0 tx=0).stop 2024-07-03T15:36:15.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:15.039+0000 7f17f22ad640 1 -- 172.21.15.28:0/1608494473 shutdown_connections 2024-07-03T15:36:15.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:15.039+0000 7f17f22ad640 1 --2- 172.21.15.28:0/1608494473 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f17d40357b0 0x7f17d4037c60 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:15.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:15.039+0000 7f17f22ad640 1 --2- 172.21.15.28:0/1608494473 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f17ec0fbfe0 0x7f17ec10c5a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:15.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:15.039+0000 7f17f22ad640 1 -- 172.21.15.28:0/1608494473 >> 172.21.15.28:0/1608494473 conn(0x7f17ec0f77c0 msgr2=0x7f17ec103180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:36:15.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:15.039+0000 7f17f22ad640 1 -- 172.21.15.28:0/1608494473 shutdown_connections 2024-07-03T15:36:15.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:15.039+0000 7f17f22ad640 1 -- 172.21.15.28:0/1608494473 wait complete. 2024-07-03T15:36:15.835 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph orch host ls --format=json 2024-07-03T15:36:16.106 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:16 smithi028 ceph-mon[29148]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:36:16.106 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:16 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:36:16.106 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:16 smithi028 ceph-mon[29148]: Added host smithi148 2024-07-03T15:36:16.288 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:36:17.555 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:17 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:36:17.555 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:17 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:36:17.555 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:17 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:36:17.555 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:17 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:36:17.555 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:17 smithi028 ceph-mon[29148]: Deploying daemon alertmanager.smithi028 on smithi028 2024-07-03T15:36:17.556 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:17 smithi028 ceph-mon[29148]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:36:17.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:17.750+0000 7fc909fa0640 1 -- 172.21.15.28:0/1843291216 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9040fbfc0 msgr2=0x7fc9040fc3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:17.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:17.750+0000 7fc909fa0640 1 --2- 172.21.15.28:0/1843291216 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9040fbfc0 0x7fc9040fc3d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4004850 tx=0x7fc8f4009da0 comp rx=0 tx=0).stop 2024-07-03T15:36:17.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:17.750+0000 7fc909fa0640 1 -- 172.21.15.28:0/1843291216 shutdown_connections 2024-07-03T15:36:17.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:17.750+0000 7fc909fa0640 1 --2- 172.21.15.28:0/1843291216 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9040fbfc0 0x7fc9040fc3d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:17.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:17.750+0000 7fc909fa0640 1 -- 172.21.15.28:0/1843291216 >> 172.21.15.28:0/1843291216 conn(0x7fc9040f77a0 msgr2=0x7fc9040f9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:36:17.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:17.750+0000 7fc909fa0640 1 -- 172.21.15.28:0/1843291216 shutdown_connections 2024-07-03T15:36:17.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:17.750+0000 7fc909fa0640 1 -- 172.21.15.28:0/1843291216 wait complete. 2024-07-03T15:36:17.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:17.750+0000 7fc909fa0640 1 Processor -- start 2024-07-03T15:36:17.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:17.751+0000 7fc909fa0640 1 -- start start 2024-07-03T15:36:17.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:17.751+0000 7fc909fa0640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9040fbfc0 0x7fc90410c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:36:17.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:17.751+0000 7fc909fa0640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90410cb50 con 0x7fc9040fbfc0 2024-07-03T15:36:17.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:17.751+0000 7fc9037fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9040fbfc0 0x7fc90410c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:36:17.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:17.751+0000 7fc9037fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9040fbfc0 0x7fc90410c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59516/0 (socket says 172.21.15.28:59516) 2024-07-03T15:36:17.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:17.751+0000 7fc9037fe640 1 -- 172.21.15.28:0/1403904435 learned_addr learned my addr 172.21.15.28:0/1403904435 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:36:17.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:17.751+0000 7fc9037fe640 1 -- 172.21.15.28:0/1403904435 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f4005040 con 0x7fc9040fbfc0 2024-07-03T15:36:17.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:17.751+0000 7fc9037fe640 1 --2- 172.21.15.28:0/1403904435 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9040fbfc0 0x7fc90410c610 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4004820 tx=0x7fc8f4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:36:17.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:17.752+0000 7fc8fbfff640 1 -- 172.21.15.28:0/1403904435 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc8f4032650 con 0x7fc9040fbfc0 2024-07-03T15:36:17.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:17.752+0000 7fc909fa0640 1 -- 172.21.15.28:0/1403904435 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc90410cd50 con 0x7fc9040fbfc0 2024-07-03T15:36:17.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:17.752+0000 7fc8fbfff640 1 -- 172.21.15.28:0/1403904435 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc8f4032c80 con 0x7fc9040fbfc0 2024-07-03T15:36:17.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:17.752+0000 7fc909fa0640 1 -- 172.21.15.28:0/1403904435 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc90410d8e0 con 0x7fc9040fbfc0 2024-07-03T15:36:17.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:17.752+0000 7fc8fbfff640 1 -- 172.21.15.28:0/1403904435 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc8f403eda0 con 0x7fc9040fbfc0 2024-07-03T15:36:17.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:17.752+0000 7fc8fbfff640 1 -- 172.21.15.28:0/1403904435 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 13) v1 ==== 42692+0+0 (secure 0 0 0) 0x7fc8f4030020 con 0x7fc9040fbfc0 2024-07-03T15:36:17.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:17.753+0000 7fc8fbfff640 1 --2- 172.21.15.28:0/1403904435 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7fc8f00357b0 0x7fc8f0037c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:36:17.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:17.753+0000 7fc8fbfff640 1 -- 172.21.15.28:0/1403904435 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fc8f4067a70 con 0x7fc9040fbfc0 2024-07-03T15:36:17.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:17.753+0000 7fc902ffd640 1 --2- 172.21.15.28:0/1403904435 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7fc8f00357b0 0x7fc8f0037c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:36:17.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:17.753+0000 7fc902ffd640 1 --2- 172.21.15.28:0/1403904435 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7fc8f00357b0 0x7fc8f0037c60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec006fd0 tx=0x7fc8ec006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:36:17.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:17.753+0000 7fc909fa0640 1 -- 172.21.15.28:0/1403904435 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc90410d220 con 0x7fc9040fbfc0 2024-07-03T15:36:17.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:17.762+0000 7fc8fbfff640 1 -- 172.21.15.28:0/1403904435 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc90410d220 con 0x7fc9040fbfc0 2024-07-03T15:36:17.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:17.951+0000 7fc909fa0640 1 -- 172.21.15.28:0/1403904435 --> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc90410d220 con 0x7fc8f00357b0 2024-07-03T15:36:17.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:17.952+0000 7fc8fbfff640 1 -- 172.21.15.28:0/1403904435 <== mgr.14160 v2:172.21.15.28:6800/577887491 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7fc90410d220 con 0x7fc8f00357b0 2024-07-03T15:36:17.954 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:36:17.955 INFO:teuthology.orchestra.run.smithi028.stdout:[{"addr": "172.21.15.28", "hostname": "smithi028", "labels": [], "status": ""}, {"addr": "172.21.15.148", "hostname": "smithi148", "labels": [], "status": ""}] 2024-07-03T15:36:17.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:17.954+0000 7fc909fa0640 1 -- 172.21.15.28:0/1403904435 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7fc8f00357b0 msgr2=0x7fc8f0037c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:17.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:17.954+0000 7fc909fa0640 1 --2- 172.21.15.28:0/1403904435 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7fc8f00357b0 0x7fc8f0037c60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec006fd0 tx=0x7fc8ec006db0 comp rx=0 tx=0).stop 2024-07-03T15:36:17.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:17.954+0000 7fc909fa0640 1 -- 172.21.15.28:0/1403904435 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9040fbfc0 msgr2=0x7fc90410c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:17.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:17.954+0000 7fc909fa0640 1 --2- 172.21.15.28:0/1403904435 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9040fbfc0 0x7fc90410c610 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4004820 tx=0x7fc8f4009da0 comp rx=0 tx=0).stop 2024-07-03T15:36:17.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:17.954+0000 7fc909fa0640 1 -- 172.21.15.28:0/1403904435 shutdown_connections 2024-07-03T15:36:17.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:17.954+0000 7fc909fa0640 1 --2- 172.21.15.28:0/1403904435 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7fc8f00357b0 0x7fc8f0037c60 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:17.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:17.954+0000 7fc909fa0640 1 --2- 172.21.15.28:0/1403904435 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9040fbfc0 0x7fc90410c610 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:17.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:17.954+0000 7fc909fa0640 1 -- 172.21.15.28:0/1403904435 >> 172.21.15.28:0/1403904435 conn(0x7fc9040f77a0 msgr2=0x7fc904103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:36:17.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:17.955+0000 7fc909fa0640 1 -- 172.21.15.28:0/1403904435 shutdown_connections 2024-07-03T15:36:17.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:17.955+0000 7fc909fa0640 1 -- 172.21.15.28:0/1403904435 wait complete. 2024-07-03T15:36:18.818 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:18 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:36:18.818 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:18 smithi028 ceph-mon[29148]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-03T15:36:19.029 INFO:tasks.cephadm:Setting crush tunables to default 2024-07-03T15:36:19.029 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph osd crush tunables default 2024-07-03T15:36:19.300 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:36:20.120 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:19 smithi028 ceph-mon[29148]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:36:21.870 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:21 smithi028 ceph-mon[29148]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:36:22.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:22.233+0000 7f4b4e20c640 1 -- 172.21.15.28:0/3722603586 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4b480fd630 msgr2=0x7f4b480fda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:22.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:22.233+0000 7f4b4e20c640 1 --2- 172.21.15.28:0/3722603586 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4b480fd630 0x7f4b480fda40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4b38004850 tx=0x7f4b38009da0 comp rx=0 tx=0).stop 2024-07-03T15:36:22.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:22.233+0000 7f4b4e20c640 1 -- 172.21.15.28:0/3722603586 shutdown_connections 2024-07-03T15:36:22.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:22.233+0000 7f4b4e20c640 1 --2- 172.21.15.28:0/3722603586 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4b480fd630 0x7f4b480fda40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:22.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:22.233+0000 7f4b4e20c640 1 -- 172.21.15.28:0/3722603586 >> 172.21.15.28:0/3722603586 conn(0x7f4b480f5e50 msgr2=0x7f4b480f8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:36:22.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:22.233+0000 7f4b4e20c640 1 -- 172.21.15.28:0/3722603586 shutdown_connections 2024-07-03T15:36:22.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:22.233+0000 7f4b4e20c640 1 -- 172.21.15.28:0/3722603586 wait complete. 2024-07-03T15:36:22.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:22.234+0000 7f4b4e20c640 1 Processor -- start 2024-07-03T15:36:22.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:22.234+0000 7f4b4e20c640 1 -- start start 2024-07-03T15:36:22.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:22.234+0000 7f4b4e20c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4b480fd630 0x7f4b4810e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:36:22.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:22.234+0000 7f4b4e20c640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4810ed00 con 0x7f4b480fd630 2024-07-03T15:36:22.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:22.234+0000 7f4b477fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4b480fd630 0x7f4b4810e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:36:22.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:22.234+0000 7f4b477fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4b480fd630 0x7f4b4810e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46502/0 (socket says 172.21.15.28:46502) 2024-07-03T15:36:22.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:22.234+0000 7f4b477fe640 1 -- 172.21.15.28:0/1486611758 learned_addr learned my addr 172.21.15.28:0/1486611758 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:36:22.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:22.235+0000 7f4b477fe640 1 -- 172.21.15.28:0/1486611758 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b38005040 con 0x7f4b480fd630 2024-07-03T15:36:22.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:22.235+0000 7f4b477fe640 1 --2- 172.21.15.28:0/1486611758 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4b480fd630 0x7f4b4810e7c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4b38004820 tx=0x7f4b38009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:36:22.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:22.235+0000 7f4b2ffff640 1 -- 172.21.15.28:0/1486611758 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4b38032650 con 0x7f4b480fd630 2024-07-03T15:36:22.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:22.235+0000 7f4b4e20c640 1 -- 172.21.15.28:0/1486611758 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b4810ef00 con 0x7f4b480fd630 2024-07-03T15:36:22.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:22.235+0000 7f4b2ffff640 1 -- 172.21.15.28:0/1486611758 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b38032c80 con 0x7f4b480fd630 2024-07-03T15:36:22.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:22.235+0000 7f4b2ffff640 1 -- 172.21.15.28:0/1486611758 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4b3803eda0 con 0x7f4b480fd630 2024-07-03T15:36:22.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:22.235+0000 7f4b4e20c640 1 -- 172.21.15.28:0/1486611758 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b4810fa90 con 0x7f4b480fd630 2024-07-03T15:36:22.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:22.236+0000 7f4b2ffff640 1 -- 172.21.15.28:0/1486611758 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 13) v1 ==== 42692+0+0 (secure 0 0 0) 0x7f4b38030020 con 0x7f4b480fd630 2024-07-03T15:36:22.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:22.236+0000 7f4b2ffff640 1 --2- 172.21.15.28:0/1486611758 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f4b300357b0 0x7f4b30037c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:36:22.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:22.236+0000 7f4b2ffff640 1 -- 172.21.15.28:0/1486611758 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f4b38067d90 con 0x7f4b480fd630 2024-07-03T15:36:22.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:22.236+0000 7f4b4e20c640 1 -- 172.21.15.28:0/1486611758 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b4810f3d0 con 0x7f4b480fd630 2024-07-03T15:36:22.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:22.236+0000 7f4b46ffd640 1 --2- 172.21.15.28:0/1486611758 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f4b300357b0 0x7f4b30037c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:36:22.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:22.237+0000 7f4b46ffd640 1 --2- 172.21.15.28:0/1486611758 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f4b300357b0 0x7f4b30037c60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c006fd0 tx=0x7f4b3c006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:36:22.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:22.242+0000 7f4b2ffff640 1 -- 172.21.15.28:0/1486611758 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4b4810f3d0 con 0x7f4b480fd630 2024-07-03T15:36:22.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:22.425+0000 7f4b4e20c640 1 -- 172.21.15.28:0/1486611758 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f4b4810f3d0 con 0x7f4b480fd630 2024-07-03T15:36:22.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:22.786+0000 7f4b2ffff640 1 -- 172.21.15.28:0/1486611758 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f4b4810f3d0 con 0x7f4b480fd630 2024-07-03T15:36:22.789 INFO:teuthology.orchestra.run.smithi028.stderr:adjusted tunables profile to default 2024-07-03T15:36:22.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:22.788+0000 7f4b4e20c640 1 -- 172.21.15.28:0/1486611758 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f4b300357b0 msgr2=0x7f4b30037c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:22.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:22.788+0000 7f4b4e20c640 1 --2- 172.21.15.28:0/1486611758 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f4b300357b0 0x7f4b30037c60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c006fd0 tx=0x7f4b3c006db0 comp rx=0 tx=0).stop 2024-07-03T15:36:22.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:22.788+0000 7f4b4e20c640 1 -- 172.21.15.28:0/1486611758 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4b480fd630 msgr2=0x7f4b4810e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:22.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:22.788+0000 7f4b4e20c640 1 --2- 172.21.15.28:0/1486611758 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4b480fd630 0x7f4b4810e7c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4b38004820 tx=0x7f4b38009da0 comp rx=0 tx=0).stop 2024-07-03T15:36:22.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:22.788+0000 7f4b4e20c640 1 -- 172.21.15.28:0/1486611758 shutdown_connections 2024-07-03T15:36:22.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:22.788+0000 7f4b4e20c640 1 --2- 172.21.15.28:0/1486611758 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f4b300357b0 0x7f4b30037c60 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:22.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:22.788+0000 7f4b4e20c640 1 --2- 172.21.15.28:0/1486611758 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4b480fd630 0x7f4b4810e7c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:22.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:22.788+0000 7f4b4e20c640 1 -- 172.21.15.28:0/1486611758 >> 172.21.15.28:0/1486611758 conn(0x7f4b480f5e50 msgr2=0x7f4b480f8230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:36:22.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:22.789+0000 7f4b4e20c640 1 -- 172.21.15.28:0/1486611758 shutdown_connections 2024-07-03T15:36:22.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:36:22.789+0000 7f4b4e20c640 1 -- 172.21.15.28:0/1486611758 wait complete. 2024-07-03T15:36:22.870 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:22 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/1486611758' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-07-03T15:36:23.554 INFO:tasks.cephadm:Adding mon.smithi028 on smithi028 2024-07-03T15:36:23.554 INFO:tasks.cephadm:Adding mon.smithi148 on smithi148 2024-07-03T15:36:23.554 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph orch apply mon '2;smithi028:172.21.15.28=smithi028;smithi148:172.21.15.148=smithi148' 2024-07-03T15:36:23.804 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-03T15:36:23.851 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-03T15:36:23.870 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:23 smithi028 ceph-mon[29148]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:36:23.870 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:23 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/1486611758' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-07-03T15:36:23.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:23 smithi028 ceph-mon[29148]: osdmap e4: 0 total, 0 up, 0 in 2024-07-03T15:36:25.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:25.318+0000 7f79d798b640 1 -- 172.21.15.148:0/3872253467 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f79d00fd630 msgr2=0x7f79d00fda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:25.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:25.318+0000 7f79d798b640 1 --2- 172.21.15.148:0/3872253467 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f79d00fd630 0x7f79d00fda40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f79cc004850 tx=0x7f79cc009da0 comp rx=0 tx=0).stop 2024-07-03T15:36:25.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:25.319+0000 7f79d798b640 1 -- 172.21.15.148:0/3872253467 shutdown_connections 2024-07-03T15:36:25.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:25.319+0000 7f79d798b640 1 --2- 172.21.15.148:0/3872253467 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f79d00fd630 0x7f79d00fda40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:25.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:25.319+0000 7f79d798b640 1 -- 172.21.15.148:0/3872253467 >> 172.21.15.148:0/3872253467 conn(0x7f79d00f5e50 msgr2=0x7f79d00f8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:36:25.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:25.319+0000 7f79d798b640 1 -- 172.21.15.148:0/3872253467 shutdown_connections 2024-07-03T15:36:25.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:25.319+0000 7f79d798b640 1 -- 172.21.15.148:0/3872253467 wait complete. 2024-07-03T15:36:25.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:25.319+0000 7f79d798b640 1 Processor -- start 2024-07-03T15:36:25.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:25.319+0000 7f79d798b640 1 -- start start 2024-07-03T15:36:25.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:25.319+0000 7f79d798b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f79d00fd630 0x7f79d010c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:36:25.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:25.319+0000 7f79d798b640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d010caf0 con 0x7f79d00fd630 2024-07-03T15:36:25.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:25.319+0000 7f79d5727640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f79d00fd630 0x7f79d010c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:36:25.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:25.319+0000 7f79d5727640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f79d00fd630 0x7f79d010c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.148:35242/0 (socket says 172.21.15.148:35242) 2024-07-03T15:36:25.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:25.319+0000 7f79d5727640 1 -- 172.21.15.148:0/567219613 learned_addr learned my addr 172.21.15.148:0/567219613 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-07-03T15:36:25.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:25.320+0000 7f79d5727640 1 -- 172.21.15.148:0/567219613 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79cc005040 con 0x7f79d00fd630 2024-07-03T15:36:25.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:25.320+0000 7f79d5727640 1 --2- 172.21.15.148:0/567219613 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f79d00fd630 0x7f79d010c5b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f79cc0098d0 tx=0x7f79cc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:36:25.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:25.320+0000 7f79c5ffb640 1 -- 172.21.15.148:0/567219613 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f79cc032650 con 0x7f79d00fd630 2024-07-03T15:36:25.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:25.320+0000 7f79c5ffb640 1 -- 172.21.15.148:0/567219613 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f79cc032c80 con 0x7f79d00fd630 2024-07-03T15:36:25.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:25.320+0000 7f79d798b640 1 -- 172.21.15.148:0/567219613 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79d010ccf0 con 0x7f79d00fd630 2024-07-03T15:36:25.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:25.320+0000 7f79c5ffb640 1 -- 172.21.15.148:0/567219613 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f79cc033c10 con 0x7f79d00fd630 2024-07-03T15:36:25.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:25.320+0000 7f79d798b640 1 -- 172.21.15.148:0/567219613 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79d010d160 con 0x7f79d00fd630 2024-07-03T15:36:25.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:25.321+0000 7f79c5ffb640 1 -- 172.21.15.148:0/567219613 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 13) v1 ==== 42692+0+0 (secure 0 0 0) 0x7f79cc030020 con 0x7f79d00fd630 2024-07-03T15:36:25.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:25.321+0000 7f79d798b640 1 -- 172.21.15.148:0/567219613 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79d005eb80 con 0x7f79d00fd630 2024-07-03T15:36:25.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:25.321+0000 7f79c5ffb640 1 --2- 172.21.15.148:0/567219613 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f79bc0357b0 0x7f79bc037c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:36:25.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:25.321+0000 7f79c5ffb640 1 -- 172.21.15.148:0/567219613 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f79cc067960 con 0x7f79d00fd630 2024-07-03T15:36:25.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:25.322+0000 7f79d4f26640 1 --2- 172.21.15.148:0/567219613 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f79bc0357b0 0x7f79bc037c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:36:25.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:25.322+0000 7f79d4f26640 1 --2- 172.21.15.148:0/567219613 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f79bc0357b0 0x7f79bc037c60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f79c0006fd0 tx=0x7f79c0006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:36:25.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:25.326+0000 7f79c5ffb640 1 -- 172.21.15.148:0/567219613 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f79cc03a4b0 con 0x7f79d00fd630 2024-07-03T15:36:25.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:25.500+0000 7f79d798b640 1 -- 172.21.15.148:0/567219613 --> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi028:172.21.15.28=smithi028;smithi148:172.21.15.148=smithi148", "target": ["mon-mgr", ""]}) v1 -- 0x7f79d0105c20 con 0x7f79bc0357b0 2024-07-03T15:36:25.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:25.505+0000 7f79c5ffb640 1 -- 172.21.15.148:0/567219613 <== mgr.14160 v2:172.21.15.28:6800/577887491 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f79d0105c20 con 0x7f79bc0357b0 2024-07-03T15:36:25.508 INFO:teuthology.orchestra.run.smithi148.stdout:Scheduled mon update... 2024-07-03T15:36:25.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:25.507+0000 7f79d798b640 1 -- 172.21.15.148:0/567219613 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f79bc0357b0 msgr2=0x7f79bc037c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:25.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:25.507+0000 7f79d798b640 1 --2- 172.21.15.148:0/567219613 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f79bc0357b0 0x7f79bc037c60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f79c0006fd0 tx=0x7f79c0006db0 comp rx=0 tx=0).stop 2024-07-03T15:36:25.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:25.507+0000 7f79d798b640 1 -- 172.21.15.148:0/567219613 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f79d00fd630 msgr2=0x7f79d010c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:25.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:25.507+0000 7f79d798b640 1 --2- 172.21.15.148:0/567219613 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f79d00fd630 0x7f79d010c5b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f79cc0098d0 tx=0x7f79cc009da0 comp rx=0 tx=0).stop 2024-07-03T15:36:25.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:25.507+0000 7f79d798b640 1 -- 172.21.15.148:0/567219613 shutdown_connections 2024-07-03T15:36:25.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:25.507+0000 7f79d798b640 1 --2- 172.21.15.148:0/567219613 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f79bc0357b0 0x7f79bc037c60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:25.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:25.507+0000 7f79d798b640 1 --2- 172.21.15.148:0/567219613 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f79d00fd630 0x7f79d010c5b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:25.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:25.507+0000 7f79d798b640 1 -- 172.21.15.148:0/567219613 >> 172.21.15.148:0/567219613 conn(0x7f79d00f5e50 msgr2=0x7f79d0103190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:36:25.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:25.508+0000 7f79d798b640 1 -- 172.21.15.148:0/567219613 shutdown_connections 2024-07-03T15:36:25.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:25.508+0000 7f79d798b640 1 -- 172.21.15.148:0/567219613 wait complete. 2024-07-03T15:36:26.043 DEBUG:teuthology.orchestra.run.smithi148:mon.smithi148> sudo journalctl -f -n 0 -u ceph-a85e4be6-3951-11ef-bcab-c7b262605968@mon.smithi148.service 2024-07-03T15:36:26.045 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-07-03T15:36:26.046 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph mon dump -f json 2024-07-03T15:36:26.061 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:25 smithi028 ceph-mon[29148]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:36:26.061 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:25 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:36:26.320 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-03T15:36:26.367 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-03T15:36:27.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:26 smithi028 ceph-mon[29148]: from='client.14192 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi028:172.21.15.28=smithi028;smithi148:172.21.15.148=smithi148", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:36:27.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:26 smithi028 ceph-mon[29148]: Saving service mon spec with placement smithi028:172.21.15.28=smithi028;smithi148:172.21.15.148=smithi148;count:2 2024-07-03T15:36:27.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:26 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:36:27.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:26 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:36:27.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:26 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:36:27.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:26 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:36:27.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:26 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:36:27.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:26 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:36:27.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:26 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-07-03T15:36:27.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:26 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:36:27.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:27.788+0000 7f2fd06fe640 1 -- 172.21.15.148:0/2464915340 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2fc80fbfe0 msgr2=0x7f2fc80fc3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:27.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:27.788+0000 7f2fd06fe640 1 --2- 172.21.15.148:0/2464915340 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2fc80fbfe0 0x7f2fc80fc3f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4004850 tx=0x7f2fc4009da0 comp rx=0 tx=0).stop 2024-07-03T15:36:27.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:27.788+0000 7f2fd06fe640 1 -- 172.21.15.148:0/2464915340 shutdown_connections 2024-07-03T15:36:27.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:27.788+0000 7f2fd06fe640 1 --2- 172.21.15.148:0/2464915340 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2fc80fbfe0 0x7f2fc80fc3f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:27.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:27.788+0000 7f2fd06fe640 1 -- 172.21.15.148:0/2464915340 >> 172.21.15.148:0/2464915340 conn(0x7f2fc80f77c0 msgr2=0x7f2fc80f9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:36:27.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:27.788+0000 7f2fd06fe640 1 -- 172.21.15.148:0/2464915340 shutdown_connections 2024-07-03T15:36:27.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:27.788+0000 7f2fd06fe640 1 -- 172.21.15.148:0/2464915340 wait complete. 2024-07-03T15:36:27.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:27.789+0000 7f2fd06fe640 1 Processor -- start 2024-07-03T15:36:27.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:27.789+0000 7f2fd06fe640 1 -- start start 2024-07-03T15:36:27.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:27.789+0000 7f2fd06fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2fc80fbfe0 0x7f2fc810c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:36:27.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:27.789+0000 7f2fd06fe640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc810cb20 con 0x7f2fc80fbfe0 2024-07-03T15:36:27.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:27.789+0000 7f2fce49a640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2fc80fbfe0 0x7f2fc810c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:36:27.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:27.789+0000 7f2fce49a640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2fc80fbfe0 0x7f2fc810c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.148:35264/0 (socket says 172.21.15.148:35264) 2024-07-03T15:36:27.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:27.789+0000 7f2fce49a640 1 -- 172.21.15.148:0/2379683878 learned_addr learned my addr 172.21.15.148:0/2379683878 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-07-03T15:36:27.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:27.790+0000 7f2fce49a640 1 -- 172.21.15.148:0/2379683878 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc4005040 con 0x7f2fc80fbfe0 2024-07-03T15:36:27.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:27.790+0000 7f2fce49a640 1 --2- 172.21.15.148:0/2379683878 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2fc80fbfe0 0x7f2fc810c5e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4004820 tx=0x7f2fc4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:36:27.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:27.790+0000 7f2fbeffd640 1 -- 172.21.15.148:0/2379683878 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2fc4032650 con 0x7f2fc80fbfe0 2024-07-03T15:36:27.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:27.790+0000 7f2fd06fe640 1 -- 172.21.15.148:0/2379683878 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fc810cd20 con 0x7f2fc80fbfe0 2024-07-03T15:36:27.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:27.790+0000 7f2fbeffd640 1 -- 172.21.15.148:0/2379683878 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2fc4032c80 con 0x7f2fc80fbfe0 2024-07-03T15:36:27.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:27.790+0000 7f2fbeffd640 1 -- 172.21.15.148:0/2379683878 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2fc403eda0 con 0x7f2fc80fbfe0 2024-07-03T15:36:27.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:27.790+0000 7f2fd06fe640 1 -- 172.21.15.148:0/2379683878 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fc810d8b0 con 0x7f2fc80fbfe0 2024-07-03T15:36:27.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:27.791+0000 7f2fd06fe640 1 -- 172.21.15.148:0/2379683878 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fc810d1f0 con 0x7f2fc80fbfe0 2024-07-03T15:36:27.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:27.791+0000 7f2fbeffd640 1 -- 172.21.15.148:0/2379683878 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 13) v1 ==== 42692+0+0 (secure 0 0 0) 0x7f2fc4030020 con 0x7f2fc80fbfe0 2024-07-03T15:36:27.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:27.792+0000 7f2fbeffd640 1 --2- 172.21.15.148:0/2379683878 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f2fb40357b0 0x7f2fb4037c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:36:27.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:27.792+0000 7f2fbeffd640 1 -- 172.21.15.148:0/2379683878 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f2fc4067b20 con 0x7f2fc80fbfe0 2024-07-03T15:36:27.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:27.792+0000 7f2fcdc99640 1 --2- 172.21.15.148:0/2379683878 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f2fb40357b0 0x7f2fb4037c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:36:27.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:27.792+0000 7f2fcdc99640 1 --2- 172.21.15.148:0/2379683878 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f2fb40357b0 0x7f2fb4037c60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8006fd0 tx=0x7f2fb8006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:36:27.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:27.796+0000 7f2fbeffd640 1 -- 172.21.15.148:0/2379683878 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2fc810d1f0 con 0x7f2fc80fbfe0 2024-07-03T15:36:28.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:28.022+0000 7f2fd06fe640 1 -- 172.21.15.148:0/2379683878 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2fc8102550 con 0x7f2fc80fbfe0 2024-07-03T15:36:28.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:28.022+0000 7f2fbeffd640 1 -- 172.21.15.148:0/2379683878 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f2fc40327b0 con 0x7f2fc80fbfe0 2024-07-03T15:36:28.025 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:36:28.025 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"a85e4be6-3951-11ef-bcab-c7b262605968","modified":"2024-07-03T15:34:53.167064Z","created":"2024-07-03T15:34:53.167064Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi028","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:3300","nonce":0},{"type":"v1","addr":"172.21.15.28:6789","nonce":0}]},"addr":"172.21.15.28:6789/0","public_addr":"172.21.15.28:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-07-03T15:36:28.025 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-07-03T15:36:28.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:28.024+0000 7f2fd06fe640 1 -- 172.21.15.148:0/2379683878 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f2fb40357b0 msgr2=0x7f2fb4037c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:28.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:28.024+0000 7f2fd06fe640 1 --2- 172.21.15.148:0/2379683878 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f2fb40357b0 0x7f2fb4037c60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8006fd0 tx=0x7f2fb8006db0 comp rx=0 tx=0).stop 2024-07-03T15:36:28.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:28.024+0000 7f2fd06fe640 1 -- 172.21.15.148:0/2379683878 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2fc80fbfe0 msgr2=0x7f2fc810c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:28.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:28.024+0000 7f2fd06fe640 1 --2- 172.21.15.148:0/2379683878 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2fc80fbfe0 0x7f2fc810c5e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4004820 tx=0x7f2fc4009da0 comp rx=0 tx=0).stop 2024-07-03T15:36:28.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:28.024+0000 7f2fd06fe640 1 -- 172.21.15.148:0/2379683878 shutdown_connections 2024-07-03T15:36:28.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:28.024+0000 7f2fd06fe640 1 --2- 172.21.15.148:0/2379683878 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f2fb40357b0 0x7f2fb4037c60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:28.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:28.024+0000 7f2fd06fe640 1 --2- 172.21.15.148:0/2379683878 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2fc80fbfe0 0x7f2fc810c5e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:28.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:28.024+0000 7f2fd06fe640 1 -- 172.21.15.148:0/2379683878 >> 172.21.15.148:0/2379683878 conn(0x7f2fc80f77c0 msgr2=0x7f2fc8103180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:36:28.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:28.024+0000 7f2fd06fe640 1 -- 172.21.15.148:0/2379683878 shutdown_connections 2024-07-03T15:36:28.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:28.024+0000 7f2fd06fe640 1 -- 172.21.15.148:0/2379683878 wait complete. 2024-07-03T15:36:28.120 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:27 smithi028 ceph-mon[29148]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-07-03T15:36:28.120 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:27 smithi028 ceph-mon[29148]: Deploying daemon grafana.smithi028 on smithi028 2024-07-03T15:36:28.120 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:27 smithi028 ceph-mon[29148]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:36:28.120 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:27 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:36:29.120 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:28 smithi028 ceph-mon[29148]: from='client.? 172.21.15.148:0/2379683878' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-03T15:36:29.562 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-07-03T15:36:29.562 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph mon dump -f json 2024-07-03T15:36:29.812 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-03T15:36:29.859 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-03T15:36:30.120 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:29 smithi028 ceph-mon[29148]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:36:31.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:31.421+0000 7f38b8eb0640 1 -- 172.21.15.148:0/999061468 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38b40faed0 msgr2=0x7f38b40fb2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:31.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:31.421+0000 7f38b8eb0640 1 --2- 172.21.15.148:0/999061468 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38b40faed0 0x7f38b40fb2e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f38ac009bd0 tx=0x7f38ac004d10 comp rx=0 tx=0).stop 2024-07-03T15:36:31.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:31.421+0000 7f38b8eb0640 1 -- 172.21.15.148:0/999061468 shutdown_connections 2024-07-03T15:36:31.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:31.421+0000 7f38b8eb0640 1 --2- 172.21.15.148:0/999061468 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38b40faed0 0x7f38b40fb2e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:31.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:31.421+0000 7f38b8eb0640 1 -- 172.21.15.148:0/999061468 >> 172.21.15.148:0/999061468 conn(0x7f38b40f6690 msgr2=0x7f38b40f8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:36:31.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:31.421+0000 7f38b8eb0640 1 -- 172.21.15.148:0/999061468 shutdown_connections 2024-07-03T15:36:31.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:31.421+0000 7f38b8eb0640 1 -- 172.21.15.148:0/999061468 wait complete. 2024-07-03T15:36:31.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:31.422+0000 7f38b8eb0640 1 Processor -- start 2024-07-03T15:36:31.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:31.422+0000 7f38b8eb0640 1 -- start start 2024-07-03T15:36:31.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:31.422+0000 7f38b8eb0640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38b40faed0 0x7f38b4065e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:36:31.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:31.422+0000 7f38b8eb0640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b410f2e0 con 0x7f38b40faed0 2024-07-03T15:36:31.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:31.422+0000 7f38b259c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38b40faed0 0x7f38b4065e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:36:31.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:31.422+0000 7f38b259c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38b40faed0 0x7f38b4065e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.148:46934/0 (socket says 172.21.15.148:46934) 2024-07-03T15:36:31.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:31.422+0000 7f38b259c640 1 -- 172.21.15.148:0/1808843447 learned_addr learned my addr 172.21.15.148:0/1808843447 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-07-03T15:36:31.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:31.423+0000 7f38b259c640 1 -- 172.21.15.148:0/1808843447 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38ac0098b0 con 0x7f38b40faed0 2024-07-03T15:36:31.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:31.423+0000 7f38b259c640 1 --2- 172.21.15.148:0/1808843447 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38b40faed0 0x7f38b4065e70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f38ac009ba0 tx=0x7f38ac004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:36:31.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:31.423+0000 7f38aaffd640 1 -- 172.21.15.148:0/1808843447 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f38ac0329b0 con 0x7f38b40faed0 2024-07-03T15:36:31.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:31.423+0000 7f38b8eb0640 1 -- 172.21.15.148:0/1808843447 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38b410f420 con 0x7f38b40faed0 2024-07-03T15:36:31.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:31.423+0000 7f38aaffd640 1 -- 172.21.15.148:0/1808843447 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f38ac03d420 con 0x7f38b40faed0 2024-07-03T15:36:31.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:31.423+0000 7f38aaffd640 1 -- 172.21.15.148:0/1808843447 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f38ac031590 con 0x7f38b40faed0 2024-07-03T15:36:31.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:31.423+0000 7f38b8eb0640 1 -- 172.21.15.148:0/1808843447 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38b410f870 con 0x7f38b40faed0 2024-07-03T15:36:31.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:31.424+0000 7f38aaffd640 1 -- 172.21.15.148:0/1808843447 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 13) v1 ==== 42692+0+0 (secure 0 0 0) 0x7f38ac02f030 con 0x7f38b40faed0 2024-07-03T15:36:31.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:31.424+0000 7f38b8eb0640 1 -- 172.21.15.148:0/1808843447 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38b4101440 con 0x7f38b40faed0 2024-07-03T15:36:31.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:31.424+0000 7f38aaffd640 1 --2- 172.21.15.148:0/1808843447 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f38a0035760 0x7f38a0037c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:36:31.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:31.424+0000 7f38aaffd640 1 -- 172.21.15.148:0/1808843447 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f38ac068e10 con 0x7f38b40faed0 2024-07-03T15:36:31.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:31.424+0000 7f38b1d9b640 1 --2- 172.21.15.148:0/1808843447 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f38a0035760 0x7f38a0037c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:36:31.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:31.425+0000 7f38b1d9b640 1 --2- 172.21.15.148:0/1808843447 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f38a0035760 0x7f38a0037c10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f389c006fd0 tx=0x7f389c006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:36:31.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:31.429+0000 7f38aaffd640 1 -- 172.21.15.148:0/1808843447 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f38ac06c030 con 0x7f38b40faed0 2024-07-03T15:36:31.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:31.660+0000 7f38b8eb0640 1 -- 172.21.15.148:0/1808843447 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f38b405eb80 con 0x7f38b40faed0 2024-07-03T15:36:31.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:31.661+0000 7f38aaffd640 1 -- 172.21.15.148:0/1808843447 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f38ac041070 con 0x7f38b40faed0 2024-07-03T15:36:31.663 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:36:31.663 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"a85e4be6-3951-11ef-bcab-c7b262605968","modified":"2024-07-03T15:34:53.167064Z","created":"2024-07-03T15:34:53.167064Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi028","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:3300","nonce":0},{"type":"v1","addr":"172.21.15.28:6789","nonce":0}]},"addr":"172.21.15.28:6789/0","public_addr":"172.21.15.28:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-07-03T15:36:31.663 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-07-03T15:36:31.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:31.663+0000 7f38b8eb0640 1 -- 172.21.15.148:0/1808843447 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f38a0035760 msgr2=0x7f38a0037c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:31.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:31.663+0000 7f38b8eb0640 1 --2- 172.21.15.148:0/1808843447 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f38a0035760 0x7f38a0037c10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f389c006fd0 tx=0x7f389c006db0 comp rx=0 tx=0).stop 2024-07-03T15:36:31.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:31.663+0000 7f38b8eb0640 1 -- 172.21.15.148:0/1808843447 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38b40faed0 msgr2=0x7f38b4065e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:31.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:31.663+0000 7f38b8eb0640 1 --2- 172.21.15.148:0/1808843447 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38b40faed0 0x7f38b4065e70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f38ac009ba0 tx=0x7f38ac004d10 comp rx=0 tx=0).stop 2024-07-03T15:36:31.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:31.663+0000 7f38b8eb0640 1 -- 172.21.15.148:0/1808843447 shutdown_connections 2024-07-03T15:36:31.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:31.663+0000 7f38b8eb0640 1 --2- 172.21.15.148:0/1808843447 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f38a0035760 0x7f38a0037c10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:31.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:31.663+0000 7f38b8eb0640 1 --2- 172.21.15.148:0/1808843447 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38b40faed0 0x7f38b4065e70 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:31.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:31.663+0000 7f38b8eb0640 1 -- 172.21.15.148:0/1808843447 >> 172.21.15.148:0/1808843447 conn(0x7f38b40f6690 msgr2=0x7f38b40f71f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:36:31.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:31.663+0000 7f38b8eb0640 1 -- 172.21.15.148:0/1808843447 shutdown_connections 2024-07-03T15:36:31.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:31.663+0000 7f38b8eb0640 1 -- 172.21.15.148:0/1808843447 wait complete. 2024-07-03T15:36:32.120 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:31 smithi028 ceph-mon[29148]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:36:32.120 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:31 smithi028 ceph-mon[29148]: from='client.? 172.21.15.148:0/1808843447' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-03T15:36:33.225 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-07-03T15:36:33.226 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph mon dump -f json 2024-07-03T15:36:33.490 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-03T15:36:33.542 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-03T15:36:34.120 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:33 smithi028 ceph-mon[29148]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:36:34.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:34.975+0000 7f2559f15640 1 -- 172.21.15.148:0/4176929033 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2554065f50 msgr2=0x7f25540fec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:34.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:34.975+0000 7f2559f15640 1 --2- 172.21.15.148:0/4176929033 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2554065f50 0x7f25540fec60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2544009bd0 tx=0x7f2544004d10 comp rx=0 tx=0).stop 2024-07-03T15:36:34.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:34.975+0000 7f2559f15640 1 -- 172.21.15.148:0/4176929033 shutdown_connections 2024-07-03T15:36:34.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:34.975+0000 7f2559f15640 1 --2- 172.21.15.148:0/4176929033 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2554065f50 0x7f25540fec60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:34.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:34.975+0000 7f2559f15640 1 -- 172.21.15.148:0/4176929033 >> 172.21.15.148:0/4176929033 conn(0x7f25540f6690 msgr2=0x7f25540f8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:36:34.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:34.975+0000 7f2559f15640 1 -- 172.21.15.148:0/4176929033 shutdown_connections 2024-07-03T15:36:34.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:34.975+0000 7f2559f15640 1 -- 172.21.15.148:0/4176929033 wait complete. 2024-07-03T15:36:34.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:34.975+0000 7f2559f15640 1 Processor -- start 2024-07-03T15:36:34.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:34.976+0000 7f2559f15640 1 -- start start 2024-07-03T15:36:34.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:34.976+0000 7f2559f15640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2554065f50 0x7f255410c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:36:34.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:34.976+0000 7f2559f15640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255410cb50 con 0x7f2554065f50 2024-07-03T15:36:34.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:34.976+0000 7f25537fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2554065f50 0x7f255410c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:36:34.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:34.976+0000 7f25537fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2554065f50 0x7f255410c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.148:46940/0 (socket says 172.21.15.148:46940) 2024-07-03T15:36:34.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:34.976+0000 7f25537fe640 1 -- 172.21.15.148:0/3218944017 learned_addr learned my addr 172.21.15.148:0/3218944017 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-07-03T15:36:34.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:34.976+0000 7f25537fe640 1 -- 172.21.15.148:0/3218944017 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25440098b0 con 0x7f2554065f50 2024-07-03T15:36:34.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:34.977+0000 7f25537fe640 1 --2- 172.21.15.148:0/3218944017 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2554065f50 0x7f255410c610 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2544009ba0 tx=0x7f2544004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:36:34.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:34.977+0000 7f253bfff640 1 -- 172.21.15.148:0/3218944017 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f25440329b0 con 0x7f2554065f50 2024-07-03T15:36:34.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:34.977+0000 7f2559f15640 1 -- 172.21.15.148:0/3218944017 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f255410cd50 con 0x7f2554065f50 2024-07-03T15:36:34.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:34.977+0000 7f253bfff640 1 -- 172.21.15.148:0/3218944017 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f254403d420 con 0x7f2554065f50 2024-07-03T15:36:34.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:34.977+0000 7f253bfff640 1 -- 172.21.15.148:0/3218944017 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2544031590 con 0x7f2554065f50 2024-07-03T15:36:34.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:34.977+0000 7f2559f15640 1 -- 172.21.15.148:0/3218944017 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f255410d1a0 con 0x7f2554065f50 2024-07-03T15:36:34.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:34.978+0000 7f253bfff640 1 -- 172.21.15.148:0/3218944017 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 13) v1 ==== 42692+0+0 (secure 0 0 0) 0x7f254402f030 con 0x7f2554065f50 2024-07-03T15:36:34.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:34.978+0000 7f2559f15640 1 -- 172.21.15.148:0/3218944017 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f255405eb80 con 0x7f2554065f50 2024-07-03T15:36:34.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:34.978+0000 7f253bfff640 1 --2- 172.21.15.148:0/3218944017 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f253c035760 0x7f253c037c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:36:34.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:34.978+0000 7f253bfff640 1 -- 172.21.15.148:0/3218944017 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f2544068e10 con 0x7f2554065f50 2024-07-03T15:36:34.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:34.978+0000 7f2552ffd640 1 --2- 172.21.15.148:0/3218944017 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f253c035760 0x7f253c037c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:36:34.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:34.978+0000 7f2552ffd640 1 --2- 172.21.15.148:0/3218944017 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f253c035760 0x7f253c037c10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2548006fd0 tx=0x7f2548006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:36:34.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:34.982+0000 7f253bfff640 1 -- 172.21.15.148:0/3218944017 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f254406c030 con 0x7f2554065f50 2024-07-03T15:36:35.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:35.209+0000 7f2559f15640 1 -- 172.21.15.148:0/3218944017 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2554101d80 con 0x7f2554065f50 2024-07-03T15:36:35.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:35.209+0000 7f253bfff640 1 -- 172.21.15.148:0/3218944017 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f2544032b10 con 0x7f2554065f50 2024-07-03T15:36:35.212 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:36:35.212 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"a85e4be6-3951-11ef-bcab-c7b262605968","modified":"2024-07-03T15:34:53.167064Z","created":"2024-07-03T15:34:53.167064Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi028","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:3300","nonce":0},{"type":"v1","addr":"172.21.15.28:6789","nonce":0}]},"addr":"172.21.15.28:6789/0","public_addr":"172.21.15.28:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-07-03T15:36:35.212 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-07-03T15:36:35.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:35.211+0000 7f2559f15640 1 -- 172.21.15.148:0/3218944017 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f253c035760 msgr2=0x7f253c037c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:35.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:35.211+0000 7f2559f15640 1 --2- 172.21.15.148:0/3218944017 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f253c035760 0x7f253c037c10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2548006fd0 tx=0x7f2548006db0 comp rx=0 tx=0).stop 2024-07-03T15:36:35.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:35.211+0000 7f2559f15640 1 -- 172.21.15.148:0/3218944017 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2554065f50 msgr2=0x7f255410c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:35.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:35.211+0000 7f2559f15640 1 --2- 172.21.15.148:0/3218944017 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2554065f50 0x7f255410c610 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2544009ba0 tx=0x7f2544004d10 comp rx=0 tx=0).stop 2024-07-03T15:36:35.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:35.211+0000 7f2559f15640 1 -- 172.21.15.148:0/3218944017 shutdown_connections 2024-07-03T15:36:35.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:35.211+0000 7f2559f15640 1 --2- 172.21.15.148:0/3218944017 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f253c035760 0x7f253c037c10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:35.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:35.211+0000 7f2559f15640 1 --2- 172.21.15.148:0/3218944017 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2554065f50 0x7f255410c610 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:35.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:35.211+0000 7f2559f15640 1 -- 172.21.15.148:0/3218944017 >> 172.21.15.148:0/3218944017 conn(0x7f25540f6690 msgr2=0x7f2554103180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:36:35.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:35.212+0000 7f2559f15640 1 -- 172.21.15.148:0/3218944017 shutdown_connections 2024-07-03T15:36:35.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:35.212+0000 7f2559f15640 1 -- 172.21.15.148:0/3218944017 wait complete. 2024-07-03T15:36:36.120 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:35 smithi028 ceph-mon[29148]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:36:36.120 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:35 smithi028 ceph-mon[29148]: from='client.? 172.21.15.148:0/3218944017' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-03T15:36:36.811 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-07-03T15:36:36.811 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph mon dump -f json 2024-07-03T15:36:37.056 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-03T15:36:37.103 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-03T15:36:38.120 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:37 smithi028 ceph-mon[29148]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:36:38.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:38.485+0000 7f35d5e54640 1 -- 172.21.15.148:0/2626263983 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35d00faed0 msgr2=0x7f35d00fb2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:38.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:38.485+0000 7f35d5e54640 1 --2- 172.21.15.148:0/2626263983 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35d00faed0 0x7f35d00fb2e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f35c0004850 tx=0x7f35c0009da0 comp rx=0 tx=0).stop 2024-07-03T15:36:38.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:38.485+0000 7f35d5e54640 1 -- 172.21.15.148:0/2626263983 shutdown_connections 2024-07-03T15:36:38.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:38.485+0000 7f35d5e54640 1 --2- 172.21.15.148:0/2626263983 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35d00faed0 0x7f35d00fb2e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:38.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:38.485+0000 7f35d5e54640 1 -- 172.21.15.148:0/2626263983 >> 172.21.15.148:0/2626263983 conn(0x7f35d00f6690 msgr2=0x7f35d00f8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:36:38.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:38.485+0000 7f35d5e54640 1 -- 172.21.15.148:0/2626263983 shutdown_connections 2024-07-03T15:36:38.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:38.485+0000 7f35d5e54640 1 -- 172.21.15.148:0/2626263983 wait complete. 2024-07-03T15:36:38.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:38.486+0000 7f35d5e54640 1 Processor -- start 2024-07-03T15:36:38.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:38.486+0000 7f35d5e54640 1 -- start start 2024-07-03T15:36:38.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:38.486+0000 7f35d5e54640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35d00faed0 0x7f35d010b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:36:38.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:38.486+0000 7f35d5e54640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d010ba00 con 0x7f35d00faed0 2024-07-03T15:36:38.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:38.486+0000 7f35cf7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35d00faed0 0x7f35d010b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:36:38.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:38.486+0000 7f35cf7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35d00faed0 0x7f35d010b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.148:46954/0 (socket says 172.21.15.148:46954) 2024-07-03T15:36:38.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:38.486+0000 7f35cf7fe640 1 -- 172.21.15.148:0/1572532964 learned_addr learned my addr 172.21.15.148:0/1572532964 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-07-03T15:36:38.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:38.487+0000 7f35cf7fe640 1 -- 172.21.15.148:0/1572532964 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c0005040 con 0x7f35d00faed0 2024-07-03T15:36:38.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:38.487+0000 7f35cf7fe640 1 --2- 172.21.15.148:0/1572532964 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35d00faed0 0x7f35d010b4c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f35c0004820 tx=0x7f35c0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:36:38.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:38.487+0000 7f35ccff9640 1 -- 172.21.15.148:0/1572532964 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f35c0032650 con 0x7f35d00faed0 2024-07-03T15:36:38.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:38.487+0000 7f35ccff9640 1 -- 172.21.15.148:0/1572532964 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f35c0032c80 con 0x7f35d00faed0 2024-07-03T15:36:38.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:38.487+0000 7f35d5e54640 1 -- 172.21.15.148:0/1572532964 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35d010bc00 con 0x7f35d00faed0 2024-07-03T15:36:38.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:38.487+0000 7f35ccff9640 1 -- 172.21.15.148:0/1572532964 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f35c003eda0 con 0x7f35d00faed0 2024-07-03T15:36:38.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:38.487+0000 7f35d5e54640 1 -- 172.21.15.148:0/1572532964 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35d010c790 con 0x7f35d00faed0 2024-07-03T15:36:38.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:38.488+0000 7f35ccff9640 1 -- 172.21.15.148:0/1572532964 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 13) v1 ==== 42692+0+0 (secure 0 0 0) 0x7f35c0030020 con 0x7f35d00faed0 2024-07-03T15:36:38.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:38.488+0000 7f35d5e54640 1 -- 172.21.15.148:0/1572532964 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35d0101440 con 0x7f35d00faed0 2024-07-03T15:36:38.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:38.488+0000 7f35ccff9640 1 --2- 172.21.15.148:0/1572532964 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f35bc0357b0 0x7f35bc037c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:36:38.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:38.488+0000 7f35ccff9640 1 -- 172.21.15.148:0/1572532964 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f35c0067b20 con 0x7f35d00faed0 2024-07-03T15:36:38.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:38.488+0000 7f35c7fff640 1 --2- 172.21.15.148:0/1572532964 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f35bc0357b0 0x7f35bc037c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:36:38.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:38.489+0000 7f35c7fff640 1 --2- 172.21.15.148:0/1572532964 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f35bc0357b0 0x7f35bc037c60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f35b8006fd0 tx=0x7f35b8006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:36:38.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:38.493+0000 7f35ccff9640 1 -- 172.21.15.148:0/1572532964 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f35c0033420 con 0x7f35d00faed0 2024-07-03T15:36:38.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:38.718+0000 7f35d5e54640 1 -- 172.21.15.148:0/1572532964 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f35d010d190 con 0x7f35d00faed0 2024-07-03T15:36:38.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:38.718+0000 7f35ccff9640 1 -- 172.21.15.148:0/1572532964 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f35c003c330 con 0x7f35d00faed0 2024-07-03T15:36:38.720 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:36:38.720 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"a85e4be6-3951-11ef-bcab-c7b262605968","modified":"2024-07-03T15:34:53.167064Z","created":"2024-07-03T15:34:53.167064Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi028","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:3300","nonce":0},{"type":"v1","addr":"172.21.15.28:6789","nonce":0}]},"addr":"172.21.15.28:6789/0","public_addr":"172.21.15.28:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-07-03T15:36:38.721 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-07-03T15:36:38.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:38.720+0000 7f35d5e54640 1 -- 172.21.15.148:0/1572532964 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f35bc0357b0 msgr2=0x7f35bc037c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:38.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:38.720+0000 7f35d5e54640 1 --2- 172.21.15.148:0/1572532964 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f35bc0357b0 0x7f35bc037c60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f35b8006fd0 tx=0x7f35b8006db0 comp rx=0 tx=0).stop 2024-07-03T15:36:38.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:38.720+0000 7f35d5e54640 1 -- 172.21.15.148:0/1572532964 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35d00faed0 msgr2=0x7f35d010b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:38.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:38.720+0000 7f35d5e54640 1 --2- 172.21.15.148:0/1572532964 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35d00faed0 0x7f35d010b4c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f35c0004820 tx=0x7f35c0009da0 comp rx=0 tx=0).stop 2024-07-03T15:36:38.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:38.720+0000 7f35d5e54640 1 -- 172.21.15.148:0/1572532964 shutdown_connections 2024-07-03T15:36:38.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:38.720+0000 7f35d5e54640 1 --2- 172.21.15.148:0/1572532964 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f35bc0357b0 0x7f35bc037c60 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:38.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:38.720+0000 7f35d5e54640 1 --2- 172.21.15.148:0/1572532964 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35d00faed0 0x7f35d010b4c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:38.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:38.720+0000 7f35d5e54640 1 -- 172.21.15.148:0/1572532964 >> 172.21.15.148:0/1572532964 conn(0x7f35d00f6690 msgr2=0x7f35d00f71f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:36:38.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:38.720+0000 7f35d5e54640 1 -- 172.21.15.148:0/1572532964 shutdown_connections 2024-07-03T15:36:38.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:38.720+0000 7f35d5e54640 1 -- 172.21.15.148:0/1572532964 wait complete. 2024-07-03T15:36:39.120 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:38 smithi028 ceph-mon[29148]: from='client.? 172.21.15.148:0/1572532964' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-03T15:36:40.120 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:39 smithi028 ceph-mon[29148]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:36:40.263 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-07-03T15:36:40.263 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph mon dump -f json 2024-07-03T15:36:40.505 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-03T15:36:40.551 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-03T15:36:41.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:41.906+0000 7fdccad8d640 1 -- 172.21.15.148:0/1396396959 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdcc40fbfe0 msgr2=0x7fdcc40fc3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:41.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:41.906+0000 7fdccad8d640 1 --2- 172.21.15.148:0/1396396959 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdcc40fbfe0 0x7fdcc40fc3f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8004850 tx=0x7fdcb8009da0 comp rx=0 tx=0).stop 2024-07-03T15:36:41.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:41.906+0000 7fdccad8d640 1 -- 172.21.15.148:0/1396396959 shutdown_connections 2024-07-03T15:36:41.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:41.906+0000 7fdccad8d640 1 --2- 172.21.15.148:0/1396396959 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdcc40fbfe0 0x7fdcc40fc3f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:41.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:41.906+0000 7fdccad8d640 1 -- 172.21.15.148:0/1396396959 >> 172.21.15.148:0/1396396959 conn(0x7fdcc40f77c0 msgr2=0x7fdcc40f9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:36:41.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:41.906+0000 7fdccad8d640 1 -- 172.21.15.148:0/1396396959 shutdown_connections 2024-07-03T15:36:41.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:41.906+0000 7fdccad8d640 1 -- 172.21.15.148:0/1396396959 wait complete. 2024-07-03T15:36:41.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:41.906+0000 7fdccad8d640 1 Processor -- start 2024-07-03T15:36:41.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:41.906+0000 7fdccad8d640 1 -- start start 2024-07-03T15:36:41.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:41.907+0000 7fdccad8d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdcc40fbfe0 0x7fdcc410c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:36:41.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:41.907+0000 7fdccad8d640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc410cae0 con 0x7fdcc40fbfe0 2024-07-03T15:36:41.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:41.907+0000 7fdcc8b29640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdcc40fbfe0 0x7fdcc410c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:36:41.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:41.907+0000 7fdcc8b29640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdcc40fbfe0 0x7fdcc410c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.148:46740/0 (socket says 172.21.15.148:46740) 2024-07-03T15:36:41.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:41.907+0000 7fdcc8b29640 1 -- 172.21.15.148:0/3877713508 learned_addr learned my addr 172.21.15.148:0/3877713508 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-07-03T15:36:41.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:41.907+0000 7fdcc8b29640 1 -- 172.21.15.148:0/3877713508 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb8005040 con 0x7fdcc40fbfe0 2024-07-03T15:36:41.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:41.907+0000 7fdcc8b29640 1 --2- 172.21.15.148:0/3877713508 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdcc40fbfe0 0x7fdcc410c5a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdcb80098d0 tx=0x7fdcb8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:36:41.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:41.908+0000 7fdcc17fa640 1 -- 172.21.15.148:0/3877713508 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdcb8032650 con 0x7fdcc40fbfe0 2024-07-03T15:36:41.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:41.908+0000 7fdccad8d640 1 -- 172.21.15.148:0/3877713508 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcc410cce0 con 0x7fdcc40fbfe0 2024-07-03T15:36:41.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:41.908+0000 7fdcc17fa640 1 -- 172.21.15.148:0/3877713508 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdcb8032c80 con 0x7fdcc40fbfe0 2024-07-03T15:36:41.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:41.908+0000 7fdcc17fa640 1 -- 172.21.15.148:0/3877713508 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdcb8033c10 con 0x7fdcc40fbfe0 2024-07-03T15:36:41.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:41.908+0000 7fdccad8d640 1 -- 172.21.15.148:0/3877713508 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcc410d150 con 0x7fdcc40fbfe0 2024-07-03T15:36:41.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:41.908+0000 7fdcc17fa640 1 -- 172.21.15.148:0/3877713508 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 13) v1 ==== 42692+0+0 (secure 0 0 0) 0x7fdcb8030020 con 0x7fdcc40fbfe0 2024-07-03T15:36:41.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:41.908+0000 7fdccad8d640 1 -- 172.21.15.148:0/3877713508 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcc405eb80 con 0x7fdcc40fbfe0 2024-07-03T15:36:41.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:41.909+0000 7fdcc17fa640 1 --2- 172.21.15.148:0/3877713508 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7fdcb40357b0 0x7fdcb4037c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:36:41.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:41.909+0000 7fdcc17fa640 1 -- 172.21.15.148:0/3877713508 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fdcb8067820 con 0x7fdcc40fbfe0 2024-07-03T15:36:41.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:41.909+0000 7fdcc3fff640 1 --2- 172.21.15.148:0/3877713508 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7fdcb40357b0 0x7fdcb4037c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:36:41.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:41.909+0000 7fdcc3fff640 1 --2- 172.21.15.148:0/3877713508 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7fdcb40357b0 0x7fdcb4037c60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0006fd0 tx=0x7fdcb0006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:36:41.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:41.913+0000 7fdcc17fa640 1 -- 172.21.15.148:0/3877713508 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fdcb8044d20 con 0x7fdcc40fbfe0 2024-07-03T15:36:42.120 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:41 smithi028 ceph-mon[29148]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:36:42.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:42.140+0000 7fdccad8d640 1 -- 172.21.15.148:0/3877713508 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fdcc404b530 con 0x7fdcc40fbfe0 2024-07-03T15:36:42.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:42.141+0000 7fdcc17fa640 1 -- 172.21.15.148:0/3877713508 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fdcb8044b40 con 0x7fdcc40fbfe0 2024-07-03T15:36:42.143 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:36:42.143 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"a85e4be6-3951-11ef-bcab-c7b262605968","modified":"2024-07-03T15:34:53.167064Z","created":"2024-07-03T15:34:53.167064Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi028","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:3300","nonce":0},{"type":"v1","addr":"172.21.15.28:6789","nonce":0}]},"addr":"172.21.15.28:6789/0","public_addr":"172.21.15.28:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-07-03T15:36:42.143 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-07-03T15:36:42.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:42.143+0000 7fdccad8d640 1 -- 172.21.15.148:0/3877713508 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7fdcb40357b0 msgr2=0x7fdcb4037c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:42.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:42.143+0000 7fdccad8d640 1 --2- 172.21.15.148:0/3877713508 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7fdcb40357b0 0x7fdcb4037c60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0006fd0 tx=0x7fdcb0006db0 comp rx=0 tx=0).stop 2024-07-03T15:36:42.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:42.143+0000 7fdccad8d640 1 -- 172.21.15.148:0/3877713508 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdcc40fbfe0 msgr2=0x7fdcc410c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:42.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:42.143+0000 7fdccad8d640 1 --2- 172.21.15.148:0/3877713508 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdcc40fbfe0 0x7fdcc410c5a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdcb80098d0 tx=0x7fdcb8009da0 comp rx=0 tx=0).stop 2024-07-03T15:36:42.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:42.143+0000 7fdccad8d640 1 -- 172.21.15.148:0/3877713508 shutdown_connections 2024-07-03T15:36:42.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:42.143+0000 7fdccad8d640 1 --2- 172.21.15.148:0/3877713508 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7fdcb40357b0 0x7fdcb4037c60 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:42.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:42.143+0000 7fdccad8d640 1 --2- 172.21.15.148:0/3877713508 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdcc40fbfe0 0x7fdcc410c5a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:42.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:42.143+0000 7fdccad8d640 1 -- 172.21.15.148:0/3877713508 >> 172.21.15.148:0/3877713508 conn(0x7fdcc40f77c0 msgr2=0x7fdcc4103180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:36:42.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:42.143+0000 7fdccad8d640 1 -- 172.21.15.148:0/3877713508 shutdown_connections 2024-07-03T15:36:42.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:42.143+0000 7fdccad8d640 1 -- 172.21.15.148:0/3877713508 wait complete. 2024-07-03T15:36:42.870 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:42 smithi028 ceph-mon[29148]: from='client.? 172.21.15.148:0/3877713508' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-03T15:36:43.734 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-07-03T15:36:43.734 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph mon dump -f json 2024-07-03T15:36:43.978 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-03T15:36:43.993 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:43 smithi028 ceph-mon[29148]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:36:44.025 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-03T15:36:45.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:45.435+0000 7f6730ea3640 1 -- 172.21.15.148:0/1322208533 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f672c0fbfc0 msgr2=0x7f672c0fc3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:45.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:45.435+0000 7f6730ea3640 1 --2- 172.21.15.148:0/1322208533 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f672c0fbfc0 0x7f672c0fc3d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f671c004850 tx=0x7f671c009da0 comp rx=0 tx=0).stop 2024-07-03T15:36:45.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:45.435+0000 7f6730ea3640 1 -- 172.21.15.148:0/1322208533 shutdown_connections 2024-07-03T15:36:45.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:45.435+0000 7f6730ea3640 1 --2- 172.21.15.148:0/1322208533 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f672c0fbfc0 0x7f672c0fc3d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:45.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:45.435+0000 7f6730ea3640 1 -- 172.21.15.148:0/1322208533 >> 172.21.15.148:0/1322208533 conn(0x7f672c0f77a0 msgr2=0x7f672c0f9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:36:45.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:45.435+0000 7f6730ea3640 1 -- 172.21.15.148:0/1322208533 shutdown_connections 2024-07-03T15:36:45.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:45.435+0000 7f6730ea3640 1 -- 172.21.15.148:0/1322208533 wait complete. 2024-07-03T15:36:45.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:45.436+0000 7f6730ea3640 1 Processor -- start 2024-07-03T15:36:45.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:45.436+0000 7f6730ea3640 1 -- start start 2024-07-03T15:36:45.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:45.436+0000 7f6730ea3640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f672c0fbfc0 0x7f672c10c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:36:45.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:45.436+0000 7f6730ea3640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672c10cb00 con 0x7f672c0fbfc0 2024-07-03T15:36:45.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:45.437+0000 7f672a59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f672c0fbfc0 0x7f672c10c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:36:45.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:45.437+0000 7f672a59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f672c0fbfc0 0x7f672c10c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.148:46756/0 (socket says 172.21.15.148:46756) 2024-07-03T15:36:45.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:45.437+0000 7f672a59c640 1 -- 172.21.15.148:0/590190099 learned_addr learned my addr 172.21.15.148:0/590190099 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-07-03T15:36:45.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:45.437+0000 7f672a59c640 1 -- 172.21.15.148:0/590190099 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f671c005040 con 0x7f672c0fbfc0 2024-07-03T15:36:45.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:45.437+0000 7f672a59c640 1 --2- 172.21.15.148:0/590190099 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f672c0fbfc0 0x7f672c10c5c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f671c004820 tx=0x7f671c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:36:45.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:45.437+0000 7f67237fe640 1 -- 172.21.15.148:0/590190099 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f671c032650 con 0x7f672c0fbfc0 2024-07-03T15:36:45.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:45.437+0000 7f6730ea3640 1 -- 172.21.15.148:0/590190099 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f672c10cd00 con 0x7f672c0fbfc0 2024-07-03T15:36:45.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:45.437+0000 7f67237fe640 1 -- 172.21.15.148:0/590190099 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f671c032c80 con 0x7f672c0fbfc0 2024-07-03T15:36:45.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:45.437+0000 7f67237fe640 1 -- 172.21.15.148:0/590190099 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f671c03eda0 con 0x7f672c0fbfc0 2024-07-03T15:36:45.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:45.437+0000 7f6730ea3640 1 -- 172.21.15.148:0/590190099 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f672c10d890 con 0x7f672c0fbfc0 2024-07-03T15:36:45.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:45.438+0000 7f6730ea3640 1 -- 172.21.15.148:0/590190099 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f672c102530 con 0x7f672c0fbfc0 2024-07-03T15:36:45.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:45.438+0000 7f67237fe640 1 -- 172.21.15.148:0/590190099 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 13) v1 ==== 42692+0+0 (secure 0 0 0) 0x7f671c030020 con 0x7f672c0fbfc0 2024-07-03T15:36:45.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:45.439+0000 7f67237fe640 1 --2- 172.21.15.148:0/590190099 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f67180357b0 0x7f6718037c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:36:45.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:45.439+0000 7f67237fe640 1 -- 172.21.15.148:0/590190099 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f671c067a60 con 0x7f672c0fbfc0 2024-07-03T15:36:45.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:45.439+0000 7f6721d9b640 1 --2- 172.21.15.148:0/590190099 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f67180357b0 0x7f6718037c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:36:45.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:45.439+0000 7f6721d9b640 1 --2- 172.21.15.148:0/590190099 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f67180357b0 0x7f6718037c60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6714006fd0 tx=0x7f6714006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:36:45.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:45.447+0000 7f67237fe640 1 -- 172.21.15.148:0/590190099 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f671c0327b0 con 0x7f672c0fbfc0 2024-07-03T15:36:45.499 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:45 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:36:45.499 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:45 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:36:45.500 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:45 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:36:45.500 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:45 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:36:45.500 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:45 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:36:45.500 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:45 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:36:45.500 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:45 smithi028 ceph-mon[29148]: Deploying daemon prometheus.smithi028 on smithi028 2024-07-03T15:36:45.500 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:45 smithi028 ceph-mon[29148]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:36:45.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:45.676+0000 7f6730ea3640 1 -- 172.21.15.148:0/590190099 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f672c10e290 con 0x7f672c0fbfc0 2024-07-03T15:36:45.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:45.677+0000 7f67237fe640 1 -- 172.21.15.148:0/590190099 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f671c03c330 con 0x7f672c0fbfc0 2024-07-03T15:36:45.679 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:36:45.679 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"a85e4be6-3951-11ef-bcab-c7b262605968","modified":"2024-07-03T15:34:53.167064Z","created":"2024-07-03T15:34:53.167064Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi028","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:3300","nonce":0},{"type":"v1","addr":"172.21.15.28:6789","nonce":0}]},"addr":"172.21.15.28:6789/0","public_addr":"172.21.15.28:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-07-03T15:36:45.679 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-07-03T15:36:45.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:45.678+0000 7f6730ea3640 1 -- 172.21.15.148:0/590190099 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f67180357b0 msgr2=0x7f6718037c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:45.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:45.678+0000 7f6730ea3640 1 --2- 172.21.15.148:0/590190099 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f67180357b0 0x7f6718037c60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6714006fd0 tx=0x7f6714006db0 comp rx=0 tx=0).stop 2024-07-03T15:36:45.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:45.678+0000 7f6730ea3640 1 -- 172.21.15.148:0/590190099 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f672c0fbfc0 msgr2=0x7f672c10c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:45.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:45.678+0000 7f6730ea3640 1 --2- 172.21.15.148:0/590190099 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f672c0fbfc0 0x7f672c10c5c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f671c004820 tx=0x7f671c009da0 comp rx=0 tx=0).stop 2024-07-03T15:36:45.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:45.679+0000 7f6730ea3640 1 -- 172.21.15.148:0/590190099 shutdown_connections 2024-07-03T15:36:45.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:45.679+0000 7f6730ea3640 1 --2- 172.21.15.148:0/590190099 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f67180357b0 0x7f6718037c60 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:45.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:45.679+0000 7f6730ea3640 1 --2- 172.21.15.148:0/590190099 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f672c0fbfc0 0x7f672c10c5c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:45.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:45.679+0000 7f6730ea3640 1 -- 172.21.15.148:0/590190099 >> 172.21.15.148:0/590190099 conn(0x7f672c0f77a0 msgr2=0x7f672c103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:36:45.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:45.679+0000 7f6730ea3640 1 -- 172.21.15.148:0/590190099 shutdown_connections 2024-07-03T15:36:45.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:45.679+0000 7f6730ea3640 1 -- 172.21.15.148:0/590190099 wait complete. 2024-07-03T15:36:46.620 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:46 smithi028 ceph-mon[29148]: from='client.? 172.21.15.148:0/590190099' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-03T15:36:47.228 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-07-03T15:36:47.229 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph mon dump -f json 2024-07-03T15:36:47.471 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-03T15:36:47.518 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-03T15:36:47.620 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:47 smithi028 ceph-mon[29148]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:36:48.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:48.872+0000 7f970d4f8640 1 -- 172.21.15.148:0/569457706 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f97080faed0 msgr2=0x7f97080fb2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:48.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:48.872+0000 7f970d4f8640 1 --2- 172.21.15.148:0/569457706 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f97080faed0 0x7f97080fb2e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f96f8004850 tx=0x7f96f8009da0 comp rx=0 tx=0).stop 2024-07-03T15:36:48.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:48.872+0000 7f970d4f8640 1 -- 172.21.15.148:0/569457706 shutdown_connections 2024-07-03T15:36:48.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:48.872+0000 7f970d4f8640 1 --2- 172.21.15.148:0/569457706 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f97080faed0 0x7f97080fb2e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:48.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:48.872+0000 7f970d4f8640 1 -- 172.21.15.148:0/569457706 >> 172.21.15.148:0/569457706 conn(0x7f97080f6690 msgr2=0x7f97080f8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:36:48.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:48.872+0000 7f970d4f8640 1 -- 172.21.15.148:0/569457706 shutdown_connections 2024-07-03T15:36:48.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:48.872+0000 7f970d4f8640 1 -- 172.21.15.148:0/569457706 wait complete. 2024-07-03T15:36:48.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:48.873+0000 7f970d4f8640 1 Processor -- start 2024-07-03T15:36:48.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:48.873+0000 7f970d4f8640 1 -- start start 2024-07-03T15:36:48.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:48.873+0000 7f970d4f8640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f97080faed0 0x7f9708065b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:36:48.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:48.873+0000 7f970d4f8640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f970810f350 con 0x7f97080faed0 2024-07-03T15:36:48.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:48.873+0000 7f9706ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f97080faed0 0x7f9708065b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:36:48.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:48.873+0000 7f9706ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f97080faed0 0x7f9708065b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.148:46784/0 (socket says 172.21.15.148:46784) 2024-07-03T15:36:48.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:48.873+0000 7f9706ffd640 1 -- 172.21.15.148:0/3523049793 learned_addr learned my addr 172.21.15.148:0/3523049793 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-07-03T15:36:48.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:48.873+0000 7f9706ffd640 1 -- 172.21.15.148:0/3523049793 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f8005040 con 0x7f97080faed0 2024-07-03T15:36:48.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:48.874+0000 7f9706ffd640 1 --2- 172.21.15.148:0/3523049793 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f97080faed0 0x7f9708065b70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f96f8004820 tx=0x7f96f8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:36:48.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:48.874+0000 7f96ff7fe640 1 -- 172.21.15.148:0/3523049793 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f96f8032650 con 0x7f97080faed0 2024-07-03T15:36:48.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:48.874+0000 7f970d4f8640 1 -- 172.21.15.148:0/3523049793 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f970810f490 con 0x7f97080faed0 2024-07-03T15:36:48.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:48.874+0000 7f96ff7fe640 1 -- 172.21.15.148:0/3523049793 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f96f8032c80 con 0x7f97080faed0 2024-07-03T15:36:48.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:48.874+0000 7f96ff7fe640 1 -- 172.21.15.148:0/3523049793 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f96f803eda0 con 0x7f97080faed0 2024-07-03T15:36:48.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:48.874+0000 7f970d4f8640 1 -- 172.21.15.148:0/3523049793 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f970810f900 con 0x7f97080faed0 2024-07-03T15:36:48.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:48.875+0000 7f96ff7fe640 1 -- 172.21.15.148:0/3523049793 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 13) v1 ==== 42692+0+0 (secure 0 0 0) 0x7f96f8030020 con 0x7f97080faed0 2024-07-03T15:36:48.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:48.875+0000 7f970d4f8640 1 -- 172.21.15.148:0/3523049793 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f970805eb80 con 0x7f97080faed0 2024-07-03T15:36:48.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:48.875+0000 7f96ff7fe640 1 --2- 172.21.15.148:0/3523049793 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f96f40357b0 0x7f96f4037c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:36:48.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:48.875+0000 7f96ff7fe640 1 -- 172.21.15.148:0/3523049793 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f96f8067ad0 con 0x7f97080faed0 2024-07-03T15:36:48.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:48.875+0000 7f96fffff640 1 --2- 172.21.15.148:0/3523049793 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f96f40357b0 0x7f96f4037c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:36:48.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:48.876+0000 7f96fffff640 1 --2- 172.21.15.148:0/3523049793 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f96f40357b0 0x7f96f4037c60 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f96f0006fd0 tx=0x7f96f0006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:36:48.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:48.880+0000 7f96ff7fe640 1 -- 172.21.15.148:0/3523049793 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f96f80327b0 con 0x7f97080faed0 2024-07-03T15:36:49.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:49.106+0000 7f970d4f8640 1 -- 172.21.15.148:0/3523049793 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f970804b530 con 0x7f97080faed0 2024-07-03T15:36:49.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:49.107+0000 7f96ff7fe640 1 -- 172.21.15.148:0/3523049793 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f96f803c330 con 0x7f97080faed0 2024-07-03T15:36:49.110 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:36:49.110 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"a85e4be6-3951-11ef-bcab-c7b262605968","modified":"2024-07-03T15:34:53.167064Z","created":"2024-07-03T15:34:53.167064Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi028","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:3300","nonce":0},{"type":"v1","addr":"172.21.15.28:6789","nonce":0}]},"addr":"172.21.15.28:6789/0","public_addr":"172.21.15.28:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-07-03T15:36:49.110 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-07-03T15:36:49.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:49.109+0000 7f970d4f8640 1 -- 172.21.15.148:0/3523049793 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f96f40357b0 msgr2=0x7f96f4037c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:49.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:49.109+0000 7f970d4f8640 1 --2- 172.21.15.148:0/3523049793 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f96f40357b0 0x7f96f4037c60 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f96f0006fd0 tx=0x7f96f0006db0 comp rx=0 tx=0).stop 2024-07-03T15:36:49.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:49.109+0000 7f970d4f8640 1 -- 172.21.15.148:0/3523049793 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f97080faed0 msgr2=0x7f9708065b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:49.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:49.109+0000 7f970d4f8640 1 --2- 172.21.15.148:0/3523049793 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f97080faed0 0x7f9708065b70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f96f8004820 tx=0x7f96f8009da0 comp rx=0 tx=0).stop 2024-07-03T15:36:49.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:49.109+0000 7f970d4f8640 1 -- 172.21.15.148:0/3523049793 shutdown_connections 2024-07-03T15:36:49.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:49.109+0000 7f970d4f8640 1 --2- 172.21.15.148:0/3523049793 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f96f40357b0 0x7f96f4037c60 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:49.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:49.109+0000 7f970d4f8640 1 --2- 172.21.15.148:0/3523049793 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f97080faed0 0x7f9708065b70 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:49.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:49.109+0000 7f970d4f8640 1 -- 172.21.15.148:0/3523049793 >> 172.21.15.148:0/3523049793 conn(0x7f97080f6690 msgr2=0x7f97080f7260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:36:49.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:49.110+0000 7f970d4f8640 1 -- 172.21.15.148:0/3523049793 shutdown_connections 2024-07-03T15:36:49.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:49.110+0000 7f970d4f8640 1 -- 172.21.15.148:0/3523049793 wait complete. 2024-07-03T15:36:49.120 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:48 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:36:50.120 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:49 smithi028 ceph-mon[29148]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:36:50.120 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:49 smithi028 ceph-mon[29148]: from='client.? 172.21.15.148:0/3523049793' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-03T15:36:50.717 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-07-03T15:36:50.717 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph mon dump -f json 2024-07-03T15:36:50.961 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-03T15:36:51.007 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-03T15:36:51.870 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:51 smithi028 ceph-mon[29148]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:36:52.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:52.441+0000 7f52acae5640 1 -- 172.21.15.148:0/925589506 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52a80fc000 msgr2=0x7f52a80fc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:52.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:52.441+0000 7f52acae5640 1 --2- 172.21.15.148:0/925589506 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52a80fc000 0x7f52a80fc410 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f52a0009bd0 tx=0x7f52a0004d10 comp rx=0 tx=0).stop 2024-07-03T15:36:52.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:52.441+0000 7f52acae5640 1 -- 172.21.15.148:0/925589506 shutdown_connections 2024-07-03T15:36:52.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:52.441+0000 7f52acae5640 1 --2- 172.21.15.148:0/925589506 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52a80fc000 0x7f52a80fc410 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:52.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:52.441+0000 7f52acae5640 1 -- 172.21.15.148:0/925589506 >> 172.21.15.148:0/925589506 conn(0x7f52a80f77c0 msgr2=0x7f52a80f9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:36:52.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:52.441+0000 7f52acae5640 1 -- 172.21.15.148:0/925589506 shutdown_connections 2024-07-03T15:36:52.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:52.441+0000 7f52acae5640 1 -- 172.21.15.148:0/925589506 wait complete. 2024-07-03T15:36:52.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:52.442+0000 7f52acae5640 1 Processor -- start 2024-07-03T15:36:52.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:52.442+0000 7f52acae5640 1 -- start start 2024-07-03T15:36:52.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:52.442+0000 7f52acae5640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52a80fc000 0x7f52a810c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:36:52.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:52.442+0000 7f52acae5640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a810cb60 con 0x7f52a80fc000 2024-07-03T15:36:52.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:52.443+0000 7f52a659c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52a80fc000 0x7f52a810c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:36:52.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:52.443+0000 7f52a659c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52a80fc000 0x7f52a810c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.148:54074/0 (socket says 172.21.15.148:54074) 2024-07-03T15:36:52.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:52.443+0000 7f52a659c640 1 -- 172.21.15.148:0/2841114716 learned_addr learned my addr 172.21.15.148:0/2841114716 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-07-03T15:36:52.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:52.443+0000 7f52a659c640 1 -- 172.21.15.148:0/2841114716 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a00098b0 con 0x7f52a80fc000 2024-07-03T15:36:52.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:52.443+0000 7f52a659c640 1 --2- 172.21.15.148:0/2841114716 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52a80fc000 0x7f52a810c620 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f52a0009ba0 tx=0x7f52a0004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:36:52.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:52.443+0000 7f529effd640 1 -- 172.21.15.148:0/2841114716 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f52a00329b0 con 0x7f52a80fc000 2024-07-03T15:36:52.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:52.443+0000 7f529effd640 1 -- 172.21.15.148:0/2841114716 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f52a003d420 con 0x7f52a80fc000 2024-07-03T15:36:52.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:52.443+0000 7f52acae5640 1 -- 172.21.15.148:0/2841114716 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52a810cd60 con 0x7f52a80fc000 2024-07-03T15:36:52.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:52.443+0000 7f529effd640 1 -- 172.21.15.148:0/2841114716 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f52a0031590 con 0x7f52a80fc000 2024-07-03T15:36:52.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:52.443+0000 7f52acae5640 1 -- 172.21.15.148:0/2841114716 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52a810d1b0 con 0x7f52a80fc000 2024-07-03T15:36:52.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:52.444+0000 7f529effd640 1 -- 172.21.15.148:0/2841114716 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 13) v1 ==== 42692+0+0 (secure 0 0 0) 0x7f52a002f030 con 0x7f52a80fc000 2024-07-03T15:36:52.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:52.444+0000 7f52acae5640 1 -- 172.21.15.148:0/2841114716 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52a805eb80 con 0x7f52a80fc000 2024-07-03T15:36:52.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:52.444+0000 7f529effd640 1 --2- 172.21.15.148:0/2841114716 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f5294035760 0x7f5294037c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:36:52.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:52.444+0000 7f529effd640 1 -- 172.21.15.148:0/2841114716 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f52a0068da0 con 0x7f52a80fc000 2024-07-03T15:36:52.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:52.445+0000 7f52a5d9b640 1 --2- 172.21.15.148:0/2841114716 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f5294035760 0x7f5294037c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:36:52.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:52.445+0000 7f52a5d9b640 1 --2- 172.21.15.148:0/2841114716 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f5294035760 0x7f5294037c10 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5290006fd0 tx=0x7f5290006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:36:52.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:52.449+0000 7f529effd640 1 -- 172.21.15.148:0/2841114716 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f52a004e940 con 0x7f52a80fc000 2024-07-03T15:36:52.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:52.674+0000 7f52acae5640 1 -- 172.21.15.148:0/2841114716 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f52a8101da0 con 0x7f52a80fc000 2024-07-03T15:36:52.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:52.675+0000 7f529effd640 1 -- 172.21.15.148:0/2841114716 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f52a004e350 con 0x7f52a80fc000 2024-07-03T15:36:52.678 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:36:52.679 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"a85e4be6-3951-11ef-bcab-c7b262605968","modified":"2024-07-03T15:34:53.167064Z","created":"2024-07-03T15:34:53.167064Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi028","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:3300","nonce":0},{"type":"v1","addr":"172.21.15.28:6789","nonce":0}]},"addr":"172.21.15.28:6789/0","public_addr":"172.21.15.28:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-07-03T15:36:52.679 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-07-03T15:36:52.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:52.677+0000 7f52acae5640 1 -- 172.21.15.148:0/2841114716 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f5294035760 msgr2=0x7f5294037c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:52.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:52.677+0000 7f52acae5640 1 --2- 172.21.15.148:0/2841114716 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f5294035760 0x7f5294037c10 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5290006fd0 tx=0x7f5290006db0 comp rx=0 tx=0).stop 2024-07-03T15:36:52.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:52.677+0000 7f52acae5640 1 -- 172.21.15.148:0/2841114716 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52a80fc000 msgr2=0x7f52a810c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:52.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:52.677+0000 7f52acae5640 1 --2- 172.21.15.148:0/2841114716 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52a80fc000 0x7f52a810c620 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f52a0009ba0 tx=0x7f52a0004d10 comp rx=0 tx=0).stop 2024-07-03T15:36:52.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:52.677+0000 7f52acae5640 1 -- 172.21.15.148:0/2841114716 shutdown_connections 2024-07-03T15:36:52.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:52.677+0000 7f52acae5640 1 --2- 172.21.15.148:0/2841114716 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f5294035760 0x7f5294037c10 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:52.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:52.677+0000 7f52acae5640 1 --2- 172.21.15.148:0/2841114716 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52a80fc000 0x7f52a810c620 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:52.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:52.677+0000 7f52acae5640 1 -- 172.21.15.148:0/2841114716 >> 172.21.15.148:0/2841114716 conn(0x7f52a80f77c0 msgr2=0x7f52a81031a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:36:52.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:52.677+0000 7f52acae5640 1 -- 172.21.15.148:0/2841114716 shutdown_connections 2024-07-03T15:36:52.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:52.677+0000 7f52acae5640 1 -- 172.21.15.148:0/2841114716 wait complete. 2024-07-03T15:36:52.891 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:52 smithi028 ceph-mon[29148]: from='client.? 172.21.15.148:0/2841114716' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-03T15:36:54.073 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:53 smithi028 ceph-mon[29148]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:36:54.254 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-07-03T15:36:54.255 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph mon dump -f json 2024-07-03T15:36:54.502 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-03T15:36:54.550 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-03T15:36:55.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:55.905+0000 7f337342a640 1 -- 172.21.15.148:0/1223738841 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f336c0faee0 msgr2=0x7f336c0fb2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:55.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:55.905+0000 7f337342a640 1 --2- 172.21.15.148:0/1223738841 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f336c0faee0 0x7f336c0fb2f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3368004850 tx=0x7f3368009da0 comp rx=0 tx=0).stop 2024-07-03T15:36:55.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:55.905+0000 7f337342a640 1 -- 172.21.15.148:0/1223738841 shutdown_connections 2024-07-03T15:36:55.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:55.905+0000 7f337342a640 1 --2- 172.21.15.148:0/1223738841 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f336c0faee0 0x7f336c0fb2f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:55.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:55.905+0000 7f337342a640 1 -- 172.21.15.148:0/1223738841 >> 172.21.15.148:0/1223738841 conn(0x7f336c0f66a0 msgr2=0x7f336c0f8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:36:55.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:55.905+0000 7f337342a640 1 -- 172.21.15.148:0/1223738841 shutdown_connections 2024-07-03T15:36:55.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:55.905+0000 7f337342a640 1 -- 172.21.15.148:0/1223738841 wait complete. 2024-07-03T15:36:55.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:55.906+0000 7f337342a640 1 Processor -- start 2024-07-03T15:36:55.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:55.906+0000 7f337342a640 1 -- start start 2024-07-03T15:36:55.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:55.906+0000 7f337342a640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f336c0faee0 0x7f336c10c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:36:55.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:55.906+0000 7f337342a640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336c10cb00 con 0x7f336c0faee0 2024-07-03T15:36:55.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:55.906+0000 7f33711c6640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f336c0faee0 0x7f336c10c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:36:55.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:55.906+0000 7f33711c6640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f336c0faee0 0x7f336c10c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.148:54096/0 (socket says 172.21.15.148:54096) 2024-07-03T15:36:55.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:55.906+0000 7f33711c6640 1 -- 172.21.15.148:0/3656626934 learned_addr learned my addr 172.21.15.148:0/3656626934 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-07-03T15:36:55.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:55.906+0000 7f33711c6640 1 -- 172.21.15.148:0/3656626934 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3368005040 con 0x7f336c0faee0 2024-07-03T15:36:55.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:55.907+0000 7f33711c6640 1 --2- 172.21.15.148:0/3656626934 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f336c0faee0 0x7f336c10c5c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3368004820 tx=0x7f3368009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:36:55.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:55.907+0000 7f3361ffb640 1 -- 172.21.15.148:0/3656626934 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3368032650 con 0x7f336c0faee0 2024-07-03T15:36:55.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:55.907+0000 7f337342a640 1 -- 172.21.15.148:0/3656626934 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f336c10cd00 con 0x7f336c0faee0 2024-07-03T15:36:55.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:55.907+0000 7f3361ffb640 1 -- 172.21.15.148:0/3656626934 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3368032c80 con 0x7f336c0faee0 2024-07-03T15:36:55.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:55.907+0000 7f3361ffb640 1 -- 172.21.15.148:0/3656626934 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f336803eda0 con 0x7f336c0faee0 2024-07-03T15:36:55.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:55.907+0000 7f337342a640 1 -- 172.21.15.148:0/3656626934 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f336c10d890 con 0x7f336c0faee0 2024-07-03T15:36:55.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:55.908+0000 7f3361ffb640 1 -- 172.21.15.148:0/3656626934 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 14) v1 ==== 42706+0+0 (secure 0 0 0) 0x7f3368030020 con 0x7f336c0faee0 2024-07-03T15:36:55.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:55.908+0000 7f3361ffb640 1 --2- 172.21.15.148:0/3656626934 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f3358035800 0x7f3358037cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:36:55.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:55.908+0000 7f337342a640 1 -- 172.21.15.148:0/3656626934 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f336c101450 con 0x7f336c0faee0 2024-07-03T15:36:55.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:55.908+0000 7f33709c5640 1 -- 172.21.15.148:0/3656626934 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f3358035800 msgr2=0x7f3358037cb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.28:6800/577887491 2024-07-03T15:36:55.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:55.908+0000 7f3361ffb640 1 -- 172.21.15.148:0/3656626934 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f3368067d10 con 0x7f336c0faee0 2024-07-03T15:36:55.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:55.908+0000 7f33709c5640 1 --2- 172.21.15.148:0/3656626934 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f3358035800 0x7f3358037cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-03T15:36:55.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:55.913+0000 7f3361ffb640 1 -- 172.21.15.148:0/3656626934 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f336803c920 con 0x7f336c0faee0 2024-07-03T15:36:55.938 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:55 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:36:55.938 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:55 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:36:55.938 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:55 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:36:55.938 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:55 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-07-03T15:36:55.939 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:55 smithi028 ceph-mon[29148]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:36:56.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:56.109+0000 7f33709c5640 1 -- 172.21.15.148:0/3656626934 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f3358035800 msgr2=0x7f3358037cb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.28:6800/577887491 2024-07-03T15:36:56.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:56.109+0000 7f33709c5640 1 --2- 172.21.15.148:0/3656626934 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f3358035800 0x7f3358037cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-03T15:36:56.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:56.146+0000 7f337342a640 1 -- 172.21.15.148:0/3656626934 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f336c10dad0 con 0x7f336c0faee0 2024-07-03T15:36:56.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:56.146+0000 7f3361ffb640 1 -- 172.21.15.148:0/3656626934 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f3368032e60 con 0x7f336c0faee0 2024-07-03T15:36:56.149 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:36:56.149 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"a85e4be6-3951-11ef-bcab-c7b262605968","modified":"2024-07-03T15:34:53.167064Z","created":"2024-07-03T15:34:53.167064Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi028","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:3300","nonce":0},{"type":"v1","addr":"172.21.15.28:6789","nonce":0}]},"addr":"172.21.15.28:6789/0","public_addr":"172.21.15.28:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-07-03T15:36:56.149 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-07-03T15:36:56.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:56.148+0000 7f337342a640 1 -- 172.21.15.148:0/3656626934 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f3358035800 msgr2=0x7f3358037cb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-03T15:36:56.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:56.148+0000 7f337342a640 1 --2- 172.21.15.148:0/3656626934 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f3358035800 0x7f3358037cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:56.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:56.148+0000 7f337342a640 1 -- 172.21.15.148:0/3656626934 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f336c0faee0 msgr2=0x7f336c10c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:56.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:56.148+0000 7f337342a640 1 --2- 172.21.15.148:0/3656626934 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f336c0faee0 0x7f336c10c5c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3368004820 tx=0x7f3368009da0 comp rx=0 tx=0).stop 2024-07-03T15:36:56.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:56.148+0000 7f337342a640 1 -- 172.21.15.148:0/3656626934 shutdown_connections 2024-07-03T15:36:56.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:56.148+0000 7f337342a640 1 --2- 172.21.15.148:0/3656626934 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f3358035800 0x7f3358037cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:56.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:56.148+0000 7f337342a640 1 --2- 172.21.15.148:0/3656626934 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f336c0faee0 0x7f336c10c5c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:56.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:56.148+0000 7f337342a640 1 -- 172.21.15.148:0/3656626934 >> 172.21.15.148:0/3656626934 conn(0x7f336c0f66a0 msgr2=0x7f336c0652f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:36:56.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:56.148+0000 7f337342a640 1 -- 172.21.15.148:0/3656626934 shutdown_connections 2024-07-03T15:36:56.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:56.148+0000 7f337342a640 1 -- 172.21.15.148:0/3656626934 wait complete. 2024-07-03T15:36:57.120 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:56 smithi028 ceph-mon[29148]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-07-03T15:36:57.120 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:56 smithi028 ceph-mon[29148]: mgrmap e14: smithi028.vtqrtz(active, since 62s) 2024-07-03T15:36:57.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:56 smithi028 ceph-mon[29148]: from='client.? 172.21.15.148:0/3656626934' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-03T15:36:57.676 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-07-03T15:36:57.676 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph mon dump -f json 2024-07-03T15:36:57.922 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-03T15:36:57.969 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-03T15:36:59.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:59.326+0000 7f579789b640 1 -- 172.21.15.148:0/2374158654 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f57900fbfe0 msgr2=0x7f57900fc3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:59.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:59.326+0000 7f579789b640 1 --2- 172.21.15.148:0/2374158654 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f57900fbfe0 0x7f57900fc3f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f578c004850 tx=0x7f578c009da0 comp rx=0 tx=0).stop 2024-07-03T15:36:59.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:59.326+0000 7f579789b640 1 -- 172.21.15.148:0/2374158654 shutdown_connections 2024-07-03T15:36:59.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:59.326+0000 7f579789b640 1 --2- 172.21.15.148:0/2374158654 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f57900fbfe0 0x7f57900fc3f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:59.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:59.326+0000 7f579789b640 1 -- 172.21.15.148:0/2374158654 >> 172.21.15.148:0/2374158654 conn(0x7f57900f77c0 msgr2=0x7f57900f9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:36:59.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:59.326+0000 7f579789b640 1 -- 172.21.15.148:0/2374158654 shutdown_connections 2024-07-03T15:36:59.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:59.326+0000 7f579789b640 1 -- 172.21.15.148:0/2374158654 wait complete. 2024-07-03T15:36:59.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:59.326+0000 7f579789b640 1 Processor -- start 2024-07-03T15:36:59.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:59.326+0000 7f579789b640 1 -- start start 2024-07-03T15:36:59.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:59.326+0000 7f579789b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f57900fbfe0 0x7f57900712a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:36:59.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:59.326+0000 7f579789b640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57900717e0 con 0x7f57900fbfe0 2024-07-03T15:36:59.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:59.327+0000 7f5795637640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f57900fbfe0 0x7f57900712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:36:59.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:59.327+0000 7f5795637640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f57900fbfe0 0x7f57900712a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.148:54120/0 (socket says 172.21.15.148:54120) 2024-07-03T15:36:59.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:59.327+0000 7f5795637640 1 -- 172.21.15.148:0/781075407 learned_addr learned my addr 172.21.15.148:0/781075407 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-07-03T15:36:59.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:59.327+0000 7f5795637640 1 -- 172.21.15.148:0/781075407 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f578c005040 con 0x7f57900fbfe0 2024-07-03T15:36:59.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:59.327+0000 7f5795637640 1 --2- 172.21.15.148:0/781075407 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f57900fbfe0 0x7f57900712a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f578c004820 tx=0x7f578c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:36:59.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:59.327+0000 7f5785ffb640 1 -- 172.21.15.148:0/781075407 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f578c032650 con 0x7f57900fbfe0 2024-07-03T15:36:59.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:59.327+0000 7f579789b640 1 -- 172.21.15.148:0/781075407 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f579006f870 con 0x7f57900fbfe0 2024-07-03T15:36:59.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:59.327+0000 7f5785ffb640 1 -- 172.21.15.148:0/781075407 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f578c032c80 con 0x7f57900fbfe0 2024-07-03T15:36:59.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:59.327+0000 7f5785ffb640 1 -- 172.21.15.148:0/781075407 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f578c03eda0 con 0x7f57900fbfe0 2024-07-03T15:36:59.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:59.328+0000 7f579789b640 1 -- 172.21.15.148:0/781075407 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f579006fd40 con 0x7f57900fbfe0 2024-07-03T15:36:59.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:59.328+0000 7f5785ffb640 1 -- 172.21.15.148:0/781075407 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 14) v1 ==== 42706+0+0 (secure 0 0 0) 0x7f578c030020 con 0x7f57900fbfe0 2024-07-03T15:36:59.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:59.328+0000 7f579789b640 1 -- 172.21.15.148:0/781075407 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5790102550 con 0x7f57900fbfe0 2024-07-03T15:36:59.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:59.329+0000 7f5785ffb640 1 --2- 172.21.15.148:0/781075407 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f577c035800 0x7f577c037cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:36:59.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:59.329+0000 7f5785ffb640 1 -- 172.21.15.148:0/781075407 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f578c067ba0 con 0x7f57900fbfe0 2024-07-03T15:36:59.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:59.329+0000 7f5794e36640 1 -- 172.21.15.148:0/781075407 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f577c035800 msgr2=0x7f577c037cb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.28:6800/577887491 2024-07-03T15:36:59.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:59.329+0000 7f5794e36640 1 --2- 172.21.15.148:0/781075407 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f577c035800 0x7f577c037cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-03T15:36:59.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:59.333+0000 7f5785ffb640 1 -- 172.21.15.148:0/781075407 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f578c03c510 con 0x7f57900fbfe0 2024-07-03T15:36:59.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:59.530+0000 7f5794e36640 1 -- 172.21.15.148:0/781075407 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f577c035800 msgr2=0x7f577c037cb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.28:6800/577887491 2024-07-03T15:36:59.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:59.530+0000 7f5794e36640 1 --2- 172.21.15.148:0/781075407 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f577c035800 0x7f577c037cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-03T15:36:59.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:59.563+0000 7f579789b640 1 -- 172.21.15.148:0/781075407 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f579005eb80 con 0x7f57900fbfe0 2024-07-03T15:36:59.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:59.564+0000 7f5785ffb640 1 -- 172.21.15.148:0/781075407 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f578c03c330 con 0x7f57900fbfe0 2024-07-03T15:36:59.566 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:36:59.567 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"a85e4be6-3951-11ef-bcab-c7b262605968","modified":"2024-07-03T15:34:53.167064Z","created":"2024-07-03T15:34:53.167064Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi028","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:3300","nonce":0},{"type":"v1","addr":"172.21.15.28:6789","nonce":0}]},"addr":"172.21.15.28:6789/0","public_addr":"172.21.15.28:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-07-03T15:36:59.567 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-07-03T15:36:59.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:59.566+0000 7f579789b640 1 -- 172.21.15.148:0/781075407 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f577c035800 msgr2=0x7f577c037cb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-03T15:36:59.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:59.566+0000 7f579789b640 1 --2- 172.21.15.148:0/781075407 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f577c035800 0x7f577c037cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:59.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:59.566+0000 7f579789b640 1 -- 172.21.15.148:0/781075407 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f57900fbfe0 msgr2=0x7f57900712a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:36:59.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:59.566+0000 7f579789b640 1 --2- 172.21.15.148:0/781075407 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f57900fbfe0 0x7f57900712a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f578c004820 tx=0x7f578c009da0 comp rx=0 tx=0).stop 2024-07-03T15:36:59.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:59.566+0000 7f579789b640 1 -- 172.21.15.148:0/781075407 shutdown_connections 2024-07-03T15:36:59.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:59.566+0000 7f579789b640 1 --2- 172.21.15.148:0/781075407 >> [v2:172.21.15.28:6800/577887491,v1:172.21.15.28:6801/577887491] conn(0x7f577c035800 0x7f577c037cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:59.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:59.566+0000 7f579789b640 1 --2- 172.21.15.148:0/781075407 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f57900fbfe0 0x7f57900712a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:36:59.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:59.566+0000 7f579789b640 1 -- 172.21.15.148:0/781075407 >> 172.21.15.148:0/781075407 conn(0x7f57900f77c0 msgr2=0x7f5790103180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:36:59.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:59.566+0000 7f579789b640 1 -- 172.21.15.148:0/781075407 shutdown_connections 2024-07-03T15:36:59.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:36:59.566+0000 7f579789b640 1 -- 172.21.15.148:0/781075407 wait complete. 2024-07-03T15:36:59.870 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:36:59 smithi028 ceph-mon[29148]: from='client.? 172.21.15.148:0/781075407' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-03T15:37:00.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:00 smithi028 ceph-mon[29148]: Active manager daemon smithi028.vtqrtz restarted 2024-07-03T15:37:00.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:00 smithi028 ceph-mon[29148]: Activating manager daemon smithi028.vtqrtz 2024-07-03T15:37:00.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:00 smithi028 ceph-mon[29148]: osdmap e5: 0 total, 0 up, 0 in 2024-07-03T15:37:00.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:00 smithi028 ceph-mon[29148]: mgrmap e15: smithi028.vtqrtz(active, starting, since 0.00288669s) 2024-07-03T15:37:00.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:00 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi028"}]: dispatch 2024-07-03T15:37:00.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:00 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mgr metadata", "who": "smithi028.vtqrtz", "id": "smithi028.vtqrtz"}]: dispatch 2024-07-03T15:37:00.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:00 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-03T15:37:00.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:00 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-03T15:37:00.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:00 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-03T15:37:00.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:00 smithi028 ceph-mon[29148]: Manager daemon smithi028.vtqrtz is now available 2024-07-03T15:37:00.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:00 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:37:00.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:00 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:37:00.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:00 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:00.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:00 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:37:00.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:00 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi028.vtqrtz/mirror_snapshot_schedule"}]: dispatch 2024-07-03T15:37:00.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:00 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi028.vtqrtz/trash_purge_schedule"}]: dispatch 2024-07-03T15:37:01.146 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-07-03T15:37:01.146 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph mon dump -f json 2024-07-03T15:37:01.395 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-03T15:37:01.444 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-03T15:37:02.319 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:02 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:02.319 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:02 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:02.319 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:02 smithi028 ceph-mon[29148]: mgrmap e16: smithi028.vtqrtz(active, since 1.00618s) 2024-07-03T15:37:02.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:02.872+0000 7f79af419640 1 -- 172.21.15.148:0/2252453459 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f79a80f9c70 msgr2=0x7f79a80fa080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:37:02.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:02.872+0000 7f79af419640 1 --2- 172.21.15.148:0/2252453459 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f79a80f9c70 0x7f79a80fa080 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f799c004850 tx=0x7f799c009da0 comp rx=0 tx=0).stop 2024-07-03T15:37:02.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:02.872+0000 7f79af419640 1 -- 172.21.15.148:0/2252453459 shutdown_connections 2024-07-03T15:37:02.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:02.872+0000 7f79af419640 1 --2- 172.21.15.148:0/2252453459 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f79a80f9c70 0x7f79a80fa080 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:37:02.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:02.872+0000 7f79af419640 1 -- 172.21.15.148:0/2252453459 >> 172.21.15.148:0/2252453459 conn(0x7f79a80f59e0 msgr2=0x7f79a80f7e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:37:02.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:02.872+0000 7f79af419640 1 -- 172.21.15.148:0/2252453459 shutdown_connections 2024-07-03T15:37:02.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:02.872+0000 7f79af419640 1 -- 172.21.15.148:0/2252453459 wait complete. 2024-07-03T15:37:02.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:02.873+0000 7f79af419640 1 Processor -- start 2024-07-03T15:37:02.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:02.873+0000 7f79af419640 1 -- start start 2024-07-03T15:37:02.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:02.873+0000 7f79af419640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f79a80f9c70 0x7f79a810c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:37:02.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:02.873+0000 7f79af419640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a810cb80 con 0x7f79a80f9c70 2024-07-03T15:37:02.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:02.873+0000 7f79ad1b5640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f79a80f9c70 0x7f79a810c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:37:02.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:02.873+0000 7f79ad1b5640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f79a80f9c70 0x7f79a810c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.148:38166/0 (socket says 172.21.15.148:38166) 2024-07-03T15:37:02.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:02.873+0000 7f79ad1b5640 1 -- 172.21.15.148:0/2556223766 learned_addr learned my addr 172.21.15.148:0/2556223766 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-07-03T15:37:02.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:02.873+0000 7f79ad1b5640 1 -- 172.21.15.148:0/2556223766 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f799c005040 con 0x7f79a80f9c70 2024-07-03T15:37:02.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:02.874+0000 7f79ad1b5640 1 --2- 172.21.15.148:0/2556223766 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f79a80f9c70 0x7f79a810c640 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f799c004820 tx=0x7f799c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:37:02.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:02.874+0000 7f7999ffb640 1 -- 172.21.15.148:0/2556223766 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f799c032650 con 0x7f79a80f9c70 2024-07-03T15:37:02.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:02.874+0000 7f79af419640 1 -- 172.21.15.148:0/2556223766 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79a810cd80 con 0x7f79a80f9c70 2024-07-03T15:37:02.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:02.874+0000 7f7999ffb640 1 -- 172.21.15.148:0/2556223766 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f799c032c80 con 0x7f79a80f9c70 2024-07-03T15:37:02.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:02.874+0000 7f7999ffb640 1 -- 172.21.15.148:0/2556223766 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f799c033d60 con 0x7f79a80f9c70 2024-07-03T15:37:02.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:02.874+0000 7f79af419640 1 -- 172.21.15.148:0/2556223766 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79a810d910 con 0x7f79a80f9c70 2024-07-03T15:37:02.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:02.875+0000 7f79af419640 1 -- 172.21.15.148:0/2556223766 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79a8102550 con 0x7f79a80f9c70 2024-07-03T15:37:02.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:02.875+0000 7f7999ffb640 1 -- 172.21.15.148:0/2556223766 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 16) v1 ==== 42557+0+0 (secure 0 0 0) 0x7f799c030020 con 0x7f79a80f9c70 2024-07-03T15:37:02.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:02.875+0000 7f7999ffb640 1 --2- 172.21.15.148:0/2556223766 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f7994035640 0x7f7994037af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:37:02.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:02.875+0000 7f7999ffb640 1 -- 172.21.15.148:0/2556223766 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f799c067780 con 0x7f79a80f9c70 2024-07-03T15:37:02.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:02.876+0000 7f79ac9b4640 1 --2- 172.21.15.148:0/2556223766 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f7994035640 0x7f7994037af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:37:02.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:02.876+0000 7f79ac9b4640 1 --2- 172.21.15.148:0/2556223766 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f7994035640 0x7f7994037af0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f79a4006fd0 tx=0x7f79a4006db0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:37:02.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:02.882+0000 7f7999ffb640 1 -- 172.21.15.148:0/2556223766 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f799c032df0 con 0x7f79a80f9c70 2024-07-03T15:37:03.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:03.079+0000 7f7999ffb640 1 -- 172.21.15.148:0/2556223766 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mgrmap(e 17) v1 ==== 42749+0+0 (secure 0 0 0) 0x7f799c0453a0 con 0x7f79a80f9c70 2024-07-03T15:37:03.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:03.131+0000 7f79af419640 1 -- 172.21.15.148:0/2556223766 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f79a810e310 con 0x7f79a80f9c70 2024-07-03T15:37:03.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:03.131+0000 7f7999ffb640 1 -- 172.21.15.148:0/2556223766 <== mon.0 v2:172.21.15.28:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f799c0304a0 con 0x7f79a80f9c70 2024-07-03T15:37:03.133 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:37:03.134 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"a85e4be6-3951-11ef-bcab-c7b262605968","modified":"2024-07-03T15:34:53.167064Z","created":"2024-07-03T15:34:53.167064Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi028","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:3300","nonce":0},{"type":"v1","addr":"172.21.15.28:6789","nonce":0}]},"addr":"172.21.15.28:6789/0","public_addr":"172.21.15.28:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-07-03T15:37:03.134 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-07-03T15:37:03.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:03.133+0000 7f79af419640 1 -- 172.21.15.148:0/2556223766 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f7994035640 msgr2=0x7f7994037af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:37:03.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:03.133+0000 7f79af419640 1 --2- 172.21.15.148:0/2556223766 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f7994035640 0x7f7994037af0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f79a4006fd0 tx=0x7f79a4006db0 comp rx=0 tx=0).stop 2024-07-03T15:37:03.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:03.133+0000 7f79af419640 1 -- 172.21.15.148:0/2556223766 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f79a80f9c70 msgr2=0x7f79a810c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:37:03.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:03.133+0000 7f79af419640 1 --2- 172.21.15.148:0/2556223766 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f79a80f9c70 0x7f79a810c640 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f799c004820 tx=0x7f799c009da0 comp rx=0 tx=0).stop 2024-07-03T15:37:03.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:03.133+0000 7f79af419640 1 -- 172.21.15.148:0/2556223766 shutdown_connections 2024-07-03T15:37:03.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:03.133+0000 7f79af419640 1 --2- 172.21.15.148:0/2556223766 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f7994035640 0x7f7994037af0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:37:03.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:03.133+0000 7f79af419640 1 --2- 172.21.15.148:0/2556223766 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f79a80f9c70 0x7f79a810c640 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:37:03.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:03.133+0000 7f79af419640 1 -- 172.21.15.148:0/2556223766 >> 172.21.15.148:0/2556223766 conn(0x7f79a80f59e0 msgr2=0x7f79a8103180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:37:03.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:03.133+0000 7f79af419640 1 -- 172.21.15.148:0/2556223766 shutdown_connections 2024-07-03T15:37:03.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:03.133+0000 7f79af419640 1 -- 172.21.15.148:0/2556223766 wait complete. 2024-07-03T15:37:03.370 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:03 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:04.340 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:04 smithi028 ceph-mon[29148]: mgrmap e17: smithi028.vtqrtz(active, since 2s) 2024-07-03T15:37:04.340 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:04 smithi028 ceph-mon[29148]: from='client.? 172.21.15.148:0/2556223766' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-03T15:37:04.341 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:04 smithi028 ceph-mon[29148]: [03/Jul/2024:15:37:03] ENGINE Bus STARTING 2024-07-03T15:37:04.341 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:04 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:04.694 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-07-03T15:37:04.695 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph mon dump -f json 2024-07-03T15:37:04.942 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-03T15:37:04.998 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-03T15:37:06.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:05 smithi028 ceph-mon[29148]: [03/Jul/2024:15:37:03] ENGINE Serving on https://172.21.15.28:7150 2024-07-03T15:37:06.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:05 smithi028 ceph-mon[29148]: [03/Jul/2024:15:37:03] ENGINE Bus STARTED 2024-07-03T15:37:06.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:05 smithi028 ceph-mon[29148]: [03/Jul/2024:15:37:03] ENGINE Error in HTTPServer.serve 2024-07-03T15:37:06.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Traceback (most recent call last): 2024-07-03T15:37:06.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-03T15:37:06.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: self._connections.run(self.expiration_interval) 2024-07-03T15:37:06.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-03T15:37:06.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: self._run(expiration_interval) 2024-07-03T15:37:06.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-03T15:37:06.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-07-03T15:37:06.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-03T15:37:06.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-03T15:37:06.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-03T15:37:06.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: s = self.context.wrap_socket( 2024-07-03T15:37:06.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-03T15:37:06.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: return self.sslsocket_class._create( 2024-07-03T15:37:06.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-03T15:37:06.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: self.do_handshake() 2024-07-03T15:37:06.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-03T15:37:06.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: self._sslobj.do_handshake() 2024-07-03T15:37:06.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-03T15:37:06.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:05 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:06.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:05 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:06.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:05 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:06.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:05 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:06.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:05 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "who": "osd/host:smithi028", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:37:07.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:07.102+0000 7f3b7b720640 1 -- 172.21.15.148:0/3998711999 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b74065400 msgr2=0x7f3b740fee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:37:07.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:07.102+0000 7f3b7b720640 1 --2- 172.21.15.148:0/3998711999 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b74065400 0x7f3b740fee60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3b70004850 tx=0x7f3b70009da0 comp rx=0 tx=0).stop 2024-07-03T15:37:07.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:07.102+0000 7f3b7b720640 1 -- 172.21.15.148:0/3998711999 shutdown_connections 2024-07-03T15:37:07.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:07.102+0000 7f3b7b720640 1 --2- 172.21.15.148:0/3998711999 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b74065400 0x7f3b740fee60 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:37:07.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:07.102+0000 7f3b7b720640 1 -- 172.21.15.148:0/3998711999 >> 172.21.15.148:0/3998711999 conn(0x7f3b740f68d0 msgr2=0x7f3b740f8ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:37:07.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:07.102+0000 7f3b7b720640 1 -- 172.21.15.148:0/3998711999 shutdown_connections 2024-07-03T15:37:07.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:07.102+0000 7f3b7b720640 1 -- 172.21.15.148:0/3998711999 wait complete. 2024-07-03T15:37:07.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:07.103+0000 7f3b7b720640 1 Processor -- start 2024-07-03T15:37:07.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:07.103+0000 7f3b7b720640 1 -- start start 2024-07-03T15:37:07.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:07.103+0000 7f3b7b720640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b74065400 0x7f3b7410c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:37:07.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:07.103+0000 7f3b7b720640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7410cd90 con 0x7f3b74065400 2024-07-03T15:37:07.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:07.103+0000 7f3b794bc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b74065400 0x7f3b7410c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:37:07.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:07.103+0000 7f3b794bc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b74065400 0x7f3b7410c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.148:38198/0 (socket says 172.21.15.148:38198) 2024-07-03T15:37:07.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:07.103+0000 7f3b794bc640 1 -- 172.21.15.148:0/2892916235 learned_addr learned my addr 172.21.15.148:0/2892916235 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-07-03T15:37:07.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:07.103+0000 7f3b794bc640 1 -- 172.21.15.148:0/2892916235 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b70005040 con 0x7f3b74065400 2024-07-03T15:37:07.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:07.103+0000 7f3b794bc640 1 --2- 172.21.15.148:0/2892916235 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b74065400 0x7f3b7410c850 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3b70003080 tx=0x7f3b70009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:37:07.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:07.104+0000 7f3b69ffb640 1 -- 172.21.15.148:0/2892916235 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3b700326c0 con 0x7f3b74065400 2024-07-03T15:37:07.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:07.104+0000 7f3b69ffb640 1 -- 172.21.15.148:0/2892916235 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3b70032cf0 con 0x7f3b74065400 2024-07-03T15:37:07.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:07.104+0000 7f3b7b720640 1 -- 172.21.15.148:0/2892916235 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b7410cf90 con 0x7f3b74065400 2024-07-03T15:37:07.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:07.104+0000 7f3b69ffb640 1 -- 172.21.15.148:0/2892916235 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3b70033db0 con 0x7f3b74065400 2024-07-03T15:37:07.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:07.104+0000 7f3b7b720640 1 -- 172.21.15.148:0/2892916235 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b7410db20 con 0x7f3b74065400 2024-07-03T15:37:07.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:07.105+0000 7f3b69ffb640 1 -- 172.21.15.148:0/2892916235 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) v1 ==== 42749+0+0 (secure 0 0 0) 0x7f3b70030020 con 0x7f3b74065400 2024-07-03T15:37:07.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:07.105+0000 7f3b7b720640 1 -- 172.21.15.148:0/2892916235 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b74102770 con 0x7f3b74065400 2024-07-03T15:37:07.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:07.105+0000 7f3b69ffb640 1 --2- 172.21.15.148:0/2892916235 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3b600358f0 0x7f3b60037da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:37:07.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:07.105+0000 7f3b69ffb640 1 -- 172.21.15.148:0/2892916235 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f3b70067c10 con 0x7f3b74065400 2024-07-03T15:37:07.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:07.105+0000 7f3b78cbb640 1 --2- 172.21.15.148:0/2892916235 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3b600358f0 0x7f3b60037da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:37:07.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:07.106+0000 7f3b78cbb640 1 --2- 172.21.15.148:0/2892916235 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3b600358f0 0x7f3b60037da0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3b64006fd0 tx=0x7f3b64006db0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:37:07.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:07.110+0000 7f3b69ffb640 1 -- 172.21.15.148:0/2892916235 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3b7002c520 con 0x7f3b74065400 2024-07-03T15:37:07.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:07.352+0000 7f3b7b720640 1 -- 172.21.15.148:0/2892916235 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3b740008d0 con 0x7f3b74065400 2024-07-03T15:37:07.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:07.352+0000 7f3b69ffb640 1 -- 172.21.15.148:0/2892916235 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f3b70032e60 con 0x7f3b74065400 2024-07-03T15:37:07.355 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:37:07.355 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"a85e4be6-3951-11ef-bcab-c7b262605968","modified":"2024-07-03T15:34:53.167064Z","created":"2024-07-03T15:34:53.167064Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi028","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:3300","nonce":0},{"type":"v1","addr":"172.21.15.28:6789","nonce":0}]},"addr":"172.21.15.28:6789/0","public_addr":"172.21.15.28:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-07-03T15:37:07.355 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-07-03T15:37:07.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:07.354+0000 7f3b7b720640 1 -- 172.21.15.148:0/2892916235 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3b600358f0 msgr2=0x7f3b60037da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:37:07.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:07.354+0000 7f3b7b720640 1 --2- 172.21.15.148:0/2892916235 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3b600358f0 0x7f3b60037da0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3b64006fd0 tx=0x7f3b64006db0 comp rx=0 tx=0).stop 2024-07-03T15:37:07.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:07.354+0000 7f3b7b720640 1 -- 172.21.15.148:0/2892916235 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b74065400 msgr2=0x7f3b7410c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:37:07.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:07.354+0000 7f3b7b720640 1 --2- 172.21.15.148:0/2892916235 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b74065400 0x7f3b7410c850 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3b70003080 tx=0x7f3b70009da0 comp rx=0 tx=0).stop 2024-07-03T15:37:07.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:07.354+0000 7f3b7b720640 1 -- 172.21.15.148:0/2892916235 shutdown_connections 2024-07-03T15:37:07.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:07.354+0000 7f3b7b720640 1 --2- 172.21.15.148:0/2892916235 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3b600358f0 0x7f3b60037da0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:37:07.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:07.354+0000 7f3b7b720640 1 --2- 172.21.15.148:0/2892916235 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b74065400 0x7f3b7410c850 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:37:07.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:07.354+0000 7f3b7b720640 1 -- 172.21.15.148:0/2892916235 >> 172.21.15.148:0/2892916235 conn(0x7f3b740f68d0 msgr2=0x7f3b741033a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:37:07.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:07.354+0000 7f3b7b720640 1 -- 172.21.15.148:0/2892916235 shutdown_connections 2024-07-03T15:37:07.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:07.354+0000 7f3b7b720640 1 -- 172.21.15.148:0/2892916235 wait complete. 2024-07-03T15:37:08.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:07 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:08.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:07 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:08.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:07 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:08.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:07 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:08.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:07 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "who": "osd/host:smithi148", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:37:08.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:07 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:37:08.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:07 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:37:08.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:07 smithi028 ceph-mon[29148]: Updating smithi028:/etc/ceph/ceph.conf 2024-07-03T15:37:08.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:07 smithi028 ceph-mon[29148]: Updating smithi148:/etc/ceph/ceph.conf 2024-07-03T15:37:08.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:07 smithi028 ceph-mon[29148]: from='client.? 172.21.15.148:0/2892916235' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-03T15:37:08.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:07 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:08.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:07 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:08.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:07 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:08.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:07 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:08.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:07 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:08.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:07 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi148", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-07-03T15:37:08.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:07 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi148", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-07-03T15:37:08.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:07 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:37:08.951 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-07-03T15:37:08.952 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph mon dump -f json 2024-07-03T15:37:09.120 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:08 smithi028 ceph-mon[29148]: Updating smithi148:/var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-03T15:37:09.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:08 smithi028 ceph-mon[29148]: Updating smithi028:/var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-03T15:37:09.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:08 smithi028 ceph-mon[29148]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-07-03T15:37:09.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:08 smithi028 ceph-mon[29148]: Updating smithi028:/etc/ceph/ceph.client.admin.keyring 2024-07-03T15:37:09.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:08 smithi028 ceph-mon[29148]: Updating smithi148:/var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/config/ceph.client.admin.keyring 2024-07-03T15:37:09.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:08 smithi028 ceph-mon[29148]: Updating smithi028:/var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/config/ceph.client.admin.keyring 2024-07-03T15:37:09.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:08 smithi028 ceph-mon[29148]: Deploying daemon crash.smithi148 on smithi148 2024-07-03T15:37:09.211 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-03T15:37:10.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:10.810+0000 7f52c838f640 1 -- 172.21.15.148:0/2909651345 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52c006f8c0 msgr2=0x7f52c006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:37:10.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:10.810+0000 7f52c838f640 1 --2- 172.21.15.148:0/2909651345 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52c006f8c0 0x7f52c006fc90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f52bc004850 tx=0x7f52bc009da0 comp rx=0 tx=0).stop 2024-07-03T15:37:10.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:10.811+0000 7f52c838f640 1 -- 172.21.15.148:0/2909651345 shutdown_connections 2024-07-03T15:37:10.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:10.811+0000 7f52c838f640 1 --2- 172.21.15.148:0/2909651345 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52c006f8c0 0x7f52c006fc90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:37:10.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:10.811+0000 7f52c838f640 1 -- 172.21.15.148:0/2909651345 >> 172.21.15.148:0/2909651345 conn(0x7f52c00f7ba0 msgr2=0x7f52c00f9fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:37:10.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:10.811+0000 7f52c838f640 1 -- 172.21.15.148:0/2909651345 shutdown_connections 2024-07-03T15:37:10.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:10.811+0000 7f52c838f640 1 -- 172.21.15.148:0/2909651345 wait complete. 2024-07-03T15:37:10.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:10.811+0000 7f52c838f640 1 Processor -- start 2024-07-03T15:37:10.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:10.811+0000 7f52c838f640 1 -- start start 2024-07-03T15:37:10.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:10.811+0000 7f52c838f640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52c006f8c0 0x7f52c01148f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:37:10.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:10.811+0000 7f52c838f640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c0114e30 con 0x7f52c006f8c0 2024-07-03T15:37:10.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:10.812+0000 7f52c612b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52c006f8c0 0x7f52c01148f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:37:10.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:10.812+0000 7f52c612b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52c006f8c0 0x7f52c01148f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.148:37110/0 (socket says 172.21.15.148:37110) 2024-07-03T15:37:10.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:10.812+0000 7f52c612b640 1 -- 172.21.15.148:0/1031446823 learned_addr learned my addr 172.21.15.148:0/1031446823 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-07-03T15:37:10.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:10.812+0000 7f52c612b640 1 -- 172.21.15.148:0/1031446823 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52bc005040 con 0x7f52c006f8c0 2024-07-03T15:37:10.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:10.812+0000 7f52c612b640 1 --2- 172.21.15.148:0/1031446823 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52c006f8c0 0x7f52c01148f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f52bc000c00 tx=0x7f52bc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:37:10.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:10.812+0000 7f52b6ffd640 1 -- 172.21.15.148:0/1031446823 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f52bc033700 con 0x7f52c006f8c0 2024-07-03T15:37:10.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:10.812+0000 7f52c838f640 1 -- 172.21.15.148:0/1031446823 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52c0115030 con 0x7f52c006f8c0 2024-07-03T15:37:10.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:10.812+0000 7f52b6ffd640 1 -- 172.21.15.148:0/1031446823 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f52bc033d30 con 0x7f52c006f8c0 2024-07-03T15:37:10.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:10.812+0000 7f52b6ffd640 1 -- 172.21.15.148:0/1031446823 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f52bc03d390 con 0x7f52c006f8c0 2024-07-03T15:37:10.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:10.812+0000 7f52c838f640 1 -- 172.21.15.148:0/1031446823 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52c0115480 con 0x7f52c006f8c0 2024-07-03T15:37:10.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:10.813+0000 7f52b6ffd640 1 -- 172.21.15.148:0/1031446823 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) v1 ==== 42749+0+0 (secure 0 0 0) 0x7f52bc030020 con 0x7f52c006f8c0 2024-07-03T15:37:10.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:10.813+0000 7f52c838f640 1 -- 172.21.15.148:0/1031446823 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52c0071000 con 0x7f52c006f8c0 2024-07-03T15:37:10.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:10.813+0000 7f52b6ffd640 1 --2- 172.21.15.148:0/1031446823 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f52ac0358a0 0x7f52ac037d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:37:10.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:10.813+0000 7f52b6ffd640 1 -- 172.21.15.148:0/1031446823 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f52bc067c80 con 0x7f52c006f8c0 2024-07-03T15:37:10.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:10.814+0000 7f52c592a640 1 --2- 172.21.15.148:0/1031446823 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f52ac0358a0 0x7f52ac037d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:37:10.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:10.814+0000 7f52c592a640 1 --2- 172.21.15.148:0/1031446823 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f52ac0358a0 0x7f52ac037d50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f52b0006fd0 tx=0x7f52b0006db0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:37:10.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:10.819+0000 7f52b6ffd640 1 -- 172.21.15.148:0/1031446823 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f52bc03bb10 con 0x7f52c006f8c0 2024-07-03T15:37:11.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:11.058+0000 7f52c838f640 1 -- 172.21.15.148:0/1031446823 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f52c004b530 con 0x7f52c006f8c0 2024-07-03T15:37:11.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:11.059+0000 7f52b6ffd640 1 -- 172.21.15.148:0/1031446823 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f52bc03bb10 con 0x7f52c006f8c0 2024-07-03T15:37:11.061 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:37:11.061 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"a85e4be6-3951-11ef-bcab-c7b262605968","modified":"2024-07-03T15:34:53.167064Z","created":"2024-07-03T15:34:53.167064Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi028","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:3300","nonce":0},{"type":"v1","addr":"172.21.15.28:6789","nonce":0}]},"addr":"172.21.15.28:6789/0","public_addr":"172.21.15.28:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-07-03T15:37:11.062 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-07-03T15:37:11.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:11.061+0000 7f52c838f640 1 -- 172.21.15.148:0/1031446823 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f52ac0358a0 msgr2=0x7f52ac037d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:37:11.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:11.061+0000 7f52c838f640 1 --2- 172.21.15.148:0/1031446823 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f52ac0358a0 0x7f52ac037d50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f52b0006fd0 tx=0x7f52b0006db0 comp rx=0 tx=0).stop 2024-07-03T15:37:11.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:11.061+0000 7f52c838f640 1 -- 172.21.15.148:0/1031446823 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52c006f8c0 msgr2=0x7f52c01148f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:37:11.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:11.061+0000 7f52c838f640 1 --2- 172.21.15.148:0/1031446823 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52c006f8c0 0x7f52c01148f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f52bc000c00 tx=0x7f52bc009da0 comp rx=0 tx=0).stop 2024-07-03T15:37:11.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:11.061+0000 7f52c838f640 1 -- 172.21.15.148:0/1031446823 shutdown_connections 2024-07-03T15:37:11.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:11.061+0000 7f52c838f640 1 --2- 172.21.15.148:0/1031446823 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f52ac0358a0 0x7f52ac037d50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:37:11.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:11.061+0000 7f52c838f640 1 --2- 172.21.15.148:0/1031446823 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52c006f8c0 0x7f52c01148f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:37:11.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:11.061+0000 7f52c838f640 1 -- 172.21.15.148:0/1031446823 >> 172.21.15.148:0/1031446823 conn(0x7f52c00f7ba0 msgr2=0x7f52c0104ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:37:11.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:11.061+0000 7f52c838f640 1 -- 172.21.15.148:0/1031446823 shutdown_connections 2024-07-03T15:37:11.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:11.061+0000 7f52c838f640 1 -- 172.21.15.148:0/1031446823 wait complete. 2024-07-03T15:37:11.370 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:11 smithi028 ceph-mon[29148]: from='client.? 172.21.15.148:0/1031446823' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-03T15:37:12.870 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:12 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:12.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:12 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:12.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:12 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:12.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:12 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:12.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:12 smithi028 ceph-mon[29148]: Deploying daemon node-exporter.smithi148 on smithi148 2024-07-03T15:37:13.032 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-07-03T15:37:13.032 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph mon dump -f json 2024-07-03T15:37:13.286 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-03T15:37:16.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:16.078+0000 7fe76425e640 1 -- 172.21.15.148:0/76284393 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe75c101ff0 msgr2=0x7fe75c1023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:37:16.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:16.078+0000 7fe76425e640 1 --2- 172.21.15.148:0/76284393 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe75c101ff0 0x7fe75c1023c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe758004850 tx=0x7fe758009da0 comp rx=0 tx=0).stop 2024-07-03T15:37:16.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:16.078+0000 7fe76425e640 1 -- 172.21.15.148:0/76284393 shutdown_connections 2024-07-03T15:37:16.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:16.078+0000 7fe76425e640 1 --2- 172.21.15.148:0/76284393 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe75c101ff0 0x7fe75c1023c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:37:16.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:16.078+0000 7fe76425e640 1 -- 172.21.15.148:0/76284393 >> 172.21.15.148:0/76284393 conn(0x7fe75c0f7ba0 msgr2=0x7fe75c0f9fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:37:16.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:16.078+0000 7fe76425e640 1 -- 172.21.15.148:0/76284393 shutdown_connections 2024-07-03T15:37:16.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:16.078+0000 7fe76425e640 1 -- 172.21.15.148:0/76284393 wait complete. 2024-07-03T15:37:16.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:16.079+0000 7fe76425e640 1 Processor -- start 2024-07-03T15:37:16.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:16.079+0000 7fe76425e640 1 -- start start 2024-07-03T15:37:16.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:16.079+0000 7fe76425e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe75c101ff0 0x7fe75c112690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:37:16.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:16.079+0000 7fe76425e640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75c112bd0 con 0x7fe75c101ff0 2024-07-03T15:37:16.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:16.079+0000 7fe761ffa640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe75c101ff0 0x7fe75c112690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:37:16.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:16.079+0000 7fe761ffa640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe75c101ff0 0x7fe75c112690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.148:37128/0 (socket says 172.21.15.148:37128) 2024-07-03T15:37:16.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:16.079+0000 7fe761ffa640 1 -- 172.21.15.148:0/3330252447 learned_addr learned my addr 172.21.15.148:0/3330252447 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-07-03T15:37:16.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:16.080+0000 7fe761ffa640 1 -- 172.21.15.148:0/3330252447 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe758005040 con 0x7fe75c101ff0 2024-07-03T15:37:16.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:16.080+0000 7fe761ffa640 1 --2- 172.21.15.148:0/3330252447 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe75c101ff0 0x7fe75c112690 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe758004820 tx=0x7fe758009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:37:16.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:16.080+0000 7fe7527fc640 1 -- 172.21.15.148:0/3330252447 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe75802cf10 con 0x7fe75c101ff0 2024-07-03T15:37:16.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:16.080+0000 7fe76425e640 1 -- 172.21.15.148:0/3330252447 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe75c112dd0 con 0x7fe75c101ff0 2024-07-03T15:37:16.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:16.080+0000 7fe7527fc640 1 -- 172.21.15.148:0/3330252447 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe758033b00 con 0x7fe75c101ff0 2024-07-03T15:37:16.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:16.080+0000 7fe7527fc640 1 -- 172.21.15.148:0/3330252447 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe75803db70 con 0x7fe75c101ff0 2024-07-03T15:37:16.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:16.080+0000 7fe76425e640 1 -- 172.21.15.148:0/3330252447 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe75c1131c0 con 0x7fe75c101ff0 2024-07-03T15:37:16.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:16.081+0000 7fe7527fc640 1 -- 172.21.15.148:0/3330252447 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) v1 ==== 42749+0+0 (secure 0 0 0) 0x7fe758030020 con 0x7fe75c101ff0 2024-07-03T15:37:16.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:16.081+0000 7fe76425e640 1 -- 172.21.15.148:0/3330252447 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe75c0fd760 con 0x7fe75c101ff0 2024-07-03T15:37:16.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:16.081+0000 7fe7527fc640 1 --2- 172.21.15.148:0/3330252447 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fe7480358a0 0x7fe748037d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:37:16.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:16.081+0000 7fe7527fc640 1 -- 172.21.15.148:0/3330252447 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fe758068220 con 0x7fe75c101ff0 2024-07-03T15:37:16.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:16.082+0000 7fe7617f9640 1 --2- 172.21.15.148:0/3330252447 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fe7480358a0 0x7fe748037d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:37:16.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:16.082+0000 7fe7617f9640 1 --2- 172.21.15.148:0/3330252447 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fe7480358a0 0x7fe748037d50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe74c006fd0 tx=0x7fe74c006db0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:37:16.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:16.086+0000 7fe7527fc640 1 -- 172.21.15.148:0/3330252447 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe758045570 con 0x7fe75c101ff0 2024-07-03T15:37:16.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:16.323+0000 7fe76425e640 1 -- 172.21.15.148:0/3330252447 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe75c10dd00 con 0x7fe75c101ff0 2024-07-03T15:37:16.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:16.324+0000 7fe7527fc640 1 -- 172.21.15.148:0/3330252447 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fe758045390 con 0x7fe75c101ff0 2024-07-03T15:37:16.326 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:37:16.326 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"a85e4be6-3951-11ef-bcab-c7b262605968","modified":"2024-07-03T15:34:53.167064Z","created":"2024-07-03T15:34:53.167064Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi028","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:3300","nonce":0},{"type":"v1","addr":"172.21.15.28:6789","nonce":0}]},"addr":"172.21.15.28:6789/0","public_addr":"172.21.15.28:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-07-03T15:37:16.327 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-07-03T15:37:16.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:16.326+0000 7fe76425e640 1 -- 172.21.15.148:0/3330252447 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fe7480358a0 msgr2=0x7fe748037d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:37:16.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:16.326+0000 7fe76425e640 1 --2- 172.21.15.148:0/3330252447 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fe7480358a0 0x7fe748037d50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe74c006fd0 tx=0x7fe74c006db0 comp rx=0 tx=0).stop 2024-07-03T15:37:16.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:16.326+0000 7fe76425e640 1 -- 172.21.15.148:0/3330252447 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe75c101ff0 msgr2=0x7fe75c112690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:37:16.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:16.326+0000 7fe76425e640 1 --2- 172.21.15.148:0/3330252447 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe75c101ff0 0x7fe75c112690 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe758004820 tx=0x7fe758009da0 comp rx=0 tx=0).stop 2024-07-03T15:37:16.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:16.326+0000 7fe76425e640 1 -- 172.21.15.148:0/3330252447 shutdown_connections 2024-07-03T15:37:16.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:16.326+0000 7fe76425e640 1 --2- 172.21.15.148:0/3330252447 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fe7480358a0 0x7fe748037d50 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:37:16.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:16.326+0000 7fe76425e640 1 --2- 172.21.15.148:0/3330252447 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe75c101ff0 0x7fe75c112690 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:37:16.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:16.326+0000 7fe76425e640 1 -- 172.21.15.148:0/3330252447 >> 172.21.15.148:0/3330252447 conn(0x7fe75c0f7ba0 msgr2=0x7fe75c0f9280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:37:16.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:16.326+0000 7fe76425e640 1 -- 172.21.15.148:0/3330252447 shutdown_connections 2024-07-03T15:37:16.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:16.326+0000 7fe76425e640 1 -- 172.21.15.148:0/3330252447 wait complete. 2024-07-03T15:37:16.665 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:16 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:16.666 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:16 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:37:16.666 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:16 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:16.666 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:16 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:16.666 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:16 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:16.666 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:16 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:16.666 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:16 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi148.rjzsar", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-03T15:37:16.666 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:16 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi148.rjzsar", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-07-03T15:37:16.666 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:16 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-03T15:37:16.666 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:16 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:37:16.666 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:16 smithi028 ceph-mon[29148]: from='client.? 172.21.15.148:0/3330252447' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-03T15:37:17.870 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:17 smithi028 ceph-mon[29148]: Deploying daemon mgr.smithi148.rjzsar on smithi148 2024-07-03T15:37:17.915 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-07-03T15:37:17.915 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph mon dump -f json 2024-07-03T15:37:18.250 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-03T15:37:19.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:19.826+0000 7f537b20e640 1 -- 172.21.15.148:0/1725232306 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5374101ff0 msgr2=0x7f53741023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:37:19.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:19.826+0000 7f537b20e640 1 --2- 172.21.15.148:0/1725232306 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5374101ff0 0x7f53741023c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f536c004850 tx=0x7f536c009da0 comp rx=0 tx=0).stop 2024-07-03T15:37:19.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:19.826+0000 7f537b20e640 1 -- 172.21.15.148:0/1725232306 shutdown_connections 2024-07-03T15:37:19.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:19.826+0000 7f537b20e640 1 --2- 172.21.15.148:0/1725232306 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5374101ff0 0x7f53741023c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:37:19.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:19.826+0000 7f537b20e640 1 -- 172.21.15.148:0/1725232306 >> 172.21.15.148:0/1725232306 conn(0x7f53740f7ba0 msgr2=0x7f53740f9fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:37:19.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:19.826+0000 7f537b20e640 1 -- 172.21.15.148:0/1725232306 shutdown_connections 2024-07-03T15:37:19.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:19.826+0000 7f537b20e640 1 -- 172.21.15.148:0/1725232306 wait complete. 2024-07-03T15:37:19.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:19.826+0000 7f537b20e640 1 Processor -- start 2024-07-03T15:37:19.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:19.826+0000 7f537b20e640 1 -- start start 2024-07-03T15:37:19.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:19.827+0000 7f537b20e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5374101ff0 0x7f5374112690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:37:19.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:19.827+0000 7f537b20e640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5374112bd0 con 0x7f5374101ff0 2024-07-03T15:37:19.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:19.827+0000 7f5378faa640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5374101ff0 0x7f5374112690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:37:19.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:19.827+0000 7f5378faa640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5374101ff0 0x7f5374112690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.148:47150/0 (socket says 172.21.15.148:47150) 2024-07-03T15:37:19.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:19.827+0000 7f5378faa640 1 -- 172.21.15.148:0/1986850832 learned_addr learned my addr 172.21.15.148:0/1986850832 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-07-03T15:37:19.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:19.827+0000 7f5378faa640 1 -- 172.21.15.148:0/1986850832 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f536c005040 con 0x7f5374101ff0 2024-07-03T15:37:19.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:19.828+0000 7f5378faa640 1 --2- 172.21.15.148:0/1986850832 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5374101ff0 0x7f5374112690 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f536c004820 tx=0x7f536c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:37:19.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:19.828+0000 7f53717fa640 1 -- 172.21.15.148:0/1986850832 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f536c02cf10 con 0x7f5374101ff0 2024-07-03T15:37:19.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:19.828+0000 7f537b20e640 1 -- 172.21.15.148:0/1986850832 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5374112dd0 con 0x7f5374101ff0 2024-07-03T15:37:19.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:19.828+0000 7f53717fa640 1 -- 172.21.15.148:0/1986850832 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f536c033b00 con 0x7f5374101ff0 2024-07-03T15:37:19.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:19.828+0000 7f537b20e640 1 -- 172.21.15.148:0/1986850832 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53741131c0 con 0x7f5374101ff0 2024-07-03T15:37:19.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:19.828+0000 7f53717fa640 1 -- 172.21.15.148:0/1986850832 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f536c03db60 con 0x7f5374101ff0 2024-07-03T15:37:19.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:19.829+0000 7f53717fa640 1 -- 172.21.15.148:0/1986850832 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) v1 ==== 42749+0+0 (secure 0 0 0) 0x7f536c030020 con 0x7f5374101ff0 2024-07-03T15:37:19.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:19.829+0000 7f53717fa640 1 --2- 172.21.15.148:0/1986850832 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f53600358a0 0x7f5360037d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:37:19.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:19.829+0000 7f537b20e640 1 -- 172.21.15.148:0/1986850832 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53740fd760 con 0x7f5374101ff0 2024-07-03T15:37:19.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:19.829+0000 7f53717fa640 1 -- 172.21.15.148:0/1986850832 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f536c0680a0 con 0x7f5374101ff0 2024-07-03T15:37:19.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:19.829+0000 7f5373fff640 1 --2- 172.21.15.148:0/1986850832 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f53600358a0 0x7f5360037d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:37:19.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:19.830+0000 7f5373fff640 1 --2- 172.21.15.148:0/1986850832 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f53600358a0 0x7f5360037d50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f536400ad90 tx=0x7f5364009410 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:37:19.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:19.834+0000 7f53717fa640 1 -- 172.21.15.148:0/1986850832 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f536c045570 con 0x7f5374101ff0 2024-07-03T15:37:20.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:20.072+0000 7f537b20e640 1 -- 172.21.15.148:0/1986850832 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f537404b530 con 0x7f5374101ff0 2024-07-03T15:37:20.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:20.073+0000 7f53717fa640 1 -- 172.21.15.148:0/1986850832 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f536c045390 con 0x7f5374101ff0 2024-07-03T15:37:20.076 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:37:20.076 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"a85e4be6-3951-11ef-bcab-c7b262605968","modified":"2024-07-03T15:34:53.167064Z","created":"2024-07-03T15:34:53.167064Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi028","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:3300","nonce":0},{"type":"v1","addr":"172.21.15.28:6789","nonce":0}]},"addr":"172.21.15.28:6789/0","public_addr":"172.21.15.28:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-07-03T15:37:20.076 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-07-03T15:37:20.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:20.075+0000 7f537b20e640 1 -- 172.21.15.148:0/1986850832 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f53600358a0 msgr2=0x7f5360037d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:37:20.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:20.075+0000 7f537b20e640 1 --2- 172.21.15.148:0/1986850832 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f53600358a0 0x7f5360037d50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f536400ad90 tx=0x7f5364009410 comp rx=0 tx=0).stop 2024-07-03T15:37:20.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:20.075+0000 7f537b20e640 1 -- 172.21.15.148:0/1986850832 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5374101ff0 msgr2=0x7f5374112690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:37:20.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:20.075+0000 7f537b20e640 1 --2- 172.21.15.148:0/1986850832 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5374101ff0 0x7f5374112690 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f536c004820 tx=0x7f536c009da0 comp rx=0 tx=0).stop 2024-07-03T15:37:20.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:20.075+0000 7f537b20e640 1 -- 172.21.15.148:0/1986850832 shutdown_connections 2024-07-03T15:37:20.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:20.075+0000 7f537b20e640 1 --2- 172.21.15.148:0/1986850832 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f53600358a0 0x7f5360037d50 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:37:20.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:20.075+0000 7f537b20e640 1 --2- 172.21.15.148:0/1986850832 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5374101ff0 0x7f5374112690 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:37:20.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:20.075+0000 7f537b20e640 1 -- 172.21.15.148:0/1986850832 >> 172.21.15.148:0/1986850832 conn(0x7f53740f7ba0 msgr2=0x7f53740f9280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:37:20.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:20.076+0000 7f537b20e640 1 -- 172.21.15.148:0/1986850832 shutdown_connections 2024-07-03T15:37:20.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:20.076+0000 7f537b20e640 1 -- 172.21.15.148:0/1986850832 wait complete. 2024-07-03T15:37:20.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:20 smithi028 ceph-mon[29148]: from='client.? 172.21.15.148:0/1986850832' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-03T15:37:20.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:20 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:20.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:20 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:20.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:20 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:20.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:20 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:21.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:21 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-03T15:37:21.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:21 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:37:21.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:21 smithi028 ceph-mon[29148]: Deploying daemon mon.smithi148 on smithi148 2024-07-03T15:37:21.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:21 smithi028 ceph-mon[29148]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:21.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:21 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:21.678 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-07-03T15:37:21.679 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph mon dump -f json 2024-07-03T15:37:22.878 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi148/config 2024-07-03T15:37:23.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:23 smithi028 ceph-mon[29148]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:24.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:24.776+0000 7fc181d31640 1 -- 172.21.15.148:0/2133117920 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc17c0fd120 msgr2=0x7fc17c0fd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:37:24.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:24.776+0000 7fc181d31640 1 --2- 172.21.15.148:0/2133117920 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc17c0fd120 0x7fc17c0fd4f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc16c004850 tx=0x7fc16c009da0 comp rx=0 tx=0).stop 2024-07-03T15:37:24.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:24.776+0000 7fc181d31640 1 -- 172.21.15.148:0/2133117920 shutdown_connections 2024-07-03T15:37:24.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:24.776+0000 7fc181d31640 1 --2- 172.21.15.148:0/2133117920 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc17c0fd120 0x7fc17c0fd4f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:37:24.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:24.776+0000 7fc181d31640 1 -- 172.21.15.148:0/2133117920 >> 172.21.15.148:0/2133117920 conn(0x7fc17c0f6c20 msgr2=0x7fc17c0f9030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:37:24.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:24.776+0000 7fc181d31640 1 -- 172.21.15.148:0/2133117920 shutdown_connections 2024-07-03T15:37:24.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:24.776+0000 7fc181d31640 1 -- 172.21.15.148:0/2133117920 wait complete. 2024-07-03T15:37:24.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:24.776+0000 7fc181d31640 1 Processor -- start 2024-07-03T15:37:24.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:24.776+0000 7fc181d31640 1 -- start start 2024-07-03T15:37:24.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:24.777+0000 7fc181d31640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc17c0fd120 0x7fc17c1127e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:37:24.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:24.777+0000 7fc181d31640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17c112d20 con 0x7fc17c0fd120 2024-07-03T15:37:24.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:24.777+0000 7fc17b7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc17c0fd120 0x7fc17c1127e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:37:24.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:24.777+0000 7fc17b7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc17c0fd120 0x7fc17c1127e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.148:47190/0 (socket says 172.21.15.148:47190) 2024-07-03T15:37:24.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:24.777+0000 7fc17b7fe640 1 -- 172.21.15.148:0/3528583221 learned_addr learned my addr 172.21.15.148:0/3528583221 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-07-03T15:37:24.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:24.777+0000 7fc17b7fe640 1 -- 172.21.15.148:0/3528583221 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc16c005040 con 0x7fc17c0fd120 2024-07-03T15:37:24.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:24.777+0000 7fc17b7fe640 1 --2- 172.21.15.148:0/3528583221 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc17c0fd120 0x7fc17c1127e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc16c004820 tx=0x7fc16c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:37:24.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:24.778+0000 7fc163fff640 1 -- 172.21.15.148:0/3528583221 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc16c02cf10 con 0x7fc17c0fd120 2024-07-03T15:37:24.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:24.778+0000 7fc181d31640 1 -- 172.21.15.148:0/3528583221 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc17c112f20 con 0x7fc17c0fd120 2024-07-03T15:37:24.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:24.778+0000 7fc163fff640 1 -- 172.21.15.148:0/3528583221 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc16c033b00 con 0x7fc17c0fd120 2024-07-03T15:37:24.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:24.778+0000 7fc163fff640 1 -- 172.21.15.148:0/3528583221 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc16c03db70 con 0x7fc17c0fd120 2024-07-03T15:37:24.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:24.778+0000 7fc181d31640 1 -- 172.21.15.148:0/3528583221 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc17c113310 con 0x7fc17c0fd120 2024-07-03T15:37:24.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:24.779+0000 7fc163fff640 1 -- 172.21.15.148:0/3528583221 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) v1 ==== 42749+0+0 (secure 0 0 0) 0x7fc16c030020 con 0x7fc17c0fd120 2024-07-03T15:37:24.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:24.779+0000 7fc181d31640 1 -- 172.21.15.148:0/3528583221 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc17c065df0 con 0x7fc17c0fd120 2024-07-03T15:37:24.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:24.779+0000 7fc163fff640 1 --2- 172.21.15.148:0/3528583221 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fc1640358a0 0x7fc164037d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:37:24.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:24.779+0000 7fc163fff640 1 -- 172.21.15.148:0/3528583221 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fc16c068160 con 0x7fc17c0fd120 2024-07-03T15:37:24.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:24.779+0000 7fc17affd640 1 --2- 172.21.15.148:0/3528583221 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fc1640358a0 0x7fc164037d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:37:24.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:24.779+0000 7fc17affd640 1 --2- 172.21.15.148:0/3528583221 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fc1640358a0 0x7fc164037d50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc170006fd0 tx=0x7fc170006db0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:37:24.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:24.784+0000 7fc163fff640 1 -- 172.21.15.148:0/3528583221 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc16c045570 con 0x7fc17c0fd120 2024-07-03T15:37:24.907 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: load: jerasure load: lrc 2024-07-03T15:37:24.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:24.935+0000 7fc163fff640 1 -- 172.21.15.148:0/3528583221 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc16c045570 con 0x7fc17c0fd120 2024-07-03T15:37:25.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:25.032+0000 7fc181d31640 1 -- 172.21.15.148:0/3528583221 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc17c106540 con 0x7fc17c0fd120 2024-07-03T15:37:25.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: RocksDB version: 6.15.5 2024-07-03T15:37:25.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-07-03T15:37:25.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Compile date Jul 1 2024 2024-07-03T15:37:25.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: DB SUMMARY 2024-07-03T15:37:25.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: DB Session ID: 3M0MZK5V1XZEA3KW8QJS 2024-07-03T15:37:25.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: CURRENT file: CURRENT 2024-07-03T15:37:25.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: IDENTITY file: IDENTITY 2024-07-03T15:37:25.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2024-07-03T15:37:25.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi148/store.db dir, Total Num: 0, files: 2024-07-03T15:37:25.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi148/store.db: 000004.log size: 511 ; 2024-07-03T15:37:25.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.error_if_exists: 0 2024-07-03T15:37:25.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.create_if_missing: 0 2024-07-03T15:37:25.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.paranoid_checks: 1 2024-07-03T15:37:25.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-07-03T15:37:25.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.env: 0x55f9ca51d440 2024-07-03T15:37:25.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.fs: Posix File System 2024-07-03T15:37:25.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.info_log: 0x55f9cbbc1120 2024-07-03T15:37:25.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.max_file_opening_threads: 16 2024-07-03T15:37:25.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.statistics: (nil) 2024-07-03T15:37:25.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.use_fsync: 0 2024-07-03T15:37:25.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.max_log_file_size: 0 2024-07-03T15:37:25.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-07-03T15:37:25.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.log_file_time_to_roll: 0 2024-07-03T15:37:25.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.keep_log_file_num: 1000 2024-07-03T15:37:25.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.recycle_log_file_num: 0 2024-07-03T15:37:25.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.allow_fallocate: 1 2024-07-03T15:37:25.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.allow_mmap_reads: 0 2024-07-03T15:37:25.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.allow_mmap_writes: 0 2024-07-03T15:37:25.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.use_direct_reads: 0 2024-07-03T15:37:25.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-07-03T15:37:25.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.create_missing_column_families: 0 2024-07-03T15:37:25.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.db_log_dir: 2024-07-03T15:37:25.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi148/store.db 2024-07-03T15:37:25.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.table_cache_numshardbits: 6 2024-07-03T15:37:25.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.WAL_ttl_seconds: 0 2024-07-03T15:37:25.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.WAL_size_limit_MB: 0 2024-07-03T15:37:25.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-07-03T15:37:25.235 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-07-03T15:37:25.235 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.is_fd_close_on_exec: 1 2024-07-03T15:37:25.235 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.advise_random_on_open: 1 2024-07-03T15:37:25.235 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.db_write_buffer_size: 0 2024-07-03T15:37:25.235 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.write_buffer_manager: 0x55f9cbc7e840 2024-07-03T15:37:25.235 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-07-03T15:37:25.235 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-07-03T15:37:25.235 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-07-03T15:37:25.235 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.use_adaptive_mutex: 0 2024-07-03T15:37:25.235 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.rate_limiter: (nil) 2024-07-03T15:37:25.235 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-07-03T15:37:25.235 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.wal_recovery_mode: 2 2024-07-03T15:37:25.235 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.enable_thread_tracking: 0 2024-07-03T15:37:25.235 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.enable_pipelined_write: 0 2024-07-03T15:37:25.236 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.unordered_write: 0 2024-07-03T15:37:25.236 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-07-03T15:37:25.236 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-07-03T15:37:25.236 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-07-03T15:37:25.236 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-07-03T15:37:25.236 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.row_cache: None 2024-07-03T15:37:25.236 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.wal_filter: None 2024-07-03T15:37:25.236 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-07-03T15:37:25.236 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.allow_ingest_behind: 0 2024-07-03T15:37:25.236 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.preserve_deletes: 0 2024-07-03T15:37:25.236 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.two_write_queues: 0 2024-07-03T15:37:25.236 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.manual_wal_flush: 0 2024-07-03T15:37:25.236 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.atomic_flush: 0 2024-07-03T15:37:25.237 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-07-03T15:37:25.237 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.persist_stats_to_disk: 0 2024-07-03T15:37:25.237 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.write_dbid_to_manifest: 0 2024-07-03T15:37:25.237 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.log_readahead_size: 0 2024-07-03T15:37:25.237 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-07-03T15:37:25.237 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.best_efforts_recovery: 0 2024-07-03T15:37:25.237 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-07-03T15:37:25.237 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-07-03T15:37:25.237 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.allow_data_in_errors: 0 2024-07-03T15:37:25.237 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.db_host_id: __hostname__ 2024-07-03T15:37:25.237 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.max_background_jobs: 2 2024-07-03T15:37:25.237 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.max_background_compactions: -1 2024-07-03T15:37:25.237 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.max_subcompactions: 1 2024-07-03T15:37:25.237 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-07-03T15:37:25.237 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-07-03T15:37:25.239 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.delayed_write_rate : 16777216 2024-07-03T15:37:25.240 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.max_total_wal_size: 0 2024-07-03T15:37:25.240 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-07-03T15:37:25.240 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.stats_dump_period_sec: 600 2024-07-03T15:37:25.240 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.stats_persist_period_sec: 600 2024-07-03T15:37:25.240 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-07-03T15:37:25.240 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.max_open_files: -1 2024-07-03T15:37:25.240 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.bytes_per_sync: 0 2024-07-03T15:37:25.240 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.wal_bytes_per_sync: 0 2024-07-03T15:37:25.240 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.strict_bytes_per_sync: 0 2024-07-03T15:37:25.240 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.compaction_readahead_size: 0 2024-07-03T15:37:25.240 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.max_background_flushes: -1 2024-07-03T15:37:25.240 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Compression algorithms supported: 2024-07-03T15:37:25.240 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: kZSTD supported: 0 2024-07-03T15:37:25.241 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: kXpressCompression supported: 0 2024-07-03T15:37:25.241 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: kBZip2Compression supported: 0 2024-07-03T15:37:25.241 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-07-03T15:37:25.241 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: kLZ4Compression supported: 1 2024-07-03T15:37:25.241 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: kZlibCompression supported: 1 2024-07-03T15:37:25.241 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: kLZ4HCCompression supported: 1 2024-07-03T15:37:25.241 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: kSnappyCompression supported: 1 2024-07-03T15:37:25.241 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Fast CRC32 supported: Supported on x86 2024-07-03T15:37:25.241 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi148/store.db/MANIFEST-000003 2024-07-03T15:37:25.241 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-07-03T15:37:25.241 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-07-03T15:37:25.241 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.merge_operator: 2024-07-03T15:37:25.242 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.compaction_filter: None 2024-07-03T15:37:25.242 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.compaction_filter_factory: None 2024-07-03T15:37:25.242 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.sst_partitioner_factory: None 2024-07-03T15:37:25.242 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.memtable_factory: SkipListFactory 2024-07-03T15:37:25.242 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.table_factory: BlockBasedTable 2024-07-03T15:37:25.242 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f9cbb67a68) 2024-07-03T15:37:25.242 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: cache_index_and_filter_blocks: 1 2024-07-03T15:37:25.242 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-07-03T15:37:25.242 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-07-03T15:37:25.242 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: pin_top_level_index_and_filter: 1 2024-07-03T15:37:25.242 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: index_type: 0 2024-07-03T15:37:25.242 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: data_block_index_type: 0 2024-07-03T15:37:25.243 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: index_shortening: 1 2024-07-03T15:37:25.243 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: data_block_hash_table_util_ratio: 0.750000 2024-07-03T15:37:25.243 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: hash_index_allow_collision: 1 2024-07-03T15:37:25.243 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: checksum: 1 2024-07-03T15:37:25.243 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: no_block_cache: 0 2024-07-03T15:37:25.243 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: block_cache: 0x55f9cbb9d1f0 2024-07-03T15:37:25.243 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: block_cache_name: BinnedLRUCache 2024-07-03T15:37:25.243 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: block_cache_options: 2024-07-03T15:37:25.243 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: capacity : 536870912 2024-07-03T15:37:25.243 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: num_shard_bits : 4 2024-07-03T15:37:25.243 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: strict_capacity_limit : 0 2024-07-03T15:37:25.243 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: high_pri_pool_ratio: 0.000 2024-07-03T15:37:25.243 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: block_cache_compressed: (nil) 2024-07-03T15:37:25.244 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: persistent_cache: (nil) 2024-07-03T15:37:25.244 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: block_size: 4096 2024-07-03T15:37:25.244 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: block_size_deviation: 10 2024-07-03T15:37:25.244 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: block_restart_interval: 16 2024-07-03T15:37:25.244 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: index_block_restart_interval: 1 2024-07-03T15:37:25.244 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: metadata_block_size: 4096 2024-07-03T15:37:25.244 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: partition_filters: 0 2024-07-03T15:37:25.244 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: use_delta_encoding: 1 2024-07-03T15:37:25.244 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-07-03T15:37:25.244 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: whole_key_filtering: 1 2024-07-03T15:37:25.244 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: verify_compression: 0 2024-07-03T15:37:25.244 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: read_amp_bytes_per_bit: 0 2024-07-03T15:37:25.244 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: format_version: 4 2024-07-03T15:37:25.244 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: enable_index_compression: 1 2024-07-03T15:37:25.245 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: block_align: 0 2024-07-03T15:37:25.245 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.write_buffer_size: 33554432 2024-07-03T15:37:25.245 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.max_write_buffer_number: 2 2024-07-03T15:37:25.245 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.compression: NoCompression 2024-07-03T15:37:25.245 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.bottommost_compression: Disabled 2024-07-03T15:37:25.245 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.prefix_extractor: nullptr 2024-07-03T15:37:25.245 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-07-03T15:37:25.245 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.num_levels: 7 2024-07-03T15:37:25.245 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-07-03T15:37:25.245 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-07-03T15:37:25.245 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-07-03T15:37:25.245 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-07-03T15:37:25.245 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-07-03T15:37:25.245 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-07-03T15:37:25.246 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-07-03T15:37:25.246 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-07-03T15:37:25.246 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-07-03T15:37:25.246 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-07-03T15:37:25.246 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.compression_opts.window_bits: -14 2024-07-03T15:37:25.246 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.compression_opts.level: 32767 2024-07-03T15:37:25.246 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.compression_opts.strategy: 0 2024-07-03T15:37:25.246 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-07-03T15:37:25.246 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-07-03T15:37:25.246 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-07-03T15:37:25.246 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.compression_opts.enabled: false 2024-07-03T15:37:25.246 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-07-03T15:37:25.246 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-07-03T15:37:25.247 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-07-03T15:37:25.247 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.target_file_size_base: 67108864 2024-07-03T15:37:25.247 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.target_file_size_multiplier: 1 2024-07-03T15:37:25.247 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-07-03T15:37:25.247 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-07-03T15:37:25.247 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-07-03T15:37:25.247 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-07-03T15:37:25.247 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-07-03T15:37:25.247 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-07-03T15:37:25.247 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-07-03T15:37:25.247 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-07-03T15:37:25.247 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-07-03T15:37:25.247 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-07-03T15:37:25.248 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-07-03T15:37:25.248 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-07-03T15:37:25.248 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.arena_block_size: 4194304 2024-07-03T15:37:25.248 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-07-03T15:37:25.248 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-07-03T15:37:25.248 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-07-03T15:37:25.248 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.disable_auto_compactions: 0 2024-07-03T15:37:25.248 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-07-03T15:37:25.248 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-07-03T15:37:25.248 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-07-03T15:37:25.248 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-07-03T15:37:25.248 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-07-03T15:37:25.248 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-07-03T15:37:25.249 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-07-03T15:37:25.249 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-07-03T15:37:25.249 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-07-03T15:37:25.249 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-07-03T15:37:25.249 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.table_properties_collectors: 2024-07-03T15:37:25.249 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.inplace_update_support: 0 2024-07-03T15:37:25.249 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.inplace_update_num_locks: 10000 2024-07-03T15:37:25.249 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-07-03T15:37:25.249 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-07-03T15:37:25.249 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.memtable_huge_page_size: 0 2024-07-03T15:37:25.249 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.bloom_locality: 0 2024-07-03T15:37:25.249 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.max_successive_merges: 0 2024-07-03T15:37:25.249 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.optimize_filters_for_hits: 0 2024-07-03T15:37:25.250 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.paranoid_file_checks: 0 2024-07-03T15:37:25.250 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.force_consistency_checks: 1 2024-07-03T15:37:25.250 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.report_bg_io_stats: 0 2024-07-03T15:37:25.250 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.ttl: 2592000 2024-07-03T15:37:25.250 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.periodic_compaction_seconds: 0 2024-07-03T15:37:25.250 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.enable_blob_files: false 2024-07-03T15:37:25.250 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.min_blob_size: 0 2024-07-03T15:37:25.250 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.blob_file_size: 268435456 2024-07-03T15:37:25.250 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.blob_compression_type: NoCompression 2024-07-03T15:37:25.250 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.enable_blob_garbage_collection: false 2024-07-03T15:37:25.250 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-07-03T15:37:25.250 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi148/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-07-03T15:37:25.250 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2024-07-03T15:37:25.251 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: [db/version_set.cc:4082] Creating manifest 7 2024-07-03T15:37:25.251 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: EVENT_LOG_v1 {"time_micros": 1720021044908672, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-07-03T15:37:25.251 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #4 mode 2 2024-07-03T15:37:25.251 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-07-03T15:37:25.251 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: EVENT_LOG_v1 {"time_micros": 1720021044908987, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1545, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1720021044, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "aa7f1c5d-03d7-4cdd-9d82-e6d25900988e", "db_session_id": "3M0MZK5V1XZEA3KW8QJS"}} 2024-07-03T15:37:25.251 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: [db/version_set.cc:4082] Creating manifest 9 2024-07-03T15:37:25.251 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: EVENT_LOG_v1 {"time_micros": 1720021044909310, "job": 1, "event": "recovery_finished"} 2024-07-03T15:37:25.251 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-smithi148/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-07-03T15:37:25.251 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x55f9cbbd0700 2024-07-03T15:37:25.251 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: DB pointer 0x55f9cbba0000 2024-07-03T15:37:25.251 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-07-03T15:37:25.251 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-07-03T15:37:25.251 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: ** DB Stats ** 2024-07-03T15:37:25.251 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-07-03T15:37:25.252 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-07-03T15:37:25.252 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-07-03T15:37:25.252 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-07-03T15:37:25.252 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-07-03T15:37:25.252 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-07-03T15:37:25.252 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-07-03T15:37:25.252 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: 2024-07-03T15:37:25.252 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: ** Compaction Stats [default] ** 2024-07-03T15:37:25.252 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-07-03T15:37:25.252 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-07-03T15:37:25.252 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: L0 1/0 1.51 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.9 0.00 0.00 1 0.000 0 0 2024-07-03T15:37:25.252 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Sum 1/0 1.51 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.9 0.00 0.00 1 0.000 0 0 2024-07-03T15:37:25.252 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.9 0.00 0.00 1 0.000 0 0 2024-07-03T15:37:25.252 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: 2024-07-03T15:37:25.252 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: ** Compaction Stats [default] ** 2024-07-03T15:37:25.253 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-07-03T15:37:25.253 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-07-03T15:37:25.253 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.9 0.00 0.00 1 0.000 0 0 2024-07-03T15:37:25.253 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-07-03T15:37:25.253 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-07-03T15:37:25.253 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-07-03T15:37:25.253 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-07-03T15:37:25.253 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-07-03T15:37:25.253 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: AddFile(Keys): cumulative 0, interval 0 2024-07-03T15:37:25.253 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Cumulative compaction: 0.00 GB write, 0.51 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-07-03T15:37:25.253 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Interval compaction: 0.00 GB write, 0.51 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-07-03T15:37:25.253 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-07-03T15:37:25.254 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: 2024-07-03T15:37:25.254 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: ** File Read Latency Histogram By Level [default] ** 2024-07-03T15:37:25.254 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: 2024-07-03T15:37:25.254 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: ** Compaction Stats [default] ** 2024-07-03T15:37:25.254 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-07-03T15:37:25.254 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-07-03T15:37:25.254 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: L0 1/0 1.51 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.9 0.00 0.00 1 0.000 0 0 2024-07-03T15:37:25.254 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Sum 1/0 1.51 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.9 0.00 0.00 1 0.000 0 0 2024-07-03T15:37:25.254 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-07-03T15:37:25.254 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: 2024-07-03T15:37:25.254 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: ** Compaction Stats [default] ** 2024-07-03T15:37:25.254 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-07-03T15:37:25.254 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-07-03T15:37:25.255 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.9 0.00 0.00 1 0.000 0 0 2024-07-03T15:37:25.255 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-07-03T15:37:25.255 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-07-03T15:37:25.255 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-07-03T15:37:25.255 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-07-03T15:37:25.255 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-07-03T15:37:25.255 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: AddFile(Keys): cumulative 0, interval 0 2024-07-03T15:37:25.255 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Cumulative compaction: 0.00 GB write, 0.50 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-07-03T15:37:25.255 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-07-03T15:37:25.255 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-07-03T15:37:25.255 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: 2024-07-03T15:37:25.255 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: ** File Read Latency Histogram By Level [default] ** 2024-07-03T15:37:25.255 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: mon.smithi148 does not exist in monmap, will attempt to join an existing cluster 2024-07-03T15:37:25.255 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: using public_addr v2:172.21.15.148:0/0 -> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] 2024-07-03T15:37:25.256 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: starting mon.smithi148 rank -1 at public addrs [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] at bind addrs [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi148 fsid a85e4be6-3951-11ef-bcab-c7b262605968 2024-07-03T15:37:25.256 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: mon.smithi148@-1(???) e0 preinit fsid a85e4be6-3951-11ef-bcab-c7b262605968 2024-07-03T15:37:25.256 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: mon.smithi148@-1(synchronizing).mds e1 new map 2024-07-03T15:37:25.256 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: mon.smithi148@-1(synchronizing).mds e1 print_map 2024-07-03T15:37:25.256 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: e1 2024-07-03T15:37:25.256 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-07-03T15:37:25.256 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:37:25.256 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: legacy client fscid: -1 2024-07-03T15:37:25.256 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: 2024-07-03T15:37:25.256 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: No filesystems configured 2024-07-03T15:37:25.256 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: mon.smithi148@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-07-03T15:37:25.256 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: mon.smithi148@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-07-03T15:37:25.256 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: mon.smithi148@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-07-03T15:37:25.256 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: mon.smithi148@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-07-03T15:37:25.257 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: mon.smithi148@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-07-03T15:37:25.257 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: mon.smithi148@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-07-03T15:37:25.257 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: mon.smithi148@-1(synchronizing).osd e5 e5: 0 total, 0 up, 0 in 2024-07-03T15:37:25.257 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: mon.smithi148@-1(synchronizing).osd e5 crush map has features 3314932999778484224, adjusting msgr requires 2024-07-03T15:37:25.257 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: mon.smithi148@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-07-03T15:37:25.257 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: mon.smithi148@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-07-03T15:37:25.257 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: mon.smithi148@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-07-03T15:37:25.257 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:25.257 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: Deploying cephadm binary to smithi148 2024-07-03T15:37:25.257 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:25.257 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.257 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: Added host smithi148 2024-07-03T15:37:25.257 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.258 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.258 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.258 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.258 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: Deploying daemon alertmanager.smithi028 on smithi028 2024-07-03T15:37:25.258 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:25.258 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.258 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-03T15:37:25.258 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:25.258 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:25.258 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/1486611758' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-07-03T15:37:25.258 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:25.258 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/1486611758' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-07-03T15:37:25.259 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: osdmap e4: 0 total, 0 up, 0 in 2024-07-03T15:37:25.259 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:25.259 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.259 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='client.14192 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi028:172.21.15.28=smithi028;smithi148:172.21.15.148=smithi148", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:37:25.259 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: Saving service mon spec with placement smithi028:172.21.15.28=smithi028;smithi148:172.21.15.148=smithi148;count:2 2024-07-03T15:37:25.259 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.259 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.259 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.259 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.259 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.259 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.259 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-07-03T15:37:25.259 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.260 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-07-03T15:37:25.260 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: Deploying daemon grafana.smithi028 on smithi028 2024-07-03T15:37:25.260 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:25.260 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.260 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='client.? 172.21.15.148:0/2379683878' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-03T15:37:25.260 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:25.260 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:25.260 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='client.? 172.21.15.148:0/1808843447' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-03T15:37:25.260 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:25.260 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:25.260 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='client.? 172.21.15.148:0/3218944017' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-03T15:37:25.260 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:25.260 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='client.? 172.21.15.148:0/1572532964' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-03T15:37:25.260 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:25.261 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:25.261 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='client.? 172.21.15.148:0/3877713508' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-03T15:37:25.261 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:25.261 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.261 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.261 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.261 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.261 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.261 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.261 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: Deploying daemon prometheus.smithi028 on smithi028 2024-07-03T15:37:25.261 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:25.261 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='client.? 172.21.15.148:0/590190099' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-03T15:37:25.261 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:25.261 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.262 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:25.262 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='client.? 172.21.15.148:0/3523049793' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-03T15:37:25.262 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:25.262 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='client.? 172.21.15.148:0/2841114716' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-03T15:37:25.262 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:25.262 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.262 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.262 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.263 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-07-03T15:37:25.263 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:25.263 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14160 172.21.15.28:0/1764392431' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-07-03T15:37:25.263 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: mgrmap e14: smithi028.vtqrtz(active, since 62s) 2024-07-03T15:37:25.263 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='client.? 172.21.15.148:0/3656626934' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-03T15:37:25.263 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='client.? 172.21.15.148:0/781075407' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-03T15:37:25.263 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: Active manager daemon smithi028.vtqrtz restarted 2024-07-03T15:37:25.263 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: Activating manager daemon smithi028.vtqrtz 2024-07-03T15:37:25.263 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: osdmap e5: 0 total, 0 up, 0 in 2024-07-03T15:37:25.263 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: mgrmap e15: smithi028.vtqrtz(active, starting, since 0.00288669s) 2024-07-03T15:37:25.263 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi028"}]: dispatch 2024-07-03T15:37:25.263 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mgr metadata", "who": "smithi028.vtqrtz", "id": "smithi028.vtqrtz"}]: dispatch 2024-07-03T15:37:25.263 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-03T15:37:25.263 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-03T15:37:25.264 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-03T15:37:25.264 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: Manager daemon smithi028.vtqrtz is now available 2024-07-03T15:37:25.264 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:37:25.264 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:37:25.264 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.264 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:37:25.264 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi028.vtqrtz/mirror_snapshot_schedule"}]: dispatch 2024-07-03T15:37:25.264 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi028.vtqrtz/trash_purge_schedule"}]: dispatch 2024-07-03T15:37:25.264 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.264 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.264 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: mgrmap e16: smithi028.vtqrtz(active, since 1.00618s) 2024-07-03T15:37:25.264 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.264 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: mgrmap e17: smithi028.vtqrtz(active, since 2s) 2024-07-03T15:37:25.264 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='client.? 172.21.15.148:0/2556223766' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-03T15:37:25.264 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: [03/Jul/2024:15:37:03] ENGINE Bus STARTING 2024-07-03T15:37:25.265 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.265 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: [03/Jul/2024:15:37:03] ENGINE Serving on https://172.21.15.28:7150 2024-07-03T15:37:25.265 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: [03/Jul/2024:15:37:03] ENGINE Bus STARTED 2024-07-03T15:37:25.265 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: [03/Jul/2024:15:37:03] ENGINE Error in HTTPServer.serve 2024-07-03T15:37:25.265 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Traceback (most recent call last): 2024-07-03T15:37:25.265 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-03T15:37:25.265 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: self._connections.run(self.expiration_interval) 2024-07-03T15:37:25.265 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-03T15:37:25.265 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: self._run(expiration_interval) 2024-07-03T15:37:25.265 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-03T15:37:25.265 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-07-03T15:37:25.265 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-03T15:37:25.265 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-03T15:37:25.265 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-03T15:37:25.265 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: s = self.context.wrap_socket( 2024-07-03T15:37:25.266 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-03T15:37:25.266 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: return self.sslsocket_class._create( 2024-07-03T15:37:25.266 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-03T15:37:25.266 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: self.do_handshake() 2024-07-03T15:37:25.266 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-03T15:37:25.266 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: self._sslobj.do_handshake() 2024-07-03T15:37:25.266 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-03T15:37:25.266 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.266 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.266 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.267 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.267 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "who": "osd/host:smithi028", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:37:25.267 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.267 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.267 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.267 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.267 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "who": "osd/host:smithi148", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:37:25.267 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:37:25.267 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:37:25.267 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: Updating smithi028:/etc/ceph/ceph.conf 2024-07-03T15:37:25.267 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: Updating smithi148:/etc/ceph/ceph.conf 2024-07-03T15:37:25.267 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='client.? 172.21.15.148:0/2892916235' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-03T15:37:25.267 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.268 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.268 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.268 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.268 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.268 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi148", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-07-03T15:37:25.268 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi148", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-07-03T15:37:25.268 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:37:25.268 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: Updating smithi148:/var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-03T15:37:25.268 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: Updating smithi028:/var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-03T15:37:25.268 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-07-03T15:37:25.268 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: Updating smithi028:/etc/ceph/ceph.client.admin.keyring 2024-07-03T15:37:25.268 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: Updating smithi148:/var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/config/ceph.client.admin.keyring 2024-07-03T15:37:25.268 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: Updating smithi028:/var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/config/ceph.client.admin.keyring 2024-07-03T15:37:25.268 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: Deploying daemon crash.smithi148 on smithi148 2024-07-03T15:37:25.268 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='client.? 172.21.15.148:0/1031446823' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-03T15:37:25.269 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.269 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.269 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.269 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.269 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: Deploying daemon node-exporter.smithi148 on smithi148 2024-07-03T15:37:25.269 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.269 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:37:25.269 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.269 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.269 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.269 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.269 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi148.rjzsar", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-03T15:37:25.269 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi148.rjzsar", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-07-03T15:37:25.269 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-03T15:37:25.269 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:37:25.270 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='client.? 172.21.15.148:0/3330252447' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-03T15:37:25.270 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: Deploying daemon mgr.smithi148.rjzsar on smithi148 2024-07-03T15:37:25.270 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='client.? 172.21.15.148:0/1986850832' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-03T15:37:25.270 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.270 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.270 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.270 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.270 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-03T15:37:25.270 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:37:25.270 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: Deploying daemon mon.smithi148 on smithi148 2024-07-03T15:37:25.270 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:25.270 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:25.270 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:25.271 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 ceph-mon[36056]: mon.smithi148@-1(synchronizing).paxosservice(auth 1..6) refresh upgraded, format 0 -> 3 2024-07-03T15:37:25.271 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 bash[35986]: 4c6c8ed2ab8514f8579f2aa29e5e662df817a7f4b577e99d77cf7dcf90067096 2024-07-03T15:37:25.271 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:24 smithi148 systemd[1]: Started Ceph mon.smithi148 for a85e4be6-3951-11ef-bcab-c7b262605968. 2024-07-03T15:37:29.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:29.946+0000 7fc163fff640 1 -- 172.21.15.148:0/3528583221 <== mon.0 v2:172.21.15.28:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1017 (secure 0 0 0) 0x7fc16c045390 con 0x7fc17c0fd120 2024-07-03T15:37:29.949 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-07-03T15:37:29.949 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":2,"fsid":"a85e4be6-3951-11ef-bcab-c7b262605968","modified":"2024-07-03T15:37:24.935576Z","created":"2024-07-03T15:34:53.167064Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi028","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:3300","nonce":0},{"type":"v1","addr":"172.21.15.28:6789","nonce":0}]},"addr":"172.21.15.28:6789/0","public_addr":"172.21.15.28:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi148","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:3300","nonce":0},{"type":"v1","addr":"172.21.15.148:6789","nonce":0}]},"addr":"172.21.15.148:6789/0","public_addr":"172.21.15.148:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-07-03T15:37:29.949 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 2 2024-07-03T15:37:29.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:29.948+0000 7fc181d31640 1 -- 172.21.15.148:0/3528583221 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fc1640358a0 msgr2=0x7fc164037d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:37:29.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:29.948+0000 7fc181d31640 1 --2- 172.21.15.148:0/3528583221 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fc1640358a0 0x7fc164037d50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc170006fd0 tx=0x7fc170006db0 comp rx=0 tx=0).stop 2024-07-03T15:37:29.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:29.948+0000 7fc181d31640 1 -- 172.21.15.148:0/3528583221 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc17c0fd120 msgr2=0x7fc17c1127e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:37:29.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:29.948+0000 7fc181d31640 1 --2- 172.21.15.148:0/3528583221 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc17c0fd120 0x7fc17c1127e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc16c004820 tx=0x7fc16c009da0 comp rx=0 tx=0).stop 2024-07-03T15:37:29.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:29.948+0000 7fc181d31640 1 -- 172.21.15.148:0/3528583221 shutdown_connections 2024-07-03T15:37:29.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:29.948+0000 7fc181d31640 1 --2- 172.21.15.148:0/3528583221 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fc1640358a0 0x7fc164037d50 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:37:29.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:29.948+0000 7fc181d31640 1 --2- 172.21.15.148:0/3528583221 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc17c0fd120 0x7fc17c1127e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:37:29.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:29.949+0000 7fc181d31640 1 -- 172.21.15.148:0/3528583221 >> 172.21.15.148:0/3528583221 conn(0x7fc17c0f6c20 msgr2=0x7fc17c102d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:37:29.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:29.949+0000 7fc181d31640 1 -- 172.21.15.148:0/3528583221 shutdown_connections 2024-07-03T15:37:29.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:37:29.949+0000 7fc181d31640 1 -- 172.21.15.148:0/3528583221 wait complete. 2024-07-03T15:37:30.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:30 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-07-03T15:37:30.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:30 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi028"}]: dispatch 2024-07-03T15:37:30.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:30 smithi148 ceph-mon[36056]: mon.smithi028 calling monitor election 2024-07-03T15:37:30.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:30 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-07-03T15:37:30.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:30 smithi148 ceph-mon[36056]: from='client.? 172.21.15.148:0/3528583221' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-03T15:37:30.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:30 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-07-03T15:37:30.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:30 smithi148 ceph-mon[36056]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:30.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:30 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-07-03T15:37:30.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:30 smithi148 ceph-mon[36056]: mon.smithi148 calling monitor election 2024-07-03T15:37:30.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:30 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-07-03T15:37:30.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:30 smithi148 ceph-mon[36056]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:30.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:30 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-07-03T15:37:30.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:30 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-07-03T15:37:30.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:30 smithi148 ceph-mon[36056]: mon.smithi028 is new leader, mons smithi028,smithi148 in quorum (ranks 0,1) 2024-07-03T15:37:30.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:30 smithi148 ceph-mon[36056]: monmap e2: 2 mons at {smithi028=[v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0],smithi148=[v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-07-03T15:37:30.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:30 smithi148 ceph-mon[36056]: fsmap 2024-07-03T15:37:30.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:30 smithi148 ceph-mon[36056]: osdmap e5: 0 total, 0 up, 0 in 2024-07-03T15:37:30.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:30 smithi148 ceph-mon[36056]: mgrmap e17: smithi028.vtqrtz(active, since 29s) 2024-07-03T15:37:30.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:30 smithi148 ceph-mon[36056]: overall HEALTH_OK 2024-07-03T15:37:30.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:30 smithi148 ceph-mon[36056]: Standby manager daemon smithi148.rjzsar started 2024-07-03T15:37:30.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:30 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:30.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:30 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:30.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:30 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:30.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:30 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:30.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:30 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:30.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:30 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-07-03T15:37:30.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:30 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi028"}]: dispatch 2024-07-03T15:37:30.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:30 smithi028 ceph-mon[29148]: mon.smithi028 calling monitor election 2024-07-03T15:37:30.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:30 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-07-03T15:37:30.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:30 smithi028 ceph-mon[29148]: from='client.? 172.21.15.148:0/3528583221' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-03T15:37:30.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:30 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-07-03T15:37:30.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:30 smithi028 ceph-mon[29148]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:30.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:30 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-07-03T15:37:30.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:30 smithi028 ceph-mon[29148]: mon.smithi148 calling monitor election 2024-07-03T15:37:30.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:30 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-07-03T15:37:30.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:30 smithi028 ceph-mon[29148]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:30.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:30 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-07-03T15:37:30.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:30 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-07-03T15:37:30.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:30 smithi028 ceph-mon[29148]: mon.smithi028 is new leader, mons smithi028,smithi148 in quorum (ranks 0,1) 2024-07-03T15:37:30.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:30 smithi028 ceph-mon[29148]: monmap e2: 2 mons at {smithi028=[v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0],smithi148=[v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-07-03T15:37:30.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:30 smithi028 ceph-mon[29148]: fsmap 2024-07-03T15:37:30.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:30 smithi028 ceph-mon[29148]: osdmap e5: 0 total, 0 up, 0 in 2024-07-03T15:37:30.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:30 smithi028 ceph-mon[29148]: mgrmap e17: smithi028.vtqrtz(active, since 29s) 2024-07-03T15:37:30.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:30 smithi028 ceph-mon[29148]: overall HEALTH_OK 2024-07-03T15:37:30.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:30 smithi028 ceph-mon[29148]: Standby manager daemon smithi148.rjzsar started 2024-07-03T15:37:30.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:30 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:30.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:30 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:30.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:30 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:30.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:30 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:30.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:30 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:30.522 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-07-03T15:37:30.522 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph config generate-minimal-conf 2024-07-03T15:37:30.798 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:37:31.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:31 smithi148 ceph-mon[36056]: mgrmap e18: smithi028.vtqrtz(active, since 29s), standbys: smithi148.rjzsar 2024-07-03T15:37:31.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:31 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mgr metadata", "who": "smithi148.rjzsar", "id": "smithi148.rjzsar"}]: dispatch 2024-07-03T15:37:31.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:31 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:37:31.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:31 smithi148 ceph-mon[36056]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:31.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:31 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:37:31.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:31 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:31.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:31 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-07-03T15:37:31.232 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:31 smithi028 ceph-mon[29148]: mgrmap e18: smithi028.vtqrtz(active, since 29s), standbys: smithi148.rjzsar 2024-07-03T15:37:31.232 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:31 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mgr metadata", "who": "smithi148.rjzsar", "id": "smithi148.rjzsar"}]: dispatch 2024-07-03T15:37:31.232 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:31 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:37:31.232 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:31 smithi028 ceph-mon[29148]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:31.232 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:31 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:37:31.232 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:31 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:31.232 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:31 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-07-03T15:37:32.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.446+0000 7fd5808c7640 1 -- 172.21.15.28:0/1762596003 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd57c101ff0 msgr2=0x7fd57c1023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:37:32.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.446+0000 7fd5808c7640 1 --2- 172.21.15.28:0/1762596003 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd57c101ff0 0x7fd57c1023c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd56c004850 tx=0x7fd56c009da0 comp rx=0 tx=0).stop 2024-07-03T15:37:32.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.446+0000 7fd5808c7640 1 -- 172.21.15.28:0/1762596003 shutdown_connections 2024-07-03T15:37:32.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.446+0000 7fd5808c7640 1 --2- 172.21.15.28:0/1762596003 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd57c101ff0 0x7fd57c1023c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:37:32.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.446+0000 7fd5808c7640 1 -- 172.21.15.28:0/1762596003 >> 172.21.15.28:0/1762596003 conn(0x7fd57c070ec0 msgr2=0x7fd57c0712c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:37:32.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.447+0000 7fd5808c7640 1 -- 172.21.15.28:0/1762596003 shutdown_connections 2024-07-03T15:37:32.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.447+0000 7fd5808c7640 1 -- 172.21.15.28:0/1762596003 wait complete. 2024-07-03T15:37:32.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.448+0000 7fd5808c7640 1 Processor -- start 2024-07-03T15:37:32.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.448+0000 7fd5808c7640 1 -- start start 2024-07-03T15:37:32.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.448+0000 7fd5808c7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd57c101ff0 0x7fd57c0faf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:37:32.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.448+0000 7fd5808c7640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd57c0fb4b0 0x7fd57c0fb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:37:32.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.448+0000 7fd5808c7640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c118a90 con 0x7fd57c0fb4b0 2024-07-03T15:37:32.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.448+0000 7fd5808c7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c118bd0 con 0x7fd57c101ff0 2024-07-03T15:37:32.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.448+0000 7fd573fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd57c0fb4b0 0x7fd57c0fb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:37:32.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.448+0000 7fd573fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd57c0fb4b0 0x7fd57c0fb900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:57808/0 (socket says 172.21.15.28:57808) 2024-07-03T15:37:32.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.448+0000 7fd573fff640 1 -- 172.21.15.28:0/2556449435 learned_addr learned my addr 172.21.15.28:0/2556449435 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:37:32.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.448+0000 7fd57a59c640 1 --2- 172.21.15.28:0/2556449435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd57c101ff0 0x7fd57c0faf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:37:32.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.448+0000 7fd57a59c640 1 -- 172.21.15.28:0/2556449435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd57c101ff0 msgr2=0x7fd57c0faf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-07-03T15:37:32.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.448+0000 7fd57a59c640 1 -- 172.21.15.28:0/2556449435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd57c101ff0 msgr2=0x7fd57c0faf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-07-03T15:37:32.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.448+0000 7fd57a59c640 1 --2- 172.21.15.28:0/2556449435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd57c101ff0 0x7fd57c0faf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-07-03T15:37:32.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.448+0000 7fd57a59c640 1 --2- 172.21.15.28:0/2556449435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd57c101ff0 0x7fd57c0faf70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-03T15:37:32.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.448+0000 7fd573fff640 1 -- 172.21.15.28:0/2556449435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd57c101ff0 msgr2=0x7fd57c0faf70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-03T15:37:32.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.448+0000 7fd573fff640 1 --2- 172.21.15.28:0/2556449435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd57c101ff0 0x7fd57c0faf70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:37:32.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.449+0000 7fd573fff640 1 -- 172.21.15.28:0/2556449435 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd56c005040 con 0x7fd57c0fb4b0 2024-07-03T15:37:32.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.449+0000 7fd573fff640 1 --2- 172.21.15.28:0/2556449435 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd57c0fb4b0 0x7fd57c0fb900 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd56400b7c0 tx=0x7fd5640091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:37:32.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.450+0000 7fd572ffd640 1 -- 172.21.15.28:0/2556449435 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd56400c730 con 0x7fd57c0fb4b0 2024-07-03T15:37:32.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.450+0000 7fd5808c7640 1 -- 172.21.15.28:0/2556449435 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd57c118dd0 con 0x7fd57c0fb4b0 2024-07-03T15:37:32.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.450+0000 7fd5808c7640 1 -- 172.21.15.28:0/2556449435 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd57c1192a0 con 0x7fd57c0fb4b0 2024-07-03T15:37:32.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.450+0000 7fd572ffd640 1 -- 172.21.15.28:0/2556449435 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd56400cd60 con 0x7fd57c0fb4b0 2024-07-03T15:37:32.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.450+0000 7fd572ffd640 1 -- 172.21.15.28:0/2556449435 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5640133a0 con 0x7fd57c0fb4b0 2024-07-03T15:37:32.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.451+0000 7fd5808c7640 1 -- 172.21.15.28:0/2556449435 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd55c005210 con 0x7fd57c0fb4b0 2024-07-03T15:37:32.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.451+0000 7fd572ffd640 1 -- 172.21.15.28:0/2556449435 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 18) v1 ==== 85069+0+0 (secure 0 0 0) 0x7fd564013500 con 0x7fd57c0fb4b0 2024-07-03T15:37:32.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.452+0000 7fd572ffd640 1 --2- 172.21.15.28:0/2556449435 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fd568067250 0x7fd568069700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:37:32.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.452+0000 7fd572ffd640 1 -- 172.21.15.28:0/2556449435 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fd564082fa0 con 0x7fd57c0fb4b0 2024-07-03T15:37:32.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.452+0000 7fd57a59c640 1 --2- 172.21.15.28:0/2556449435 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fd568067250 0x7fd568069700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:37:32.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.453+0000 7fd57a59c640 1 --2- 172.21.15.28:0/2556449435 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fd568067250 0x7fd568069700 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd56c02ab60 tx=0x7fd56c009da0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:37:32.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.456+0000 7fd572ffd640 1 -- 172.21.15.28:0/2556449435 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd56404d250 con 0x7fd57c0fb4b0 2024-07-03T15:37:32.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.637+0000 7fd5808c7640 1 -- 172.21.15.28:0/2556449435 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fd55c0054c0 con 0x7fd57c0fb4b0 2024-07-03T15:37:32.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.637+0000 7fd572ffd640 1 -- 172.21.15.28:0/2556449435 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+225 (secure 0 0 0) 0x7fd564019030 con 0x7fd57c0fb4b0 2024-07-03T15:37:32.640 INFO:teuthology.orchestra.run.smithi028.stdout:# minimal ceph.conf for a85e4be6-3951-11ef-bcab-c7b262605968 2024-07-03T15:37:32.641 INFO:teuthology.orchestra.run.smithi028.stdout:[global] 2024-07-03T15:37:32.641 INFO:teuthology.orchestra.run.smithi028.stdout: fsid = a85e4be6-3951-11ef-bcab-c7b262605968 2024-07-03T15:37:32.641 INFO:teuthology.orchestra.run.smithi028.stdout: mon_host = [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] 2024-07-03T15:37:32.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.639+0000 7fd5808c7640 1 -- 172.21.15.28:0/2556449435 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fd568067250 msgr2=0x7fd568069700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:37:32.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.639+0000 7fd5808c7640 1 --2- 172.21.15.28:0/2556449435 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fd568067250 0x7fd568069700 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd56c02ab60 tx=0x7fd56c009da0 comp rx=0 tx=0).stop 2024-07-03T15:37:32.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.639+0000 7fd5808c7640 1 -- 172.21.15.28:0/2556449435 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd57c0fb4b0 msgr2=0x7fd57c0fb900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:37:32.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.639+0000 7fd5808c7640 1 --2- 172.21.15.28:0/2556449435 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd57c0fb4b0 0x7fd57c0fb900 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd56400b7c0 tx=0x7fd5640091d0 comp rx=0 tx=0).stop 2024-07-03T15:37:32.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.639+0000 7fd5808c7640 1 -- 172.21.15.28:0/2556449435 shutdown_connections 2024-07-03T15:37:32.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.639+0000 7fd5808c7640 1 --2- 172.21.15.28:0/2556449435 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fd568067250 0x7fd568069700 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:37:32.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.639+0000 7fd5808c7640 1 --2- 172.21.15.28:0/2556449435 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd57c0fb4b0 0x7fd57c0fb900 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:37:32.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.639+0000 7fd5808c7640 1 --2- 172.21.15.28:0/2556449435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd57c101ff0 0x7fd57c0faf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:37:32.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.639+0000 7fd5808c7640 1 -- 172.21.15.28:0/2556449435 >> 172.21.15.28:0/2556449435 conn(0x7fd57c070ec0 msgr2=0x7fd57c0f9e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:37:32.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.639+0000 7fd5808c7640 1 -- 172.21.15.28:0/2556449435 shutdown_connections 2024-07-03T15:37:32.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:32.639+0000 7fd5808c7640 1 -- 172.21.15.28:0/2556449435 wait complete. 2024-07-03T15:37:33.260 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:32 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:33.260 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:32 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:33.260 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:32 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:37:33.260 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:32 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:37:33.261 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:32 smithi028 ceph-mon[29148]: Updating smithi028:/etc/ceph/ceph.conf 2024-07-03T15:37:33.261 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:32 smithi028 ceph-mon[29148]: Updating smithi148:/etc/ceph/ceph.conf 2024-07-03T15:37:33.261 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:32 smithi028 ceph-mon[29148]: Updating smithi148:/var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-03T15:37:33.261 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:32 smithi028 ceph-mon[29148]: Updating smithi028:/var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-03T15:37:33.261 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:32 smithi028 ceph-mon[29148]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:33.261 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:32 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:33.261 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:32 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:33.261 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:32 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:33.261 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:32 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:33.261 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:32 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:33.261 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:32 smithi028 ceph-mon[29148]: Reconfiguring mon.smithi028 (unknown last config time)... 2024-07-03T15:37:33.261 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:32 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-03T15:37:33.261 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:32 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-03T15:37:33.261 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:32 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:37:33.262 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:32 smithi028 ceph-mon[29148]: Reconfiguring daemon mon.smithi028 on smithi028 2024-07-03T15:37:33.262 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:32 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/2556449435' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:37:33.284 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-07-03T15:37:33.285 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-07-03T15:37:33.285 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/etc/ceph/ceph.conf 2024-07-03T15:37:33.313 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-07-03T15:37:33.313 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-07-03T15:37:33.379 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-07-03T15:37:33.380 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/ceph/ceph.conf 2024-07-03T15:37:33.407 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:32 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:33.407 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:32 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:33.407 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:32 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:37:33.407 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:32 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:37:33.407 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:32 smithi148 ceph-mon[36056]: Updating smithi028:/etc/ceph/ceph.conf 2024-07-03T15:37:33.408 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:32 smithi148 ceph-mon[36056]: Updating smithi148:/etc/ceph/ceph.conf 2024-07-03T15:37:33.408 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:32 smithi148 ceph-mon[36056]: Updating smithi148:/var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-03T15:37:33.408 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:32 smithi148 ceph-mon[36056]: Updating smithi028:/var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-03T15:37:33.408 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:32 smithi148 ceph-mon[36056]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:33.408 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:32 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:33.408 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:32 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:33.408 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:32 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:33.408 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:32 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:33.408 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:32 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:33.408 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:32 smithi148 ceph-mon[36056]: Reconfiguring mon.smithi028 (unknown last config time)... 2024-07-03T15:37:33.408 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:32 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-03T15:37:33.408 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:32 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-03T15:37:33.408 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:32 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:37:33.408 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:32 smithi148 ceph-mon[36056]: Reconfiguring daemon mon.smithi028 on smithi028 2024-07-03T15:37:33.408 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:32 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/2556449435' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:37:33.413 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-07-03T15:37:33.413 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-07-03T15:37:33.481 INFO:tasks.cephadm:Deploying OSDs... 2024-07-03T15:37:33.481 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-07-03T15:37:33.481 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/scratch_devs of=/dev/stdout 2024-07-03T15:37:33.497 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-07-03T15:37:33.498 DEBUG:teuthology.orchestra.run.smithi028:> stat /dev/vg_nvme/lv_1 2024-07-03T15:37:33.556 INFO:teuthology.orchestra.run.smithi028.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-07-03T15:37:33.556 INFO:teuthology.orchestra.run.smithi028.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-03T15:37:33.556 INFO:teuthology.orchestra.run.smithi028.stdout:Device: 5h/5d Inode: 676 Links: 1 2024-07-03T15:37:33.556 INFO:teuthology.orchestra.run.smithi028.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-03T15:37:33.556 INFO:teuthology.orchestra.run.smithi028.stdout:Context: system_u:object_r:device_t:s0 2024-07-03T15:37:33.556 INFO:teuthology.orchestra.run.smithi028.stdout:Access: 2024-07-03 15:36:54.003017835 +0000 2024-07-03T15:37:33.556 INFO:teuthology.orchestra.run.smithi028.stdout:Modify: 2024-07-03 15:30:47.923687366 +0000 2024-07-03T15:37:33.556 INFO:teuthology.orchestra.run.smithi028.stdout:Change: 2024-07-03 15:30:47.923687366 +0000 2024-07-03T15:37:33.556 INFO:teuthology.orchestra.run.smithi028.stdout: Birth: 2024-07-03 15:30:47.923687366 +0000 2024-07-03T15:37:33.556 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-07-03T15:37:33.621 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records in 2024-07-03T15:37:33.621 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records out 2024-07-03T15:37:33.621 INFO:teuthology.orchestra.run.smithi028.stderr:512 bytes copied, 9.9533e-05 s, 5.1 MB/s 2024-07-03T15:37:33.622 DEBUG:teuthology.orchestra.run.smithi028:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-07-03T15:37:33.683 DEBUG:teuthology.orchestra.run.smithi028:> stat /dev/vg_nvme/lv_2 2024-07-03T15:37:33.739 INFO:teuthology.orchestra.run.smithi028.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-07-03T15:37:33.739 INFO:teuthology.orchestra.run.smithi028.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-03T15:37:33.739 INFO:teuthology.orchestra.run.smithi028.stdout:Device: 5h/5d Inode: 692 Links: 1 2024-07-03T15:37:33.739 INFO:teuthology.orchestra.run.smithi028.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-03T15:37:33.739 INFO:teuthology.orchestra.run.smithi028.stdout:Context: system_u:object_r:device_t:s0 2024-07-03T15:37:33.740 INFO:teuthology.orchestra.run.smithi028.stdout:Access: 2024-07-03 15:36:54.003017835 +0000 2024-07-03T15:37:33.740 INFO:teuthology.orchestra.run.smithi028.stdout:Modify: 2024-07-03 15:30:48.274677467 +0000 2024-07-03T15:37:33.740 INFO:teuthology.orchestra.run.smithi028.stdout:Change: 2024-07-03 15:30:48.274677467 +0000 2024-07-03T15:37:33.740 INFO:teuthology.orchestra.run.smithi028.stdout: Birth: 2024-07-03 15:30:48.274677467 +0000 2024-07-03T15:37:33.740 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-07-03T15:37:33.804 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records in 2024-07-03T15:37:33.804 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records out 2024-07-03T15:37:33.804 INFO:teuthology.orchestra.run.smithi028.stderr:512 bytes copied, 0.000113226 s, 4.5 MB/s 2024-07-03T15:37:33.806 DEBUG:teuthology.orchestra.run.smithi028:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-07-03T15:37:33.863 DEBUG:teuthology.orchestra.run.smithi028:> stat /dev/vg_nvme/lv_3 2024-07-03T15:37:33.920 INFO:teuthology.orchestra.run.smithi028.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-07-03T15:37:33.920 INFO:teuthology.orchestra.run.smithi028.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-03T15:37:33.920 INFO:teuthology.orchestra.run.smithi028.stdout:Device: 5h/5d Inode: 711 Links: 1 2024-07-03T15:37:33.920 INFO:teuthology.orchestra.run.smithi028.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-03T15:37:33.920 INFO:teuthology.orchestra.run.smithi028.stdout:Context: system_u:object_r:device_t:s0 2024-07-03T15:37:33.920 INFO:teuthology.orchestra.run.smithi028.stdout:Access: 2024-07-03 15:36:54.003017835 +0000 2024-07-03T15:37:33.920 INFO:teuthology.orchestra.run.smithi028.stdout:Modify: 2024-07-03 15:30:48.602668217 +0000 2024-07-03T15:37:33.920 INFO:teuthology.orchestra.run.smithi028.stdout:Change: 2024-07-03 15:30:48.602668217 +0000 2024-07-03T15:37:33.920 INFO:teuthology.orchestra.run.smithi028.stdout: Birth: 2024-07-03 15:30:48.602668217 +0000 2024-07-03T15:37:33.921 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-07-03T15:37:33.985 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records in 2024-07-03T15:37:33.985 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records out 2024-07-03T15:37:33.985 INFO:teuthology.orchestra.run.smithi028.stderr:512 bytes copied, 0.000104726 s, 4.9 MB/s 2024-07-03T15:37:33.987 DEBUG:teuthology.orchestra.run.smithi028:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-07-03T15:37:34.044 DEBUG:teuthology.orchestra.run.smithi028:> stat /dev/vg_nvme/lv_4 2024-07-03T15:37:34.100 INFO:teuthology.orchestra.run.smithi028.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-07-03T15:37:34.100 INFO:teuthology.orchestra.run.smithi028.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-03T15:37:34.101 INFO:teuthology.orchestra.run.smithi028.stdout:Device: 5h/5d Inode: 731 Links: 1 2024-07-03T15:37:34.101 INFO:teuthology.orchestra.run.smithi028.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-03T15:37:34.101 INFO:teuthology.orchestra.run.smithi028.stdout:Context: system_u:object_r:device_t:s0 2024-07-03T15:37:34.101 INFO:teuthology.orchestra.run.smithi028.stdout:Access: 2024-07-03 15:36:54.004017807 +0000 2024-07-03T15:37:34.101 INFO:teuthology.orchestra.run.smithi028.stdout:Modify: 2024-07-03 15:30:48.948658459 +0000 2024-07-03T15:37:34.101 INFO:teuthology.orchestra.run.smithi028.stdout:Change: 2024-07-03 15:30:48.948658459 +0000 2024-07-03T15:37:34.101 INFO:teuthology.orchestra.run.smithi028.stdout: Birth: 2024-07-03 15:30:48.948658459 +0000 2024-07-03T15:37:34.101 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-07-03T15:37:34.166 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records in 2024-07-03T15:37:34.166 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records out 2024-07-03T15:37:34.166 INFO:teuthology.orchestra.run.smithi028.stderr:512 bytes copied, 9.2486e-05 s, 5.5 MB/s 2024-07-03T15:37:34.167 DEBUG:teuthology.orchestra.run.smithi028:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-07-03T15:37:34.224 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-07-03T15:37:34.224 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/scratch_devs of=/dev/stdout 2024-07-03T15:37:34.241 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-07-03T15:37:34.241 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_1 2024-07-03T15:37:34.296 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-07-03T15:37:34.296 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-03T15:37:34.296 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-07-03T15:37:34.296 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-03T15:37:34.296 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-07-03T15:37:34.296 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-07-03 15:37:23.125250984 +0000 2024-07-03T15:37:34.296 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-07-03 15:30:41.732223358 +0000 2024-07-03T15:37:34.296 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-07-03 15:30:41.732223358 +0000 2024-07-03T15:37:34.296 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-07-03 15:30:41.732223358 +0000 2024-07-03T15:37:34.297 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-07-03T15:37:34.359 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-07-03T15:37:34.359 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-07-03T15:37:34.360 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000214801 s, 2.4 MB/s 2024-07-03T15:37:34.361 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-07-03T15:37:34.417 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_2 2024-07-03T15:37:34.473 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-07-03T15:37:34.473 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-03T15:37:34.473 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-07-03T15:37:34.473 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-03T15:37:34.473 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-07-03T15:37:34.473 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-07-03 15:37:23.126250961 +0000 2024-07-03T15:37:34.474 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-07-03 15:30:42.071215229 +0000 2024-07-03T15:37:34.474 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-07-03 15:30:42.071215229 +0000 2024-07-03T15:37:34.474 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-07-03 15:30:42.071215229 +0000 2024-07-03T15:37:34.474 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-07-03T15:37:34.536 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-07-03T15:37:34.537 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-07-03T15:37:34.537 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000230815 s, 2.2 MB/s 2024-07-03T15:37:34.538 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-07-03T15:37:34.594 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_3 2024-07-03T15:37:34.649 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-07-03T15:37:34.650 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-03T15:37:34.650 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-07-03T15:37:34.650 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-03T15:37:34.650 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-07-03T15:37:34.650 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-07-03 15:37:23.126250961 +0000 2024-07-03T15:37:34.650 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-07-03 15:30:42.389207603 +0000 2024-07-03T15:37:34.650 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-07-03 15:30:42.389207603 +0000 2024-07-03T15:37:34.650 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-07-03 15:30:42.389207603 +0000 2024-07-03T15:37:34.650 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-07-03T15:37:34.713 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-07-03T15:37:34.713 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-07-03T15:37:34.713 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000198393 s, 2.6 MB/s 2024-07-03T15:37:34.714 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-07-03T15:37:34.771 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_4 2024-07-03T15:37:34.826 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-07-03T15:37:34.826 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-03T15:37:34.826 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-07-03T15:37:34.826 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-03T15:37:34.826 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-07-03T15:37:34.826 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-07-03 15:37:23.127250938 +0000 2024-07-03T15:37:34.827 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-07-03 15:30:42.708199954 +0000 2024-07-03T15:37:34.827 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-07-03 15:30:42.708199954 +0000 2024-07-03T15:37:34.827 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-07-03 15:30:42.708199954 +0000 2024-07-03T15:37:34.827 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-07-03T15:37:34.889 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-07-03T15:37:34.889 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-07-03T15:37:34.889 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.00019627 s, 2.6 MB/s 2024-07-03T15:37:34.890 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-07-03T15:37:34.947 INFO:tasks.cephadm:Deploying osd.0 on smithi028 with /dev/vg_nvme/lv_4... 2024-07-03T15:37:34.947 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-07-03T15:37:35.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:35 smithi148 ceph-mon[36056]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:35.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:35 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:35.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:35 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:35.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:35 smithi148 ceph-mon[36056]: Reconfiguring mgr.smithi028.vtqrtz (unknown last config time)... 2024-07-03T15:37:35.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:35 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi028.vtqrtz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-03T15:37:35.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:35 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-03T15:37:35.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:35 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:37:35.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:35 smithi148 ceph-mon[36056]: Reconfiguring daemon mgr.smithi028.vtqrtz on smithi028 2024-07-03T15:37:35.791 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:35 smithi028 ceph-mon[29148]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:35.791 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:35 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:35.791 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:35 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:35.791 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:35 smithi028 ceph-mon[29148]: Reconfiguring mgr.smithi028.vtqrtz (unknown last config time)... 2024-07-03T15:37:35.791 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:35 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi028.vtqrtz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-03T15:37:35.791 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:35 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-03T15:37:35.791 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:35 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:37:35.791 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:35 smithi028 ceph-mon[29148]: Reconfiguring daemon mgr.smithi028.vtqrtz on smithi028 2024-07-03T15:37:35.870 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:37:36.931 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:36 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:36.931 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:36 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:36.931 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:36 smithi028 ceph-mon[29148]: Reconfiguring crash.smithi028 (monmap changed)... 2024-07-03T15:37:36.931 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:36 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi028", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-07-03T15:37:36.932 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:36 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:37:36.932 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:36 smithi028 ceph-mon[29148]: Reconfiguring daemon crash.smithi028 on smithi028 2024-07-03T15:37:36.932 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:36 smithi028 ceph-mon[29148]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:37.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:36 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:37.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:36 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:37.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:36 smithi148 ceph-mon[36056]: Reconfiguring crash.smithi028 (monmap changed)... 2024-07-03T15:37:37.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:36 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi028", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-07-03T15:37:37.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:36 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:37:37.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:36 smithi148 ceph-mon[36056]: Reconfiguring daemon crash.smithi028 on smithi028 2024-07-03T15:37:37.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:36 smithi148 ceph-mon[36056]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:38.660 INFO:teuthology.orchestra.run.smithi028.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-07-03T15:37:38.754 INFO:teuthology.orchestra.run.smithi028.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-07-03T15:37:38.765 INFO:teuthology.orchestra.run.smithi028.stderr: stderr: 10+0 records in 2024-07-03T15:37:38.765 INFO:teuthology.orchestra.run.smithi028.stderr:10+0 records out 2024-07-03T15:37:38.765 INFO:teuthology.orchestra.run.smithi028.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00879082 s, 1.2 GB/s 2024-07-03T15:37:38.765 INFO:teuthology.orchestra.run.smithi028.stderr:--> Zapping successful for: 2024-07-03T15:37:39.304 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:39 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:39.305 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:39 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:39.305 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:39 smithi028 ceph-mon[29148]: Reconfiguring alertmanager.smithi028 (dependencies changed)... 2024-07-03T15:37:39.305 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:39 smithi028 ceph-mon[29148]: Reconfiguring daemon alertmanager.smithi028 on smithi028 2024-07-03T15:37:39.305 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:39 smithi028 ceph-mon[29148]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:39.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:39 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:39.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:39 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:39.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:39 smithi148 ceph-mon[36056]: Reconfiguring alertmanager.smithi028 (dependencies changed)... 2024-07-03T15:37:39.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:39 smithi148 ceph-mon[36056]: Reconfiguring daemon alertmanager.smithi028 on smithi028 2024-07-03T15:37:39.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:39 smithi148 ceph-mon[36056]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:40.366 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph orch daemon add osd smithi028:vg_nvme/lv_4 2024-07-03T15:37:41.660 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:37:41.694 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:41 smithi028 ceph-mon[29148]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:41.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:41 smithi148 ceph-mon[36056]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:43.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:43.300+0000 7fb655682640 1 -- 172.21.15.28:0/1511155075 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6500fc510 msgr2=0x7fb6500fc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:37:43.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:43.300+0000 7fb655682640 1 --2- 172.21.15.28:0/1511155075 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6500fc510 0x7fb6500fc980 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb6440048d0 tx=0x7fb644009da0 comp rx=0 tx=0).stop 2024-07-03T15:37:43.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:43.300+0000 7fb655682640 1 -- 172.21.15.28:0/1511155075 shutdown_connections 2024-07-03T15:37:43.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:43.300+0000 7fb655682640 1 --2- 172.21.15.28:0/1511155075 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6500fc510 0x7fb6500fc980 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:37:43.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:43.300+0000 7fb655682640 1 --2- 172.21.15.28:0/1511155075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6500fb240 0x7fb6500fb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:37:43.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:43.300+0000 7fb655682640 1 -- 172.21.15.28:0/1511155075 >> 172.21.15.28:0/1511155075 conn(0x7fb6500f6a20 msgr2=0x7fb6500f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:37:43.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:43.300+0000 7fb655682640 1 -- 172.21.15.28:0/1511155075 shutdown_connections 2024-07-03T15:37:43.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:43.300+0000 7fb655682640 1 -- 172.21.15.28:0/1511155075 wait complete. 2024-07-03T15:37:43.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:43.300+0000 7fb655682640 1 Processor -- start 2024-07-03T15:37:43.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:43.300+0000 7fb655682640 1 -- start start 2024-07-03T15:37:43.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:43.301+0000 7fb655682640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6500fb240 0x7fb65010cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:37:43.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:43.301+0000 7fb655682640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6500fc510 0x7fb65010d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:37:43.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:43.301+0000 7fb655682640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65010e380 con 0x7fb6500fc510 2024-07-03T15:37:43.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:43.301+0000 7fb655682640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65010e4f0 con 0x7fb6500fb240 2024-07-03T15:37:43.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:43.301+0000 7fb64e7fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6500fc510 0x7fb65010d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:37:43.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:43.301+0000 7fb64e7fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6500fc510 0x7fb65010d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49434/0 (socket says 172.21.15.28:49434) 2024-07-03T15:37:43.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:43.301+0000 7fb64e7fc640 1 -- 172.21.15.28:0/1762706035 learned_addr learned my addr 172.21.15.28:0/1762706035 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:37:43.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:43.301+0000 7fb64effd640 1 --2- 172.21.15.28:0/1762706035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6500fb240 0x7fb65010cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:37:43.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:43.301+0000 7fb64e7fc640 1 -- 172.21.15.28:0/1762706035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6500fb240 msgr2=0x7fb65010cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:37:43.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:43.301+0000 7fb64e7fc640 1 --2- 172.21.15.28:0/1762706035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6500fb240 0x7fb65010cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:37:43.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:43.301+0000 7fb64e7fc640 1 -- 172.21.15.28:0/1762706035 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb644005040 con 0x7fb6500fc510 2024-07-03T15:37:43.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:43.301+0000 7fb64e7fc640 1 --2- 172.21.15.28:0/1762706035 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6500fc510 0x7fb65010d510 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb6440048a0 tx=0x7fb644009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:37:43.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:43.301+0000 7fb6377fe640 1 -- 172.21.15.28:0/1762706035 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb644038070 con 0x7fb6500fc510 2024-07-03T15:37:43.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:43.302+0000 7fb6377fe640 1 -- 172.21.15.28:0/1762706035 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb644032970 con 0x7fb6500fc510 2024-07-03T15:37:43.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:43.302+0000 7fb655682640 1 -- 172.21.15.28:0/1762706035 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb65010e770 con 0x7fb6500fc510 2024-07-03T15:37:43.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:43.302+0000 7fb6377fe640 1 -- 172.21.15.28:0/1762706035 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb64403d9a0 con 0x7fb6500fc510 2024-07-03T15:37:43.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:43.302+0000 7fb655682640 1 -- 172.21.15.28:0/1762706035 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb65010ec30 con 0x7fb6500fc510 2024-07-03T15:37:43.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:43.303+0000 7fb655682640 1 -- 172.21.15.28:0/1762706035 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6500fa6a0 con 0x7fb6500fc510 2024-07-03T15:37:43.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:43.303+0000 7fb6377fe640 1 -- 172.21.15.28:0/1762706035 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 18) v1 ==== 85069+0+0 (secure 0 0 0) 0x7fb644030020 con 0x7fb6500fc510 2024-07-03T15:37:43.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:43.303+0000 7fb6377fe640 1 --2- 172.21.15.28:0/1762706035 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fb638067110 0x7fb6380695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:37:43.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:43.304+0000 7fb6377fe640 1 -- 172.21.15.28:0/1762706035 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fb6440a40f0 con 0x7fb6500fc510 2024-07-03T15:37:43.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:43.304+0000 7fb64effd640 1 --2- 172.21.15.28:0/1762706035 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fb638067110 0x7fb6380695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:37:43.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:43.304+0000 7fb64effd640 1 --2- 172.21.15.28:0/1762706035 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fb638067110 0x7fb6380695c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb640006fd0 tx=0x7fb640008040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:37:43.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:43.308+0000 7fb6377fe640 1 -- 172.21.15.28:0/1762706035 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb644072530 con 0x7fb6500fc510 2024-07-03T15:37:43.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:37:43.492+0000 7fb655682640 1 -- 172.21.15.28:0/1762706035 --> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi028:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fb650103e10 con 0x7fb638067110 2024-07-03T15:37:43.495 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:43 smithi028 ceph-mon[29148]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:43.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:43 smithi148 ceph-mon[36056]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:44.467 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:44 smithi028 ceph-mon[29148]: from='client.14242 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi028:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:37:44.467 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:44 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-03T15:37:44.467 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:44 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-03T15:37:44.468 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:44 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:37:44.682 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:44 smithi148 ceph-mon[36056]: from='client.14242 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi028:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:37:44.682 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:44 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-03T15:37:44.682 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:44 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-03T15:37:44.682 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:44 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:37:45.597 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:45 smithi028 ceph-mon[29148]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:45.597 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:45 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:37:45.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:45 smithi148 ceph-mon[36056]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:45.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:45 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:37:47.296 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:47 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:47.296 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:47 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:47.297 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:47 smithi028 ceph-mon[29148]: Reconfiguring grafana.smithi028 (dependencies changed)... 2024-07-03T15:37:47.297 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:47 smithi028 ceph-mon[29148]: Reconfiguring daemon grafana.smithi028 on smithi028 2024-07-03T15:37:47.297 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:47 smithi028 ceph-mon[29148]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:47.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:47 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:47.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:47 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:47.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:47 smithi148 ceph-mon[36056]: Reconfiguring grafana.smithi028 (dependencies changed)... 2024-07-03T15:37:47.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:47 smithi148 ceph-mon[36056]: Reconfiguring daemon grafana.smithi028 on smithi028 2024-07-03T15:37:47.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:47 smithi148 ceph-mon[36056]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:48.370 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:48 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/3042653894' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ecd90b5-3715-4ba7-a696-588843e3582c"}]: dispatch 2024-07-03T15:37:48.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:48 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/3042653894' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7ecd90b5-3715-4ba7-a696-588843e3582c"}]': finished 2024-07-03T15:37:48.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:48 smithi028 ceph-mon[29148]: osdmap e6: 1 total, 0 up, 1 in 2024-07-03T15:37:48.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:48 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-03T15:37:48.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:48 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/58244266' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-03T15:37:48.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:48 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/3042653894' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ecd90b5-3715-4ba7-a696-588843e3582c"}]: dispatch 2024-07-03T15:37:48.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:48 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/3042653894' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7ecd90b5-3715-4ba7-a696-588843e3582c"}]': finished 2024-07-03T15:37:48.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:48 smithi148 ceph-mon[36056]: osdmap e6: 1 total, 0 up, 1 in 2024-07-03T15:37:48.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:48 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-03T15:37:48.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:48 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/58244266' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-03T15:37:49.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:49 smithi028 ceph-mon[29148]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:49.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:49 smithi148 ceph-mon[36056]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:51.604 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:51 smithi028 ceph-mon[29148]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:51.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:51 smithi148 ceph-mon[36056]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:53.620 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:53 smithi028 ceph-mon[29148]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:53.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:53 smithi148 ceph-mon[36056]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:55.562 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:55 smithi028 ceph-mon[29148]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:55.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:55 smithi148 ceph-mon[36056]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:57.590 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:57 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:57.590 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:57 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:57.590 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:57 smithi028 ceph-mon[29148]: Reconfiguring prometheus.smithi028 (dependencies changed)... 2024-07-03T15:37:57.591 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:57 smithi028 ceph-mon[29148]: Reconfiguring daemon prometheus.smithi028 on smithi028 2024-07-03T15:37:57.591 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:57 smithi028 ceph-mon[29148]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:57.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:57 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:57.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:57 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:37:57.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:57 smithi148 ceph-mon[36056]: Reconfiguring prometheus.smithi028 (dependencies changed)... 2024-07-03T15:37:57.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:57 smithi148 ceph-mon[36056]: Reconfiguring daemon prometheus.smithi028 on smithi028 2024-07-03T15:37:57.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:57 smithi148 ceph-mon[36056]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:58.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:58 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-07-03T15:37:58.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:58 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:37:58.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:58 smithi028 ceph-mon[29148]: Deploying daemon osd.0 on smithi028 2024-07-03T15:37:58.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:58 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-07-03T15:37:58.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:58 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:37:58.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:58 smithi148 ceph-mon[36056]: Deploying daemon osd.0 on smithi028 2024-07-03T15:37:59.381 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:37:59 smithi028 ceph-mon[29148]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:37:59.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:37:59 smithi148 ceph-mon[36056]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:38:00.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:00 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:38:00.755 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:00 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:38:01.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:01 smithi148 ceph-mon[36056]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:38:01.733 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:01 smithi028 ceph-mon[29148]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:38:02.990 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:02 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:02.990 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:02 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:02.990 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:02 smithi028 ceph-mon[29148]: Reconfiguring crash.smithi148 (monmap changed)... 2024-07-03T15:38:02.990 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:02 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi148", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-07-03T15:38:02.990 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:02 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:38:02.990 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:02 smithi028 ceph-mon[29148]: Reconfiguring daemon crash.smithi148 on smithi148 2024-07-03T15:38:02.991 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:02 smithi028 ceph-mon[29148]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:38:03.131 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:02 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:03.132 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:02 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:03.132 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:02 smithi148 ceph-mon[36056]: Reconfiguring crash.smithi148 (monmap changed)... 2024-07-03T15:38:03.132 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:02 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi148", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-07-03T15:38:03.132 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:02 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:38:03.132 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:02 smithi148 ceph-mon[36056]: Reconfiguring daemon crash.smithi148 on smithi148 2024-07-03T15:38:03.132 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:02 smithi148 ceph-mon[36056]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:38:04.308 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:04 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:04.308 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:04 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:04.308 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:04 smithi028 ceph-mon[29148]: Reconfiguring mgr.smithi148.rjzsar (monmap changed)... 2024-07-03T15:38:04.308 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:04 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi148.rjzsar", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-03T15:38:04.308 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:04 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-03T15:38:04.308 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:04 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:38:04.308 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:04 smithi028 ceph-mon[29148]: Reconfiguring daemon mgr.smithi148.rjzsar on smithi148 2024-07-03T15:38:04.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:04 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:04.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:04 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:04.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:04 smithi148 ceph-mon[36056]: Reconfiguring mgr.smithi148.rjzsar (monmap changed)... 2024-07-03T15:38:04.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:04 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi148.rjzsar", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-03T15:38:04.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:04 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-03T15:38:04.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:04 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:38:04.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:04 smithi148 ceph-mon[36056]: Reconfiguring daemon mgr.smithi148.rjzsar on smithi148 2024-07-03T15:38:05.578 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:05 smithi148 ceph-mon[36056]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:38:05.578 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:05 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:05.578 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:05 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:05.578 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:05 smithi148 ceph-mon[36056]: Reconfiguring mon.smithi148 (monmap changed)... 2024-07-03T15:38:05.578 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:05 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-03T15:38:05.578 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:05 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-03T15:38:05.578 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:05 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:38:05.578 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:05 smithi148 ceph-mon[36056]: Reconfiguring daemon mon.smithi148 on smithi148 2024-07-03T15:38:05.759 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:05 smithi028 ceph-mon[29148]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:38:05.759 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:05 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:05.759 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:05 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:05.759 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:05 smithi028 ceph-mon[29148]: Reconfiguring mon.smithi148 (monmap changed)... 2024-07-03T15:38:05.759 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:05 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-03T15:38:05.759 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:05 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-03T15:38:05.759 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:05 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:38:05.759 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:05 smithi028 ceph-mon[29148]: Reconfiguring daemon mon.smithi148 on smithi148 2024-07-03T15:38:06.832 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:06 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:06.832 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:06 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:06.832 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:06 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-07-03T15:38:06.832 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:06 smithi028 ceph-mon[29148]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-07-03T15:38:06.832 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:06 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi028.front.sepia.ceph.com:9093"}]: dispatch 2024-07-03T15:38:06.832 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:06 smithi028 ceph-mon[29148]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi028.front.sepia.ceph.com:9093"}]: dispatch 2024-07-03T15:38:06.832 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:06 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:06.832 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:06 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-07-03T15:38:06.832 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:06 smithi028 ceph-mon[29148]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-07-03T15:38:06.833 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:06 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi028.front.sepia.ceph.com:3000"}]: dispatch 2024-07-03T15:38:06.833 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:06 smithi028 ceph-mon[29148]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi028.front.sepia.ceph.com:3000"}]: dispatch 2024-07-03T15:38:06.833 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:06 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:06.833 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:06 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-07-03T15:38:06.833 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:06 smithi028 ceph-mon[29148]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-07-03T15:38:06.833 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:06 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi028.front.sepia.ceph.com:9095"}]: dispatch 2024-07-03T15:38:06.833 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:06 smithi028 ceph-mon[29148]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi028.front.sepia.ceph.com:9095"}]: dispatch 2024-07-03T15:38:06.833 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:06 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:06.833 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:06 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:38:06.933 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:06 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:06.933 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:06 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:06.933 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:06 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-07-03T15:38:06.933 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:06 smithi148 ceph-mon[36056]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-07-03T15:38:06.933 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:06 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi028.front.sepia.ceph.com:9093"}]: dispatch 2024-07-03T15:38:06.933 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:06 smithi148 ceph-mon[36056]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi028.front.sepia.ceph.com:9093"}]: dispatch 2024-07-03T15:38:06.934 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:06 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:06.934 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:06 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-07-03T15:38:06.934 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:06 smithi148 ceph-mon[36056]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-07-03T15:38:06.934 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:06 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi028.front.sepia.ceph.com:3000"}]: dispatch 2024-07-03T15:38:06.934 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:06 smithi148 ceph-mon[36056]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi028.front.sepia.ceph.com:3000"}]: dispatch 2024-07-03T15:38:06.934 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:06 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:06.934 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:06 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-07-03T15:38:06.934 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:06 smithi148 ceph-mon[36056]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-07-03T15:38:06.934 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:06 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi028.front.sepia.ceph.com:9095"}]: dispatch 2024-07-03T15:38:06.934 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:06 smithi148 ceph-mon[36056]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi028.front.sepia.ceph.com:9095"}]: dispatch 2024-07-03T15:38:06.934 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:06 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:06.934 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:06 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:38:07.820 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:07 smithi028 ceph-mon[29148]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:38:07.820 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:07 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:07.821 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:07 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:07.821 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:07 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:07.821 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:07 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:07.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:07 smithi148 ceph-mon[36056]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:38:07.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:07 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:07.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:07 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:07.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:07 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:07.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:07 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:09.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:09 smithi148 ceph-mon[36056]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:38:10.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:09 smithi028 ceph-mon[29148]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:38:10.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:10.615+0000 7fb6377fe640 1 -- 172.21.15.28:0/1762706035 <== mgr.14210 v2:172.21.15.28:6800/421057178 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb650103e10 con 0x7fb638067110 2024-07-03T15:38:10.618 INFO:teuthology.orchestra.run.smithi028.stdout:Created osd(s) 0 on host 'smithi028' 2024-07-03T15:38:10.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:10.617+0000 7fb655682640 1 -- 172.21.15.28:0/1762706035 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fb638067110 msgr2=0x7fb6380695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:38:10.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:10.617+0000 7fb655682640 1 --2- 172.21.15.28:0/1762706035 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fb638067110 0x7fb6380695c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb640006fd0 tx=0x7fb640008040 comp rx=0 tx=0).stop 2024-07-03T15:38:10.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:10.617+0000 7fb655682640 1 -- 172.21.15.28:0/1762706035 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6500fc510 msgr2=0x7fb65010d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:38:10.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:10.617+0000 7fb655682640 1 --2- 172.21.15.28:0/1762706035 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6500fc510 0x7fb65010d510 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb6440048a0 tx=0x7fb644009da0 comp rx=0 tx=0).stop 2024-07-03T15:38:10.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:10.617+0000 7fb655682640 1 -- 172.21.15.28:0/1762706035 shutdown_connections 2024-07-03T15:38:10.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:10.617+0000 7fb655682640 1 --2- 172.21.15.28:0/1762706035 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fb638067110 0x7fb6380695c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:38:10.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:10.617+0000 7fb655682640 1 --2- 172.21.15.28:0/1762706035 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6500fc510 0x7fb65010d510 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:38:10.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:10.617+0000 7fb655682640 1 --2- 172.21.15.28:0/1762706035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6500fb240 0x7fb65010cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:38:10.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:10.617+0000 7fb655682640 1 -- 172.21.15.28:0/1762706035 >> 172.21.15.28:0/1762706035 conn(0x7fb6500f6a20 msgr2=0x7fb6500f80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:38:10.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:10.617+0000 7fb655682640 1 -- 172.21.15.28:0/1762706035 shutdown_connections 2024-07-03T15:38:10.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:10.617+0000 7fb655682640 1 -- 172.21.15.28:0/1762706035 wait complete. 2024-07-03T15:38:10.858 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:10 smithi028 ceph-mon[29148]: from='osd.0 [v2:172.21.15.28:6802/3973972034,v1:172.21.15.28:6803/3973972034]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-07-03T15:38:10.859 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:10 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:10.859 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:10 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:10.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:10 smithi148 ceph-mon[36056]: from='osd.0 [v2:172.21.15.28:6802/3973972034,v1:172.21.15.28:6803/3973972034]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-07-03T15:38:10.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:10 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:10.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:10 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:11.501 DEBUG:teuthology.orchestra.run.smithi028:osd.0> sudo journalctl -f -n 0 -u ceph-a85e4be6-3951-11ef-bcab-c7b262605968@osd.0.service 2024-07-03T15:38:11.504 INFO:tasks.cephadm:Deploying osd.1 on smithi028 with /dev/vg_nvme/lv_3... 2024-07-03T15:38:11.505 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-07-03T15:38:11.727 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:11 smithi028 ceph-mon[29148]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:38:11.727 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:11 smithi028 ceph-mon[29148]: from='osd.0 [v2:172.21.15.28:6802/3973972034,v1:172.21.15.28:6803/3973972034]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-07-03T15:38:11.727 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:11 smithi028 ceph-mon[29148]: osdmap e7: 1 total, 0 up, 1 in 2024-07-03T15:38:11.727 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:11 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-03T15:38:11.727 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:11 smithi028 ceph-mon[29148]: from='osd.0 [v2:172.21.15.28:6802/3973972034,v1:172.21.15.28:6803/3973972034]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi028", "root=default"]}]: dispatch 2024-07-03T15:38:11.727 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:38:11 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-0[47392]: 2024-07-03T15:38:11.719+0000 7f7f2db27640 -1 osd.0 0 waiting for initial osdmap 2024-07-03T15:38:11.727 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:38:11 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-0[47392]: 2024-07-03T15:38:11.724+0000 7f7f29960640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-03T15:38:11.898 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:38:11.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:11 smithi148 ceph-mon[36056]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:38:11.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:11 smithi148 ceph-mon[36056]: from='osd.0 [v2:172.21.15.28:6802/3973972034,v1:172.21.15.28:6803/3973972034]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-07-03T15:38:11.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:11 smithi148 ceph-mon[36056]: osdmap e7: 1 total, 0 up, 1 in 2024-07-03T15:38:11.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:11 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-03T15:38:11.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:11 smithi148 ceph-mon[36056]: from='osd.0 [v2:172.21.15.28:6802/3973972034,v1:172.21.15.28:6803/3973972034]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi028", "root=default"]}]: dispatch 2024-07-03T15:38:12.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:12 smithi148 ceph-mon[36056]: from='osd.0 [v2:172.21.15.28:6802/3973972034,v1:172.21.15.28:6803/3973972034]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi028", "root=default"]}]': finished 2024-07-03T15:38:12.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:12 smithi148 ceph-mon[36056]: osdmap e8: 1 total, 0 up, 1 in 2024-07-03T15:38:12.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:12 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-03T15:38:12.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:12 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-03T15:38:12.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:12 smithi148 ceph-mon[36056]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:38:13.012 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:12 smithi028 ceph-mon[29148]: from='osd.0 [v2:172.21.15.28:6802/3973972034,v1:172.21.15.28:6803/3973972034]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi028", "root=default"]}]': finished 2024-07-03T15:38:13.012 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:12 smithi028 ceph-mon[29148]: osdmap e8: 1 total, 0 up, 1 in 2024-07-03T15:38:13.012 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:12 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-03T15:38:13.012 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:12 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-03T15:38:13.012 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:12 smithi028 ceph-mon[29148]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-03T15:38:13.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:13 smithi148 ceph-mon[36056]: purged_snaps scrub starts 2024-07-03T15:38:13.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:13 smithi148 ceph-mon[36056]: purged_snaps scrub ok 2024-07-03T15:38:13.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:13 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-03T15:38:13.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:13 smithi148 ceph-mon[36056]: osd.0 [v2:172.21.15.28:6802/3973972034,v1:172.21.15.28:6803/3973972034] boot 2024-07-03T15:38:13.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:13 smithi148 ceph-mon[36056]: osdmap e9: 1 total, 1 up, 1 in 2024-07-03T15:38:13.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:13 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-03T15:38:13.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:13 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:13.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:13 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:14.059 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:13 smithi028 ceph-mon[29148]: purged_snaps scrub starts 2024-07-03T15:38:14.059 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:13 smithi028 ceph-mon[29148]: purged_snaps scrub ok 2024-07-03T15:38:14.059 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:13 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-03T15:38:14.059 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:13 smithi028 ceph-mon[29148]: osd.0 [v2:172.21.15.28:6802/3973972034,v1:172.21.15.28:6803/3973972034] boot 2024-07-03T15:38:14.059 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:13 smithi028 ceph-mon[29148]: osdmap e9: 1 total, 1 up, 1 in 2024-07-03T15:38:14.060 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:13 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-03T15:38:14.060 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:13 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:14.060 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:13 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:15.120 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:14 smithi028 ceph-mon[29148]: osdmap e10: 1 total, 1 up, 1 in 2024-07-03T15:38:15.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:14 smithi028 ceph-mon[29148]: pgmap v35: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-03T15:38:15.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:14 smithi148 ceph-mon[36056]: osdmap e10: 1 total, 1 up, 1 in 2024-07-03T15:38:15.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:14 smithi148 ceph-mon[36056]: pgmap v35: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-03T15:38:15.292 INFO:teuthology.orchestra.run.smithi028.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-07-03T15:38:15.387 INFO:teuthology.orchestra.run.smithi028.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-07-03T15:38:15.397 INFO:teuthology.orchestra.run.smithi028.stderr: stderr: 10+0 records in 2024-07-03T15:38:15.397 INFO:teuthology.orchestra.run.smithi028.stderr:10+0 records out 2024-07-03T15:38:15.398 INFO:teuthology.orchestra.run.smithi028.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00854241 s, 1.2 GB/s 2024-07-03T15:38:15.398 INFO:teuthology.orchestra.run.smithi028.stderr:--> Zapping successful for: 2024-07-03T15:38:15.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:15 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:38:16.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:15 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:38:16.175 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph orch daemon add osd smithi028:vg_nvme/lv_3 2024-07-03T15:38:16.646 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:38:17.035 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:16 smithi028 ceph-mon[29148]: pgmap v36: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-03T15:38:17.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:16 smithi148 ceph-mon[36056]: pgmap v36: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-03T15:38:18.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:18.953+0000 7fd62b605640 1 -- 172.21.15.28:0/4212855445 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6240fc330 msgr2=0x7fd6240fc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:38:18.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:18.953+0000 7fd62b605640 1 --2- 172.21.15.28:0/4212855445 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6240fc330 0x7fd6240fc740 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd6200048d0 tx=0x7fd620009da0 comp rx=0 tx=0).stop 2024-07-03T15:38:18.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:18.953+0000 7fd62b605640 1 -- 172.21.15.28:0/4212855445 shutdown_connections 2024-07-03T15:38:18.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:18.953+0000 7fd62b605640 1 --2- 172.21.15.28:0/4212855445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6240fd600 0x7fd6240fda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:38:18.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:18.953+0000 7fd62b605640 1 --2- 172.21.15.28:0/4212855445 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6240fc330 0x7fd6240fc740 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:38:18.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:18.953+0000 7fd62b605640 1 -- 172.21.15.28:0/4212855445 >> 172.21.15.28:0/4212855445 conn(0x7fd6240f7af0 msgr2=0x7fd6240f9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:38:18.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:18.953+0000 7fd62b605640 1 -- 172.21.15.28:0/4212855445 shutdown_connections 2024-07-03T15:38:18.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:18.953+0000 7fd62b605640 1 -- 172.21.15.28:0/4212855445 wait complete. 2024-07-03T15:38:18.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:18.953+0000 7fd62b605640 1 Processor -- start 2024-07-03T15:38:18.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:18.953+0000 7fd62b605640 1 -- start start 2024-07-03T15:38:18.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:18.954+0000 7fd62b605640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6240fc330 0x7fd62410f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:38:18.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:18.954+0000 7fd62b605640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6240fd600 0x7fd62410f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:38:18.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:18.954+0000 7fd62b605640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62410fcf0 con 0x7fd6240fc330 2024-07-03T15:38:18.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:18.954+0000 7fd62b605640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd624110510 con 0x7fd6240fd600 2024-07-03T15:38:18.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:18.954+0000 7fd6293a1640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6240fc330 0x7fd62410f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:38:18.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:18.954+0000 7fd6293a1640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6240fc330 0x7fd62410f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50716/0 (socket says 172.21.15.28:50716) 2024-07-03T15:38:18.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:18.954+0000 7fd6293a1640 1 -- 172.21.15.28:0/968865384 learned_addr learned my addr 172.21.15.28:0/968865384 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:38:18.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:18.954+0000 7fd628ba0640 1 --2- 172.21.15.28:0/968865384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6240fd600 0x7fd62410f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:38:18.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:18.954+0000 7fd6293a1640 1 -- 172.21.15.28:0/968865384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6240fd600 msgr2=0x7fd62410f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:38:18.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:18.954+0000 7fd6293a1640 1 --2- 172.21.15.28:0/968865384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6240fd600 0x7fd62410f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:38:18.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:18.954+0000 7fd6293a1640 1 -- 172.21.15.28:0/968865384 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd614005040 con 0x7fd6240fc330 2024-07-03T15:38:18.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:18.954+0000 7fd6293a1640 1 --2- 172.21.15.28:0/968865384 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6240fc330 0x7fd62410f160 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd6200041d0 tx=0x7fd620009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:38:18.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:18.955+0000 7fd619ffb640 1 -- 172.21.15.28:0/968865384 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd620038070 con 0x7fd6240fc330 2024-07-03T15:38:18.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:18.955+0000 7fd619ffb640 1 -- 172.21.15.28:0/968865384 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd6200092b0 con 0x7fd6240fc330 2024-07-03T15:38:18.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:18.955+0000 7fd619ffb640 1 -- 172.21.15.28:0/968865384 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd620033710 con 0x7fd6240fc330 2024-07-03T15:38:18.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:18.955+0000 7fd62b605640 1 -- 172.21.15.28:0/968865384 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd620005040 con 0x7fd6240fc330 2024-07-03T15:38:18.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:18.955+0000 7fd62b605640 1 -- 172.21.15.28:0/968865384 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd624110af0 con 0x7fd6240fc330 2024-07-03T15:38:18.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:18.956+0000 7fd62b605640 1 -- 172.21.15.28:0/968865384 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd624105690 con 0x7fd6240fc330 2024-07-03T15:38:18.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:18.956+0000 7fd619ffb640 1 -- 172.21.15.28:0/968865384 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 18) v1 ==== 85069+0+0 (secure 0 0 0) 0x7fd620030020 con 0x7fd6240fc330 2024-07-03T15:38:18.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:18.957+0000 7fd619ffb640 1 --2- 172.21.15.28:0/968865384 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fd610067110 0x7fd6100695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:38:18.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:18.957+0000 7fd619ffb640 1 -- 172.21.15.28:0/968865384 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1978+0+0 (secure 0 0 0) 0x7fd6200a3940 con 0x7fd6240fc330 2024-07-03T15:38:18.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:18.957+0000 7fd628ba0640 1 --2- 172.21.15.28:0/968865384 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fd610067110 0x7fd6100695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:38:18.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:18.957+0000 7fd628ba0640 1 --2- 172.21.15.28:0/968865384 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fd610067110 0x7fd6100695c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd62410ef10 tx=0x7fd61400b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:38:18.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:18.961+0000 7fd619ffb640 1 -- 172.21.15.28:0/968865384 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd620071b90 con 0x7fd6240fc330 2024-07-03T15:38:19.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:19.157+0000 7fd62b605640 1 -- 172.21.15.28:0/968865384 --> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi028:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fd624104f60 con 0x7fd610067110 2024-07-03T15:38:19.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:18 smithi148 ceph-mon[36056]: Detected new or changed devices on smithi028 2024-07-03T15:38:19.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:18 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:19.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:18 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:19.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:18 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:38:19.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:18 smithi148 ceph-mon[36056]: Adjusting osd_memory_target on smithi028 to 12868M 2024-07-03T15:38:19.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:18 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:19.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:18 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:38:19.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:18 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:38:19.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:18 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:19.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:18 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:38:19.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:18 smithi148 ceph-mon[36056]: pgmap v37: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-03T15:38:19.246 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:18 smithi028 ceph-mon[29148]: Detected new or changed devices on smithi028 2024-07-03T15:38:19.247 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:18 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:19.247 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:18 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:19.247 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:18 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:38:19.247 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:18 smithi028 ceph-mon[29148]: Adjusting osd_memory_target on smithi028 to 12868M 2024-07-03T15:38:19.247 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:18 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:19.247 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:18 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:38:19.247 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:18 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:38:19.247 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:18 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:19.247 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:18 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:38:19.247 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:18 smithi028 ceph-mon[29148]: pgmap v37: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-03T15:38:20.175 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:20 smithi028 ceph-mon[29148]: from='client.14260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi028:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:38:20.176 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:20 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-03T15:38:20.176 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:20 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-03T15:38:20.176 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:20 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:38:20.176 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:20 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:20.176 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:20 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:20.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:20 smithi148 ceph-mon[36056]: from='client.14260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi028:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:38:20.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:20 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-03T15:38:20.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:20 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-03T15:38:20.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:20 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:38:20.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:20 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:20.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:20 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:21.119 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:21 smithi028 ceph-mon[29148]: pgmap v38: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-03T15:38:21.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:21 smithi148 ceph-mon[36056]: pgmap v38: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-03T15:38:22.090 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:22 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/2999575212' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31c5ae0c-6c57-4615-902b-72bf49fa5b72"}]: dispatch 2024-07-03T15:38:22.090 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:22 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/2999575212' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "31c5ae0c-6c57-4615-902b-72bf49fa5b72"}]': finished 2024-07-03T15:38:22.091 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:22 smithi028 ceph-mon[29148]: osdmap e11: 2 total, 1 up, 2 in 2024-07-03T15:38:22.091 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:22 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-03T15:38:22.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:22 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/2999575212' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31c5ae0c-6c57-4615-902b-72bf49fa5b72"}]: dispatch 2024-07-03T15:38:22.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:22 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/2999575212' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "31c5ae0c-6c57-4615-902b-72bf49fa5b72"}]': finished 2024-07-03T15:38:22.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:22 smithi148 ceph-mon[36056]: osdmap e11: 2 total, 1 up, 2 in 2024-07-03T15:38:22.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:22 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-03T15:38:23.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:23 smithi028 ceph-mon[29148]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-03T15:38:23.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:23 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/1347228385' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-03T15:38:23.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:23 smithi148 ceph-mon[36056]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-03T15:38:23.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:23 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/1347228385' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-03T15:38:25.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:25 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:25.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:25 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:25.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:25 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:38:25.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:25 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:38:25.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:25 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:25.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:25 smithi028 ceph-mon[29148]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-03T15:38:25.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:25 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:25.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:25 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:25.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:25 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:38:25.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:25 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:38:25.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:25 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:25.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:25 smithi148 ceph-mon[36056]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-03T15:38:27.440 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:27 smithi028 ceph-mon[29148]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-03T15:38:27.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:27 smithi148 ceph-mon[36056]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-03T15:38:29.538 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:29 smithi028 ceph-mon[29148]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-03T15:38:29.538 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:29 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-07-03T15:38:29.539 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:29 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:38:29.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:29 smithi148 ceph-mon[36056]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-03T15:38:29.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:29 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-07-03T15:38:29.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:29 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:38:30.661 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:30 smithi028 ceph-mon[29148]: Deploying daemon osd.1 on smithi028 2024-07-03T15:38:30.661 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:30 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:38:30.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:30 smithi148 ceph-mon[36056]: Deploying daemon osd.1 on smithi028 2024-07-03T15:38:30.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:30 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:38:31.618 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:31 smithi028 ceph-mon[29148]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-03T15:38:31.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:31 smithi148 ceph-mon[36056]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-03T15:38:33.438 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:33 smithi028 ceph-mon[29148]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-03T15:38:33.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:33 smithi148 ceph-mon[36056]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-03T15:38:34.501 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:34 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:38:34.501 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:34 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:34.501 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:34 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:34.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:34 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:38:34.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:34 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:34.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:34 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:35.538 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:35 smithi028 ceph-mon[29148]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-03T15:38:35.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:35 smithi148 ceph-mon[36056]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-03T15:38:36.481 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:36 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:36.482 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:36 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:36.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:36 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:36.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:36 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:37.690 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:37 smithi028 ceph-mon[29148]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-03T15:38:37.727 INFO:teuthology.orchestra.run.smithi028.stdout:Created osd(s) 1 on host 'smithi028' 2024-07-03T15:38:37.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:37.724+0000 7fd619ffb640 1 -- 172.21.15.28:0/968865384 <== mgr.14210 v2:172.21.15.28:6800/421057178 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd624104f60 con 0x7fd610067110 2024-07-03T15:38:37.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:37.727+0000 7fd62b605640 1 -- 172.21.15.28:0/968865384 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fd610067110 msgr2=0x7fd6100695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:38:37.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:37.727+0000 7fd62b605640 1 --2- 172.21.15.28:0/968865384 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fd610067110 0x7fd6100695c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd62410ef10 tx=0x7fd61400b040 comp rx=0 tx=0).stop 2024-07-03T15:38:37.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:37.727+0000 7fd62b605640 1 -- 172.21.15.28:0/968865384 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6240fc330 msgr2=0x7fd62410f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:38:37.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:37.727+0000 7fd62b605640 1 --2- 172.21.15.28:0/968865384 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6240fc330 0x7fd62410f160 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd6200041d0 tx=0x7fd620009da0 comp rx=0 tx=0).stop 2024-07-03T15:38:37.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:37.727+0000 7fd62b605640 1 -- 172.21.15.28:0/968865384 shutdown_connections 2024-07-03T15:38:37.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:37.727+0000 7fd62b605640 1 --2- 172.21.15.28:0/968865384 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fd610067110 0x7fd6100695c0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:38:37.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:37.727+0000 7fd62b605640 1 --2- 172.21.15.28:0/968865384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6240fd600 0x7fd62410f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:38:37.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:37.727+0000 7fd62b605640 1 --2- 172.21.15.28:0/968865384 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6240fc330 0x7fd62410f160 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:38:37.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:37.727+0000 7fd62b605640 1 -- 172.21.15.28:0/968865384 >> 172.21.15.28:0/968865384 conn(0x7fd6240f7af0 msgr2=0x7fd6240f90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:38:37.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:37.727+0000 7fd62b605640 1 -- 172.21.15.28:0/968865384 shutdown_connections 2024-07-03T15:38:37.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:37.727+0000 7fd62b605640 1 -- 172.21.15.28:0/968865384 wait complete. 2024-07-03T15:38:37.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:37 smithi148 ceph-mon[36056]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-03T15:38:38.341 DEBUG:teuthology.orchestra.run.smithi028:osd.1> sudo journalctl -f -n 0 -u ceph-a85e4be6-3951-11ef-bcab-c7b262605968@osd.1.service 2024-07-03T15:38:38.344 INFO:tasks.cephadm:Deploying osd.2 on smithi028 with /dev/vg_nvme/lv_2... 2024-07-03T15:38:38.345 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-07-03T15:38:38.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:38 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:38.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:38 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:38.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:38 smithi028 ceph-mon[29148]: from='osd.1 [v2:172.21.15.28:6810/1642389568,v1:172.21.15.28:6811/1642389568]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-07-03T15:38:38.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:38 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:38.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:38 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:38.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:38 smithi148 ceph-mon[36056]: from='osd.1 [v2:172.21.15.28:6810/1642389568,v1:172.21.15.28:6811/1642389568]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-07-03T15:38:39.018 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:38:39.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:39 smithi148 ceph-mon[36056]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-03T15:38:39.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:39 smithi148 ceph-mon[36056]: from='osd.1 [v2:172.21.15.28:6810/1642389568,v1:172.21.15.28:6811/1642389568]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-07-03T15:38:39.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:39 smithi148 ceph-mon[36056]: osdmap e12: 2 total, 1 up, 2 in 2024-07-03T15:38:39.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:39 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-03T15:38:39.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:39 smithi148 ceph-mon[36056]: from='osd.1 [v2:172.21.15.28:6810/1642389568,v1:172.21.15.28:6811/1642389568]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi028", "root=default"]}]: dispatch 2024-07-03T15:38:40.025 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:39 smithi028 ceph-mon[29148]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-03T15:38:40.026 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:39 smithi028 ceph-mon[29148]: from='osd.1 [v2:172.21.15.28:6810/1642389568,v1:172.21.15.28:6811/1642389568]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-07-03T15:38:40.026 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:39 smithi028 ceph-mon[29148]: osdmap e12: 2 total, 1 up, 2 in 2024-07-03T15:38:40.026 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:39 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-03T15:38:40.026 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:39 smithi028 ceph-mon[29148]: from='osd.1 [v2:172.21.15.28:6810/1642389568,v1:172.21.15.28:6811/1642389568]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi028", "root=default"]}]: dispatch 2024-07-03T15:38:40.026 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:38:39 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-1[54714]: 2024-07-03T15:38:39.730+0000 7fe43ad9d640 -1 osd.1 0 waiting for initial osdmap 2024-07-03T15:38:40.026 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:38:39 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-1[54714]: 2024-07-03T15:38:39.734+0000 7fe435bd4640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-03T15:38:41.031 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:40 smithi028 ceph-mon[29148]: from='osd.1 [v2:172.21.15.28:6810/1642389568,v1:172.21.15.28:6811/1642389568]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi028", "root=default"]}]': finished 2024-07-03T15:38:41.032 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:40 smithi028 ceph-mon[29148]: osdmap e13: 2 total, 1 up, 2 in 2024-07-03T15:38:41.032 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:40 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-03T15:38:41.032 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:40 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-03T15:38:41.032 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:40 smithi028 ceph-mon[29148]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-03T15:38:41.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:40 smithi148 ceph-mon[36056]: from='osd.1 [v2:172.21.15.28:6810/1642389568,v1:172.21.15.28:6811/1642389568]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi028", "root=default"]}]': finished 2024-07-03T15:38:41.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:40 smithi148 ceph-mon[36056]: osdmap e13: 2 total, 1 up, 2 in 2024-07-03T15:38:41.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:40 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-03T15:38:41.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:40 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-03T15:38:41.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:40 smithi148 ceph-mon[36056]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-03T15:38:41.378 INFO:teuthology.orchestra.run.smithi028.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-07-03T15:38:41.469 INFO:teuthology.orchestra.run.smithi028.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-07-03T15:38:41.481 INFO:teuthology.orchestra.run.smithi028.stderr: stderr: 10+0 records in 2024-07-03T15:38:41.481 INFO:teuthology.orchestra.run.smithi028.stderr:10+0 records out 2024-07-03T15:38:41.481 INFO:teuthology.orchestra.run.smithi028.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00985325 s, 1.1 GB/s 2024-07-03T15:38:41.481 INFO:teuthology.orchestra.run.smithi028.stderr:--> Zapping successful for: 2024-07-03T15:38:42.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:41 smithi028 ceph-mon[29148]: purged_snaps scrub starts 2024-07-03T15:38:42.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:41 smithi028 ceph-mon[29148]: purged_snaps scrub ok 2024-07-03T15:38:42.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:41 smithi028 ceph-mon[29148]: osd.1 [v2:172.21.15.28:6810/1642389568,v1:172.21.15.28:6811/1642389568] boot 2024-07-03T15:38:42.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:41 smithi028 ceph-mon[29148]: osdmap e14: 2 total, 2 up, 2 in 2024-07-03T15:38:42.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:41 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-03T15:38:42.165 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph orch daemon add osd smithi028:vg_nvme/lv_2 2024-07-03T15:38:42.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:41 smithi148 ceph-mon[36056]: purged_snaps scrub starts 2024-07-03T15:38:42.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:41 smithi148 ceph-mon[36056]: purged_snaps scrub ok 2024-07-03T15:38:42.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:41 smithi148 ceph-mon[36056]: osd.1 [v2:172.21.15.28:6810/1642389568,v1:172.21.15.28:6811/1642389568] boot 2024-07-03T15:38:42.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:41 smithi148 ceph-mon[36056]: osdmap e14: 2 total, 2 up, 2 in 2024-07-03T15:38:42.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:41 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-03T15:38:42.574 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:38:42.948 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:42 smithi028 ceph-mon[29148]: osdmap e15: 2 total, 2 up, 2 in 2024-07-03T15:38:42.949 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:42 smithi028 ceph-mon[29148]: pgmap v54: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-03T15:38:42.949 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:42 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:42.949 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:42 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:43.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:42 smithi148 ceph-mon[36056]: osdmap e15: 2 total, 2 up, 2 in 2024-07-03T15:38:43.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:42 smithi148 ceph-mon[36056]: pgmap v54: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-03T15:38:43.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:42 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:43.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:42 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:44.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:44.835+0000 7f01df48f640 1 -- 172.21.15.28:0/2220987494 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f01d81018b0 msgr2=0x7f01d8103c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:38:44.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:44.835+0000 7f01df48f640 1 --2- 172.21.15.28:0/2220987494 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f01d81018b0 0x7f01d8103c90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f01c80048d0 tx=0x7f01c8009da0 comp rx=0 tx=0).stop 2024-07-03T15:38:44.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:44.835+0000 7f01df48f640 1 -- 172.21.15.28:0/2220987494 shutdown_connections 2024-07-03T15:38:44.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:44.835+0000 7f01df48f640 1 --2- 172.21.15.28:0/2220987494 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f01d81018b0 0x7f01d8103c90 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:38:44.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:44.836+0000 7f01df48f640 1 --2- 172.21.15.28:0/2220987494 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01d8066520 0x7f01d8101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:38:44.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:44.836+0000 7f01df48f640 1 -- 172.21.15.28:0/2220987494 >> 172.21.15.28:0/2220987494 conn(0x7f01d80f6a20 msgr2=0x7f01d80f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:38:44.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:44.836+0000 7f01df48f640 1 -- 172.21.15.28:0/2220987494 shutdown_connections 2024-07-03T15:38:44.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:44.836+0000 7f01df48f640 1 -- 172.21.15.28:0/2220987494 wait complete. 2024-07-03T15:38:44.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:44.836+0000 7f01df48f640 1 Processor -- start 2024-07-03T15:38:44.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:44.836+0000 7f01df48f640 1 -- start start 2024-07-03T15:38:44.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:44.836+0000 7f01df48f640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f01d8066520 0x7f01d810f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:38:44.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:44.836+0000 7f01df48f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01d81018b0 0x7f01d810f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:38:44.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:44.836+0000 7f01df48f640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d8110620 con 0x7f01d8066520 2024-07-03T15:38:44.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:44.836+0000 7f01df48f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d8110790 con 0x7f01d81018b0 2024-07-03T15:38:44.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:44.837+0000 7f01dd22b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f01d8066520 0x7f01d810f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:38:44.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:44.837+0000 7f01dd22b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f01d8066520 0x7f01d810f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39938/0 (socket says 172.21.15.28:39938) 2024-07-03T15:38:44.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:44.837+0000 7f01dd22b640 1 -- 172.21.15.28:0/1419335504 learned_addr learned my addr 172.21.15.28:0/1419335504 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:38:44.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:44.837+0000 7f01dd22b640 1 -- 172.21.15.28:0/1419335504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01d81018b0 msgr2=0x7f01d810f7b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-03T15:38:44.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:44.837+0000 7f01dd22b640 1 --2- 172.21.15.28:0/1419335504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01d81018b0 0x7f01d810f7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:38:44.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:44.837+0000 7f01dd22b640 1 -- 172.21.15.28:0/1419335504 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c8005040 con 0x7f01d8066520 2024-07-03T15:38:44.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:44.837+0000 7f01dd22b640 1 --2- 172.21.15.28:0/1419335504 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f01d8066520 0x7f01d810f270 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f01d400b560 tx=0x7f01d40090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:38:44.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:44.837+0000 7f01cdffb640 1 -- 172.21.15.28:0/1419335504 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01d400f6f0 con 0x7f01d8066520 2024-07-03T15:38:44.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:44.837+0000 7f01cdffb640 1 -- 172.21.15.28:0/1419335504 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f01d400fd20 con 0x7f01d8066520 2024-07-03T15:38:44.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:44.837+0000 7f01df48f640 1 -- 172.21.15.28:0/1419335504 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01d8110a70 con 0x7f01d8066520 2024-07-03T15:38:44.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:44.837+0000 7f01cdffb640 1 -- 172.21.15.28:0/1419335504 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01d40103a0 con 0x7f01d8066520 2024-07-03T15:38:44.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:44.838+0000 7f01df48f640 1 -- 172.21.15.28:0/1419335504 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01d8111010 con 0x7f01d8066520 2024-07-03T15:38:44.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:44.839+0000 7f01df48f640 1 -- 172.21.15.28:0/1419335504 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01d80fa6c0 con 0x7f01d8066520 2024-07-03T15:38:44.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:44.839+0000 7f01cdffb640 1 -- 172.21.15.28:0/1419335504 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 18) v1 ==== 85069+0+0 (secure 0 0 0) 0x7f01d4011400 con 0x7f01d8066520 2024-07-03T15:38:44.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:44.840+0000 7f01cdffb640 1 --2- 172.21.15.28:0/1419335504 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f01c4067100 0x7f01c40695b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:38:44.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:44.840+0000 7f01cdffb640 1 -- 172.21.15.28:0/1419335504 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2410+0+0 (secure 0 0 0) 0x7f01d4083b20 con 0x7f01d8066520 2024-07-03T15:38:44.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:44.840+0000 7f01dca2a640 1 --2- 172.21.15.28:0/1419335504 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f01c4067100 0x7f01c40695b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:38:44.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:44.840+0000 7f01dca2a640 1 --2- 172.21.15.28:0/1419335504 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f01c4067100 0x7f01c40695b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f01c8007ac0 tx=0x7f01c8009da0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:38:44.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:44.844+0000 7f01cdffb640 1 -- 172.21.15.28:0/1419335504 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f01d4051bb0 con 0x7f01d8066520 2024-07-03T15:38:45.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:38:45.029+0000 7f01df48f640 1 -- 172.21.15.28:0/1419335504 --> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi028:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f01d81112e0 con 0x7f01c4067100 2024-07-03T15:38:45.696 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:45 smithi028 ceph-mon[29148]: pgmap v55: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-03T15:38:45.696 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:45 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-03T15:38:45.696 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:45 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-03T15:38:45.696 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:45 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:38:45.696 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:45 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:38:45.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:45 smithi148 ceph-mon[36056]: pgmap v55: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-03T15:38:45.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:45 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-03T15:38:45.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:45 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-03T15:38:45.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:45 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:38:45.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:45 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:38:46.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:46 smithi028 ceph-mon[29148]: from='client.14278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi028:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:38:46.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:46 smithi148 ceph-mon[36056]: from='client.14278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi028:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:38:47.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:47 smithi028 ceph-mon[29148]: pgmap v56: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-03T15:38:47.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:47 smithi148 ceph-mon[36056]: pgmap v56: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-03T15:38:48.862 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:48 smithi028 ceph-mon[29148]: Detected new or changed devices on smithi028 2024-07-03T15:38:48.863 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:48 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:48.863 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:48 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:48.863 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:48 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:38:48.863 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:48 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:38:48.863 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:48 smithi028 ceph-mon[29148]: Adjusting osd_memory_target on smithi028 to 6434M 2024-07-03T15:38:48.863 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:48 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:48.863 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:48 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:38:48.863 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:48 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:38:48.863 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:48 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:48.863 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:48 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:38:48.863 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:48 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/1166701640' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5ec8caa6-fb62-443f-bbd6-fbe52df5034c"}]: dispatch 2024-07-03T15:38:48.863 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:48 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/1166701640' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5ec8caa6-fb62-443f-bbd6-fbe52df5034c"}]': finished 2024-07-03T15:38:48.864 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:48 smithi028 ceph-mon[29148]: osdmap e16: 3 total, 2 up, 3 in 2024-07-03T15:38:48.864 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:48 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-03T15:38:48.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:48 smithi148 ceph-mon[36056]: Detected new or changed devices on smithi028 2024-07-03T15:38:48.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:48 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:48.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:48 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:48.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:48 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:38:48.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:48 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:38:48.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:48 smithi148 ceph-mon[36056]: Adjusting osd_memory_target on smithi028 to 6434M 2024-07-03T15:38:48.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:48 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:48.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:48 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:38:48.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:48 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:38:48.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:48 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:48.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:48 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:38:48.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:48 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/1166701640' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5ec8caa6-fb62-443f-bbd6-fbe52df5034c"}]: dispatch 2024-07-03T15:38:48.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:48 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/1166701640' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5ec8caa6-fb62-443f-bbd6-fbe52df5034c"}]': finished 2024-07-03T15:38:48.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:48 smithi148 ceph-mon[36056]: osdmap e16: 3 total, 2 up, 3 in 2024-07-03T15:38:48.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:48 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-03T15:38:49.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:49 smithi148 ceph-mon[36056]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-03T15:38:49.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:49 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/129690986' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-03T15:38:50.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:49 smithi028 ceph-mon[29148]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-03T15:38:50.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:49 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/129690986' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-03T15:38:51.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:51 smithi148 ceph-mon[36056]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-03T15:38:52.015 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:51 smithi028 ceph-mon[29148]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-03T15:38:53.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:53 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:53.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:53 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:53.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:53 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:38:53.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:53 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:38:53.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:53 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:53.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:53 smithi028 ceph-mon[29148]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-03T15:38:53.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:53 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:53.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:53 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:53.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:53 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:38:53.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:53 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:38:53.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:53 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:38:53.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:53 smithi148 ceph-mon[36056]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-03T15:38:55.713 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:55 smithi028 ceph-mon[29148]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-03T15:38:55.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:55 smithi148 ceph-mon[36056]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-03T15:38:56.482 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:56 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-07-03T15:38:56.482 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:56 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:38:56.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:56 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-07-03T15:38:56.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:56 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:38:57.681 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:57 smithi028 ceph-mon[29148]: Deploying daemon osd.2 on smithi028 2024-07-03T15:38:57.681 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:57 smithi028 ceph-mon[29148]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-03T15:38:57.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:57 smithi148 ceph-mon[36056]: Deploying daemon osd.2 on smithi028 2024-07-03T15:38:57.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:57 smithi148 ceph-mon[36056]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-03T15:38:59.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:38:59 smithi028 ceph-mon[29148]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-03T15:38:59.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:38:59 smithi148 ceph-mon[36056]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-03T15:39:00.457 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:00 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:39:00.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:00 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:39:01.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:01 smithi028 ceph-mon[29148]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-03T15:39:01.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:01 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:39:01.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:01 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:01.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:01 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:01.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:01 smithi148 ceph-mon[36056]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-03T15:39:01.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:01 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:39:01.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:01 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:01.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:01 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:03.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:03 smithi028 ceph-mon[29148]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-03T15:39:03.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:03 smithi148 ceph-mon[36056]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-03T15:39:04.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:39:04.559+0000 7f01cdffb640 1 -- 172.21.15.28:0/1419335504 <== mgr.14210 v2:172.21.15.28:6800/421057178 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f01d81112e0 con 0x7f01c4067100 2024-07-03T15:39:04.562 INFO:teuthology.orchestra.run.smithi028.stdout:Created osd(s) 2 on host 'smithi028' 2024-07-03T15:39:04.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:39:04.561+0000 7f01df48f640 1 -- 172.21.15.28:0/1419335504 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f01c4067100 msgr2=0x7f01c40695b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:39:04.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:39:04.561+0000 7f01df48f640 1 --2- 172.21.15.28:0/1419335504 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f01c4067100 0x7f01c40695b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f01c8007ac0 tx=0x7f01c8009da0 comp rx=0 tx=0).stop 2024-07-03T15:39:04.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:39:04.561+0000 7f01df48f640 1 -- 172.21.15.28:0/1419335504 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f01d8066520 msgr2=0x7f01d810f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:39:04.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:39:04.561+0000 7f01df48f640 1 --2- 172.21.15.28:0/1419335504 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f01d8066520 0x7f01d810f270 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f01d400b560 tx=0x7f01d40090f0 comp rx=0 tx=0).stop 2024-07-03T15:39:04.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:39:04.561+0000 7f01df48f640 1 -- 172.21.15.28:0/1419335504 shutdown_connections 2024-07-03T15:39:04.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:39:04.561+0000 7f01df48f640 1 --2- 172.21.15.28:0/1419335504 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f01c4067100 0x7f01c40695b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:39:04.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:39:04.561+0000 7f01df48f640 1 --2- 172.21.15.28:0/1419335504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01d81018b0 0x7f01d810f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:39:04.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:39:04.561+0000 7f01df48f640 1 --2- 172.21.15.28:0/1419335504 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f01d8066520 0x7f01d810f270 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:39:04.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:39:04.561+0000 7f01df48f640 1 -- 172.21.15.28:0/1419335504 >> 172.21.15.28:0/1419335504 conn(0x7f01d80f6a20 msgr2=0x7f01d80f6e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:39:04.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:39:04.562+0000 7f01df48f640 1 -- 172.21.15.28:0/1419335504 shutdown_connections 2024-07-03T15:39:04.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:39:04.562+0000 7f01df48f640 1 -- 172.21.15.28:0/1419335504 wait complete. 2024-07-03T15:39:05.178 DEBUG:teuthology.orchestra.run.smithi028:osd.2> sudo journalctl -f -n 0 -u ceph-a85e4be6-3951-11ef-bcab-c7b262605968@osd.2.service 2024-07-03T15:39:05.181 INFO:tasks.cephadm:Deploying osd.3 on smithi148 with /dev/vg_nvme/lv_4... 2024-07-03T15:39:05.181 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-07-03T15:39:05.438 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi148/config 2024-07-03T15:39:05.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:05 smithi148 ceph-mon[36056]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-03T15:39:05.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:05 smithi148 ceph-mon[36056]: from='osd.2 [v2:172.21.15.28:6818/3104474806,v1:172.21.15.28:6819/3104474806]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-07-03T15:39:05.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:05 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:05.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:05 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:05.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:05 smithi028 ceph-mon[29148]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-03T15:39:05.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:05 smithi028 ceph-mon[29148]: from='osd.2 [v2:172.21.15.28:6818/3104474806,v1:172.21.15.28:6819/3104474806]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-07-03T15:39:05.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:05 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:05.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:05 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:06.686 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:06 smithi148 ceph-mon[36056]: from='osd.2 [v2:172.21.15.28:6818/3104474806,v1:172.21.15.28:6819/3104474806]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-07-03T15:39:06.686 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:06 smithi148 ceph-mon[36056]: osdmap e17: 3 total, 2 up, 3 in 2024-07-03T15:39:06.686 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:06 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-03T15:39:06.686 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:06 smithi148 ceph-mon[36056]: from='osd.2 [v2:172.21.15.28:6818/3104474806,v1:172.21.15.28:6819/3104474806]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi028", "root=default"]}]: dispatch 2024-07-03T15:39:06.765 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:06 smithi028 ceph-mon[29148]: from='osd.2 [v2:172.21.15.28:6818/3104474806,v1:172.21.15.28:6819/3104474806]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-07-03T15:39:06.765 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:06 smithi028 ceph-mon[29148]: osdmap e17: 3 total, 2 up, 3 in 2024-07-03T15:39:06.765 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:06 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-03T15:39:06.765 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:06 smithi028 ceph-mon[29148]: from='osd.2 [v2:172.21.15.28:6818/3104474806,v1:172.21.15.28:6819/3104474806]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi028", "root=default"]}]: dispatch 2024-07-03T15:39:06.765 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:39:06 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-2[62448]: 2024-07-03T15:39:06.437+0000 7fe3646ec640 -1 osd.2 0 waiting for initial osdmap 2024-07-03T15:39:06.765 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:39:06 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-2[62448]: 2024-07-03T15:39:06.442+0000 7fe35e521640 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-03T15:39:07.025 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-07-03T15:39:07.111 INFO:teuthology.orchestra.run.smithi148.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-07-03T15:39:07.124 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10+0 records in 2024-07-03T15:39:07.125 INFO:teuthology.orchestra.run.smithi148.stderr:10+0 records out 2024-07-03T15:39:07.125 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011829 s, 886 MB/s 2024-07-03T15:39:07.125 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping successful for: 2024-07-03T15:39:07.699 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:07 smithi148 ceph-mon[36056]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-03T15:39:07.699 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:07 smithi148 ceph-mon[36056]: from='osd.2 [v2:172.21.15.28:6818/3104474806,v1:172.21.15.28:6819/3104474806]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi028", "root=default"]}]': finished 2024-07-03T15:39:07.699 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:07 smithi148 ceph-mon[36056]: osdmap e18: 3 total, 2 up, 3 in 2024-07-03T15:39:07.699 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:07 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-03T15:39:07.699 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:07 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-03T15:39:07.720 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph orch daemon add osd smithi148:vg_nvme/lv_4 2024-07-03T15:39:07.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:07 smithi028 ceph-mon[29148]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-03T15:39:07.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:07 smithi028 ceph-mon[29148]: from='osd.2 [v2:172.21.15.28:6818/3104474806,v1:172.21.15.28:6819/3104474806]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi028", "root=default"]}]': finished 2024-07-03T15:39:07.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:07 smithi028 ceph-mon[29148]: osdmap e18: 3 total, 2 up, 3 in 2024-07-03T15:39:07.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:07 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-03T15:39:07.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:07 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-03T15:39:07.969 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi148/config 2024-07-03T15:39:09.103 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:08 smithi028 ceph-mon[29148]: purged_snaps scrub starts 2024-07-03T15:39:09.103 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:08 smithi028 ceph-mon[29148]: purged_snaps scrub ok 2024-07-03T15:39:09.103 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:08 smithi028 ceph-mon[29148]: osd.2 [v2:172.21.15.28:6818/3104474806,v1:172.21.15.28:6819/3104474806] boot 2024-07-03T15:39:09.103 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:08 smithi028 ceph-mon[29148]: osdmap e19: 3 total, 3 up, 3 in 2024-07-03T15:39:09.103 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:08 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-03T15:39:09.103 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:08 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:09.103 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:08 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:09.125 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:08 smithi148 ceph-mon[36056]: purged_snaps scrub starts 2024-07-03T15:39:09.126 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:08 smithi148 ceph-mon[36056]: purged_snaps scrub ok 2024-07-03T15:39:09.126 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:08 smithi148 ceph-mon[36056]: osd.2 [v2:172.21.15.28:6818/3104474806,v1:172.21.15.28:6819/3104474806] boot 2024-07-03T15:39:09.126 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:08 smithi148 ceph-mon[36056]: osdmap e19: 3 total, 3 up, 3 in 2024-07-03T15:39:09.126 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:08 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-03T15:39:09.126 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:08 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:09.126 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:08 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:09.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:09.304+0000 7f49231aa640 1 -- 172.21.15.148:0/424947542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f491c0fc530 msgr2=0x7f491c0fc9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:39:09.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:09.304+0000 7f49231aa640 1 --2- 172.21.15.148:0/424947542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f491c0fc530 0x7f491c0fc9a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f490c009c50 tx=0x7f490c004d10 comp rx=0 tx=0).stop 2024-07-03T15:39:09.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:09.305+0000 7f49231aa640 1 -- 172.21.15.148:0/424947542 shutdown_connections 2024-07-03T15:39:09.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:09.305+0000 7f49231aa640 1 --2- 172.21.15.148:0/424947542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f491c0fc530 0x7f491c0fc9a0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:39:09.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:09.305+0000 7f49231aa640 1 --2- 172.21.15.148:0/424947542 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f491c0fb260 0x7f491c0fb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:39:09.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:09.305+0000 7f49231aa640 1 -- 172.21.15.148:0/424947542 >> 172.21.15.148:0/424947542 conn(0x7f491c0f6a20 msgr2=0x7f491c0f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:39:09.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:09.305+0000 7f49231aa640 1 -- 172.21.15.148:0/424947542 shutdown_connections 2024-07-03T15:39:09.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:09.305+0000 7f49231aa640 1 -- 172.21.15.148:0/424947542 wait complete. 2024-07-03T15:39:09.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:09.305+0000 7f49231aa640 1 Processor -- start 2024-07-03T15:39:09.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:09.305+0000 7f49231aa640 1 -- start start 2024-07-03T15:39:09.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:09.306+0000 7f49231aa640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f491c0fb260 0x7f491c10cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:39:09.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:09.306+0000 7f49231aa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f491c0fc530 0x7f491c10d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:39:09.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:09.306+0000 7f49231aa640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491c10e3a0 con 0x7f491c0fb260 2024-07-03T15:39:09.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:09.306+0000 7f49231aa640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491c10e510 con 0x7f491c0fc530 2024-07-03T15:39:09.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:09.306+0000 7f491bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f491c0fc530 0x7f491c10d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:39:09.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:09.306+0000 7f491bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f491c0fc530 0x7f491c10d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34052/0 (socket says 172.21.15.148:34052) 2024-07-03T15:39:09.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:09.306+0000 7f4920f46640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f491c0fb260 0x7f491c10cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:39:09.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:09.306+0000 7f491bfff640 1 -- 172.21.15.148:0/3936002942 learned_addr learned my addr 172.21.15.148:0/3936002942 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-07-03T15:39:09.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:09.306+0000 7f491bfff640 1 -- 172.21.15.148:0/3936002942 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f491c0fb260 msgr2=0x7f491c10cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:39:09.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:09.306+0000 7f491bfff640 1 --2- 172.21.15.148:0/3936002942 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f491c0fb260 0x7f491c10cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:39:09.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:09.306+0000 7f491bfff640 1 -- 172.21.15.148:0/3936002942 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f490c0098b0 con 0x7f491c0fc530 2024-07-03T15:39:09.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:09.306+0000 7f491bfff640 1 --2- 172.21.15.148:0/3936002942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f491c0fc530 0x7f491c10d530 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f490c003400 tx=0x7f490c004d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:39:09.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:09.306+0000 7f49197fa640 1 -- 172.21.15.148:0/3936002942 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f490c032910 con 0x7f491c0fc530 2024-07-03T15:39:09.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:09.306+0000 7f49197fa640 1 -- 172.21.15.148:0/3936002942 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f490c008ee0 con 0x7f491c0fc530 2024-07-03T15:39:09.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:09.306+0000 7f49231aa640 1 -- 172.21.15.148:0/3936002942 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f491c10e790 con 0x7f491c0fc530 2024-07-03T15:39:09.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:09.306+0000 7f49197fa640 1 -- 172.21.15.148:0/3936002942 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f490c03dcf0 con 0x7f491c0fc530 2024-07-03T15:39:09.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:09.307+0000 7f49231aa640 1 -- 172.21.15.148:0/3936002942 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f491c10ecb0 con 0x7f491c0fc530 2024-07-03T15:39:09.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:09.308+0000 7f49231aa640 1 -- 172.21.15.148:0/3936002942 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f491c0fa6c0 con 0x7f491c0fc530 2024-07-03T15:39:09.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:09.308+0000 7f49197fa640 1 -- 172.21.15.148:0/3936002942 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) v1 ==== 85069+0+0 (secure 0 0 0) 0x7f490c02f030 con 0x7f491c0fc530 2024-07-03T15:39:09.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:09.308+0000 7f49197fa640 1 --2- 172.21.15.148:0/3936002942 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f4908067110 0x7f49080695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:39:09.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:09.308+0000 7f49197fa640 1 -- 172.21.15.148:0/3936002942 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3217+0+0 (secure 0 0 0) 0x7f490c0a5820 con 0x7f491c0fc530 2024-07-03T15:39:09.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:09.309+0000 7f4920f46640 1 --2- 172.21.15.148:0/3936002942 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f4908067110 0x7f49080695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:39:09.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:09.309+0000 7f4920f46640 1 --2- 172.21.15.148:0/3936002942 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f4908067110 0x7f49080695c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4914009b50 tx=0x7f4914009120 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:39:09.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:09.313+0000 7f49197fa640 1 -- 172.21.15.148:0/3936002942 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f490c0734e0 con 0x7f491c0fc530 2024-07-03T15:39:09.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:09.486+0000 7f49231aa640 1 -- 172.21.15.148:0/3936002942 --> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f491c103e30 con 0x7f4908067110 2024-07-03T15:39:10.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:09 smithi028 ceph-mon[29148]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-03T15:39:10.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:09 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-07-03T15:39:10.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:09 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-07-03T15:39:10.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:09 smithi028 ceph-mon[29148]: osdmap e20: 3 total, 3 up, 3 in 2024-07-03T15:39:10.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:09 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-07-03T15:39:10.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:09 smithi028 ceph-mon[29148]: from='client.24125 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:39:10.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:09 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-03T15:39:10.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:09 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-03T15:39:10.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:09 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:39:10.128 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:09 smithi148 ceph-mon[36056]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-03T15:39:10.129 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:09 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-07-03T15:39:10.129 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:09 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-07-03T15:39:10.129 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:09 smithi148 ceph-mon[36056]: osdmap e20: 3 total, 3 up, 3 in 2024-07-03T15:39:10.129 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:09 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-07-03T15:39:10.129 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:09 smithi148 ceph-mon[36056]: from='client.24125 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:39:10.129 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:09 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-03T15:39:10.129 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:09 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-03T15:39:10.129 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:09 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:39:10.851 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:10 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-07-03T15:39:10.852 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:10 smithi148 ceph-mon[36056]: osdmap e21: 3 total, 3 up, 3 in 2024-07-03T15:39:10.852 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:10 smithi148 ceph-mon[36056]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-07-03T15:39:10.852 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:10 smithi148 ceph-mon[36056]: Detected new or changed devices on smithi028 2024-07-03T15:39:10.852 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:10 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:10.852 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:10 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:10.852 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:10 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:39:10.852 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:10 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:39:10.852 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:10 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:39:10.852 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:10 smithi148 ceph-mon[36056]: Adjusting osd_memory_target on smithi028 to 4289M 2024-07-03T15:39:10.852 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:10 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:10.852 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:10 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:39:10.852 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:10 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:39:10.853 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:10 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:10.853 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:10 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:39:11.121 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:39:11 smithi028 sudo[66708]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-07-03T15:39:11.121 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:39:11 smithi028 sudo[66708]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-03T15:39:11.121 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:39:11 smithi028 sudo[66708]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-03T15:39:11.121 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:39:11 smithi028 sudo[66708]: pam_unix(sudo:session): session closed for user root 2024-07-03T15:39:11.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:10 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-07-03T15:39:11.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:10 smithi028 ceph-mon[29148]: osdmap e21: 3 total, 3 up, 3 in 2024-07-03T15:39:11.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:10 smithi028 ceph-mon[29148]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-07-03T15:39:11.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:10 smithi028 ceph-mon[29148]: Detected new or changed devices on smithi028 2024-07-03T15:39:11.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:10 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:11.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:10 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:11.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:10 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:39:11.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:10 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:39:11.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:10 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:39:11.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:10 smithi028 ceph-mon[29148]: Adjusting osd_memory_target on smithi028 to 4289M 2024-07-03T15:39:11.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:10 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:11.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:10 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:39:11.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:10 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:39:11.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:10 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:11.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:10 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:39:11.871 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:39:11 smithi028 sudo[66728]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-07-03T15:39:11.871 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:39:11 smithi028 sudo[66728]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-03T15:39:11.871 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:39:11 smithi028 sudo[66728]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-03T15:39:11.871 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:39:11 smithi028 sudo[66728]: pam_unix(sudo:session): session closed for user root 2024-07-03T15:39:12.302 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:39:11 smithi028 sudo[66812]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-07-03T15:39:12.302 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:39:11 smithi028 sudo[66812]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-03T15:39:12.302 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:39:11 smithi028 sudo[66812]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-03T15:39:12.302 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:39:12 smithi028 sudo[66812]: pam_unix(sudo:session): session closed for user root 2024-07-03T15:39:12.621 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:39:12 smithi028 sudo[66837]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-07-03T15:39:12.621 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:39:12 smithi028 sudo[66837]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-03T15:39:12.621 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:39:12 smithi028 sudo[66837]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-03T15:39:12.621 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:39:12 smithi028 sudo[66837]: pam_unix(sudo:session): session closed for user root 2024-07-03T15:39:12.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:12 smithi028 ceph-mon[29148]: osdmap e22: 3 total, 3 up, 3 in 2024-07-03T15:39:12.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:12 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:12.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:12 smithi028 ceph-mon[29148]: from='client.? 172.21.15.148:0/3406433033' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3117e1c-a230-4fc8-9cfc-37e4dc7e3d17"}]: dispatch 2024-07-03T15:39:12.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:12 smithi028 ceph-mon[29148]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3117e1c-a230-4fc8-9cfc-37e4dc7e3d17"}]: dispatch 2024-07-03T15:39:12.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:12 smithi028 ceph-mon[29148]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d3117e1c-a230-4fc8-9cfc-37e4dc7e3d17"}]': finished 2024-07-03T15:39:12.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:12 smithi028 ceph-mon[29148]: osdmap e23: 4 total, 3 up, 4 in 2024-07-03T15:39:12.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:12 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-03T15:39:12.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:12 smithi028 ceph-mon[29148]: from='client.? 172.21.15.148:0/621970082' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-03T15:39:12.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:12 smithi148 ceph-mon[36056]: osdmap e22: 3 total, 3 up, 3 in 2024-07-03T15:39:12.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:12 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:12.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:12 smithi148 ceph-mon[36056]: from='client.? 172.21.15.148:0/3406433033' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3117e1c-a230-4fc8-9cfc-37e4dc7e3d17"}]: dispatch 2024-07-03T15:39:12.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:12 smithi148 ceph-mon[36056]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3117e1c-a230-4fc8-9cfc-37e4dc7e3d17"}]: dispatch 2024-07-03T15:39:12.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:12 smithi148 ceph-mon[36056]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d3117e1c-a230-4fc8-9cfc-37e4dc7e3d17"}]': finished 2024-07-03T15:39:12.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:12 smithi148 ceph-mon[36056]: osdmap e23: 4 total, 3 up, 4 in 2024-07-03T15:39:12.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:12 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-03T15:39:12.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:12 smithi148 ceph-mon[36056]: from='client.? 172.21.15.148:0/621970082' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-03T15:39:12.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:12.743+0000 7f49197fa640 1 -- 172.21.15.148:0/3936002942 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f490c03d520 con 0x7f491c0fc530 2024-07-03T15:39:13.030 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:39:12 smithi028 sudo[66911]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-07-03T15:39:13.030 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:39:12 smithi028 sudo[66911]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-03T15:39:13.030 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:39:12 smithi028 sudo[66911]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-03T15:39:13.030 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:39:12 smithi028 sudo[66911]: pam_unix(sudo:session): session closed for user root 2024-07-03T15:39:13.621 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:39:13 smithi028 sudo[66960]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-07-03T15:39:13.621 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:39:13 smithi028 sudo[66960]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-03T15:39:13.621 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:39:13 smithi028 sudo[66960]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-03T15:39:13.621 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:39:13 smithi028 sudo[66960]: pam_unix(sudo:session): session closed for user root 2024-07-03T15:39:14.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:13 smithi028 sudo[66991]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-07-03T15:39:14.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:13 smithi028 sudo[66991]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-03T15:39:14.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:13 smithi028 sudo[66991]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-03T15:39:14.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:13 smithi028 sudo[66991]: pam_unix(sudo:session): session closed for user root 2024-07-03T15:39:14.537 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:14 smithi148 ceph-mon[36056]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 872 MiB used, 267 GiB / 268 GiB avail 2024-07-03T15:39:14.537 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:14 smithi148 ceph-mon[36056]: mgrmap e19: smithi028.vtqrtz(active, since 2m), standbys: smithi148.rjzsar 2024-07-03T15:39:14.537 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:14 smithi148 ceph-mon[36056]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-03T15:39:14.537 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:14 smithi148 sudo[40147]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-07-03T15:39:14.538 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:14 smithi148 sudo[40147]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-03T15:39:14.538 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:14 smithi148 sudo[40147]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-03T15:39:14.538 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:14 smithi148 sudo[40147]: pam_unix(sudo:session): session closed for user root 2024-07-03T15:39:14.545 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:14 smithi028 sudo[67047]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-07-03T15:39:14.545 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:14 smithi028 sudo[67047]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-03T15:39:14.545 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:14 smithi028 sudo[67047]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-03T15:39:14.545 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:14 smithi028 sudo[67047]: pam_unix(sudo:session): session closed for user root 2024-07-03T15:39:14.546 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:14 smithi028 ceph-mon[29148]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 872 MiB used, 267 GiB / 268 GiB avail 2024-07-03T15:39:14.546 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:14 smithi028 ceph-mon[29148]: mgrmap e19: smithi028.vtqrtz(active, since 2m), standbys: smithi148.rjzsar 2024-07-03T15:39:14.546 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:14 smithi028 ceph-mon[29148]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-03T15:39:14.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:14 smithi148 sudo[40335]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-07-03T15:39:14.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:14 smithi148 sudo[40335]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-03T15:39:14.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:14 smithi148 sudo[40335]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-03T15:39:14.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:14 smithi148 sudo[40335]: pam_unix(sudo:session): session closed for user root 2024-07-03T15:39:15.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:15 smithi028 ceph-mon[29148]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-03T15:39:15.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:15 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi028"}]: dispatch 2024-07-03T15:39:15.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:15 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-07-03T15:39:15.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:15 smithi028 ceph-mon[29148]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-03T15:39:15.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:15 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi028"}]: dispatch 2024-07-03T15:39:15.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:15 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-07-03T15:39:15.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:15 smithi028 ceph-mon[29148]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-07-03T15:39:15.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:15 smithi028 ceph-mon[29148]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-03T15:39:15.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:15 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:15.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:15 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:15.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:15 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:15.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:15 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:39:15.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:15 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:39:15.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:15 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:15.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:15 smithi148 ceph-mon[36056]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-03T15:39:15.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:15 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi028"}]: dispatch 2024-07-03T15:39:15.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:15 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-07-03T15:39:15.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:15 smithi148 ceph-mon[36056]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-03T15:39:15.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:15 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi028"}]: dispatch 2024-07-03T15:39:15.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:15 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-07-03T15:39:15.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:15 smithi148 ceph-mon[36056]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-07-03T15:39:15.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:15 smithi148 ceph-mon[36056]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-03T15:39:15.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:15 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:15.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:15 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:15.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:15 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:15.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:15 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:39:15.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:15 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:39:15.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:15 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:16.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:16 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:39:16.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:16 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:39:17.413 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:17 smithi148 ceph-mon[36056]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-07-03T15:39:17.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:17 smithi028 ceph-mon[29148]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-07-03T15:39:19.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:19 smithi148 ceph-mon[36056]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.7 KiB/s rd, 118 KiB/s wr, 8 op/s 2024-07-03T15:39:19.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:19 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-07-03T15:39:19.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:19 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:39:19.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:19 smithi148 ceph-mon[36056]: Deploying daemon osd.3 on smithi148 2024-07-03T15:39:19.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:19 smithi028 ceph-mon[29148]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.7 KiB/s rd, 118 KiB/s wr, 8 op/s 2024-07-03T15:39:19.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:19 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-07-03T15:39:19.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:19 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:39:19.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:19 smithi028 ceph-mon[29148]: Deploying daemon osd.3 on smithi148 2024-07-03T15:39:21.672 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:21 smithi148 ceph-mon[36056]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.1 KiB/s rd, 91 KiB/s wr, 6 op/s 2024-07-03T15:39:21.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:21 smithi028 ceph-mon[29148]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.1 KiB/s rd, 91 KiB/s wr, 6 op/s 2024-07-03T15:39:23.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:23 smithi148 ceph-mon[36056]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.7 KiB/s rd, 74 KiB/s wr, 5 op/s 2024-07-03T15:39:23.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:23 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:39:23.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:23 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:23.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:23 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:23.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:23 smithi028 ceph-mon[29148]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.7 KiB/s rd, 74 KiB/s wr, 5 op/s 2024-07-03T15:39:23.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:23 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:39:23.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:23 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:23.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:23 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:25.703 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:25 smithi148 ceph-mon[36056]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-07-03T15:39:25.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:25 smithi028 ceph-mon[29148]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-07-03T15:39:26.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:26.804+0000 7f49197fa640 1 -- 172.21.15.148:0/3936002942 <== mgr.14210 v2:172.21.15.28:6800/421057178 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f491c103e30 con 0x7f4908067110 2024-07-03T15:39:26.808 INFO:teuthology.orchestra.run.smithi148.stdout:Created osd(s) 3 on host 'smithi148' 2024-07-03T15:39:26.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:26.807+0000 7f49231aa640 1 -- 172.21.15.148:0/3936002942 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f4908067110 msgr2=0x7f49080695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:39:26.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:26.807+0000 7f49231aa640 1 --2- 172.21.15.148:0/3936002942 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f4908067110 0x7f49080695c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4914009b50 tx=0x7f4914009120 comp rx=0 tx=0).stop 2024-07-03T15:39:26.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:26.807+0000 7f49231aa640 1 -- 172.21.15.148:0/3936002942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f491c0fc530 msgr2=0x7f491c10d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:39:26.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:26.807+0000 7f49231aa640 1 --2- 172.21.15.148:0/3936002942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f491c0fc530 0x7f491c10d530 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f490c003400 tx=0x7f490c004d10 comp rx=0 tx=0).stop 2024-07-03T15:39:26.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:26.807+0000 7f49231aa640 1 -- 172.21.15.148:0/3936002942 shutdown_connections 2024-07-03T15:39:26.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:26.807+0000 7f49231aa640 1 --2- 172.21.15.148:0/3936002942 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f4908067110 0x7f49080695c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:39:26.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:26.807+0000 7f49231aa640 1 --2- 172.21.15.148:0/3936002942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f491c0fc530 0x7f491c10d530 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:39:26.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:26.807+0000 7f49231aa640 1 --2- 172.21.15.148:0/3936002942 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f491c0fb260 0x7f491c10cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:39:26.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:26.807+0000 7f49231aa640 1 -- 172.21.15.148:0/3936002942 >> 172.21.15.148:0/3936002942 conn(0x7f491c0f6a20 msgr2=0x7f491c065d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:39:26.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:26.807+0000 7f49231aa640 1 -- 172.21.15.148:0/3936002942 shutdown_connections 2024-07-03T15:39:26.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:26.807+0000 7f49231aa640 1 -- 172.21.15.148:0/3936002942 wait complete. 2024-07-03T15:39:27.421 DEBUG:teuthology.orchestra.run.smithi148:osd.3> sudo journalctl -f -n 0 -u ceph-a85e4be6-3951-11ef-bcab-c7b262605968@osd.3.service 2024-07-03T15:39:27.430 INFO:tasks.cephadm:Deploying osd.4 on smithi148 with /dev/vg_nvme/lv_3... 2024-07-03T15:39:27.430 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-07-03T15:39:27.729 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi148/config 2024-07-03T15:39:27.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:27 smithi148 ceph-mon[36056]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-07-03T15:39:27.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:27 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:27.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:27 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:27.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:27 smithi148 ceph-mon[36056]: from='osd.3 [v2:172.21.15.148:6800/2002097126,v1:172.21.15.148:6801/2002097126]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-07-03T15:39:27.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:27 smithi148 ceph-mon[36056]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-07-03T15:39:27.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:27 smithi028 ceph-mon[29148]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-07-03T15:39:27.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:27 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:27.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:27 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:27.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:27 smithi028 ceph-mon[29148]: from='osd.3 [v2:172.21.15.148:6800/2002097126,v1:172.21.15.148:6801/2002097126]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-07-03T15:39:27.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:27 smithi028 ceph-mon[29148]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-07-03T15:39:28.793 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:28 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:28.793 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:28 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:28.793 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:28 smithi148 ceph-mon[36056]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-07-03T15:39:28.793 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:28 smithi148 ceph-mon[36056]: osdmap e24: 4 total, 3 up, 4 in 2024-07-03T15:39:28.793 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:28 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-03T15:39:28.793 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:28 smithi148 ceph-mon[36056]: from='osd.3 [v2:172.21.15.148:6800/2002097126,v1:172.21.15.148:6801/2002097126]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-07-03T15:39:28.793 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:28 smithi148 ceph-mon[36056]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-07-03T15:39:28.855 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:28 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:28.855 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:28 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:28.855 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:28 smithi028 ceph-mon[29148]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-07-03T15:39:28.855 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:28 smithi028 ceph-mon[29148]: osdmap e24: 4 total, 3 up, 4 in 2024-07-03T15:39:28.855 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:28 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-03T15:39:28.855 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:28 smithi028 ceph-mon[29148]: from='osd.3 [v2:172.21.15.148:6800/2002097126,v1:172.21.15.148:6801/2002097126]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-07-03T15:39:28.855 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:28 smithi028 ceph-mon[29148]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-07-03T15:39:29.069 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:39:28 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3[41393]: 2024-07-03T15:39:28.813+0000 7f4ad9083640 -1 osd.3 0 waiting for initial osdmap 2024-07-03T15:39:29.069 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:39:28 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3[41393]: 2024-07-03T15:39:28.822+0000 7f4ad46bb640 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-03T15:39:29.400 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-07-03T15:39:29.497 INFO:teuthology.orchestra.run.smithi148.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-07-03T15:39:29.511 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10+0 records in 2024-07-03T15:39:29.511 INFO:teuthology.orchestra.run.smithi148.stderr:10+0 records out 2024-07-03T15:39:29.511 INFO:teuthology.orchestra.run.smithi148.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0126904 s, 826 MB/s 2024-07-03T15:39:29.511 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping successful for: 2024-07-03T15:39:29.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:29 smithi148 ceph-mon[36056]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-07-03T15:39:29.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:29 smithi148 ceph-mon[36056]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-07-03T15:39:29.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:29 smithi148 ceph-mon[36056]: osdmap e25: 4 total, 3 up, 4 in 2024-07-03T15:39:29.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:29 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-03T15:39:29.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:29 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-03T15:39:29.874 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:29 smithi028 ceph-mon[29148]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-07-03T15:39:29.874 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:29 smithi028 ceph-mon[29148]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-07-03T15:39:29.874 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:29 smithi028 ceph-mon[29148]: osdmap e25: 4 total, 3 up, 4 in 2024-07-03T15:39:29.874 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:29 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-03T15:39:29.874 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:29 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-03T15:39:30.573 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph orch daemon add osd smithi148:vg_nvme/lv_3 2024-07-03T15:39:31.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:30 smithi028 ceph-mon[29148]: purged_snaps scrub starts 2024-07-03T15:39:31.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:30 smithi028 ceph-mon[29148]: purged_snaps scrub ok 2024-07-03T15:39:31.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:30 smithi028 ceph-mon[29148]: osd.3 [v2:172.21.15.148:6800/2002097126,v1:172.21.15.148:6801/2002097126] boot 2024-07-03T15:39:31.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:30 smithi028 ceph-mon[29148]: osdmap e26: 4 total, 4 up, 4 in 2024-07-03T15:39:31.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:30 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-03T15:39:31.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:30 smithi028 ceph-mon[29148]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-07-03T15:39:31.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:30 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:39:31.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:30 smithi028 ceph-mon[29148]: osdmap e27: 4 total, 4 up, 4 in 2024-07-03T15:39:31.161 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:30 smithi148 ceph-mon[36056]: purged_snaps scrub starts 2024-07-03T15:39:31.161 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:30 smithi148 ceph-mon[36056]: purged_snaps scrub ok 2024-07-03T15:39:31.161 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:30 smithi148 ceph-mon[36056]: osd.3 [v2:172.21.15.148:6800/2002097126,v1:172.21.15.148:6801/2002097126] boot 2024-07-03T15:39:31.161 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:30 smithi148 ceph-mon[36056]: osdmap e26: 4 total, 4 up, 4 in 2024-07-03T15:39:31.161 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:30 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-03T15:39:31.161 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:30 smithi148 ceph-mon[36056]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-07-03T15:39:31.161 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:30 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:39:31.161 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:30 smithi148 ceph-mon[36056]: osdmap e27: 4 total, 4 up, 4 in 2024-07-03T15:39:31.540 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi148/config 2024-07-03T15:39:33.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:32 smithi028 ceph-mon[29148]: osdmap e28: 4 total, 4 up, 4 in 2024-07-03T15:39:33.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:32 smithi028 ceph-mon[29148]: pgmap v92: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-07-03T15:39:33.203 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:32 smithi148 ceph-mon[36056]: osdmap e28: 4 total, 4 up, 4 in 2024-07-03T15:39:33.203 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:32 smithi148 ceph-mon[36056]: pgmap v92: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-07-03T15:39:33.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:33.666+0000 7fed916bb640 1 -- 172.21.15.148:0/3346157871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed8c0fc510 msgr2=0x7fed8c0fc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:39:33.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:33.666+0000 7fed916bb640 1 --2- 172.21.15.148:0/3346157871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed8c0fc510 0x7fed8c0fc980 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fed800048d0 tx=0x7fed80009da0 comp rx=0 tx=0).stop 2024-07-03T15:39:33.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:33.666+0000 7fed916bb640 1 -- 172.21.15.148:0/3346157871 shutdown_connections 2024-07-03T15:39:33.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:33.666+0000 7fed916bb640 1 --2- 172.21.15.148:0/3346157871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed8c0fc510 0x7fed8c0fc980 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:39:33.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:33.666+0000 7fed916bb640 1 --2- 172.21.15.148:0/3346157871 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed8c0fb240 0x7fed8c0fb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:39:33.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:33.666+0000 7fed916bb640 1 -- 172.21.15.148:0/3346157871 >> 172.21.15.148:0/3346157871 conn(0x7fed8c0f6a20 msgr2=0x7fed8c0f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:39:33.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:33.666+0000 7fed916bb640 1 -- 172.21.15.148:0/3346157871 shutdown_connections 2024-07-03T15:39:33.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:33.667+0000 7fed916bb640 1 -- 172.21.15.148:0/3346157871 wait complete. 2024-07-03T15:39:33.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:33.667+0000 7fed916bb640 1 Processor -- start 2024-07-03T15:39:33.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:33.667+0000 7fed916bb640 1 -- start start 2024-07-03T15:39:33.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:33.667+0000 7fed916bb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed8c0fb240 0x7fed8c0ff360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:39:33.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:33.667+0000 7fed916bb640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed8c0fc510 0x7fed8c0fd850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:39:33.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:33.667+0000 7fed916bb640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed8c0fde70 con 0x7fed8c0fc510 2024-07-03T15:39:33.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:33.667+0000 7fed916bb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed8c0fdfb0 con 0x7fed8c0fb240 2024-07-03T15:39:33.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:33.667+0000 7fed8affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed8c0fb240 0x7fed8c0ff360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:39:33.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:33.667+0000 7fed8affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed8c0fb240 0x7fed8c0ff360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37204/0 (socket says 172.21.15.148:37204) 2024-07-03T15:39:33.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:33.667+0000 7fed8affd640 1 -- 172.21.15.148:0/3047095449 learned_addr learned my addr 172.21.15.148:0/3047095449 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-07-03T15:39:33.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:33.668+0000 7fed8affd640 1 -- 172.21.15.148:0/3047095449 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed8c0fc510 msgr2=0x7fed8c0fd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:39:33.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:33.668+0000 7fed8a7fc640 1 --2- 172.21.15.148:0/3047095449 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed8c0fc510 0x7fed8c0fd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:39:33.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:33.668+0000 7fed8affd640 1 --2- 172.21.15.148:0/3047095449 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed8c0fc510 0x7fed8c0fd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:39:33.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:33.668+0000 7fed8affd640 1 -- 172.21.15.148:0/3047095449 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed80005040 con 0x7fed8c0fb240 2024-07-03T15:39:33.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:33.668+0000 7fed8a7fc640 1 --2- 172.21.15.148:0/3047095449 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed8c0fc510 0x7fed8c0fd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-03T15:39:33.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:33.668+0000 7fed8affd640 1 --2- 172.21.15.148:0/3047095449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed8c0fb240 0x7fed8c0ff360 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fed7c00bbb0 tx=0x7fed7c009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:39:33.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:33.668+0000 7fed737fe640 1 -- 172.21.15.148:0/3047095449 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed7c00c9c0 con 0x7fed8c0fb240 2024-07-03T15:39:33.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:33.668+0000 7fed737fe640 1 -- 172.21.15.148:0/3047095449 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed7c0092e0 con 0x7fed8c0fb240 2024-07-03T15:39:33.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:33.668+0000 7fed916bb640 1 -- 172.21.15.148:0/3047095449 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed8c0fe290 con 0x7fed8c0fb240 2024-07-03T15:39:33.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:33.668+0000 7fed737fe640 1 -- 172.21.15.148:0/3047095449 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed7c007640 con 0x7fed8c0fb240 2024-07-03T15:39:33.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:33.668+0000 7fed916bb640 1 -- 172.21.15.148:0/3047095449 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed8c115030 con 0x7fed8c0fb240 2024-07-03T15:39:33.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:33.669+0000 7fed916bb640 1 -- 172.21.15.148:0/3047095449 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed8c0fa6a0 con 0x7fed8c0fb240 2024-07-03T15:39:33.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:33.670+0000 7fed737fe640 1 -- 172.21.15.148:0/3047095449 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7fed7c012070 con 0x7fed8c0fb240 2024-07-03T15:39:33.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:33.670+0000 7fed737fe640 1 --2- 172.21.15.148:0/3047095449 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fed74067190 0x7fed74069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:39:33.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:33.670+0000 7fed737fe640 1 -- 172.21.15.148:0/3047095449 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 3770+0+0 (secure 0 0 0) 0x7fed7c084400 con 0x7fed8c0fb240 2024-07-03T15:39:33.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:33.670+0000 7fed8a7fc640 1 --2- 172.21.15.148:0/3047095449 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fed74067190 0x7fed74069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:39:33.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:33.671+0000 7fed8a7fc640 1 --2- 172.21.15.148:0/3047095449 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fed74067190 0x7fed74069640 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fed800047f0 tx=0x7fed80009da0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:39:33.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:33.674+0000 7fed737fe640 1 -- 172.21.15.148:0/3047095449 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fed7c051dc0 con 0x7fed8c0fb240 2024-07-03T15:39:33.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:33.848+0000 7fed916bb640 1 -- 172.21.15.148:0/3047095449 --> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fed8c103f80 con 0x7fed74067190 2024-07-03T15:39:34.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:33 smithi028 ceph-mon[29148]: Detected new or changed devices on smithi148 2024-07-03T15:39:34.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:33 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:34.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:33 smithi028 ceph-mon[29148]: osdmap e29: 4 total, 4 up, 4 in 2024-07-03T15:39:34.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:33 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:34.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:33 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:39:34.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:33 smithi028 ceph-mon[29148]: Adjusting osd_memory_target on smithi148 to 15936M 2024-07-03T15:39:34.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:33 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:34.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:33 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:39:34.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:33 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:39:34.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:33 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:34.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:33 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:39:34.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:33 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:39:34.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:33 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:39:34.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:33 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:34.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:33 smithi148 ceph-mon[36056]: Detected new or changed devices on smithi148 2024-07-03T15:39:34.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:33 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:34.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:33 smithi148 ceph-mon[36056]: osdmap e29: 4 total, 4 up, 4 in 2024-07-03T15:39:34.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:33 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:34.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:33 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:39:34.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:33 smithi148 ceph-mon[36056]: Adjusting osd_memory_target on smithi148 to 15936M 2024-07-03T15:39:34.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:33 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:34.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:33 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:39:34.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:33 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:39:34.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:33 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:34.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:33 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:39:34.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:33 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:39:34.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:33 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:39:34.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:33 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:35.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:34 smithi028 ceph-mon[29148]: osdmap e30: 4 total, 4 up, 4 in 2024-07-03T15:39:35.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:34 smithi028 ceph-mon[29148]: from='client.24147 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:39:35.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:34 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-03T15:39:35.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:34 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-03T15:39:35.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:34 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:39:35.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:34 smithi028 ceph-mon[29148]: pgmap v95: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-07-03T15:39:35.129 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:34 smithi148 ceph-mon[36056]: osdmap e30: 4 total, 4 up, 4 in 2024-07-03T15:39:35.130 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:34 smithi148 ceph-mon[36056]: from='client.24147 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:39:35.130 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:34 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-03T15:39:35.130 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:34 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-03T15:39:35.130 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:34 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:39:35.130 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:34 smithi148 ceph-mon[36056]: pgmap v95: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-07-03T15:39:36.279 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:36 smithi148 ceph-mon[36056]: from='client.? 172.21.15.148:0/303848540' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d338fd80-0a76-441e-9cba-9cfd27cccc46"}]: dispatch 2024-07-03T15:39:36.280 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:36 smithi148 ceph-mon[36056]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d338fd80-0a76-441e-9cba-9cfd27cccc46"}]: dispatch 2024-07-03T15:39:36.280 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:36 smithi148 ceph-mon[36056]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d338fd80-0a76-441e-9cba-9cfd27cccc46"}]': finished 2024-07-03T15:39:36.280 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:36 smithi148 ceph-mon[36056]: osdmap e31: 5 total, 4 up, 5 in 2024-07-03T15:39:36.280 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:36 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-03T15:39:36.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:36 smithi028 ceph-mon[29148]: from='client.? 172.21.15.148:0/303848540' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d338fd80-0a76-441e-9cba-9cfd27cccc46"}]: dispatch 2024-07-03T15:39:36.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:36 smithi028 ceph-mon[29148]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d338fd80-0a76-441e-9cba-9cfd27cccc46"}]: dispatch 2024-07-03T15:39:36.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:36 smithi028 ceph-mon[29148]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d338fd80-0a76-441e-9cba-9cfd27cccc46"}]': finished 2024-07-03T15:39:36.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:36 smithi028 ceph-mon[29148]: osdmap e31: 5 total, 4 up, 5 in 2024-07-03T15:39:36.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:36 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-03T15:39:37.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:37 smithi148 ceph-mon[36056]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-07-03T15:39:37.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:37 smithi148 ceph-mon[36056]: from='client.? 172.21.15.148:0/456402844' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-03T15:39:37.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:37 smithi028 ceph-mon[29148]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-07-03T15:39:37.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:37 smithi028 ceph-mon[29148]: from='client.? 172.21.15.148:0/456402844' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-03T15:39:39.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:39 smithi148 ceph-mon[36056]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-07-03T15:39:39.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:39 smithi028 ceph-mon[29148]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-07-03T15:39:41.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:41 smithi148 ceph-mon[36056]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-07-03T15:39:41.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:41 smithi028 ceph-mon[29148]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-07-03T15:39:43.720 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:43 smithi148 ceph-mon[36056]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-07-03T15:39:43.720 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:43 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-07-03T15:39:43.720 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:43 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:39:43.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:43 smithi028 ceph-mon[29148]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-07-03T15:39:43.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:43 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-07-03T15:39:43.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:43 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:39:44.715 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:44 smithi148 ceph-mon[36056]: Deploying daemon osd.4 on smithi148 2024-07-03T15:39:44.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:44 smithi028 ceph-mon[29148]: Deploying daemon osd.4 on smithi148 2024-07-03T15:39:45.686 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:45 smithi148 ceph-mon[36056]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-03T15:39:45.687 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:45 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:39:45.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:45 smithi028 ceph-mon[29148]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-03T15:39:45.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:45 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:39:47.701 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:47 smithi148 ceph-mon[36056]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-07-03T15:39:47.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:47 smithi028 ceph-mon[29148]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-07-03T15:39:48.676 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:48 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:39:48.676 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:48 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:48.677 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:48 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:48.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:48 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:39:48.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:48 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:48.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:48 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:49.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:49 smithi148 ceph-mon[36056]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-07-03T15:39:49.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:49 smithi028 ceph-mon[29148]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-07-03T15:39:51.654 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:51 smithi148 ceph-mon[36056]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-07-03T15:39:51.654 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:51 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:51.654 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:51 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:51.654 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:51 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:39:51.654 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:51 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:39:51.655 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:51 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:51.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:51.687+0000 7fed737fe640 1 -- 172.21.15.148:0/3047095449 <== mgr.14210 v2:172.21.15.28:6800/421057178 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fed8c103f80 con 0x7fed74067190 2024-07-03T15:39:51.691 INFO:teuthology.orchestra.run.smithi148.stdout:Created osd(s) 4 on host 'smithi148' 2024-07-03T15:39:51.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:51.689+0000 7fed916bb640 1 -- 172.21.15.148:0/3047095449 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fed74067190 msgr2=0x7fed74069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:39:51.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:51.689+0000 7fed916bb640 1 --2- 172.21.15.148:0/3047095449 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fed74067190 0x7fed74069640 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fed800047f0 tx=0x7fed80009da0 comp rx=0 tx=0).stop 2024-07-03T15:39:51.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:51.689+0000 7fed916bb640 1 -- 172.21.15.148:0/3047095449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed8c0fb240 msgr2=0x7fed8c0ff360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:39:51.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:51.689+0000 7fed916bb640 1 --2- 172.21.15.148:0/3047095449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed8c0fb240 0x7fed8c0ff360 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fed7c00bbb0 tx=0x7fed7c009840 comp rx=0 tx=0).stop 2024-07-03T15:39:51.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:51.689+0000 7fed916bb640 1 -- 172.21.15.148:0/3047095449 shutdown_connections 2024-07-03T15:39:51.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:51.689+0000 7fed916bb640 1 --2- 172.21.15.148:0/3047095449 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fed74067190 0x7fed74069640 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:39:51.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:51.689+0000 7fed916bb640 1 --2- 172.21.15.148:0/3047095449 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed8c0fc510 0x7fed8c0fd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:39:51.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:51.689+0000 7fed916bb640 1 --2- 172.21.15.148:0/3047095449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed8c0fb240 0x7fed8c0ff360 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:39:51.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:51.689+0000 7fed916bb640 1 -- 172.21.15.148:0/3047095449 >> 172.21.15.148:0/3047095449 conn(0x7fed8c0f6a20 msgr2=0x7fed8c0653a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:39:51.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:51.689+0000 7fed916bb640 1 -- 172.21.15.148:0/3047095449 shutdown_connections 2024-07-03T15:39:51.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:51.689+0000 7fed916bb640 1 -- 172.21.15.148:0/3047095449 wait complete. 2024-07-03T15:39:51.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:51 smithi028 ceph-mon[29148]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-07-03T15:39:51.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:51 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:51.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:51 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:51.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:51 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:39:51.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:51 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:39:51.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:51 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:52.299 DEBUG:teuthology.orchestra.run.smithi148:osd.4> sudo journalctl -f -n 0 -u ceph-a85e4be6-3951-11ef-bcab-c7b262605968@osd.4.service 2024-07-03T15:39:52.302 INFO:tasks.cephadm:Deploying osd.5 on smithi148 with /dev/vg_nvme/lv_2... 2024-07-03T15:39:52.302 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-07-03T15:39:52.541 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:52 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:39:52.541 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:52 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:52.541 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:52 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:52.541 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:52 smithi148 ceph-mon[36056]: from='osd.4 [v2:172.21.15.148:6808/1943011533,v1:172.21.15.148:6809/1943011533]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-03T15:39:52.541 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:52 smithi148 ceph-mon[36056]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-03T15:39:52.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:52 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:39:52.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:52 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:52.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:52 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:52.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:52 smithi028 ceph-mon[29148]: from='osd.4 [v2:172.21.15.148:6808/1943011533,v1:172.21.15.148:6809/1943011533]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-03T15:39:52.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:52 smithi028 ceph-mon[29148]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-03T15:39:53.878 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi148/config 2024-07-03T15:39:53.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:53 smithi148 ceph-mon[36056]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-07-03T15:39:53.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:53 smithi148 ceph-mon[36056]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-07-03T15:39:53.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:53 smithi148 ceph-mon[36056]: osdmap e32: 5 total, 4 up, 5 in 2024-07-03T15:39:53.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:53 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-03T15:39:53.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:53 smithi148 ceph-mon[36056]: from='osd.4 [v2:172.21.15.148:6808/1943011533,v1:172.21.15.148:6809/1943011533]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-07-03T15:39:53.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:53 smithi148 ceph-mon[36056]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-07-03T15:39:53.982 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:39:53 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4[47919]: 2024-07-03T15:39:53.699+0000 7ff1cadf2640 -1 osd.4 0 waiting for initial osdmap 2024-07-03T15:39:53.982 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:39:53 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4[47919]: 2024-07-03T15:39:53.707+0000 7ff1c7c2d640 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-03T15:39:54.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:53 smithi028 ceph-mon[29148]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-07-03T15:39:54.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:53 smithi028 ceph-mon[29148]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-07-03T15:39:54.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:53 smithi028 ceph-mon[29148]: osdmap e32: 5 total, 4 up, 5 in 2024-07-03T15:39:54.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:53 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-03T15:39:54.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:53 smithi028 ceph-mon[29148]: from='osd.4 [v2:172.21.15.148:6808/1943011533,v1:172.21.15.148:6809/1943011533]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-07-03T15:39:54.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:53 smithi028 ceph-mon[29148]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-07-03T15:39:54.887 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:54 smithi148 ceph-mon[36056]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-07-03T15:39:54.888 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:54 smithi148 ceph-mon[36056]: osdmap e33: 5 total, 4 up, 5 in 2024-07-03T15:39:54.888 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:54 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-03T15:39:54.888 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:54 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-03T15:39:55.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:54 smithi028 ceph-mon[29148]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-07-03T15:39:55.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:54 smithi028 ceph-mon[29148]: osdmap e33: 5 total, 4 up, 5 in 2024-07-03T15:39:55.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:54 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-03T15:39:55.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:54 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-03T15:39:55.899 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:55 smithi148 ceph-mon[36056]: purged_snaps scrub starts 2024-07-03T15:39:55.899 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:55 smithi148 ceph-mon[36056]: purged_snaps scrub ok 2024-07-03T15:39:55.899 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:55 smithi148 ceph-mon[36056]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-07-03T15:39:55.899 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:55 smithi148 ceph-mon[36056]: osd.4 [v2:172.21.15.148:6808/1943011533,v1:172.21.15.148:6809/1943011533] boot 2024-07-03T15:39:55.899 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:55 smithi148 ceph-mon[36056]: osdmap e34: 5 total, 5 up, 5 in 2024-07-03T15:39:55.899 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:55 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-03T15:39:55.899 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:55 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:55.899 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:55 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:55.900 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:55 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:39:55.900 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:55 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:39:55.900 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:55 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:55.900 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:55 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:39:55.900 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:55 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:39:55.900 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:55 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:56.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:55 smithi028 ceph-mon[29148]: purged_snaps scrub starts 2024-07-03T15:39:56.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:55 smithi028 ceph-mon[29148]: purged_snaps scrub ok 2024-07-03T15:39:56.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:55 smithi028 ceph-mon[29148]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-07-03T15:39:56.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:55 smithi028 ceph-mon[29148]: osd.4 [v2:172.21.15.148:6808/1943011533,v1:172.21.15.148:6809/1943011533] boot 2024-07-03T15:39:56.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:55 smithi028 ceph-mon[29148]: osdmap e34: 5 total, 5 up, 5 in 2024-07-03T15:39:56.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:55 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-03T15:39:56.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:55 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:56.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:55 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:56.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:55 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:39:56.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:55 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:39:56.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:55 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:56.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:55 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:39:56.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:55 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:39:56.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:55 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:39:56.261 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-07-03T15:39:56.340 INFO:teuthology.orchestra.run.smithi148.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-07-03T15:39:56.355 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10+0 records in 2024-07-03T15:39:56.355 INFO:teuthology.orchestra.run.smithi148.stderr:10+0 records out 2024-07-03T15:39:56.355 INFO:teuthology.orchestra.run.smithi148.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0132377 s, 792 MB/s 2024-07-03T15:39:56.355 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping successful for: 2024-07-03T15:39:56.977 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:56 smithi148 ceph-mon[36056]: Detected new or changed devices on smithi148 2024-07-03T15:39:56.977 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:56 smithi148 ceph-mon[36056]: Adjusting osd_memory_target on smithi148 to 7968M 2024-07-03T15:39:56.977 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:56 smithi148 ceph-mon[36056]: osdmap e35: 5 total, 5 up, 5 in 2024-07-03T15:39:56.979 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph orch daemon add osd smithi148:vg_nvme/lv_2 2024-07-03T15:39:57.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:56 smithi028 ceph-mon[29148]: Detected new or changed devices on smithi148 2024-07-03T15:39:57.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:56 smithi028 ceph-mon[29148]: Adjusting osd_memory_target on smithi148 to 7968M 2024-07-03T15:39:57.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:56 smithi028 ceph-mon[29148]: osdmap e35: 5 total, 5 up, 5 in 2024-07-03T15:39:57.233 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi148/config 2024-07-03T15:39:57.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:57 smithi148 ceph-mon[36056]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-03T15:39:58.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:57 smithi028 ceph-mon[29148]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-03T15:39:58.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:58.679+0000 7f7d9c248640 1 -- 172.21.15.148:0/3135773873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d940fc520 msgr2=0x7f7d940fc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:39:58.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:58.679+0000 7f7d9c248640 1 --2- 172.21.15.148:0/3135773873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d940fc520 0x7f7d940fc990 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7d840048d0 tx=0x7f7d84009da0 comp rx=0 tx=0).stop 2024-07-03T15:39:58.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:58.679+0000 7f7d9c248640 1 -- 172.21.15.148:0/3135773873 shutdown_connections 2024-07-03T15:39:58.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:58.679+0000 7f7d9c248640 1 --2- 172.21.15.148:0/3135773873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d940fc520 0x7f7d940fc990 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:39:58.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:58.679+0000 7f7d9c248640 1 --2- 172.21.15.148:0/3135773873 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7d940fb250 0x7f7d940fb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:39:58.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:58.679+0000 7f7d9c248640 1 -- 172.21.15.148:0/3135773873 >> 172.21.15.148:0/3135773873 conn(0x7f7d940f6a10 msgr2=0x7f7d940f8e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:39:58.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:58.679+0000 7f7d9c248640 1 -- 172.21.15.148:0/3135773873 shutdown_connections 2024-07-03T15:39:58.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:58.679+0000 7f7d9c248640 1 -- 172.21.15.148:0/3135773873 wait complete. 2024-07-03T15:39:58.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:58.680+0000 7f7d9c248640 1 Processor -- start 2024-07-03T15:39:58.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:58.680+0000 7f7d9c248640 1 -- start start 2024-07-03T15:39:58.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:58.680+0000 7f7d9c248640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7d940fb250 0x7f7d9410fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:39:58.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:58.680+0000 7f7d9c248640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d940fc520 0x7f7d94112200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:39:58.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:58.680+0000 7f7d9c248640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d94066670 con 0x7f7d940fb250 2024-07-03T15:39:58.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:58.680+0000 7f7d9c248640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d940667e0 con 0x7f7d940fc520 2024-07-03T15:39:58.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:58.680+0000 7f7d997e3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d940fc520 0x7f7d94112200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:39:58.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:58.680+0000 7f7d997e3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d940fc520 0x7f7d94112200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44334/0 (socket says 172.21.15.148:44334) 2024-07-03T15:39:58.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:58.680+0000 7f7d997e3640 1 -- 172.21.15.148:0/1323947115 learned_addr learned my addr 172.21.15.148:0/1323947115 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-07-03T15:39:58.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:58.681+0000 7f7d997e3640 1 -- 172.21.15.148:0/1323947115 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7d940fb250 msgr2=0x7f7d9410fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:39:58.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:58.681+0000 7f7d997e3640 1 --2- 172.21.15.148:0/1323947115 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7d940fb250 0x7f7d9410fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:39:58.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:58.681+0000 7f7d997e3640 1 -- 172.21.15.148:0/1323947115 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d84005040 con 0x7f7d940fc520 2024-07-03T15:39:58.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:58.681+0000 7f7d997e3640 1 --2- 172.21.15.148:0/1323947115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d940fc520 0x7f7d94112200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7d84003080 tx=0x7f7d84009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:39:58.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:58.681+0000 7f7d8a7fc640 1 -- 172.21.15.148:0/1323947115 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d84038070 con 0x7f7d940fc520 2024-07-03T15:39:58.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:58.681+0000 7f7d9c248640 1 -- 172.21.15.148:0/1323947115 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d94112800 con 0x7f7d940fc520 2024-07-03T15:39:58.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:58.681+0000 7f7d8a7fc640 1 -- 172.21.15.148:0/1323947115 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7d84032970 con 0x7f7d940fc520 2024-07-03T15:39:58.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:58.681+0000 7f7d8a7fc640 1 -- 172.21.15.148:0/1323947115 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d84033930 con 0x7f7d940fc520 2024-07-03T15:39:58.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:58.681+0000 7f7d9c248640 1 -- 172.21.15.148:0/1323947115 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d94112cc0 con 0x7f7d940fc520 2024-07-03T15:39:58.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:58.682+0000 7f7d9c248640 1 -- 172.21.15.148:0/1323947115 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d940fa6b0 con 0x7f7d940fc520 2024-07-03T15:39:58.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:58.682+0000 7f7d8a7fc640 1 -- 172.21.15.148:0/1323947115 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f7d84030020 con 0x7f7d940fc520 2024-07-03T15:39:58.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:58.683+0000 7f7d8a7fc640 1 --2- 172.21.15.148:0/1323947115 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f7d80067190 0x7f7d80069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:39:58.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:58.683+0000 7f7d8a7fc640 1 -- 172.21.15.148:0/1323947115 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4202+0+0 (secure 0 0 0) 0x7f7d840a46c0 con 0x7f7d940fc520 2024-07-03T15:39:58.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:58.683+0000 7f7d99fe4640 1 --2- 172.21.15.148:0/1323947115 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f7d80067190 0x7f7d80069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:39:58.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:58.683+0000 7f7d99fe4640 1 --2- 172.21.15.148:0/1323947115 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f7d80067190 0x7f7d80069640 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7d940662d0 tx=0x7f7d90006d20 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:39:58.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:58.687+0000 7f7d8a7fc640 1 -- 172.21.15.148:0/1323947115 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7d84071e50 con 0x7f7d940fc520 2024-07-03T15:39:58.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:39:58.860+0000 7f7d9c248640 1 -- 172.21.15.148:0/1323947115 --> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d94100840 con 0x7f7d80067190 2024-07-03T15:39:59.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:59 smithi148 ceph-mon[36056]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-03T15:39:59.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:59 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-03T15:39:59.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:59 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-03T15:39:59.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:39:59 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:40:00.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:59 smithi028 ceph-mon[29148]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-03T15:40:00.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:59 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-03T15:40:00.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:59 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-03T15:40:00.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:39:59 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:40:00.847 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:00 smithi148 ceph-mon[36056]: from='client.24165 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:40:00.847 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:00 smithi148 ceph-mon[36056]: overall HEALTH_OK 2024-07-03T15:40:00.847 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:00 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:40:01.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:00 smithi028 ceph-mon[29148]: from='client.24165 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:40:01.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:00 smithi028 ceph-mon[29148]: overall HEALTH_OK 2024-07-03T15:40:01.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:00 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:40:01.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:01 smithi148 ceph-mon[36056]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-03T15:40:01.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:01 smithi148 ceph-mon[36056]: from='client.? 172.21.15.148:0/2415985361' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5c87b168-85e8-4d65-9763-8f1626e474ac"}]: dispatch 2024-07-03T15:40:01.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:01 smithi148 ceph-mon[36056]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5c87b168-85e8-4d65-9763-8f1626e474ac"}]: dispatch 2024-07-03T15:40:01.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:01 smithi148 ceph-mon[36056]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5c87b168-85e8-4d65-9763-8f1626e474ac"}]': finished 2024-07-03T15:40:01.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:01 smithi148 ceph-mon[36056]: osdmap e36: 6 total, 5 up, 6 in 2024-07-03T15:40:01.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:01 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-03T15:40:02.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:01 smithi028 ceph-mon[29148]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-03T15:40:02.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:01 smithi028 ceph-mon[29148]: from='client.? 172.21.15.148:0/2415985361' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5c87b168-85e8-4d65-9763-8f1626e474ac"}]: dispatch 2024-07-03T15:40:02.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:01 smithi028 ceph-mon[29148]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5c87b168-85e8-4d65-9763-8f1626e474ac"}]: dispatch 2024-07-03T15:40:02.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:01 smithi028 ceph-mon[29148]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5c87b168-85e8-4d65-9763-8f1626e474ac"}]': finished 2024-07-03T15:40:02.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:01 smithi028 ceph-mon[29148]: osdmap e36: 6 total, 5 up, 6 in 2024-07-03T15:40:02.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:01 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-03T15:40:03.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:02 smithi028 ceph-mon[29148]: from='client.? 172.21.15.148:0/2615056971' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-03T15:40:03.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:02 smithi148 ceph-mon[36056]: from='client.? 172.21.15.148:0/2615056971' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-03T15:40:04.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:03 smithi028 ceph-mon[29148]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-03T15:40:04.133 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:03 smithi148 ceph-mon[36056]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-03T15:40:05.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:05 smithi148 ceph-mon[36056]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-03T15:40:06.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:05 smithi028 ceph-mon[29148]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-03T15:40:06.949 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:06 smithi148 ceph-mon[36056]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-03T15:40:07.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:06 smithi028 ceph-mon[29148]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-03T15:40:08.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:08 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-07-03T15:40:08.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:08 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:40:08.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:08 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-07-03T15:40:08.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:08 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:40:09.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:09 smithi148 ceph-mon[36056]: Deploying daemon osd.5 on smithi148 2024-07-03T15:40:09.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:09 smithi148 ceph-mon[36056]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-03T15:40:09.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:09 smithi028 ceph-mon[29148]: Deploying daemon osd.5 on smithi148 2024-07-03T15:40:09.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:09 smithi028 ceph-mon[29148]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-03T15:40:11.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:11 smithi148 ceph-mon[36056]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-03T15:40:11.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:11 smithi028 ceph-mon[29148]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-03T15:40:13.514 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:13 smithi148 ceph-mon[36056]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-03T15:40:13.514 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:13 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:40:13.514 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:13 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:40:13.514 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:13 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:40:13.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:13 smithi028 ceph-mon[29148]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-03T15:40:13.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:13 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:40:13.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:13 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:40:13.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:13 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:40:15.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:15.481+0000 7f7d8a7fc640 1 -- 172.21.15.148:0/1323947115 <== mgr.14210 v2:172.21.15.28:6800/421057178 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7d94100840 con 0x7f7d80067190 2024-07-03T15:40:15.485 INFO:teuthology.orchestra.run.smithi148.stdout:Created osd(s) 5 on host 'smithi148' 2024-07-03T15:40:15.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:15.483+0000 7f7d9c248640 1 -- 172.21.15.148:0/1323947115 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f7d80067190 msgr2=0x7f7d80069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:15.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:15.483+0000 7f7d9c248640 1 --2- 172.21.15.148:0/1323947115 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f7d80067190 0x7f7d80069640 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7d940662d0 tx=0x7f7d90006d20 comp rx=0 tx=0).stop 2024-07-03T15:40:15.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:15.483+0000 7f7d9c248640 1 -- 172.21.15.148:0/1323947115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d940fc520 msgr2=0x7f7d94112200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:15.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:15.483+0000 7f7d9c248640 1 --2- 172.21.15.148:0/1323947115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d940fc520 0x7f7d94112200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7d84003080 tx=0x7f7d84009da0 comp rx=0 tx=0).stop 2024-07-03T15:40:15.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:15.483+0000 7f7d9c248640 1 -- 172.21.15.148:0/1323947115 shutdown_connections 2024-07-03T15:40:15.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:15.483+0000 7f7d9c248640 1 --2- 172.21.15.148:0/1323947115 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f7d80067190 0x7f7d80069640 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:15.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:15.483+0000 7f7d9c248640 1 --2- 172.21.15.148:0/1323947115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d940fc520 0x7f7d94112200 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:15.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:15.483+0000 7f7d9c248640 1 --2- 172.21.15.148:0/1323947115 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7d940fb250 0x7f7d9410fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:15.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:15.484+0000 7f7d9c248640 1 -- 172.21.15.148:0/1323947115 >> 172.21.15.148:0/1323947115 conn(0x7f7d940f6a10 msgr2=0x7f7d940f80a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:40:15.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:15.484+0000 7f7d9c248640 1 -- 172.21.15.148:0/1323947115 shutdown_connections 2024-07-03T15:40:15.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:15.484+0000 7f7d9c248640 1 -- 172.21.15.148:0/1323947115 wait complete. 2024-07-03T15:40:15.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:15 smithi148 ceph-mon[36056]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-03T15:40:15.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:15 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:40:15.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:15 smithi028 ceph-mon[29148]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-03T15:40:15.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:15 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:40:16.104 DEBUG:teuthology.orchestra.run.smithi148:osd.5> sudo journalctl -f -n 0 -u ceph-a85e4be6-3951-11ef-bcab-c7b262605968@osd.5.service 2024-07-03T15:40:16.106 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-07-03T15:40:16.106 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph osd stat -f json 2024-07-03T15:40:16.387 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:40:16.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:16 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:40:16.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:16 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:40:16.732 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:40:16 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5[54485]: 2024-07-03T15:40:16.683+0000 7fc3aa7b52c0 -1 osd.5 0 log_to_monitors true 2024-07-03T15:40:16.778 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:16 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:40:16.778 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:16 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:40:17.841 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:17 smithi028 ceph-mon[29148]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-03T15:40:17.841 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:17 smithi028 ceph-mon[29148]: from='osd.5 [v2:172.21.15.148:6816/1578977000,v1:172.21.15.148:6817/1578977000]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-07-03T15:40:17.841 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:17 smithi028 ceph-mon[29148]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-07-03T15:40:17.841 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:17 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:40:17.841 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:17 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:40:17.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:17 smithi148 ceph-mon[36056]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-03T15:40:17.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:17 smithi148 ceph-mon[36056]: from='osd.5 [v2:172.21.15.148:6816/1578977000,v1:172.21.15.148:6817/1578977000]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-07-03T15:40:17.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:17 smithi148 ceph-mon[36056]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-07-03T15:40:17.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:17 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:40:17.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:17 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:40:18.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.145+0000 7f8128b40640 1 -- 172.21.15.28:0/1817922654 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f81240fc560 msgr2=0x7f81240fc9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:18.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.145+0000 7f8128b40640 1 --2- 172.21.15.28:0/1817922654 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f81240fc560 0x7f81240fc9d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f810c0048d0 tx=0x7f810c009da0 comp rx=0 tx=0).stop 2024-07-03T15:40:18.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.145+0000 7f8128b40640 1 -- 172.21.15.28:0/1817922654 shutdown_connections 2024-07-03T15:40:18.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.145+0000 7f8128b40640 1 --2- 172.21.15.28:0/1817922654 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f81240fc560 0x7f81240fc9d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:18.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.145+0000 7f8128b40640 1 --2- 172.21.15.28:0/1817922654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81240fb290 0x7f81240fb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:18.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.145+0000 7f8128b40640 1 -- 172.21.15.28:0/1817922654 >> 172.21.15.28:0/1817922654 conn(0x7f81240f6a50 msgr2=0x7f81240f8e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:40:18.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.145+0000 7f8128b40640 1 -- 172.21.15.28:0/1817922654 shutdown_connections 2024-07-03T15:40:18.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.145+0000 7f8128b40640 1 -- 172.21.15.28:0/1817922654 wait complete. 2024-07-03T15:40:18.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.146+0000 7f8128b40640 1 Processor -- start 2024-07-03T15:40:18.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.146+0000 7f8128b40640 1 -- start start 2024-07-03T15:40:18.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.146+0000 7f8128b40640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81240fb290 0x7f812410bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:18.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.146+0000 7f8128b40640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f81240fc560 0x7f812410c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:18.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.146+0000 7f8128b40640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812410d300 con 0x7f81240fc560 2024-07-03T15:40:18.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.146+0000 7f8128b40640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812410d470 con 0x7f81240fb290 2024-07-03T15:40:18.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.146+0000 7f811bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f81240fc560 0x7f812410c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:18.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.146+0000 7f811bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f81240fc560 0x7f812410c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46052/0 (socket says 172.21.15.28:46052) 2024-07-03T15:40:18.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.146+0000 7f811bfff640 1 -- 172.21.15.28:0/2507590837 learned_addr learned my addr 172.21.15.28:0/2507590837 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:40:18.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.146+0000 7f812259c640 1 --2- 172.21.15.28:0/2507590837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81240fb290 0x7f812410bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:18.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.146+0000 7f811bfff640 1 -- 172.21.15.28:0/2507590837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81240fb290 msgr2=0x7f812410bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:18.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.146+0000 7f811bfff640 1 --2- 172.21.15.28:0/2507590837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81240fb290 0x7f812410bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:18.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.146+0000 7f811bfff640 1 -- 172.21.15.28:0/2507590837 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f810c005040 con 0x7f81240fc560 2024-07-03T15:40:18.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.147+0000 7f811bfff640 1 --2- 172.21.15.28:0/2507590837 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f81240fc560 0x7f812410c490 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f810c003080 tx=0x7f810c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:40:18.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.147+0000 7f811affd640 1 -- 172.21.15.28:0/2507590837 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f810c038070 con 0x7f81240fc560 2024-07-03T15:40:18.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.147+0000 7f811affd640 1 -- 172.21.15.28:0/2507590837 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f810c032970 con 0x7f81240fc560 2024-07-03T15:40:18.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.147+0000 7f8128b40640 1 -- 172.21.15.28:0/2507590837 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f812410d6f0 con 0x7f81240fc560 2024-07-03T15:40:18.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.147+0000 7f811affd640 1 -- 172.21.15.28:0/2507590837 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f810c033820 con 0x7f81240fc560 2024-07-03T15:40:18.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.147+0000 7f8128b40640 1 -- 172.21.15.28:0/2507590837 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f812410dbb0 con 0x7f81240fc560 2024-07-03T15:40:18.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.148+0000 7f8128b40640 1 -- 172.21.15.28:0/2507590837 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81240fa6f0 con 0x7f81240fc560 2024-07-03T15:40:18.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.148+0000 7f811affd640 1 -- 172.21.15.28:0/2507590837 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f810c030020 con 0x7f81240fc560 2024-07-03T15:40:18.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.149+0000 7f811affd640 1 --2- 172.21.15.28:0/2507590837 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f8110067190 0x7f8110069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:18.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.149+0000 7f811affd640 1 -- 172.21.15.28:0/2507590837 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4338+0+0 (secure 0 0 0) 0x7f810c0a45a0 con 0x7f81240fc560 2024-07-03T15:40:18.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.149+0000 7f812259c640 1 --2- 172.21.15.28:0/2507590837 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f8110067190 0x7f8110069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:18.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.149+0000 7f812259c640 1 --2- 172.21.15.28:0/2507590837 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f8110067190 0x7f8110069640 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8114009b50 tx=0x7f8114009120 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:40:18.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.153+0000 7f811affd640 1 -- 172.21.15.28:0/2507590837 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f810c071d50 con 0x7f81240fc560 2024-07-03T15:40:18.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.328+0000 7f8128b40640 1 -- 172.21.15.28:0/2507590837 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f812410de90 con 0x7f81240fc560 2024-07-03T15:40:18.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.328+0000 7f811affd640 1 -- 172.21.15.28:0/2507590837 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v37) v1 ==== 74+0+130 (secure 0 0 0) 0x7f810c075370 con 0x7f81240fc560 2024-07-03T15:40:18.332 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:40:18.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.330+0000 7f8128b40640 1 -- 172.21.15.28:0/2507590837 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f8110067190 msgr2=0x7f8110069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:18.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.330+0000 7f8128b40640 1 --2- 172.21.15.28:0/2507590837 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f8110067190 0x7f8110069640 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8114009b50 tx=0x7f8114009120 comp rx=0 tx=0).stop 2024-07-03T15:40:18.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.330+0000 7f8128b40640 1 -- 172.21.15.28:0/2507590837 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f81240fc560 msgr2=0x7f812410c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:18.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.330+0000 7f8128b40640 1 --2- 172.21.15.28:0/2507590837 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f81240fc560 0x7f812410c490 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f810c003080 tx=0x7f810c009da0 comp rx=0 tx=0).stop 2024-07-03T15:40:18.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.330+0000 7f8128b40640 1 -- 172.21.15.28:0/2507590837 shutdown_connections 2024-07-03T15:40:18.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.330+0000 7f8128b40640 1 --2- 172.21.15.28:0/2507590837 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f8110067190 0x7f8110069640 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:18.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.330+0000 7f8128b40640 1 --2- 172.21.15.28:0/2507590837 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f81240fc560 0x7f812410c490 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:18.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.330+0000 7f8128b40640 1 --2- 172.21.15.28:0/2507590837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81240fb290 0x7f812410bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:18.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.330+0000 7f8128b40640 1 -- 172.21.15.28:0/2507590837 >> 172.21.15.28:0/2507590837 conn(0x7f81240f6a50 msgr2=0x7f81240f8150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:40:18.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.330+0000 7f8128b40640 1 -- 172.21.15.28:0/2507590837 shutdown_connections 2024-07-03T15:40:18.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:18.330+0000 7f8128b40640 1 -- 172.21.15.28:0/2507590837 wait complete. 2024-07-03T15:40:18.731 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:40:18 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5[54485]: 2024-07-03T15:40:18.492+0000 7fc3a5f27640 -1 osd.5 0 waiting for initial osdmap 2024-07-03T15:40:18.732 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:40:18 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5[54485]: 2024-07-03T15:40:18.500+0000 7fc3a055d640 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-03T15:40:18.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:18 smithi148 ceph-mon[36056]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-07-03T15:40:18.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:18 smithi148 ceph-mon[36056]: osdmap e37: 6 total, 5 up, 6 in 2024-07-03T15:40:18.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:18 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-03T15:40:18.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:18 smithi148 ceph-mon[36056]: from='osd.5 [v2:172.21.15.148:6816/1578977000,v1:172.21.15.148:6817/1578977000]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-07-03T15:40:18.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:18 smithi148 ceph-mon[36056]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-07-03T15:40:18.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:18 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/2507590837' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-07-03T15:40:18.811 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:18 smithi028 ceph-mon[29148]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-07-03T15:40:18.811 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:18 smithi028 ceph-mon[29148]: osdmap e37: 6 total, 5 up, 6 in 2024-07-03T15:40:18.811 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:18 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-03T15:40:18.811 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:18 smithi028 ceph-mon[29148]: from='osd.5 [v2:172.21.15.148:6816/1578977000,v1:172.21.15.148:6817/1578977000]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-07-03T15:40:18.811 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:18 smithi028 ceph-mon[29148]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-07-03T15:40:18.811 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:18 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/2507590837' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-07-03T15:40:19.005 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":37,"num_osds":6,"num_up_osds":5,"osd_up_since":1720021194,"num_in_osds":6,"osd_in_since":1720021201,"num_remapped_pgs":0} 2024-07-03T15:40:19.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:19 smithi148 ceph-mon[36056]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-03T15:40:19.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:19 smithi148 ceph-mon[36056]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-07-03T15:40:19.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:19 smithi148 ceph-mon[36056]: osdmap e38: 6 total, 5 up, 6 in 2024-07-03T15:40:19.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:19 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-03T15:40:19.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:19 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-03T15:40:19.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:19 smithi028 ceph-mon[29148]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-03T15:40:19.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:19 smithi028 ceph-mon[29148]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-07-03T15:40:19.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:19 smithi028 ceph-mon[29148]: osdmap e38: 6 total, 5 up, 6 in 2024-07-03T15:40:19.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:19 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-03T15:40:19.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:19 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-03T15:40:20.007 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph osd stat -f json 2024-07-03T15:40:20.278 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:40:20.708 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:20 smithi028 ceph-mon[29148]: purged_snaps scrub starts 2024-07-03T15:40:20.708 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:20 smithi028 ceph-mon[29148]: purged_snaps scrub ok 2024-07-03T15:40:20.708 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:20 smithi028 ceph-mon[29148]: osd.5 [v2:172.21.15.148:6816/1578977000,v1:172.21.15.148:6817/1578977000] boot 2024-07-03T15:40:20.708 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:20 smithi028 ceph-mon[29148]: osdmap e39: 6 total, 6 up, 6 in 2024-07-03T15:40:20.709 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:20 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-03T15:40:20.709 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:20 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:40:20.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:20 smithi148 ceph-mon[36056]: purged_snaps scrub starts 2024-07-03T15:40:20.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:20 smithi148 ceph-mon[36056]: purged_snaps scrub ok 2024-07-03T15:40:20.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:20 smithi148 ceph-mon[36056]: osd.5 [v2:172.21.15.148:6816/1578977000,v1:172.21.15.148:6817/1578977000] boot 2024-07-03T15:40:20.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:20 smithi148 ceph-mon[36056]: osdmap e39: 6 total, 6 up, 6 in 2024-07-03T15:40:20.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:20 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-03T15:40:20.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:20 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:40:21.754 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:21 smithi028 ceph-mon[29148]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-03T15:40:21.754 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:21 smithi028 ceph-mon[29148]: Detected new or changed devices on smithi148 2024-07-03T15:40:21.754 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:21 smithi028 ceph-mon[29148]: osdmap e40: 6 total, 6 up, 6 in 2024-07-03T15:40:21.755 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:21 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:40:21.755 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:21 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:40:21.755 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:21 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:40:21.755 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:21 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:40:21.755 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:21 smithi028 ceph-mon[29148]: Adjusting osd_memory_target on smithi148 to 5312M 2024-07-03T15:40:21.755 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:21 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:40:21.755 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:21 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:40:21.755 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:21 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:40:21.755 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:21 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:40:21.756 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:21 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:40:21.756 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:21 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:40:21.756 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:21 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:40:21.756 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:21 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:40:21.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:21 smithi148 ceph-mon[36056]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-03T15:40:21.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:21 smithi148 ceph-mon[36056]: Detected new or changed devices on smithi148 2024-07-03T15:40:21.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:21 smithi148 ceph-mon[36056]: osdmap e40: 6 total, 6 up, 6 in 2024-07-03T15:40:21.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:21 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:40:21.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:21 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:40:21.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:21 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:40:21.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:21 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:40:21.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:21 smithi148 ceph-mon[36056]: Adjusting osd_memory_target on smithi148 to 5312M 2024-07-03T15:40:21.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:21 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:40:21.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:21 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:40:21.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:21 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:40:21.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:21 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:40:21.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:21 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:40:21.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:21 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:40:21.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:21 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:40:21.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:21 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:40:22.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.058+0000 7fb48054f640 1 -- 172.21.15.28:0/1103176437 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb4780fe3c0 msgr2=0x7fb4780fa520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:22.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.058+0000 7fb48054f640 1 --2- 172.21.15.28:0/1103176437 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb4780fe3c0 0x7fb4780fa520 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb4680048d0 tx=0x7fb468009da0 comp rx=0 tx=0).stop 2024-07-03T15:40:22.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.059+0000 7fb48054f640 1 -- 172.21.15.28:0/1103176437 shutdown_connections 2024-07-03T15:40:22.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.059+0000 7fb48054f640 1 --2- 172.21.15.28:0/1103176437 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb4780fe3c0 0x7fb4780fa520 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:22.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.059+0000 7fb48054f640 1 --2- 172.21.15.28:0/1103176437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4780fda70 0x7fb4780fde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:22.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.059+0000 7fb48054f640 1 -- 172.21.15.28:0/1103176437 >> 172.21.15.28:0/1103176437 conn(0x7fb4780f61b0 msgr2=0x7fb4780f85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:40:22.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.059+0000 7fb48054f640 1 -- 172.21.15.28:0/1103176437 shutdown_connections 2024-07-03T15:40:22.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.059+0000 7fb48054f640 1 -- 172.21.15.28:0/1103176437 wait complete. 2024-07-03T15:40:22.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.059+0000 7fb48054f640 1 Processor -- start 2024-07-03T15:40:22.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.059+0000 7fb48054f640 1 -- start start 2024-07-03T15:40:22.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.059+0000 7fb48054f640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb4780fda70 0x7fb4780fd260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:22.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.059+0000 7fb48054f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4780fe3c0 0x7fb4780fb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:22.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.059+0000 7fb48054f640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4780fbce0 con 0x7fb4780fda70 2024-07-03T15:40:22.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.059+0000 7fb48054f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4780fbe20 con 0x7fb4780fe3c0 2024-07-03T15:40:22.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.060+0000 7fb47e2eb640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb4780fda70 0x7fb4780fd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:22.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.060+0000 7fb47e2eb640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb4780fda70 0x7fb4780fd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48156/0 (socket says 172.21.15.28:48156) 2024-07-03T15:40:22.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.060+0000 7fb47e2eb640 1 -- 172.21.15.28:0/1301682467 learned_addr learned my addr 172.21.15.28:0/1301682467 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:40:22.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.060+0000 7fb47daea640 1 --2- 172.21.15.28:0/1301682467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4780fe3c0 0x7fb4780fb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:22.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.060+0000 7fb47e2eb640 1 -- 172.21.15.28:0/1301682467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4780fe3c0 msgr2=0x7fb4780fb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:22.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.060+0000 7fb47e2eb640 1 --2- 172.21.15.28:0/1301682467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4780fe3c0 0x7fb4780fb750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:22.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.060+0000 7fb47e2eb640 1 -- 172.21.15.28:0/1301682467 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb468005040 con 0x7fb4780fda70 2024-07-03T15:40:22.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.060+0000 7fb47e2eb640 1 --2- 172.21.15.28:0/1301682467 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb4780fda70 0x7fb4780fd260 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb474002a50 tx=0x7fb474004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:40:22.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.060+0000 7fb46effd640 1 -- 172.21.15.28:0/1301682467 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb474007c30 con 0x7fb4780fda70 2024-07-03T15:40:22.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.060+0000 7fb46effd640 1 -- 172.21.15.28:0/1301682467 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb47400f040 con 0x7fb4780fda70 2024-07-03T15:40:22.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.060+0000 7fb48054f640 1 -- 172.21.15.28:0/1301682467 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4780fc100 con 0x7fb4780fda70 2024-07-03T15:40:22.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.060+0000 7fb46effd640 1 -- 172.21.15.28:0/1301682467 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb47400db60 con 0x7fb4780fda70 2024-07-03T15:40:22.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.060+0000 7fb48054f640 1 -- 172.21.15.28:0/1301682467 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb478114fa0 con 0x7fb4780fda70 2024-07-03T15:40:22.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.062+0000 7fb48054f640 1 -- 172.21.15.28:0/1301682467 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4780f9e50 con 0x7fb4780fda70 2024-07-03T15:40:22.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.062+0000 7fb46effd640 1 -- 172.21.15.28:0/1301682467 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7fb47400dcc0 con 0x7fb4780fda70 2024-07-03T15:40:22.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.062+0000 7fb46effd640 1 --2- 172.21.15.28:0/1301682467 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fb464067190 0x7fb464069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:22.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.063+0000 7fb46effd640 1 -- 172.21.15.28:0/1301682467 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fb474084550 con 0x7fb4780fda70 2024-07-03T15:40:22.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.063+0000 7fb47daea640 1 --2- 172.21.15.28:0/1301682467 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fb464067190 0x7fb464069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:22.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.063+0000 7fb47daea640 1 --2- 172.21.15.28:0/1301682467 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fb464067190 0x7fb464069640 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb468000c00 tx=0x7fb468035040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:40:22.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.067+0000 7fb46effd640 1 -- 172.21.15.28:0/1301682467 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb474051bb0 con 0x7fb4780fda70 2024-07-03T15:40:22.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.240+0000 7fb48054f640 1 -- 172.21.15.28:0/1301682467 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fb47804b530 con 0x7fb4780fda70 2024-07-03T15:40:22.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.240+0000 7fb46effd640 1 -- 172.21.15.28:0/1301682467 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v40) v1 ==== 74+0+130 (secure 0 0 0) 0x7fb4740551d0 con 0x7fb4780fda70 2024-07-03T15:40:22.243 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:40:22.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.242+0000 7fb48054f640 1 -- 172.21.15.28:0/1301682467 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fb464067190 msgr2=0x7fb464069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:22.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.242+0000 7fb48054f640 1 --2- 172.21.15.28:0/1301682467 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fb464067190 0x7fb464069640 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb468000c00 tx=0x7fb468035040 comp rx=0 tx=0).stop 2024-07-03T15:40:22.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.242+0000 7fb48054f640 1 -- 172.21.15.28:0/1301682467 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb4780fda70 msgr2=0x7fb4780fd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:22.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.242+0000 7fb48054f640 1 --2- 172.21.15.28:0/1301682467 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb4780fda70 0x7fb4780fd260 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb474002a50 tx=0x7fb474004600 comp rx=0 tx=0).stop 2024-07-03T15:40:22.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.242+0000 7fb48054f640 1 -- 172.21.15.28:0/1301682467 shutdown_connections 2024-07-03T15:40:22.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.242+0000 7fb48054f640 1 --2- 172.21.15.28:0/1301682467 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fb464067190 0x7fb464069640 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:22.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.242+0000 7fb48054f640 1 --2- 172.21.15.28:0/1301682467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4780fe3c0 0x7fb4780fb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:22.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.242+0000 7fb48054f640 1 --2- 172.21.15.28:0/1301682467 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb4780fda70 0x7fb4780fd260 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:22.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.242+0000 7fb48054f640 1 -- 172.21.15.28:0/1301682467 >> 172.21.15.28:0/1301682467 conn(0x7fb4780f61b0 msgr2=0x7fb4780f7830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:40:22.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.242+0000 7fb48054f640 1 -- 172.21.15.28:0/1301682467 shutdown_connections 2024-07-03T15:40:22.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:22.242+0000 7fb48054f640 1 -- 172.21.15.28:0/1301682467 wait complete. 2024-07-03T15:40:22.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:22 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/1301682467' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-07-03T15:40:22.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:22 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/1301682467' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-07-03T15:40:23.100 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":40,"num_osds":6,"num_up_osds":6,"osd_up_since":1720021219,"num_in_osds":6,"osd_in_since":1720021201,"num_remapped_pgs":0} 2024-07-03T15:40:23.101 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph osd dump --format=json 2024-07-03T15:40:23.373 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:40:23.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:23 smithi148 ceph-mon[36056]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:40:23.813 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:23 smithi028 ceph-mon[29148]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:40:25.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.156+0000 7fb3be3bb640 1 -- 172.21.15.28:0/3221353665 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3b8102cf0 msgr2=0x7fb3b80fc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:25.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.156+0000 7fb3be3bb640 1 --2- 172.21.15.28:0/3221353665 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3b8102cf0 0x7fb3b80fc380 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac0048d0 tx=0x7fb3ac009da0 comp rx=0 tx=0).stop 2024-07-03T15:40:25.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.156+0000 7fb3be3bb640 1 -- 172.21.15.28:0/3221353665 shutdown_connections 2024-07-03T15:40:25.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.156+0000 7fb3be3bb640 1 --2- 172.21.15.28:0/3221353665 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3b8102cf0 0x7fb3b80fc380 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:25.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.156+0000 7fb3be3bb640 1 --2- 172.21.15.28:0/3221353665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3b8102350 0x7fb3b8102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:25.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.156+0000 7fb3be3bb640 1 -- 172.21.15.28:0/3221353665 >> 172.21.15.28:0/3221353665 conn(0x7fb3b80f7f20 msgr2=0x7fb3b80fa330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:40:25.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.156+0000 7fb3be3bb640 1 -- 172.21.15.28:0/3221353665 shutdown_connections 2024-07-03T15:40:25.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.156+0000 7fb3be3bb640 1 -- 172.21.15.28:0/3221353665 wait complete. 2024-07-03T15:40:25.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.156+0000 7fb3be3bb640 1 Processor -- start 2024-07-03T15:40:25.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.156+0000 7fb3be3bb640 1 -- start start 2024-07-03T15:40:25.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.157+0000 7fb3be3bb640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3b8102350 0x7fb3b81130b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:25.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.157+0000 7fb3be3bb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3b8102cf0 0x7fb3b81135f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:25.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.157+0000 7fb3be3bb640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b8113bf0 con 0x7fb3b8102350 2024-07-03T15:40:25.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.157+0000 7fb3be3bb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b810d030 con 0x7fb3b8102cf0 2024-07-03T15:40:25.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.157+0000 7fb3b7fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3b8102350 0x7fb3b81130b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:25.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.157+0000 7fb3b7fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3b8102350 0x7fb3b81130b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48180/0 (socket says 172.21.15.28:48180) 2024-07-03T15:40:25.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.157+0000 7fb3b7fff640 1 -- 172.21.15.28:0/1411409752 learned_addr learned my addr 172.21.15.28:0/1411409752 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:40:25.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.157+0000 7fb3b77fe640 1 --2- 172.21.15.28:0/1411409752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3b8102cf0 0x7fb3b81135f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:25.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.157+0000 7fb3b7fff640 1 -- 172.21.15.28:0/1411409752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3b8102cf0 msgr2=0x7fb3b81135f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:25.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.157+0000 7fb3b7fff640 1 --2- 172.21.15.28:0/1411409752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3b8102cf0 0x7fb3b81135f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:25.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.157+0000 7fb3b7fff640 1 -- 172.21.15.28:0/1411409752 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3ac005040 con 0x7fb3b8102350 2024-07-03T15:40:25.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.157+0000 7fb3b7fff640 1 --2- 172.21.15.28:0/1411409752 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3b8102350 0x7fb3b81130b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb3a800c810 tx=0x7fb3a80043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:40:25.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.157+0000 7fb3b4ff9640 1 -- 172.21.15.28:0/1411409752 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3a800a7a0 con 0x7fb3b8102350 2024-07-03T15:40:25.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.157+0000 7fb3b4ff9640 1 -- 172.21.15.28:0/1411409752 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb3a800add0 con 0x7fb3b8102350 2024-07-03T15:40:25.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.157+0000 7fb3b4ff9640 1 -- 172.21.15.28:0/1411409752 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3a80115c0 con 0x7fb3b8102350 2024-07-03T15:40:25.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.157+0000 7fb3be3bb640 1 -- 172.21.15.28:0/1411409752 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3b810d310 con 0x7fb3b8102350 2024-07-03T15:40:25.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.158+0000 7fb3be3bb640 1 -- 172.21.15.28:0/1411409752 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b810d8b0 con 0x7fb3b8102350 2024-07-03T15:40:25.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.158+0000 7fb3be3bb640 1 -- 172.21.15.28:0/1411409752 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3b80fdac0 con 0x7fb3b8102350 2024-07-03T15:40:25.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.159+0000 7fb3b4ff9640 1 -- 172.21.15.28:0/1411409752 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7fb3a8012420 con 0x7fb3b8102350 2024-07-03T15:40:25.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.159+0000 7fb3b4ff9640 1 --2- 172.21.15.28:0/1411409752 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fb3a0067190 0x7fb3a0069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:25.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.160+0000 7fb3b4ff9640 1 -- 172.21.15.28:0/1411409752 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fb3a8084c20 con 0x7fb3b8102350 2024-07-03T15:40:25.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.160+0000 7fb3b77fe640 1 --2- 172.21.15.28:0/1411409752 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fb3a0067190 0x7fb3a0069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:25.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.160+0000 7fb3b77fe640 1 --2- 172.21.15.28:0/1411409752 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fb3a0067190 0x7fb3a0069640 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb3b810e970 tx=0x7fb3ac035040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:40:25.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.163+0000 7fb3b4ff9640 1 -- 172.21.15.28:0/1411409752 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb3a8052280 con 0x7fb3b8102350 2024-07-03T15:40:25.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.339+0000 7fb3be3bb640 1 -- 172.21.15.28:0/1411409752 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3b80fbb40 con 0x7fb3b8102350 2024-07-03T15:40:25.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.340+0000 7fb3b4ff9640 1 -- 172.21.15.28:0/1411409752 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11093 (secure 0 0 0) 0x7fb3a80558a0 con 0x7fb3b8102350 2024-07-03T15:40:25.343 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:40:25.343 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":40,"fsid":"a85e4be6-3951-11ef-bcab-c7b262605968","created":"2024-07-03T15:34:56.026978+0000","modified":"2024-07-03T15:40:20.495051+0000","last_up_change":"2024-07-03T15:40:19.490284+0000","last_in_change":"2024-07-03T15:40:01.150891+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-03T15:39:08.833339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"7ecd90b5-3715-4ba7-a696-588843e3582c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3973972034},{"type":"v1","addr":"172.21.15.28:6803","nonce":3973972034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3973972034},{"type":"v1","addr":"172.21.15.28:6805","nonce":3973972034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":3973972034},{"type":"v1","addr":"172.21.15.28:6809","nonce":3973972034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3973972034},{"type":"v1","addr":"172.21.15.28:6807","nonce":3973972034}]},"public_addr":"172.21.15.28:6803/3973972034","cluster_addr":"172.21.15.28:6805/3973972034","heartbeat_back_addr":"172.21.15.28:6809/3973972034","heartbeat_front_addr":"172.21.15.28:6807/3973972034","state":["exists","up"]},{"osd":1,"uuid":"31c5ae0c-6c57-4615-902b-72bf49fa5b72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1642389568},{"type":"v1","addr":"172.21.15.28:6811","nonce":1642389568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1642389568},{"type":"v1","addr":"172.21.15.28:6813","nonce":1642389568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":1642389568},{"type":"v1","addr":"172.21.15.28:6817","nonce":1642389568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1642389568},{"type":"v1","addr":"172.21.15.28:6815","nonce":1642389568}]},"public_addr":"172.21.15.28:6811/1642389568","cluster_addr":"172.21.15.28:6813/1642389568","heartbeat_back_addr":"172.21.15.28:6817/1642389568","heartbeat_front_addr":"172.21.15.28:6815/1642389568","state":["exists","up"]},{"osd":2,"uuid":"5ec8caa6-fb62-443f-bbd6-fbe52df5034c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":3104474806},{"type":"v1","addr":"172.21.15.28:6819","nonce":3104474806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":3104474806},{"type":"v1","addr":"172.21.15.28:6821","nonce":3104474806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":3104474806},{"type":"v1","addr":"172.21.15.28:6825","nonce":3104474806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":3104474806},{"type":"v1","addr":"172.21.15.28:6823","nonce":3104474806}]},"public_addr":"172.21.15.28:6819/3104474806","cluster_addr":"172.21.15.28:6821/3104474806","heartbeat_back_addr":"172.21.15.28:6825/3104474806","heartbeat_front_addr":"172.21.15.28:6823/3104474806","state":["exists","up"]},{"osd":3,"uuid":"d3117e1c-a230-4fc8-9cfc-37e4dc7e3d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2002097126},{"type":"v1","addr":"172.21.15.148:6801","nonce":2002097126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2002097126},{"type":"v1","addr":"172.21.15.148:6803","nonce":2002097126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2002097126},{"type":"v1","addr":"172.21.15.148:6807","nonce":2002097126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2002097126},{"type":"v1","addr":"172.21.15.148:6805","nonce":2002097126}]},"public_addr":"172.21.15.148:6801/2002097126","cluster_addr":"172.21.15.148:6803/2002097126","heartbeat_back_addr":"172.21.15.148:6807/2002097126","heartbeat_front_addr":"172.21.15.148:6805/2002097126","state":["exists","up"]},{"osd":4,"uuid":"d338fd80-0a76-441e-9cba-9cfd27cccc46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1943011533},{"type":"v1","addr":"172.21.15.148:6809","nonce":1943011533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1943011533},{"type":"v1","addr":"172.21.15.148:6811","nonce":1943011533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1943011533},{"type":"v1","addr":"172.21.15.148:6815","nonce":1943011533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1943011533},{"type":"v1","addr":"172.21.15.148:6813","nonce":1943011533}]},"public_addr":"172.21.15.148:6809/1943011533","cluster_addr":"172.21.15.148:6811/1943011533","heartbeat_back_addr":"172.21.15.148:6815/1943011533","heartbeat_front_addr":"172.21.15.148:6813/1943011533","state":["exists","up"]},{"osd":5,"uuid":"5c87b168-85e8-4d65-9763-8f1626e474ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":1578977000},{"type":"v1","addr":"172.21.15.148:6817","nonce":1578977000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1578977000},{"type":"v1","addr":"172.21.15.148:6819","nonce":1578977000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1578977000},{"type":"v1","addr":"172.21.15.148:6823","nonce":1578977000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1578977000},{"type":"v1","addr":"172.21.15.148:6821","nonce":1578977000}]},"public_addr":"172.21.15.148:6817/1578977000","cluster_addr":"172.21.15.148:6819/1578977000","heartbeat_back_addr":"172.21.15.148:6823/1578977000","heartbeat_front_addr":"172.21.15.148:6821/1578977000","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-07-03T15:38:11.596686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-07-03T15:38:38.765936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-07-03T15:39:05.464618+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-07-03T15:39:28.180164+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-07-03T15:39:52.836379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-07-03T15:40:17.711279+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.28:0/3219967940":"2024-07-04T15:37:00.336312+0000","172.21.15.28:0/1009019065":"2024-07-04T15:35:22.603924+0000","172.21.15.28:6801/1358048352":"2024-07-04T15:35:22.603924+0000","172.21.15.28:0/4294623705":"2024-07-04T15:37:00.336312+0000","172.21.15.28:0/390942453":"2024-07-04T15:35:52.741581+0000","172.21.15.28:0/423260870":"2024-07-04T15:35:22.603924+0000","172.21.15.28:6800/1358048352":"2024-07-04T15:35:22.603924+0000","172.21.15.28:6800/577887491":"2024-07-04T15:37:00.336312+0000","172.21.15.28:6800/1434683766":"2024-07-04T15:35:52.741581+0000","172.21.15.28:6801/1434683766":"2024-07-04T15:35:52.741581+0000","172.21.15.28:0/2750216161":"2024-07-04T15:35:52.741581+0000","172.21.15.28:0/3813763982":"2024-07-04T15:35:52.741581+0000","172.21.15.28:0/2748059179":"2024-07-04T15:35:22.603924+0000","172.21.15.28:6801/577887491":"2024-07-04T15:37:00.336312+0000","172.21.15.28:0/2964865891":"2024-07-04T15:37:00.336312+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-03T15:40:25.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.342+0000 7fb3be3bb640 1 -- 172.21.15.28:0/1411409752 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fb3a0067190 msgr2=0x7fb3a0069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:25.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.342+0000 7fb3be3bb640 1 --2- 172.21.15.28:0/1411409752 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fb3a0067190 0x7fb3a0069640 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb3b810e970 tx=0x7fb3ac035040 comp rx=0 tx=0).stop 2024-07-03T15:40:25.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.342+0000 7fb3be3bb640 1 -- 172.21.15.28:0/1411409752 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3b8102350 msgr2=0x7fb3b81130b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:25.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.342+0000 7fb3be3bb640 1 --2- 172.21.15.28:0/1411409752 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3b8102350 0x7fb3b81130b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb3a800c810 tx=0x7fb3a80043c0 comp rx=0 tx=0).stop 2024-07-03T15:40:25.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.342+0000 7fb3be3bb640 1 -- 172.21.15.28:0/1411409752 shutdown_connections 2024-07-03T15:40:25.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.342+0000 7fb3be3bb640 1 --2- 172.21.15.28:0/1411409752 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fb3a0067190 0x7fb3a0069640 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:25.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.342+0000 7fb3be3bb640 1 --2- 172.21.15.28:0/1411409752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3b8102cf0 0x7fb3b81135f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:25.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.342+0000 7fb3be3bb640 1 --2- 172.21.15.28:0/1411409752 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3b8102350 0x7fb3b81130b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:25.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.342+0000 7fb3be3bb640 1 -- 172.21.15.28:0/1411409752 >> 172.21.15.28:0/1411409752 conn(0x7fb3b80f7f20 msgr2=0x7fb3b80f8f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:40:25.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.342+0000 7fb3be3bb640 1 -- 172.21.15.28:0/1411409752 shutdown_connections 2024-07-03T15:40:25.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:25.342+0000 7fb3be3bb640 1 -- 172.21.15.28:0/1411409752 wait complete. 2024-07-03T15:40:25.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:25 smithi028 ceph-mon[29148]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:40:25.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:25 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/1411409752' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-03T15:40:25.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:25 smithi148 ceph-mon[36056]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:40:25.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:25 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/1411409752' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-03T15:40:26.033 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-03T15:39:08.833339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-07-03T15:40:26.034 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-07-03T15:40:26.300 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:40:27.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:27 smithi028 ceph-mon[29148]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:40:27.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:27 smithi148 ceph-mon[36056]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:40:28.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.197+0000 7fb2a701f640 1 -- 172.21.15.28:0/3156812622 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2a01023c0 msgr2=0x7fb2a0102790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:28.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.197+0000 7fb2a701f640 1 --2- 172.21.15.28:0/3156812622 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2a01023c0 0x7fb2a0102790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb2900048d0 tx=0x7fb290009da0 comp rx=0 tx=0).stop 2024-07-03T15:40:28.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.197+0000 7fb2a701f640 1 -- 172.21.15.28:0/3156812622 shutdown_connections 2024-07-03T15:40:28.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.197+0000 7fb2a701f640 1 --2- 172.21.15.28:0/3156812622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2a0102cd0 0x7fb2a00faa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:28.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.197+0000 7fb2a701f640 1 --2- 172.21.15.28:0/3156812622 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2a01023c0 0x7fb2a0102790 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:28.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.197+0000 7fb2a701f640 1 -- 172.21.15.28:0/3156812622 >> 172.21.15.28:0/3156812622 conn(0x7fb2a00709b0 msgr2=0x7fb2a0070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:40:28.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.197+0000 7fb2a701f640 1 -- 172.21.15.28:0/3156812622 shutdown_connections 2024-07-03T15:40:28.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.197+0000 7fb2a701f640 1 -- 172.21.15.28:0/3156812622 wait complete. 2024-07-03T15:40:28.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.198+0000 7fb2a701f640 1 Processor -- start 2024-07-03T15:40:28.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.198+0000 7fb2a701f640 1 -- start start 2024-07-03T15:40:28.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.198+0000 7fb2a701f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2a01023c0 0x7fb2a0113100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:28.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.199+0000 7fb2a701f640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2a0102cd0 0x7fb2a0113640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:28.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.199+0000 7fb2a701f640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a0113cd0 con 0x7fb2a0102cd0 2024-07-03T15:40:28.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.199+0000 7fb2a701f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a010d0d0 con 0x7fb2a01023c0 2024-07-03T15:40:28.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.199+0000 7fb29ffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2a0102cd0 0x7fb2a0113640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:28.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.199+0000 7fb29ffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2a0102cd0 0x7fb2a0113640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48210/0 (socket says 172.21.15.28:48210) 2024-07-03T15:40:28.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.199+0000 7fb29ffff640 1 -- 172.21.15.28:0/3816471717 learned_addr learned my addr 172.21.15.28:0/3816471717 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:40:28.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.199+0000 7fb2a4dbb640 1 --2- 172.21.15.28:0/3816471717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2a01023c0 0x7fb2a0113100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:28.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.199+0000 7fb29ffff640 1 -- 172.21.15.28:0/3816471717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2a01023c0 msgr2=0x7fb2a0113100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:28.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.199+0000 7fb29ffff640 1 --2- 172.21.15.28:0/3816471717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2a01023c0 0x7fb2a0113100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:28.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.199+0000 7fb29ffff640 1 -- 172.21.15.28:0/3816471717 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb290005040 con 0x7fb2a0102cd0 2024-07-03T15:40:28.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.199+0000 7fb29ffff640 1 --2- 172.21.15.28:0/3816471717 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2a0102cd0 0x7fb2a0113640 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb29400b580 tx=0x7fb2940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:40:28.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.199+0000 7fb29d7fa640 1 -- 172.21.15.28:0/3816471717 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb29400f6f0 con 0x7fb2a0102cd0 2024-07-03T15:40:28.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.199+0000 7fb29d7fa640 1 -- 172.21.15.28:0/3816471717 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb29400fd20 con 0x7fb2a0102cd0 2024-07-03T15:40:28.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.199+0000 7fb2a701f640 1 -- 172.21.15.28:0/3816471717 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2a010d3b0 con 0x7fb2a0102cd0 2024-07-03T15:40:28.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.199+0000 7fb29d7fa640 1 -- 172.21.15.28:0/3816471717 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2940103a0 con 0x7fb2a0102cd0 2024-07-03T15:40:28.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.199+0000 7fb2a701f640 1 -- 172.21.15.28:0/3816471717 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2a010d8d0 con 0x7fb2a0102cd0 2024-07-03T15:40:28.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.200+0000 7fb2a701f640 1 -- 172.21.15.28:0/3816471717 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2a0065490 con 0x7fb2a0102cd0 2024-07-03T15:40:28.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.201+0000 7fb29d7fa640 1 -- 172.21.15.28:0/3816471717 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7fb294011400 con 0x7fb2a0102cd0 2024-07-03T15:40:28.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.201+0000 7fb29d7fa640 1 --2- 172.21.15.28:0/3816471717 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fb288067190 0x7fb288069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:28.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.201+0000 7fb29d7fa640 1 -- 172.21.15.28:0/3816471717 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fb294084690 con 0x7fb2a0102cd0 2024-07-03T15:40:28.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.201+0000 7fb2a4dbb640 1 --2- 172.21.15.28:0/3816471717 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fb288067190 0x7fb288069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:28.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.202+0000 7fb2a4dbb640 1 --2- 172.21.15.28:0/3816471717 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fb288067190 0x7fb288069640 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb290009fa0 tx=0x7fb290035040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:40:28.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.205+0000 7fb29d7fa640 1 -- 172.21.15.28:0/3816471717 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb294051cf0 con 0x7fb2a0102cd0 2024-07-03T15:40:28.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.381+0000 7fb2a701f640 1 -- 172.21.15.28:0/3816471717 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb2a0109590 con 0x7fb2a0102cd0 2024-07-03T15:40:28.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.382+0000 7fb29d7fa640 1 -- 172.21.15.28:0/3816471717 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb294055310 con 0x7fb2a0102cd0 2024-07-03T15:40:28.385 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2024-07-03T15:40:28.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.384+0000 7fb2a701f640 1 -- 172.21.15.28:0/3816471717 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fb288067190 msgr2=0x7fb288069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:28.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.384+0000 7fb2a701f640 1 --2- 172.21.15.28:0/3816471717 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fb288067190 0x7fb288069640 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb290009fa0 tx=0x7fb290035040 comp rx=0 tx=0).stop 2024-07-03T15:40:28.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.384+0000 7fb2a701f640 1 -- 172.21.15.28:0/3816471717 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2a0102cd0 msgr2=0x7fb2a0113640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:28.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.384+0000 7fb2a701f640 1 --2- 172.21.15.28:0/3816471717 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2a0102cd0 0x7fb2a0113640 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb29400b580 tx=0x7fb2940091a0 comp rx=0 tx=0).stop 2024-07-03T15:40:28.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.384+0000 7fb2a701f640 1 -- 172.21.15.28:0/3816471717 shutdown_connections 2024-07-03T15:40:28.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.384+0000 7fb2a701f640 1 --2- 172.21.15.28:0/3816471717 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fb288067190 0x7fb288069640 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:28.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.384+0000 7fb2a701f640 1 --2- 172.21.15.28:0/3816471717 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2a0102cd0 0x7fb2a0113640 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:28.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.384+0000 7fb2a701f640 1 --2- 172.21.15.28:0/3816471717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2a01023c0 0x7fb2a0113100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:28.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.384+0000 7fb2a701f640 1 -- 172.21.15.28:0/3816471717 >> 172.21.15.28:0/3816471717 conn(0x7fb2a00709b0 msgr2=0x7fb2a01050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:40:28.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.384+0000 7fb2a701f640 1 -- 172.21.15.28:0/3816471717 shutdown_connections 2024-07-03T15:40:28.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:28.384+0000 7fb2a701f640 1 -- 172.21.15.28:0/3816471717 wait complete. 2024-07-03T15:40:28.743 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:28 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/3816471717' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-03T15:40:28.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:28 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/3816471717' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-03T15:40:29.098 INFO:tasks.cephadm:Setting up client nodes... 2024-07-03T15:40:29.098 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-07-03T15:40:29.363 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:40:29.830 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:29 smithi028 ceph-mon[29148]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:40:29.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:29 smithi148 ceph-mon[36056]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:40:30.741 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:30 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:40:30.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:30.965+0000 7f6206ab3640 1 -- 172.21.15.28:0/1761511915 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62000fc360 msgr2=0x7f62000fc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:30.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:30.965+0000 7f6206ab3640 1 --2- 172.21.15.28:0/1761511915 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62000fc360 0x7f62000fc770 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f61f40048d0 tx=0x7f61f4009da0 comp rx=0 tx=0).stop 2024-07-03T15:40:30.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:30.965+0000 7f6206ab3640 1 -- 172.21.15.28:0/1761511915 shutdown_connections 2024-07-03T15:40:30.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:30.965+0000 7f6206ab3640 1 --2- 172.21.15.28:0/1761511915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62000fd630 0x7f62000fdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:30.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:30.965+0000 7f6206ab3640 1 --2- 172.21.15.28:0/1761511915 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62000fc360 0x7f62000fc770 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:30.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:30.965+0000 7f6206ab3640 1 -- 172.21.15.28:0/1761511915 >> 172.21.15.28:0/1761511915 conn(0x7f62000f7b20 msgr2=0x7f62000f9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:40:30.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:30.965+0000 7f6206ab3640 1 -- 172.21.15.28:0/1761511915 shutdown_connections 2024-07-03T15:40:30.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:30.965+0000 7f6206ab3640 1 -- 172.21.15.28:0/1761511915 wait complete. 2024-07-03T15:40:30.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:30.966+0000 7f6206ab3640 1 Processor -- start 2024-07-03T15:40:30.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:30.966+0000 7f6206ab3640 1 -- start start 2024-07-03T15:40:30.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:30.966+0000 7f6206ab3640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62000fc360 0x7f6200071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:30.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:30.966+0000 7f6206ab3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62000fd630 0x7f620006f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:30.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:30.966+0000 7f6206ab3640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620006fe90 con 0x7f62000fc360 2024-07-03T15:40:30.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:30.966+0000 7f6206ab3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620006ffd0 con 0x7f62000fd630 2024-07-03T15:40:30.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:30.966+0000 7f620484f640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62000fc360 0x7f6200071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:30.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:30.966+0000 7f620484f640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62000fc360 0x7f6200071380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:40624/0 (socket says 172.21.15.28:40624) 2024-07-03T15:40:30.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:30.966+0000 7f620484f640 1 -- 172.21.15.28:0/1181831236 learned_addr learned my addr 172.21.15.28:0/1181831236 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:40:30.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:30.966+0000 7f61fffff640 1 --2- 172.21.15.28:0/1181831236 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62000fd630 0x7f620006f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:30.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:30.966+0000 7f620484f640 1 -- 172.21.15.28:0/1181831236 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62000fd630 msgr2=0x7f620006f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:30.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:30.967+0000 7f620484f640 1 --2- 172.21.15.28:0/1181831236 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62000fd630 0x7f620006f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:30.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:30.967+0000 7f620484f640 1 -- 172.21.15.28:0/1181831236 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f4005040 con 0x7f62000fc360 2024-07-03T15:40:30.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:30.967+0000 7f620484f640 1 --2- 172.21.15.28:0/1181831236 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62000fc360 0x7f6200071380 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f61f4003080 tx=0x7f61f4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:40:30.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:30.967+0000 7f61fd7fa640 1 -- 172.21.15.28:0/1181831236 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61f4038070 con 0x7f62000fc360 2024-07-03T15:40:30.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:30.967+0000 7f61fd7fa640 1 -- 172.21.15.28:0/1181831236 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f61f4032970 con 0x7f62000fc360 2024-07-03T15:40:30.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:30.967+0000 7f6206ab3640 1 -- 172.21.15.28:0/1181831236 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6200070250 con 0x7f62000fc360 2024-07-03T15:40:30.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:30.967+0000 7f61fd7fa640 1 -- 172.21.15.28:0/1181831236 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61f4033900 con 0x7f62000fc360 2024-07-03T15:40:30.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:30.967+0000 7f6206ab3640 1 -- 172.21.15.28:0/1181831236 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6200115030 con 0x7f62000fc360 2024-07-03T15:40:30.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:30.968+0000 7f6206ab3640 1 -- 172.21.15.28:0/1181831236 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62000fb7c0 con 0x7f62000fc360 2024-07-03T15:40:30.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:30.968+0000 7f61fd7fa640 1 -- 172.21.15.28:0/1181831236 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f61f4030020 con 0x7f62000fc360 2024-07-03T15:40:30.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:30.969+0000 7f61fd7fa640 1 --2- 172.21.15.28:0/1181831236 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f61f0067190 0x7f61f0069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:30.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:30.969+0000 7f61fffff640 1 --2- 172.21.15.28:0/1181831236 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f61f0067190 0x7f61f0069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:30.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:30.969+0000 7f61fd7fa640 1 -- 172.21.15.28:0/1181831236 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f61f40a4800 con 0x7f62000fc360 2024-07-03T15:40:30.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:30.969+0000 7f61fffff640 1 --2- 172.21.15.28:0/1181831236 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f61f0067190 0x7f61f0069640 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f61ec00a900 tx=0x7f61ec008040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:40:30.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:30.973+0000 7f61fd7fa640 1 -- 172.21.15.28:0/1181831236 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f61f4071e60 con 0x7f62000fc360 2024-07-03T15:40:30.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:30 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:40:31.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:31.214+0000 7f6206ab3640 1 -- 172.21.15.28:0/1181831236 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f620004b530 con 0x7f62000fc360 2024-07-03T15:40:31.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:31.218+0000 7f61fd7fa640 1 -- 172.21.15.28:0/1181831236 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v14) v1 ==== 170+0+59 (secure 0 0 0) 0x7f61f4041050 con 0x7f62000fc360 2024-07-03T15:40:31.222 INFO:teuthology.orchestra.run.smithi028.stdout:[client.0] 2024-07-03T15:40:31.222 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQDvcIVmZGboDBAAGwjWoTSEcOSdnFniMgXoMQ== 2024-07-03T15:40:31.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:31.220+0000 7f6206ab3640 1 -- 172.21.15.28:0/1181831236 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f61f0067190 msgr2=0x7f61f0069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:31.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:31.220+0000 7f6206ab3640 1 --2- 172.21.15.28:0/1181831236 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f61f0067190 0x7f61f0069640 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f61ec00a900 tx=0x7f61ec008040 comp rx=0 tx=0).stop 2024-07-03T15:40:31.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:31.220+0000 7f6206ab3640 1 -- 172.21.15.28:0/1181831236 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62000fc360 msgr2=0x7f6200071380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:31.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:31.220+0000 7f6206ab3640 1 --2- 172.21.15.28:0/1181831236 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62000fc360 0x7f6200071380 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f61f4003080 tx=0x7f61f4009da0 comp rx=0 tx=0).stop 2024-07-03T15:40:31.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:31.220+0000 7f6206ab3640 1 -- 172.21.15.28:0/1181831236 shutdown_connections 2024-07-03T15:40:31.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:31.220+0000 7f6206ab3640 1 --2- 172.21.15.28:0/1181831236 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f61f0067190 0x7f61f0069640 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:31.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:31.220+0000 7f6206ab3640 1 --2- 172.21.15.28:0/1181831236 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62000fd630 0x7f620006f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:31.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:31.220+0000 7f6206ab3640 1 --2- 172.21.15.28:0/1181831236 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62000fc360 0x7f6200071380 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:31.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:31.220+0000 7f6206ab3640 1 -- 172.21.15.28:0/1181831236 >> 172.21.15.28:0/1181831236 conn(0x7f62000f7b20 msgr2=0x7f62000f91f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:40:31.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:31.220+0000 7f6206ab3640 1 -- 172.21.15.28:0/1181831236 shutdown_connections 2024-07-03T15:40:31.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:31.220+0000 7f6206ab3640 1 -- 172.21.15.28:0/1181831236 wait complete. 2024-07-03T15:40:31.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:31 smithi028 ceph-mon[29148]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:40:31.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:31 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/1181831236' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-07-03T15:40:31.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:31 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/1181831236' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-07-03T15:40:31.935 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-07-03T15:40:31.936 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-07-03T15:40:31.936 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-07-03T15:40:31.977 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-07-03T15:40:31.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:31 smithi148 ceph-mon[36056]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:40:31.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:31 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/1181831236' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-07-03T15:40:31.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:31 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/1181831236' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-07-03T15:40:32.238 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi148/config 2024-07-03T15:40:33.610 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:33 smithi148 ceph-mon[36056]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:40:33.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:33.793+0000 7fb76f7cc640 1 -- 172.21.15.148:0/3077320924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7680fc360 msgr2=0x7fb7680fc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:33.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:33.793+0000 7fb76f7cc640 1 --2- 172.21.15.148:0/3077320924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7680fc360 0x7fb7680fc770 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb7640048d0 tx=0x7fb764009da0 comp rx=0 tx=0).stop 2024-07-03T15:40:33.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:33.793+0000 7fb76f7cc640 1 -- 172.21.15.148:0/3077320924 shutdown_connections 2024-07-03T15:40:33.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:33.793+0000 7fb76f7cc640 1 --2- 172.21.15.148:0/3077320924 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7680fd630 0x7fb7680fdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:33.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:33.793+0000 7fb76f7cc640 1 --2- 172.21.15.148:0/3077320924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7680fc360 0x7fb7680fc770 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:33.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:33.793+0000 7fb76f7cc640 1 -- 172.21.15.148:0/3077320924 >> 172.21.15.148:0/3077320924 conn(0x7fb7680f7b20 msgr2=0x7fb7680f9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:40:33.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:33.793+0000 7fb76f7cc640 1 -- 172.21.15.148:0/3077320924 shutdown_connections 2024-07-03T15:40:33.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:33.793+0000 7fb76f7cc640 1 -- 172.21.15.148:0/3077320924 wait complete. 2024-07-03T15:40:33.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:33.793+0000 7fb76f7cc640 1 Processor -- start 2024-07-03T15:40:33.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:33.794+0000 7fb76f7cc640 1 -- start start 2024-07-03T15:40:33.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:33.794+0000 7fb76f7cc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7680fc360 0x7fb76810cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:33.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:33.794+0000 7fb76f7cc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7680fd630 0x7fb76810d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:33.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:33.794+0000 7fb76f7cc640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb76810e370 con 0x7fb7680fc360 2024-07-03T15:40:33.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:33.794+0000 7fb76f7cc640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb76810e4e0 con 0x7fb7680fd630 2024-07-03T15:40:33.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:33.794+0000 7fb76cd67640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7680fd630 0x7fb76810d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:33.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:33.794+0000 7fb76cd67640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7680fd630 0x7fb76810d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52028/0 (socket says 172.21.15.148:52028) 2024-07-03T15:40:33.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:33.794+0000 7fb76cd67640 1 -- 172.21.15.148:0/1643526611 learned_addr learned my addr 172.21.15.148:0/1643526611 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-07-03T15:40:33.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:33.794+0000 7fb76d568640 1 --2- 172.21.15.148:0/1643526611 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7680fc360 0x7fb76810cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:33.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:33.794+0000 7fb76cd67640 1 -- 172.21.15.148:0/1643526611 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7680fc360 msgr2=0x7fb76810cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:33.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:33.794+0000 7fb76cd67640 1 --2- 172.21.15.148:0/1643526611 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7680fc360 0x7fb76810cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:33.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:33.794+0000 7fb76cd67640 1 -- 172.21.15.148:0/1643526611 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb764005040 con 0x7fb7680fd630 2024-07-03T15:40:33.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:33.794+0000 7fb76d568640 1 --2- 172.21.15.148:0/1643526611 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7680fc360 0x7fb76810cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-03T15:40:33.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:33.794+0000 7fb76cd67640 1 --2- 172.21.15.148:0/1643526611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7680fd630 0x7fb76810d500 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb76000bbb0 tx=0x7fb760009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:40:33.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:33.794+0000 7fb75dffb640 1 -- 172.21.15.148:0/1643526611 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb76000c960 con 0x7fb7680fd630 2024-07-03T15:40:33.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:33.794+0000 7fb75dffb640 1 -- 172.21.15.148:0/1643526611 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb7600092e0 con 0x7fb7680fd630 2024-07-03T15:40:33.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:33.794+0000 7fb75dffb640 1 -- 172.21.15.148:0/1643526611 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb760007640 con 0x7fb7680fd630 2024-07-03T15:40:33.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:33.794+0000 7fb76f7cc640 1 -- 172.21.15.148:0/1643526611 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb76810e7c0 con 0x7fb7680fd630 2024-07-03T15:40:33.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:33.795+0000 7fb76f7cc640 1 -- 172.21.15.148:0/1643526611 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb76810ece0 con 0x7fb7680fd630 2024-07-03T15:40:33.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:33.796+0000 7fb76f7cc640 1 -- 172.21.15.148:0/1643526611 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7680fb7c0 con 0x7fb7680fd630 2024-07-03T15:40:33.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:33.796+0000 7fb75dffb640 1 -- 172.21.15.148:0/1643526611 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7fb760012070 con 0x7fb7680fd630 2024-07-03T15:40:33.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:33.796+0000 7fb75dffb640 1 --2- 172.21.15.148:0/1643526611 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fb754067190 0x7fb754069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:33.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:33.796+0000 7fb75dffb640 1 -- 172.21.15.148:0/1643526611 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fb760084910 con 0x7fb7680fd630 2024-07-03T15:40:33.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:33.797+0000 7fb76d568640 1 --2- 172.21.15.148:0/1643526611 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fb754067190 0x7fb754069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:33.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:33.797+0000 7fb76d568640 1 --2- 172.21.15.148:0/1643526611 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fb754067190 0x7fb754069640 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb764000c00 tx=0x7fb764009da0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:40:33.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:33.801+0000 7fb75dffb640 1 -- 172.21.15.148:0/1643526611 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb760051ef0 con 0x7fb7680fd630 2024-07-03T15:40:33.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:33 smithi028 ceph-mon[29148]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:40:34.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:34.028+0000 7fb76f7cc640 1 -- 172.21.15.148:0/1643526611 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fb76804b530 con 0x7fb7680fd630 2024-07-03T15:40:34.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:34.032+0000 7fb75dffb640 1 -- 172.21.15.148:0/1643526611 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v15) v1 ==== 170+0+59 (secure 0 0 0) 0x7fb760055510 con 0x7fb7680fd630 2024-07-03T15:40:34.035 INFO:teuthology.orchestra.run.smithi148.stdout:[client.1] 2024-07-03T15:40:34.035 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQDycIVmLtjGARAADhtrFQID3adYPsPyWh98Vg== 2024-07-03T15:40:34.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:34.034+0000 7fb76f7cc640 1 -- 172.21.15.148:0/1643526611 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fb754067190 msgr2=0x7fb754069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:34.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:34.034+0000 7fb76f7cc640 1 --2- 172.21.15.148:0/1643526611 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fb754067190 0x7fb754069640 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb764000c00 tx=0x7fb764009da0 comp rx=0 tx=0).stop 2024-07-03T15:40:34.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:34.034+0000 7fb76f7cc640 1 -- 172.21.15.148:0/1643526611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7680fd630 msgr2=0x7fb76810d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:34.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:34.034+0000 7fb76f7cc640 1 --2- 172.21.15.148:0/1643526611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7680fd630 0x7fb76810d500 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb76000bbb0 tx=0x7fb760009840 comp rx=0 tx=0).stop 2024-07-03T15:40:34.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:34.034+0000 7fb76f7cc640 1 -- 172.21.15.148:0/1643526611 shutdown_connections 2024-07-03T15:40:34.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:34.034+0000 7fb76f7cc640 1 --2- 172.21.15.148:0/1643526611 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fb754067190 0x7fb754069640 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:34.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:34.034+0000 7fb76f7cc640 1 --2- 172.21.15.148:0/1643526611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7680fd630 0x7fb76810d500 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:34.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:34.034+0000 7fb76f7cc640 1 --2- 172.21.15.148:0/1643526611 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7680fc360 0x7fb76810cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:34.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:34.034+0000 7fb76f7cc640 1 -- 172.21.15.148:0/1643526611 >> 172.21.15.148:0/1643526611 conn(0x7fb7680f7b20 msgr2=0x7fb7680f9160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:40:34.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:34.035+0000 7fb76f7cc640 1 -- 172.21.15.148:0/1643526611 shutdown_connections 2024-07-03T15:40:34.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:40:34.035+0000 7fb76f7cc640 1 -- 172.21.15.148:0/1643526611 wait complete. 2024-07-03T15:40:34.621 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-07-03T15:40:34.621 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-07-03T15:40:34.621 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-07-03T15:40:34.698 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-07-03T15:40:34.698 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-07-03T15:40:34.698 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph mgr dump --format=json 2024-07-03T15:40:34.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:34 smithi028 ceph-mon[29148]: from='client.? 172.21.15.148:0/1643526611' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-07-03T15:40:34.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:34 smithi028 ceph-mon[29148]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-07-03T15:40:34.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:34 smithi028 ceph-mon[29148]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-07-03T15:40:34.972 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:40:34.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:34 smithi148 ceph-mon[36056]: from='client.? 172.21.15.148:0/1643526611' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-07-03T15:40:34.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:34 smithi148 ceph-mon[36056]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-07-03T15:40:34.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:34 smithi148 ceph-mon[36056]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-07-03T15:40:35.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:35 smithi028 ceph-mon[29148]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:40:35.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:35 smithi148 ceph-mon[36056]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:40:36.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.606+0000 7fd97e7a6640 1 -- 172.21.15.28:0/3380961313 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd978102350 msgr2=0x7fd978102720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:36.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.606+0000 7fd97e7a6640 1 --2- 172.21.15.28:0/3380961313 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd978102350 0x7fd978102720 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd9680048d0 tx=0x7fd968009da0 comp rx=0 tx=0).stop 2024-07-03T15:40:36.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.606+0000 7fd97e7a6640 1 -- 172.21.15.28:0/3380961313 shutdown_connections 2024-07-03T15:40:36.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.606+0000 7fd97e7a6640 1 --2- 172.21.15.28:0/3380961313 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd978102cf0 0x7fd9780fc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:36.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.606+0000 7fd97e7a6640 1 --2- 172.21.15.28:0/3380961313 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd978102350 0x7fd978102720 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:36.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.606+0000 7fd97e7a6640 1 -- 172.21.15.28:0/3380961313 >> 172.21.15.28:0/3380961313 conn(0x7fd9780f7f00 msgr2=0x7fd9780fa310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:40:36.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.606+0000 7fd97e7a6640 1 -- 172.21.15.28:0/3380961313 shutdown_connections 2024-07-03T15:40:36.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.606+0000 7fd97e7a6640 1 -- 172.21.15.28:0/3380961313 wait complete. 2024-07-03T15:40:36.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.606+0000 7fd97e7a6640 1 Processor -- start 2024-07-03T15:40:36.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.606+0000 7fd97e7a6640 1 -- start start 2024-07-03T15:40:36.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.607+0000 7fd97e7a6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd978102350 0x7fd97806f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:36.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.607+0000 7fd97e7a6640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd978102cf0 0x7fd97806fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:36.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.607+0000 7fd97e7a6640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd978070330 con 0x7fd978102cf0 2024-07-03T15:40:36.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.607+0000 7fd97e7a6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9780704a0 con 0x7fd978102350 2024-07-03T15:40:36.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.607+0000 7fd9777fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd978102cf0 0x7fd97806fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:36.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.607+0000 7fd9777fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd978102cf0 0x7fd97806fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:40636/0 (socket says 172.21.15.28:40636) 2024-07-03T15:40:36.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.607+0000 7fd9777fe640 1 -- 172.21.15.28:0/3291924533 learned_addr learned my addr 172.21.15.28:0/3291924533 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:40:36.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.607+0000 7fd977fff640 1 --2- 172.21.15.28:0/3291924533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd978102350 0x7fd97806f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:36.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.607+0000 7fd9777fe640 1 -- 172.21.15.28:0/3291924533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd978102350 msgr2=0x7fd97806f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:36.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.607+0000 7fd9777fe640 1 --2- 172.21.15.28:0/3291924533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd978102350 0x7fd97806f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:36.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.607+0000 7fd9777fe640 1 -- 172.21.15.28:0/3291924533 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd968005040 con 0x7fd978102cf0 2024-07-03T15:40:36.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.607+0000 7fd9777fe640 1 --2- 172.21.15.28:0/3291924533 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd978102cf0 0x7fd97806fd60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd96c00b790 tx=0x7fd96c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:40:36.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.607+0000 7fd974ff9640 1 -- 172.21.15.28:0/3291924533 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd96c00c700 con 0x7fd978102cf0 2024-07-03T15:40:36.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.608+0000 7fd974ff9640 1 -- 172.21.15.28:0/3291924533 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd96c00cd30 con 0x7fd978102cf0 2024-07-03T15:40:36.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.608+0000 7fd97e7a6640 1 -- 172.21.15.28:0/3291924533 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd978119430 con 0x7fd978102cf0 2024-07-03T15:40:36.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.608+0000 7fd974ff9640 1 -- 172.21.15.28:0/3291924533 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd96c013370 con 0x7fd978102cf0 2024-07-03T15:40:36.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.608+0000 7fd97e7a6640 1 -- 172.21.15.28:0/3291924533 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd978119870 con 0x7fd978102cf0 2024-07-03T15:40:36.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.609+0000 7fd97e7a6640 1 -- 172.21.15.28:0/3291924533 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9780fdac0 con 0x7fd978102cf0 2024-07-03T15:40:36.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.609+0000 7fd974ff9640 1 -- 172.21.15.28:0/3291924533 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7fd96c012070 con 0x7fd978102cf0 2024-07-03T15:40:36.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.609+0000 7fd974ff9640 1 --2- 172.21.15.28:0/3291924533 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fd960067190 0x7fd960069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:36.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.610+0000 7fd974ff9640 1 -- 172.21.15.28:0/3291924533 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fd96c084040 con 0x7fd978102cf0 2024-07-03T15:40:36.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.610+0000 7fd977fff640 1 --2- 172.21.15.28:0/3291924533 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fd960067190 0x7fd960069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:36.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.610+0000 7fd977fff640 1 --2- 172.21.15.28:0/3291924533 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fd960067190 0x7fd960069640 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd968009f80 tx=0x7fd968035040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:40:36.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.614+0000 7fd974ff9640 1 -- 172.21.15.28:0/3291924533 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd96c0516a0 con 0x7fd978102cf0 2024-07-03T15:40:36.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.834+0000 7fd97e7a6640 1 -- 172.21.15.28:0/3291924533 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fd9780fbb40 con 0x7fd978102cf0 2024-07-03T15:40:36.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.837+0000 7fd974ff9640 1 -- 172.21.15.28:0/3291924533 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+164104 (secure 0 0 0) 0x7fd96c054cc0 con 0x7fd978102cf0 2024-07-03T15:40:36.842 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:40:36.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.840+0000 7fd97e7a6640 1 -- 172.21.15.28:0/3291924533 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fd960067190 msgr2=0x7fd960069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:36.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.840+0000 7fd97e7a6640 1 --2- 172.21.15.28:0/3291924533 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fd960067190 0x7fd960069640 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd968009f80 tx=0x7fd968035040 comp rx=0 tx=0).stop 2024-07-03T15:40:36.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.840+0000 7fd97e7a6640 1 -- 172.21.15.28:0/3291924533 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd978102cf0 msgr2=0x7fd97806fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:36.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.840+0000 7fd97e7a6640 1 --2- 172.21.15.28:0/3291924533 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd978102cf0 0x7fd97806fd60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd96c00b790 tx=0x7fd96c0091d0 comp rx=0 tx=0).stop 2024-07-03T15:40:36.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.840+0000 7fd97e7a6640 1 -- 172.21.15.28:0/3291924533 shutdown_connections 2024-07-03T15:40:36.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.840+0000 7fd97e7a6640 1 --2- 172.21.15.28:0/3291924533 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fd960067190 0x7fd960069640 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:36.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.840+0000 7fd97e7a6640 1 --2- 172.21.15.28:0/3291924533 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd978102cf0 0x7fd97806fd60 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:36.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.840+0000 7fd97e7a6640 1 --2- 172.21.15.28:0/3291924533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd978102350 0x7fd97806f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:36.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.840+0000 7fd97e7a6640 1 -- 172.21.15.28:0/3291924533 >> 172.21.15.28:0/3291924533 conn(0x7fd9780f7f00 msgr2=0x7fd9780f8e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:40:36.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.841+0000 7fd97e7a6640 1 -- 172.21.15.28:0/3291924533 shutdown_connections 2024-07-03T15:40:36.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:36.841+0000 7fd97e7a6640 1 -- 172.21.15.28:0/3291924533 wait complete. 2024-07-03T15:40:37.572 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":19,"flags":0,"active_gid":14210,"active_name":"smithi028.vtqrtz","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":421057178},{"type":"v1","addr":"172.21.15.28:6801","nonce":421057178}]},"active_addr":"172.21.15.28:6801/421057178","active_change":"2024-07-03T15:37:00.336422+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":14232,"name":"smithi148.rjzsar","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.28:8443/","prometheus":"http://172.21.15.28:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.28:0","nonce":1083276374}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.28:0","nonce":1938941600}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.28:0","nonce":1365299779}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.28:0","nonce":3188588861}]}]} 2024-07-03T15:40:37.576 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-07-03T15:40:37.576 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-07-03T15:40:37.576 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph osd dump --format=json 2024-07-03T15:40:37.845 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:40:37.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:37 smithi028 ceph-mon[29148]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:40:37.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:37 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/3291924533' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-07-03T15:40:37.894 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:37 smithi148 ceph-mon[36056]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:40:37.894 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:37 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/3291924533' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-07-03T15:40:39.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.409+0000 7f4d29931640 1 -- 172.21.15.28:0/3497009195 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d24102350 msgr2=0x7f4d24102720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:39.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.409+0000 7f4d29931640 1 --2- 172.21.15.28:0/3497009195 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d24102350 0x7f4d24102720 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4d140048d0 tx=0x7f4d14009da0 comp rx=0 tx=0).stop 2024-07-03T15:40:39.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.409+0000 7f4d29931640 1 -- 172.21.15.28:0/3497009195 shutdown_connections 2024-07-03T15:40:39.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.409+0000 7f4d29931640 1 --2- 172.21.15.28:0/3497009195 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d24102cf0 0x7f4d240fc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:39.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.409+0000 7f4d29931640 1 --2- 172.21.15.28:0/3497009195 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d24102350 0x7f4d24102720 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:39.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.409+0000 7f4d29931640 1 -- 172.21.15.28:0/3497009195 >> 172.21.15.28:0/3497009195 conn(0x7f4d240f7f20 msgr2=0x7f4d240fa330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:40:39.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.409+0000 7f4d29931640 1 -- 172.21.15.28:0/3497009195 shutdown_connections 2024-07-03T15:40:39.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.409+0000 7f4d29931640 1 -- 172.21.15.28:0/3497009195 wait complete. 2024-07-03T15:40:39.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.410+0000 7f4d29931640 1 Processor -- start 2024-07-03T15:40:39.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.410+0000 7f4d29931640 1 -- start start 2024-07-03T15:40:39.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.410+0000 7f4d29931640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d24102350 0x7f4d24113140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:39.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.410+0000 7f4d29931640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d24102cf0 0x7f4d24113680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:39.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.410+0000 7f4d29931640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d24113d10 con 0x7f4d24102cf0 2024-07-03T15:40:39.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.410+0000 7f4d29931640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2410d0c0 con 0x7f4d24102350 2024-07-03T15:40:39.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.410+0000 7f4d1a7fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d24102cf0 0x7f4d24113680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:39.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.410+0000 7f4d1a7fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d24102cf0 0x7f4d24113680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:40668/0 (socket says 172.21.15.28:40668) 2024-07-03T15:40:39.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.410+0000 7f4d1a7fc640 1 -- 172.21.15.28:0/3703529296 learned_addr learned my addr 172.21.15.28:0/3703529296 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:40:39.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.410+0000 7f4d22ffd640 1 --2- 172.21.15.28:0/3703529296 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d24102350 0x7f4d24113140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:39.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.411+0000 7f4d1a7fc640 1 -- 172.21.15.28:0/3703529296 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d24102350 msgr2=0x7f4d24113140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:39.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.411+0000 7f4d1a7fc640 1 --2- 172.21.15.28:0/3703529296 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d24102350 0x7f4d24113140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:39.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.411+0000 7f4d1a7fc640 1 -- 172.21.15.28:0/3703529296 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d14005040 con 0x7f4d24102cf0 2024-07-03T15:40:39.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.411+0000 7f4d1a7fc640 1 --2- 172.21.15.28:0/3703529296 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d24102cf0 0x7f4d24113680 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c00b790 tx=0x7f4d0c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:40:39.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.411+0000 7f4d1bfff640 1 -- 172.21.15.28:0/3703529296 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d0c00c700 con 0x7f4d24102cf0 2024-07-03T15:40:39.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.411+0000 7f4d29931640 1 -- 172.21.15.28:0/3703529296 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d2410d3a0 con 0x7f4d24102cf0 2024-07-03T15:40:39.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.411+0000 7f4d1bfff640 1 -- 172.21.15.28:0/3703529296 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4d0c00cd30 con 0x7f4d24102cf0 2024-07-03T15:40:39.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.411+0000 7f4d1bfff640 1 -- 172.21.15.28:0/3703529296 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d0c013370 con 0x7f4d24102cf0 2024-07-03T15:40:39.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.411+0000 7f4d29931640 1 -- 172.21.15.28:0/3703529296 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d2410d8c0 con 0x7f4d24102cf0 2024-07-03T15:40:39.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.412+0000 7f4d29931640 1 -- 172.21.15.28:0/3703529296 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d240fdac0 con 0x7f4d24102cf0 2024-07-03T15:40:39.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.413+0000 7f4d1bfff640 1 -- 172.21.15.28:0/3703529296 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f4d0c012070 con 0x7f4d24102cf0 2024-07-03T15:40:39.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.413+0000 7f4d1bfff640 1 --2- 172.21.15.28:0/3703529296 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f4d10067190 0x7f4d10069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:39.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.413+0000 7f4d1bfff640 1 -- 172.21.15.28:0/3703529296 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f4d0c0840d0 con 0x7f4d24102cf0 2024-07-03T15:40:39.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.413+0000 7f4d22ffd640 1 --2- 172.21.15.28:0/3703529296 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f4d10067190 0x7f4d10069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:39.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.414+0000 7f4d22ffd640 1 --2- 172.21.15.28:0/3703529296 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f4d10067190 0x7f4d10069640 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4d14032430 tx=0x7f4d14035040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:40:39.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.417+0000 7f4d1bfff640 1 -- 172.21.15.28:0/3703529296 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4d0c0516b0 con 0x7f4d24102cf0 2024-07-03T15:40:39.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.604+0000 7f4d29931640 1 -- 172.21.15.28:0/3703529296 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4d240fbb40 con 0x7f4d24102cf0 2024-07-03T15:40:39.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.604+0000 7f4d1bfff640 1 -- 172.21.15.28:0/3703529296 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11093 (secure 0 0 0) 0x7f4d0c054cd0 con 0x7f4d24102cf0 2024-07-03T15:40:39.608 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:40:39.608 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":40,"fsid":"a85e4be6-3951-11ef-bcab-c7b262605968","created":"2024-07-03T15:34:56.026978+0000","modified":"2024-07-03T15:40:20.495051+0000","last_up_change":"2024-07-03T15:40:19.490284+0000","last_in_change":"2024-07-03T15:40:01.150891+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-03T15:39:08.833339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"7ecd90b5-3715-4ba7-a696-588843e3582c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3973972034},{"type":"v1","addr":"172.21.15.28:6803","nonce":3973972034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3973972034},{"type":"v1","addr":"172.21.15.28:6805","nonce":3973972034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":3973972034},{"type":"v1","addr":"172.21.15.28:6809","nonce":3973972034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3973972034},{"type":"v1","addr":"172.21.15.28:6807","nonce":3973972034}]},"public_addr":"172.21.15.28:6803/3973972034","cluster_addr":"172.21.15.28:6805/3973972034","heartbeat_back_addr":"172.21.15.28:6809/3973972034","heartbeat_front_addr":"172.21.15.28:6807/3973972034","state":["exists","up"]},{"osd":1,"uuid":"31c5ae0c-6c57-4615-902b-72bf49fa5b72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1642389568},{"type":"v1","addr":"172.21.15.28:6811","nonce":1642389568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1642389568},{"type":"v1","addr":"172.21.15.28:6813","nonce":1642389568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":1642389568},{"type":"v1","addr":"172.21.15.28:6817","nonce":1642389568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1642389568},{"type":"v1","addr":"172.21.15.28:6815","nonce":1642389568}]},"public_addr":"172.21.15.28:6811/1642389568","cluster_addr":"172.21.15.28:6813/1642389568","heartbeat_back_addr":"172.21.15.28:6817/1642389568","heartbeat_front_addr":"172.21.15.28:6815/1642389568","state":["exists","up"]},{"osd":2,"uuid":"5ec8caa6-fb62-443f-bbd6-fbe52df5034c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":3104474806},{"type":"v1","addr":"172.21.15.28:6819","nonce":3104474806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":3104474806},{"type":"v1","addr":"172.21.15.28:6821","nonce":3104474806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":3104474806},{"type":"v1","addr":"172.21.15.28:6825","nonce":3104474806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":3104474806},{"type":"v1","addr":"172.21.15.28:6823","nonce":3104474806}]},"public_addr":"172.21.15.28:6819/3104474806","cluster_addr":"172.21.15.28:6821/3104474806","heartbeat_back_addr":"172.21.15.28:6825/3104474806","heartbeat_front_addr":"172.21.15.28:6823/3104474806","state":["exists","up"]},{"osd":3,"uuid":"d3117e1c-a230-4fc8-9cfc-37e4dc7e3d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2002097126},{"type":"v1","addr":"172.21.15.148:6801","nonce":2002097126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2002097126},{"type":"v1","addr":"172.21.15.148:6803","nonce":2002097126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2002097126},{"type":"v1","addr":"172.21.15.148:6807","nonce":2002097126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2002097126},{"type":"v1","addr":"172.21.15.148:6805","nonce":2002097126}]},"public_addr":"172.21.15.148:6801/2002097126","cluster_addr":"172.21.15.148:6803/2002097126","heartbeat_back_addr":"172.21.15.148:6807/2002097126","heartbeat_front_addr":"172.21.15.148:6805/2002097126","state":["exists","up"]},{"osd":4,"uuid":"d338fd80-0a76-441e-9cba-9cfd27cccc46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1943011533},{"type":"v1","addr":"172.21.15.148:6809","nonce":1943011533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1943011533},{"type":"v1","addr":"172.21.15.148:6811","nonce":1943011533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1943011533},{"type":"v1","addr":"172.21.15.148:6815","nonce":1943011533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1943011533},{"type":"v1","addr":"172.21.15.148:6813","nonce":1943011533}]},"public_addr":"172.21.15.148:6809/1943011533","cluster_addr":"172.21.15.148:6811/1943011533","heartbeat_back_addr":"172.21.15.148:6815/1943011533","heartbeat_front_addr":"172.21.15.148:6813/1943011533","state":["exists","up"]},{"osd":5,"uuid":"5c87b168-85e8-4d65-9763-8f1626e474ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":1578977000},{"type":"v1","addr":"172.21.15.148:6817","nonce":1578977000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1578977000},{"type":"v1","addr":"172.21.15.148:6819","nonce":1578977000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1578977000},{"type":"v1","addr":"172.21.15.148:6823","nonce":1578977000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1578977000},{"type":"v1","addr":"172.21.15.148:6821","nonce":1578977000}]},"public_addr":"172.21.15.148:6817/1578977000","cluster_addr":"172.21.15.148:6819/1578977000","heartbeat_back_addr":"172.21.15.148:6823/1578977000","heartbeat_front_addr":"172.21.15.148:6821/1578977000","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-07-03T15:38:11.596686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-07-03T15:38:38.765936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-07-03T15:39:05.464618+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-07-03T15:39:28.180164+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-07-03T15:39:52.836379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-07-03T15:40:17.711279+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.28:0/3219967940":"2024-07-04T15:37:00.336312+0000","172.21.15.28:0/1009019065":"2024-07-04T15:35:22.603924+0000","172.21.15.28:6801/1358048352":"2024-07-04T15:35:22.603924+0000","172.21.15.28:0/4294623705":"2024-07-04T15:37:00.336312+0000","172.21.15.28:0/390942453":"2024-07-04T15:35:52.741581+0000","172.21.15.28:0/423260870":"2024-07-04T15:35:22.603924+0000","172.21.15.28:6800/1358048352":"2024-07-04T15:35:22.603924+0000","172.21.15.28:6800/577887491":"2024-07-04T15:37:00.336312+0000","172.21.15.28:6800/1434683766":"2024-07-04T15:35:52.741581+0000","172.21.15.28:6801/1434683766":"2024-07-04T15:35:52.741581+0000","172.21.15.28:0/2750216161":"2024-07-04T15:35:52.741581+0000","172.21.15.28:0/3813763982":"2024-07-04T15:35:52.741581+0000","172.21.15.28:0/2748059179":"2024-07-04T15:35:22.603924+0000","172.21.15.28:6801/577887491":"2024-07-04T15:37:00.336312+0000","172.21.15.28:0/2964865891":"2024-07-04T15:37:00.336312+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-03T15:40:39.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.607+0000 7f4d29931640 1 -- 172.21.15.28:0/3703529296 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f4d10067190 msgr2=0x7f4d10069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:39.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.607+0000 7f4d29931640 1 --2- 172.21.15.28:0/3703529296 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f4d10067190 0x7f4d10069640 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4d14032430 tx=0x7f4d14035040 comp rx=0 tx=0).stop 2024-07-03T15:40:39.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.607+0000 7f4d29931640 1 -- 172.21.15.28:0/3703529296 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d24102cf0 msgr2=0x7f4d24113680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:39.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.607+0000 7f4d29931640 1 --2- 172.21.15.28:0/3703529296 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d24102cf0 0x7f4d24113680 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c00b790 tx=0x7f4d0c0091d0 comp rx=0 tx=0).stop 2024-07-03T15:40:39.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.607+0000 7f4d29931640 1 -- 172.21.15.28:0/3703529296 shutdown_connections 2024-07-03T15:40:39.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.607+0000 7f4d29931640 1 --2- 172.21.15.28:0/3703529296 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f4d10067190 0x7f4d10069640 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:39.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.607+0000 7f4d29931640 1 --2- 172.21.15.28:0/3703529296 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d24102cf0 0x7f4d24113680 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:39.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.607+0000 7f4d29931640 1 --2- 172.21.15.28:0/3703529296 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d24102350 0x7f4d24113140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:39.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.607+0000 7f4d29931640 1 -- 172.21.15.28:0/3703529296 >> 172.21.15.28:0/3703529296 conn(0x7f4d240f7f20 msgr2=0x7f4d240f8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:40:39.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.607+0000 7f4d29931640 1 -- 172.21.15.28:0/3703529296 shutdown_connections 2024-07-03T15:40:39.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:39.607+0000 7f4d29931640 1 -- 172.21.15.28:0/3703529296 wait complete. 2024-07-03T15:40:39.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:39 smithi028 ceph-mon[29148]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:40:39.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:39 smithi148 ceph-mon[36056]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:40:40.406 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-07-03T15:40:40.406 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph osd dump --format=json 2024-07-03T15:40:40.678 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:40:40.801 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:40 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/3703529296' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-03T15:40:40.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:40 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/3703529296' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-03T15:40:41.833 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:41 smithi028 ceph-mon[29148]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:40:41.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:41 smithi148 ceph-mon[36056]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:40:42.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.355+0000 7f9b901a4640 1 -- 172.21.15.28:0/4137680419 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b88102d10 msgr2=0x7f9b880fc3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:42.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.355+0000 7f9b901a4640 1 --2- 172.21.15.28:0/4137680419 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b88102d10 0x7f9b880fc3a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9b780048d0 tx=0x7f9b78009da0 comp rx=0 tx=0).stop 2024-07-03T15:40:42.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.355+0000 7f9b901a4640 1 -- 172.21.15.28:0/4137680419 shutdown_connections 2024-07-03T15:40:42.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.355+0000 7f9b901a4640 1 --2- 172.21.15.28:0/4137680419 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b88102d10 0x7f9b880fc3a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:42.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.355+0000 7f9b901a4640 1 --2- 172.21.15.28:0/4137680419 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b88102370 0x7f9b88102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:42.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.355+0000 7f9b901a4640 1 -- 172.21.15.28:0/4137680419 >> 172.21.15.28:0/4137680419 conn(0x7f9b880f7f00 msgr2=0x7f9b880fa310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:40:42.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.355+0000 7f9b901a4640 1 -- 172.21.15.28:0/4137680419 shutdown_connections 2024-07-03T15:40:42.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.355+0000 7f9b901a4640 1 -- 172.21.15.28:0/4137680419 wait complete. 2024-07-03T15:40:42.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.355+0000 7f9b901a4640 1 Processor -- start 2024-07-03T15:40:42.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.356+0000 7f9b901a4640 1 -- start start 2024-07-03T15:40:42.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.356+0000 7f9b901a4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b88102370 0x7f9b881133c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:42.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.356+0000 7f9b901a4640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b88102d10 0x7f9b88113900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:42.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.356+0000 7f9b901a4640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b88113f00 con 0x7f9b88102d10 2024-07-03T15:40:42.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.356+0000 7f9b901a4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b88114070 con 0x7f9b88102370 2024-07-03T15:40:42.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.356+0000 7f9b8d73f640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b88102d10 0x7f9b88113900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:42.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.356+0000 7f9b8d73f640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b88102d10 0x7f9b88113900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55008/0 (socket says 172.21.15.28:55008) 2024-07-03T15:40:42.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.356+0000 7f9b8d73f640 1 -- 172.21.15.28:0/1727640409 learned_addr learned my addr 172.21.15.28:0/1727640409 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:40:42.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.356+0000 7f9b8df40640 1 --2- 172.21.15.28:0/1727640409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b88102370 0x7f9b881133c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:42.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.356+0000 7f9b8d73f640 1 -- 172.21.15.28:0/1727640409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b88102370 msgr2=0x7f9b881133c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:42.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.356+0000 7f9b8d73f640 1 --2- 172.21.15.28:0/1727640409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b88102370 0x7f9b881133c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:42.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.356+0000 7f9b8d73f640 1 -- 172.21.15.28:0/1727640409 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b78005040 con 0x7f9b88102d10 2024-07-03T15:40:42.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.356+0000 7f9b8df40640 1 --2- 172.21.15.28:0/1727640409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b88102370 0x7f9b881133c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-03T15:40:42.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.356+0000 7f9b8d73f640 1 --2- 172.21.15.28:0/1727640409 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b88102d10 0x7f9b88113900 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9b78032430 tx=0x7f9b78009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:40:42.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.357+0000 7f9b7e7fc640 1 -- 172.21.15.28:0/1727640409 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b78038070 con 0x7f9b88102d10 2024-07-03T15:40:42.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.357+0000 7f9b901a4640 1 -- 172.21.15.28:0/1727640409 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b8810d490 con 0x7f9b88102d10 2024-07-03T15:40:42.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.357+0000 7f9b7e7fc640 1 -- 172.21.15.28:0/1727640409 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9b78008ee0 con 0x7f9b88102d10 2024-07-03T15:40:42.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.357+0000 7f9b7e7fc640 1 -- 172.21.15.28:0/1727640409 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b7803d7a0 con 0x7f9b88102d10 2024-07-03T15:40:42.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.357+0000 7f9b901a4640 1 -- 172.21.15.28:0/1727640409 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b8810d9d0 con 0x7f9b88102d10 2024-07-03T15:40:42.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.358+0000 7f9b901a4640 1 -- 172.21.15.28:0/1727640409 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b880fdae0 con 0x7f9b88102d10 2024-07-03T15:40:42.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.358+0000 7f9b7e7fc640 1 -- 172.21.15.28:0/1727640409 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f9b78030020 con 0x7f9b88102d10 2024-07-03T15:40:42.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.358+0000 7f9b7e7fc640 1 --2- 172.21.15.28:0/1727640409 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f9b74067190 0x7f9b74069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:42.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.358+0000 7f9b7e7fc640 1 -- 172.21.15.28:0/1727640409 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f9b780a4e40 con 0x7f9b88102d10 2024-07-03T15:40:42.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.359+0000 7f9b8df40640 1 --2- 172.21.15.28:0/1727640409 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f9b74067190 0x7f9b74069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:42.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.359+0000 7f9b8df40640 1 --2- 172.21.15.28:0/1727640409 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f9b74067190 0x7f9b74069640 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9b84004c60 tx=0x7f9b84009040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:40:42.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.363+0000 7f9b7e7fc640 1 -- 172.21.15.28:0/1727640409 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9b780724e0 con 0x7f9b88102d10 2024-07-03T15:40:42.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.540+0000 7f9b901a4640 1 -- 172.21.15.28:0/1727640409 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9b881094b0 con 0x7f9b88102d10 2024-07-03T15:40:42.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.541+0000 7f9b7e7fc640 1 -- 172.21.15.28:0/1727640409 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11093 (secure 0 0 0) 0x7f9b78075b00 con 0x7f9b88102d10 2024-07-03T15:40:42.544 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:40:42.545 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":40,"fsid":"a85e4be6-3951-11ef-bcab-c7b262605968","created":"2024-07-03T15:34:56.026978+0000","modified":"2024-07-03T15:40:20.495051+0000","last_up_change":"2024-07-03T15:40:19.490284+0000","last_in_change":"2024-07-03T15:40:01.150891+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-03T15:39:08.833339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"7ecd90b5-3715-4ba7-a696-588843e3582c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3973972034},{"type":"v1","addr":"172.21.15.28:6803","nonce":3973972034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3973972034},{"type":"v1","addr":"172.21.15.28:6805","nonce":3973972034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":3973972034},{"type":"v1","addr":"172.21.15.28:6809","nonce":3973972034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3973972034},{"type":"v1","addr":"172.21.15.28:6807","nonce":3973972034}]},"public_addr":"172.21.15.28:6803/3973972034","cluster_addr":"172.21.15.28:6805/3973972034","heartbeat_back_addr":"172.21.15.28:6809/3973972034","heartbeat_front_addr":"172.21.15.28:6807/3973972034","state":["exists","up"]},{"osd":1,"uuid":"31c5ae0c-6c57-4615-902b-72bf49fa5b72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1642389568},{"type":"v1","addr":"172.21.15.28:6811","nonce":1642389568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1642389568},{"type":"v1","addr":"172.21.15.28:6813","nonce":1642389568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":1642389568},{"type":"v1","addr":"172.21.15.28:6817","nonce":1642389568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1642389568},{"type":"v1","addr":"172.21.15.28:6815","nonce":1642389568}]},"public_addr":"172.21.15.28:6811/1642389568","cluster_addr":"172.21.15.28:6813/1642389568","heartbeat_back_addr":"172.21.15.28:6817/1642389568","heartbeat_front_addr":"172.21.15.28:6815/1642389568","state":["exists","up"]},{"osd":2,"uuid":"5ec8caa6-fb62-443f-bbd6-fbe52df5034c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":3104474806},{"type":"v1","addr":"172.21.15.28:6819","nonce":3104474806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":3104474806},{"type":"v1","addr":"172.21.15.28:6821","nonce":3104474806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":3104474806},{"type":"v1","addr":"172.21.15.28:6825","nonce":3104474806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":3104474806},{"type":"v1","addr":"172.21.15.28:6823","nonce":3104474806}]},"public_addr":"172.21.15.28:6819/3104474806","cluster_addr":"172.21.15.28:6821/3104474806","heartbeat_back_addr":"172.21.15.28:6825/3104474806","heartbeat_front_addr":"172.21.15.28:6823/3104474806","state":["exists","up"]},{"osd":3,"uuid":"d3117e1c-a230-4fc8-9cfc-37e4dc7e3d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2002097126},{"type":"v1","addr":"172.21.15.148:6801","nonce":2002097126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2002097126},{"type":"v1","addr":"172.21.15.148:6803","nonce":2002097126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2002097126},{"type":"v1","addr":"172.21.15.148:6807","nonce":2002097126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2002097126},{"type":"v1","addr":"172.21.15.148:6805","nonce":2002097126}]},"public_addr":"172.21.15.148:6801/2002097126","cluster_addr":"172.21.15.148:6803/2002097126","heartbeat_back_addr":"172.21.15.148:6807/2002097126","heartbeat_front_addr":"172.21.15.148:6805/2002097126","state":["exists","up"]},{"osd":4,"uuid":"d338fd80-0a76-441e-9cba-9cfd27cccc46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1943011533},{"type":"v1","addr":"172.21.15.148:6809","nonce":1943011533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1943011533},{"type":"v1","addr":"172.21.15.148:6811","nonce":1943011533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1943011533},{"type":"v1","addr":"172.21.15.148:6815","nonce":1943011533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1943011533},{"type":"v1","addr":"172.21.15.148:6813","nonce":1943011533}]},"public_addr":"172.21.15.148:6809/1943011533","cluster_addr":"172.21.15.148:6811/1943011533","heartbeat_back_addr":"172.21.15.148:6815/1943011533","heartbeat_front_addr":"172.21.15.148:6813/1943011533","state":["exists","up"]},{"osd":5,"uuid":"5c87b168-85e8-4d65-9763-8f1626e474ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":1578977000},{"type":"v1","addr":"172.21.15.148:6817","nonce":1578977000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1578977000},{"type":"v1","addr":"172.21.15.148:6819","nonce":1578977000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1578977000},{"type":"v1","addr":"172.21.15.148:6823","nonce":1578977000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1578977000},{"type":"v1","addr":"172.21.15.148:6821","nonce":1578977000}]},"public_addr":"172.21.15.148:6817/1578977000","cluster_addr":"172.21.15.148:6819/1578977000","heartbeat_back_addr":"172.21.15.148:6823/1578977000","heartbeat_front_addr":"172.21.15.148:6821/1578977000","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-07-03T15:38:11.596686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-07-03T15:38:38.765936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-07-03T15:39:05.464618+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-07-03T15:39:28.180164+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-07-03T15:39:52.836379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-07-03T15:40:17.711279+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.28:0/3219967940":"2024-07-04T15:37:00.336312+0000","172.21.15.28:0/1009019065":"2024-07-04T15:35:22.603924+0000","172.21.15.28:6801/1358048352":"2024-07-04T15:35:22.603924+0000","172.21.15.28:0/4294623705":"2024-07-04T15:37:00.336312+0000","172.21.15.28:0/390942453":"2024-07-04T15:35:52.741581+0000","172.21.15.28:0/423260870":"2024-07-04T15:35:22.603924+0000","172.21.15.28:6800/1358048352":"2024-07-04T15:35:22.603924+0000","172.21.15.28:6800/577887491":"2024-07-04T15:37:00.336312+0000","172.21.15.28:6800/1434683766":"2024-07-04T15:35:52.741581+0000","172.21.15.28:6801/1434683766":"2024-07-04T15:35:52.741581+0000","172.21.15.28:0/2750216161":"2024-07-04T15:35:52.741581+0000","172.21.15.28:0/3813763982":"2024-07-04T15:35:52.741581+0000","172.21.15.28:0/2748059179":"2024-07-04T15:35:22.603924+0000","172.21.15.28:6801/577887491":"2024-07-04T15:37:00.336312+0000","172.21.15.28:0/2964865891":"2024-07-04T15:37:00.336312+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-03T15:40:42.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.543+0000 7f9b901a4640 1 -- 172.21.15.28:0/1727640409 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f9b74067190 msgr2=0x7f9b74069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:42.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.543+0000 7f9b901a4640 1 --2- 172.21.15.28:0/1727640409 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f9b74067190 0x7f9b74069640 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9b84004c60 tx=0x7f9b84009040 comp rx=0 tx=0).stop 2024-07-03T15:40:42.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.543+0000 7f9b901a4640 1 -- 172.21.15.28:0/1727640409 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b88102d10 msgr2=0x7f9b88113900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:42.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.543+0000 7f9b901a4640 1 --2- 172.21.15.28:0/1727640409 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b88102d10 0x7f9b88113900 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9b78032430 tx=0x7f9b78009da0 comp rx=0 tx=0).stop 2024-07-03T15:40:42.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.543+0000 7f9b901a4640 1 -- 172.21.15.28:0/1727640409 shutdown_connections 2024-07-03T15:40:42.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.543+0000 7f9b901a4640 1 --2- 172.21.15.28:0/1727640409 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f9b74067190 0x7f9b74069640 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:42.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.543+0000 7f9b901a4640 1 --2- 172.21.15.28:0/1727640409 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b88102d10 0x7f9b88113900 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:42.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.543+0000 7f9b901a4640 1 --2- 172.21.15.28:0/1727640409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b88102370 0x7f9b881133c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:42.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.543+0000 7f9b901a4640 1 -- 172.21.15.28:0/1727640409 >> 172.21.15.28:0/1727640409 conn(0x7f9b880f7f00 msgr2=0x7f9b880f8e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:40:42.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.543+0000 7f9b901a4640 1 -- 172.21.15.28:0/1727640409 shutdown_connections 2024-07-03T15:40:42.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:42.543+0000 7f9b901a4640 1 -- 172.21.15.28:0/1727640409 wait complete. 2024-07-03T15:40:42.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:42 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/1727640409' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-03T15:40:42.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:42 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/1727640409' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-03T15:40:43.262 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-07-03T15:40:43.263 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-07-03T15:40:43.263 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-07-03T15:40:43.264 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-07-03T15:40:43.265 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-07-03T15:40:43.265 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-07-03T15:40:43.554 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:43 smithi028 ceph-mon[29148]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:40:43.591 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:40:43.596 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:40:43.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:43 smithi148 ceph-mon[36056]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:40:44.264 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:40:44.277 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:40:45.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:45 smithi028 ceph-mon[29148]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:40:45.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:45 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:40:45.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:45 smithi148 ceph-mon[36056]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:40:45.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:45 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:40:47.473 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:40:47.716 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:47 smithi028 ceph-mon[29148]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:40:47.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:47 smithi148 ceph-mon[36056]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:40:48.089 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:40:49.831 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:49 smithi028 ceph-mon[29148]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:40:49.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:49 smithi148 ceph-mon[36056]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:40:50.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.050+0000 7f5e21e1b640 1 -- 172.21.15.28:0/352492749 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e1c102350 msgr2=0x7f5e1c102720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:50.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.050+0000 7f5e21e1b640 1 --2- 172.21.15.28:0/352492749 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e1c102350 0x7f5e1c102720 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c0048d0 tx=0x7f5e0c009da0 comp rx=0 tx=0).stop 2024-07-03T15:40:50.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.050+0000 7f5e21e1b640 1 -- 172.21.15.28:0/352492749 shutdown_connections 2024-07-03T15:40:50.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.050+0000 7f5e21e1b640 1 --2- 172.21.15.28:0/352492749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e1c102cf0 0x7f5e1c0fc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:50.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.050+0000 7f5e21e1b640 1 --2- 172.21.15.28:0/352492749 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e1c102350 0x7f5e1c102720 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:50.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.050+0000 7f5e21e1b640 1 -- 172.21.15.28:0/352492749 >> 172.21.15.28:0/352492749 conn(0x7f5e1c0f7f00 msgr2=0x7f5e1c0fa310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:40:50.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.050+0000 7f5e21e1b640 1 -- 172.21.15.28:0/352492749 shutdown_connections 2024-07-03T15:40:50.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.050+0000 7f5e21e1b640 1 -- 172.21.15.28:0/352492749 wait complete. 2024-07-03T15:40:50.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.051+0000 7f5e21e1b640 1 Processor -- start 2024-07-03T15:40:50.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.051+0000 7f5e21e1b640 1 -- start start 2024-07-03T15:40:50.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.051+0000 7f5e21e1b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e1c102350 0x7f5e1c113110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:50.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.051+0000 7f5e21e1b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e1c102cf0 0x7f5e1c113650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:50.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.051+0000 7f5e21e1b640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1c113ce0 con 0x7f5e1c102cf0 2024-07-03T15:40:50.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.051+0000 7f5e21e1b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1c10d090 con 0x7f5e1c102350 2024-07-03T15:40:50.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.051+0000 7f5e12ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e1c102cf0 0x7f5e1c113650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:50.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.051+0000 7f5e12ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e1c102cf0 0x7f5e1c113650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56904/0 (socket says 172.21.15.28:56904) 2024-07-03T15:40:50.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.051+0000 7f5e12ffd640 1 -- 172.21.15.28:0/3582748629 learned_addr learned my addr 172.21.15.28:0/3582748629 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:40:50.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.051+0000 7f5e1b7fe640 1 --2- 172.21.15.28:0/3582748629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e1c102350 0x7f5e1c113110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:50.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.052+0000 7f5e12ffd640 1 -- 172.21.15.28:0/3582748629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e1c102350 msgr2=0x7f5e1c113110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:50.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.052+0000 7f5e12ffd640 1 --2- 172.21.15.28:0/3582748629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e1c102350 0x7f5e1c113110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:50.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.052+0000 7f5e12ffd640 1 -- 172.21.15.28:0/3582748629 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e0c005040 con 0x7f5e1c102cf0 2024-07-03T15:40:50.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.052+0000 7f5e12ffd640 1 --2- 172.21.15.28:0/3582748629 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e1c102cf0 0x7f5e1c113650 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5e0400b790 tx=0x7f5e040091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:40:50.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.052+0000 7f5e18ff9640 1 -- 172.21.15.28:0/3582748629 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e0400c700 con 0x7f5e1c102cf0 2024-07-03T15:40:50.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.052+0000 7f5e18ff9640 1 -- 172.21.15.28:0/3582748629 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5e0400cd30 con 0x7f5e1c102cf0 2024-07-03T15:40:50.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.052+0000 7f5e21e1b640 1 -- 172.21.15.28:0/3582748629 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e1c10d370 con 0x7f5e1c102cf0 2024-07-03T15:40:50.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.052+0000 7f5e18ff9640 1 -- 172.21.15.28:0/3582748629 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e04013370 con 0x7f5e1c102cf0 2024-07-03T15:40:50.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.052+0000 7f5e21e1b640 1 -- 172.21.15.28:0/3582748629 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e1c10d890 con 0x7f5e1c102cf0 2024-07-03T15:40:50.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.053+0000 7f5e21e1b640 1 -- 172.21.15.28:0/3582748629 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5e1c10ddd0 con 0x7f5e1c102cf0 2024-07-03T15:40:50.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.053+0000 7f5e18ff9640 1 -- 172.21.15.28:0/3582748629 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f5e04012070 con 0x7f5e1c102cf0 2024-07-03T15:40:50.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.054+0000 7f5e18ff9640 1 --2- 172.21.15.28:0/3582748629 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f5e08067190 0x7f5e08069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:50.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.054+0000 7f5e18ff9640 1 -- 172.21.15.28:0/3582748629 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f5e04084470 con 0x7f5e1c102cf0 2024-07-03T15:40:50.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.054+0000 7f5e1b7fe640 1 --2- 172.21.15.28:0/3582748629 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f5e08067190 0x7f5e08069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:50.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.054+0000 7f5e18ff9640 1 --2- 172.21.15.28:0/3582748629 >> [v2:172.21.15.28:6802/3973972034,v1:172.21.15.28:6803/3973972034] conn(0x7f5e0806cf10 0x7f5e0806f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:50.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.054+0000 7f5e18ff9640 1 -- 172.21.15.28:0/3582748629 --> [v2:172.21.15.28:6802/3973972034,v1:172.21.15.28:6803/3973972034] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5e0806f9d0 con 0x7f5e0806cf10 2024-07-03T15:40:50.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.054+0000 7f5e18ff9640 1 -- 172.21.15.28:0/3582748629 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e04086a20 con 0x7f5e1c102cf0 2024-07-03T15:40:50.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.054+0000 7f5e1bfff640 1 --2- 172.21.15.28:0/3582748629 >> [v2:172.21.15.28:6802/3973972034,v1:172.21.15.28:6803/3973972034] conn(0x7f5e0806cf10 0x7f5e0806f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:50.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.055+0000 7f5e1b7fe640 1 --2- 172.21.15.28:0/3582748629 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f5e08067190 0x7f5e08069640 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c000c00 tx=0x7f5e0c035040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:40:50.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.055+0000 7f5e1bfff640 1 --2- 172.21.15.28:0/3582748629 >> [v2:172.21.15.28:6802/3973972034,v1:172.21.15.28:6803/3973972034] conn(0x7f5e0806cf10 0x7f5e0806f320 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:40:50.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.055+0000 7f5e18ff9640 1 -- 172.21.15.28:0/3582748629 <== osd.0 v2:172.21.15.28:6802/3973972034 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f5e0806f9d0 con 0x7f5e0806cf10 2024-07-03T15:40:50.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.069+0000 7f5e21e1b640 1 -- 172.21.15.28:0/3582748629 --> [v2:172.21.15.28:6802/3973972034,v1:172.21.15.28:6803/3973972034] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5e1c0f9890 con 0x7f5e0806cf10 2024-07-03T15:40:50.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.069+0000 7f5e18ff9640 1 -- 172.21.15.28:0/3582748629 <== osd.0 v2:172.21.15.28:6802/3973972034 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5e1c0f9890 con 0x7f5e0806cf10 2024-07-03T15:40:50.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.069+0000 7f5e21e1b640 1 -- 172.21.15.28:0/3582748629 >> [v2:172.21.15.28:6802/3973972034,v1:172.21.15.28:6803/3973972034] conn(0x7f5e0806cf10 msgr2=0x7f5e0806f320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:50.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.069+0000 7f5e21e1b640 1 --2- 172.21.15.28:0/3582748629 >> [v2:172.21.15.28:6802/3973972034,v1:172.21.15.28:6803/3973972034] conn(0x7f5e0806cf10 0x7f5e0806f320 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:50.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.069+0000 7f5e21e1b640 1 -- 172.21.15.28:0/3582748629 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f5e08067190 msgr2=0x7f5e08069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:50.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.069+0000 7f5e21e1b640 1 --2- 172.21.15.28:0/3582748629 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f5e08067190 0x7f5e08069640 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c000c00 tx=0x7f5e0c035040 comp rx=0 tx=0).stop 2024-07-03T15:40:50.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.069+0000 7f5e21e1b640 1 -- 172.21.15.28:0/3582748629 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e1c102cf0 msgr2=0x7f5e1c113650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:50.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.069+0000 7f5e21e1b640 1 --2- 172.21.15.28:0/3582748629 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e1c102cf0 0x7f5e1c113650 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5e0400b790 tx=0x7f5e040091d0 comp rx=0 tx=0).stop 2024-07-03T15:40:50.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.069+0000 7f5e21e1b640 1 -- 172.21.15.28:0/3582748629 shutdown_connections 2024-07-03T15:40:50.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.069+0000 7f5e21e1b640 1 --2- 172.21.15.28:0/3582748629 >> [v2:172.21.15.28:6802/3973972034,v1:172.21.15.28:6803/3973972034] conn(0x7f5e0806cf10 0x7f5e0806f320 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:50.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.069+0000 7f5e21e1b640 1 --2- 172.21.15.28:0/3582748629 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f5e08067190 0x7f5e08069640 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:50.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.069+0000 7f5e21e1b640 1 --2- 172.21.15.28:0/3582748629 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e1c102cf0 0x7f5e1c113650 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:50.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.069+0000 7f5e21e1b640 1 --2- 172.21.15.28:0/3582748629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e1c102350 0x7f5e1c113110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:50.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.069+0000 7f5e21e1b640 1 -- 172.21.15.28:0/3582748629 >> 172.21.15.28:0/3582748629 conn(0x7f5e1c0f7f00 msgr2=0x7f5e1c0f8e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:40:50.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.069+0000 7f5e21e1b640 1 -- 172.21.15.28:0/3582748629 shutdown_connections 2024-07-03T15:40:50.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.069+0000 7f5e21e1b640 1 -- 172.21.15.28:0/3582748629 wait complete. 2024-07-03T15:40:50.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.519+0000 7f66aa92a640 1 -- 172.21.15.28:0/4018598456 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66a40faa10 msgr2=0x7f66a40fae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:50.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.519+0000 7f66aa92a640 1 --2- 172.21.15.28:0/4018598456 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66a40faa10 0x7f66a40fae60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f668c0048d0 tx=0x7f668c009da0 comp rx=0 tx=0).stop 2024-07-03T15:40:50.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.520+0000 7f66aa92a640 1 -- 172.21.15.28:0/4018598456 shutdown_connections 2024-07-03T15:40:50.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.520+0000 7f66aa92a640 1 --2- 172.21.15.28:0/4018598456 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66a40faa10 0x7f66a40fae60 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:50.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.520+0000 7f66aa92a640 1 --2- 172.21.15.28:0/4018598456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66a41023e0 0x7f66a41027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:50.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.520+0000 7f66aa92a640 1 -- 172.21.15.28:0/4018598456 >> 172.21.15.28:0/4018598456 conn(0x7f66a4070a90 msgr2=0x7f66a4070e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:40:50.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.520+0000 7f66aa92a640 1 -- 172.21.15.28:0/4018598456 shutdown_connections 2024-07-03T15:40:50.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.520+0000 7f66aa92a640 1 -- 172.21.15.28:0/4018598456 wait complete. 2024-07-03T15:40:50.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.520+0000 7f66aa92a640 1 Processor -- start 2024-07-03T15:40:50.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.520+0000 7f66aa92a640 1 -- start start 2024-07-03T15:40:50.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.520+0000 7f66aa92a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66a40faa10 0x7f66a41130a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:50.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.520+0000 7f66aa92a640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66a41023e0 0x7f66a41135e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:50.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.520+0000 7f66aa92a640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a4113be0 con 0x7f66a41023e0 2024-07-03T15:40:50.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.520+0000 7f66aa92a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a410d020 con 0x7f66a40faa10 2024-07-03T15:40:50.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.520+0000 7f669bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66a41023e0 0x7f66a41135e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:50.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.520+0000 7f669bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66a41023e0 0x7f66a41135e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56926/0 (socket says 172.21.15.28:56926) 2024-07-03T15:40:50.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.521+0000 7f669bfff640 1 -- 172.21.15.28:0/1112635159 learned_addr learned my addr 172.21.15.28:0/1112635159 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:40:50.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.521+0000 7f66a3fff640 1 --2- 172.21.15.28:0/1112635159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66a40faa10 0x7f66a41130a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:50.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.521+0000 7f669bfff640 1 -- 172.21.15.28:0/1112635159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66a40faa10 msgr2=0x7f66a41130a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:50.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.521+0000 7f669bfff640 1 --2- 172.21.15.28:0/1112635159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66a40faa10 0x7f66a41130a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:50.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.521+0000 7f669bfff640 1 -- 172.21.15.28:0/1112635159 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f668c005040 con 0x7f66a41023e0 2024-07-03T15:40:50.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.521+0000 7f669bfff640 1 --2- 172.21.15.28:0/1112635159 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66a41023e0 0x7f66a41135e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f668c0043d0 tx=0x7f668c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:40:50.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.521+0000 7f66a17fa640 1 -- 172.21.15.28:0/1112635159 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f668c038070 con 0x7f66a41023e0 2024-07-03T15:40:50.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.521+0000 7f66a17fa640 1 -- 172.21.15.28:0/1112635159 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f668c032970 con 0x7f66a41023e0 2024-07-03T15:40:50.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.521+0000 7f66aa92a640 1 -- 172.21.15.28:0/1112635159 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66a410d2a0 con 0x7f66a41023e0 2024-07-03T15:40:50.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.521+0000 7f66a17fa640 1 -- 172.21.15.28:0/1112635159 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f668c03da60 con 0x7f66a41023e0 2024-07-03T15:40:50.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.521+0000 7f66aa92a640 1 -- 172.21.15.28:0/1112635159 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66a410d760 con 0x7f66a41023e0 2024-07-03T15:40:50.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.522+0000 7f66aa92a640 1 -- 172.21.15.28:0/1112635159 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f66a410dc40 con 0x7f66a41023e0 2024-07-03T15:40:50.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.523+0000 7f66a17fa640 1 -- 172.21.15.28:0/1112635159 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f668c030020 con 0x7f66a41023e0 2024-07-03T15:40:50.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.523+0000 7f66a17fa640 1 --2- 172.21.15.28:0/1112635159 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f6690067190 0x7f6690069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:50.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.523+0000 7f66a17fa640 1 -- 172.21.15.28:0/1112635159 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f668c0a4900 con 0x7f66a41023e0 2024-07-03T15:40:50.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.523+0000 7f66a3fff640 1 --2- 172.21.15.28:0/1112635159 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f6690067190 0x7f6690069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:50.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.523+0000 7f66a17fa640 1 --2- 172.21.15.28:0/1112635159 >> [v2:172.21.15.148:6800/2002097126,v1:172.21.15.148:6801/2002097126] conn(0x7f669006cf10 0x7f669006f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:50.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.523+0000 7f66a17fa640 1 -- 172.21.15.28:0/1112635159 --> [v2:172.21.15.148:6800/2002097126,v1:172.21.15.148:6801/2002097126] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f669006f9d0 con 0x7f669006cf10 2024-07-03T15:40:50.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.523+0000 7f66a17fa640 1 -- 172.21.15.28:0/1112635159 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f668c0a9220 con 0x7f66a41023e0 2024-07-03T15:40:50.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.524+0000 7f66a8ec7640 1 --2- 172.21.15.28:0/1112635159 >> [v2:172.21.15.148:6800/2002097126,v1:172.21.15.148:6801/2002097126] conn(0x7f669006cf10 0x7f669006f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:50.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.524+0000 7f66a3fff640 1 --2- 172.21.15.28:0/1112635159 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f6690067190 0x7f6690069640 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6694004eb0 tx=0x7f669400b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:40:50.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.524+0000 7f66a8ec7640 1 --2- 172.21.15.28:0/1112635159 >> [v2:172.21.15.148:6800/2002097126,v1:172.21.15.148:6801/2002097126] conn(0x7f669006cf10 0x7f669006f320 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:40:50.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.525+0000 7f66a17fa640 1 -- 172.21.15.28:0/1112635159 <== osd.3 v2:172.21.15.148:6800/2002097126 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f669006f9d0 con 0x7f669006cf10 2024-07-03T15:40:50.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.538+0000 7f66aa92a640 1 -- 172.21.15.28:0/1112635159 --> [v2:172.21.15.148:6800/2002097126,v1:172.21.15.148:6801/2002097126] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f66a405dc60 con 0x7f669006cf10 2024-07-03T15:40:50.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.539+0000 7f66a17fa640 1 -- 172.21.15.28:0/1112635159 <== osd.3 v2:172.21.15.148:6800/2002097126 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f66a405dc60 con 0x7f669006cf10 2024-07-03T15:40:50.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.539+0000 7f66aa92a640 1 -- 172.21.15.28:0/1112635159 >> [v2:172.21.15.148:6800/2002097126,v1:172.21.15.148:6801/2002097126] conn(0x7f669006cf10 msgr2=0x7f669006f320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:50.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.539+0000 7f66aa92a640 1 --2- 172.21.15.28:0/1112635159 >> [v2:172.21.15.148:6800/2002097126,v1:172.21.15.148:6801/2002097126] conn(0x7f669006cf10 0x7f669006f320 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:50.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.539+0000 7f66aa92a640 1 -- 172.21.15.28:0/1112635159 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f6690067190 msgr2=0x7f6690069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:50.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.539+0000 7f66aa92a640 1 --2- 172.21.15.28:0/1112635159 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f6690067190 0x7f6690069640 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6694004eb0 tx=0x7f669400b040 comp rx=0 tx=0).stop 2024-07-03T15:40:50.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.539+0000 7f66aa92a640 1 -- 172.21.15.28:0/1112635159 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66a41023e0 msgr2=0x7f66a41135e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:50.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.539+0000 7f66aa92a640 1 --2- 172.21.15.28:0/1112635159 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66a41023e0 0x7f66a41135e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f668c0043d0 tx=0x7f668c009da0 comp rx=0 tx=0).stop 2024-07-03T15:40:50.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.539+0000 7f66aa92a640 1 -- 172.21.15.28:0/1112635159 shutdown_connections 2024-07-03T15:40:50.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.539+0000 7f66aa92a640 1 --2- 172.21.15.28:0/1112635159 >> [v2:172.21.15.148:6800/2002097126,v1:172.21.15.148:6801/2002097126] conn(0x7f669006cf10 0x7f669006f320 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:50.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.539+0000 7f66aa92a640 1 --2- 172.21.15.28:0/1112635159 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f6690067190 0x7f6690069640 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:50.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.539+0000 7f66aa92a640 1 --2- 172.21.15.28:0/1112635159 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66a41023e0 0x7f66a41135e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:50.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.539+0000 7f66aa92a640 1 --2- 172.21.15.28:0/1112635159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66a40faa10 0x7f66a41130a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:50.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.539+0000 7f66aa92a640 1 -- 172.21.15.28:0/1112635159 >> 172.21.15.28:0/1112635159 conn(0x7f66a4070a90 msgr2=0x7f66a40f9d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:40:50.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.539+0000 7f66aa92a640 1 -- 172.21.15.28:0/1112635159 shutdown_connections 2024-07-03T15:40:50.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:50.539+0000 7f66aa92a640 1 -- 172.21.15.28:0/1112635159 wait complete. 2024-07-03T15:40:51.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.252+0000 7f6ddcda1640 1 -- 172.21.15.28:0/259108279 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6dd8102b60 msgr2=0x7f6dd8064c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:51.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.252+0000 7f6ddcda1640 1 --2- 172.21.15.28:0/259108279 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6dd8102b60 0x7f6dd8064c40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc0048d0 tx=0x7f6dcc009da0 comp rx=0 tx=0).stop 2024-07-03T15:40:51.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.252+0000 7f6ddcda1640 1 -- 172.21.15.28:0/259108279 shutdown_connections 2024-07-03T15:40:51.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.252+0000 7f6ddcda1640 1 --2- 172.21.15.28:0/259108279 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6dd8102b60 0x7f6dd8064c40 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:51.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.252+0000 7f6ddcda1640 1 --2- 172.21.15.28:0/259108279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6dd81021c0 0x7f6dd8102590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:51.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.252+0000 7f6ddcda1640 1 -- 172.21.15.28:0/259108279 >> 172.21.15.28:0/259108279 conn(0x7f6dd8071180 msgr2=0x7f6dd8071580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:40:51.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.252+0000 7f6ddcda1640 1 -- 172.21.15.28:0/259108279 shutdown_connections 2024-07-03T15:40:51.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.252+0000 7f6ddcda1640 1 -- 172.21.15.28:0/259108279 wait complete. 2024-07-03T15:40:51.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.252+0000 7f6ddcda1640 1 Processor -- start 2024-07-03T15:40:51.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.252+0000 7f6ddcda1640 1 -- start start 2024-07-03T15:40:51.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.253+0000 7f6ddcda1640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6dd81021c0 0x7f6dd81151e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:51.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.253+0000 7f6ddcda1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6dd8102b60 0x7f6dd8115720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:51.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.253+0000 7f6ddcda1640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd8115d20 con 0x7f6dd81021c0 2024-07-03T15:40:51.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.253+0000 7f6ddcda1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd810f160 con 0x7f6dd8102b60 2024-07-03T15:40:51.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.253+0000 7f6dd77fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6dd81021c0 0x7f6dd81151e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:51.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.253+0000 7f6dd77fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6dd81021c0 0x7f6dd81151e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56946/0 (socket says 172.21.15.28:56946) 2024-07-03T15:40:51.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.253+0000 7f6dd77fe640 1 -- 172.21.15.28:0/1148281535 learned_addr learned my addr 172.21.15.28:0/1148281535 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:40:51.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.253+0000 7f6dd6ffd640 1 --2- 172.21.15.28:0/1148281535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6dd8102b60 0x7f6dd8115720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:51.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.253+0000 7f6dd77fe640 1 -- 172.21.15.28:0/1148281535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6dd8102b60 msgr2=0x7f6dd8115720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:51.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.253+0000 7f6dd77fe640 1 --2- 172.21.15.28:0/1148281535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6dd8102b60 0x7f6dd8115720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:51.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.253+0000 7f6dd77fe640 1 -- 172.21.15.28:0/1148281535 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dcc005040 con 0x7f6dd81021c0 2024-07-03T15:40:51.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.253+0000 7f6dd77fe640 1 --2- 172.21.15.28:0/1148281535 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6dd81021c0 0x7f6dd81151e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6dc800cb10 tx=0x7f6dc8007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:40:51.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.254+0000 7f6dbffff640 1 -- 172.21.15.28:0/1148281535 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6dc8004200 con 0x7f6dd81021c0 2024-07-03T15:40:51.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.254+0000 7f6dbffff640 1 -- 172.21.15.28:0/1148281535 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6dc800ac70 con 0x7f6dd81021c0 2024-07-03T15:40:51.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.254+0000 7f6ddcda1640 1 -- 172.21.15.28:0/1148281535 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dd810f440 con 0x7f6dd81021c0 2024-07-03T15:40:51.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.254+0000 7f6dbffff640 1 -- 172.21.15.28:0/1148281535 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6dc8003db0 con 0x7f6dd81021c0 2024-07-03T15:40:51.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.254+0000 7f6ddcda1640 1 -- 172.21.15.28:0/1148281535 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dd810f930 con 0x7f6dd81021c0 2024-07-03T15:40:51.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.255+0000 7f6ddcda1640 1 -- 172.21.15.28:0/1148281535 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6dd810fe40 con 0x7f6dd81021c0 2024-07-03T15:40:51.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.255+0000 7f6dbffff640 1 -- 172.21.15.28:0/1148281535 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f6dc800a7a0 con 0x7f6dd81021c0 2024-07-03T15:40:51.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.256+0000 7f6dbffff640 1 --2- 172.21.15.28:0/1148281535 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f6dc00671a0 0x7f6dc0069650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:51.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.256+0000 7f6dbffff640 1 -- 172.21.15.28:0/1148281535 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f6dc8084d50 con 0x7f6dd81021c0 2024-07-03T15:40:51.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.256+0000 7f6dd6ffd640 1 --2- 172.21.15.28:0/1148281535 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f6dc00671a0 0x7f6dc0069650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:51.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.256+0000 7f6dbffff640 1 --2- 172.21.15.28:0/1148281535 >> [v2:172.21.15.28:6810/1642389568,v1:172.21.15.28:6811/1642389568] conn(0x7f6dc006cf20 0x7f6dc006f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:51.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.256+0000 7f6dbffff640 1 -- 172.21.15.28:0/1148281535 --> [v2:172.21.15.28:6810/1642389568,v1:172.21.15.28:6811/1642389568] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6dc006f9e0 con 0x7f6dc006cf20 2024-07-03T15:40:51.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.256+0000 7f6dbffff640 1 -- 172.21.15.28:0/1148281535 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6dc808a3b0 con 0x7f6dd81021c0 2024-07-03T15:40:51.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.256+0000 7f6dd7fff640 1 --2- 172.21.15.28:0/1148281535 >> [v2:172.21.15.28:6810/1642389568,v1:172.21.15.28:6811/1642389568] conn(0x7f6dc006cf20 0x7f6dc006f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:51.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.256+0000 7f6dd6ffd640 1 --2- 172.21.15.28:0/1148281535 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f6dc00671a0 0x7f6dc0069650 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc0048a0 tx=0x7f6dcc035040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:40:51.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.256+0000 7f6dd7fff640 1 --2- 172.21.15.28:0/1148281535 >> [v2:172.21.15.28:6810/1642389568,v1:172.21.15.28:6811/1642389568] conn(0x7f6dc006cf20 0x7f6dc006f330 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:40:51.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.257+0000 7f6dbffff640 1 -- 172.21.15.28:0/1148281535 <== osd.1 v2:172.21.15.28:6810/1642389568 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f6dc006f9e0 con 0x7f6dc006cf20 2024-07-03T15:40:51.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.271+0000 7f6ddcda1640 1 -- 172.21.15.28:0/1148281535 --> [v2:172.21.15.28:6810/1642389568,v1:172.21.15.28:6811/1642389568] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6dd810ffb0 con 0x7f6dc006cf20 2024-07-03T15:40:51.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.271+0000 7f6dbffff640 1 -- 172.21.15.28:0/1148281535 <== osd.1 v2:172.21.15.28:6810/1642389568 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6dd810ffb0 con 0x7f6dc006cf20 2024-07-03T15:40:51.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.271+0000 7f6ddcda1640 1 -- 172.21.15.28:0/1148281535 >> [v2:172.21.15.28:6810/1642389568,v1:172.21.15.28:6811/1642389568] conn(0x7f6dc006cf20 msgr2=0x7f6dc006f330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:51.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.271+0000 7f6ddcda1640 1 --2- 172.21.15.28:0/1148281535 >> [v2:172.21.15.28:6810/1642389568,v1:172.21.15.28:6811/1642389568] conn(0x7f6dc006cf20 0x7f6dc006f330 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:51.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.271+0000 7f6ddcda1640 1 -- 172.21.15.28:0/1148281535 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f6dc00671a0 msgr2=0x7f6dc0069650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:51.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.271+0000 7f6ddcda1640 1 --2- 172.21.15.28:0/1148281535 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f6dc00671a0 0x7f6dc0069650 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc0048a0 tx=0x7f6dcc035040 comp rx=0 tx=0).stop 2024-07-03T15:40:51.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.271+0000 7f6ddcda1640 1 -- 172.21.15.28:0/1148281535 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6dd81021c0 msgr2=0x7f6dd81151e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:51.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.271+0000 7f6ddcda1640 1 --2- 172.21.15.28:0/1148281535 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6dd81021c0 0x7f6dd81151e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6dc800cb10 tx=0x7f6dc8007ee0 comp rx=0 tx=0).stop 2024-07-03T15:40:51.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.271+0000 7f6ddcda1640 1 -- 172.21.15.28:0/1148281535 shutdown_connections 2024-07-03T15:40:51.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.271+0000 7f6ddcda1640 1 --2- 172.21.15.28:0/1148281535 >> [v2:172.21.15.28:6810/1642389568,v1:172.21.15.28:6811/1642389568] conn(0x7f6dc006cf20 0x7f6dc006f330 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:51.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.271+0000 7f6ddcda1640 1 --2- 172.21.15.28:0/1148281535 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f6dc00671a0 0x7f6dc0069650 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:51.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.271+0000 7f6ddcda1640 1 --2- 172.21.15.28:0/1148281535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6dd8102b60 0x7f6dd8115720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:51.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.271+0000 7f6ddcda1640 1 --2- 172.21.15.28:0/1148281535 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6dd81021c0 0x7f6dd81151e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:51.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.271+0000 7f6ddcda1640 1 -- 172.21.15.28:0/1148281535 >> 172.21.15.28:0/1148281535 conn(0x7f6dd8071180 msgr2=0x7f6dd80f9610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:40:51.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.271+0000 7f6ddcda1640 1 -- 172.21.15.28:0/1148281535 shutdown_connections 2024-07-03T15:40:51.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.272+0000 7f6ddcda1640 1 -- 172.21.15.28:0/1148281535 wait complete. 2024-07-03T15:40:51.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.624+0000 7f6af0b04640 1 -- 172.21.15.28:0/491573297 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6aec0fdf50 msgr2=0x7f6aec0fe3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:51.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.624+0000 7f6af0b04640 1 --2- 172.21.15.28:0/491573297 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6aec0fdf50 0x7f6aec0fe3c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6ad40048d0 tx=0x7f6ad4009da0 comp rx=0 tx=0).stop 2024-07-03T15:40:51.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.625+0000 7f6af0b04640 1 -- 172.21.15.28:0/491573297 shutdown_connections 2024-07-03T15:40:51.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.625+0000 7f6af0b04640 1 --2- 172.21.15.28:0/491573297 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6aec0fdf50 0x7f6aec0fe3c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:51.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.625+0000 7f6af0b04640 1 --2- 172.21.15.28:0/491573297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6aec0650e0 0x7f6aec0654b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:51.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.625+0000 7f6af0b04640 1 -- 172.21.15.28:0/491573297 >> 172.21.15.28:0/491573297 conn(0x7f6aec0721c0 msgr2=0x7f6aec0725c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:40:51.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.625+0000 7f6af0b04640 1 -- 172.21.15.28:0/491573297 shutdown_connections 2024-07-03T15:40:51.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.625+0000 7f6af0b04640 1 -- 172.21.15.28:0/491573297 wait complete. 2024-07-03T15:40:51.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.625+0000 7f6af0b04640 1 Processor -- start 2024-07-03T15:40:51.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.625+0000 7f6af0b04640 1 -- start start 2024-07-03T15:40:51.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.625+0000 7f6af0b04640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6aec0650e0 0x7f6aec113120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:51.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.625+0000 7f6af0b04640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6aec0fdf50 0x7f6aec113660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:51.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.625+0000 7f6af0b04640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec113c60 con 0x7f6aec0650e0 2024-07-03T15:40:51.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.625+0000 7f6af0b04640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec10d0f0 con 0x7f6aec0fdf50 2024-07-03T15:40:51.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.625+0000 7f6aea59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6aec0650e0 0x7f6aec113120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:51.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.625+0000 7f6aea59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6aec0650e0 0x7f6aec113120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56972/0 (socket says 172.21.15.28:56972) 2024-07-03T15:40:51.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.625+0000 7f6aea59c640 1 -- 172.21.15.28:0/2422533212 learned_addr learned my addr 172.21.15.28:0/2422533212 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:40:51.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.626+0000 7f6ae3fff640 1 --2- 172.21.15.28:0/2422533212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6aec0fdf50 0x7f6aec113660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:51.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.626+0000 7f6aea59c640 1 -- 172.21.15.28:0/2422533212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6aec0fdf50 msgr2=0x7f6aec113660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:51.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.626+0000 7f6aea59c640 1 --2- 172.21.15.28:0/2422533212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6aec0fdf50 0x7f6aec113660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:51.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.626+0000 7f6aea59c640 1 -- 172.21.15.28:0/2422533212 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad4005040 con 0x7f6aec0650e0 2024-07-03T15:40:51.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.626+0000 7f6aea59c640 1 --2- 172.21.15.28:0/2422533212 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6aec0650e0 0x7f6aec113120 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6adc002a80 tx=0x7f6adc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:40:51.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.626+0000 7f6ae2ffd640 1 -- 172.21.15.28:0/2422533212 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6adc007c30 con 0x7f6aec0650e0 2024-07-03T15:40:51.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.626+0000 7f6ae2ffd640 1 -- 172.21.15.28:0/2422533212 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6adc00f040 con 0x7f6aec0650e0 2024-07-03T15:40:51.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.626+0000 7f6af0b04640 1 -- 172.21.15.28:0/2422533212 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aec10d3d0 con 0x7f6aec0650e0 2024-07-03T15:40:51.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.626+0000 7f6ae2ffd640 1 -- 172.21.15.28:0/2422533212 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6adc0096b0 con 0x7f6aec0650e0 2024-07-03T15:40:51.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.626+0000 7f6af0b04640 1 -- 172.21.15.28:0/2422533212 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aec10d8f0 con 0x7f6aec0650e0 2024-07-03T15:40:51.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.627+0000 7f6af0b04640 1 -- 172.21.15.28:0/2422533212 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6aec10de20 con 0x7f6aec0650e0 2024-07-03T15:40:51.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.628+0000 7f6ae2ffd640 1 -- 172.21.15.28:0/2422533212 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f6adc00d6b0 con 0x7f6aec0650e0 2024-07-03T15:40:51.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.628+0000 7f6ae2ffd640 1 --2- 172.21.15.28:0/2422533212 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f6ad8067190 0x7f6ad8069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:51.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.628+0000 7f6ae2ffd640 1 -- 172.21.15.28:0/2422533212 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f6adc084110 con 0x7f6aec0650e0 2024-07-03T15:40:51.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.628+0000 7f6ae3fff640 1 --2- 172.21.15.28:0/2422533212 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f6ad8067190 0x7f6ad8069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:51.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.628+0000 7f6ae2ffd640 1 --2- 172.21.15.28:0/2422533212 >> [v2:172.21.15.28:6818/3104474806,v1:172.21.15.28:6819/3104474806] conn(0x7f6ad806cf10 0x7f6ad806f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:51.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.628+0000 7f6ae2ffd640 1 -- 172.21.15.28:0/2422533212 --> [v2:172.21.15.28:6818/3104474806,v1:172.21.15.28:6819/3104474806] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6ad806f9d0 con 0x7f6ad806cf10 2024-07-03T15:40:51.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.628+0000 7f6ae2ffd640 1 -- 172.21.15.28:0/2422533212 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6adc086760 con 0x7f6aec0650e0 2024-07-03T15:40:51.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.629+0000 7f6aead9d640 1 --2- 172.21.15.28:0/2422533212 >> [v2:172.21.15.28:6818/3104474806,v1:172.21.15.28:6819/3104474806] conn(0x7f6ad806cf10 0x7f6ad806f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:51.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.629+0000 7f6ae3fff640 1 --2- 172.21.15.28:0/2422533212 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f6ad8067190 0x7f6ad8069640 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6aec10e9e0 tx=0x7f6ad4035040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:40:51.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.629+0000 7f6aead9d640 1 --2- 172.21.15.28:0/2422533212 >> [v2:172.21.15.28:6818/3104474806,v1:172.21.15.28:6819/3104474806] conn(0x7f6ad806cf10 0x7f6ad806f320 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:40:51.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.630+0000 7f6ae2ffd640 1 -- 172.21.15.28:0/2422533212 <== osd.2 v2:172.21.15.28:6818/3104474806 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f6ad806f9d0 con 0x7f6ad806cf10 2024-07-03T15:40:51.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.643+0000 7f6af0b04640 1 -- 172.21.15.28:0/2422533212 --> [v2:172.21.15.28:6818/3104474806,v1:172.21.15.28:6819/3104474806] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6aec10df90 con 0x7f6ad806cf10 2024-07-03T15:40:51.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.643+0000 7f6ae2ffd640 1 -- 172.21.15.28:0/2422533212 <== osd.2 v2:172.21.15.28:6818/3104474806 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6aec10df90 con 0x7f6ad806cf10 2024-07-03T15:40:51.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.643+0000 7f6af0b04640 1 -- 172.21.15.28:0/2422533212 >> [v2:172.21.15.28:6818/3104474806,v1:172.21.15.28:6819/3104474806] conn(0x7f6ad806cf10 msgr2=0x7f6ad806f320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:51.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.643+0000 7f6af0b04640 1 --2- 172.21.15.28:0/2422533212 >> [v2:172.21.15.28:6818/3104474806,v1:172.21.15.28:6819/3104474806] conn(0x7f6ad806cf10 0x7f6ad806f320 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:51.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.643+0000 7f6af0b04640 1 -- 172.21.15.28:0/2422533212 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f6ad8067190 msgr2=0x7f6ad8069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:51.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.643+0000 7f6af0b04640 1 --2- 172.21.15.28:0/2422533212 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f6ad8067190 0x7f6ad8069640 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6aec10e9e0 tx=0x7f6ad4035040 comp rx=0 tx=0).stop 2024-07-03T15:40:51.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.643+0000 7f6af0b04640 1 -- 172.21.15.28:0/2422533212 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6aec0650e0 msgr2=0x7f6aec113120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:51.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.643+0000 7f6af0b04640 1 --2- 172.21.15.28:0/2422533212 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6aec0650e0 0x7f6aec113120 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6adc002a80 tx=0x7f6adc004600 comp rx=0 tx=0).stop 2024-07-03T15:40:51.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.643+0000 7f6af0b04640 1 -- 172.21.15.28:0/2422533212 shutdown_connections 2024-07-03T15:40:51.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.643+0000 7f6af0b04640 1 --2- 172.21.15.28:0/2422533212 >> [v2:172.21.15.28:6818/3104474806,v1:172.21.15.28:6819/3104474806] conn(0x7f6ad806cf10 0x7f6ad806f320 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:51.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.643+0000 7f6af0b04640 1 --2- 172.21.15.28:0/2422533212 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f6ad8067190 0x7f6ad8069640 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:51.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.643+0000 7f6af0b04640 1 --2- 172.21.15.28:0/2422533212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6aec0fdf50 0x7f6aec113660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:51.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.643+0000 7f6af0b04640 1 --2- 172.21.15.28:0/2422533212 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6aec0650e0 0x7f6aec113120 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:51.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.643+0000 7f6af0b04640 1 -- 172.21.15.28:0/2422533212 >> 172.21.15.28:0/2422533212 conn(0x7f6aec0721c0 msgr2=0x7f6aec1051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:40:51.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.643+0000 7f6af0b04640 1 -- 172.21.15.28:0/2422533212 shutdown_connections 2024-07-03T15:40:51.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:51.643+0000 7f6af0b04640 1 -- 172.21.15.28:0/2422533212 wait complete. 2024-07-03T15:40:51.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:51 smithi028 ceph-mon[29148]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:40:51.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:51 smithi148 ceph-mon[36056]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:40:52.353 INFO:teuthology.orchestra.run.smithi028.stdout:81604378647 2024-07-03T15:40:52.354 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-07-03T15:40:52.770 INFO:teuthology.orchestra.run.smithi028.stdout:38654705699 2024-07-03T15:40:52.771 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-07-03T15:40:53.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:53 smithi028 ceph-mon[29148]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:40:53.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:53 smithi148 ceph-mon[36056]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:40:54.328 INFO:teuthology.orchestra.run.smithi028.stdout:60129542173 2024-07-03T15:40:54.329 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-07-03T15:40:55.178 INFO:teuthology.orchestra.run.smithi028.stdout:111669149713 2024-07-03T15:40:55.178 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-07-03T15:40:55.283 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:40:55.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.690+0000 7f3b8bc10640 1 -- 172.21.15.28:0/415935337 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b841023a0 msgr2=0x7f3b84102770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:55.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.690+0000 7f3b8bc10640 1 --2- 172.21.15.28:0/415935337 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b841023a0 0x7f3b84102770 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3b800048d0 tx=0x7f3b80009da0 comp rx=0 tx=0).stop 2024-07-03T15:40:55.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.690+0000 7f3b8bc10640 1 -- 172.21.15.28:0/415935337 shutdown_connections 2024-07-03T15:40:55.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.690+0000 7f3b8bc10640 1 --2- 172.21.15.28:0/415935337 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b84102d40 0x7f3b840fc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:55.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.690+0000 7f3b8bc10640 1 --2- 172.21.15.28:0/415935337 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b841023a0 0x7f3b84102770 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:55.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.690+0000 7f3b8bc10640 1 -- 172.21.15.28:0/415935337 >> 172.21.15.28:0/415935337 conn(0x7f3b840f7fb0 msgr2=0x7f3b840fa3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:40:55.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.690+0000 7f3b8bc10640 1 -- 172.21.15.28:0/415935337 shutdown_connections 2024-07-03T15:40:55.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.690+0000 7f3b8bc10640 1 -- 172.21.15.28:0/415935337 wait complete. 2024-07-03T15:40:55.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.691+0000 7f3b8bc10640 1 Processor -- start 2024-07-03T15:40:55.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.691+0000 7f3b8bc10640 1 -- start start 2024-07-03T15:40:55.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.691+0000 7f3b8bc10640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b841023a0 0x7f3b8406f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:55.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.691+0000 7f3b8bc10640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b84102d40 0x7f3b8406fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:55.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.691+0000 7f3b8bc10640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b840702a0 con 0x7f3b84102d40 2024-07-03T15:40:55.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.691+0000 7f3b8bc10640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b84070410 con 0x7f3b841023a0 2024-07-03T15:40:55.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.691+0000 7f3b891ab640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b84102d40 0x7f3b8406fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:55.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.691+0000 7f3b891ab640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b84102d40 0x7f3b8406fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56984/0 (socket says 172.21.15.28:56984) 2024-07-03T15:40:55.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.692+0000 7f3b891ab640 1 -- 172.21.15.28:0/1750143525 learned_addr learned my addr 172.21.15.28:0/1750143525 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:40:55.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.692+0000 7f3b899ac640 1 --2- 172.21.15.28:0/1750143525 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b841023a0 0x7f3b8406f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:55.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.692+0000 7f3b891ab640 1 -- 172.21.15.28:0/1750143525 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b841023a0 msgr2=0x7f3b8406f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:55.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.692+0000 7f3b891ab640 1 --2- 172.21.15.28:0/1750143525 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b841023a0 0x7f3b8406f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:55.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.692+0000 7f3b891ab640 1 -- 172.21.15.28:0/1750143525 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b80005040 con 0x7f3b84102d40 2024-07-03T15:40:55.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.692+0000 7f3b891ab640 1 --2- 172.21.15.28:0/1750143525 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b84102d40 0x7f3b8406fd60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3b74002a50 tx=0x7f3b74004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:40:55.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.692+0000 7f3b7a7fc640 1 -- 172.21.15.28:0/1750143525 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b74007c30 con 0x7f3b84102d40 2024-07-03T15:40:55.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.692+0000 7f3b7a7fc640 1 -- 172.21.15.28:0/1750143525 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3b7400f040 con 0x7f3b84102d40 2024-07-03T15:40:55.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.692+0000 7f3b8bc10640 1 -- 172.21.15.28:0/1750143525 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b840706a0 con 0x7f3b84102d40 2024-07-03T15:40:55.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.692+0000 7f3b7a7fc640 1 -- 172.21.15.28:0/1750143525 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b7400dad0 con 0x7f3b84102d40 2024-07-03T15:40:55.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.692+0000 7f3b8bc10640 1 -- 172.21.15.28:0/1750143525 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b841197d0 con 0x7f3b84102d40 2024-07-03T15:40:55.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.694+0000 7f3b8bc10640 1 -- 172.21.15.28:0/1750143525 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3b840f98d0 con 0x7f3b84102d40 2024-07-03T15:40:55.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.694+0000 7f3b7a7fc640 1 -- 172.21.15.28:0/1750143525 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f3b7401c400 con 0x7f3b84102d40 2024-07-03T15:40:55.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.694+0000 7f3b7a7fc640 1 --2- 172.21.15.28:0/1750143525 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3b70067190 0x7f3b70069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:55.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.694+0000 7f3b7a7fc640 1 -- 172.21.15.28:0/1750143525 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f3b74084000 con 0x7f3b84102d40 2024-07-03T15:40:55.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.694+0000 7f3b899ac640 1 --2- 172.21.15.28:0/1750143525 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3b70067190 0x7f3b70069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:55.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.694+0000 7f3b7a7fc640 1 --2- 172.21.15.28:0/1750143525 >> [v2:172.21.15.148:6816/1578977000,v1:172.21.15.148:6817/1578977000] conn(0x7f3b7006cf10 0x7f3b7006f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:55.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.694+0000 7f3b7a7fc640 1 -- 172.21.15.28:0/1750143525 --> [v2:172.21.15.148:6816/1578977000,v1:172.21.15.148:6817/1578977000] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3b7006f9d0 con 0x7f3b7006cf10 2024-07-03T15:40:55.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.694+0000 7f3b7a7fc640 1 -- 172.21.15.28:0/1750143525 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b74086650 con 0x7f3b84102d40 2024-07-03T15:40:55.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.695+0000 7f3b8a1ad640 1 --2- 172.21.15.28:0/1750143525 >> [v2:172.21.15.148:6816/1578977000,v1:172.21.15.148:6817/1578977000] conn(0x7f3b7006cf10 0x7f3b7006f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:55.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.695+0000 7f3b899ac640 1 --2- 172.21.15.28:0/1750143525 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3b70067190 0x7f3b70069640 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3b80009f80 tx=0x7f3b80035040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:40:55.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.695+0000 7f3b8a1ad640 1 --2- 172.21.15.28:0/1750143525 >> [v2:172.21.15.148:6816/1578977000,v1:172.21.15.148:6817/1578977000] conn(0x7f3b7006cf10 0x7f3b7006f320 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:40:55.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.696+0000 7f3b7a7fc640 1 -- 172.21.15.28:0/1750143525 <== osd.5 v2:172.21.15.148:6816/1578977000 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f3b7006f9d0 con 0x7f3b7006cf10 2024-07-03T15:40:55.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.710+0000 7f3b8bc10640 1 -- 172.21.15.28:0/1750143525 --> [v2:172.21.15.148:6816/1578977000,v1:172.21.15.148:6817/1578977000] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3b840fdb10 con 0x7f3b7006cf10 2024-07-03T15:40:55.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.710+0000 7f3b7a7fc640 1 -- 172.21.15.28:0/1750143525 <== osd.5 v2:172.21.15.148:6816/1578977000 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3b840fdb10 con 0x7f3b7006cf10 2024-07-03T15:40:55.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.711+0000 7f3b8bc10640 1 -- 172.21.15.28:0/1750143525 >> [v2:172.21.15.148:6816/1578977000,v1:172.21.15.148:6817/1578977000] conn(0x7f3b7006cf10 msgr2=0x7f3b7006f320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:55.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.711+0000 7f3b8bc10640 1 --2- 172.21.15.28:0/1750143525 >> [v2:172.21.15.148:6816/1578977000,v1:172.21.15.148:6817/1578977000] conn(0x7f3b7006cf10 0x7f3b7006f320 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:55.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.711+0000 7f3b8bc10640 1 -- 172.21.15.28:0/1750143525 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3b70067190 msgr2=0x7f3b70069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:55.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.711+0000 7f3b8bc10640 1 --2- 172.21.15.28:0/1750143525 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3b70067190 0x7f3b70069640 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3b80009f80 tx=0x7f3b80035040 comp rx=0 tx=0).stop 2024-07-03T15:40:55.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.711+0000 7f3b8bc10640 1 -- 172.21.15.28:0/1750143525 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b84102d40 msgr2=0x7f3b8406fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:55.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.711+0000 7f3b8bc10640 1 --2- 172.21.15.28:0/1750143525 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b84102d40 0x7f3b8406fd60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3b74002a50 tx=0x7f3b74004600 comp rx=0 tx=0).stop 2024-07-03T15:40:55.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.711+0000 7f3b8bc10640 1 -- 172.21.15.28:0/1750143525 shutdown_connections 2024-07-03T15:40:55.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.711+0000 7f3b8bc10640 1 --2- 172.21.15.28:0/1750143525 >> [v2:172.21.15.148:6816/1578977000,v1:172.21.15.148:6817/1578977000] conn(0x7f3b7006cf10 0x7f3b7006f320 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:55.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.711+0000 7f3b8bc10640 1 --2- 172.21.15.28:0/1750143525 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3b70067190 0x7f3b70069640 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:55.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.711+0000 7f3b8bc10640 1 --2- 172.21.15.28:0/1750143525 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b84102d40 0x7f3b8406fd60 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:55.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.711+0000 7f3b8bc10640 1 --2- 172.21.15.28:0/1750143525 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b841023a0 0x7f3b8406f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:55.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.711+0000 7f3b8bc10640 1 -- 172.21.15.28:0/1750143525 >> 172.21.15.28:0/1750143525 conn(0x7f3b840f7fb0 msgr2=0x7f3b840f8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:40:55.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.711+0000 7f3b8bc10640 1 -- 172.21.15.28:0/1750143525 shutdown_connections 2024-07-03T15:40:55.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:55.711+0000 7f3b8bc10640 1 -- 172.21.15.28:0/1750143525 wait complete. 2024-07-03T15:40:55.820 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:55 smithi028 ceph-mon[29148]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:40:55.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:55 smithi148 ceph-mon[36056]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:40:56.167 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:40:56.167 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:40:57.724 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:57 smithi028 ceph-mon[29148]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:40:57.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:57 smithi148 ceph-mon[36056]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:40:58.048 INFO:teuthology.orchestra.run.smithi028.stdout:167503724554 2024-07-03T15:40:58.048 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-07-03T15:40:58.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.331+0000 7f151e09d640 1 -- 172.21.15.28:0/695403966 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1518102350 msgr2=0x7f1518102720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:58.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.331+0000 7f151e09d640 1 --2- 172.21.15.28:0/695403966 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1518102350 0x7f1518102720 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f15080048d0 tx=0x7f1508009da0 comp rx=0 tx=0).stop 2024-07-03T15:40:58.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.331+0000 7f151e09d640 1 -- 172.21.15.28:0/695403966 shutdown_connections 2024-07-03T15:40:58.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.332+0000 7f151e09d640 1 --2- 172.21.15.28:0/695403966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1518102cf0 0x7f15180fc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:58.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.332+0000 7f151e09d640 1 --2- 172.21.15.28:0/695403966 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1518102350 0x7f1518102720 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:58.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.332+0000 7f151e09d640 1 -- 172.21.15.28:0/695403966 >> 172.21.15.28:0/695403966 conn(0x7f15180f7f00 msgr2=0x7f15180fa310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:40:58.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.332+0000 7f151e09d640 1 -- 172.21.15.28:0/695403966 shutdown_connections 2024-07-03T15:40:58.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.332+0000 7f151e09d640 1 -- 172.21.15.28:0/695403966 wait complete. 2024-07-03T15:40:58.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.333+0000 7f151e09d640 1 Processor -- start 2024-07-03T15:40:58.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.333+0000 7f151e09d640 1 -- start start 2024-07-03T15:40:58.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.333+0000 7f151e09d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1518102350 0x7f151806f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:58.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.333+0000 7f151e09d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1518102cf0 0x7f151806fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:58.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.333+0000 7f151e09d640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15180702a0 con 0x7f1518102cf0 2024-07-03T15:40:58.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.333+0000 7f151e09d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1518070410 con 0x7f1518102350 2024-07-03T15:40:58.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.333+0000 7f1516ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1518102cf0 0x7f151806fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:58.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.334+0000 7f1516ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1518102cf0 0x7f151806fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:57000/0 (socket says 172.21.15.28:57000) 2024-07-03T15:40:58.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.334+0000 7f1516ffd640 1 -- 172.21.15.28:0/1687388670 learned_addr learned my addr 172.21.15.28:0/1687388670 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:40:58.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.334+0000 7f15177fe640 1 --2- 172.21.15.28:0/1687388670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1518102350 0x7f151806f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:58.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.334+0000 7f1516ffd640 1 -- 172.21.15.28:0/1687388670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1518102350 msgr2=0x7f151806f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:58.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.334+0000 7f1516ffd640 1 --2- 172.21.15.28:0/1687388670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1518102350 0x7f151806f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:58.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.334+0000 7f1516ffd640 1 -- 172.21.15.28:0/1687388670 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1508005040 con 0x7f1518102cf0 2024-07-03T15:40:58.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.334+0000 7f1516ffd640 1 --2- 172.21.15.28:0/1687388670 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1518102cf0 0x7f151806fd60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f150c00eb70 tx=0x7f150c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:40:58.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.334+0000 7f14fffff640 1 -- 172.21.15.28:0/1687388670 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f150c007c30 con 0x7f1518102cf0 2024-07-03T15:40:58.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.334+0000 7f14fffff640 1 -- 172.21.15.28:0/1687388670 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f150c00f040 con 0x7f1518102cf0 2024-07-03T15:40:58.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.334+0000 7f151e09d640 1 -- 172.21.15.28:0/1687388670 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1518070640 con 0x7f1518102cf0 2024-07-03T15:40:58.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.334+0000 7f14fffff640 1 -- 172.21.15.28:0/1687388670 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f150c00cad0 con 0x7f1518102cf0 2024-07-03T15:40:58.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.334+0000 7f151e09d640 1 -- 172.21.15.28:0/1687388670 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1518119780 con 0x7f1518102cf0 2024-07-03T15:40:58.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.335+0000 7f151e09d640 1 -- 172.21.15.28:0/1687388670 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f15180f98d0 con 0x7f1518102cf0 2024-07-03T15:40:58.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.336+0000 7f14fffff640 1 -- 172.21.15.28:0/1687388670 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f150c00cc50 con 0x7f1518102cf0 2024-07-03T15:40:58.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.336+0000 7f14fffff640 1 --2- 172.21.15.28:0/1687388670 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f1500067190 0x7f1500069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:58.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.336+0000 7f14fffff640 1 -- 172.21.15.28:0/1687388670 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f150c084410 con 0x7f1518102cf0 2024-07-03T15:40:58.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.336+0000 7f15177fe640 1 --2- 172.21.15.28:0/1687388670 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f1500067190 0x7f1500069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:58.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.336+0000 7f14fffff640 1 --2- 172.21.15.28:0/1687388670 >> [v2:172.21.15.148:6808/1943011533,v1:172.21.15.148:6809/1943011533] conn(0x7f150006cf10 0x7f150006f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:58.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.336+0000 7f14fffff640 1 -- 172.21.15.28:0/1687388670 --> [v2:172.21.15.148:6808/1943011533,v1:172.21.15.148:6809/1943011533] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f150006f9d0 con 0x7f150006cf10 2024-07-03T15:40:58.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.336+0000 7f14fffff640 1 -- 172.21.15.28:0/1687388670 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f150c086a60 con 0x7f1518102cf0 2024-07-03T15:40:58.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.337+0000 7f1517fff640 1 --2- 172.21.15.28:0/1687388670 >> [v2:172.21.15.148:6808/1943011533,v1:172.21.15.148:6809/1943011533] conn(0x7f150006cf10 0x7f150006f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:58.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.337+0000 7f15177fe640 1 --2- 172.21.15.28:0/1687388670 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f1500067190 0x7f1500069640 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f15080048a0 tx=0x7f1508035040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:40:58.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.337+0000 7f1517fff640 1 --2- 172.21.15.28:0/1687388670 >> [v2:172.21.15.148:6808/1943011533,v1:172.21.15.148:6809/1943011533] conn(0x7f150006cf10 0x7f150006f320 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:40:58.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.338+0000 7f14fffff640 1 -- 172.21.15.28:0/1687388670 <== osd.4 v2:172.21.15.148:6808/1943011533 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f150006f9d0 con 0x7f150006cf10 2024-07-03T15:40:58.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.354+0000 7f151e09d640 1 -- 172.21.15.28:0/1687388670 --> [v2:172.21.15.148:6808/1943011533,v1:172.21.15.148:6809/1943011533] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f15180fdac0 con 0x7f150006cf10 2024-07-03T15:40:58.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.355+0000 7f14fffff640 1 -- 172.21.15.28:0/1687388670 <== osd.4 v2:172.21.15.148:6808/1943011533 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f15180fdac0 con 0x7f150006cf10 2024-07-03T15:40:58.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.355+0000 7f151e09d640 1 -- 172.21.15.28:0/1687388670 >> [v2:172.21.15.148:6808/1943011533,v1:172.21.15.148:6809/1943011533] conn(0x7f150006cf10 msgr2=0x7f150006f320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:58.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.355+0000 7f151e09d640 1 --2- 172.21.15.28:0/1687388670 >> [v2:172.21.15.148:6808/1943011533,v1:172.21.15.148:6809/1943011533] conn(0x7f150006cf10 0x7f150006f320 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:58.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.355+0000 7f151e09d640 1 -- 172.21.15.28:0/1687388670 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f1500067190 msgr2=0x7f1500069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:58.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.355+0000 7f151e09d640 1 --2- 172.21.15.28:0/1687388670 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f1500067190 0x7f1500069640 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f15080048a0 tx=0x7f1508035040 comp rx=0 tx=0).stop 2024-07-03T15:40:58.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.355+0000 7f151e09d640 1 -- 172.21.15.28:0/1687388670 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1518102cf0 msgr2=0x7f151806fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:58.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.355+0000 7f151e09d640 1 --2- 172.21.15.28:0/1687388670 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1518102cf0 0x7f151806fd60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f150c00eb70 tx=0x7f150c00b040 comp rx=0 tx=0).stop 2024-07-03T15:40:58.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.355+0000 7f151e09d640 1 -- 172.21.15.28:0/1687388670 shutdown_connections 2024-07-03T15:40:58.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.355+0000 7f151e09d640 1 --2- 172.21.15.28:0/1687388670 >> [v2:172.21.15.148:6808/1943011533,v1:172.21.15.148:6809/1943011533] conn(0x7f150006cf10 0x7f150006f320 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:58.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.355+0000 7f151e09d640 1 --2- 172.21.15.28:0/1687388670 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f1500067190 0x7f1500069640 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:58.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.355+0000 7f151e09d640 1 --2- 172.21.15.28:0/1687388670 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1518102cf0 0x7f151806fd60 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:58.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.355+0000 7f151e09d640 1 --2- 172.21.15.28:0/1687388670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1518102350 0x7f151806f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:58.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.355+0000 7f151e09d640 1 -- 172.21.15.28:0/1687388670 >> 172.21.15.28:0/1687388670 conn(0x7f15180f7f00 msgr2=0x7f15180f8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:40:58.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.356+0000 7f151e09d640 1 -- 172.21.15.28:0/1687388670 shutdown_connections 2024-07-03T15:40:58.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:58.356+0000 7f151e09d640 1 -- 172.21.15.28:0/1687388670 wait complete. 2024-07-03T15:40:59.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.577+0000 7f88c3476640 1 -- 172.21.15.28:0/4125494103 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f88bc0fdd50 msgr2=0x7f88bc106390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:59.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.577+0000 7f88c3476640 1 --2- 172.21.15.28:0/4125494103 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f88bc0fdd50 0x7f88bc106390 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f88ac004900 tx=0x7f88ac009da0 comp rx=0 tx=0).stop 2024-07-03T15:40:59.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.577+0000 7f88c3476640 1 -- 172.21.15.28:0/4125494103 shutdown_connections 2024-07-03T15:40:59.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.577+0000 7f88c3476640 1 --2- 172.21.15.28:0/4125494103 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f88bc0fdd50 0x7f88bc106390 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:59.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.577+0000 7f88c3476640 1 --2- 172.21.15.28:0/4125494103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88bc0fd3b0 0x7f88bc0fd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:59.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.577+0000 7f88c3476640 1 -- 172.21.15.28:0/4125494103 >> 172.21.15.28:0/4125494103 conn(0x7f88bc0f6e40 msgr2=0x7f88bc0f9250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:40:59.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.577+0000 7f88c3476640 1 -- 172.21.15.28:0/4125494103 shutdown_connections 2024-07-03T15:40:59.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.577+0000 7f88c3476640 1 -- 172.21.15.28:0/4125494103 wait complete. 2024-07-03T15:40:59.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.578+0000 7f88c3476640 1 Processor -- start 2024-07-03T15:40:59.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.578+0000 7f88c3476640 1 -- start start 2024-07-03T15:40:59.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.578+0000 7f88c3476640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f88bc0fd3b0 0x7f88bc113160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:59.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.578+0000 7f88c3476640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88bc0fdd50 0x7f88bc1136a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:59.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.578+0000 7f88c3476640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc113ca0 con 0x7f88bc0fd3b0 2024-07-03T15:40:59.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.578+0000 7f88c3476640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc10d0e0 con 0x7f88bc0fdd50 2024-07-03T15:40:59.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.578+0000 7f88c1212640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f88bc0fd3b0 0x7f88bc113160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:59.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.578+0000 7f88c1212640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f88bc0fd3b0 0x7f88bc113160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:57016/0 (socket says 172.21.15.28:57016) 2024-07-03T15:40:59.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.578+0000 7f88c1212640 1 -- 172.21.15.28:0/3759927529 learned_addr learned my addr 172.21.15.28:0/3759927529 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:40:59.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.578+0000 7f88c0a11640 1 --2- 172.21.15.28:0/3759927529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88bc0fdd50 0x7f88bc1136a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:59.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.578+0000 7f88c1212640 1 -- 172.21.15.28:0/3759927529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88bc0fdd50 msgr2=0x7f88bc1136a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:59.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.578+0000 7f88c1212640 1 --2- 172.21.15.28:0/3759927529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88bc0fdd50 0x7f88bc1136a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:59.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.578+0000 7f88c1212640 1 -- 172.21.15.28:0/3759927529 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88ac005040 con 0x7f88bc0fd3b0 2024-07-03T15:40:59.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.579+0000 7f88c1212640 1 --2- 172.21.15.28:0/3759927529 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f88bc0fd3b0 0x7f88bc113160 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f88b8002a50 tx=0x7f88b8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:40:59.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.579+0000 7f88b1ffb640 1 -- 172.21.15.28:0/3759927529 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88b8007c30 con 0x7f88bc0fd3b0 2024-07-03T15:40:59.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.579+0000 7f88b1ffb640 1 -- 172.21.15.28:0/3759927529 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f88b800f040 con 0x7f88bc0fd3b0 2024-07-03T15:40:59.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.579+0000 7f88b1ffb640 1 -- 172.21.15.28:0/3759927529 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88b800db50 con 0x7f88bc0fd3b0 2024-07-03T15:40:59.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.579+0000 7f88c3476640 1 -- 172.21.15.28:0/3759927529 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88bc10d3c0 con 0x7f88bc0fd3b0 2024-07-03T15:40:59.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.579+0000 7f88c3476640 1 -- 172.21.15.28:0/3759927529 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88bc10d8e0 con 0x7f88bc0fd3b0 2024-07-03T15:40:59.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.581+0000 7f88c3476640 1 -- 172.21.15.28:0/3759927529 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88bc065ec0 con 0x7f88bc0fd3b0 2024-07-03T15:40:59.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.581+0000 7f88b1ffb640 1 -- 172.21.15.28:0/3759927529 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f88b801c400 con 0x7f88bc0fd3b0 2024-07-03T15:40:59.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.582+0000 7f88b1ffb640 1 --2- 172.21.15.28:0/3759927529 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f88a8067190 0x7f88a8069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:40:59.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.582+0000 7f88b1ffb640 1 -- 172.21.15.28:0/3759927529 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f88b80840a0 con 0x7f88bc0fd3b0 2024-07-03T15:40:59.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.582+0000 7f88c0a11640 1 --2- 172.21.15.28:0/3759927529 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f88a8067190 0x7f88a8069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:40:59.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.583+0000 7f88c0a11640 1 --2- 172.21.15.28:0/3759927529 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f88a8067190 0x7f88a8069640 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f88ac000c00 tx=0x7f88ac009da0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:40:59.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.586+0000 7f88b1ffb640 1 -- 172.21.15.28:0/3759927529 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f88b804d250 con 0x7f88bc0fd3b0 2024-07-03T15:40:59.591 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:40:59.591 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:40:59.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:40:59 smithi028 ceph-mon[29148]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:40:59.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.768+0000 7f88c3476640 1 -- 172.21.15.28:0/3759927529 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f88bc04b530 con 0x7f88bc0fd3b0 2024-07-03T15:40:59.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.769+0000 7f88b1ffb640 1 -- 172.21.15.28:0/3759927529 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f88b8051680 con 0x7f88bc0fd3b0 2024-07-03T15:40:59.772 INFO:teuthology.orchestra.run.smithi028.stdout:38654705700 2024-07-03T15:40:59.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.771+0000 7f88c3476640 1 -- 172.21.15.28:0/3759927529 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f88a8067190 msgr2=0x7f88a8069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:59.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.771+0000 7f88c3476640 1 --2- 172.21.15.28:0/3759927529 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f88a8067190 0x7f88a8069640 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f88ac000c00 tx=0x7f88ac009da0 comp rx=0 tx=0).stop 2024-07-03T15:40:59.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.771+0000 7f88c3476640 1 -- 172.21.15.28:0/3759927529 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f88bc0fd3b0 msgr2=0x7f88bc113160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:40:59.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.771+0000 7f88c3476640 1 --2- 172.21.15.28:0/3759927529 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f88bc0fd3b0 0x7f88bc113160 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f88b8002a50 tx=0x7f88b8004600 comp rx=0 tx=0).stop 2024-07-03T15:40:59.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.771+0000 7f88c3476640 1 -- 172.21.15.28:0/3759927529 shutdown_connections 2024-07-03T15:40:59.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.771+0000 7f88c3476640 1 --2- 172.21.15.28:0/3759927529 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f88a8067190 0x7f88a8069640 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:59.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.771+0000 7f88c3476640 1 --2- 172.21.15.28:0/3759927529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88bc0fdd50 0x7f88bc1136a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:59.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.771+0000 7f88c3476640 1 --2- 172.21.15.28:0/3759927529 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f88bc0fd3b0 0x7f88bc113160 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:40:59.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.771+0000 7f88c3476640 1 -- 172.21.15.28:0/3759927529 >> 172.21.15.28:0/3759927529 conn(0x7f88bc0f6e40 msgr2=0x7f88bc0f8410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:40:59.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.771+0000 7f88c3476640 1 -- 172.21.15.28:0/3759927529 shutdown_connections 2024-07-03T15:40:59.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:40:59.771+0000 7f88c3476640 1 -- 172.21.15.28:0/3759927529 wait complete. 2024-07-03T15:40:59.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:40:59 smithi148 ceph-mon[36056]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:00.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:00 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/3759927529' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-07-03T15:41:00.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:00 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:41:00.947 INFO:teuthology.orchestra.run.smithi028.stdout:146028888079 2024-07-03T15:41:00.947 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-07-03T15:41:00.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:00 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/3759927529' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-07-03T15:41:00.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:00 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:41:01.850 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:01 smithi028 ceph-mon[29148]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:01.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:01 smithi148 ceph-mon[36056]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:02.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.317+0000 7f90fc3d1640 1 -- 172.21.15.28:0/3722190879 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f90f4101c00 msgr2=0x7f90f40fb290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:02.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.317+0000 7f90fc3d1640 1 --2- 172.21.15.28:0/3722190879 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f90f4101c00 0x7f90f40fb290 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f90e40048d0 tx=0x7f90e4009da0 comp rx=0 tx=0).stop 2024-07-03T15:41:02.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.317+0000 7f90fc3d1640 1 -- 172.21.15.28:0/3722190879 shutdown_connections 2024-07-03T15:41:02.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.317+0000 7f90fc3d1640 1 --2- 172.21.15.28:0/3722190879 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f90f4101c00 0x7f90f40fb290 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:02.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.317+0000 7f90fc3d1640 1 --2- 172.21.15.28:0/3722190879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90f4101260 0x7f90f4101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:02.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.317+0000 7f90fc3d1640 1 -- 172.21.15.28:0/3722190879 >> 172.21.15.28:0/3722190879 conn(0x7f90f40f6e10 msgr2=0x7f90f40f9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:41:02.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.317+0000 7f90fc3d1640 1 -- 172.21.15.28:0/3722190879 shutdown_connections 2024-07-03T15:41:02.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.317+0000 7f90fc3d1640 1 -- 172.21.15.28:0/3722190879 wait complete. 2024-07-03T15:41:02.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.318+0000 7f90fc3d1640 1 Processor -- start 2024-07-03T15:41:02.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.318+0000 7f90fc3d1640 1 -- start start 2024-07-03T15:41:02.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.318+0000 7f90fc3d1640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f90f4101260 0x7f90f4113180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:02.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.318+0000 7f90fc3d1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90f4101c00 0x7f90f41136c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:02.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.318+0000 7f90fc3d1640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f4113d50 con 0x7f90f4101260 2024-07-03T15:41:02.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.318+0000 7f90fc3d1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f410d100 con 0x7f90f4101c00 2024-07-03T15:41:02.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.318+0000 7f90fa16d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f90f4101260 0x7f90f4113180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:02.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.318+0000 7f90fa16d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f90f4101260 0x7f90f4113180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35534/0 (socket says 172.21.15.28:35534) 2024-07-03T15:41:02.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.318+0000 7f90fa16d640 1 -- 172.21.15.28:0/2810033426 learned_addr learned my addr 172.21.15.28:0/2810033426 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:41:02.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.319+0000 7f90f996c640 1 --2- 172.21.15.28:0/2810033426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90f4101c00 0x7f90f41136c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:02.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.319+0000 7f90fa16d640 1 -- 172.21.15.28:0/2810033426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90f4101c00 msgr2=0x7f90f41136c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:02.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.319+0000 7f90fa16d640 1 --2- 172.21.15.28:0/2810033426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90f4101c00 0x7f90f41136c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:02.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.319+0000 7f90fa16d640 1 -- 172.21.15.28:0/2810033426 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e4005040 con 0x7f90f4101260 2024-07-03T15:41:02.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.319+0000 7f90f996c640 1 --2- 172.21.15.28:0/2810033426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90f4101c00 0x7f90f41136c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-03T15:41:02.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.319+0000 7f90fa16d640 1 --2- 172.21.15.28:0/2810033426 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f90f4101260 0x7f90f4113180 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f90f000b790 tx=0x7f90f00091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:41:02.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.319+0000 7f90eaffd640 1 -- 172.21.15.28:0/2810033426 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90f000c700 con 0x7f90f4101260 2024-07-03T15:41:02.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.319+0000 7f90eaffd640 1 -- 172.21.15.28:0/2810033426 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f90f000cd30 con 0x7f90f4101260 2024-07-03T15:41:02.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.319+0000 7f90fc3d1640 1 -- 172.21.15.28:0/2810033426 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90f410d3e0 con 0x7f90f4101260 2024-07-03T15:41:02.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.319+0000 7f90eaffd640 1 -- 172.21.15.28:0/2810033426 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90f0013370 con 0x7f90f4101260 2024-07-03T15:41:02.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.319+0000 7f90fc3d1640 1 -- 172.21.15.28:0/2810033426 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90f410d900 con 0x7f90f4101260 2024-07-03T15:41:02.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.320+0000 7f90fc3d1640 1 -- 172.21.15.28:0/2810033426 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90f40fc9d0 con 0x7f90f4101260 2024-07-03T15:41:02.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.321+0000 7f90eaffd640 1 -- 172.21.15.28:0/2810033426 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f90f0012070 con 0x7f90f4101260 2024-07-03T15:41:02.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.321+0000 7f90eaffd640 1 --2- 172.21.15.28:0/2810033426 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f90e0067190 0x7f90e0069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:02.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.321+0000 7f90eaffd640 1 -- 172.21.15.28:0/2810033426 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f90f00843d0 con 0x7f90f4101260 2024-07-03T15:41:02.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.321+0000 7f90f996c640 1 --2- 172.21.15.28:0/2810033426 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f90e0067190 0x7f90e0069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:02.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.322+0000 7f90f996c640 1 --2- 172.21.15.28:0/2810033426 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f90e0067190 0x7f90e0069640 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f90e4009f80 tx=0x7f90e4009da0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:41:02.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.325+0000 7f90eaffd640 1 -- 172.21.15.28:0/2810033426 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f90f00519b0 con 0x7f90f4101260 2024-07-03T15:41:02.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.503+0000 7f90fc3d1640 1 -- 172.21.15.28:0/2810033426 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f90f404b590 con 0x7f90f4101260 2024-07-03T15:41:02.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.503+0000 7f90eaffd640 1 -- 172.21.15.28:0/2810033426 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f90f0054fd0 con 0x7f90f4101260 2024-07-03T15:41:02.507 INFO:teuthology.orchestra.run.smithi028.stdout:81604378649 2024-07-03T15:41:02.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.505+0000 7f90fc3d1640 1 -- 172.21.15.28:0/2810033426 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f90e0067190 msgr2=0x7f90e0069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:02.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.505+0000 7f90fc3d1640 1 --2- 172.21.15.28:0/2810033426 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f90e0067190 0x7f90e0069640 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f90e4009f80 tx=0x7f90e4009da0 comp rx=0 tx=0).stop 2024-07-03T15:41:02.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.505+0000 7f90fc3d1640 1 -- 172.21.15.28:0/2810033426 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f90f4101260 msgr2=0x7f90f4113180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:02.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.505+0000 7f90fc3d1640 1 --2- 172.21.15.28:0/2810033426 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f90f4101260 0x7f90f4113180 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f90f000b790 tx=0x7f90f00091d0 comp rx=0 tx=0).stop 2024-07-03T15:41:02.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.505+0000 7f90fc3d1640 1 -- 172.21.15.28:0/2810033426 shutdown_connections 2024-07-03T15:41:02.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.505+0000 7f90fc3d1640 1 --2- 172.21.15.28:0/2810033426 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f90e0067190 0x7f90e0069640 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:02.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.505+0000 7f90fc3d1640 1 --2- 172.21.15.28:0/2810033426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90f4101c00 0x7f90f41136c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:02.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.505+0000 7f90fc3d1640 1 --2- 172.21.15.28:0/2810033426 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f90f4101260 0x7f90f4113180 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:02.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.505+0000 7f90fc3d1640 1 -- 172.21.15.28:0/2810033426 >> 172.21.15.28:0/2810033426 conn(0x7f90f40f6e10 msgr2=0x7f90f40f7dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:41:02.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.505+0000 7f90fc3d1640 1 -- 172.21.15.28:0/2810033426 shutdown_connections 2024-07-03T15:41:02.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:02.505+0000 7f90fc3d1640 1 -- 172.21.15.28:0/2810033426 wait complete. 2024-07-03T15:41:02.836 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:02 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/2810033426' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-07-03T15:41:02.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:02 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/2810033426' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-07-03T15:41:03.693 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:03 smithi028 ceph-mon[29148]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:03.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.771+0000 7f2b61b76640 1 -- 172.21.15.28:0/3346102843 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2b5c0fb620 msgr2=0x7f2b5c0fb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:03.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.771+0000 7f2b61b76640 1 --2- 172.21.15.28:0/3346102843 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2b5c0fb620 0x7f2b5c0fb9f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c009c50 tx=0x7f2b4c004d10 comp rx=0 tx=0).stop 2024-07-03T15:41:03.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.771+0000 7f2b61b76640 1 -- 172.21.15.28:0/3346102843 shutdown_connections 2024-07-03T15:41:03.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.771+0000 7f2b61b76640 1 --2- 172.21.15.28:0/3346102843 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b5c0fbf30 0x7f2b5c0fce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:03.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.771+0000 7f2b61b76640 1 --2- 172.21.15.28:0/3346102843 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2b5c0fb620 0x7f2b5c0fb9f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:03.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.771+0000 7f2b61b76640 1 -- 172.21.15.28:0/3346102843 >> 172.21.15.28:0/3346102843 conn(0x7f2b5c0721c0 msgr2=0x7f2b5c0725c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:41:03.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.771+0000 7f2b61b76640 1 -- 172.21.15.28:0/3346102843 shutdown_connections 2024-07-03T15:41:03.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.771+0000 7f2b61b76640 1 -- 172.21.15.28:0/3346102843 wait complete. 2024-07-03T15:41:03.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.772+0000 7f2b61b76640 1 Processor -- start 2024-07-03T15:41:03.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.772+0000 7f2b61b76640 1 -- start start 2024-07-03T15:41:03.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.772+0000 7f2b61b76640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b5c0fb620 0x7f2b5c1130d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:03.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.772+0000 7f2b61b76640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2b5c0fbf30 0x7f2b5c113610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:03.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.772+0000 7f2b61b76640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5c113c10 con 0x7f2b5c0fbf30 2024-07-03T15:41:03.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.772+0000 7f2b61b76640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5c10d050 con 0x7f2b5c0fb620 2024-07-03T15:41:03.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.772+0000 7f2b5affd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2b5c0fbf30 0x7f2b5c113610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:03.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.773+0000 7f2b5affd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2b5c0fbf30 0x7f2b5c113610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35542/0 (socket says 172.21.15.28:35542) 2024-07-03T15:41:03.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.773+0000 7f2b5affd640 1 -- 172.21.15.28:0/583086998 learned_addr learned my addr 172.21.15.28:0/583086998 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:41:03.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.773+0000 7f2b5b7fe640 1 --2- 172.21.15.28:0/583086998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b5c0fb620 0x7f2b5c1130d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:03.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.773+0000 7f2b5affd640 1 -- 172.21.15.28:0/583086998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b5c0fb620 msgr2=0x7f2b5c1130d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:03.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.773+0000 7f2b5affd640 1 --2- 172.21.15.28:0/583086998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b5c0fb620 0x7f2b5c1130d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:03.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.773+0000 7f2b5affd640 1 -- 172.21.15.28:0/583086998 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b4c0098b0 con 0x7f2b5c0fbf30 2024-07-03T15:41:03.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.773+0000 7f2b5affd640 1 --2- 172.21.15.28:0/583086998 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2b5c0fbf30 0x7f2b5c113610 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2b50002a50 tx=0x7f2b50004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:41:03.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.773+0000 7f2b43fff640 1 -- 172.21.15.28:0/583086998 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b50007c30 con 0x7f2b5c0fbf30 2024-07-03T15:41:03.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.773+0000 7f2b43fff640 1 -- 172.21.15.28:0/583086998 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2b5000f040 con 0x7f2b5c0fbf30 2024-07-03T15:41:03.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.773+0000 7f2b61b76640 1 -- 172.21.15.28:0/583086998 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b5c10d330 con 0x7f2b5c0fbf30 2024-07-03T15:41:03.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.773+0000 7f2b43fff640 1 -- 172.21.15.28:0/583086998 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b5000db50 con 0x7f2b5c0fbf30 2024-07-03T15:41:03.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.773+0000 7f2b61b76640 1 -- 172.21.15.28:0/583086998 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b5c10d850 con 0x7f2b5c0fbf30 2024-07-03T15:41:03.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.774+0000 7f2b61b76640 1 -- 172.21.15.28:0/583086998 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b5c0fc9d0 con 0x7f2b5c0fbf30 2024-07-03T15:41:03.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.775+0000 7f2b43fff640 1 -- 172.21.15.28:0/583086998 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f2b5001c400 con 0x7f2b5c0fbf30 2024-07-03T15:41:03.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.775+0000 7f2b43fff640 1 --2- 172.21.15.28:0/583086998 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f2b44067190 0x7f2b44069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:03.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.775+0000 7f2b43fff640 1 -- 172.21.15.28:0/583086998 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f2b50084120 con 0x7f2b5c0fbf30 2024-07-03T15:41:03.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.775+0000 7f2b5b7fe640 1 --2- 172.21.15.28:0/583086998 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f2b44067190 0x7f2b44069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:03.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.776+0000 7f2b5b7fe640 1 --2- 172.21.15.28:0/583086998 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f2b44067190 0x7f2b44069640 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c004f50 tx=0x7f2b4c004d10 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:41:03.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.779+0000 7f2b43fff640 1 -- 172.21.15.28:0/583086998 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2b50051700 con 0x7f2b5c0fbf30 2024-07-03T15:41:03.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.959+0000 7f2b61b76640 1 -- 172.21.15.28:0/583086998 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2b5c10e740 con 0x7f2b5c0fbf30 2024-07-03T15:41:03.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.959+0000 7f2b43fff640 1 -- 172.21.15.28:0/583086998 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2b50054d20 con 0x7f2b5c0fbf30 2024-07-03T15:41:03.963 INFO:teuthology.orchestra.run.smithi028.stdout:60129542176 2024-07-03T15:41:03.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.961+0000 7f2b61b76640 1 -- 172.21.15.28:0/583086998 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f2b44067190 msgr2=0x7f2b44069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:03.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.961+0000 7f2b61b76640 1 --2- 172.21.15.28:0/583086998 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f2b44067190 0x7f2b44069640 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c004f50 tx=0x7f2b4c004d10 comp rx=0 tx=0).stop 2024-07-03T15:41:03.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.961+0000 7f2b61b76640 1 -- 172.21.15.28:0/583086998 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2b5c0fbf30 msgr2=0x7f2b5c113610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:03.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.961+0000 7f2b61b76640 1 --2- 172.21.15.28:0/583086998 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2b5c0fbf30 0x7f2b5c113610 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2b50002a50 tx=0x7f2b50004600 comp rx=0 tx=0).stop 2024-07-03T15:41:03.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.961+0000 7f2b61b76640 1 -- 172.21.15.28:0/583086998 shutdown_connections 2024-07-03T15:41:03.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.961+0000 7f2b61b76640 1 --2- 172.21.15.28:0/583086998 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f2b44067190 0x7f2b44069640 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:03.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.961+0000 7f2b61b76640 1 --2- 172.21.15.28:0/583086998 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2b5c0fbf30 0x7f2b5c113610 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:03.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.961+0000 7f2b61b76640 1 --2- 172.21.15.28:0/583086998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b5c0fb620 0x7f2b5c1130d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:03.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.961+0000 7f2b61b76640 1 -- 172.21.15.28:0/583086998 >> 172.21.15.28:0/583086998 conn(0x7f2b5c0721c0 msgr2=0x7f2b5c0fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:41:03.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.961+0000 7f2b61b76640 1 -- 172.21.15.28:0/583086998 shutdown_connections 2024-07-03T15:41:03.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:03.961+0000 7f2b61b76640 1 -- 172.21.15.28:0/583086998 wait complete. 2024-07-03T15:41:03.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:03 smithi148 ceph-mon[36056]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:04.313 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705699 got 38654705700 for osd.0 2024-07-03T15:41:04.313 DEBUG:teuthology.parallel:result is None 2024-07-03T15:41:04.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:04 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/583086998' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-07-03T15:41:04.883 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:04 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/583086998' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-07-03T15:41:05.314 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542173 got 60129542176 for osd.1 2024-07-03T15:41:05.314 DEBUG:teuthology.parallel:result is None 2024-07-03T15:41:05.847 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:05 smithi028 ceph-mon[29148]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:05.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:05 smithi148 ceph-mon[36056]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:06.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.531+0000 7f63dfd31640 1 -- 172.21.15.28:0/3248329924 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f63d80650e0 msgr2=0x7f63d80654b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:06.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.531+0000 7f63dfd31640 1 --2- 172.21.15.28:0/3248329924 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f63d80650e0 0x7f63d80654b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f63d40048d0 tx=0x7f63d4009da0 comp rx=0 tx=0).stop 2024-07-03T15:41:06.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.531+0000 7f63dfd31640 1 -- 172.21.15.28:0/3248329924 shutdown_connections 2024-07-03T15:41:06.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.531+0000 7f63dfd31640 1 --2- 172.21.15.28:0/3248329924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63d80fdf50 0x7f63d80fe3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:06.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.531+0000 7f63dfd31640 1 --2- 172.21.15.28:0/3248329924 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f63d80650e0 0x7f63d80654b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:06.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.531+0000 7f63dfd31640 1 -- 172.21.15.28:0/3248329924 >> 172.21.15.28:0/3248329924 conn(0x7f63d80721c0 msgr2=0x7f63d80725c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:41:06.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.531+0000 7f63dfd31640 1 -- 172.21.15.28:0/3248329924 shutdown_connections 2024-07-03T15:41:06.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.531+0000 7f63dfd31640 1 -- 172.21.15.28:0/3248329924 wait complete. 2024-07-03T15:41:06.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.532+0000 7f63dfd31640 1 Processor -- start 2024-07-03T15:41:06.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.532+0000 7f63dfd31640 1 -- start start 2024-07-03T15:41:06.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.532+0000 7f63dfd31640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f63d80650e0 0x7f63d8115320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:06.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.532+0000 7f63dfd31640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63d80fdf50 0x7f63d8115860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:06.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.532+0000 7f63dfd31640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d8115e60 con 0x7f63d80650e0 2024-07-03T15:41:06.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.532+0000 7f63dfd31640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d810f2a0 con 0x7f63d80fdf50 2024-07-03T15:41:06.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.532+0000 7f63ddacd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f63d80650e0 0x7f63d8115320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:06.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.532+0000 7f63ddacd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f63d80650e0 0x7f63d8115320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35560/0 (socket says 172.21.15.28:35560) 2024-07-03T15:41:06.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.532+0000 7f63ddacd640 1 -- 172.21.15.28:0/1091199270 learned_addr learned my addr 172.21.15.28:0/1091199270 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:41:06.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.533+0000 7f63dd2cc640 1 --2- 172.21.15.28:0/1091199270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63d80fdf50 0x7f63d8115860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:06.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.533+0000 7f63ddacd640 1 -- 172.21.15.28:0/1091199270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63d80fdf50 msgr2=0x7f63d8115860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:06.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.533+0000 7f63ddacd640 1 --2- 172.21.15.28:0/1091199270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63d80fdf50 0x7f63d8115860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:06.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.533+0000 7f63ddacd640 1 -- 172.21.15.28:0/1091199270 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d4005040 con 0x7f63d80650e0 2024-07-03T15:41:06.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.533+0000 7f63dd2cc640 1 --2- 172.21.15.28:0/1091199270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63d80fdf50 0x7f63d8115860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-03T15:41:06.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.533+0000 7f63ddacd640 1 --2- 172.21.15.28:0/1091199270 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f63d80650e0 0x7f63d8115320 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f63d40043d0 tx=0x7f63d4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:41:06.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.533+0000 7f63ce7fc640 1 -- 172.21.15.28:0/1091199270 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63d4038070 con 0x7f63d80650e0 2024-07-03T15:41:06.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.533+0000 7f63ce7fc640 1 -- 172.21.15.28:0/1091199270 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f63d4032970 con 0x7f63d80650e0 2024-07-03T15:41:06.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.533+0000 7f63dfd31640 1 -- 172.21.15.28:0/1091199270 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63d810f520 con 0x7f63d80650e0 2024-07-03T15:41:06.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.533+0000 7f63ce7fc640 1 -- 172.21.15.28:0/1091199270 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63d403da60 con 0x7f63d80650e0 2024-07-03T15:41:06.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.533+0000 7f63dfd31640 1 -- 172.21.15.28:0/1091199270 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d810f9e0 con 0x7f63d80650e0 2024-07-03T15:41:06.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.534+0000 7f63dfd31640 1 -- 172.21.15.28:0/1091199270 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63d80fdaf0 con 0x7f63d80650e0 2024-07-03T15:41:06.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.535+0000 7f63ce7fc640 1 -- 172.21.15.28:0/1091199270 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f63d4030020 con 0x7f63d80650e0 2024-07-03T15:41:06.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.535+0000 7f63ce7fc640 1 --2- 172.21.15.28:0/1091199270 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f63c4067190 0x7f63c4069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:06.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.535+0000 7f63ce7fc640 1 -- 172.21.15.28:0/1091199270 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f63d40a48a0 con 0x7f63d80650e0 2024-07-03T15:41:06.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.535+0000 7f63dd2cc640 1 --2- 172.21.15.28:0/1091199270 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f63c4067190 0x7f63c4069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:06.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.536+0000 7f63dd2cc640 1 --2- 172.21.15.28:0/1091199270 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f63c4067190 0x7f63c4069640 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f63d8110be0 tx=0x7f63c800b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:41:06.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.539+0000 7f63ce7fc640 1 -- 172.21.15.28:0/1091199270 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f63d4071e80 con 0x7f63d80650e0 2024-07-03T15:41:06.682 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378647 got 81604378649 for osd.2 2024-07-03T15:41:06.683 DEBUG:teuthology.parallel:result is None 2024-07-03T15:41:06.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.717+0000 7f63dfd31640 1 -- 172.21.15.28:0/1091199270 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f63d804b530 con 0x7f63d80650e0 2024-07-03T15:41:06.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.717+0000 7f63ce7fc640 1 -- 172.21.15.28:0/1091199270 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f63d40754a0 con 0x7f63d80650e0 2024-07-03T15:41:06.722 INFO:teuthology.orchestra.run.smithi028.stdout:111669149717 2024-07-03T15:41:06.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.720+0000 7f63dfd31640 1 -- 172.21.15.28:0/1091199270 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f63c4067190 msgr2=0x7f63c4069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:06.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.720+0000 7f63dfd31640 1 --2- 172.21.15.28:0/1091199270 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f63c4067190 0x7f63c4069640 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f63d8110be0 tx=0x7f63c800b040 comp rx=0 tx=0).stop 2024-07-03T15:41:06.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.720+0000 7f63dfd31640 1 -- 172.21.15.28:0/1091199270 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f63d80650e0 msgr2=0x7f63d8115320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:06.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.720+0000 7f63dfd31640 1 --2- 172.21.15.28:0/1091199270 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f63d80650e0 0x7f63d8115320 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f63d40043d0 tx=0x7f63d4009da0 comp rx=0 tx=0).stop 2024-07-03T15:41:06.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.720+0000 7f63dfd31640 1 -- 172.21.15.28:0/1091199270 shutdown_connections 2024-07-03T15:41:06.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.720+0000 7f63dfd31640 1 --2- 172.21.15.28:0/1091199270 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f63c4067190 0x7f63c4069640 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:06.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.720+0000 7f63dfd31640 1 --2- 172.21.15.28:0/1091199270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63d80fdf50 0x7f63d8115860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:06.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.720+0000 7f63dfd31640 1 --2- 172.21.15.28:0/1091199270 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f63d80650e0 0x7f63d8115320 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:06.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.720+0000 7f63dfd31640 1 -- 172.21.15.28:0/1091199270 >> 172.21.15.28:0/1091199270 conn(0x7f63d80721c0 msgr2=0x7f63d81031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:41:06.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.720+0000 7f63dfd31640 1 -- 172.21.15.28:0/1091199270 shutdown_connections 2024-07-03T15:41:06.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:06.720+0000 7f63dfd31640 1 -- 172.21.15.28:0/1091199270 wait complete. 2024-07-03T15:41:06.883 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:41:07.612 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149713 got 111669149717 for osd.3 2024-07-03T15:41:07.612 DEBUG:teuthology.parallel:result is None 2024-07-03T15:41:07.842 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:07 smithi028 ceph-mon[29148]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:07.842 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:07 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/1091199270' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-07-03T15:41:07.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:07 smithi148 ceph-mon[36056]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:07.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:07 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/1091199270' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-07-03T15:41:09.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.631+0000 7f67350aa640 1 -- 172.21.15.28:0/456421904 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6730102d40 msgr2=0x7f67300fc3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:09.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.631+0000 7f67350aa640 1 --2- 172.21.15.28:0/456421904 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6730102d40 0x7f67300fc3d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f67240048d0 tx=0x7f6724009da0 comp rx=0 tx=0).stop 2024-07-03T15:41:09.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.631+0000 7f67350aa640 1 -- 172.21.15.28:0/456421904 shutdown_connections 2024-07-03T15:41:09.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.631+0000 7f67350aa640 1 --2- 172.21.15.28:0/456421904 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6730102d40 0x7f67300fc3d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:09.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.631+0000 7f67350aa640 1 --2- 172.21.15.28:0/456421904 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67301023a0 0x7f6730102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:09.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.631+0000 7f67350aa640 1 -- 172.21.15.28:0/456421904 >> 172.21.15.28:0/456421904 conn(0x7f67300f7fb0 msgr2=0x7f67300fa3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:41:09.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.631+0000 7f67350aa640 1 -- 172.21.15.28:0/456421904 shutdown_connections 2024-07-03T15:41:09.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.631+0000 7f67350aa640 1 -- 172.21.15.28:0/456421904 wait complete. 2024-07-03T15:41:09.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.632+0000 7f67350aa640 1 Processor -- start 2024-07-03T15:41:09.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.632+0000 7f67350aa640 1 -- start start 2024-07-03T15:41:09.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.632+0000 7f67350aa640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f67301023a0 0x7f6730113160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:09.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.632+0000 7f67350aa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6730102d40 0x7f67301136a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:09.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.632+0000 7f67350aa640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6730113d30 con 0x7f67301023a0 2024-07-03T15:41:09.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.632+0000 7f67350aa640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f673010d0e0 con 0x7f6730102d40 2024-07-03T15:41:09.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.632+0000 7f672ed9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f67301023a0 0x7f6730113160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:09.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.632+0000 7f672ed9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f67301023a0 0x7f6730113160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35576/0 (socket says 172.21.15.28:35576) 2024-07-03T15:41:09.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.632+0000 7f672ed9d640 1 -- 172.21.15.28:0/2208348974 learned_addr learned my addr 172.21.15.28:0/2208348974 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:41:09.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.633+0000 7f672e59c640 1 --2- 172.21.15.28:0/2208348974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6730102d40 0x7f67301136a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:09.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.633+0000 7f672ed9d640 1 -- 172.21.15.28:0/2208348974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6730102d40 msgr2=0x7f67301136a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:09.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.633+0000 7f672ed9d640 1 --2- 172.21.15.28:0/2208348974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6730102d40 0x7f67301136a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:09.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.633+0000 7f672ed9d640 1 -- 172.21.15.28:0/2208348974 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6724005040 con 0x7f67301023a0 2024-07-03T15:41:09.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.633+0000 7f672ed9d640 1 --2- 172.21.15.28:0/2208348974 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f67301023a0 0x7f6730113160 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f672000b790 tx=0x7f67200091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:41:09.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.633+0000 7f67177fe640 1 -- 172.21.15.28:0/2208348974 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f672000c700 con 0x7f67301023a0 2024-07-03T15:41:09.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.633+0000 7f67177fe640 1 -- 172.21.15.28:0/2208348974 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f672000cd30 con 0x7f67301023a0 2024-07-03T15:41:09.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.633+0000 7f67350aa640 1 -- 172.21.15.28:0/2208348974 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f673010d3c0 con 0x7f67301023a0 2024-07-03T15:41:09.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.633+0000 7f67177fe640 1 -- 172.21.15.28:0/2208348974 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6720013370 con 0x7f67301023a0 2024-07-03T15:41:09.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.633+0000 7f67350aa640 1 -- 172.21.15.28:0/2208348974 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f673010d8e0 con 0x7f67301023a0 2024-07-03T15:41:09.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.634+0000 7f67350aa640 1 -- 172.21.15.28:0/2208348974 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67300fdb10 con 0x7f67301023a0 2024-07-03T15:41:09.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.635+0000 7f67177fe640 1 -- 172.21.15.28:0/2208348974 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f6720012070 con 0x7f67301023a0 2024-07-03T15:41:09.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.635+0000 7f67177fe640 1 --2- 172.21.15.28:0/2208348974 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f6718067190 0x7f6718069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:09.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.635+0000 7f67177fe640 1 -- 172.21.15.28:0/2208348974 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f67200842f0 con 0x7f67301023a0 2024-07-03T15:41:09.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.635+0000 7f672e59c640 1 --2- 172.21.15.28:0/2208348974 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f6718067190 0x7f6718069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:09.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.635+0000 7f672e59c640 1 --2- 172.21.15.28:0/2208348974 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f6718067190 0x7f6718069640 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f673010ea20 tx=0x7f6724035040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:41:09.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.639+0000 7f67177fe640 1 -- 172.21.15.28:0/2208348974 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f6720051950 con 0x7f67301023a0 2024-07-03T15:41:09.661 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:09 smithi028 ceph-mon[29148]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:09.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.821+0000 7f67350aa640 1 -- 172.21.15.28:0/2208348974 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f673004b560 con 0x7f67301023a0 2024-07-03T15:41:09.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.821+0000 7f67177fe640 1 -- 172.21.15.28:0/2208348974 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6720054f70 con 0x7f67301023a0 2024-07-03T15:41:09.824 INFO:teuthology.orchestra.run.smithi028.stdout:167503724556 2024-07-03T15:41:09.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.823+0000 7f67350aa640 1 -- 172.21.15.28:0/2208348974 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f6718067190 msgr2=0x7f6718069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:09.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.823+0000 7f67350aa640 1 --2- 172.21.15.28:0/2208348974 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f6718067190 0x7f6718069640 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f673010ea20 tx=0x7f6724035040 comp rx=0 tx=0).stop 2024-07-03T15:41:09.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.823+0000 7f67350aa640 1 -- 172.21.15.28:0/2208348974 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f67301023a0 msgr2=0x7f6730113160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:09.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.823+0000 7f67350aa640 1 --2- 172.21.15.28:0/2208348974 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f67301023a0 0x7f6730113160 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f672000b790 tx=0x7f67200091d0 comp rx=0 tx=0).stop 2024-07-03T15:41:09.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.823+0000 7f67350aa640 1 -- 172.21.15.28:0/2208348974 shutdown_connections 2024-07-03T15:41:09.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.823+0000 7f67350aa640 1 --2- 172.21.15.28:0/2208348974 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f6718067190 0x7f6718069640 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:09.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.823+0000 7f67350aa640 1 --2- 172.21.15.28:0/2208348974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6730102d40 0x7f67301136a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:09.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.823+0000 7f67350aa640 1 --2- 172.21.15.28:0/2208348974 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f67301023a0 0x7f6730113160 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:09.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.823+0000 7f67350aa640 1 -- 172.21.15.28:0/2208348974 >> 172.21.15.28:0/2208348974 conn(0x7f67300f7fb0 msgr2=0x7f67300f8e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:41:09.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.823+0000 7f67350aa640 1 -- 172.21.15.28:0/2208348974 shutdown_connections 2024-07-03T15:41:09.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:09.823+0000 7f67350aa640 1 -- 172.21.15.28:0/2208348974 wait complete. 2024-07-03T15:41:09.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:09 smithi148 ceph-mon[36056]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:10.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:10.376+0000 7f8a193c3640 1 -- 172.21.15.28:0/3207026592 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a14102d20 msgr2=0x7f8a140fc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:10.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:10.376+0000 7f8a193c3640 1 --2- 172.21.15.28:0/3207026592 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a14102d20 0x7f8a140fc3b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8a080048d0 tx=0x7f8a08009da0 comp rx=0 tx=0).stop 2024-07-03T15:41:10.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:10.376+0000 7f8a193c3640 1 -- 172.21.15.28:0/3207026592 shutdown_connections 2024-07-03T15:41:10.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:10.376+0000 7f8a193c3640 1 --2- 172.21.15.28:0/3207026592 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a14102d20 0x7f8a140fc3b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:10.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:10.376+0000 7f8a193c3640 1 --2- 172.21.15.28:0/3207026592 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a14102380 0x7f8a14102750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:10.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:10.376+0000 7f8a193c3640 1 -- 172.21.15.28:0/3207026592 >> 172.21.15.28:0/3207026592 conn(0x7f8a140f7f30 msgr2=0x7f8a140fa340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:41:10.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:10.376+0000 7f8a193c3640 1 -- 172.21.15.28:0/3207026592 shutdown_connections 2024-07-03T15:41:10.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:10.376+0000 7f8a193c3640 1 -- 172.21.15.28:0/3207026592 wait complete. 2024-07-03T15:41:10.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:10.377+0000 7f8a193c3640 1 Processor -- start 2024-07-03T15:41:10.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:10.377+0000 7f8a193c3640 1 -- start start 2024-07-03T15:41:10.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:10.377+0000 7f8a193c3640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a14102380 0x7f8a141130f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:10.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:10.377+0000 7f8a193c3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a14102d20 0x7f8a14113630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:10.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:10.377+0000 7f8a193c3640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a14113c30 con 0x7f8a14102380 2024-07-03T15:41:10.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:10.377+0000 7f8a193c3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1410d070 con 0x7f8a14102d20 2024-07-03T15:41:10.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:10.377+0000 7f8a12ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a14102380 0x7f8a141130f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:10.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:10.377+0000 7f8a12ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a14102380 0x7f8a141130f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36382/0 (socket says 172.21.15.28:36382) 2024-07-03T15:41:10.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:10.377+0000 7f8a12ffd640 1 -- 172.21.15.28:0/100850375 learned_addr learned my addr 172.21.15.28:0/100850375 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:41:10.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:10.377+0000 7f8a12ffd640 1 -- 172.21.15.28:0/100850375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a14102d20 msgr2=0x7f8a14113630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:10.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:10.377+0000 7f8a12ffd640 1 --2- 172.21.15.28:0/100850375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a14102d20 0x7f8a14113630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:10.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:10.377+0000 7f8a12ffd640 1 -- 172.21.15.28:0/100850375 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a08005040 con 0x7f8a14102380 2024-07-03T15:41:10.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:10.377+0000 7f8a12ffd640 1 --2- 172.21.15.28:0/100850375 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a14102380 0x7f8a141130f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8a0400eb70 tx=0x7f8a0400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:41:10.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:10.377+0000 7f89fb7fe640 1 -- 172.21.15.28:0/100850375 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a04007c30 con 0x7f8a14102380 2024-07-03T15:41:10.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:10.378+0000 7f8a193c3640 1 -- 172.21.15.28:0/100850375 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a1410d2f0 con 0x7f8a14102380 2024-07-03T15:41:10.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:10.378+0000 7f89fb7fe640 1 -- 172.21.15.28:0/100850375 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8a0400f040 con 0x7f8a14102380 2024-07-03T15:41:10.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:10.378+0000 7f89fb7fe640 1 -- 172.21.15.28:0/100850375 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a040095b0 con 0x7f8a14102380 2024-07-03T15:41:10.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:10.378+0000 7f8a193c3640 1 -- 172.21.15.28:0/100850375 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a1410d890 con 0x7f8a14102380 2024-07-03T15:41:10.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:10.379+0000 7f8a193c3640 1 -- 172.21.15.28:0/100850375 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a140fdaf0 con 0x7f8a14102380 2024-07-03T15:41:10.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:10.379+0000 7f89fb7fe640 1 -- 172.21.15.28:0/100850375 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f8a04009730 con 0x7f8a14102380 2024-07-03T15:41:10.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:10.379+0000 7f89fb7fe640 1 --2- 172.21.15.28:0/100850375 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f89fc067190 0x7f89fc069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:10.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:10.380+0000 7f89fb7fe640 1 -- 172.21.15.28:0/100850375 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f8a040844f0 con 0x7f8a14102380 2024-07-03T15:41:10.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:10.380+0000 7f8a127fc640 1 --2- 172.21.15.28:0/100850375 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f89fc067190 0x7f89fc069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:10.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:10.380+0000 7f8a127fc640 1 --2- 172.21.15.28:0/100850375 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f89fc067190 0x7f89fc069640 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8a0802a530 tx=0x7f8a08009da0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:41:10.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:10.384+0000 7f89fb7fe640 1 -- 172.21.15.28:0/100850375 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8a04051ad0 con 0x7f8a14102380 2024-07-03T15:41:10.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:10.563+0000 7f8a193c3640 1 -- 172.21.15.28:0/100850375 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f8a140fbb70 con 0x7f8a14102380 2024-07-03T15:41:10.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:10.563+0000 7f89fb7fe640 1 -- 172.21.15.28:0/100850375 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8a040550f0 con 0x7f8a14102380 2024-07-03T15:41:10.567 INFO:teuthology.orchestra.run.smithi028.stdout:146028888081 2024-07-03T15:41:10.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:10.565+0000 7f8a193c3640 1 -- 172.21.15.28:0/100850375 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f89fc067190 msgr2=0x7f89fc069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:10.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:10.565+0000 7f8a193c3640 1 --2- 172.21.15.28:0/100850375 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f89fc067190 0x7f89fc069640 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8a0802a530 tx=0x7f8a08009da0 comp rx=0 tx=0).stop 2024-07-03T15:41:10.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:10.565+0000 7f8a193c3640 1 -- 172.21.15.28:0/100850375 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a14102380 msgr2=0x7f8a141130f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:10.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:10.565+0000 7f8a193c3640 1 --2- 172.21.15.28:0/100850375 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a14102380 0x7f8a141130f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8a0400eb70 tx=0x7f8a0400b040 comp rx=0 tx=0).stop 2024-07-03T15:41:10.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:10.565+0000 7f8a193c3640 1 -- 172.21.15.28:0/100850375 shutdown_connections 2024-07-03T15:41:10.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:10.565+0000 7f8a193c3640 1 --2- 172.21.15.28:0/100850375 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f89fc067190 0x7f89fc069640 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:10.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:10.565+0000 7f8a193c3640 1 --2- 172.21.15.28:0/100850375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a14102d20 0x7f8a14113630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:10.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:10.565+0000 7f8a193c3640 1 --2- 172.21.15.28:0/100850375 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a14102380 0x7f8a141130f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:10.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:10.566+0000 7f8a193c3640 1 -- 172.21.15.28:0/100850375 >> 172.21.15.28:0/100850375 conn(0x7f8a140f7f30 msgr2=0x7f8a140f8e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:41:10.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:10.566+0000 7f8a193c3640 1 -- 172.21.15.28:0/100850375 shutdown_connections 2024-07-03T15:41:10.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:10.566+0000 7f8a193c3640 1 -- 172.21.15.28:0/100850375 wait complete. 2024-07-03T15:41:10.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:10 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/2208348974' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-07-03T15:41:10.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:10 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/100850375' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-07-03T15:41:10.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:10 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/2208348974' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-07-03T15:41:10.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:10 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/100850375' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-07-03T15:41:10.993 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724554 got 167503724556 for osd.5 2024-07-03T15:41:10.993 DEBUG:teuthology.parallel:result is None 2024-07-03T15:41:11.769 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:11 smithi028 ceph-mon[29148]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:11.800 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888079 got 146028888081 for osd.4 2024-07-03T15:41:11.800 DEBUG:teuthology.parallel:result is None 2024-07-03T15:41:11.801 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-07-03T15:41:11.801 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph pg dump --format=json 2024-07-03T15:41:11.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:11 smithi148 ceph-mon[36056]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:12.074 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:41:13.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.691+0000 7fdc06bab640 1 -- 172.21.15.28:0/234918834 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdc000650c0 msgr2=0x7fdc00065490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:13.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.691+0000 7fdc06bab640 1 --2- 172.21.15.28:0/234918834 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdc000650c0 0x7fdc00065490 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8009c50 tx=0x7fdbf8004d10 comp rx=0 tx=0).stop 2024-07-03T15:41:13.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.691+0000 7fdc06bab640 1 -- 172.21.15.28:0/234918834 shutdown_connections 2024-07-03T15:41:13.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.691+0000 7fdc06bab640 1 --2- 172.21.15.28:0/234918834 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc000fdf50 0x7fdc000fe3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:13.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.691+0000 7fdc06bab640 1 --2- 172.21.15.28:0/234918834 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdc000650c0 0x7fdc00065490 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:13.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.691+0000 7fdc06bab640 1 -- 172.21.15.28:0/234918834 >> 172.21.15.28:0/234918834 conn(0x7fdc000721c0 msgr2=0x7fdc000725c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:41:13.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.691+0000 7fdc06bab640 1 -- 172.21.15.28:0/234918834 shutdown_connections 2024-07-03T15:41:13.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.691+0000 7fdc06bab640 1 -- 172.21.15.28:0/234918834 wait complete. 2024-07-03T15:41:13.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.691+0000 7fdc06bab640 1 Processor -- start 2024-07-03T15:41:13.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.691+0000 7fdc06bab640 1 -- start start 2024-07-03T15:41:13.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.692+0000 7fdc06bab640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdc000650c0 0x7fdc00113130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:13.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.692+0000 7fdc06bab640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc000fdf50 0x7fdc00113670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:13.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.692+0000 7fdc06bab640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc00113d00 con 0x7fdc000650c0 2024-07-03T15:41:13.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.692+0000 7fdc06bab640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0010d0b0 con 0x7fdc000fdf50 2024-07-03T15:41:13.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.692+0000 7fdc04947640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdc000650c0 0x7fdc00113130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:13.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.692+0000 7fdc04947640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdc000650c0 0x7fdc00113130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36398/0 (socket says 172.21.15.28:36398) 2024-07-03T15:41:13.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.692+0000 7fdc04947640 1 -- 172.21.15.28:0/926386530 learned_addr learned my addr 172.21.15.28:0/926386530 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:41:13.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.692+0000 7fdbfffff640 1 --2- 172.21.15.28:0/926386530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc000fdf50 0x7fdc00113670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:13.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.692+0000 7fdc04947640 1 -- 172.21.15.28:0/926386530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc000fdf50 msgr2=0x7fdc00113670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:13.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.692+0000 7fdc04947640 1 --2- 172.21.15.28:0/926386530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc000fdf50 0x7fdc00113670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:13.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.692+0000 7fdc04947640 1 -- 172.21.15.28:0/926386530 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf80098b0 con 0x7fdc000650c0 2024-07-03T15:41:13.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.692+0000 7fdc04947640 1 --2- 172.21.15.28:0/926386530 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdc000650c0 0x7fdc00113130 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8004f80 tx=0x7fdbf8004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:41:13.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.692+0000 7fdbfd7fa640 1 -- 172.21.15.28:0/926386530 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdbf8008de0 con 0x7fdc000650c0 2024-07-03T15:41:13.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.692+0000 7fdc06bab640 1 -- 172.21.15.28:0/926386530 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc0010d330 con 0x7fdc000650c0 2024-07-03T15:41:13.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.693+0000 7fdbfd7fa640 1 -- 172.21.15.28:0/926386530 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdbf8032c00 con 0x7fdc000650c0 2024-07-03T15:41:13.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.693+0000 7fdbfd7fa640 1 -- 172.21.15.28:0/926386530 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdbf803dbc0 con 0x7fdc000650c0 2024-07-03T15:41:13.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.693+0000 7fdc06bab640 1 -- 172.21.15.28:0/926386530 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc0010d850 con 0x7fdc000650c0 2024-07-03T15:41:13.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.693+0000 7fdc06bab640 1 -- 172.21.15.28:0/926386530 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc000fdaf0 con 0x7fdc000650c0 2024-07-03T15:41:13.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.694+0000 7fdbfd7fa640 1 -- 172.21.15.28:0/926386530 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7fdbf802f030 con 0x7fdc000650c0 2024-07-03T15:41:13.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.694+0000 7fdbfd7fa640 1 --2- 172.21.15.28:0/926386530 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fdbe8067190 0x7fdbe8069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:13.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.694+0000 7fdbfd7fa640 1 -- 172.21.15.28:0/926386530 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fdbf80a5ec0 con 0x7fdc000650c0 2024-07-03T15:41:13.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.694+0000 7fdbfffff640 1 --2- 172.21.15.28:0/926386530 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fdbe8067190 0x7fdbe8069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:13.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.695+0000 7fdbfffff640 1 --2- 172.21.15.28:0/926386530 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fdbe8067190 0x7fdbe8069640 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4006fd0 tx=0x7fdbf4008040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:41:13.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.699+0000 7fdbfd7fa640 1 -- 172.21.15.28:0/926386530 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fdbf80734a0 con 0x7fdc000650c0 2024-07-03T15:41:13.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:13 smithi028 ceph-mon[29148]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:13.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.871+0000 7fdc06bab640 1 -- 172.21.15.28:0/926386530 --> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdc0004b560 con 0x7fdbe8067190 2024-07-03T15:41:13.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.873+0000 7fdbfd7fa640 1 -- 172.21.15.28:0/926386530 <== mgr.14210 v2:172.21.15.28:6800/421057178 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28230 (secure 0 0 0) 0x7fdc0004b560 con 0x7fdbe8067190 2024-07-03T15:41:13.876 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:41:13.876 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-07-03T15:41:13.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.875+0000 7fdc06bab640 1 -- 172.21.15.28:0/926386530 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fdbe8067190 msgr2=0x7fdbe8069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:13.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.875+0000 7fdc06bab640 1 --2- 172.21.15.28:0/926386530 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fdbe8067190 0x7fdbe8069640 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4006fd0 tx=0x7fdbf4008040 comp rx=0 tx=0).stop 2024-07-03T15:41:13.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.875+0000 7fdc06bab640 1 -- 172.21.15.28:0/926386530 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdc000650c0 msgr2=0x7fdc00113130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:13.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.875+0000 7fdc06bab640 1 --2- 172.21.15.28:0/926386530 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdc000650c0 0x7fdc00113130 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8004f80 tx=0x7fdbf8004d10 comp rx=0 tx=0).stop 2024-07-03T15:41:13.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.875+0000 7fdc06bab640 1 -- 172.21.15.28:0/926386530 shutdown_connections 2024-07-03T15:41:13.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.875+0000 7fdc06bab640 1 --2- 172.21.15.28:0/926386530 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fdbe8067190 0x7fdbe8069640 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:13.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.875+0000 7fdc06bab640 1 --2- 172.21.15.28:0/926386530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc000fdf50 0x7fdc00113670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:13.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.875+0000 7fdc06bab640 1 --2- 172.21.15.28:0/926386530 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdc000650c0 0x7fdc00113130 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:13.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.875+0000 7fdc06bab640 1 -- 172.21.15.28:0/926386530 >> 172.21.15.28:0/926386530 conn(0x7fdc000721c0 msgr2=0x7fdc000fa010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:41:13.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.875+0000 7fdc06bab640 1 -- 172.21.15.28:0/926386530 shutdown_connections 2024-07-03T15:41:13.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:13.875+0000 7fdc06bab640 1 -- 172.21.15.28:0/926386530 wait complete. 2024-07-03T15:41:13.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:13 smithi148 ceph-mon[36056]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:14.507 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":154,"stamp":"2024-07-03T15:41:12.382108+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1788052,"kb_used_data":3876,"kb_used_omap":9,"kb_used_meta":1784054,"kb_avail":560682860,"statfs":{"total":575970213888,"available":574139248640,"internally_reserved":0,"allocated":3969024,"data_stored":2606262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1826871312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002113"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":190,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-03T15:40:19.493447+0000","last_change":"2024-07-03T15:39:54.699987+0000","last_active":"2024-07-03T15:40:19.493447+0000","last_peered":"2024-07-03T15:40:19.493447+0000","last_clean":"2024-07-03T15:40:19.493447+0000","last_became_active":"2024-07-03T15:39:54.699620+0000","last_became_peered":"2024-07-03T15:39:54.699620+0000","last_unstale":"2024-07-03T15:40:19.493447+0000","last_undegraded":"2024-07-03T15:40:19.493447+0000","last_fullsized":"2024-07-03T15:40:19.493447+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-03T15:39:08.833730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-03T15:39:08.833730+0000","last_clean_scrub_stamp":"2024-07-03T15:39:08.833730+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-04T19:19:07.782643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]}]},{"osd":4,"up_from":34,"seq":146028888082,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297716,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447436,"statfs":{"total":95995035648,"available":95690174464,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jul 3 15:40:59 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.47799999999999998}]},{"osd":1,"last update":"Wed Jul 3 15:40:59 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.48999999999999999}]},{"osd":2,"last update":"Wed Jul 3 15:40:59 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.51000000000000001}]},{"osd":3,"last update":"Wed Jul 3 15:40:59 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.39700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32600000000000001}]}]},{"osd":3,"up_from":26,"seq":111669149718,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298300,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446852,"statfs":{"total":95995035648,"available":95689576448,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jul 3 15:40:32 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.40999999999999998}]},{"osd":1,"last update":"Wed Jul 3 15:40:32 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.371},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.38600000000000001}]},{"osd":2,"last update":"Wed Jul 3 15:40:32 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.45100000000000001}]},{"osd":4,"last update":"Wed Jul 3 15:41:00 2024","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"last":0.435},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.42299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]}]},{"osd":2,"up_from":19,"seq":81604378651,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297724,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447428,"statfs":{"total":95995035648,"available":95690166272,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jul 3 15:40:08 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"min":{"1min":0.16700000000000001,"5min":0.16700000000000001,"15min":0.16700000000000001},"max":{"1min":0.5,"5min":0.5,"15min":0.5},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.19,"5min":0.19,"15min":0.19},"max":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"last":0.54500000000000004}]},{"osd":1,"last update":"Wed Jul 3 15:40:08 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"last":0.57299999999999995}]},{"osd":3,"last update":"Wed Jul 3 15:40:29 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.70099999999999996}]},{"osd":4,"last update":"Wed Jul 3 15:40:54 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.621}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]}]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298300,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446852,"statfs":{"total":95995035648,"available":95689576448,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jul 3 15:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.35299999999999998,"15min":0.35199999999999998},"min":{"1min":0.26800000000000002,"5min":0.215,"15min":0.215},"max":{"1min":0.442,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.36499999999999999,"15min":0.36699999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.56899999999999995,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.38900000000000001}]},{"osd":2,"last update":"Wed Jul 3 15:41:09 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.38300000000000001,"15min":0.38900000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.42399999999999999,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.375,"15min":0.38300000000000001},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.46200000000000002,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.36899999999999999}]},{"osd":3,"last update":"Wed Jul 3 15:40:31 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.57499999999999996}]},{"osd":4,"last update":"Wed Jul 3 15:40:58 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.51800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]}]},{"osd":0,"up_from":9,"seq":38654705703,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298300,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446852,"statfs":{"total":95995035648,"available":95689576448,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jul 3 15:40:47 2024","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.33700000000000002,"15min":0.33900000000000002},"min":{"1min":0.23000000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.40899999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.35499999999999998,"15min":0.36099999999999999},"min":{"1min":0.23100000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.45600000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.38400000000000001}]},{"osd":2,"last update":"Wed Jul 3 15:40:11 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.501,"5min":0.501,"15min":0.501},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.39700000000000002}]},{"osd":3,"last update":"Wed Jul 3 15:40:30 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.57799999999999996}]},{"osd":4,"last update":"Wed Jul 3 15:40:55 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.54500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-03T15:41:14.509 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph pg dump --format=json 2024-07-03T15:41:14.774 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:41:15.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:15 smithi028 ceph-mon[29148]: from='client.14406 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-03T15:41:15.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:15 smithi028 ceph-mon[29148]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:15.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:15 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:41:15.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:15 smithi148 ceph-mon[36056]: from='client.14406 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-03T15:41:15.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:15 smithi148 ceph-mon[36056]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:15.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:15 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:41:16.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:16.495+0000 7f581c5b1640 1 -- 172.21.15.28:0/673683931 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5814102350 msgr2=0x7f5814102720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:16.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:16.495+0000 7f581c5b1640 1 --2- 172.21.15.28:0/673683931 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5814102350 0x7f5814102720 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f58040048d0 tx=0x7f5804009da0 comp rx=0 tx=0).stop 2024-07-03T15:41:16.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:16.495+0000 7f581c5b1640 1 -- 172.21.15.28:0/673683931 shutdown_connections 2024-07-03T15:41:16.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:16.495+0000 7f581c5b1640 1 --2- 172.21.15.28:0/673683931 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5814102cf0 0x7f58140fc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:16.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:16.495+0000 7f581c5b1640 1 --2- 172.21.15.28:0/673683931 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5814102350 0x7f5814102720 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:16.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:16.495+0000 7f581c5b1640 1 -- 172.21.15.28:0/673683931 >> 172.21.15.28:0/673683931 conn(0x7f58140f7f20 msgr2=0x7f58140fa330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:41:16.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:16.495+0000 7f581c5b1640 1 -- 172.21.15.28:0/673683931 shutdown_connections 2024-07-03T15:41:16.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:16.495+0000 7f581c5b1640 1 -- 172.21.15.28:0/673683931 wait complete. 2024-07-03T15:41:16.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:16.496+0000 7f581c5b1640 1 Processor -- start 2024-07-03T15:41:16.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:16.496+0000 7f581c5b1640 1 -- start start 2024-07-03T15:41:16.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:16.496+0000 7f581c5b1640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5814102350 0x7f5814113090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:16.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:16.496+0000 7f581c5b1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5814102cf0 0x7f58141135d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:16.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:16.496+0000 7f581c5b1640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5814113c60 con 0x7f5814102350 2024-07-03T15:41:16.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:16.496+0000 7f581c5b1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581410d0c0 con 0x7f5814102cf0 2024-07-03T15:41:16.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:16.496+0000 7f581a34d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5814102350 0x7f5814113090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:16.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:16.496+0000 7f581a34d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5814102350 0x7f5814113090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36416/0 (socket says 172.21.15.28:36416) 2024-07-03T15:41:16.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:16.496+0000 7f581a34d640 1 -- 172.21.15.28:0/2315996197 learned_addr learned my addr 172.21.15.28:0/2315996197 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:41:16.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:16.496+0000 7f581a34d640 1 -- 172.21.15.28:0/2315996197 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5814102cf0 msgr2=0x7f58141135d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:16.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:16.496+0000 7f581a34d640 1 --2- 172.21.15.28:0/2315996197 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5814102cf0 0x7f58141135d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:16.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:16.496+0000 7f581a34d640 1 -- 172.21.15.28:0/2315996197 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5804005040 con 0x7f5814102350 2024-07-03T15:41:16.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:16.496+0000 7f581a34d640 1 --2- 172.21.15.28:0/2315996197 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5814102350 0x7f5814113090 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5804003080 tx=0x7f5804009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:41:16.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:16.497+0000 7f5812ffd640 1 -- 172.21.15.28:0/2315996197 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5804038070 con 0x7f5814102350 2024-07-03T15:41:16.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:16.497+0000 7f581c5b1640 1 -- 172.21.15.28:0/2315996197 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f581410d340 con 0x7f5814102350 2024-07-03T15:41:16.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:16.497+0000 7f5812ffd640 1 -- 172.21.15.28:0/2315996197 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5804008ee0 con 0x7f5814102350 2024-07-03T15:41:16.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:16.497+0000 7f5812ffd640 1 -- 172.21.15.28:0/2315996197 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f580403d870 con 0x7f5814102350 2024-07-03T15:41:16.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:16.497+0000 7f581c5b1640 1 -- 172.21.15.28:0/2315996197 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f581410d800 con 0x7f5814102350 2024-07-03T15:41:16.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:16.498+0000 7f581c5b1640 1 -- 172.21.15.28:0/2315996197 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58140fdac0 con 0x7f5814102350 2024-07-03T15:41:16.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:16.498+0000 7f5812ffd640 1 -- 172.21.15.28:0/2315996197 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f5804030020 con 0x7f5814102350 2024-07-03T15:41:16.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:16.499+0000 7f5812ffd640 1 --2- 172.21.15.28:0/2315996197 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f57fc067190 0x7f57fc069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:16.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:16.499+0000 7f5812ffd640 1 -- 172.21.15.28:0/2315996197 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f58040a4520 con 0x7f5814102350 2024-07-03T15:41:16.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:16.499+0000 7f5819b4c640 1 --2- 172.21.15.28:0/2315996197 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f57fc067190 0x7f57fc069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:16.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:16.499+0000 7f5819b4c640 1 --2- 172.21.15.28:0/2315996197 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f57fc067190 0x7f57fc069640 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f58141065f0 tx=0x7f5808006d10 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:41:16.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:16.503+0000 7f5812ffd640 1 -- 172.21.15.28:0/2315996197 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5804071b80 con 0x7f5814102350 2024-07-03T15:41:16.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:16.676+0000 7f581c5b1640 1 -- 172.21.15.28:0/2315996197 --> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5814109580 con 0x7f57fc067190 2024-07-03T15:41:16.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:16.677+0000 7f5812ffd640 1 -- 172.21.15.28:0/2315996197 <== mgr.14210 v2:172.21.15.28:6800/421057178 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28277 (secure 0 0 0) 0x7f5814109580 con 0x7f57fc067190 2024-07-03T15:41:16.681 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:41:16.681 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-07-03T15:41:16.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:16.679+0000 7f581c5b1640 1 -- 172.21.15.28:0/2315996197 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f57fc067190 msgr2=0x7f57fc069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:16.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:16.679+0000 7f581c5b1640 1 --2- 172.21.15.28:0/2315996197 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f57fc067190 0x7f57fc069640 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f58141065f0 tx=0x7f5808006d10 comp rx=0 tx=0).stop 2024-07-03T15:41:16.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:16.679+0000 7f581c5b1640 1 -- 172.21.15.28:0/2315996197 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5814102350 msgr2=0x7f5814113090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:16.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:16.679+0000 7f581c5b1640 1 --2- 172.21.15.28:0/2315996197 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5814102350 0x7f5814113090 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5804003080 tx=0x7f5804009da0 comp rx=0 tx=0).stop 2024-07-03T15:41:16.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:16.679+0000 7f581c5b1640 1 -- 172.21.15.28:0/2315996197 shutdown_connections 2024-07-03T15:41:16.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:16.679+0000 7f581c5b1640 1 --2- 172.21.15.28:0/2315996197 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f57fc067190 0x7f57fc069640 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:16.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:16.679+0000 7f581c5b1640 1 --2- 172.21.15.28:0/2315996197 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5814102cf0 0x7f58141135d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:16.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:16.679+0000 7f581c5b1640 1 --2- 172.21.15.28:0/2315996197 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5814102350 0x7f5814113090 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:16.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:16.679+0000 7f581c5b1640 1 -- 172.21.15.28:0/2315996197 >> 172.21.15.28:0/2315996197 conn(0x7f58140f7f20 msgr2=0x7f58140f8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:41:16.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:16.680+0000 7f581c5b1640 1 -- 172.21.15.28:0/2315996197 shutdown_connections 2024-07-03T15:41:16.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:16.680+0000 7f581c5b1640 1 -- 172.21.15.28:0/2315996197 wait complete. 2024-07-03T15:41:17.559 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":156,"stamp":"2024-07-03T15:41:16.382774+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1788052,"kb_used_data":3876,"kb_used_omap":9,"kb_used_meta":1784054,"kb_avail":560682860,"statfs":{"total":575970213888,"available":574139248640,"internally_reserved":0,"allocated":3969024,"data_stored":2606262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1826871312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002210"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":190,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-03T15:40:19.493447+0000","last_change":"2024-07-03T15:39:54.699987+0000","last_active":"2024-07-03T15:40:19.493447+0000","last_peered":"2024-07-03T15:40:19.493447+0000","last_clean":"2024-07-03T15:40:19.493447+0000","last_became_active":"2024-07-03T15:39:54.699620+0000","last_became_peered":"2024-07-03T15:39:54.699620+0000","last_unstale":"2024-07-03T15:40:19.493447+0000","last_undegraded":"2024-07-03T15:40:19.493447+0000","last_fullsized":"2024-07-03T15:40:19.493447+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-03T15:39:08.833730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-03T15:39:08.833730+0000","last_clean_scrub_stamp":"2024-07-03T15:39:08.833730+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-04T19:19:07.782643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724558,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]}]},{"osd":4,"up_from":34,"seq":146028888083,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297716,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447436,"statfs":{"total":95995035648,"available":95690174464,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jul 3 15:40:59 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.44},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.53600000000000003}]},{"osd":1,"last update":"Wed Jul 3 15:40:59 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.48199999999999998}]},{"osd":2,"last update":"Wed Jul 3 15:40:59 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.55800000000000005}]},{"osd":3,"last update":"Wed Jul 3 15:40:59 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.46100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]}]},{"osd":3,"up_from":26,"seq":111669149719,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298300,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446852,"statfs":{"total":95995035648,"available":95689576448,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jul 3 15:40:32 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.48799999999999999}]},{"osd":1,"last update":"Wed Jul 3 15:40:32 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.45800000000000002}]},{"osd":2,"last update":"Wed Jul 3 15:40:32 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.433},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.47399999999999998}]},{"osd":4,"last update":"Wed Jul 3 15:41:00 2024","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.505}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]}]},{"osd":2,"up_from":19,"seq":81604378652,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297724,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447428,"statfs":{"total":95995035648,"available":95690166272,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jul 3 15:41:09 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.34300000000000003,"15min":0.34000000000000002},"min":{"1min":0.20300000000000001,"5min":0.16700000000000001,"15min":0.16700000000000001},"max":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.441},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.35499999999999998,"15min":0.35299999999999998},"min":{"1min":0.24399999999999999,"5min":0.19,"15min":0.19},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.63500000000000001}]},{"osd":1,"last update":"Wed Jul 3 15:41:09 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.35899999999999999,"15min":0.35699999999999998},"min":{"1min":0.25600000000000001,"5min":0.218,"15min":0.218},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.36499999999999999,"15min":0.36099999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.48299999999999998}]},{"osd":3,"last update":"Wed Jul 3 15:40:29 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.66600000000000004}]},{"osd":4,"last update":"Wed Jul 3 15:40:54 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.60899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]}]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298300,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446852,"statfs":{"total":95995035648,"available":95689576448,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jul 3 15:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.35299999999999998,"15min":0.35199999999999998},"min":{"1min":0.26800000000000002,"5min":0.215,"15min":0.215},"max":{"1min":0.442,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.36499999999999999,"15min":0.36699999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.56899999999999995,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.38900000000000001}]},{"osd":2,"last update":"Wed Jul 3 15:41:09 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.38300000000000001,"15min":0.38900000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.42399999999999999,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.375,"15min":0.38300000000000001},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.46200000000000002,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.36899999999999999}]},{"osd":3,"last update":"Wed Jul 3 15:40:31 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.57499999999999996}]},{"osd":4,"last update":"Wed Jul 3 15:40:58 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.51800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]}]},{"osd":0,"up_from":9,"seq":38654705704,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298300,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446852,"statfs":{"total":95995035648,"available":95689576448,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jul 3 15:40:47 2024","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.33700000000000002,"15min":0.33900000000000002},"min":{"1min":0.23000000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.40899999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.35499999999999998,"15min":0.36099999999999999},"min":{"1min":0.23100000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.45600000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.46700000000000003}]},{"osd":2,"last update":"Wed Jul 3 15:40:11 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.501,"5min":0.501,"15min":0.501},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.51000000000000001}]},{"osd":3,"last update":"Wed Jul 3 15:40:30 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.73199999999999998}]},{"osd":4,"last update":"Wed Jul 3 15:40:55 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.71299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-03T15:41:17.560 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-07-03T15:41:17.561 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-07-03T15:41:17.561 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-07-03T15:41:17.561 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph health --format=json 2024-07-03T15:41:17.830 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:41:17.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:17 smithi028 ceph-mon[29148]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:17.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:17 smithi028 ceph-mon[29148]: from='client.14410 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-03T15:41:17.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:17 smithi148 ceph-mon[36056]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:17.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:17 smithi148 ceph-mon[36056]: from='client.14410 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-03T15:41:19.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.515+0000 7fdd7fd0e640 1 -- 172.21.15.28:0/1082007978 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd78101c00 msgr2=0x7fdd780fb290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:19.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.515+0000 7fdd7fd0e640 1 --2- 172.21.15.28:0/1082007978 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd78101c00 0x7fdd780fb290 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdd680048d0 tx=0x7fdd68009da0 comp rx=0 tx=0).stop 2024-07-03T15:41:19.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.515+0000 7fdd7fd0e640 1 -- 172.21.15.28:0/1082007978 shutdown_connections 2024-07-03T15:41:19.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.515+0000 7fdd7fd0e640 1 --2- 172.21.15.28:0/1082007978 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd78101c00 0x7fdd780fb290 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:19.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.515+0000 7fdd7fd0e640 1 --2- 172.21.15.28:0/1082007978 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd78101260 0x7fdd78101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:19.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.515+0000 7fdd7fd0e640 1 -- 172.21.15.28:0/1082007978 >> 172.21.15.28:0/1082007978 conn(0x7fdd780f6e10 msgr2=0x7fdd780f9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:41:19.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.515+0000 7fdd7fd0e640 1 -- 172.21.15.28:0/1082007978 shutdown_connections 2024-07-03T15:41:19.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.515+0000 7fdd7fd0e640 1 -- 172.21.15.28:0/1082007978 wait complete. 2024-07-03T15:41:19.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.516+0000 7fdd7fd0e640 1 Processor -- start 2024-07-03T15:41:19.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.516+0000 7fdd7fd0e640 1 -- start start 2024-07-03T15:41:19.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.516+0000 7fdd7fd0e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd78101260 0x7fdd781152f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:19.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.516+0000 7fdd7fd0e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd78101c00 0x7fdd78115830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:19.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.516+0000 7fdd7fd0e640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd78115e30 con 0x7fdd78101c00 2024-07-03T15:41:19.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.516+0000 7fdd7fd0e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd78115fa0 con 0x7fdd78101260 2024-07-03T15:41:19.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.516+0000 7fdd7d2a9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd78101c00 0x7fdd78115830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:19.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.516+0000 7fdd7d2a9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd78101c00 0x7fdd78115830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36444/0 (socket says 172.21.15.28:36444) 2024-07-03T15:41:19.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.516+0000 7fdd7d2a9640 1 -- 172.21.15.28:0/597501252 learned_addr learned my addr 172.21.15.28:0/597501252 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:41:19.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.516+0000 7fdd7daaa640 1 --2- 172.21.15.28:0/597501252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd78101260 0x7fdd781152f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:19.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.517+0000 7fdd7d2a9640 1 -- 172.21.15.28:0/597501252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd78101260 msgr2=0x7fdd781152f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:19.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.517+0000 7fdd7d2a9640 1 --2- 172.21.15.28:0/597501252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd78101260 0x7fdd781152f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:19.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.517+0000 7fdd7d2a9640 1 -- 172.21.15.28:0/597501252 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd68005040 con 0x7fdd78101c00 2024-07-03T15:41:19.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.517+0000 7fdd7d2a9640 1 --2- 172.21.15.28:0/597501252 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd78101c00 0x7fdd78115830 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdd680043d0 tx=0x7fdd68009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:41:19.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.517+0000 7fdd6e7fc640 1 -- 172.21.15.28:0/597501252 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd68038070 con 0x7fdd78101c00 2024-07-03T15:41:19.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.517+0000 7fdd6e7fc640 1 -- 172.21.15.28:0/597501252 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd68032970 con 0x7fdd78101c00 2024-07-03T15:41:19.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.517+0000 7fdd7fd0e640 1 -- 172.21.15.28:0/597501252 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd7810f3c0 con 0x7fdd78101c00 2024-07-03T15:41:19.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.517+0000 7fdd6e7fc640 1 -- 172.21.15.28:0/597501252 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd6803da60 con 0x7fdd78101c00 2024-07-03T15:41:19.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.517+0000 7fdd7fd0e640 1 -- 172.21.15.28:0/597501252 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd7810f880 con 0x7fdd78101c00 2024-07-03T15:41:19.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.518+0000 7fdd7fd0e640 1 -- 172.21.15.28:0/597501252 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd780fc9d0 con 0x7fdd78101c00 2024-07-03T15:41:19.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.519+0000 7fdd6e7fc640 1 -- 172.21.15.28:0/597501252 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7fdd68030020 con 0x7fdd78101c00 2024-07-03T15:41:19.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.519+0000 7fdd6e7fc640 1 --2- 172.21.15.28:0/597501252 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fdd64067190 0x7fdd64069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:19.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.519+0000 7fdd6e7fc640 1 -- 172.21.15.28:0/597501252 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fdd680a4900 con 0x7fdd78101c00 2024-07-03T15:41:19.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.519+0000 7fdd7daaa640 1 --2- 172.21.15.28:0/597501252 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fdd64067190 0x7fdd64069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:19.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.520+0000 7fdd7daaa640 1 --2- 172.21.15.28:0/597501252 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fdd64067190 0x7fdd64069640 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdd74007fd0 tx=0x7fdd74009040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:41:19.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.523+0000 7fdd6e7fc640 1 -- 172.21.15.28:0/597501252 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fdd68071ee0 con 0x7fdd78101c00 2024-07-03T15:41:19.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.734+0000 7fdd7fd0e640 1 -- 172.21.15.28:0/597501252 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fdd781083d0 con 0x7fdd78101c00 2024-07-03T15:41:19.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.735+0000 7fdd6e7fc640 1 -- 172.21.15.28:0/597501252 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fdd68075500 con 0x7fdd78101c00 2024-07-03T15:41:19.738 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:41:19.738 INFO:teuthology.orchestra.run.smithi028.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-07-03T15:41:19.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.736+0000 7fdd7fd0e640 1 -- 172.21.15.28:0/597501252 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fdd64067190 msgr2=0x7fdd64069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:19.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.736+0000 7fdd7fd0e640 1 --2- 172.21.15.28:0/597501252 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fdd64067190 0x7fdd64069640 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdd74007fd0 tx=0x7fdd74009040 comp rx=0 tx=0).stop 2024-07-03T15:41:19.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.736+0000 7fdd7fd0e640 1 -- 172.21.15.28:0/597501252 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd78101c00 msgr2=0x7fdd78115830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:19.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.736+0000 7fdd7fd0e640 1 --2- 172.21.15.28:0/597501252 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd78101c00 0x7fdd78115830 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdd680043d0 tx=0x7fdd68009da0 comp rx=0 tx=0).stop 2024-07-03T15:41:19.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.736+0000 7fdd7fd0e640 1 -- 172.21.15.28:0/597501252 shutdown_connections 2024-07-03T15:41:19.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.736+0000 7fdd7fd0e640 1 --2- 172.21.15.28:0/597501252 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fdd64067190 0x7fdd64069640 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:19.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.736+0000 7fdd7fd0e640 1 --2- 172.21.15.28:0/597501252 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd78101c00 0x7fdd78115830 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:19.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.736+0000 7fdd7fd0e640 1 --2- 172.21.15.28:0/597501252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd78101260 0x7fdd781152f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:19.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.736+0000 7fdd7fd0e640 1 -- 172.21.15.28:0/597501252 >> 172.21.15.28:0/597501252 conn(0x7fdd780f6e10 msgr2=0x7fdd78065860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:41:19.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.737+0000 7fdd7fd0e640 1 -- 172.21.15.28:0/597501252 shutdown_connections 2024-07-03T15:41:19.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:19.737+0000 7fdd7fd0e640 1 -- 172.21.15.28:0/597501252 wait complete. 2024-07-03T15:41:19.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:19 smithi028 ceph-mon[29148]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:19.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:19 smithi148 ceph-mon[36056]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:20.505 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-07-03T15:41:20.505 INFO:tasks.cephadm:Setup complete, yielding 2024-07-03T15:41:20.506 INFO:teuthology.run_tasks:Running task print... 2024-07-03T15:41:20.515 INFO:teuthology.task.print:**** done end installing quincy cephadm ... 2024-07-03T15:41:20.515 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-07-03T15:41:20.551 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi028.front.sepia.ceph.com 2024-07-03T15:41:20.551 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-07-03T15:41:20.822 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:41:20.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:20 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/597501252' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-07-03T15:41:20.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:20 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/597501252' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-07-03T15:41:21.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:21 smithi028 ceph-mon[29148]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:21.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:21 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:41:21.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:21 smithi148 ceph-mon[36056]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:21.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:21 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:41:22.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.557+0000 7fb8c27cf640 1 -- 172.21.15.28:0/3011279028 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8bc0fc510 msgr2=0x7fb8bc0fc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:22.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.557+0000 7fb8c27cf640 1 --2- 172.21.15.28:0/3011279028 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8bc0fc510 0x7fb8bc0fc980 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb8b00048d0 tx=0x7fb8b0009da0 comp rx=0 tx=0).stop 2024-07-03T15:41:22.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.558+0000 7fb8c27cf640 1 -- 172.21.15.28:0/3011279028 shutdown_connections 2024-07-03T15:41:22.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.558+0000 7fb8c27cf640 1 --2- 172.21.15.28:0/3011279028 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8bc0fc510 0x7fb8bc0fc980 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:22.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.558+0000 7fb8c27cf640 1 --2- 172.21.15.28:0/3011279028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8bc0fb240 0x7fb8bc0fb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:22.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.558+0000 7fb8c27cf640 1 -- 172.21.15.28:0/3011279028 >> 172.21.15.28:0/3011279028 conn(0x7fb8bc0f6a20 msgr2=0x7fb8bc0f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:41:22.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.558+0000 7fb8c27cf640 1 -- 172.21.15.28:0/3011279028 shutdown_connections 2024-07-03T15:41:22.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.558+0000 7fb8c27cf640 1 -- 172.21.15.28:0/3011279028 wait complete. 2024-07-03T15:41:22.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.558+0000 7fb8c27cf640 1 Processor -- start 2024-07-03T15:41:22.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.558+0000 7fb8c27cf640 1 -- start start 2024-07-03T15:41:22.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.559+0000 7fb8c27cf640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8bc0fb240 0x7fb8bc10f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:22.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.559+0000 7fb8c27cf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8bc0fc510 0x7fb8bc10f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:22.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.559+0000 7fb8c27cf640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8bc10fd30 con 0x7fb8bc0fb240 2024-07-03T15:41:22.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.559+0000 7fb8c27cf640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8bc110550 con 0x7fb8bc0fc510 2024-07-03T15:41:22.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.559+0000 7fb8bbfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8bc0fb240 0x7fb8bc10f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:22.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.559+0000 7fb8bbfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8bc0fb240 0x7fb8bc10f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46244/0 (socket says 172.21.15.28:46244) 2024-07-03T15:41:22.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.559+0000 7fb8bbfff640 1 -- 172.21.15.28:0/319256082 learned_addr learned my addr 172.21.15.28:0/319256082 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:41:22.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.559+0000 7fb8bb7fe640 1 --2- 172.21.15.28:0/319256082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8bc0fc510 0x7fb8bc10f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:22.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.559+0000 7fb8bbfff640 1 -- 172.21.15.28:0/319256082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8bc0fc510 msgr2=0x7fb8bc10f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:22.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.559+0000 7fb8bbfff640 1 --2- 172.21.15.28:0/319256082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8bc0fc510 0x7fb8bc10f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:22.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.559+0000 7fb8bbfff640 1 -- 172.21.15.28:0/319256082 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b0005040 con 0x7fb8bc0fb240 2024-07-03T15:41:22.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.559+0000 7fb8bbfff640 1 --2- 172.21.15.28:0/319256082 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8bc0fb240 0x7fb8bc10f1a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac00eb70 tx=0x7fb8ac00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:41:22.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.560+0000 7fb8b8ff9640 1 -- 172.21.15.28:0/319256082 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8ac007c30 con 0x7fb8bc0fb240 2024-07-03T15:41:22.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.560+0000 7fb8c27cf640 1 -- 172.21.15.28:0/319256082 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8bc110780 con 0x7fb8bc0fb240 2024-07-03T15:41:22.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.560+0000 7fb8b8ff9640 1 -- 172.21.15.28:0/319256082 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb8ac00f040 con 0x7fb8bc0fb240 2024-07-03T15:41:22.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.560+0000 7fb8b8ff9640 1 -- 172.21.15.28:0/319256082 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8ac00cad0 con 0x7fb8bc0fb240 2024-07-03T15:41:22.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.560+0000 7fb8c27cf640 1 -- 172.21.15.28:0/319256082 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8bc110cf0 con 0x7fb8bc0fb240 2024-07-03T15:41:22.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.561+0000 7fb8c27cf640 1 -- 172.21.15.28:0/319256082 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8bc1045c0 con 0x7fb8bc0fb240 2024-07-03T15:41:22.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.561+0000 7fb8b8ff9640 1 -- 172.21.15.28:0/319256082 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7fb8ac00cc50 con 0x7fb8bc0fb240 2024-07-03T15:41:22.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.562+0000 7fb8b8ff9640 1 --2- 172.21.15.28:0/319256082 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fb8a4067190 0x7fb8a4069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:22.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.562+0000 7fb8b8ff9640 1 -- 172.21.15.28:0/319256082 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fb8ac084410 con 0x7fb8bc0fb240 2024-07-03T15:41:22.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.562+0000 7fb8bb7fe640 1 --2- 172.21.15.28:0/319256082 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fb8a4067190 0x7fb8a4069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:22.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.562+0000 7fb8bb7fe640 1 --2- 172.21.15.28:0/319256082 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fb8a4067190 0x7fb8a4069640 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0009f80 tx=0x7fb8b0035040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:41:22.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.566+0000 7fb8b8ff9640 1 -- 172.21.15.28:0/319256082 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb8ac051aa0 con 0x7fb8bc0fb240 2024-07-03T15:41:22.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.742+0000 7fb8c27cf640 1 -- 172.21.15.28:0/319256082 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7fb8bc103ef0 con 0x7fb8bc0fb240 2024-07-03T15:41:22.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.746+0000 7fb8b8ff9640 1 -- 172.21.15.28:0/319256082 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7fb8ac0550c0 con 0x7fb8bc0fb240 2024-07-03T15:41:22.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.748+0000 7fb8c27cf640 1 -- 172.21.15.28:0/319256082 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fb8a4067190 msgr2=0x7fb8a4069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:22.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.748+0000 7fb8c27cf640 1 --2- 172.21.15.28:0/319256082 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fb8a4067190 0x7fb8a4069640 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0009f80 tx=0x7fb8b0035040 comp rx=0 tx=0).stop 2024-07-03T15:41:22.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.749+0000 7fb8c27cf640 1 -- 172.21.15.28:0/319256082 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8bc0fb240 msgr2=0x7fb8bc10f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:22.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.749+0000 7fb8c27cf640 1 --2- 172.21.15.28:0/319256082 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8bc0fb240 0x7fb8bc10f1a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac00eb70 tx=0x7fb8ac00b040 comp rx=0 tx=0).stop 2024-07-03T15:41:22.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.749+0000 7fb8c27cf640 1 -- 172.21.15.28:0/319256082 shutdown_connections 2024-07-03T15:41:22.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.749+0000 7fb8c27cf640 1 --2- 172.21.15.28:0/319256082 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fb8a4067190 0x7fb8a4069640 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:22.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.749+0000 7fb8c27cf640 1 --2- 172.21.15.28:0/319256082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8bc0fc510 0x7fb8bc10f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:22.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.749+0000 7fb8c27cf640 1 --2- 172.21.15.28:0/319256082 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8bc0fb240 0x7fb8bc10f1a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:22.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.749+0000 7fb8c27cf640 1 -- 172.21.15.28:0/319256082 >> 172.21.15.28:0/319256082 conn(0x7fb8bc0f6a20 msgr2=0x7fb8bc065320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:41:22.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.749+0000 7fb8c27cf640 1 -- 172.21.15.28:0/319256082 shutdown_connections 2024-07-03T15:41:22.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:22.749+0000 7fb8c27cf640 1 -- 172.21.15.28:0/319256082 wait complete. 2024-07-03T15:41:22.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:22 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:41:22.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:22 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:41:22.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:22 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:41:22.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:22 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:41:22.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:22 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:41:22.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:22 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:41:23.493 INFO:teuthology.run_tasks:Running task print... 2024-07-03T15:41:23.504 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-07-03T15:41:23.505 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-07-03T15:41:23.532 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi028.front.sepia.ceph.com 2024-07-03T15:41:23.533 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- bash -c 'ceph orch status' 2024-07-03T15:41:23.802 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:41:23.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:23 smithi028 ceph-mon[29148]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:23.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:23 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/319256082' entity='client.admin' 2024-07-03T15:41:23.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:23 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:41:23.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:23 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:41:23.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:23 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:41:23.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:23 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:41:23.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:23 smithi148 ceph-mon[36056]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:23.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:23 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/319256082' entity='client.admin' 2024-07-03T15:41:23.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:23 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:41:23.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:23 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:41:23.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:23 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:41:23.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:23 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:41:25.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.525+0000 7fd3e5852640 1 -- 172.21.15.28:0/3387177586 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd3e00fd600 msgr2=0x7fd3e00fda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:25.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.525+0000 7fd3e5852640 1 --2- 172.21.15.28:0/3387177586 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd3e00fd600 0x7fd3e00fda70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd3d40048d0 tx=0x7fd3d4009da0 comp rx=0 tx=0).stop 2024-07-03T15:41:25.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.525+0000 7fd3e5852640 1 -- 172.21.15.28:0/3387177586 shutdown_connections 2024-07-03T15:41:25.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.525+0000 7fd3e5852640 1 --2- 172.21.15.28:0/3387177586 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd3e00fd600 0x7fd3e00fda70 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:25.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.525+0000 7fd3e5852640 1 --2- 172.21.15.28:0/3387177586 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3e00fc330 0x7fd3e00fc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:25.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.525+0000 7fd3e5852640 1 -- 172.21.15.28:0/3387177586 >> 172.21.15.28:0/3387177586 conn(0x7fd3e00f7af0 msgr2=0x7fd3e00f9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:41:25.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.525+0000 7fd3e5852640 1 -- 172.21.15.28:0/3387177586 shutdown_connections 2024-07-03T15:41:25.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.525+0000 7fd3e5852640 1 -- 172.21.15.28:0/3387177586 wait complete. 2024-07-03T15:41:25.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.526+0000 7fd3e5852640 1 Processor -- start 2024-07-03T15:41:25.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.526+0000 7fd3e5852640 1 -- start start 2024-07-03T15:41:25.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.526+0000 7fd3e5852640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd3e00fc330 0x7fd3e010cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:25.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.526+0000 7fd3e5852640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3e00fd600 0x7fd3e010d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:25.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.526+0000 7fd3e5852640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e010e370 con 0x7fd3e00fc330 2024-07-03T15:41:25.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.526+0000 7fd3e5852640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e010e4e0 con 0x7fd3e00fd600 2024-07-03T15:41:25.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.527+0000 7fd3deffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd3e00fc330 0x7fd3e010cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:25.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.527+0000 7fd3deffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd3e00fc330 0x7fd3e010cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46272/0 (socket says 172.21.15.28:46272) 2024-07-03T15:41:25.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.527+0000 7fd3deffd640 1 -- 172.21.15.28:0/1958693235 learned_addr learned my addr 172.21.15.28:0/1958693235 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:41:25.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.527+0000 7fd3de7fc640 1 --2- 172.21.15.28:0/1958693235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3e00fd600 0x7fd3e010d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:25.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.527+0000 7fd3deffd640 1 -- 172.21.15.28:0/1958693235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3e00fd600 msgr2=0x7fd3e010d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:25.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.527+0000 7fd3deffd640 1 --2- 172.21.15.28:0/1958693235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3e00fd600 0x7fd3e010d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:25.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.527+0000 7fd3deffd640 1 -- 172.21.15.28:0/1958693235 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d4005040 con 0x7fd3e00fc330 2024-07-03T15:41:25.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.527+0000 7fd3deffd640 1 --2- 172.21.15.28:0/1958693235 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd3e00fc330 0x7fd3e010cfc0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd3d000b790 tx=0x7fd3d00091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:41:25.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.527+0000 7fd3c77fe640 1 -- 172.21.15.28:0/1958693235 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3d000c700 con 0x7fd3e00fc330 2024-07-03T15:41:25.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.527+0000 7fd3c77fe640 1 -- 172.21.15.28:0/1958693235 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd3d000cd30 con 0x7fd3e00fc330 2024-07-03T15:41:25.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.527+0000 7fd3c77fe640 1 -- 172.21.15.28:0/1958693235 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3d0013370 con 0x7fd3e00fc330 2024-07-03T15:41:25.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.527+0000 7fd3e5852640 1 -- 172.21.15.28:0/1958693235 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3e010e7c0 con 0x7fd3e00fc330 2024-07-03T15:41:25.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.527+0000 7fd3e5852640 1 -- 172.21.15.28:0/1958693235 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3e010ece0 con 0x7fd3e00fc330 2024-07-03T15:41:25.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.528+0000 7fd3e5852640 1 -- 172.21.15.28:0/1958693235 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3e0105710 con 0x7fd3e00fc330 2024-07-03T15:41:25.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.529+0000 7fd3c77fe640 1 -- 172.21.15.28:0/1958693235 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7fd3d0012070 con 0x7fd3e00fc330 2024-07-03T15:41:25.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.529+0000 7fd3c77fe640 1 --2- 172.21.15.28:0/1958693235 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fd3c8067190 0x7fd3c8069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:25.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.529+0000 7fd3c77fe640 1 -- 172.21.15.28:0/1958693235 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fd3d00840d0 con 0x7fd3e00fc330 2024-07-03T15:41:25.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.529+0000 7fd3de7fc640 1 --2- 172.21.15.28:0/1958693235 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fd3c8067190 0x7fd3c8069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:25.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.530+0000 7fd3de7fc640 1 --2- 172.21.15.28:0/1958693235 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fd3c8067190 0x7fd3c8069640 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd3e010cd70 tx=0x7fd3d4035040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:41:25.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.533+0000 7fd3c77fe640 1 -- 172.21.15.28:0/1958693235 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd3d00516b0 con 0x7fd3e00fc330 2024-07-03T15:41:25.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.720+0000 7fd3e5852640 1 -- 172.21.15.28:0/1958693235 --> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd3e010ef60 con 0x7fd3c8067190 2024-07-03T15:41:25.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.721+0000 7fd3c77fe640 1 -- 172.21.15.28:0/1958693235 <== mgr.14210 v2:172.21.15.28:6800/421057178 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fd3e010ef60 con 0x7fd3c8067190 2024-07-03T15:41:25.725 INFO:teuthology.orchestra.run.smithi028.stdout:Backend: cephadm 2024-07-03T15:41:25.725 INFO:teuthology.orchestra.run.smithi028.stdout:Available: Yes 2024-07-03T15:41:25.725 INFO:teuthology.orchestra.run.smithi028.stdout:Paused: No 2024-07-03T15:41:25.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.723+0000 7fd3e5852640 1 -- 172.21.15.28:0/1958693235 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fd3c8067190 msgr2=0x7fd3c8069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:25.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.723+0000 7fd3e5852640 1 --2- 172.21.15.28:0/1958693235 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fd3c8067190 0x7fd3c8069640 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd3e010cd70 tx=0x7fd3d4035040 comp rx=0 tx=0).stop 2024-07-03T15:41:25.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.723+0000 7fd3e5852640 1 -- 172.21.15.28:0/1958693235 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd3e00fc330 msgr2=0x7fd3e010cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:25.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.723+0000 7fd3e5852640 1 --2- 172.21.15.28:0/1958693235 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd3e00fc330 0x7fd3e010cfc0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd3d000b790 tx=0x7fd3d00091d0 comp rx=0 tx=0).stop 2024-07-03T15:41:25.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.723+0000 7fd3e5852640 1 -- 172.21.15.28:0/1958693235 shutdown_connections 2024-07-03T15:41:25.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.723+0000 7fd3e5852640 1 --2- 172.21.15.28:0/1958693235 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fd3c8067190 0x7fd3c8069640 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:25.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.723+0000 7fd3e5852640 1 --2- 172.21.15.28:0/1958693235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3e00fd600 0x7fd3e010d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:25.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.723+0000 7fd3e5852640 1 --2- 172.21.15.28:0/1958693235 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd3e00fc330 0x7fd3e010cfc0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:25.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.723+0000 7fd3e5852640 1 -- 172.21.15.28:0/1958693235 >> 172.21.15.28:0/1958693235 conn(0x7fd3e00f7af0 msgr2=0x7fd3e00f9180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:41:25.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.723+0000 7fd3e5852640 1 -- 172.21.15.28:0/1958693235 shutdown_connections 2024-07-03T15:41:25.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:25.723+0000 7fd3e5852640 1 -- 172.21.15.28:0/1958693235 wait complete. 2024-07-03T15:41:25.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:25 smithi028 ceph-mon[29148]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:25.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:25 smithi148 ceph-mon[36056]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:26.322 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- bash -c 'ceph orch ps' 2024-07-03T15:41:26.587 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:41:26.949 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:26 smithi028 ceph-mon[29148]: from='client.14422 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:41:26.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:26 smithi148 ceph-mon[36056]: from='client.14422 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:41:27.711 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:27 smithi028 ceph-mon[29148]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:27.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:27 smithi148 ceph-mon[36056]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:28.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.184+0000 7f730a073640 1 -- 172.21.15.28:0/1851256758 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73040fc550 msgr2=0x7f73040fc9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:28.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.184+0000 7f730a073640 1 --2- 172.21.15.28:0/1851256758 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73040fc550 0x7f73040fc9c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f72f40048d0 tx=0x7f72f4009da0 comp rx=0 tx=0).stop 2024-07-03T15:41:28.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.184+0000 7f730a073640 1 -- 172.21.15.28:0/1851256758 shutdown_connections 2024-07-03T15:41:28.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.184+0000 7f730a073640 1 --2- 172.21.15.28:0/1851256758 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73040fc550 0x7f73040fc9c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:28.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.184+0000 7f730a073640 1 --2- 172.21.15.28:0/1851256758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73040fb280 0x7f73040fb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:28.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.184+0000 7f730a073640 1 -- 172.21.15.28:0/1851256758 >> 172.21.15.28:0/1851256758 conn(0x7f73040f6a40 msgr2=0x7f73040f8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:41:28.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.184+0000 7f730a073640 1 -- 172.21.15.28:0/1851256758 shutdown_connections 2024-07-03T15:41:28.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.184+0000 7f730a073640 1 -- 172.21.15.28:0/1851256758 wait complete. 2024-07-03T15:41:28.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.185+0000 7f730a073640 1 Processor -- start 2024-07-03T15:41:28.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.185+0000 7f730a073640 1 -- start start 2024-07-03T15:41:28.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.185+0000 7f730a073640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73040fb280 0x7f730410d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:28.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.185+0000 7f730a073640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73040fc550 0x7f730410d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:28.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.185+0000 7f730a073640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730410e420 con 0x7f73040fc550 2024-07-03T15:41:28.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.185+0000 7f730a073640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730410e590 con 0x7f73040fb280 2024-07-03T15:41:28.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.185+0000 7f7302ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73040fc550 0x7f730410d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:28.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.185+0000 7f7302ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73040fc550 0x7f730410d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46298/0 (socket says 172.21.15.28:46298) 2024-07-03T15:41:28.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.185+0000 7f7302ffd640 1 -- 172.21.15.28:0/1204074160 learned_addr learned my addr 172.21.15.28:0/1204074160 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:41:28.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.185+0000 7f7302ffd640 1 -- 172.21.15.28:0/1204074160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73040fb280 msgr2=0x7f730410d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:28.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.185+0000 7f73037fe640 1 --2- 172.21.15.28:0/1204074160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73040fb280 0x7f730410d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:28.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.185+0000 7f7302ffd640 1 --2- 172.21.15.28:0/1204074160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73040fb280 0x7f730410d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:28.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.185+0000 7f7302ffd640 1 -- 172.21.15.28:0/1204074160 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f4005040 con 0x7f73040fc550 2024-07-03T15:41:28.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.186+0000 7f7302ffd640 1 --2- 172.21.15.28:0/1204074160 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73040fc550 0x7f730410d5b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f72f4003080 tx=0x7f72f4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:41:28.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.186+0000 7f72fbfff640 1 -- 172.21.15.28:0/1204074160 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72f4038070 con 0x7f73040fc550 2024-07-03T15:41:28.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.186+0000 7f730a073640 1 -- 172.21.15.28:0/1204074160 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f730410e810 con 0x7f73040fc550 2024-07-03T15:41:28.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.186+0000 7f72fbfff640 1 -- 172.21.15.28:0/1204074160 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f72f4032970 con 0x7f73040fc550 2024-07-03T15:41:28.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.186+0000 7f72fbfff640 1 -- 172.21.15.28:0/1204074160 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72f4033820 con 0x7f73040fc550 2024-07-03T15:41:28.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.186+0000 7f730a073640 1 -- 172.21.15.28:0/1204074160 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f730410ecd0 con 0x7f73040fc550 2024-07-03T15:41:28.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.187+0000 7f730a073640 1 -- 172.21.15.28:0/1204074160 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73040fa6e0 con 0x7f73040fc550 2024-07-03T15:41:28.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.187+0000 7f72fbfff640 1 -- 172.21.15.28:0/1204074160 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f72f4030020 con 0x7f73040fc550 2024-07-03T15:41:28.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.188+0000 7f72fbfff640 1 --2- 172.21.15.28:0/1204074160 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f72f0067190 0x7f72f0069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:28.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.188+0000 7f72fbfff640 1 -- 172.21.15.28:0/1204074160 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f72f40a47b0 con 0x7f73040fc550 2024-07-03T15:41:28.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.188+0000 7f73037fe640 1 --2- 172.21.15.28:0/1204074160 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f72f0067190 0x7f72f0069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:28.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.188+0000 7f73037fe640 1 --2- 172.21.15.28:0/1204074160 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f72f0067190 0x7f72f0069640 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f72ec00a900 tx=0x7f72ec008040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:41:28.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.192+0000 7f72fbfff640 1 -- 172.21.15.28:0/1204074160 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f72f4071d90 con 0x7f73040fc550 2024-07-03T15:41:28.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.374+0000 7f730a073640 1 -- 172.21.15.28:0/1204074160 --> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7304103e50 con 0x7f72f0067190 2024-07-03T15:41:28.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.382+0000 7f72fbfff640 1 -- 172.21.15.28:0/1204074160 <== mgr.14210 v2:172.21.15.28:6800/421057178 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2736 (secure 0 0 0) 0x7f7304103e50 con 0x7f72f0067190 2024-07-03T15:41:28.385 INFO:teuthology.orchestra.run.smithi028.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-03T15:41:28.386 INFO:teuthology.orchestra.run.smithi028.stdout:alertmanager.smithi028 smithi028 *:9093,9094 running (3m) 2m ago 5m 26.1M - 0.25.0 c8568f914cd2 b9a1a484a0a2 2024-07-03T15:41:28.386 INFO:teuthology.orchestra.run.smithi028.stdout:crash.smithi028 smithi028 running (5m) 2m ago 5m 6899k - 17.2.7-1047-gd4b063ab 4ca399d9e65d 535660e448f9 2024-07-03T15:41:28.386 INFO:teuthology.orchestra.run.smithi028.stdout:crash.smithi148 smithi148 running (4m) 71s ago 4m 6899k - 17.2.7-1047-gd4b063ab 4ca399d9e65d 4bb7675895e7 2024-07-03T15:41:28.386 INFO:teuthology.orchestra.run.smithi028.stdout:grafana.smithi028 smithi028 *:3000 running (3m) 2m ago 4m 80.7M - 9.4.7 954c08fa6188 07e0ae3c15aa 2024-07-03T15:41:28.386 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.smithi028.vtqrtz smithi028 *:9283 running (6m) 2m ago 6m 438M - 17.2.7-1047-gd4b063ab 4ca399d9e65d 0d26e74cb154 2024-07-03T15:41:28.386 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.smithi148.rjzsar smithi148 *:8443,9283 running (4m) 71s ago 4m 400M - 17.2.7-1047-gd4b063ab 4ca399d9e65d c637b3ae9fce 2024-07-03T15:41:28.386 INFO:teuthology.orchestra.run.smithi028.stdout:mon.smithi028 smithi028 running (6m) 2m ago 6m 42.9M 2048M 17.2.7-1047-gd4b063ab 4ca399d9e65d 7461f9bef4e0 2024-07-03T15:41:28.386 INFO:teuthology.orchestra.run.smithi028.stdout:mon.smithi148 smithi148 running (4m) 71s ago 4m 38.7M 2048M 17.2.7-1047-gd4b063ab 4ca399d9e65d 4c6c8ed2ab85 2024-07-03T15:41:28.386 INFO:teuthology.orchestra.run.smithi028.stdout:node-exporter.smithi028 smithi028 *:9100 running (5m) 2m ago 5m 16.0M - 1.5.0 0da6a335fe13 68318a0ca520 2024-07-03T15:41:28.386 INFO:teuthology.orchestra.run.smithi028.stdout:node-exporter.smithi148 smithi148 *:9100 running (4m) 71s ago 4m 17.0M - 1.5.0 0da6a335fe13 4994e60bd57c 2024-07-03T15:41:28.386 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0 smithi028 running (3m) 2m ago 3m 42.7M 4289M 17.2.7-1047-gd4b063ab 4ca399d9e65d b18089a439ba 2024-07-03T15:41:28.386 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1 smithi028 running (2m) 2m ago 2m 38.4M 4289M 17.2.7-1047-gd4b063ab 4ca399d9e65d 87d415dde231 2024-07-03T15:41:28.386 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2 smithi028 running (2m) 2m ago 2m 40.1M 4289M 17.2.7-1047-gd4b063ab 4ca399d9e65d b2cbb312212d 2024-07-03T15:41:28.387 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3 smithi148 running (2m) 71s ago 2m 39.4M 5312M 17.2.7-1047-gd4b063ab 4ca399d9e65d 5be69a27df31 2024-07-03T15:41:28.387 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4 smithi148 running (100s) 71s ago 100s 38.8M 5312M 17.2.7-1047-gd4b063ab 4ca399d9e65d 54f7c26c77ca 2024-07-03T15:41:28.387 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5 smithi148 running (76s) 71s ago 75s 11.2M 5312M 17.2.7-1047-gd4b063ab 4ca399d9e65d d185c6905384 2024-07-03T15:41:28.387 INFO:teuthology.orchestra.run.smithi028.stdout:prometheus.smithi028 smithi028 *:9095 running (3m) 2m ago 4m 39.2M - 2.43.0 a07b618ecd1d 97f838e118a3 2024-07-03T15:41:28.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.384+0000 7f730a073640 1 -- 172.21.15.28:0/1204074160 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f72f0067190 msgr2=0x7f72f0069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:28.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.384+0000 7f730a073640 1 --2- 172.21.15.28:0/1204074160 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f72f0067190 0x7f72f0069640 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f72ec00a900 tx=0x7f72ec008040 comp rx=0 tx=0).stop 2024-07-03T15:41:28.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.384+0000 7f730a073640 1 -- 172.21.15.28:0/1204074160 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73040fc550 msgr2=0x7f730410d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:28.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.384+0000 7f730a073640 1 --2- 172.21.15.28:0/1204074160 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73040fc550 0x7f730410d5b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f72f4003080 tx=0x7f72f4009da0 comp rx=0 tx=0).stop 2024-07-03T15:41:28.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.384+0000 7f730a073640 1 -- 172.21.15.28:0/1204074160 shutdown_connections 2024-07-03T15:41:28.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.384+0000 7f730a073640 1 --2- 172.21.15.28:0/1204074160 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f72f0067190 0x7f72f0069640 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:28.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.384+0000 7f730a073640 1 --2- 172.21.15.28:0/1204074160 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73040fc550 0x7f730410d5b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:28.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.384+0000 7f730a073640 1 --2- 172.21.15.28:0/1204074160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73040fb280 0x7f730410d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:28.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.384+0000 7f730a073640 1 -- 172.21.15.28:0/1204074160 >> 172.21.15.28:0/1204074160 conn(0x7f73040f6a40 msgr2=0x7f7304065ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:41:28.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.385+0000 7f730a073640 1 -- 172.21.15.28:0/1204074160 shutdown_connections 2024-07-03T15:41:28.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:28.385+0000 7f730a073640 1 -- 172.21.15.28:0/1204074160 wait complete. 2024-07-03T15:41:29.048 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- bash -c 'ceph orch ls' 2024-07-03T15:41:29.318 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:41:29.705 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:29 smithi028 ceph-mon[29148]: from='client.14426 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:41:29.705 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:29 smithi028 ceph-mon[29148]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:29.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:29 smithi148 ceph-mon[36056]: from='client.14426 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:41:29.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:29 smithi148 ceph-mon[36056]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:30.886 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:30 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:41:30.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:30.881+0000 7f303be91640 1 -- 172.21.15.28:0/2199260688 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f30340fd610 msgr2=0x7f30340fda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:30.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:30.881+0000 7f303be91640 1 --2- 172.21.15.28:0/2199260688 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f30340fd610 0x7f30340fda80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f30240048d0 tx=0x7f3024009da0 comp rx=0 tx=0).stop 2024-07-03T15:41:30.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:30.881+0000 7f303be91640 1 -- 172.21.15.28:0/2199260688 shutdown_connections 2024-07-03T15:41:30.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:30.881+0000 7f303be91640 1 --2- 172.21.15.28:0/2199260688 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f30340fd610 0x7f30340fda80 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:30.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:30.881+0000 7f303be91640 1 --2- 172.21.15.28:0/2199260688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30340fc340 0x7f30340fc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:30.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:30.881+0000 7f303be91640 1 -- 172.21.15.28:0/2199260688 >> 172.21.15.28:0/2199260688 conn(0x7f30340f7b20 msgr2=0x7f30340f9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:41:30.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:30.881+0000 7f303be91640 1 -- 172.21.15.28:0/2199260688 shutdown_connections 2024-07-03T15:41:30.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:30.881+0000 7f303be91640 1 -- 172.21.15.28:0/2199260688 wait complete. 2024-07-03T15:41:30.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:30.882+0000 7f303be91640 1 Processor -- start 2024-07-03T15:41:30.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:30.882+0000 7f303be91640 1 -- start start 2024-07-03T15:41:30.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:30.882+0000 7f303be91640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f30340fc340 0x7f303410cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:30.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:30.882+0000 7f303be91640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30340fd610 0x7f303410d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:30.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:30.882+0000 7f303be91640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303410dae0 con 0x7f30340fc340 2024-07-03T15:41:30.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:30.882+0000 7f303be91640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303410e300 con 0x7f30340fd610 2024-07-03T15:41:30.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:30.882+0000 7f3039c2d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f30340fc340 0x7f303410cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:30.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:30.882+0000 7f3039c2d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f30340fc340 0x7f303410cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51906/0 (socket says 172.21.15.28:51906) 2024-07-03T15:41:30.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:30.882+0000 7f3039c2d640 1 -- 172.21.15.28:0/410292277 learned_addr learned my addr 172.21.15.28:0/410292277 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:41:30.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:30.882+0000 7f303942c640 1 --2- 172.21.15.28:0/410292277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30340fd610 0x7f303410d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:30.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:30.882+0000 7f3039c2d640 1 -- 172.21.15.28:0/410292277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30340fd610 msgr2=0x7f303410d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:30.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:30.882+0000 7f3039c2d640 1 --2- 172.21.15.28:0/410292277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30340fd610 0x7f303410d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:30.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:30.882+0000 7f3039c2d640 1 -- 172.21.15.28:0/410292277 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3024005040 con 0x7f30340fc340 2024-07-03T15:41:30.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:30.883+0000 7f3039c2d640 1 --2- 172.21.15.28:0/410292277 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f30340fc340 0x7f303410cf50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f303000c830 tx=0x7f3030004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:41:30.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:30.883+0000 7f302a7fc640 1 -- 172.21.15.28:0/410292277 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f303000a7d0 con 0x7f30340fc340 2024-07-03T15:41:30.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:30.883+0000 7f302a7fc640 1 -- 172.21.15.28:0/410292277 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f303000ae00 con 0x7f30340fc340 2024-07-03T15:41:30.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:30.883+0000 7f303be91640 1 -- 172.21.15.28:0/410292277 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f303410e590 con 0x7f30340fc340 2024-07-03T15:41:30.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:30.883+0000 7f302a7fc640 1 -- 172.21.15.28:0/410292277 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30300113a0 con 0x7f30340fc340 2024-07-03T15:41:30.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:30.883+0000 7f303be91640 1 -- 172.21.15.28:0/410292277 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f303410eb00 con 0x7f30340fc340 2024-07-03T15:41:30.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:30.884+0000 7f303be91640 1 -- 172.21.15.28:0/410292277 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30341056a0 con 0x7f30340fc340 2024-07-03T15:41:30.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:30.884+0000 7f302a7fc640 1 -- 172.21.15.28:0/410292277 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f3030012400 con 0x7f30340fc340 2024-07-03T15:41:30.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:30.885+0000 7f302a7fc640 1 --2- 172.21.15.28:0/410292277 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3020067190 0x7f3020069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:30.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:30.885+0000 7f302a7fc640 1 -- 172.21.15.28:0/410292277 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f30300847f0 con 0x7f30340fc340 2024-07-03T15:41:30.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:30.885+0000 7f303942c640 1 --2- 172.21.15.28:0/410292277 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3020067190 0x7f3020069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:30.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:30.885+0000 7f303942c640 1 --2- 172.21.15.28:0/410292277 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3020067190 0x7f3020069640 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3024000c00 tx=0x7f3024035040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:41:30.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:30.889+0000 7f302a7fc640 1 -- 172.21.15.28:0/410292277 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3030051fb0 con 0x7f30340fc340 2024-07-03T15:41:30.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:30 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:41:31.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:31.070+0000 7f303be91640 1 -- 172.21.15.28:0/410292277 --> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f303410eee0 con 0x7f3020067190 2024-07-03T15:41:31.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:31.074+0000 7f302a7fc640 1 -- 172.21.15.28:0/410292277 <== mgr.14210 v2:172.21.15.28:6800/421057178 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1170 (secure 0 0 0) 0x7f303410eee0 con 0x7f3020067190 2024-07-03T15:41:31.077 INFO:teuthology.orchestra.run.smithi028.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-07-03T15:41:31.077 INFO:teuthology.orchestra.run.smithi028.stdout:alertmanager ?:9093,9094 1/1 2m ago 5m count:1 2024-07-03T15:41:31.077 INFO:teuthology.orchestra.run.smithi028.stdout:crash 2/2 2m ago 5m * 2024-07-03T15:41:31.077 INFO:teuthology.orchestra.run.smithi028.stdout:grafana ?:3000 1/1 2m ago 5m count:1 2024-07-03T15:41:31.077 INFO:teuthology.orchestra.run.smithi028.stdout:mgr 2/2 2m ago 5m count:2 2024-07-03T15:41:31.077 INFO:teuthology.orchestra.run.smithi028.stdout:mon 2/2 2m ago 5m smithi028:172.21.15.28=smithi028;smithi148:172.21.15.148=smithi148;count:2 2024-07-03T15:41:31.077 INFO:teuthology.orchestra.run.smithi028.stdout:node-exporter ?:9100 2/2 2m ago 5m * 2024-07-03T15:41:31.077 INFO:teuthology.orchestra.run.smithi028.stdout:osd 6 2m ago - 2024-07-03T15:41:31.078 INFO:teuthology.orchestra.run.smithi028.stdout:prometheus ?:9095 1/1 2m ago 5m count:1 2024-07-03T15:41:31.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:31.076+0000 7f303be91640 1 -- 172.21.15.28:0/410292277 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3020067190 msgr2=0x7f3020069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:31.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:31.076+0000 7f303be91640 1 --2- 172.21.15.28:0/410292277 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3020067190 0x7f3020069640 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3024000c00 tx=0x7f3024035040 comp rx=0 tx=0).stop 2024-07-03T15:41:31.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:31.076+0000 7f303be91640 1 -- 172.21.15.28:0/410292277 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f30340fc340 msgr2=0x7f303410cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:31.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:31.076+0000 7f303be91640 1 --2- 172.21.15.28:0/410292277 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f30340fc340 0x7f303410cf50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f303000c830 tx=0x7f3030004430 comp rx=0 tx=0).stop 2024-07-03T15:41:31.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:31.076+0000 7f303be91640 1 -- 172.21.15.28:0/410292277 shutdown_connections 2024-07-03T15:41:31.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:31.076+0000 7f303be91640 1 --2- 172.21.15.28:0/410292277 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3020067190 0x7f3020069640 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:31.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:31.076+0000 7f303be91640 1 --2- 172.21.15.28:0/410292277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30340fd610 0x7f303410d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:31.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:31.076+0000 7f303be91640 1 --2- 172.21.15.28:0/410292277 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f30340fc340 0x7f303410cf50 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:31.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:31.076+0000 7f303be91640 1 -- 172.21.15.28:0/410292277 >> 172.21.15.28:0/410292277 conn(0x7f30340f7b20 msgr2=0x7f30340f91e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:41:31.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:31.076+0000 7f303be91640 1 -- 172.21.15.28:0/410292277 shutdown_connections 2024-07-03T15:41:31.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:31.076+0000 7f303be91640 1 -- 172.21.15.28:0/410292277 wait complete. 2024-07-03T15:41:31.749 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:31 smithi028 ceph-mon[29148]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:31.781 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- bash -c 'ceph orch host ls' 2024-07-03T15:41:31.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:31 smithi148 ceph-mon[36056]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:32.052 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:41:32.847 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:32 smithi028 ceph-mon[29148]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:41:32.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:32 smithi148 ceph-mon[36056]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:41:33.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.643+0000 7fccd3734640 1 -- 172.21.15.28:0/1316377777 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcccc0fc370 msgr2=0x7fcccc0fc780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:33.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.643+0000 7fccd3734640 1 --2- 172.21.15.28:0/1316377777 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcccc0fc370 0x7fcccc0fc780 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fccc80048d0 tx=0x7fccc8009da0 comp rx=0 tx=0).stop 2024-07-03T15:41:33.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.643+0000 7fccd3734640 1 -- 172.21.15.28:0/1316377777 shutdown_connections 2024-07-03T15:41:33.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.643+0000 7fccd3734640 1 --2- 172.21.15.28:0/1316377777 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcccc0fd640 0x7fcccc0fdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:33.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.643+0000 7fccd3734640 1 --2- 172.21.15.28:0/1316377777 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcccc0fc370 0x7fcccc0fc780 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:33.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.643+0000 7fccd3734640 1 -- 172.21.15.28:0/1316377777 >> 172.21.15.28:0/1316377777 conn(0x7fcccc0f7b50 msgr2=0x7fcccc0f9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:41:33.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.643+0000 7fccd3734640 1 -- 172.21.15.28:0/1316377777 shutdown_connections 2024-07-03T15:41:33.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.643+0000 7fccd3734640 1 -- 172.21.15.28:0/1316377777 wait complete. 2024-07-03T15:41:33.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.644+0000 7fccd3734640 1 Processor -- start 2024-07-03T15:41:33.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.644+0000 7fccd3734640 1 -- start start 2024-07-03T15:41:33.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.644+0000 7fccd3734640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcccc0fc370 0x7fcccc10cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:33.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.644+0000 7fccd3734640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcccc0fd640 0x7fcccc10d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:33.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.644+0000 7fccd3734640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcccc10dad0 con 0x7fcccc0fd640 2024-07-03T15:41:33.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.644+0000 7fccd3734640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcccc10e2f0 con 0x7fcccc0fc370 2024-07-03T15:41:33.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.644+0000 7fccd0ccf640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcccc0fd640 0x7fcccc10d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:33.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.644+0000 7fccd0ccf640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcccc0fd640 0x7fcccc10d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51916/0 (socket says 172.21.15.28:51916) 2024-07-03T15:41:33.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.644+0000 7fccd0ccf640 1 -- 172.21.15.28:0/1213030364 learned_addr learned my addr 172.21.15.28:0/1213030364 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:41:33.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.644+0000 7fccd14d0640 1 --2- 172.21.15.28:0/1213030364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcccc0fc370 0x7fcccc10cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:33.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.644+0000 7fccd0ccf640 1 -- 172.21.15.28:0/1213030364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcccc0fc370 msgr2=0x7fcccc10cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:33.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.644+0000 7fccd0ccf640 1 --2- 172.21.15.28:0/1213030364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcccc0fc370 0x7fcccc10cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:33.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.644+0000 7fccd0ccf640 1 -- 172.21.15.28:0/1213030364 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc8005040 con 0x7fcccc0fd640 2024-07-03T15:41:33.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.645+0000 7fccd0ccf640 1 --2- 172.21.15.28:0/1213030364 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcccc0fd640 0x7fcccc10d480 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fccbc002a50 tx=0x7fccbc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:41:33.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.645+0000 7fccc1ffb640 1 -- 172.21.15.28:0/1213030364 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fccbc007c30 con 0x7fcccc0fd640 2024-07-03T15:41:33.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.645+0000 7fccc1ffb640 1 -- 172.21.15.28:0/1213030364 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fccbc00f040 con 0x7fcccc0fd640 2024-07-03T15:41:33.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.645+0000 7fccd3734640 1 -- 172.21.15.28:0/1213030364 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcccc10e580 con 0x7fcccc0fd640 2024-07-03T15:41:33.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.645+0000 7fccc1ffb640 1 -- 172.21.15.28:0/1213030364 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fccbc00db60 con 0x7fcccc0fd640 2024-07-03T15:41:33.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.645+0000 7fccd3734640 1 -- 172.21.15.28:0/1213030364 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcccc10eb20 con 0x7fcccc0fd640 2024-07-03T15:41:33.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.646+0000 7fccd3734640 1 -- 172.21.15.28:0/1213030364 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcccc0fb7d0 con 0x7fcccc0fd640 2024-07-03T15:41:33.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.646+0000 7fccc1ffb640 1 -- 172.21.15.28:0/1213030364 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7fccbc00dcc0 con 0x7fcccc0fd640 2024-07-03T15:41:33.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.647+0000 7fccc1ffb640 1 --2- 172.21.15.28:0/1213030364 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fccb8067190 0x7fccb8069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:33.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.647+0000 7fccc1ffb640 1 -- 172.21.15.28:0/1213030364 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fccbc084640 con 0x7fcccc0fd640 2024-07-03T15:41:33.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.647+0000 7fccd14d0640 1 --2- 172.21.15.28:0/1213030364 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fccb8067190 0x7fccb8069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:33.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.647+0000 7fccd14d0640 1 --2- 172.21.15.28:0/1213030364 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fccb8067190 0x7fccb8069640 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fccc800b020 tx=0x7fccc8035040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:41:33.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.651+0000 7fccc1ffb640 1 -- 172.21.15.28:0/1213030364 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fccbc051c20 con 0x7fcccc0fd640 2024-07-03T15:41:33.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.834+0000 7fccd3734640 1 -- 172.21.15.28:0/1213030364 --> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fcccc10ee50 con 0x7fccb8067190 2024-07-03T15:41:33.837 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:33 smithi028 ceph-mon[29148]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:33.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.835+0000 7fccc1ffb640 1 -- 172.21.15.28:0/1213030364 <== mgr.14210 v2:172.21.15.28:6800/421057178 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7fcccc10ee50 con 0x7fccb8067190 2024-07-03T15:41:33.838 INFO:teuthology.orchestra.run.smithi028.stdout:HOST ADDR LABELS STATUS 2024-07-03T15:41:33.838 INFO:teuthology.orchestra.run.smithi028.stdout:smithi028 172.21.15.28 2024-07-03T15:41:33.838 INFO:teuthology.orchestra.run.smithi028.stdout:smithi148 172.21.15.148 2024-07-03T15:41:33.839 INFO:teuthology.orchestra.run.smithi028.stdout:2 hosts in cluster 2024-07-03T15:41:33.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.838+0000 7fccd3734640 1 -- 172.21.15.28:0/1213030364 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fccb8067190 msgr2=0x7fccb8069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:33.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.838+0000 7fccd3734640 1 --2- 172.21.15.28:0/1213030364 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fccb8067190 0x7fccb8069640 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fccc800b020 tx=0x7fccc8035040 comp rx=0 tx=0).stop 2024-07-03T15:41:33.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.838+0000 7fccd3734640 1 -- 172.21.15.28:0/1213030364 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcccc0fd640 msgr2=0x7fcccc10d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:33.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.838+0000 7fccd3734640 1 --2- 172.21.15.28:0/1213030364 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcccc0fd640 0x7fcccc10d480 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fccbc002a50 tx=0x7fccbc004600 comp rx=0 tx=0).stop 2024-07-03T15:41:33.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.838+0000 7fccd3734640 1 -- 172.21.15.28:0/1213030364 shutdown_connections 2024-07-03T15:41:33.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.838+0000 7fccd3734640 1 --2- 172.21.15.28:0/1213030364 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fccb8067190 0x7fccb8069640 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:33.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.838+0000 7fccd3734640 1 --2- 172.21.15.28:0/1213030364 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcccc0fd640 0x7fcccc10d480 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:33.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.838+0000 7fccd3734640 1 --2- 172.21.15.28:0/1213030364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcccc0fc370 0x7fcccc10cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:33.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.838+0000 7fccd3734640 1 -- 172.21.15.28:0/1213030364 >> 172.21.15.28:0/1213030364 conn(0x7fcccc0f7b50 msgr2=0x7fcccc0f91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:41:33.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.838+0000 7fccd3734640 1 -- 172.21.15.28:0/1213030364 shutdown_connections 2024-07-03T15:41:33.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:33.838+0000 7fccd3734640 1 -- 172.21.15.28:0/1213030364 wait complete. 2024-07-03T15:41:33.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:33 smithi148 ceph-mon[36056]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:34.466 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- bash -c 'ceph orch device ls' 2024-07-03T15:41:34.736 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:41:35.930 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:35 smithi028 ceph-mon[29148]: from='client.14434 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:41:35.931 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:35 smithi028 ceph-mon[29148]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:35.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:35 smithi148 ceph-mon[36056]: from='client.14434 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:41:35.984 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:35 smithi148 ceph-mon[36056]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:36.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.254+0000 7f20307a5640 1 -- 172.21.15.28:0/678537605 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f20280fd640 msgr2=0x7f20280fdab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:36.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.254+0000 7f20307a5640 1 --2- 172.21.15.28:0/678537605 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f20280fd640 0x7f20280fdab0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f20180048d0 tx=0x7f2018009da0 comp rx=0 tx=0).stop 2024-07-03T15:41:36.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.254+0000 7f20307a5640 1 -- 172.21.15.28:0/678537605 shutdown_connections 2024-07-03T15:41:36.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.254+0000 7f20307a5640 1 --2- 172.21.15.28:0/678537605 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f20280fd640 0x7f20280fdab0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:36.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.254+0000 7f20307a5640 1 --2- 172.21.15.28:0/678537605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20280fc370 0x7f20280fc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:36.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.254+0000 7f20307a5640 1 -- 172.21.15.28:0/678537605 >> 172.21.15.28:0/678537605 conn(0x7f20280f7b30 msgr2=0x7f20280f9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:41:36.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.254+0000 7f20307a5640 1 -- 172.21.15.28:0/678537605 shutdown_connections 2024-07-03T15:41:36.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.254+0000 7f20307a5640 1 -- 172.21.15.28:0/678537605 wait complete. 2024-07-03T15:41:36.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.255+0000 7f20307a5640 1 Processor -- start 2024-07-03T15:41:36.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.255+0000 7f20307a5640 1 -- start start 2024-07-03T15:41:36.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.255+0000 7f20307a5640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f20280fc370 0x7f202810cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:36.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.255+0000 7f20307a5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20280fd640 0x7f202810d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:36.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.255+0000 7f20307a5640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202810daf0 con 0x7f20280fc370 2024-07-03T15:41:36.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.255+0000 7f20307a5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202810e2a0 con 0x7f20280fd640 2024-07-03T15:41:36.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.255+0000 7f202e541640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f20280fc370 0x7f202810cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:36.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.255+0000 7f202e541640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f20280fc370 0x7f202810cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51936/0 (socket says 172.21.15.28:51936) 2024-07-03T15:41:36.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.255+0000 7f202e541640 1 -- 172.21.15.28:0/196318603 learned_addr learned my addr 172.21.15.28:0/196318603 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:41:36.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.255+0000 7f202dd40640 1 --2- 172.21.15.28:0/196318603 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20280fd640 0x7f202810d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:36.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.255+0000 7f202e541640 1 -- 172.21.15.28:0/196318603 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20280fd640 msgr2=0x7f202810d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:36.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.255+0000 7f202e541640 1 --2- 172.21.15.28:0/196318603 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20280fd640 0x7f202810d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:36.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.255+0000 7f202e541640 1 -- 172.21.15.28:0/196318603 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2018005040 con 0x7f20280fc370 2024-07-03T15:41:36.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.256+0000 7f202e541640 1 --2- 172.21.15.28:0/196318603 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f20280fc370 0x7f202810cf60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f202400eb70 tx=0x7f202400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:41:36.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.256+0000 7f201effd640 1 -- 172.21.15.28:0/196318603 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2024007c30 con 0x7f20280fc370 2024-07-03T15:41:36.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.256+0000 7f201effd640 1 -- 172.21.15.28:0/196318603 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f202400f040 con 0x7f20280fc370 2024-07-03T15:41:36.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.256+0000 7f20307a5640 1 -- 172.21.15.28:0/196318603 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f202810e4d0 con 0x7f20280fc370 2024-07-03T15:41:36.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.256+0000 7f201effd640 1 -- 172.21.15.28:0/196318603 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f202400cad0 con 0x7f20280fc370 2024-07-03T15:41:36.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.256+0000 7f20307a5640 1 -- 172.21.15.28:0/196318603 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f202810ea40 con 0x7f20280fc370 2024-07-03T15:41:36.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.257+0000 7f20307a5640 1 -- 172.21.15.28:0/196318603 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20281056f0 con 0x7f20280fc370 2024-07-03T15:41:36.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.257+0000 7f201effd640 1 -- 172.21.15.28:0/196318603 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f202400cc50 con 0x7f20280fc370 2024-07-03T15:41:36.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.258+0000 7f201effd640 1 --2- 172.21.15.28:0/196318603 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f2014067190 0x7f2014069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:36.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.258+0000 7f201effd640 1 -- 172.21.15.28:0/196318603 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f2024084410 con 0x7f20280fc370 2024-07-03T15:41:36.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.258+0000 7f202dd40640 1 --2- 172.21.15.28:0/196318603 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f2014067190 0x7f2014069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:36.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.258+0000 7f202dd40640 1 --2- 172.21.15.28:0/196318603 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f2014067190 0x7f2014069640 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2018009f80 tx=0x7f2018035040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:41:36.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.262+0000 7f201effd640 1 -- 172.21.15.28:0/196318603 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2024051aa0 con 0x7f20280fc370 2024-07-03T15:41:36.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.445+0000 7f20307a5640 1 -- 172.21.15.28:0/196318603 --> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f202810ed70 con 0x7f2014067190 2024-07-03T15:41:36.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.446+0000 7f201effd640 1 -- 172.21.15.28:0/196318603 <== mgr.14210 v2:172.21.15.28:6800/421057178 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7f202810ed70 con 0x7f2014067190 2024-07-03T15:41:36.449 INFO:teuthology.orchestra.run.smithi028.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-07-03T15:41:36.450 INFO:teuthology.orchestra.run.smithi028.stdout:smithi028 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB747401F4480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-07-03T15:41:36.450 INFO:teuthology.orchestra.run.smithi028.stdout:smithi148 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233005G400BGN 372G No 75s ago Has a FileSystem, LVM detected 2024-07-03T15:41:36.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.448+0000 7f20307a5640 1 -- 172.21.15.28:0/196318603 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f2014067190 msgr2=0x7f2014069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:36.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.448+0000 7f20307a5640 1 --2- 172.21.15.28:0/196318603 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f2014067190 0x7f2014069640 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2018009f80 tx=0x7f2018035040 comp rx=0 tx=0).stop 2024-07-03T15:41:36.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.448+0000 7f20307a5640 1 -- 172.21.15.28:0/196318603 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f20280fc370 msgr2=0x7f202810cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:36.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.448+0000 7f20307a5640 1 --2- 172.21.15.28:0/196318603 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f20280fc370 0x7f202810cf60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f202400eb70 tx=0x7f202400b040 comp rx=0 tx=0).stop 2024-07-03T15:41:36.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.448+0000 7f20307a5640 1 -- 172.21.15.28:0/196318603 shutdown_connections 2024-07-03T15:41:36.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.448+0000 7f20307a5640 1 --2- 172.21.15.28:0/196318603 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f2014067190 0x7f2014069640 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:36.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.448+0000 7f20307a5640 1 --2- 172.21.15.28:0/196318603 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20280fd640 0x7f202810d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:36.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.448+0000 7f20307a5640 1 --2- 172.21.15.28:0/196318603 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f20280fc370 0x7f202810cf60 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:36.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.448+0000 7f20307a5640 1 -- 172.21.15.28:0/196318603 >> 172.21.15.28:0/196318603 conn(0x7f20280f7b30 msgr2=0x7f20280f9180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:41:36.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.448+0000 7f20307a5640 1 -- 172.21.15.28:0/196318603 shutdown_connections 2024-07-03T15:41:36.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:36.448+0000 7f20307a5640 1 -- 172.21.15.28:0/196318603 wait complete. 2024-07-03T15:41:37.119 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-07-03T15:41:37.129 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi028.front.sepia.ceph.com 2024-07-03T15:41:37.129 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-07-03T15:41:37.391 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:41:37.829 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:37 smithi028 ceph-mon[29148]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:37.829 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:37 smithi028 ceph-mon[29148]: from='client.14438 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:41:37.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:37 smithi148 ceph-mon[36056]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:37.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:37 smithi148 ceph-mon[36056]: from='client.14438 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:41:38.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:38.943+0000 7f461f587640 1 -- 172.21.15.28:0/152391444 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f46180fc510 msgr2=0x7f46180fc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:38.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:38.943+0000 7f461f587640 1 --2- 172.21.15.28:0/152391444 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f46180fc510 0x7f46180fc980 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f46080048d0 tx=0x7f4608009da0 comp rx=0 tx=0).stop 2024-07-03T15:41:38.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:38.943+0000 7f461f587640 1 -- 172.21.15.28:0/152391444 shutdown_connections 2024-07-03T15:41:38.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:38.943+0000 7f461f587640 1 --2- 172.21.15.28:0/152391444 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f46180fc510 0x7f46180fc980 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:38.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:38.943+0000 7f461f587640 1 --2- 172.21.15.28:0/152391444 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46180fb240 0x7f46180fb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:38.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:38.943+0000 7f461f587640 1 -- 172.21.15.28:0/152391444 >> 172.21.15.28:0/152391444 conn(0x7f46180f6a20 msgr2=0x7f46180f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:41:38.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:38.943+0000 7f461f587640 1 -- 172.21.15.28:0/152391444 shutdown_connections 2024-07-03T15:41:38.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:38.943+0000 7f461f587640 1 -- 172.21.15.28:0/152391444 wait complete. 2024-07-03T15:41:38.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:38.943+0000 7f461f587640 1 Processor -- start 2024-07-03T15:41:38.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:38.944+0000 7f461f587640 1 -- start start 2024-07-03T15:41:38.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:38.944+0000 7f461f587640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46180fb240 0x7f461810bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:38.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:38.944+0000 7f461f587640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f46180fc510 0x7f461810c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:38.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:38.944+0000 7f461f587640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461810d270 con 0x7f46180fc510 2024-07-03T15:41:38.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:38.944+0000 7f461f587640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461810d3e0 con 0x7f46180fb240 2024-07-03T15:41:38.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:38.944+0000 7f461cb22640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f46180fc510 0x7f461810c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:38.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:38.944+0000 7f461cb22640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f46180fc510 0x7f461810c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51960/0 (socket says 172.21.15.28:51960) 2024-07-03T15:41:38.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:38.944+0000 7f461cb22640 1 -- 172.21.15.28:0/2345833947 learned_addr learned my addr 172.21.15.28:0/2345833947 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:41:38.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:38.944+0000 7f461d323640 1 --2- 172.21.15.28:0/2345833947 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46180fb240 0x7f461810bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:38.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:38.944+0000 7f461cb22640 1 -- 172.21.15.28:0/2345833947 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46180fb240 msgr2=0x7f461810bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:38.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:38.944+0000 7f461cb22640 1 --2- 172.21.15.28:0/2345833947 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46180fb240 0x7f461810bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:38.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:38.944+0000 7f461cb22640 1 -- 172.21.15.28:0/2345833947 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4608005040 con 0x7f46180fc510 2024-07-03T15:41:38.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:38.944+0000 7f461cb22640 1 --2- 172.21.15.28:0/2345833947 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f46180fc510 0x7f461810c400 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4608003080 tx=0x7f4608009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:41:38.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:38.945+0000 7f460dffb640 1 -- 172.21.15.28:0/2345833947 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f460803a070 con 0x7f46180fc510 2024-07-03T15:41:38.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:38.945+0000 7f460dffb640 1 -- 172.21.15.28:0/2345833947 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4608035b00 con 0x7f46180fc510 2024-07-03T15:41:38.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:38.945+0000 7f461f587640 1 -- 172.21.15.28:0/2345833947 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f461810d660 con 0x7f46180fc510 2024-07-03T15:41:38.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:38.945+0000 7f460dffb640 1 -- 172.21.15.28:0/2345833947 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4608032b00 con 0x7f46180fc510 2024-07-03T15:41:38.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:38.945+0000 7f461f587640 1 -- 172.21.15.28:0/2345833947 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f461810db20 con 0x7f46180fc510 2024-07-03T15:41:38.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:38.946+0000 7f461f587640 1 -- 172.21.15.28:0/2345833947 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46180fa6a0 con 0x7f46180fc510 2024-07-03T15:41:38.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:38.946+0000 7f460dffb640 1 -- 172.21.15.28:0/2345833947 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f4608030020 con 0x7f46180fc510 2024-07-03T15:41:38.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:38.947+0000 7f460dffb640 1 --2- 172.21.15.28:0/2345833947 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f4604067190 0x7f4604069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:38.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:38.947+0000 7f461d323640 1 --2- 172.21.15.28:0/2345833947 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f4604067190 0x7f4604069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:38.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:38.947+0000 7f460dffb640 1 -- 172.21.15.28:0/2345833947 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f46080a6ae0 con 0x7f46180fc510 2024-07-03T15:41:38.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:38.947+0000 7f461d323640 1 --2- 172.21.15.28:0/2345833947 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f4604067190 0x7f4604069640 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4614006fd0 tx=0x7f4614008040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:41:38.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:38.951+0000 7f460dffb640 1 -- 172.21.15.28:0/2345833947 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f46080740c0 con 0x7f46180fc510 2024-07-03T15:41:39.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:39.150+0000 7f461f587640 1 -- 172.21.15.28:0/2345833947 --> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f461810dd40 con 0x7f4604067190 2024-07-03T15:41:40.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:40 smithi028 ceph-mon[29148]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:40.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:40 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-07-03T15:41:40.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:40 smithi148 ceph-mon[36056]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:40.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:40 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-07-03T15:41:40.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:40.674+0000 7f460dffb640 1 -- 172.21.15.28:0/2345833947 <== mgr.14210 v2:172.21.15.28:6800/421057178 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f461810dd40 con 0x7f4604067190 2024-07-03T15:41:40.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:40.678+0000 7f461f587640 1 -- 172.21.15.28:0/2345833947 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f4604067190 msgr2=0x7f4604069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:40.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:40.678+0000 7f461f587640 1 --2- 172.21.15.28:0/2345833947 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f4604067190 0x7f4604069640 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4614006fd0 tx=0x7f4614008040 comp rx=0 tx=0).stop 2024-07-03T15:41:40.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:40.678+0000 7f461f587640 1 -- 172.21.15.28:0/2345833947 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f46180fc510 msgr2=0x7f461810c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:40.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:40.678+0000 7f461f587640 1 --2- 172.21.15.28:0/2345833947 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f46180fc510 0x7f461810c400 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4608003080 tx=0x7f4608009da0 comp rx=0 tx=0).stop 2024-07-03T15:41:40.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:40.678+0000 7f461f587640 1 -- 172.21.15.28:0/2345833947 shutdown_connections 2024-07-03T15:41:40.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:40.678+0000 7f461f587640 1 --2- 172.21.15.28:0/2345833947 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f4604067190 0x7f4604069640 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:40.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:40.678+0000 7f461f587640 1 --2- 172.21.15.28:0/2345833947 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f46180fc510 0x7f461810c400 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:40.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:40.678+0000 7f461f587640 1 --2- 172.21.15.28:0/2345833947 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46180fb240 0x7f461810bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:40.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:40.678+0000 7f461f587640 1 -- 172.21.15.28:0/2345833947 >> 172.21.15.28:0/2345833947 conn(0x7f46180f6a20 msgr2=0x7f46180f80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:41:40.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:40.678+0000 7f461f587640 1 -- 172.21.15.28:0/2345833947 shutdown_connections 2024-07-03T15:41:40.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:40.678+0000 7f461f587640 1 -- 172.21.15.28:0/2345833947 wait complete. 2024-07-03T15:41:41.072 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:40 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-mon-smithi028[29144]: 2024-07-03T15:41:40.654+0000 7f308bcf0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-03T15:41:41.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:41 smithi028 ceph-mon[29148]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:41:41.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:41 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-07-03T15:41:41.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:41 smithi028 ceph-mon[29148]: osdmap e41: 6 total, 6 up, 6 in 2024-07-03T15:41:41.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:41 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-07-03T15:41:41.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:41 smithi028 ceph-mon[29148]: pgmap v169: 33 pgs: 3 creating+peering, 29 unknown, 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:41.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:41 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-07-03T15:41:41.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:41 smithi028 ceph-mon[29148]: osdmap e42: 6 total, 6 up, 6 in 2024-07-03T15:41:41.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:41 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-07-03T15:41:41.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:41 smithi028 ceph-mon[29148]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-03T15:41:41.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:41 smithi028 ceph-mon[29148]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-07-03T15:41:41.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:41 smithi028 ceph-mon[29148]: osdmap e43: 6 total, 6 up, 6 in 2024-07-03T15:41:41.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:41 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-07-03T15:41:41.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:41 smithi028 ceph-mon[29148]: fsmap cephfs:0 2024-07-03T15:41:41.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:41 smithi028 ceph-mon[29148]: Saving service mds.cephfs spec with placement count:4 2024-07-03T15:41:41.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:41 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:41:41.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:41 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:41:41.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:41 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:41:41.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:41 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:41:41.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:41 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:41:41.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:41 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi028.rhpvve", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-03T15:41:41.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:41 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi028.rhpvve", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-03T15:41:41.374 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:41 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:41:41.374 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:41 smithi028 ceph-mon[29148]: Deploying daemon mds.cephfs.smithi028.rhpvve on smithi028 2024-07-03T15:41:41.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:41 smithi148 ceph-mon[36056]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:41:41.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:41 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-07-03T15:41:41.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:41 smithi148 ceph-mon[36056]: osdmap e41: 6 total, 6 up, 6 in 2024-07-03T15:41:41.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:41 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-07-03T15:41:41.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:41 smithi148 ceph-mon[36056]: pgmap v169: 33 pgs: 3 creating+peering, 29 unknown, 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:41.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:41 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-07-03T15:41:41.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:41 smithi148 ceph-mon[36056]: osdmap e42: 6 total, 6 up, 6 in 2024-07-03T15:41:41.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:41 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-07-03T15:41:41.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:41 smithi148 ceph-mon[36056]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-03T15:41:41.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:41 smithi148 ceph-mon[36056]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-07-03T15:41:41.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:41 smithi148 ceph-mon[36056]: osdmap e43: 6 total, 6 up, 6 in 2024-07-03T15:41:41.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:41 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-07-03T15:41:41.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:41 smithi148 ceph-mon[36056]: fsmap cephfs:0 2024-07-03T15:41:41.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:41 smithi148 ceph-mon[36056]: Saving service mds.cephfs spec with placement count:4 2024-07-03T15:41:41.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:41 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:41:41.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:41 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:41:41.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:41 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:41:41.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:41 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:41:41.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:41 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:41:41.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:41 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi028.rhpvve", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-03T15:41:41.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:41 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi028.rhpvve", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-03T15:41:41.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:41 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:41:41.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:41 smithi148 ceph-mon[36056]: Deploying daemon mds.cephfs.smithi028.rhpvve on smithi028 2024-07-03T15:41:41.501 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- bash -c 'ceph fs dump' 2024-07-03T15:41:42.240 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:41:42.355 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:42 smithi028 ceph-mon[29148]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-07-03T15:41:42.355 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:42 smithi028 ceph-mon[29148]: osdmap e44: 6 total, 6 up, 6 in 2024-07-03T15:41:42.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:42 smithi148 ceph-mon[36056]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-07-03T15:41:42.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:42 smithi148 ceph-mon[36056]: osdmap e44: 6 total, 6 up, 6 in 2024-07-03T15:41:43.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:43 smithi148 ceph-mon[36056]: pgmap v173: 65 pgs: 22 active+clean, 14 creating+peering, 29 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:43.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:43 smithi148 ceph-mon[36056]: osdmap e45: 6 total, 6 up, 6 in 2024-07-03T15:41:43.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:43 smithi148 ceph-mon[36056]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-07-03T15:41:44.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:43 smithi028 ceph-mon[29148]: pgmap v173: 65 pgs: 22 active+clean, 14 creating+peering, 29 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:44.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:43 smithi028 ceph-mon[29148]: osdmap e45: 6 total, 6 up, 6 in 2024-07-03T15:41:44.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:43 smithi028 ceph-mon[29148]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-07-03T15:41:44.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:44.880+0000 7f3860a03640 1 -- 172.21.15.28:0/64258822 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f385c0ff360 msgr2=0x7f385c101740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:44.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:44.880+0000 7f3860a03640 1 --2- 172.21.15.28:0/64258822 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f385c0ff360 0x7f385c101740 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f38440048d0 tx=0x7f3844009da0 comp rx=0 tx=0).stop 2024-07-03T15:41:44.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:44.881+0000 7f3860a03640 1 -- 172.21.15.28:0/64258822 shutdown_connections 2024-07-03T15:41:44.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:44.881+0000 7f3860a03640 1 --2- 172.21.15.28:0/64258822 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f385c0ff360 0x7f385c101740 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:44.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:44.881+0000 7f3860a03640 1 --2- 172.21.15.28:0/64258822 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f385c0662d0 0x7f385c0fee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:44.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:44.881+0000 7f3860a03640 1 -- 172.21.15.28:0/64258822 >> 172.21.15.28:0/64258822 conn(0x7f385c0f6870 msgr2=0x7f385c0f8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:41:44.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:44.881+0000 7f3860a03640 1 -- 172.21.15.28:0/64258822 shutdown_connections 2024-07-03T15:41:44.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:44.881+0000 7f3860a03640 1 -- 172.21.15.28:0/64258822 wait complete. 2024-07-03T15:41:44.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:44.881+0000 7f3860a03640 1 Processor -- start 2024-07-03T15:41:44.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:44.881+0000 7f3860a03640 1 -- start start 2024-07-03T15:41:44.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:44.881+0000 7f3860a03640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f385c0662d0 0x7f385c10ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:44.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:44.881+0000 7f3860a03640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f385c0ff360 0x7f385c10d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:44.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:44.881+0000 7f3860a03640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385c10da00 con 0x7f385c0ff360 2024-07-03T15:41:44.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:44.881+0000 7f3860a03640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385c10e0f0 con 0x7f385c0662d0 2024-07-03T15:41:44.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:44.882+0000 7f3853fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f385c0ff360 0x7f385c10d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:44.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:44.882+0000 7f3853fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f385c0ff360 0x7f385c10d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56032/0 (socket says 172.21.15.28:56032) 2024-07-03T15:41:44.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:44.882+0000 7f3853fff640 1 -- 172.21.15.28:0/2984943535 learned_addr learned my addr 172.21.15.28:0/2984943535 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:41:44.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:44.882+0000 7f385ad9d640 1 --2- 172.21.15.28:0/2984943535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f385c0662d0 0x7f385c10ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:44.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:44.882+0000 7f3853fff640 1 -- 172.21.15.28:0/2984943535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f385c0662d0 msgr2=0x7f385c10ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:44.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:44.882+0000 7f3853fff640 1 --2- 172.21.15.28:0/2984943535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f385c0662d0 0x7f385c10ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:44.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:44.882+0000 7f3853fff640 1 -- 172.21.15.28:0/2984943535 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3844005040 con 0x7f385c0ff360 2024-07-03T15:41:44.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:44.882+0000 7f3853fff640 1 --2- 172.21.15.28:0/2984943535 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f385c0ff360 0x7f385c10d380 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f38440048a0 tx=0x7f3844009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:41:44.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:44.882+0000 7f38537fe640 1 -- 172.21.15.28:0/2984943535 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3844038070 con 0x7f385c0ff360 2024-07-03T15:41:44.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:44.882+0000 7f38537fe640 1 -- 172.21.15.28:0/2984943535 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3844032970 con 0x7f385c0ff360 2024-07-03T15:41:44.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:44.882+0000 7f3860a03640 1 -- 172.21.15.28:0/2984943535 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f385c10e310 con 0x7f385c0ff360 2024-07-03T15:41:44.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:44.882+0000 7f38537fe640 1 -- 172.21.15.28:0/2984943535 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f384403d8c0 con 0x7f385c0ff360 2024-07-03T15:41:44.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:44.882+0000 7f3860a03640 1 -- 172.21.15.28:0/2984943535 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f385c10e8b0 con 0x7f385c0ff360 2024-07-03T15:41:44.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:44.883+0000 7f3860a03640 1 -- 172.21.15.28:0/2984943535 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f385c0fa4f0 con 0x7f385c0ff360 2024-07-03T15:41:44.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:44.884+0000 7f38537fe640 1 -- 172.21.15.28:0/2984943535 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f3844030020 con 0x7f385c0ff360 2024-07-03T15:41:44.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:44.884+0000 7f38537fe640 1 --2- 172.21.15.28:0/2984943535 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3848067190 0x7f3848069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:44.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:44.884+0000 7f38537fe640 1 -- 172.21.15.28:0/2984943535 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f38440a51f0 con 0x7f385c0ff360 2024-07-03T15:41:44.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:44.884+0000 7f385ad9d640 1 --2- 172.21.15.28:0/2984943535 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3848067190 0x7f3848069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:44.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:44.885+0000 7f385ad9d640 1 --2- 172.21.15.28:0/2984943535 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3848067190 0x7f3848069640 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f385c0fef40 tx=0x7f384c009040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:41:44.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:44.889+0000 7f38537fe640 1 -- 172.21.15.28:0/2984943535 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3844072550 con 0x7f385c0ff360 2024-07-03T15:41:45.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:45.103+0000 7f3860a03640 1 -- 172.21.15.28:0/2984943535 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f385c04b530 con 0x7f385c0ff360 2024-07-03T15:41:45.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:45.103+0000 7f38537fe640 1 -- 172.21.15.28:0/2984943535 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 2 v2) v1 ==== 75+0+1076 (secure 0 0 0) 0x7f3844075b70 con 0x7f385c0ff360 2024-07-03T15:41:45.107 INFO:teuthology.orchestra.run.smithi028.stdout:e2 2024-07-03T15:41:45.107 INFO:teuthology.orchestra.run.smithi028.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-03T15:41:45.107 INFO:teuthology.orchestra.run.smithi028.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:41:45.107 INFO:teuthology.orchestra.run.smithi028.stdout:legacy client fscid: 1 2024-07-03T15:41:45.107 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:41:45.107 INFO:teuthology.orchestra.run.smithi028.stdout:Filesystem 'cephfs' (1) 2024-07-03T15:41:45.107 INFO:teuthology.orchestra.run.smithi028.stdout:fs_name cephfs 2024-07-03T15:41:45.107 INFO:teuthology.orchestra.run.smithi028.stdout:epoch 2 2024-07-03T15:41:45.107 INFO:teuthology.orchestra.run.smithi028.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-07-03T15:41:45.108 INFO:teuthology.orchestra.run.smithi028.stdout:created 2024-07-03T15:41:40.656298+0000 2024-07-03T15:41:45.108 INFO:teuthology.orchestra.run.smithi028.stdout:modified 2024-07-03T15:41:40.656352+0000 2024-07-03T15:41:45.108 INFO:teuthology.orchestra.run.smithi028.stdout:tableserver 0 2024-07-03T15:41:45.108 INFO:teuthology.orchestra.run.smithi028.stdout:root 0 2024-07-03T15:41:45.108 INFO:teuthology.orchestra.run.smithi028.stdout:session_timeout 60 2024-07-03T15:41:45.108 INFO:teuthology.orchestra.run.smithi028.stdout:session_autoclose 300 2024-07-03T15:41:45.108 INFO:teuthology.orchestra.run.smithi028.stdout:max_file_size 1099511627776 2024-07-03T15:41:45.108 INFO:teuthology.orchestra.run.smithi028.stdout:required_client_features {} 2024-07-03T15:41:45.108 INFO:teuthology.orchestra.run.smithi028.stdout:last_failure 0 2024-07-03T15:41:45.108 INFO:teuthology.orchestra.run.smithi028.stdout:last_failure_osd_epoch 0 2024-07-03T15:41:45.108 INFO:teuthology.orchestra.run.smithi028.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:41:45.109 INFO:teuthology.orchestra.run.smithi028.stdout:max_mds 1 2024-07-03T15:41:45.109 INFO:teuthology.orchestra.run.smithi028.stdout:in 2024-07-03T15:41:45.109 INFO:teuthology.orchestra.run.smithi028.stdout:up {} 2024-07-03T15:41:45.109 INFO:teuthology.orchestra.run.smithi028.stdout:failed 2024-07-03T15:41:45.109 INFO:teuthology.orchestra.run.smithi028.stdout:damaged 2024-07-03T15:41:45.109 INFO:teuthology.orchestra.run.smithi028.stdout:stopped 2024-07-03T15:41:45.109 INFO:teuthology.orchestra.run.smithi028.stdout:data_pools [3] 2024-07-03T15:41:45.109 INFO:teuthology.orchestra.run.smithi028.stdout:metadata_pool 2 2024-07-03T15:41:45.109 INFO:teuthology.orchestra.run.smithi028.stdout:inline_data disabled 2024-07-03T15:41:45.109 INFO:teuthology.orchestra.run.smithi028.stdout:balancer 2024-07-03T15:41:45.109 INFO:teuthology.orchestra.run.smithi028.stdout:standby_count_wanted 0 2024-07-03T15:41:45.109 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:41:45.109 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:41:45.110 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 2 2024-07-03T15:41:45.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:45.105+0000 7f3860a03640 1 -- 172.21.15.28:0/2984943535 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3848067190 msgr2=0x7f3848069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:45.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:45.105+0000 7f3860a03640 1 --2- 172.21.15.28:0/2984943535 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3848067190 0x7f3848069640 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f385c0fef40 tx=0x7f384c009040 comp rx=0 tx=0).stop 2024-07-03T15:41:45.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:45.105+0000 7f3860a03640 1 -- 172.21.15.28:0/2984943535 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f385c0ff360 msgr2=0x7f385c10d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:45.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:45.105+0000 7f3860a03640 1 --2- 172.21.15.28:0/2984943535 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f385c0ff360 0x7f385c10d380 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f38440048a0 tx=0x7f3844009da0 comp rx=0 tx=0).stop 2024-07-03T15:41:45.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:45.105+0000 7f3860a03640 1 -- 172.21.15.28:0/2984943535 shutdown_connections 2024-07-03T15:41:45.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:45.105+0000 7f3860a03640 1 --2- 172.21.15.28:0/2984943535 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3848067190 0x7f3848069640 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:45.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:45.105+0000 7f3860a03640 1 --2- 172.21.15.28:0/2984943535 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f385c0ff360 0x7f385c10d380 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:45.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:45.105+0000 7f3860a03640 1 --2- 172.21.15.28:0/2984943535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f385c0662d0 0x7f385c10ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:45.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:45.105+0000 7f3860a03640 1 -- 172.21.15.28:0/2984943535 >> 172.21.15.28:0/2984943535 conn(0x7f385c0f6870 msgr2=0x7f385c0f6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:41:45.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:45.105+0000 7f3860a03640 1 -- 172.21.15.28:0/2984943535 shutdown_connections 2024-07-03T15:41:45.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:45.105+0000 7f3860a03640 1 -- 172.21.15.28:0/2984943535 wait complete. 2024-07-03T15:41:45.799 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:45 smithi028 ceph-mon[29148]: pgmap v175: 65 pgs: 40 active+clean, 14 creating+peering, 11 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:45.799 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:45 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/2984943535' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-03T15:41:45.799 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:45 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:41:45.799 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:45 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:41:45.799 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:45 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:41:45.799 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:45 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.xgpuld", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-03T15:41:45.799 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:45 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.xgpuld", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-03T15:41:45.799 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:45 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:41:45.799 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:45 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:41:45.799 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:45 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:41:45.824 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-07-03T15:41:45.834 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi028.front.sepia.ceph.com 2024-07-03T15:41:45.834 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- bash -c 'ceph fs set cephfs max_mds 1' 2024-07-03T15:41:45.906 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:45 smithi148 ceph-mon[36056]: pgmap v175: 65 pgs: 40 active+clean, 14 creating+peering, 11 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-03T15:41:45.906 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:45 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/2984943535' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-03T15:41:45.907 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:45 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:41:45.907 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:45 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:41:45.907 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:45 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:41:45.907 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:45 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.xgpuld", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-03T15:41:45.907 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:45 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.xgpuld", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-03T15:41:45.907 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:45 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:41:45.907 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:45 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:41:45.907 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:45 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:41:46.102 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:41:46.764 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:46 smithi148 ceph-mon[36056]: Deploying daemon mds.cephfs.smithi148.xgpuld on smithi148 2024-07-03T15:41:46.764 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:46 smithi148 ceph-mon[36056]: mds.? [v2:172.21.15.28:6826/1613936042,v1:172.21.15.28:6827/1613936042] up:boot 2024-07-03T15:41:46.764 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:46 smithi148 ceph-mon[36056]: daemon mds.cephfs.smithi028.rhpvve assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-07-03T15:41:46.764 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:46 smithi148 ceph-mon[36056]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-07-03T15:41:46.764 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:46 smithi148 ceph-mon[36056]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-07-03T15:41:46.764 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:46 smithi148 ceph-mon[36056]: Cluster is now healthy 2024-07-03T15:41:46.764 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:46 smithi148 ceph-mon[36056]: fsmap cephfs:0 1 up:standby 2024-07-03T15:41:46.764 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:46 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi028.rhpvve"}]: dispatch 2024-07-03T15:41:46.765 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:46 smithi148 ceph-mon[36056]: fsmap cephfs:1 {0=cephfs.smithi028.rhpvve=up:creating} 2024-07-03T15:41:46.765 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:46 smithi148 ceph-mon[36056]: daemon mds.cephfs.smithi028.rhpvve is now active in filesystem cephfs as rank 0 2024-07-03T15:41:46.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:46 smithi028 ceph-mon[29148]: Deploying daemon mds.cephfs.smithi148.xgpuld on smithi148 2024-07-03T15:41:46.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:46 smithi028 ceph-mon[29148]: mds.? [v2:172.21.15.28:6826/1613936042,v1:172.21.15.28:6827/1613936042] up:boot 2024-07-03T15:41:46.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:46 smithi028 ceph-mon[29148]: daemon mds.cephfs.smithi028.rhpvve assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-07-03T15:41:46.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:46 smithi028 ceph-mon[29148]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-07-03T15:41:46.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:46 smithi028 ceph-mon[29148]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-07-03T15:41:46.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:46 smithi028 ceph-mon[29148]: Cluster is now healthy 2024-07-03T15:41:46.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:46 smithi028 ceph-mon[29148]: fsmap cephfs:0 1 up:standby 2024-07-03T15:41:46.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:46 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi028.rhpvve"}]: dispatch 2024-07-03T15:41:46.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:46 smithi028 ceph-mon[29148]: fsmap cephfs:1 {0=cephfs.smithi028.rhpvve=up:creating} 2024-07-03T15:41:46.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:46 smithi028 ceph-mon[29148]: daemon mds.cephfs.smithi028.rhpvve is now active in filesystem cephfs as rank 0 2024-07-03T15:41:47.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:47.646+0000 7f8dcceef640 1 -- 172.21.15.28:0/2912182107 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8dc80ff490 msgr2=0x7f8dc8101870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:47.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:47.646+0000 7f8dcceef640 1 --2- 172.21.15.28:0/2912182107 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8dc80ff490 0x7f8dc8101870 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc0048d0 tx=0x7f8dbc009da0 comp rx=0 tx=0).stop 2024-07-03T15:41:47.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:47.646+0000 7f8dcceef640 1 -- 172.21.15.28:0/2912182107 shutdown_connections 2024-07-03T15:41:47.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:47.646+0000 7f8dcceef640 1 --2- 172.21.15.28:0/2912182107 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8dc80ff490 0x7f8dc8101870 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:47.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:47.646+0000 7f8dcceef640 1 --2- 172.21.15.28:0/2912182107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8dc8065a50 0x7f8dc80fef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:47.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:47.646+0000 7f8dcceef640 1 -- 172.21.15.28:0/2912182107 >> 172.21.15.28:0/2912182107 conn(0x7f8dc80f6a20 msgr2=0x7f8dc80f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:41:47.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:47.646+0000 7f8dcceef640 1 -- 172.21.15.28:0/2912182107 shutdown_connections 2024-07-03T15:41:47.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:47.646+0000 7f8dcceef640 1 -- 172.21.15.28:0/2912182107 wait complete. 2024-07-03T15:41:47.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:47.647+0000 7f8dcceef640 1 Processor -- start 2024-07-03T15:41:47.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:47.647+0000 7f8dcceef640 1 -- start start 2024-07-03T15:41:47.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:47.647+0000 7f8dcceef640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8dc8065a50 0x7f8dc810cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:47.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:47.647+0000 7f8dcceef640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8dc80ff490 0x7f8dc810d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:47.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:47.647+0000 7f8dcceef640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc810db20 con 0x7f8dc8065a50 2024-07-03T15:41:47.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:47.647+0000 7f8dcceef640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc810e2f0 con 0x7f8dc80ff490 2024-07-03T15:41:47.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:47.647+0000 7f8dc659c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8dc8065a50 0x7f8dc810cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:47.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:47.647+0000 7f8dc659c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8dc8065a50 0x7f8dc810cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56064/0 (socket says 172.21.15.28:56064) 2024-07-03T15:41:47.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:47.647+0000 7f8dc659c640 1 -- 172.21.15.28:0/1442832455 learned_addr learned my addr 172.21.15.28:0/1442832455 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:41:47.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:47.647+0000 7f8dc5d9b640 1 --2- 172.21.15.28:0/1442832455 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8dc80ff490 0x7f8dc810d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:47.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:47.647+0000 7f8dc659c640 1 -- 172.21.15.28:0/1442832455 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8dc80ff490 msgr2=0x7f8dc810d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:47.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:47.647+0000 7f8dc659c640 1 --2- 172.21.15.28:0/1442832455 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8dc80ff490 0x7f8dc810d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:47.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:47.647+0000 7f8dc659c640 1 -- 172.21.15.28:0/1442832455 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dbc005040 con 0x7f8dc8065a50 2024-07-03T15:41:47.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:47.648+0000 7f8dc659c640 1 --2- 172.21.15.28:0/1442832455 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8dc8065a50 0x7f8dc810cf90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8db800db00 tx=0x7f8db8004e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:41:47.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:47.648+0000 7f8db6ffd640 1 -- 172.21.15.28:0/1442832455 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8db800b910 con 0x7f8dc8065a50 2024-07-03T15:41:47.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:47.648+0000 7f8db6ffd640 1 -- 172.21.15.28:0/1442832455 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8db800f040 con 0x7f8dc8065a50 2024-07-03T15:41:47.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:47.648+0000 7f8dcceef640 1 -- 172.21.15.28:0/1442832455 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dc810e580 con 0x7f8dc8065a50 2024-07-03T15:41:47.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:47.648+0000 7f8db6ffd640 1 -- 172.21.15.28:0/1442832455 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8db8012390 con 0x7f8dc8065a50 2024-07-03T15:41:47.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:47.648+0000 7f8dcceef640 1 -- 172.21.15.28:0/1442832455 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dc810eb20 con 0x7f8dc8065a50 2024-07-03T15:41:47.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:47.649+0000 7f8dcceef640 1 -- 172.21.15.28:0/1442832455 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dc80fa6a0 con 0x7f8dc8065a50 2024-07-03T15:41:47.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:47.649+0000 7f8db6ffd640 1 -- 172.21.15.28:0/1442832455 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f8db8012610 con 0x7f8dc8065a50 2024-07-03T15:41:47.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:47.650+0000 7f8db6ffd640 1 --2- 172.21.15.28:0/1442832455 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f8db0067190 0x7f8db0069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:47.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:47.650+0000 7f8db6ffd640 1 -- 172.21.15.28:0/1442832455 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f8db8085020 con 0x7f8dc8065a50 2024-07-03T15:41:47.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:47.650+0000 7f8dc5d9b640 1 --2- 172.21.15.28:0/1442832455 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f8db0067190 0x7f8db0069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:47.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:47.650+0000 7f8dc5d9b640 1 --2- 172.21.15.28:0/1442832455 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f8db0067190 0x7f8db0069640 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc009f80 tx=0x7f8dbc035040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:41:47.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:47.656+0000 7f8db6ffd640 1 -- 172.21.15.28:0/1442832455 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8db80522c0 con 0x7f8dc8065a50 2024-07-03T15:41:47.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:47.885+0000 7f8dcceef640 1 -- 172.21.15.28:0/1442832455 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) v1 -- 0x7f8dc810eed0 con 0x7f8dc8065a50 2024-07-03T15:41:47.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:47 smithi148 ceph-mon[36056]: pgmap v176: 65 pgs: 65 active+clean; 578 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 535 B/s wr, 2 op/s 2024-07-03T15:41:47.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:47 smithi148 ceph-mon[36056]: mds.? [v2:172.21.15.28:6826/1613936042,v1:172.21.15.28:6827/1613936042] up:active 2024-07-03T15:41:47.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:47 smithi148 ceph-mon[36056]: fsmap cephfs:1 {0=cephfs.smithi028.rhpvve=up:active} 2024-07-03T15:41:47.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:47 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:41:47.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:47 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:41:47.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:47 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:41:47.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:47 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi028.wmbxew", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-03T15:41:47.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:47 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi028.wmbxew", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-03T15:41:47.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:47 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:41:48.044 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:47 smithi028 ceph-mon[29148]: pgmap v176: 65 pgs: 65 active+clean; 578 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 535 B/s wr, 2 op/s 2024-07-03T15:41:48.044 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:47 smithi028 ceph-mon[29148]: mds.? [v2:172.21.15.28:6826/1613936042,v1:172.21.15.28:6827/1613936042] up:active 2024-07-03T15:41:48.044 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:47 smithi028 ceph-mon[29148]: fsmap cephfs:1 {0=cephfs.smithi028.rhpvve=up:active} 2024-07-03T15:41:48.044 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:47 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:41:48.044 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:47 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:41:48.044 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:47 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:41:48.044 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:47 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi028.wmbxew", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-03T15:41:48.044 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:47 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi028.wmbxew", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-03T15:41:48.044 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:47 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:41:48.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:48.694+0000 7f8db6ffd640 1 -- 172.21.15.28:0/1442832455 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v8) v1 ==== 105+0+0 (secure 0 0 0) 0x7f8db80558e0 con 0x7f8dc8065a50 2024-07-03T15:41:48.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:48.696+0000 7f8dcceef640 1 -- 172.21.15.28:0/1442832455 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f8db0067190 msgr2=0x7f8db0069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:48.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:48.697+0000 7f8dcceef640 1 --2- 172.21.15.28:0/1442832455 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f8db0067190 0x7f8db0069640 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc009f80 tx=0x7f8dbc035040 comp rx=0 tx=0).stop 2024-07-03T15:41:48.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:48.697+0000 7f8dcceef640 1 -- 172.21.15.28:0/1442832455 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8dc8065a50 msgr2=0x7f8dc810cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:48.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:48.697+0000 7f8dcceef640 1 --2- 172.21.15.28:0/1442832455 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8dc8065a50 0x7f8dc810cf90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8db800db00 tx=0x7f8db8004e80 comp rx=0 tx=0).stop 2024-07-03T15:41:48.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:48.697+0000 7f8dcceef640 1 -- 172.21.15.28:0/1442832455 shutdown_connections 2024-07-03T15:41:48.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:48.697+0000 7f8dcceef640 1 --2- 172.21.15.28:0/1442832455 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f8db0067190 0x7f8db0069640 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:48.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:48.697+0000 7f8dcceef640 1 --2- 172.21.15.28:0/1442832455 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8dc80ff490 0x7f8dc810d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:48.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:48.697+0000 7f8dcceef640 1 --2- 172.21.15.28:0/1442832455 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8dc8065a50 0x7f8dc810cf90 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:48.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:48.697+0000 7f8dcceef640 1 -- 172.21.15.28:0/1442832455 >> 172.21.15.28:0/1442832455 conn(0x7f8dc80f6a20 msgr2=0x7f8dc80f6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:41:48.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:48.697+0000 7f8dcceef640 1 -- 172.21.15.28:0/1442832455 shutdown_connections 2024-07-03T15:41:48.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:48.697+0000 7f8dcceef640 1 -- 172.21.15.28:0/1442832455 wait complete. 2024-07-03T15:41:48.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:48 smithi028 ceph-mon[29148]: Deploying daemon mds.cephfs.smithi028.wmbxew on smithi028 2024-07-03T15:41:48.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:48 smithi028 ceph-mon[29148]: mds.? [v2:172.21.15.148:6824/2597855286,v1:172.21.15.148:6825/2597855286] up:boot 2024-07-03T15:41:48.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:48 smithi028 ceph-mon[29148]: fsmap cephfs:1 {0=cephfs.smithi028.rhpvve=up:active} 1 up:standby 2024-07-03T15:41:48.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:48 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.xgpuld"}]: dispatch 2024-07-03T15:41:48.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:48 smithi028 ceph-mon[29148]: fsmap cephfs:1 {0=cephfs.smithi028.rhpvve=up:active} 1 up:standby 2024-07-03T15:41:48.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:48 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/1442832455' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-07-03T15:41:48.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:48 smithi148 ceph-mon[36056]: Deploying daemon mds.cephfs.smithi028.wmbxew on smithi028 2024-07-03T15:41:48.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:48 smithi148 ceph-mon[36056]: mds.? [v2:172.21.15.148:6824/2597855286,v1:172.21.15.148:6825/2597855286] up:boot 2024-07-03T15:41:48.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:48 smithi148 ceph-mon[36056]: fsmap cephfs:1 {0=cephfs.smithi028.rhpvve=up:active} 1 up:standby 2024-07-03T15:41:48.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:48 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.xgpuld"}]: dispatch 2024-07-03T15:41:48.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:48 smithi148 ceph-mon[36056]: fsmap cephfs:1 {0=cephfs.smithi028.rhpvve=up:active} 1 up:standby 2024-07-03T15:41:48.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:48 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/1442832455' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-07-03T15:41:49.762 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-07-03T15:41:49.772 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi028.front.sepia.ceph.com 2024-07-03T15:41:49.772 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- bash -c 'ceph fs set cephfs allow_standby_replay true' 2024-07-03T15:41:49.980 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:49 smithi028 ceph-mon[29148]: pgmap v177: 65 pgs: 65 active+clean; 578 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 662 B/s wr, 2 op/s 2024-07-03T15:41:49.980 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:49 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/1442832455' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-07-03T15:41:49.980 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:49 smithi028 ceph-mon[29148]: fsmap cephfs:1 {0=cephfs.smithi028.rhpvve=up:active} 1 up:standby 2024-07-03T15:41:49.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:49 smithi148 ceph-mon[36056]: pgmap v177: 65 pgs: 65 active+clean; 578 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 662 B/s wr, 2 op/s 2024-07-03T15:41:49.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:49 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/1442832455' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-07-03T15:41:49.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:49 smithi148 ceph-mon[36056]: fsmap cephfs:1 {0=cephfs.smithi028.rhpvve=up:active} 1 up:standby 2024-07-03T15:41:50.439 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:41:51.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:51 smithi148 ceph-mon[36056]: pgmap v178: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s wr, 5 op/s 2024-07-03T15:41:51.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:51 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:41:51.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:51 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:41:51.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:51 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:41:51.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:51 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.szomfl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-03T15:41:51.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:51 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.szomfl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-03T15:41:51.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:51 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:41:51.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:51 smithi148 ceph-mon[36056]: Deploying daemon mds.cephfs.smithi148.szomfl on smithi148 2024-07-03T15:41:51.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:51 smithi148 ceph-mon[36056]: mds.? [v2:172.21.15.28:6828/1730336821,v1:172.21.15.28:6829/1730336821] up:boot 2024-07-03T15:41:51.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:51 smithi148 ceph-mon[36056]: fsmap cephfs:1 {0=cephfs.smithi028.rhpvve=up:active} 2 up:standby 2024-07-03T15:41:51.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:51 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi028.wmbxew"}]: dispatch 2024-07-03T15:41:51.733 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:51 smithi028 ceph-mon[29148]: pgmap v178: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s wr, 5 op/s 2024-07-03T15:41:51.733 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:51 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:41:51.733 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:51 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:41:51.733 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:51 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:41:51.733 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:51 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.szomfl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-03T15:41:51.733 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:51 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.szomfl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-03T15:41:51.734 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:51 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:41:51.734 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:51 smithi028 ceph-mon[29148]: Deploying daemon mds.cephfs.smithi148.szomfl on smithi148 2024-07-03T15:41:51.734 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:51 smithi028 ceph-mon[29148]: mds.? [v2:172.21.15.28:6828/1730336821,v1:172.21.15.28:6829/1730336821] up:boot 2024-07-03T15:41:51.734 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:51 smithi028 ceph-mon[29148]: fsmap cephfs:1 {0=cephfs.smithi028.rhpvve=up:active} 2 up:standby 2024-07-03T15:41:51.734 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:51 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi028.wmbxew"}]: dispatch 2024-07-03T15:41:52.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.083+0000 7fed35b26640 1 -- 172.21.15.28:0/3859369432 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed300fd9b0 msgr2=0x7fed300fddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:52.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.083+0000 7fed35b26640 1 --2- 172.21.15.28:0/3859369432 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed300fd9b0 0x7fed300fddc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fed200048d0 tx=0x7fed20009da0 comp rx=0 tx=0).stop 2024-07-03T15:41:52.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.083+0000 7fed35b26640 1 -- 172.21.15.28:0/3859369432 shutdown_connections 2024-07-03T15:41:52.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.083+0000 7fed35b26640 1 --2- 172.21.15.28:0/3859369432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed300fe300 0x7fed300fa460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:52.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.083+0000 7fed35b26640 1 --2- 172.21.15.28:0/3859369432 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed300fd9b0 0x7fed300fddc0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:52.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.083+0000 7fed35b26640 1 -- 172.21.15.28:0/3859369432 >> 172.21.15.28:0/3859369432 conn(0x7fed300f61b0 msgr2=0x7fed300f85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:41:52.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.083+0000 7fed35b26640 1 -- 172.21.15.28:0/3859369432 shutdown_connections 2024-07-03T15:41:52.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.083+0000 7fed35b26640 1 -- 172.21.15.28:0/3859369432 wait complete. 2024-07-03T15:41:52.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.084+0000 7fed35b26640 1 Processor -- start 2024-07-03T15:41:52.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.084+0000 7fed35b26640 1 -- start start 2024-07-03T15:41:52.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.084+0000 7fed35b26640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed300fd9b0 0x7fed300fd150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:52.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.084+0000 7fed35b26640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed300fe300 0x7fed300fb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:52.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.084+0000 7fed35b26640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed300fbbd0 con 0x7fed300fd9b0 2024-07-03T15:41:52.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.084+0000 7fed35b26640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed300fbd10 con 0x7fed300fe300 2024-07-03T15:41:52.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.084+0000 7fed2f7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed300fd9b0 0x7fed300fd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:52.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.084+0000 7fed2f7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed300fd9b0 0x7fed300fd150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47230/0 (socket says 172.21.15.28:47230) 2024-07-03T15:41:52.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.084+0000 7fed2f7fe640 1 -- 172.21.15.28:0/3917791513 learned_addr learned my addr 172.21.15.28:0/3917791513 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:41:52.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.084+0000 7fed26ffd640 1 --2- 172.21.15.28:0/3917791513 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed300fe300 0x7fed300fb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:52.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.085+0000 7fed2f7fe640 1 -- 172.21.15.28:0/3917791513 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed300fe300 msgr2=0x7fed300fb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:52.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.085+0000 7fed2f7fe640 1 --2- 172.21.15.28:0/3917791513 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed300fe300 0x7fed300fb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:52.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.085+0000 7fed2f7fe640 1 -- 172.21.15.28:0/3917791513 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed20005040 con 0x7fed300fd9b0 2024-07-03T15:41:52.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.085+0000 7fed2f7fe640 1 --2- 172.21.15.28:0/3917791513 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed300fd9b0 0x7fed300fd150 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fed200048a0 tx=0x7fed20009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:41:52.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.085+0000 7fed2cff9640 1 -- 172.21.15.28:0/3917791513 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed20038070 con 0x7fed300fd9b0 2024-07-03T15:41:52.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.085+0000 7fed2cff9640 1 -- 172.21.15.28:0/3917791513 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed200092b0 con 0x7fed300fd9b0 2024-07-03T15:41:52.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.085+0000 7fed35b26640 1 -- 172.21.15.28:0/3917791513 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed300fbf90 con 0x7fed300fd9b0 2024-07-03T15:41:52.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.085+0000 7fed2cff9640 1 -- 172.21.15.28:0/3917791513 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed20033780 con 0x7fed300fd9b0 2024-07-03T15:41:52.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.085+0000 7fed35b26640 1 -- 172.21.15.28:0/3917791513 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed300fc4d0 con 0x7fed300fd9b0 2024-07-03T15:41:52.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.086+0000 7fed35b26640 1 -- 172.21.15.28:0/3917791513 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed300f9e50 con 0x7fed300fd9b0 2024-07-03T15:41:52.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.087+0000 7fed2cff9640 1 -- 172.21.15.28:0/3917791513 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7fed20030020 con 0x7fed300fd9b0 2024-07-03T15:41:52.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.087+0000 7fed2cff9640 1 --2- 172.21.15.28:0/3917791513 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fed1c067190 0x7fed1c069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:52.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.087+0000 7fed2cff9640 1 -- 172.21.15.28:0/3917791513 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fed200a48f0 con 0x7fed300fd9b0 2024-07-03T15:41:52.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.087+0000 7fed26ffd640 1 --2- 172.21.15.28:0/3917791513 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fed1c067190 0x7fed1c069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:52.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.088+0000 7fed26ffd640 1 --2- 172.21.15.28:0/3917791513 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fed1c067190 0x7fed1c069640 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fed18009af0 tx=0x7fed1800b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:41:52.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.092+0000 7fed2cff9640 1 -- 172.21.15.28:0/3917791513 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fed20071c40 con 0x7fed300fd9b0 2024-07-03T15:41:52.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.305+0000 7fed35b26640 1 -- 172.21.15.28:0/3917791513 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fed3004b530 con 0x7fed300fd9b0 2024-07-03T15:41:52.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.440+0000 7fed2cff9640 1 -- 172.21.15.28:0/3917791513 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v10) v1 ==== 121+0+0 (secure 0 0 0) 0x7fed20075260 con 0x7fed300fd9b0 2024-07-03T15:41:52.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.443+0000 7fed35b26640 1 -- 172.21.15.28:0/3917791513 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fed1c067190 msgr2=0x7fed1c069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:52.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.443+0000 7fed35b26640 1 --2- 172.21.15.28:0/3917791513 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fed1c067190 0x7fed1c069640 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fed18009af0 tx=0x7fed1800b040 comp rx=0 tx=0).stop 2024-07-03T15:41:52.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.443+0000 7fed35b26640 1 -- 172.21.15.28:0/3917791513 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed300fd9b0 msgr2=0x7fed300fd150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:52.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.443+0000 7fed35b26640 1 --2- 172.21.15.28:0/3917791513 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed300fd9b0 0x7fed300fd150 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fed200048a0 tx=0x7fed20009da0 comp rx=0 tx=0).stop 2024-07-03T15:41:52.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.443+0000 7fed35b26640 1 -- 172.21.15.28:0/3917791513 shutdown_connections 2024-07-03T15:41:52.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.443+0000 7fed35b26640 1 --2- 172.21.15.28:0/3917791513 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fed1c067190 0x7fed1c069640 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:52.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.443+0000 7fed35b26640 1 --2- 172.21.15.28:0/3917791513 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed300fe300 0x7fed300fb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:52.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.443+0000 7fed35b26640 1 --2- 172.21.15.28:0/3917791513 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed300fd9b0 0x7fed300fd150 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:52.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.443+0000 7fed35b26640 1 -- 172.21.15.28:0/3917791513 >> 172.21.15.28:0/3917791513 conn(0x7fed300f61b0 msgr2=0x7fed300f7ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:41:52.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.443+0000 7fed35b26640 1 -- 172.21.15.28:0/3917791513 shutdown_connections 2024-07-03T15:41:52.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:52.443+0000 7fed35b26640 1 -- 172.21.15.28:0/3917791513 wait complete. 2024-07-03T15:41:52.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:52 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/3917791513' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-07-03T15:41:52.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:52 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/3917791513' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-07-03T15:41:53.124 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-07-03T15:41:53.134 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi028.front.sepia.ceph.com 2024-07-03T15:41:53.135 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- bash -c 'ceph fs set cephfs inline_data false' 2024-07-03T15:41:53.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:53 smithi148 ceph-mon[36056]: pgmap v179: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s wr, 4 op/s 2024-07-03T15:41:53.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:53 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/3917791513' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-07-03T15:41:53.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:53 smithi148 ceph-mon[36056]: fsmap cephfs:1 {0=cephfs.smithi028.rhpvve=up:active} 2 up:standby 2024-07-03T15:41:53.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:53 smithi148 ceph-mon[36056]: fsmap cephfs:1 {0=cephfs.smithi028.rhpvve=up:active} 1 up:standby-replay 1 up:standby 2024-07-03T15:41:53.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:53 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:41:53.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:53 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:41:53.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:53 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:41:53.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:53 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:41:53.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:53 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:41:53.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:53 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:41:53.751 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:53 smithi028 ceph-mon[29148]: pgmap v179: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s wr, 4 op/s 2024-07-03T15:41:53.751 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:53 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/3917791513' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-07-03T15:41:53.752 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:53 smithi028 ceph-mon[29148]: fsmap cephfs:1 {0=cephfs.smithi028.rhpvve=up:active} 2 up:standby 2024-07-03T15:41:53.752 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:53 smithi028 ceph-mon[29148]: fsmap cephfs:1 {0=cephfs.smithi028.rhpvve=up:active} 1 up:standby-replay 1 up:standby 2024-07-03T15:41:53.752 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:53 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:41:53.752 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:53 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:41:53.752 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:53 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:41:53.752 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:53 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:41:53.752 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:53 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:41:53.752 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:53 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:41:53.975 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:41:54.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:54 smithi148 ceph-mon[36056]: mds.? [v2:172.21.15.148:6826/4171231858,v1:172.21.15.148:6827/4171231858] up:boot 2024-07-03T15:41:54.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:54 smithi148 ceph-mon[36056]: fsmap cephfs:1 {0=cephfs.smithi028.rhpvve=up:active} 1 up:standby-replay 2 up:standby 2024-07-03T15:41:54.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:54 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.szomfl"}]: dispatch 2024-07-03T15:41:54.774 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:54 smithi028 ceph-mon[29148]: mds.? [v2:172.21.15.148:6826/4171231858,v1:172.21.15.148:6827/4171231858] up:boot 2024-07-03T15:41:54.774 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:54 smithi028 ceph-mon[29148]: fsmap cephfs:1 {0=cephfs.smithi028.rhpvve=up:active} 1 up:standby-replay 2 up:standby 2024-07-03T15:41:54.774 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:54 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.szomfl"}]: dispatch 2024-07-03T15:41:55.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:55 smithi028 ceph-mon[29148]: pgmap v180: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 174 B/s rd, 1.1 KiB/s wr, 4 op/s 2024-07-03T15:41:55.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:55 smithi028 ceph-mon[29148]: mds.? [v2:172.21.15.28:6826/1613936042,v1:172.21.15.28:6827/1613936042] up:active 2024-07-03T15:41:55.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:55 smithi028 ceph-mon[29148]: mds.? [v2:172.21.15.28:6828/1730336821,v1:172.21.15.28:6829/1730336821] up:standby 2024-07-03T15:41:55.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:55 smithi028 ceph-mon[29148]: Dropping low affinity standby-replay daemon mds.cephfs.smithi148.xgpuld in favor of higher affinity standby. 2024-07-03T15:41:55.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:55 smithi028 ceph-mon[29148]: fsmap cephfs:1 {0=cephfs.smithi028.rhpvve=up:active} 1 up:standby-replay 2 up:standby 2024-07-03T15:41:55.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:55 smithi028 ceph-mon[29148]: fsmap cephfs:1 {0=cephfs.smithi028.rhpvve=up:active} 1 up:standby-replay 1 up:standby 2024-07-03T15:41:55.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:55 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:41:55.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:55 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:41:55.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:55 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:41:55.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:55 smithi148 ceph-mon[36056]: pgmap v180: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 174 B/s rd, 1.1 KiB/s wr, 4 op/s 2024-07-03T15:41:55.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:55 smithi148 ceph-mon[36056]: mds.? [v2:172.21.15.28:6826/1613936042,v1:172.21.15.28:6827/1613936042] up:active 2024-07-03T15:41:55.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:55 smithi148 ceph-mon[36056]: mds.? [v2:172.21.15.28:6828/1730336821,v1:172.21.15.28:6829/1730336821] up:standby 2024-07-03T15:41:55.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:55 smithi148 ceph-mon[36056]: Dropping low affinity standby-replay daemon mds.cephfs.smithi148.xgpuld in favor of higher affinity standby. 2024-07-03T15:41:55.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:55 smithi148 ceph-mon[36056]: fsmap cephfs:1 {0=cephfs.smithi028.rhpvve=up:active} 1 up:standby-replay 2 up:standby 2024-07-03T15:41:55.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:55 smithi148 ceph-mon[36056]: fsmap cephfs:1 {0=cephfs.smithi028.rhpvve=up:active} 1 up:standby-replay 1 up:standby 2024-07-03T15:41:55.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:55 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:41:55.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:55 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:41:55.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:55 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:41:55.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:55.750+0000 7f8a66a88640 1 -- 172.21.15.28:0/3682829904 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a600fd630 msgr2=0x7f8a600fdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:55.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:55.750+0000 7f8a66a88640 1 --2- 172.21.15.28:0/3682829904 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a600fd630 0x7f8a600fdaa0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c0048d0 tx=0x7f8a4c009da0 comp rx=0 tx=0).stop 2024-07-03T15:41:55.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:55.750+0000 7f8a66a88640 1 -- 172.21.15.28:0/3682829904 shutdown_connections 2024-07-03T15:41:55.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:55.750+0000 7f8a66a88640 1 --2- 172.21.15.28:0/3682829904 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a600fd630 0x7f8a600fdaa0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:55.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:55.750+0000 7f8a66a88640 1 --2- 172.21.15.28:0/3682829904 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a600fc360 0x7f8a600fc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:55.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:55.750+0000 7f8a66a88640 1 -- 172.21.15.28:0/3682829904 >> 172.21.15.28:0/3682829904 conn(0x7f8a600f7b40 msgr2=0x7f8a600f9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:41:55.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:55.750+0000 7f8a66a88640 1 -- 172.21.15.28:0/3682829904 shutdown_connections 2024-07-03T15:41:55.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:55.750+0000 7f8a66a88640 1 -- 172.21.15.28:0/3682829904 wait complete. 2024-07-03T15:41:55.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:55.751+0000 7f8a66a88640 1 Processor -- start 2024-07-03T15:41:55.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:55.751+0000 7f8a66a88640 1 -- start start 2024-07-03T15:41:55.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:55.751+0000 7f8a66a88640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a600fc360 0x7f8a60071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:55.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:55.751+0000 7f8a66a88640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a600fd630 0x7f8a6006f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:55.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:55.751+0000 7f8a66a88640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6006fe00 con 0x7f8a600fd630 2024-07-03T15:41:55.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:55.751+0000 7f8a66a88640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6006ff40 con 0x7f8a600fc360 2024-07-03T15:41:55.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:55.751+0000 7f8a5ffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a600fd630 0x7f8a6006f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:55.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:55.751+0000 7f8a5ffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a600fd630 0x7f8a6006f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47250/0 (socket says 172.21.15.28:47250) 2024-07-03T15:41:55.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:55.751+0000 7f8a5ffff640 1 -- 172.21.15.28:0/1862605900 learned_addr learned my addr 172.21.15.28:0/1862605900 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:41:55.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:55.751+0000 7f8a64824640 1 --2- 172.21.15.28:0/1862605900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a600fc360 0x7f8a60071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:55.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:55.752+0000 7f8a5ffff640 1 -- 172.21.15.28:0/1862605900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a600fc360 msgr2=0x7f8a60071380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:55.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:55.752+0000 7f8a5ffff640 1 --2- 172.21.15.28:0/1862605900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a600fc360 0x7f8a60071380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:55.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:55.752+0000 7f8a5ffff640 1 -- 172.21.15.28:0/1862605900 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a4c005040 con 0x7f8a600fd630 2024-07-03T15:41:55.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:55.752+0000 7f8a5ffff640 1 --2- 172.21.15.28:0/1862605900 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a600fd630 0x7f8a6006f870 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c0048a0 tx=0x7f8a4c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:41:55.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:55.752+0000 7f8a5d7fa640 1 -- 172.21.15.28:0/1862605900 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a4c038070 con 0x7f8a600fd630 2024-07-03T15:41:55.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:55.752+0000 7f8a5d7fa640 1 -- 172.21.15.28:0/1862605900 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8a4c0092b0 con 0x7f8a600fd630 2024-07-03T15:41:55.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:55.752+0000 7f8a66a88640 1 -- 172.21.15.28:0/1862605900 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a600701c0 con 0x7f8a600fd630 2024-07-03T15:41:55.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:55.752+0000 7f8a5d7fa640 1 -- 172.21.15.28:0/1862605900 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a4c0338a0 con 0x7f8a600fd630 2024-07-03T15:41:55.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:55.752+0000 7f8a66a88640 1 -- 172.21.15.28:0/1862605900 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a60114f90 con 0x7f8a600fd630 2024-07-03T15:41:55.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:55.753+0000 7f8a66a88640 1 -- 172.21.15.28:0/1862605900 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a600fb7c0 con 0x7f8a600fd630 2024-07-03T15:41:55.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:55.754+0000 7f8a5d7fa640 1 -- 172.21.15.28:0/1862605900 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f8a4c030020 con 0x7f8a600fd630 2024-07-03T15:41:55.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:55.754+0000 7f8a5d7fa640 1 --2- 172.21.15.28:0/1862605900 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f8a50067190 0x7f8a50069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:41:55.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:55.754+0000 7f8a64824640 1 --2- 172.21.15.28:0/1862605900 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f8a50067190 0x7f8a50069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:41:55.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:55.754+0000 7f8a5d7fa640 1 -- 172.21.15.28:0/1862605900 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f8a4c0a48f0 con 0x7f8a600fd630 2024-07-03T15:41:55.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:55.755+0000 7f8a64824640 1 --2- 172.21.15.28:0/1862605900 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f8a50067190 0x7f8a50069640 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8a60071130 tx=0x7f8a5400b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:41:55.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:55.758+0000 7f8a5d7fa640 1 -- 172.21.15.28:0/1862605900 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8a4c071c40 con 0x7f8a600fd630 2024-07-03T15:41:55.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:55.978+0000 7f8a66a88640 1 -- 172.21.15.28:0/1862605900 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"} v 0) v1 -- 0x7f8a6004b530 con 0x7f8a600fd630 2024-07-03T15:41:56.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:56.460+0000 7f8a5d7fa640 1 -- 172.21.15.28:0/1862605900 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]=0 inline data disabled v16) v1 ==== 133+0+0 (secure 0 0 0) 0x7f8a4c075260 con 0x7f8a600fd630 2024-07-03T15:41:56.464 INFO:teuthology.orchestra.run.smithi028.stderr:inline data disabled 2024-07-03T15:41:56.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:56.462+0000 7f8a66a88640 1 -- 172.21.15.28:0/1862605900 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f8a50067190 msgr2=0x7f8a50069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:56.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:56.462+0000 7f8a66a88640 1 --2- 172.21.15.28:0/1862605900 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f8a50067190 0x7f8a50069640 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8a60071130 tx=0x7f8a5400b040 comp rx=0 tx=0).stop 2024-07-03T15:41:56.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:56.462+0000 7f8a66a88640 1 -- 172.21.15.28:0/1862605900 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a600fd630 msgr2=0x7f8a6006f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:41:56.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:56.462+0000 7f8a66a88640 1 --2- 172.21.15.28:0/1862605900 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a600fd630 0x7f8a6006f870 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c0048a0 tx=0x7f8a4c009da0 comp rx=0 tx=0).stop 2024-07-03T15:41:56.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:56.463+0000 7f8a66a88640 1 -- 172.21.15.28:0/1862605900 shutdown_connections 2024-07-03T15:41:56.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:56.463+0000 7f8a66a88640 1 --2- 172.21.15.28:0/1862605900 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f8a50067190 0x7f8a50069640 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:56.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:56.463+0000 7f8a66a88640 1 --2- 172.21.15.28:0/1862605900 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a600fd630 0x7f8a6006f870 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:56.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:56.463+0000 7f8a66a88640 1 --2- 172.21.15.28:0/1862605900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a600fc360 0x7f8a60071380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:41:56.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:56.463+0000 7f8a66a88640 1 -- 172.21.15.28:0/1862605900 >> 172.21.15.28:0/1862605900 conn(0x7f8a600f7b40 msgr2=0x7f8a600f91e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:41:56.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:56.463+0000 7f8a66a88640 1 -- 172.21.15.28:0/1862605900 shutdown_connections 2024-07-03T15:41:56.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:41:56.463+0000 7f8a66a88640 1 -- 172.21.15.28:0/1862605900 wait complete. 2024-07-03T15:41:56.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:56 smithi148 ceph-mon[36056]: mds.? [v2:172.21.15.148:6824/2347536398,v1:172.21.15.148:6825/2347536398] up:boot 2024-07-03T15:41:56.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:56 smithi148 ceph-mon[36056]: fsmap cephfs:1 {0=cephfs.smithi028.rhpvve=up:active} 1 up:standby-replay 2 up:standby 2024-07-03T15:41:56.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:56 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.xgpuld"}]: dispatch 2024-07-03T15:41:56.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:56 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/1862605900' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-07-03T15:41:56.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:56 smithi028 ceph-mon[29148]: mds.? [v2:172.21.15.148:6824/2347536398,v1:172.21.15.148:6825/2347536398] up:boot 2024-07-03T15:41:56.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:56 smithi028 ceph-mon[29148]: fsmap cephfs:1 {0=cephfs.smithi028.rhpvve=up:active} 1 up:standby-replay 2 up:standby 2024-07-03T15:41:56.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:56 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.xgpuld"}]: dispatch 2024-07-03T15:41:56.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:56 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/1862605900' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-07-03T15:41:57.198 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-07-03T15:41:57.208 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi028.front.sepia.ceph.com 2024-07-03T15:41:57.208 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- bash -c 'ceph fs dump' 2024-07-03T15:41:57.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:57 smithi148 ceph-mon[36056]: pgmap v181: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-07-03T15:41:57.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:57 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/1862605900' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-07-03T15:41:57.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:57 smithi148 ceph-mon[36056]: fsmap cephfs:1 {0=cephfs.smithi028.rhpvve=up:active} 1 up:standby-replay 2 up:standby 2024-07-03T15:41:57.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:57 smithi028 ceph-mon[29148]: pgmap v181: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-07-03T15:41:57.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:57 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/1862605900' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-07-03T15:41:57.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:57 smithi028 ceph-mon[29148]: fsmap cephfs:1 {0=cephfs.smithi028.rhpvve=up:active} 1 up:standby-replay 2 up:standby 2024-07-03T15:41:58.088 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:41:58.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:58 smithi028 ceph-mon[29148]: mds.? [v2:172.21.15.148:6826/4171231858,v1:172.21.15.148:6827/4171231858] up:standby 2024-07-03T15:41:58.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:58 smithi028 ceph-mon[29148]: fsmap cephfs:1 {0=cephfs.smithi028.rhpvve=up:active} 1 up:standby-replay 2 up:standby 2024-07-03T15:41:58.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:58 smithi148 ceph-mon[36056]: mds.? [v2:172.21.15.148:6826/4171231858,v1:172.21.15.148:6827/4171231858] up:standby 2024-07-03T15:41:58.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:58 smithi148 ceph-mon[36056]: fsmap cephfs:1 {0=cephfs.smithi028.rhpvve=up:active} 1 up:standby-replay 2 up:standby 2024-07-03T15:41:59.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:41:59 smithi148 ceph-mon[36056]: pgmap v182: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 853 B/s wr, 4 op/s 2024-07-03T15:41:59.793 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:41:59 smithi028 ceph-mon[29148]: pgmap v182: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 853 B/s wr, 4 op/s 2024-07-03T15:42:00.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.541+0000 7f298b1cc640 1 -- 172.21.15.28:0/1130921797 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f29840fd9b0 msgr2=0x7f29840fddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:00.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.541+0000 7f298b1cc640 1 --2- 172.21.15.28:0/1130921797 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f29840fd9b0 0x7f29840fddc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f29800048d0 tx=0x7f2980009da0 comp rx=0 tx=0).stop 2024-07-03T15:42:00.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.541+0000 7f298b1cc640 1 -- 172.21.15.28:0/1130921797 shutdown_connections 2024-07-03T15:42:00.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.541+0000 7f298b1cc640 1 --2- 172.21.15.28:0/1130921797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29840fe300 0x7f29840fa460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:00.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.541+0000 7f298b1cc640 1 --2- 172.21.15.28:0/1130921797 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f29840fd9b0 0x7f29840fddc0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:00.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.541+0000 7f298b1cc640 1 -- 172.21.15.28:0/1130921797 >> 172.21.15.28:0/1130921797 conn(0x7f29840f61b0 msgr2=0x7f29840f85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:42:00.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.541+0000 7f298b1cc640 1 -- 172.21.15.28:0/1130921797 shutdown_connections 2024-07-03T15:42:00.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.541+0000 7f298b1cc640 1 -- 172.21.15.28:0/1130921797 wait complete. 2024-07-03T15:42:00.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.541+0000 7f298b1cc640 1 Processor -- start 2024-07-03T15:42:00.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.541+0000 7f298b1cc640 1 -- start start 2024-07-03T15:42:00.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.542+0000 7f298b1cc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29840fd9b0 0x7f298410d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:00.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.542+0000 7f298b1cc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f29840fe300 0x7f298410d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:00.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.542+0000 7f298b1cc640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298410dbc0 con 0x7f29840fe300 2024-07-03T15:42:00.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.542+0000 7f298b1cc640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298410e390 con 0x7f29840fd9b0 2024-07-03T15:42:00.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.542+0000 7f297bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f29840fe300 0x7f298410d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:00.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.542+0000 7f297bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f29840fe300 0x7f298410d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46656/0 (socket says 172.21.15.28:46656) 2024-07-03T15:42:00.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.542+0000 7f297bfff640 1 -- 172.21.15.28:0/1144082654 learned_addr learned my addr 172.21.15.28:0/1144082654 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:42:00.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.542+0000 7f2988f68640 1 --2- 172.21.15.28:0/1144082654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29840fd9b0 0x7f298410d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:00.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.542+0000 7f297bfff640 1 -- 172.21.15.28:0/1144082654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29840fd9b0 msgr2=0x7f298410d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:00.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.542+0000 7f297bfff640 1 --2- 172.21.15.28:0/1144082654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29840fd9b0 0x7f298410d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:00.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.542+0000 7f297bfff640 1 -- 172.21.15.28:0/1144082654 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2980005040 con 0x7f29840fe300 2024-07-03T15:42:00.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.542+0000 7f2988f68640 1 --2- 172.21.15.28:0/1144082654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29840fd9b0 0x7f298410d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-03T15:42:00.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.542+0000 7f297bfff640 1 --2- 172.21.15.28:0/1144082654 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f29840fe300 0x7f298410d570 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2974002870 tx=0x7f2974004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:42:00.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.542+0000 7f29797fa640 1 -- 172.21.15.28:0/1144082654 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2974007c30 con 0x7f29840fe300 2024-07-03T15:42:00.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.543+0000 7f298b1cc640 1 -- 172.21.15.28:0/1144082654 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f298410e620 con 0x7f29840fe300 2024-07-03T15:42:00.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.543+0000 7f29797fa640 1 -- 172.21.15.28:0/1144082654 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f297400f040 con 0x7f29840fe300 2024-07-03T15:42:00.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.543+0000 7f29797fa640 1 -- 172.21.15.28:0/1144082654 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29740095f0 con 0x7f29840fe300 2024-07-03T15:42:00.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.543+0000 7f298b1cc640 1 -- 172.21.15.28:0/1144082654 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f298410ebc0 con 0x7f29840fe300 2024-07-03T15:42:00.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.544+0000 7f298b1cc640 1 -- 172.21.15.28:0/1144082654 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29840f9e50 con 0x7f29840fe300 2024-07-03T15:42:00.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.544+0000 7f29797fa640 1 -- 172.21.15.28:0/1144082654 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f297400d6b0 con 0x7f29840fe300 2024-07-03T15:42:00.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.545+0000 7f29797fa640 1 --2- 172.21.15.28:0/1144082654 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f296c067190 0x7f296c069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:00.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.545+0000 7f29797fa640 1 -- 172.21.15.28:0/1144082654 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f2974084350 con 0x7f29840fe300 2024-07-03T15:42:00.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.545+0000 7f2988f68640 1 --2- 172.21.15.28:0/1144082654 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f296c067190 0x7f296c069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:00.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.545+0000 7f2988f68640 1 --2- 172.21.15.28:0/1144082654 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f296c067190 0x7f296c069640 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f29800047f0 tx=0x7f2980009da0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:42:00.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.549+0000 7f29797fa640 1 -- 172.21.15.28:0/1144082654 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f29740516b0 con 0x7f29840fe300 2024-07-03T15:42:00.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.782+0000 7f298b1cc640 1 -- 172.21.15.28:0/1144082654 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f298410ee80 con 0x7f29840fe300 2024-07-03T15:42:00.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.783+0000 7f29797fa640 1 -- 172.21.15.28:0/1144082654 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1838 (secure 0 0 0) 0x7f2974054cd0 con 0x7f29840fe300 2024-07-03T15:42:00.787 INFO:teuthology.orchestra.run.smithi028.stdout:e17 2024-07-03T15:42:00.787 INFO:teuthology.orchestra.run.smithi028.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-03T15:42:00.787 INFO:teuthology.orchestra.run.smithi028.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:42:00.787 INFO:teuthology.orchestra.run.smithi028.stdout:legacy client fscid: 1 2024-07-03T15:42:00.787 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:42:00.787 INFO:teuthology.orchestra.run.smithi028.stdout:Filesystem 'cephfs' (1) 2024-07-03T15:42:00.787 INFO:teuthology.orchestra.run.smithi028.stdout:fs_name cephfs 2024-07-03T15:42:00.787 INFO:teuthology.orchestra.run.smithi028.stdout:epoch 16 2024-07-03T15:42:00.787 INFO:teuthology.orchestra.run.smithi028.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-03T15:42:00.788 INFO:teuthology.orchestra.run.smithi028.stdout:created 2024-07-03T15:41:40.656298+0000 2024-07-03T15:42:00.788 INFO:teuthology.orchestra.run.smithi028.stdout:modified 2024-07-03T15:41:56.454984+0000 2024-07-03T15:42:00.788 INFO:teuthology.orchestra.run.smithi028.stdout:tableserver 0 2024-07-03T15:42:00.788 INFO:teuthology.orchestra.run.smithi028.stdout:root 0 2024-07-03T15:42:00.788 INFO:teuthology.orchestra.run.smithi028.stdout:session_timeout 60 2024-07-03T15:42:00.788 INFO:teuthology.orchestra.run.smithi028.stdout:session_autoclose 300 2024-07-03T15:42:00.788 INFO:teuthology.orchestra.run.smithi028.stdout:max_file_size 1099511627776 2024-07-03T15:42:00.788 INFO:teuthology.orchestra.run.smithi028.stdout:required_client_features {} 2024-07-03T15:42:00.788 INFO:teuthology.orchestra.run.smithi028.stdout:last_failure 0 2024-07-03T15:42:00.788 INFO:teuthology.orchestra.run.smithi028.stdout:last_failure_osd_epoch 0 2024-07-03T15:42:00.788 INFO:teuthology.orchestra.run.smithi028.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:42:00.788 INFO:teuthology.orchestra.run.smithi028.stdout:max_mds 1 2024-07-03T15:42:00.789 INFO:teuthology.orchestra.run.smithi028.stdout:in 0 2024-07-03T15:42:00.789 INFO:teuthology.orchestra.run.smithi028.stdout:up {0=14450} 2024-07-03T15:42:00.789 INFO:teuthology.orchestra.run.smithi028.stdout:failed 2024-07-03T15:42:00.789 INFO:teuthology.orchestra.run.smithi028.stdout:damaged 2024-07-03T15:42:00.789 INFO:teuthology.orchestra.run.smithi028.stdout:stopped 2024-07-03T15:42:00.789 INFO:teuthology.orchestra.run.smithi028.stdout:data_pools [3] 2024-07-03T15:42:00.789 INFO:teuthology.orchestra.run.smithi028.stdout:metadata_pool 2 2024-07-03T15:42:00.789 INFO:teuthology.orchestra.run.smithi028.stdout:inline_data disabled 2024-07-03T15:42:00.789 INFO:teuthology.orchestra.run.smithi028.stdout:balancer 2024-07-03T15:42:00.789 INFO:teuthology.orchestra.run.smithi028.stdout:standby_count_wanted 1 2024-07-03T15:42:00.789 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi028.rhpvve{0:14450} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.28:6826/1613936042,v1:172.21.15.28:6827/1613936042] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:42:00.789 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi028.wmbxew{0:14460} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.28:6828/1730336821,v1:172.21.15.28:6829/1730336821] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:42:00.790 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:42:00.790 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:42:00.790 INFO:teuthology.orchestra.run.smithi028.stdout:Standby daemons: 2024-07-03T15:42:00.790 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:42:00.790 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi148.szomfl{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/4171231858,v1:172.21.15.148:6827/4171231858] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:42:00.790 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi148.xgpuld{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/2347536398,v1:172.21.15.148:6825/2347536398] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:42:00.791 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 17 2024-07-03T15:42:00.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.785+0000 7f298b1cc640 1 -- 172.21.15.28:0/1144082654 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f296c067190 msgr2=0x7f296c069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:00.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.785+0000 7f298b1cc640 1 --2- 172.21.15.28:0/1144082654 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f296c067190 0x7f296c069640 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f29800047f0 tx=0x7f2980009da0 comp rx=0 tx=0).stop 2024-07-03T15:42:00.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.785+0000 7f298b1cc640 1 -- 172.21.15.28:0/1144082654 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f29840fe300 msgr2=0x7f298410d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:00.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.785+0000 7f298b1cc640 1 --2- 172.21.15.28:0/1144082654 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f29840fe300 0x7f298410d570 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2974002870 tx=0x7f2974004600 comp rx=0 tx=0).stop 2024-07-03T15:42:00.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.785+0000 7f298b1cc640 1 -- 172.21.15.28:0/1144082654 shutdown_connections 2024-07-03T15:42:00.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.785+0000 7f298b1cc640 1 --2- 172.21.15.28:0/1144082654 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f296c067190 0x7f296c069640 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:00.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.785+0000 7f298b1cc640 1 --2- 172.21.15.28:0/1144082654 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f29840fe300 0x7f298410d570 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:00.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.785+0000 7f298b1cc640 1 --2- 172.21.15.28:0/1144082654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29840fd9b0 0x7f298410d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:00.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.785+0000 7f298b1cc640 1 -- 172.21.15.28:0/1144082654 >> 172.21.15.28:0/1144082654 conn(0x7f29840f61b0 msgr2=0x7f29840f7ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:42:00.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.785+0000 7f298b1cc640 1 -- 172.21.15.28:0/1144082654 shutdown_connections 2024-07-03T15:42:00.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:00.785+0000 7f298b1cc640 1 -- 172.21.15.28:0/1144082654 wait complete. 2024-07-03T15:42:01.490 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:01 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:42:01.490 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:01 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:42:01.490 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:01 smithi028 ceph-mon[29148]: pgmap v183: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 682 B/s wr, 4 op/s 2024-07-03T15:42:01.490 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:01 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:42:01.490 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:01 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:42:01.490 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:01 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:42:01.490 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:01 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:42:01.490 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:01 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/1144082654' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-03T15:42:01.490 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:01 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:42:01.490 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:01 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:42:01.491 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-07-03T15:42:01.686 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:01 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:42:01.686 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:01 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:42:01.686 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:01 smithi148 ceph-mon[36056]: pgmap v183: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 682 B/s wr, 4 op/s 2024-07-03T15:42:01.686 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:01 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:42:01.686 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:01 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:42:01.686 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:01 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:42:01.686 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:01 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:42:01.686 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:01 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/1144082654' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-03T15:42:01.687 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:01 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:42:01.687 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:01 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:42:02.339 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:42:03.689 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:03 smithi028 ceph-mon[29148]: pgmap v184: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-07-03T15:42:03.689 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:03 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:42:03.689 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:03 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:42:03.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:03 smithi148 ceph-mon[36056]: pgmap v184: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-07-03T15:42:03.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:03 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:42:03.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:03 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:42:03.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:03.883+0000 7f21fe607640 1 -- 172.21.15.28:0/1853124372 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21f80fd630 msgr2=0x7f21f80fdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:03.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:03.883+0000 7f21fe607640 1 --2- 172.21.15.28:0/1853124372 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21f80fd630 0x7f21f80fdaa0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f21e80048d0 tx=0x7f21e8009da0 comp rx=0 tx=0).stop 2024-07-03T15:42:03.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:03.883+0000 7f21fe607640 1 -- 172.21.15.28:0/1853124372 shutdown_connections 2024-07-03T15:42:03.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:03.883+0000 7f21fe607640 1 --2- 172.21.15.28:0/1853124372 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21f80fd630 0x7f21f80fdaa0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:03.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:03.883+0000 7f21fe607640 1 --2- 172.21.15.28:0/1853124372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21f80fc360 0x7f21f80fc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:03.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:03.883+0000 7f21fe607640 1 -- 172.21.15.28:0/1853124372 >> 172.21.15.28:0/1853124372 conn(0x7f21f80f7b40 msgr2=0x7f21f80f9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:42:03.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:03.883+0000 7f21fe607640 1 -- 172.21.15.28:0/1853124372 shutdown_connections 2024-07-03T15:42:03.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:03.883+0000 7f21fe607640 1 -- 172.21.15.28:0/1853124372 wait complete. 2024-07-03T15:42:03.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:03.884+0000 7f21fe607640 1 Processor -- start 2024-07-03T15:42:03.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:03.884+0000 7f21fe607640 1 -- start start 2024-07-03T15:42:03.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:03.884+0000 7f21fe607640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21f80fc360 0x7f21f8071310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:03.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:03.884+0000 7f21fe607640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21f80fd630 0x7f21f806f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:03.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:03.884+0000 7f21fe607640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f806fdb0 con 0x7f21f80fc360 2024-07-03T15:42:03.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:03.884+0000 7f21fe607640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f806ff20 con 0x7f21f80fd630 2024-07-03T15:42:03.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:03.884+0000 7f21f7fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21f80fc360 0x7f21f8071310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:03.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:03.884+0000 7f21f7fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21f80fc360 0x7f21f8071310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46674/0 (socket says 172.21.15.28:46674) 2024-07-03T15:42:03.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:03.884+0000 7f21f7fff640 1 -- 172.21.15.28:0/1275721475 learned_addr learned my addr 172.21.15.28:0/1275721475 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:42:03.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:03.885+0000 7f21f77fe640 1 --2- 172.21.15.28:0/1275721475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21f80fd630 0x7f21f806f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:03.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:03.885+0000 7f21f7fff640 1 -- 172.21.15.28:0/1275721475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21f80fd630 msgr2=0x7f21f806f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:03.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:03.885+0000 7f21f7fff640 1 --2- 172.21.15.28:0/1275721475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21f80fd630 0x7f21f806f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:03.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:03.885+0000 7f21f7fff640 1 -- 172.21.15.28:0/1275721475 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e8005040 con 0x7f21f80fc360 2024-07-03T15:42:03.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:03.885+0000 7f21f7fff640 1 --2- 172.21.15.28:0/1275721475 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21f80fc360 0x7f21f8071310 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f21e000c790 tx=0x7f21e00043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:42:03.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:03.885+0000 7f21f4ff9640 1 -- 172.21.15.28:0/1275721475 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21e000a820 con 0x7f21f80fc360 2024-07-03T15:42:03.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:03.885+0000 7f21f4ff9640 1 -- 172.21.15.28:0/1275721475 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f21e000ae50 con 0x7f21f80fc360 2024-07-03T15:42:03.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:03.885+0000 7f21fe607640 1 -- 172.21.15.28:0/1275721475 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21f80701b0 con 0x7f21f80fc360 2024-07-03T15:42:03.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:03.885+0000 7f21f4ff9640 1 -- 172.21.15.28:0/1275721475 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21e00115e0 con 0x7f21f80fc360 2024-07-03T15:42:03.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:03.885+0000 7f21fe607640 1 -- 172.21.15.28:0/1275721475 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21f8114fd0 con 0x7f21f80fc360 2024-07-03T15:42:03.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:03.886+0000 7f21fe607640 1 -- 172.21.15.28:0/1275721475 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21f80fb7c0 con 0x7f21f80fc360 2024-07-03T15:42:03.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:03.887+0000 7f21f4ff9640 1 -- 172.21.15.28:0/1275721475 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f21e0012400 con 0x7f21f80fc360 2024-07-03T15:42:03.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:03.887+0000 7f21f4ff9640 1 --2- 172.21.15.28:0/1275721475 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f21e4067190 0x7f21e4069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:03.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:03.887+0000 7f21f4ff9640 1 -- 172.21.15.28:0/1275721475 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f21e0084ce0 con 0x7f21f80fc360 2024-07-03T15:42:03.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:03.887+0000 7f21f77fe640 1 --2- 172.21.15.28:0/1275721475 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f21e4067190 0x7f21e4069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:03.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:03.888+0000 7f21f77fe640 1 --2- 172.21.15.28:0/1275721475 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f21e4067190 0x7f21e4069640 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f21e8000c00 tx=0x7f21e8035040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:42:03.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:03.891+0000 7f21f4ff9640 1 -- 172.21.15.28:0/1275721475 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f21e0052000 con 0x7f21f80fc360 2024-07-03T15:42:04.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:04.109+0000 7f21fe607640 1 -- 172.21.15.28:0/1275721475 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f21f804b530 con 0x7f21f80fc360 2024-07-03T15:42:04.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:04.110+0000 7f21f4ff9640 1 -- 172.21.15.28:0/1275721475 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4700 (secure 0 0 0) 0x7f21e0055620 con 0x7f21f80fc360 2024-07-03T15:42:04.113 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 17 2024-07-03T15:42:04.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:04.112+0000 7f21fe607640 1 -- 172.21.15.28:0/1275721475 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f21e4067190 msgr2=0x7f21e4069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:04.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:04.112+0000 7f21fe607640 1 --2- 172.21.15.28:0/1275721475 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f21e4067190 0x7f21e4069640 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f21e8000c00 tx=0x7f21e8035040 comp rx=0 tx=0).stop 2024-07-03T15:42:04.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:04.112+0000 7f21fe607640 1 -- 172.21.15.28:0/1275721475 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21f80fc360 msgr2=0x7f21f8071310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:04.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:04.112+0000 7f21fe607640 1 --2- 172.21.15.28:0/1275721475 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21f80fc360 0x7f21f8071310 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f21e000c790 tx=0x7f21e00043c0 comp rx=0 tx=0).stop 2024-07-03T15:42:04.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:04.112+0000 7f21fe607640 1 -- 172.21.15.28:0/1275721475 shutdown_connections 2024-07-03T15:42:04.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:04.112+0000 7f21fe607640 1 --2- 172.21.15.28:0/1275721475 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f21e4067190 0x7f21e4069640 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:04.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:04.112+0000 7f21fe607640 1 --2- 172.21.15.28:0/1275721475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21f80fd630 0x7f21f806f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:04.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:04.112+0000 7f21fe607640 1 --2- 172.21.15.28:0/1275721475 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21f80fc360 0x7f21f8071310 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:04.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:04.112+0000 7f21fe607640 1 -- 172.21.15.28:0/1275721475 >> 172.21.15.28:0/1275721475 conn(0x7f21f80f7b40 msgr2=0x7f21f80f9230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:42:04.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:04.112+0000 7f21fe607640 1 -- 172.21.15.28:0/1275721475 shutdown_connections 2024-07-03T15:42:04.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:04.112+0000 7f21fe607640 1 -- 172.21.15.28:0/1275721475 wait complete. 2024-07-03T15:42:04.124 INFO:teuthology.orchestra.run.smithi028.stdout:true 2024-07-03T15:42:04.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:04 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/1275721475' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-03T15:42:04.861 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:04 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/1275721475' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-03T15:42:04.889 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-07-03T15:42:05.251 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:42:05.690 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:05 smithi028 ceph-mon[29148]: pgmap v185: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 2 op/s 2024-07-03T15:42:05.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:05 smithi148 ceph-mon[36056]: pgmap v185: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 2 op/s 2024-07-03T15:42:07.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:07 smithi028 ceph-mon[29148]: pgmap v186: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 3 op/s 2024-07-03T15:42:07.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:07 smithi148 ceph-mon[36056]: pgmap v186: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 3 op/s 2024-07-03T15:42:08.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.147+0000 7f2edd2de640 1 -- 172.21.15.28:0/2206129135 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ed80ff4b0 msgr2=0x7f2ed8101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:08.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.147+0000 7f2edd2de640 1 --2- 172.21.15.28:0/2206129135 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ed80ff4b0 0x7f2ed8101890 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2ec00048d0 tx=0x7f2ec0009da0 comp rx=0 tx=0).stop 2024-07-03T15:42:08.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.147+0000 7f2edd2de640 1 -- 172.21.15.28:0/2206129135 shutdown_connections 2024-07-03T15:42:08.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.147+0000 7f2edd2de640 1 --2- 172.21.15.28:0/2206129135 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ed80ff4b0 0x7f2ed8101890 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:08.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.147+0000 7f2edd2de640 1 --2- 172.21.15.28:0/2206129135 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ed8066520 0x7f2ed80fef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:08.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.147+0000 7f2edd2de640 1 -- 172.21.15.28:0/2206129135 >> 172.21.15.28:0/2206129135 conn(0x7f2ed80f6a20 msgr2=0x7f2ed80f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:42:08.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.147+0000 7f2edd2de640 1 -- 172.21.15.28:0/2206129135 shutdown_connections 2024-07-03T15:42:08.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.147+0000 7f2edd2de640 1 -- 172.21.15.28:0/2206129135 wait complete. 2024-07-03T15:42:08.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.148+0000 7f2edd2de640 1 Processor -- start 2024-07-03T15:42:08.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.148+0000 7f2edd2de640 1 -- start start 2024-07-03T15:42:08.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.148+0000 7f2edd2de640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ed8066520 0x7f2ed810d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:08.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.148+0000 7f2edd2de640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ed80ff4b0 0x7f2ed810d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:08.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.148+0000 7f2edd2de640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed810e440 con 0x7f2ed80ff4b0 2024-07-03T15:42:08.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.148+0000 7f2edd2de640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed810e5b0 con 0x7f2ed8066520 2024-07-03T15:42:08.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.148+0000 7f2ecffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ed80ff4b0 0x7f2ed810d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:08.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.148+0000 7f2ecffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ed80ff4b0 0x7f2ed810d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46692/0 (socket says 172.21.15.28:46692) 2024-07-03T15:42:08.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.148+0000 7f2ecffff640 1 -- 172.21.15.28:0/1264242927 learned_addr learned my addr 172.21.15.28:0/1264242927 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:42:08.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.148+0000 7f2ed6ffd640 1 --2- 172.21.15.28:0/1264242927 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ed8066520 0x7f2ed810d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:08.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.148+0000 7f2ecffff640 1 -- 172.21.15.28:0/1264242927 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ed8066520 msgr2=0x7f2ed810d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:08.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.148+0000 7f2ecffff640 1 --2- 172.21.15.28:0/1264242927 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ed8066520 0x7f2ed810d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:08.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.148+0000 7f2ecffff640 1 -- 172.21.15.28:0/1264242927 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec0005040 con 0x7f2ed80ff4b0 2024-07-03T15:42:08.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.148+0000 7f2ed6ffd640 1 --2- 172.21.15.28:0/1264242927 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ed8066520 0x7f2ed810d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-03T15:42:08.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.149+0000 7f2ecffff640 1 --2- 172.21.15.28:0/1264242927 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ed80ff4b0 0x7f2ed810d5d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0003080 tx=0x7f2ec0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:42:08.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.149+0000 7f2ecf7fe640 1 -- 172.21.15.28:0/1264242927 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ec0038070 con 0x7f2ed80ff4b0 2024-07-03T15:42:08.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.149+0000 7f2ecf7fe640 1 -- 172.21.15.28:0/1264242927 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2ec0032970 con 0x7f2ed80ff4b0 2024-07-03T15:42:08.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.149+0000 7f2edd2de640 1 -- 172.21.15.28:0/1264242927 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ed810e830 con 0x7f2ed80ff4b0 2024-07-03T15:42:08.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.149+0000 7f2ecf7fe640 1 -- 172.21.15.28:0/1264242927 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ec0033820 con 0x7f2ed80ff4b0 2024-07-03T15:42:08.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.149+0000 7f2edd2de640 1 -- 172.21.15.28:0/1264242927 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ed810ecf0 con 0x7f2ed80ff4b0 2024-07-03T15:42:08.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.150+0000 7f2edd2de640 1 -- 172.21.15.28:0/1264242927 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ed80fa6c0 con 0x7f2ed80ff4b0 2024-07-03T15:42:08.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.150+0000 7f2ecf7fe640 1 -- 172.21.15.28:0/1264242927 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f2ec0030020 con 0x7f2ed80ff4b0 2024-07-03T15:42:08.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.151+0000 7f2ecf7fe640 1 --2- 172.21.15.28:0/1264242927 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f2ec4067190 0x7f2ec4069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:08.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.151+0000 7f2ecf7fe640 1 -- 172.21.15.28:0/1264242927 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f2ec00a4a40 con 0x7f2ed80ff4b0 2024-07-03T15:42:08.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.151+0000 7f2ed6ffd640 1 --2- 172.21.15.28:0/1264242927 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f2ec4067190 0x7f2ec4069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:08.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.152+0000 7f2ed6ffd640 1 --2- 172.21.15.28:0/1264242927 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f2ec4067190 0x7f2ec4069640 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8006fd0 tx=0x7f2ec8008040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:42:08.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.155+0000 7f2ecf7fe640 1 -- 172.21.15.28:0/1264242927 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2ec0071d60 con 0x7f2ed80ff4b0 2024-07-03T15:42:08.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.405+0000 7f2edd2de640 1 -- 172.21.15.28:0/1264242927 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7f2ed804b530 con 0x7f2ed80ff4b0 2024-07-03T15:42:08.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.406+0000 7f2ecf7fe640 1 -- 172.21.15.28:0/1264242927 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v17) v1 ==== 78+0+100 (secure 0 0 0) 0x7f2ec0041080 con 0x7f2ed80ff4b0 2024-07-03T15:42:08.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.408+0000 7f2edd2de640 1 -- 172.21.15.28:0/1264242927 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f2ec4067190 msgr2=0x7f2ec4069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:08.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.408+0000 7f2edd2de640 1 --2- 172.21.15.28:0/1264242927 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f2ec4067190 0x7f2ec4069640 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8006fd0 tx=0x7f2ec8008040 comp rx=0 tx=0).stop 2024-07-03T15:42:08.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.408+0000 7f2edd2de640 1 -- 172.21.15.28:0/1264242927 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ed80ff4b0 msgr2=0x7f2ed810d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:08.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.408+0000 7f2edd2de640 1 --2- 172.21.15.28:0/1264242927 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ed80ff4b0 0x7f2ed810d5d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0003080 tx=0x7f2ec0009da0 comp rx=0 tx=0).stop 2024-07-03T15:42:08.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.408+0000 7f2edd2de640 1 -- 172.21.15.28:0/1264242927 shutdown_connections 2024-07-03T15:42:08.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.408+0000 7f2edd2de640 1 --2- 172.21.15.28:0/1264242927 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f2ec4067190 0x7f2ec4069640 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:08.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.408+0000 7f2edd2de640 1 --2- 172.21.15.28:0/1264242927 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ed80ff4b0 0x7f2ed810d5d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:08.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.408+0000 7f2edd2de640 1 --2- 172.21.15.28:0/1264242927 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ed8066520 0x7f2ed810d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:08.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.408+0000 7f2edd2de640 1 -- 172.21.15.28:0/1264242927 >> 172.21.15.28:0/1264242927 conn(0x7f2ed80f6a20 msgr2=0x7f2ed80f6e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:42:08.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.408+0000 7f2edd2de640 1 -- 172.21.15.28:0/1264242927 shutdown_connections 2024-07-03T15:42:08.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:08.408+0000 7f2edd2de640 1 -- 172.21.15.28:0/1264242927 wait complete. 2024-07-03T15:42:08.420 INFO:teuthology.orchestra.run.smithi028.stdout:true 2024-07-03T15:42:08.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:08 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/1264242927' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-07-03T15:42:08.785 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:08 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/1264242927' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-07-03T15:42:09.114 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-07-03T15:42:09.126 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-03T15:42:09.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.269+0000 7f436711d640 1 -- 172.21.15.28:0/1750355655 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43600ff390 msgr2=0x7f4360101770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:09.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.269+0000 7f436711d640 1 --2- 172.21.15.28:0/1750355655 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43600ff390 0x7f4360101770 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f43500048d0 tx=0x7f4350009da0 comp rx=0 tx=0).stop 2024-07-03T15:42:09.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.269+0000 7f436711d640 1 -- 172.21.15.28:0/1750355655 shutdown_connections 2024-07-03T15:42:09.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.269+0000 7f436711d640 1 --2- 172.21.15.28:0/1750355655 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43600ff390 0x7f4360101770 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:09.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.269+0000 7f436711d640 1 --2- 172.21.15.28:0/1750355655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4360066240 0x7f43600fee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:09.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.269+0000 7f436711d640 1 -- 172.21.15.28:0/1750355655 >> 172.21.15.28:0/1750355655 conn(0x7f43600f6900 msgr2=0x7f43600f8d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:42:09.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.269+0000 7f436711d640 1 -- 172.21.15.28:0/1750355655 shutdown_connections 2024-07-03T15:42:09.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.269+0000 7f436711d640 1 -- 172.21.15.28:0/1750355655 wait complete. 2024-07-03T15:42:09.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.269+0000 7f436711d640 1 Processor -- start 2024-07-03T15:42:09.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.269+0000 7f436711d640 1 -- start start 2024-07-03T15:42:09.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.270+0000 7f436711d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4360066240 0x7f436010cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:09.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.270+0000 7f436711d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43600ff390 0x7f436010d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:09.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.270+0000 7f436711d640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436010d8f0 con 0x7f4360066240 2024-07-03T15:42:09.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.270+0000 7f436711d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43601122b0 con 0x7f43600ff390 2024-07-03T15:42:09.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.270+0000 7f4364eb9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4360066240 0x7f436010cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:09.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.270+0000 7f4364eb9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4360066240 0x7f436010cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46702/0 (socket says 172.21.15.28:46702) 2024-07-03T15:42:09.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.270+0000 7f4364eb9640 1 -- 172.21.15.28:0/2402049562 learned_addr learned my addr 172.21.15.28:0/2402049562 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:42:09.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.270+0000 7f435ffff640 1 --2- 172.21.15.28:0/2402049562 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43600ff390 0x7f436010d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:09.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.270+0000 7f4364eb9640 1 -- 172.21.15.28:0/2402049562 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43600ff390 msgr2=0x7f436010d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:09.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.270+0000 7f4364eb9640 1 --2- 172.21.15.28:0/2402049562 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43600ff390 0x7f436010d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:09.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.270+0000 7f4364eb9640 1 -- 172.21.15.28:0/2402049562 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4350005040 con 0x7f4360066240 2024-07-03T15:42:09.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.270+0000 7f4364eb9640 1 --2- 172.21.15.28:0/2402049562 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4360066240 0x7f436010cd60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f435400c810 tx=0x7f43540043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:42:09.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.270+0000 7f435d7fa640 1 -- 172.21.15.28:0/2402049562 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f435400a820 con 0x7f4360066240 2024-07-03T15:42:09.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.270+0000 7f436711d640 1 -- 172.21.15.28:0/2402049562 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4360112450 con 0x7f4360066240 2024-07-03T15:42:09.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.270+0000 7f435d7fa640 1 -- 172.21.15.28:0/2402049562 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f435400ae50 con 0x7f4360066240 2024-07-03T15:42:09.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.270+0000 7f435d7fa640 1 -- 172.21.15.28:0/2402049562 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43540115c0 con 0x7f4360066240 2024-07-03T15:42:09.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.270+0000 7f436711d640 1 -- 172.21.15.28:0/2402049562 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4360112920 con 0x7f4360066240 2024-07-03T15:42:09.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.271+0000 7f436711d640 1 -- 172.21.15.28:0/2402049562 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f436004bd70 con 0x7f4360066240 2024-07-03T15:42:09.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.272+0000 7f435d7fa640 1 -- 172.21.15.28:0/2402049562 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f4354012420 con 0x7f4360066240 2024-07-03T15:42:09.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.272+0000 7f435d7fa640 1 --2- 172.21.15.28:0/2402049562 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f4348067190 0x7f4348069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:09.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.273+0000 7f435ffff640 1 --2- 172.21.15.28:0/2402049562 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f4348067190 0x7f4348069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:09.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.273+0000 7f435d7fa640 1 -- 172.21.15.28:0/2402049562 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f4354084ef0 con 0x7f4360066240 2024-07-03T15:42:09.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.273+0000 7f435ffff640 1 --2- 172.21.15.28:0/2402049562 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f4348067190 0x7f4348069640 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f43500317e0 tx=0x7f4350009da0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:42:09.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.277+0000 7f435d7fa640 1 -- 172.21.15.28:0/2402049562 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4354052190 con 0x7f4360066240 2024-07-03T15:42:09.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.454+0000 7f436711d640 1 -- 172.21.15.28:0/2402049562 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4360104d90 con 0x7f4360066240 2024-07-03T15:42:09.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.456+0000 7f435d7fa640 1 -- 172.21.15.28:0/2402049562 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+14278 (secure 0 0 0) 0x7f43540030b0 con 0x7f4360066240 2024-07-03T15:42:09.459 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:42:09.459 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":45,"fsid":"a85e4be6-3951-11ef-bcab-c7b262605968","created":"2024-07-03T15:34:56.026978+0000","modified":"2024-07-03T15:41:42.660381+0000","last_up_change":"2024-07-03T15:40:19.490284+0000","last_in_change":"2024-07-03T15:40:01.150891+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-03T15:39:08.833339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-07-03T15:41:39.583625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-07-03T15:41:40.084538+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7ecd90b5-3715-4ba7-a696-588843e3582c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3973972034},{"type":"v1","addr":"172.21.15.28:6803","nonce":3973972034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3973972034},{"type":"v1","addr":"172.21.15.28:6805","nonce":3973972034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":3973972034},{"type":"v1","addr":"172.21.15.28:6809","nonce":3973972034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3973972034},{"type":"v1","addr":"172.21.15.28:6807","nonce":3973972034}]},"public_addr":"172.21.15.28:6803/3973972034","cluster_addr":"172.21.15.28:6805/3973972034","heartbeat_back_addr":"172.21.15.28:6809/3973972034","heartbeat_front_addr":"172.21.15.28:6807/3973972034","state":["exists","up"]},{"osd":1,"uuid":"31c5ae0c-6c57-4615-902b-72bf49fa5b72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1642389568},{"type":"v1","addr":"172.21.15.28:6811","nonce":1642389568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1642389568},{"type":"v1","addr":"172.21.15.28:6813","nonce":1642389568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":1642389568},{"type":"v1","addr":"172.21.15.28:6817","nonce":1642389568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1642389568},{"type":"v1","addr":"172.21.15.28:6815","nonce":1642389568}]},"public_addr":"172.21.15.28:6811/1642389568","cluster_addr":"172.21.15.28:6813/1642389568","heartbeat_back_addr":"172.21.15.28:6817/1642389568","heartbeat_front_addr":"172.21.15.28:6815/1642389568","state":["exists","up"]},{"osd":2,"uuid":"5ec8caa6-fb62-443f-bbd6-fbe52df5034c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":3104474806},{"type":"v1","addr":"172.21.15.28:6819","nonce":3104474806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":3104474806},{"type":"v1","addr":"172.21.15.28:6821","nonce":3104474806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":3104474806},{"type":"v1","addr":"172.21.15.28:6825","nonce":3104474806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":3104474806},{"type":"v1","addr":"172.21.15.28:6823","nonce":3104474806}]},"public_addr":"172.21.15.28:6819/3104474806","cluster_addr":"172.21.15.28:6821/3104474806","heartbeat_back_addr":"172.21.15.28:6825/3104474806","heartbeat_front_addr":"172.21.15.28:6823/3104474806","state":["exists","up"]},{"osd":3,"uuid":"d3117e1c-a230-4fc8-9cfc-37e4dc7e3d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2002097126},{"type":"v1","addr":"172.21.15.148:6801","nonce":2002097126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2002097126},{"type":"v1","addr":"172.21.15.148:6803","nonce":2002097126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2002097126},{"type":"v1","addr":"172.21.15.148:6807","nonce":2002097126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2002097126},{"type":"v1","addr":"172.21.15.148:6805","nonce":2002097126}]},"public_addr":"172.21.15.148:6801/2002097126","cluster_addr":"172.21.15.148:6803/2002097126","heartbeat_back_addr":"172.21.15.148:6807/2002097126","heartbeat_front_addr":"172.21.15.148:6805/2002097126","state":["exists","up"]},{"osd":4,"uuid":"d338fd80-0a76-441e-9cba-9cfd27cccc46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1943011533},{"type":"v1","addr":"172.21.15.148:6809","nonce":1943011533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1943011533},{"type":"v1","addr":"172.21.15.148:6811","nonce":1943011533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1943011533},{"type":"v1","addr":"172.21.15.148:6815","nonce":1943011533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1943011533},{"type":"v1","addr":"172.21.15.148:6813","nonce":1943011533}]},"public_addr":"172.21.15.148:6809/1943011533","cluster_addr":"172.21.15.148:6811/1943011533","heartbeat_back_addr":"172.21.15.148:6815/1943011533","heartbeat_front_addr":"172.21.15.148:6813/1943011533","state":["exists","up"]},{"osd":5,"uuid":"5c87b168-85e8-4d65-9763-8f1626e474ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":1578977000},{"type":"v1","addr":"172.21.15.148:6817","nonce":1578977000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1578977000},{"type":"v1","addr":"172.21.15.148:6819","nonce":1578977000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1578977000},{"type":"v1","addr":"172.21.15.148:6823","nonce":1578977000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1578977000},{"type":"v1","addr":"172.21.15.148:6821","nonce":1578977000}]},"public_addr":"172.21.15.148:6817/1578977000","cluster_addr":"172.21.15.148:6819/1578977000","heartbeat_back_addr":"172.21.15.148:6823/1578977000","heartbeat_front_addr":"172.21.15.148:6821/1578977000","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-07-03T15:38:11.596686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-07-03T15:38:38.765936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-07-03T15:39:05.464618+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-07-03T15:39:28.180164+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-07-03T15:39:52.836379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-07-03T15:40:17.711279+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.28:0/3219967940":"2024-07-04T15:37:00.336312+0000","172.21.15.28:0/1009019065":"2024-07-04T15:35:22.603924+0000","172.21.15.28:6801/1358048352":"2024-07-04T15:35:22.603924+0000","172.21.15.28:0/4294623705":"2024-07-04T15:37:00.336312+0000","172.21.15.28:0/390942453":"2024-07-04T15:35:52.741581+0000","172.21.15.28:0/423260870":"2024-07-04T15:35:22.603924+0000","172.21.15.28:6800/1358048352":"2024-07-04T15:35:22.603924+0000","172.21.15.28:6800/577887491":"2024-07-04T15:37:00.336312+0000","172.21.15.28:6800/1434683766":"2024-07-04T15:35:52.741581+0000","172.21.15.28:6801/1434683766":"2024-07-04T15:35:52.741581+0000","172.21.15.28:0/2750216161":"2024-07-04T15:35:52.741581+0000","172.21.15.28:0/3813763982":"2024-07-04T15:35:52.741581+0000","172.21.15.28:0/2748059179":"2024-07-04T15:35:22.603924+0000","172.21.15.28:6801/577887491":"2024-07-04T15:37:00.336312+0000","172.21.15.28:0/2964865891":"2024-07-04T15:37:00.336312+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-03T15:42:09.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.458+0000 7f436711d640 1 -- 172.21.15.28:0/2402049562 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f4348067190 msgr2=0x7f4348069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:09.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.458+0000 7f436711d640 1 --2- 172.21.15.28:0/2402049562 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f4348067190 0x7f4348069640 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f43500317e0 tx=0x7f4350009da0 comp rx=0 tx=0).stop 2024-07-03T15:42:09.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.458+0000 7f436711d640 1 -- 172.21.15.28:0/2402049562 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4360066240 msgr2=0x7f436010cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:09.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.458+0000 7f436711d640 1 --2- 172.21.15.28:0/2402049562 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4360066240 0x7f436010cd60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f435400c810 tx=0x7f43540043c0 comp rx=0 tx=0).stop 2024-07-03T15:42:09.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.458+0000 7f436711d640 1 -- 172.21.15.28:0/2402049562 shutdown_connections 2024-07-03T15:42:09.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.458+0000 7f436711d640 1 --2- 172.21.15.28:0/2402049562 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f4348067190 0x7f4348069640 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:09.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.458+0000 7f436711d640 1 --2- 172.21.15.28:0/2402049562 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43600ff390 0x7f436010d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:09.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.458+0000 7f436711d640 1 --2- 172.21.15.28:0/2402049562 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4360066240 0x7f436010cd60 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:09.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.458+0000 7f436711d640 1 -- 172.21.15.28:0/2402049562 >> 172.21.15.28:0/2402049562 conn(0x7f43600f6900 msgr2=0x7f43600f6d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:42:09.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.458+0000 7f436711d640 1 -- 172.21.15.28:0/2402049562 shutdown_connections 2024-07-03T15:42:09.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.458+0000 7f436711d640 1 -- 172.21.15.28:0/2402049562 wait complete. 2024-07-03T15:42:09.476 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-03T15:39:08.833339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-07-03T15:41:39.583625+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-07-03T15:41:40.084538+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-07-03T15:42:09.476 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-07-03T15:42:09.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.585+0000 7fd6dd754640 1 -- 172.21.15.28:0/1412671172 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6d80ff8e0 msgr2=0x7fd6d8101cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:09.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.585+0000 7fd6dd754640 1 --2- 172.21.15.28:0/1412671172 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6d80ff8e0 0x7fd6d8101cc0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd6c80048d0 tx=0x7fd6c8009da0 comp rx=0 tx=0).stop 2024-07-03T15:42:09.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.585+0000 7fd6dd754640 1 -- 172.21.15.28:0/1412671172 shutdown_connections 2024-07-03T15:42:09.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.585+0000 7fd6dd754640 1 --2- 172.21.15.28:0/1412671172 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6d80ff8e0 0x7fd6d8101cc0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:09.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.585+0000 7fd6dd754640 1 --2- 172.21.15.28:0/1412671172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6d80fcfc0 0x7fd6d80ff3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:09.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.585+0000 7fd6dd754640 1 -- 172.21.15.28:0/1412671172 >> 172.21.15.28:0/1412671172 conn(0x7fd6d80f6a60 msgr2=0x7fd6d80f8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:42:09.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.585+0000 7fd6dd754640 1 -- 172.21.15.28:0/1412671172 shutdown_connections 2024-07-03T15:42:09.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.585+0000 7fd6dd754640 1 -- 172.21.15.28:0/1412671172 wait complete. 2024-07-03T15:42:09.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.586+0000 7fd6dd754640 1 Processor -- start 2024-07-03T15:42:09.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.586+0000 7fd6dd754640 1 -- start start 2024-07-03T15:42:09.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.587+0000 7fd6dd754640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6d80fcfc0 0x7fd6d810cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:09.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.587+0000 7fd6dd754640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6d80ff8e0 0x7fd6d810d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:09.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.587+0000 7fd6dd754640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d810db80 con 0x7fd6d80fcfc0 2024-07-03T15:42:09.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.587+0000 7fd6dd754640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d8112530 con 0x7fd6d80ff8e0 2024-07-03T15:42:09.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.587+0000 7fd6d6ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6d80fcfc0 0x7fd6d810cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:09.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.587+0000 7fd6d6ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6d80fcfc0 0x7fd6d810cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46730/0 (socket says 172.21.15.28:46730) 2024-07-03T15:42:09.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.587+0000 7fd6d6ffd640 1 -- 172.21.15.28:0/1734907452 learned_addr learned my addr 172.21.15.28:0/1734907452 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:42:09.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.587+0000 7fd6d67fc640 1 --2- 172.21.15.28:0/1734907452 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6d80ff8e0 0x7fd6d810d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:09.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.587+0000 7fd6d6ffd640 1 -- 172.21.15.28:0/1734907452 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6d80ff8e0 msgr2=0x7fd6d810d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:09.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.587+0000 7fd6d6ffd640 1 --2- 172.21.15.28:0/1734907452 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6d80ff8e0 0x7fd6d810d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:09.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.587+0000 7fd6d6ffd640 1 -- 172.21.15.28:0/1734907452 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c8005040 con 0x7fd6d80fcfc0 2024-07-03T15:42:09.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.587+0000 7fd6d6ffd640 1 --2- 172.21.15.28:0/1734907452 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6d80fcfc0 0x7fd6d810cff0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd6c000c790 tx=0x7fd6c00043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:42:09.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.587+0000 7fd6cf7fe640 1 -- 172.21.15.28:0/1734907452 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6c000a820 con 0x7fd6d80fcfc0 2024-07-03T15:42:09.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.587+0000 7fd6dd754640 1 -- 172.21.15.28:0/1734907452 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6d81126d0 con 0x7fd6d80fcfc0 2024-07-03T15:42:09.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.587+0000 7fd6cf7fe640 1 -- 172.21.15.28:0/1734907452 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd6c000ae50 con 0x7fd6d80fcfc0 2024-07-03T15:42:09.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.587+0000 7fd6cf7fe640 1 -- 172.21.15.28:0/1734907452 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6c00115e0 con 0x7fd6d80fcfc0 2024-07-03T15:42:09.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.588+0000 7fd6dd754640 1 -- 172.21.15.28:0/1734907452 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6d8112bc0 con 0x7fd6d80fcfc0 2024-07-03T15:42:09.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.588+0000 7fd6dd754640 1 -- 172.21.15.28:0/1734907452 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6d804bd70 con 0x7fd6d80fcfc0 2024-07-03T15:42:09.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.589+0000 7fd6cf7fe640 1 -- 172.21.15.28:0/1734907452 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7fd6c0012400 con 0x7fd6d80fcfc0 2024-07-03T15:42:09.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.590+0000 7fd6cf7fe640 1 --2- 172.21.15.28:0/1734907452 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fd6c4067190 0x7fd6c4069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:09.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.590+0000 7fd6cf7fe640 1 -- 172.21.15.28:0/1734907452 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fd6c0084c10 con 0x7fd6d80fcfc0 2024-07-03T15:42:09.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.590+0000 7fd6d67fc640 1 --2- 172.21.15.28:0/1734907452 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fd6c4067190 0x7fd6c4069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:09.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.590+0000 7fd6d67fc640 1 --2- 172.21.15.28:0/1734907452 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fd6c4067190 0x7fd6c4069640 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8000c00 tx=0x7fd6c8009da0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:42:09.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.594+0000 7fd6cf7fe640 1 -- 172.21.15.28:0/1734907452 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd6c0051f30 con 0x7fd6d80fcfc0 2024-07-03T15:42:09.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:09 smithi148 ceph-mon[36056]: pgmap v187: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-07-03T15:42:09.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:09 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/2402049562' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-03T15:42:09.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.789+0000 7fd6dd754640 1 -- 172.21.15.28:0/1734907452 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd6d8112f90 con 0x7fd6d80fcfc0 2024-07-03T15:42:09.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.789+0000 7fd6cf7fe640 1 -- 172.21.15.28:0/1734907452 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd6c0055550 con 0x7fd6d80fcfc0 2024-07-03T15:42:09.792 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2024-07-03T15:42:09.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.791+0000 7fd6dd754640 1 -- 172.21.15.28:0/1734907452 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fd6c4067190 msgr2=0x7fd6c4069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:09.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.791+0000 7fd6dd754640 1 --2- 172.21.15.28:0/1734907452 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fd6c4067190 0x7fd6c4069640 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8000c00 tx=0x7fd6c8009da0 comp rx=0 tx=0).stop 2024-07-03T15:42:09.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.791+0000 7fd6dd754640 1 -- 172.21.15.28:0/1734907452 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6d80fcfc0 msgr2=0x7fd6d810cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:09.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.791+0000 7fd6dd754640 1 --2- 172.21.15.28:0/1734907452 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6d80fcfc0 0x7fd6d810cff0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd6c000c790 tx=0x7fd6c00043c0 comp rx=0 tx=0).stop 2024-07-03T15:42:09.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.791+0000 7fd6dd754640 1 -- 172.21.15.28:0/1734907452 shutdown_connections 2024-07-03T15:42:09.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.791+0000 7fd6dd754640 1 --2- 172.21.15.28:0/1734907452 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fd6c4067190 0x7fd6c4069640 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:09.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.791+0000 7fd6dd754640 1 --2- 172.21.15.28:0/1734907452 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6d80ff8e0 0x7fd6d810d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:09.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.791+0000 7fd6dd754640 1 --2- 172.21.15.28:0/1734907452 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6d80fcfc0 0x7fd6d810cff0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:09.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.791+0000 7fd6dd754640 1 -- 172.21.15.28:0/1734907452 >> 172.21.15.28:0/1734907452 conn(0x7fd6d80f6a60 msgr2=0x7fd6d80f6ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:42:09.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.791+0000 7fd6dd754640 1 -- 172.21.15.28:0/1734907452 shutdown_connections 2024-07-03T15:42:09.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.791+0000 7fd6dd754640 1 -- 172.21.15.28:0/1734907452 wait complete. 2024-07-03T15:42:09.804 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:09 smithi028 ceph-mon[29148]: pgmap v187: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-07-03T15:42:09.804 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:09 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/2402049562' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-03T15:42:09.805 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-07-03T15:42:09.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.904+0000 7f1d6899d640 1 -- 172.21.15.28:0/2472963240 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1d640fb0d0 msgr2=0x7f1d640fb4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:09.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.904+0000 7f1d6899d640 1 --2- 172.21.15.28:0/2472963240 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1d640fb0d0 0x7f1d640fb4e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1d540048d0 tx=0x7f1d54009da0 comp rx=0 tx=0).stop 2024-07-03T15:42:09.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.904+0000 7f1d6899d640 1 -- 172.21.15.28:0/2472963240 shutdown_connections 2024-07-03T15:42:09.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.904+0000 7f1d6899d640 1 --2- 172.21.15.28:0/2472963240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d640fc300 0x7f1d640fc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:09.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.904+0000 7f1d6899d640 1 --2- 172.21.15.28:0/2472963240 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1d640fb0d0 0x7f1d640fb4e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:09.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.904+0000 7f1d6899d640 1 -- 172.21.15.28:0/2472963240 >> 172.21.15.28:0/2472963240 conn(0x7f1d640f6830 msgr2=0x7f1d640f8c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:42:09.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.904+0000 7f1d6899d640 1 -- 172.21.15.28:0/2472963240 shutdown_connections 2024-07-03T15:42:09.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.904+0000 7f1d6899d640 1 -- 172.21.15.28:0/2472963240 wait complete. 2024-07-03T15:42:09.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.905+0000 7f1d6899d640 1 Processor -- start 2024-07-03T15:42:09.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.905+0000 7f1d6899d640 1 -- start start 2024-07-03T15:42:09.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.905+0000 7f1d6899d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d640fb0d0 0x7f1d6410ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:09.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.905+0000 7f1d6899d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1d640fc300 0x7f1d6410f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:09.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.905+0000 7f1d6899d640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6410fad0 con 0x7f1d640fc300 2024-07-03T15:42:09.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.905+0000 7f1d6899d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d641144d0 con 0x7f1d640fb0d0 2024-07-03T15:42:09.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.905+0000 7f1d61d9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1d640fc300 0x7f1d6410f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:09.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.905+0000 7f1d61d9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1d640fc300 0x7f1d6410f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:40868/0 (socket says 172.21.15.28:40868) 2024-07-03T15:42:09.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.905+0000 7f1d61d9b640 1 -- 172.21.15.28:0/387878631 learned_addr learned my addr 172.21.15.28:0/387878631 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:42:09.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.905+0000 7f1d6259c640 1 --2- 172.21.15.28:0/387878631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d640fb0d0 0x7f1d6410ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:09.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.905+0000 7f1d61d9b640 1 -- 172.21.15.28:0/387878631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d640fb0d0 msgr2=0x7f1d6410ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:09.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.905+0000 7f1d61d9b640 1 --2- 172.21.15.28:0/387878631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d640fb0d0 0x7f1d6410ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:09.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.905+0000 7f1d61d9b640 1 -- 172.21.15.28:0/387878631 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d54005040 con 0x7f1d640fc300 2024-07-03T15:42:09.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.905+0000 7f1d61d9b640 1 --2- 172.21.15.28:0/387878631 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1d640fc300 0x7f1d6410f480 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800c830 tx=0x7f1d58004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:42:09.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.906+0000 7f1d52ffd640 1 -- 172.21.15.28:0/387878631 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d5800a7d0 con 0x7f1d640fc300 2024-07-03T15:42:09.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.906+0000 7f1d52ffd640 1 -- 172.21.15.28:0/387878631 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1d5800ae00 con 0x7f1d640fc300 2024-07-03T15:42:09.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.906+0000 7f1d6899d640 1 -- 172.21.15.28:0/387878631 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d64114670 con 0x7f1d640fc300 2024-07-03T15:42:09.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.906+0000 7f1d52ffd640 1 -- 172.21.15.28:0/387878631 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d580113a0 con 0x7f1d640fc300 2024-07-03T15:42:09.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.906+0000 7f1d6899d640 1 -- 172.21.15.28:0/387878631 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d64114b60 con 0x7f1d640fc300 2024-07-03T15:42:09.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.907+0000 7f1d6899d640 1 -- 172.21.15.28:0/387878631 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d6404bd70 con 0x7f1d640fc300 2024-07-03T15:42:09.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.907+0000 7f1d52ffd640 1 -- 172.21.15.28:0/387878631 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f1d58012400 con 0x7f1d640fc300 2024-07-03T15:42:09.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.908+0000 7f1d52ffd640 1 --2- 172.21.15.28:0/387878631 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f1d4c0671a0 0x7f1d4c069650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:09.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.908+0000 7f1d52ffd640 1 -- 172.21.15.28:0/387878631 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f1d58084c10 con 0x7f1d640fc300 2024-07-03T15:42:09.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.908+0000 7f1d6259c640 1 --2- 172.21.15.28:0/387878631 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f1d4c0671a0 0x7f1d4c069650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:09.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.908+0000 7f1d6259c640 1 --2- 172.21.15.28:0/387878631 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f1d4c0671a0 0x7f1d4c069650 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1d5402bda0 tx=0x7f1d54009da0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:42:09.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:09.912+0000 7f1d52ffd640 1 -- 172.21.15.28:0/387878631 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f1d58051f30 con 0x7f1d640fc300 2024-07-03T15:42:10.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.085+0000 7f1d6899d640 1 -- 172.21.15.28:0/387878631 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f1d64103c20 con 0x7f1d640fc300 2024-07-03T15:42:10.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.086+0000 7f1d52ffd640 1 -- 172.21.15.28:0/387878631 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v45) v1 ==== 107+0+11 (secure 0 0 0) 0x7f1d58055550 con 0x7f1d640fc300 2024-07-03T15:42:10.089 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2024-07-03T15:42:10.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.087+0000 7f1d6899d640 1 -- 172.21.15.28:0/387878631 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f1d4c0671a0 msgr2=0x7f1d4c069650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:10.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.087+0000 7f1d6899d640 1 --2- 172.21.15.28:0/387878631 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f1d4c0671a0 0x7f1d4c069650 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1d5402bda0 tx=0x7f1d54009da0 comp rx=0 tx=0).stop 2024-07-03T15:42:10.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.087+0000 7f1d6899d640 1 -- 172.21.15.28:0/387878631 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1d640fc300 msgr2=0x7f1d6410f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:10.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.087+0000 7f1d6899d640 1 --2- 172.21.15.28:0/387878631 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1d640fc300 0x7f1d6410f480 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800c830 tx=0x7f1d58004430 comp rx=0 tx=0).stop 2024-07-03T15:42:10.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.087+0000 7f1d6899d640 1 -- 172.21.15.28:0/387878631 shutdown_connections 2024-07-03T15:42:10.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.087+0000 7f1d6899d640 1 --2- 172.21.15.28:0/387878631 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f1d4c0671a0 0x7f1d4c069650 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:10.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.087+0000 7f1d6899d640 1 --2- 172.21.15.28:0/387878631 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1d640fc300 0x7f1d6410f480 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:10.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.087+0000 7f1d6899d640 1 --2- 172.21.15.28:0/387878631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d640fb0d0 0x7f1d6410ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:10.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.087+0000 7f1d6899d640 1 -- 172.21.15.28:0/387878631 >> 172.21.15.28:0/387878631 conn(0x7f1d640f6830 msgr2=0x7f1d640f8320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:42:10.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.088+0000 7f1d6899d640 1 -- 172.21.15.28:0/387878631 shutdown_connections 2024-07-03T15:42:10.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.088+0000 7f1d6899d640 1 -- 172.21.15.28:0/387878631 wait complete. 2024-07-03T15:42:10.102 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-07-03T15:42:10.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.200+0000 7f76a335d640 1 -- 172.21.15.28:0/2401078448 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f769c0fc3e0 msgr2=0x7f769c0fc850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:10.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.200+0000 7f76a335d640 1 --2- 172.21.15.28:0/2401078448 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f769c0fc3e0 0x7f769c0fc850 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f768c0048d0 tx=0x7f768c009da0 comp rx=0 tx=0).stop 2024-07-03T15:42:10.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.200+0000 7f76a335d640 1 -- 172.21.15.28:0/2401078448 shutdown_connections 2024-07-03T15:42:10.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.200+0000 7f76a335d640 1 --2- 172.21.15.28:0/2401078448 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f769c0fc3e0 0x7f769c0fc850 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:10.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.200+0000 7f76a335d640 1 --2- 172.21.15.28:0/2401078448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f769c0fb110 0x7f769c0fb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:10.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.200+0000 7f76a335d640 1 -- 172.21.15.28:0/2401078448 >> 172.21.15.28:0/2401078448 conn(0x7f769c0f68d0 msgr2=0x7f769c0f8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:42:10.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.200+0000 7f76a335d640 1 -- 172.21.15.28:0/2401078448 shutdown_connections 2024-07-03T15:42:10.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.200+0000 7f76a335d640 1 -- 172.21.15.28:0/2401078448 wait complete. 2024-07-03T15:42:10.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.201+0000 7f76a335d640 1 Processor -- start 2024-07-03T15:42:10.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.201+0000 7f76a335d640 1 -- start start 2024-07-03T15:42:10.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.201+0000 7f76a335d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f769c0fb110 0x7f769c10cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:10.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.201+0000 7f76a335d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f769c0fc3e0 0x7f769c10d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:10.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.201+0000 7f76a335d640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769c10d900 con 0x7f769c0fc3e0 2024-07-03T15:42:10.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.201+0000 7f76a335d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769c112300 con 0x7f769c0fb110 2024-07-03T15:42:10.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.201+0000 7f76a08f8640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f769c0fc3e0 0x7f769c10d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:10.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.201+0000 7f76a08f8640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f769c0fc3e0 0x7f769c10d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:40892/0 (socket says 172.21.15.28:40892) 2024-07-03T15:42:10.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.201+0000 7f76a08f8640 1 -- 172.21.15.28:0/3774678542 learned_addr learned my addr 172.21.15.28:0/3774678542 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:42:10.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.201+0000 7f76a08f8640 1 -- 172.21.15.28:0/3774678542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f769c0fb110 msgr2=0x7f769c10cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:10.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.201+0000 7f76a10f9640 1 --2- 172.21.15.28:0/3774678542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f769c0fb110 0x7f769c10cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:10.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.201+0000 7f76a08f8640 1 --2- 172.21.15.28:0/3774678542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f769c0fb110 0x7f769c10cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:10.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.201+0000 7f76a08f8640 1 -- 172.21.15.28:0/3774678542 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f768c005040 con 0x7f769c0fc3e0 2024-07-03T15:42:10.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.201+0000 7f76a08f8640 1 --2- 172.21.15.28:0/3774678542 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f769c0fc3e0 0x7f769c10d2b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f768c007d20 tx=0x7f768c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:42:10.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.202+0000 7f7691ffb640 1 -- 172.21.15.28:0/3774678542 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f768c009590 con 0x7f769c0fc3e0 2024-07-03T15:42:10.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.202+0000 7f7691ffb640 1 -- 172.21.15.28:0/3774678542 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f768c0096f0 con 0x7f769c0fc3e0 2024-07-03T15:42:10.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.202+0000 7f76a335d640 1 -- 172.21.15.28:0/3774678542 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f769c112440 con 0x7f769c0fc3e0 2024-07-03T15:42:10.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.202+0000 7f7691ffb640 1 -- 172.21.15.28:0/3774678542 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f768c03b6d0 con 0x7f769c0fc3e0 2024-07-03T15:42:10.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.202+0000 7f76a335d640 1 -- 172.21.15.28:0/3774678542 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f769c112930 con 0x7f769c0fc3e0 2024-07-03T15:42:10.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.203+0000 7f76a335d640 1 -- 172.21.15.28:0/3774678542 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f769c04bd70 con 0x7f769c0fc3e0 2024-07-03T15:42:10.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.203+0000 7f7691ffb640 1 -- 172.21.15.28:0/3774678542 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f768c02f070 con 0x7f769c0fc3e0 2024-07-03T15:42:10.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.204+0000 7f7691ffb640 1 --2- 172.21.15.28:0/3774678542 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f7688067190 0x7f7688069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:10.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.204+0000 7f76a10f9640 1 --2- 172.21.15.28:0/3774678542 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f7688067190 0x7f7688069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:10.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.204+0000 7f7691ffb640 1 -- 172.21.15.28:0/3774678542 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f768c0a71f0 con 0x7f769c0fc3e0 2024-07-03T15:42:10.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.204+0000 7f76a10f9640 1 --2- 172.21.15.28:0/3774678542 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f7688067190 0x7f7688069640 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7698007fd0 tx=0x7f7698009040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:42:10.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.208+0000 7f7691ffb640 1 -- 172.21.15.28:0/3774678542 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f768c074470 con 0x7f769c0fc3e0 2024-07-03T15:42:10.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.381+0000 7f76a335d640 1 -- 172.21.15.28:0/3774678542 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f769c103d30 con 0x7f769c0fc3e0 2024-07-03T15:42:10.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.381+0000 7f7691ffb640 1 -- 172.21.15.28:0/3774678542 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v45) v1 ==== 107+0+11 (secure 0 0 0) 0x7f768c077a90 con 0x7f769c0fc3e0 2024-07-03T15:42:10.385 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2024-07-03T15:42:10.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.383+0000 7f76a335d640 1 -- 172.21.15.28:0/3774678542 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f7688067190 msgr2=0x7f7688069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:10.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.383+0000 7f76a335d640 1 --2- 172.21.15.28:0/3774678542 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f7688067190 0x7f7688069640 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7698007fd0 tx=0x7f7698009040 comp rx=0 tx=0).stop 2024-07-03T15:42:10.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.383+0000 7f76a335d640 1 -- 172.21.15.28:0/3774678542 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f769c0fc3e0 msgr2=0x7f769c10d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:10.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.383+0000 7f76a335d640 1 --2- 172.21.15.28:0/3774678542 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f769c0fc3e0 0x7f769c10d2b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f768c007d20 tx=0x7f768c009da0 comp rx=0 tx=0).stop 2024-07-03T15:42:10.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.383+0000 7f76a335d640 1 -- 172.21.15.28:0/3774678542 shutdown_connections 2024-07-03T15:42:10.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.383+0000 7f76a335d640 1 --2- 172.21.15.28:0/3774678542 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f7688067190 0x7f7688069640 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:10.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.383+0000 7f76a335d640 1 --2- 172.21.15.28:0/3774678542 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f769c0fc3e0 0x7f769c10d2b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:10.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.383+0000 7f76a335d640 1 --2- 172.21.15.28:0/3774678542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f769c0fb110 0x7f769c10cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:10.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.383+0000 7f76a335d640 1 -- 172.21.15.28:0/3774678542 >> 172.21.15.28:0/3774678542 conn(0x7f769c0f68d0 msgr2=0x7f769c0f7f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:42:10.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.383+0000 7f76a335d640 1 -- 172.21.15.28:0/3774678542 shutdown_connections 2024-07-03T15:42:10.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.383+0000 7f76a335d640 1 -- 172.21.15.28:0/3774678542 wait complete. 2024-07-03T15:42:10.397 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-03T15:42:10.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.493+0000 7f59240ae640 1 -- 172.21.15.28:0/3718382576 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f591c0ff360 msgr2=0x7f591c101740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:10.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.493+0000 7f59240ae640 1 --2- 172.21.15.28:0/3718382576 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f591c0ff360 0x7f591c101740 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f59100048d0 tx=0x7f5910009da0 comp rx=0 tx=0).stop 2024-07-03T15:42:10.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.493+0000 7f59240ae640 1 -- 172.21.15.28:0/3718382576 shutdown_connections 2024-07-03T15:42:10.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.493+0000 7f59240ae640 1 --2- 172.21.15.28:0/3718382576 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f591c0ff360 0x7f591c101740 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:10.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.493+0000 7f59240ae640 1 --2- 172.21.15.28:0/3718382576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f591c0650c0 0x7f591c0fee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:10.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.493+0000 7f59240ae640 1 -- 172.21.15.28:0/3718382576 >> 172.21.15.28:0/3718382576 conn(0x7f591c0f68d0 msgr2=0x7f591c0f8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:42:10.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.494+0000 7f59240ae640 1 -- 172.21.15.28:0/3718382576 shutdown_connections 2024-07-03T15:42:10.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.494+0000 7f59240ae640 1 -- 172.21.15.28:0/3718382576 wait complete. 2024-07-03T15:42:10.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.494+0000 7f59240ae640 1 Processor -- start 2024-07-03T15:42:10.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.494+0000 7f59240ae640 1 -- start start 2024-07-03T15:42:10.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.494+0000 7f59240ae640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f591c0650c0 0x7f591c10cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:10.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.494+0000 7f59240ae640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f591c0ff360 0x7f591c10d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:10.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.494+0000 7f59240ae640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591c10d940 con 0x7f591c0650c0 2024-07-03T15:42:10.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.494+0000 7f59240ae640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591c1122f0 con 0x7f591c0ff360 2024-07-03T15:42:10.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.494+0000 7f5921e4a640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f591c0650c0 0x7f591c10cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:10.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.494+0000 7f5921e4a640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f591c0650c0 0x7f591c10cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:40906/0 (socket says 172.21.15.28:40906) 2024-07-03T15:42:10.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.494+0000 7f5921e4a640 1 -- 172.21.15.28:0/2273677623 learned_addr learned my addr 172.21.15.28:0/2273677623 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:42:10.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.494+0000 7f5921649640 1 --2- 172.21.15.28:0/2273677623 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f591c0ff360 0x7f591c10d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:10.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.495+0000 7f5921e4a640 1 -- 172.21.15.28:0/2273677623 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f591c0ff360 msgr2=0x7f591c10d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:10.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.495+0000 7f5921e4a640 1 --2- 172.21.15.28:0/2273677623 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f591c0ff360 0x7f591c10d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:10.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.495+0000 7f5921e4a640 1 -- 172.21.15.28:0/2273677623 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5910005040 con 0x7f591c0650c0 2024-07-03T15:42:10.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.495+0000 7f5921649640 1 --2- 172.21.15.28:0/2273677623 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f591c0ff360 0x7f591c10d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-03T15:42:10.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.495+0000 7f5921e4a640 1 --2- 172.21.15.28:0/2273677623 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f591c0650c0 0x7f591c10cdb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5914002840 tx=0x7f5914004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:42:10.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.495+0000 7f591a7fc640 1 -- 172.21.15.28:0/2273677623 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5914007c30 con 0x7f591c0650c0 2024-07-03T15:42:10.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.495+0000 7f59240ae640 1 -- 172.21.15.28:0/2273677623 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f591c112490 con 0x7f591c0650c0 2024-07-03T15:42:10.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.495+0000 7f591a7fc640 1 -- 172.21.15.28:0/2273677623 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f591400f040 con 0x7f591c0650c0 2024-07-03T15:42:10.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.495+0000 7f591a7fc640 1 -- 172.21.15.28:0/2273677623 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5914009590 con 0x7f591c0650c0 2024-07-03T15:42:10.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.495+0000 7f59240ae640 1 -- 172.21.15.28:0/2273677623 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f591c112980 con 0x7f591c0650c0 2024-07-03T15:42:10.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.496+0000 7f59240ae640 1 -- 172.21.15.28:0/2273677623 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f591c04bd70 con 0x7f591c0650c0 2024-07-03T15:42:10.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.496+0000 7f591a7fc640 1 -- 172.21.15.28:0/2273677623 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f591400d6b0 con 0x7f591c0650c0 2024-07-03T15:42:10.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.497+0000 7f591a7fc640 1 --2- 172.21.15.28:0/2273677623 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f5904067190 0x7f5904069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:10.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.497+0000 7f5921649640 1 --2- 172.21.15.28:0/2273677623 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f5904067190 0x7f5904069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:10.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.497+0000 7f591a7fc640 1 -- 172.21.15.28:0/2273677623 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f59140849e0 con 0x7f591c0650c0 2024-07-03T15:42:10.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.498+0000 7f5921649640 1 --2- 172.21.15.28:0/2273677623 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f5904067190 0x7f5904069640 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f59100048a0 tx=0x7f5910009da0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:42:10.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.501+0000 7f591a7fc640 1 -- 172.21.15.28:0/2273677623 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5914051c80 con 0x7f591c0650c0 2024-07-03T15:42:10.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.713+0000 7f59240ae640 1 -- 172.21.15.28:0/2273677623 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f591c112b70 con 0x7f591c0650c0 2024-07-03T15:42:10.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.714+0000 7f591a7fc640 1 -- 172.21.15.28:0/2273677623 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4700 (secure 0 0 0) 0x7f59140552a0 con 0x7f591c0650c0 2024-07-03T15:42:10.718 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:42:10.718 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24277,"name":"cephfs.smithi148.szomfl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.148:6827/4171231858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":4171231858},{"type":"v1","addr":"172.21.15.148:6827","nonce":4171231858}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":24281,"name":"cephfs.smithi148.xgpuld","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6825/2347536398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2347536398},{"type":"v1","addr":"172.21.15.148:6825","nonce":2347536398}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":16,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-03T15:41:40.656298+0000","modified":"2024-07-03T15:41:56.454984+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14450},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14450":{"gid":14450,"name":"cephfs.smithi028.rhpvve","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.28:6827/1613936042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1613936042},{"type":"v1","addr":"172.21.15.28:6827","nonce":1613936042}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14460":{"gid":14460,"name":"cephfs.smithi028.wmbxew","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.28:6829/1730336821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1730336821},{"type":"v1","addr":"172.21.15.28:6829","nonce":1730336821}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-07-03T15:42:10.718 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 17 2024-07-03T15:42:10.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.716+0000 7f59240ae640 1 -- 172.21.15.28:0/2273677623 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f5904067190 msgr2=0x7f5904069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:10.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.716+0000 7f59240ae640 1 --2- 172.21.15.28:0/2273677623 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f5904067190 0x7f5904069640 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f59100048a0 tx=0x7f5910009da0 comp rx=0 tx=0).stop 2024-07-03T15:42:10.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.716+0000 7f59240ae640 1 -- 172.21.15.28:0/2273677623 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f591c0650c0 msgr2=0x7f591c10cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:10.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.716+0000 7f59240ae640 1 --2- 172.21.15.28:0/2273677623 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f591c0650c0 0x7f591c10cdb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5914002840 tx=0x7f5914004570 comp rx=0 tx=0).stop 2024-07-03T15:42:10.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.716+0000 7f59240ae640 1 -- 172.21.15.28:0/2273677623 shutdown_connections 2024-07-03T15:42:10.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.716+0000 7f59240ae640 1 --2- 172.21.15.28:0/2273677623 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f5904067190 0x7f5904069640 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:10.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.716+0000 7f59240ae640 1 --2- 172.21.15.28:0/2273677623 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f591c0ff360 0x7f591c10d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:10.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.716+0000 7f59240ae640 1 --2- 172.21.15.28:0/2273677623 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f591c0650c0 0x7f591c10cdb0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:10.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.716+0000 7f59240ae640 1 -- 172.21.15.28:0/2273677623 >> 172.21.15.28:0/2273677623 conn(0x7f591c0f68d0 msgr2=0x7f591c0f6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:42:10.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.716+0000 7f59240ae640 1 -- 172.21.15.28:0/2273677623 shutdown_connections 2024-07-03T15:42:10.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:10.716+0000 7f59240ae640 1 -- 172.21.15.28:0/2273677623 wait complete. 2024-07-03T15:42:10.730 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 16, 'max_mds': 1, 'flags': 50} 2024-07-03T15:42:10.730 INFO:teuthology.run_tasks:Running task kclient... 2024-07-03T15:42:10.758 INFO:tasks.kclient:Mounting kernel clients... 2024-07-03T15:42:10.758 INFO:tasks.kclient:config is {'client.0': {}, 'client.1': {}} 2024-07-03T15:42:10.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-03T15:42:10.759 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2024-07-03T15:42:10.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-03T15:42:10.775 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link delete ceph-brx 2024-07-03T15:42:10.843 INFO:teuthology.orchestra.run.smithi028.stderr:Cannot find device "ceph-brx" 2024-07-03T15:42:10.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-03T15:42:10.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-03T15:42:10.845 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-07-03T15:42:10.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-03T15:42:10.865 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link delete ceph-brx 2024-07-03T15:42:10.932 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:10 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:42:10.932 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:10 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:42:10.932 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:10 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:42:10.932 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:10 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:42:10.932 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:10 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:42:10.933 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:10 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/1734907452' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-03T15:42:10.933 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:10 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/387878631' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-07-03T15:42:10.933 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:10 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/3774678542' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-07-03T15:42:10.940 INFO:teuthology.orchestra.run.smithi148.stderr:Cannot find device "ceph-brx" 2024-07-03T15:42:10.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-03T15:42:10.941 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-07-03T15:42:10.941 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-07-03T15:42:10.941 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-03T15:42:10.942 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph fs ls 2024-07-03T15:42:10.965 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:10 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:42:10.965 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:10 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:42:10.965 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:10 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:42:10.965 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:10 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:42:10.965 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:10 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:42:10.965 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:10 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/1734907452' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-03T15:42:10.965 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:10 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/387878631' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-07-03T15:42:10.965 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:10 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/3774678542' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-07-03T15:42:11.205 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:42:11.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:11 smithi028 ceph-mon[29148]: pgmap v188: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-07-03T15:42:11.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:11 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/2273677623' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-03T15:42:11.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:11 smithi148 ceph-mon[36056]: pgmap v188: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-07-03T15:42:11.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:11 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/2273677623' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-03T15:42:12.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:12.867+0000 7f3c44613640 1 -- 172.21.15.28:0/2512576010 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c3c0650c0 msgr2=0x7f3c3c065490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:12.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:12.867+0000 7f3c44613640 1 --2- 172.21.15.28:0/2512576010 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c3c0650c0 0x7f3c3c065490 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3c38009c50 tx=0x7f3c38004d10 comp rx=0 tx=0).stop 2024-07-03T15:42:12.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:12.867+0000 7f3c44613640 1 -- 172.21.15.28:0/2512576010 shutdown_connections 2024-07-03T15:42:12.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:12.867+0000 7f3c44613640 1 --2- 172.21.15.28:0/2512576010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c3c0fdf50 0x7f3c3c0fe3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:12.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:12.867+0000 7f3c44613640 1 --2- 172.21.15.28:0/2512576010 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c3c0650c0 0x7f3c3c065490 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:12.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:12.867+0000 7f3c44613640 1 -- 172.21.15.28:0/2512576010 >> 172.21.15.28:0/2512576010 conn(0x7f3c3c0721c0 msgr2=0x7f3c3c0725c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:42:12.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:12.867+0000 7f3c44613640 1 -- 172.21.15.28:0/2512576010 shutdown_connections 2024-07-03T15:42:12.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:12.867+0000 7f3c44613640 1 -- 172.21.15.28:0/2512576010 wait complete. 2024-07-03T15:42:12.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:12.868+0000 7f3c44613640 1 Processor -- start 2024-07-03T15:42:12.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:12.868+0000 7f3c44613640 1 -- start start 2024-07-03T15:42:12.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:12.868+0000 7f3c44613640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c3c0650c0 0x7f3c3c1130a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:12.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:12.868+0000 7f3c44613640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c3c0fdf50 0x7f3c3c1135e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:12.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:12.868+0000 7f3c44613640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3c113be0 con 0x7f3c3c0650c0 2024-07-03T15:42:12.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:12.868+0000 7f3c44613640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3c113d50 con 0x7f3c3c0fdf50 2024-07-03T15:42:12.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:12.869+0000 7f3c423af640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c3c0650c0 0x7f3c3c1130a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:12.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:12.869+0000 7f3c423af640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c3c0650c0 0x7f3c3c1130a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:40922/0 (socket says 172.21.15.28:40922) 2024-07-03T15:42:12.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:12.869+0000 7f3c423af640 1 -- 172.21.15.28:0/380211002 learned_addr learned my addr 172.21.15.28:0/380211002 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:42:12.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:12.869+0000 7f3c41bae640 1 --2- 172.21.15.28:0/380211002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c3c0fdf50 0x7f3c3c1135e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:12.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:12.869+0000 7f3c423af640 1 -- 172.21.15.28:0/380211002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c3c0fdf50 msgr2=0x7f3c3c1135e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:12.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:12.869+0000 7f3c423af640 1 --2- 172.21.15.28:0/380211002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c3c0fdf50 0x7f3c3c1135e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:12.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:12.869+0000 7f3c423af640 1 -- 172.21.15.28:0/380211002 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c380098b0 con 0x7f3c3c0650c0 2024-07-03T15:42:12.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:12.869+0000 7f3c423af640 1 --2- 172.21.15.28:0/380211002 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c3c0650c0 0x7f3c3c1130a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3c38004f80 tx=0x7f3c38004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:42:12.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:12.869+0000 7f3c32ffd640 1 -- 172.21.15.28:0/380211002 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c38008de0 con 0x7f3c3c0650c0 2024-07-03T15:42:12.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:12.869+0000 7f3c44613640 1 -- 172.21.15.28:0/380211002 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c3c10d170 con 0x7f3c3c0650c0 2024-07-03T15:42:12.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:12.869+0000 7f3c32ffd640 1 -- 172.21.15.28:0/380211002 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3c38032c00 con 0x7f3c3c0650c0 2024-07-03T15:42:12.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:12.869+0000 7f3c32ffd640 1 -- 172.21.15.28:0/380211002 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c3803dbc0 con 0x7f3c3c0650c0 2024-07-03T15:42:12.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:12.869+0000 7f3c44613640 1 -- 172.21.15.28:0/380211002 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c3c10d690 con 0x7f3c3c0650c0 2024-07-03T15:42:12.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:12.870+0000 7f3c44613640 1 -- 172.21.15.28:0/380211002 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c3c0fdaf0 con 0x7f3c3c0650c0 2024-07-03T15:42:12.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:12.871+0000 7f3c32ffd640 1 -- 172.21.15.28:0/380211002 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f3c3802f030 con 0x7f3c3c0650c0 2024-07-03T15:42:12.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:12.871+0000 7f3c32ffd640 1 --2- 172.21.15.28:0/380211002 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3c28067190 0x7f3c28069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:12.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:12.871+0000 7f3c32ffd640 1 -- 172.21.15.28:0/380211002 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f3c380a6360 con 0x7f3c3c0650c0 2024-07-03T15:42:12.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:12.871+0000 7f3c41bae640 1 --2- 172.21.15.28:0/380211002 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3c28067190 0x7f3c28069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:12.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:12.872+0000 7f3c41bae640 1 --2- 172.21.15.28:0/380211002 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3c28067190 0x7f3c28069640 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c007fd0 tx=0x7f3c2c009040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:42:12.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:12.875+0000 7f3c32ffd640 1 -- 172.21.15.28:0/380211002 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3c380736b0 con 0x7f3c3c0650c0 2024-07-03T15:42:13.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:13.086+0000 7f3c44613640 1 -- 172.21.15.28:0/380211002 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3c3c072cd0 con 0x7f3c3c0650c0 2024-07-03T15:42:13.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:13.086+0000 7f3c32ffd640 1 -- 172.21.15.28:0/380211002 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+83 (secure 0 0 0) 0x7f3c38032d70 con 0x7f3c3c0650c0 2024-07-03T15:42:13.090 INFO:teuthology.orchestra.run.smithi028.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-07-03T15:42:13.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:13.088+0000 7f3c44613640 1 -- 172.21.15.28:0/380211002 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3c28067190 msgr2=0x7f3c28069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:13.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:13.088+0000 7f3c44613640 1 --2- 172.21.15.28:0/380211002 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3c28067190 0x7f3c28069640 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c007fd0 tx=0x7f3c2c009040 comp rx=0 tx=0).stop 2024-07-03T15:42:13.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:13.088+0000 7f3c44613640 1 -- 172.21.15.28:0/380211002 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c3c0650c0 msgr2=0x7f3c3c1130a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:13.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:13.088+0000 7f3c44613640 1 --2- 172.21.15.28:0/380211002 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c3c0650c0 0x7f3c3c1130a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3c38004f80 tx=0x7f3c38004d10 comp rx=0 tx=0).stop 2024-07-03T15:42:13.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:13.088+0000 7f3c44613640 1 -- 172.21.15.28:0/380211002 shutdown_connections 2024-07-03T15:42:13.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:13.089+0000 7f3c44613640 1 --2- 172.21.15.28:0/380211002 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3c28067190 0x7f3c28069640 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:13.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:13.089+0000 7f3c44613640 1 --2- 172.21.15.28:0/380211002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c3c0fdf50 0x7f3c3c1135e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:13.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:13.089+0000 7f3c44613640 1 --2- 172.21.15.28:0/380211002 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c3c0650c0 0x7f3c3c1130a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:13.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:13.089+0000 7f3c44613640 1 -- 172.21.15.28:0/380211002 >> 172.21.15.28:0/380211002 conn(0x7f3c3c0721c0 msgr2=0x7f3c3c0fa030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:42:13.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:13.089+0000 7f3c44613640 1 -- 172.21.15.28:0/380211002 shutdown_connections 2024-07-03T15:42:13.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:13.089+0000 7f3c44613640 1 -- 172.21.15.28:0/380211002 wait complete. 2024-07-03T15:42:13.959 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:13 smithi028 ceph-mon[29148]: pgmap v189: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:42:13.959 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:13 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/380211002' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-07-03T15:42:13.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:13 smithi148 ceph-mon[36056]: pgmap v189: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:42:13.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:13 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/380211002' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-07-03T15:42:13.989 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-07-03T15:42:13.990 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-03T15:42:13.990 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi028.front.sepia.ceph.com 2024-07-03T15:42:13.990 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-07-03T15:42:13.990 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-03T15:42:13.990 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-07-03T15:42:13.990 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-07-03T15:42:13.990 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-03T15:42:13.990 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-03T15:42:13.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-03T15:42:13.990 DEBUG:teuthology.orchestra.run.smithi028:> ip addr 2024-07-03T15:42:14.049 INFO:teuthology.orchestra.run.smithi028.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-03T15:42:14.050 INFO:teuthology.orchestra.run.smithi028.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-03T15:42:14.050 INFO:teuthology.orchestra.run.smithi028.stdout: inet 127.0.0.1/8 scope host lo 2024-07-03T15:42:14.050 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-07-03T15:42:14.050 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 ::1/128 scope host 2024-07-03T15:42:14.050 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-07-03T15:42:14.050 INFO:teuthology.orchestra.run.smithi028.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-03T15:42:14.050 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:16 brd ff:ff:ff:ff:ff:ff 2024-07-03T15:42:14.050 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f0 2024-07-03T15:42:14.050 INFO:teuthology.orchestra.run.smithi028.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-03T15:42:14.050 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:17 brd ff:ff:ff:ff:ff:ff 2024-07-03T15:42:14.050 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f1 2024-07-03T15:42:14.051 INFO:teuthology.orchestra.run.smithi028.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-03T15:42:14.051 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3c brd ff:ff:ff:ff:ff:ff 2024-07-03T15:42:14.051 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f0 2024-07-03T15:42:14.051 INFO:teuthology.orchestra.run.smithi028.stdout: inet 172.21.15.28/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-07-03T15:42:14.051 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft 42232sec preferred_lft 42232sec 2024-07-03T15:42:14.051 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ae1f:6bff:fea5:ca3c/64 scope link noprefixroute 2024-07-03T15:42:14.051 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-07-03T15:42:14.051 INFO:teuthology.orchestra.run.smithi028.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-03T15:42:14.051 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3d brd ff:ff:ff:ff:ff:ff 2024-07-03T15:42:14.051 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f1 2024-07-03T15:42:14.052 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-07-03T15:42:14.052 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-03T15:42:14.052 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-07-03T15:42:14.052 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link add name ceph-brx type bridge 2024-07-03T15:42:14.052 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip addr flush dev ceph-brx 2024-07-03T15:42:14.052 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set ceph-brx up 2024-07-03T15:42:14.052 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-07-03T15:42:14.052 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-07-03T15:42:14.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-03T15:42:14.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-03T15:42:14.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-03T15:42:14.273 DEBUG:teuthology.orchestra.run.smithi028:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-03T15:42:14.300 INFO:teuthology.orchestra.run.smithi028.stdout:1 2024-07-03T15:42:14.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-03T15:42:14.302 DEBUG:teuthology.orchestra.run.smithi028:> route 2024-07-03T15:42:14.373 INFO:teuthology.orchestra.run.smithi028.stdout:Kernel IP routing table 2024-07-03T15:42:14.373 INFO:teuthology.orchestra.run.smithi028.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-03T15:42:14.373 INFO:teuthology.orchestra.run.smithi028.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-07-03T15:42:14.373 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-07-03T15:42:14.373 INFO:teuthology.orchestra.run.smithi028.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-03T15:42:14.374 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-03T15:42:14.374 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-07-03T15:42:14.374 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-07-03T15:42:14.374 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-07-03T15:42:14.374 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-07-03T15:42:14.374 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-07-03T15:42:14.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-03T15:42:14.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-03T15:42:14.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-03T15:42:14.715 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2024-07-03T15:42:14.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-03T15:42:14.732 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list-id 2024-07-03T15:42:14.787 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-03T15:42:14.787 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-07-03T15:42:14.787 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-03T15:42:14.787 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-07-03T15:42:14.787 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-07-03T15:42:14.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-03T15:42:14.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-03T15:42:14.907 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-07-03T15:42:14.907 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-03T15:42:14.907 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-07-03T15:42:14.908 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-07-03T15:42:14.908 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-07-03T15:42:14.908 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-07-03T15:42:14.908 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-07-03T15:42:14.908 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-07-03T15:42:14.908 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-07-03T15:42:14.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-03T15:42:15.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-03T15:42:15.036 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-03T15:42:15.036 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-07-03T15:42:15.036 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set brx.0 up 2024-07-03T15:42:15.036 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set dev brx.0 master ceph-brx 2024-07-03T15:42:15.036 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-07-03T15:42:15.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-03T15:42:15.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-03T15:42:15.116 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-03T15:42:15.116 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-07-03T15:42:15.145 INFO:teuthology.orchestra.run.smithi028.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-07-03T15:42:15.145 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-03T15:42:15.145 DEBUG:teuthology.orchestra.run.smithi028:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-07-03T15:42:15.200 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-07-03T15:42:15.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-03T15:42:15.201 DEBUG:teuthology.orchestra.run.smithi028:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-03T15:42:15.300 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-03T15:42:15.300 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-03T15:42:15.300 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-07-03T15:42:15.438 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-03T15:42:15.439 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-03T15:42:15.439 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-07-03T15:42:15.439 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.148:3300" 2024-07-03T15:42:15.439 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-03T15:42:15.443 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-03T15:42:15.444 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-03T15:42:15.444 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-07-03T15:42:15.444 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.148:3300" 2024-07-03T15:42:15.444 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-03T15:42:15.444 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:6789,172.21.15.148:6789" 2024-07-03T15:42:15.444 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.28:6789,172.21.15.148:6789:/ 2024-07-03T15:42:15.444 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=a85e4be6-3951-11ef-bcab-c7b262605968" will pass to kernel 2024-07-03T15:42:15.444 INFO:teuthology.orchestra.run.smithi028.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-07-03T15:42:15.445 INFO:teuthology.orchestra.run.smithi028.stdout: You just mounted a file system that supports labels which does not 2024-07-03T15:42:15.445 INFO:teuthology.orchestra.run.smithi028.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-07-03T15:42:15.445 INFO:teuthology.orchestra.run.smithi028.stdout: applications will generate AVC messages and not be allowed access to 2024-07-03T15:42:15.445 INFO:teuthology.orchestra.run.smithi028.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-07-03T15:42:15.446 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-07-03T15:42:15.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-03T15:42:15.447 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-07-03T15:42:15.472 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-03T15:42:15.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-03T15:42:15.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-03T15:42:15.579 INFO:teuthology.orchestra.run.smithi028.stdout:client24301 2024-07-03T15:42:15.579 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-07-03T15:42:15.579 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-03T15:42:15.597 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-03T15:42:15.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-03T15:42:15.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-03T15:42:15.702 INFO:teuthology.orchestra.run.smithi028.stdout:client24301 2024-07-03T15:42:15.706 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-03T15:42:15.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-03T15:42:15.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-03T15:42:15.768 INFO:teuthology.orchestra.run.smithi028.stdout:client24301 2024-07-03T15:42:15.768 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/a85e4be6-3951-11ef-bcab-c7b262605968.client24301/status') 2024-07-03T15:42:15.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-03T15:42:15.832 INFO:teuthology.orchestra.run.smithi028.stdout:instance: client.24301 (3)172.21.15.28:0/1387201540 2024-07-03T15:42:15.832 INFO:teuthology.orchestra.run.smithi028.stdout:blocklisted: false 2024-07-03T15:42:15.832 INFO:teuthology.orchestra.run.smithi028.stderr:0+1 records in 2024-07-03T15:42:15.832 INFO:teuthology.orchestra.run.smithi028.stderr:0+1 records out 2024-07-03T15:42:15.832 INFO:teuthology.orchestra.run.smithi028.stderr:71 bytes copied, 2.4452e-05 s, 2.9 MB/s 2024-07-03T15:42:15.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-03T15:42:15.839 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-07-03T15:42:15.840 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-07-03T15:42:15.840 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-07-03T15:42:15.840 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- ceph fs ls 2024-07-03T15:42:15.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:15 smithi148 ceph-mon[36056]: pgmap v190: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:42:15.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:15 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:42:16.105 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:42:16.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:15 smithi028 ceph-mon[29148]: pgmap v190: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:42:16.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:15 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:42:17.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.663+0000 7f5ce0f7c640 1 -- 172.21.15.28:0/2353969036 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cdc102d40 msgr2=0x7f5cdc0faa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:17.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.663+0000 7f5ce0f7c640 1 --2- 172.21.15.28:0/2353969036 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cdc102d40 0x7f5cdc0faa90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5cc40048d0 tx=0x7f5cc4009da0 comp rx=0 tx=0).stop 2024-07-03T15:42:17.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.663+0000 7f5ce0f7c640 1 -- 172.21.15.28:0/2353969036 shutdown_connections 2024-07-03T15:42:17.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.663+0000 7f5ce0f7c640 1 --2- 172.21.15.28:0/2353969036 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cdc102d40 0x7f5cdc0faa90 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:17.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.663+0000 7f5ce0f7c640 1 --2- 172.21.15.28:0/2353969036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cdc1023a0 0x7f5cdc102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:17.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.663+0000 7f5ce0f7c640 1 -- 172.21.15.28:0/2353969036 >> 172.21.15.28:0/2353969036 conn(0x7f5cdc070af0 msgr2=0x7f5cdc070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:42:17.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.663+0000 7f5ce0f7c640 1 -- 172.21.15.28:0/2353969036 shutdown_connections 2024-07-03T15:42:17.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.663+0000 7f5ce0f7c640 1 -- 172.21.15.28:0/2353969036 wait complete. 2024-07-03T15:42:17.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.664+0000 7f5ce0f7c640 1 Processor -- start 2024-07-03T15:42:17.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.664+0000 7f5ce0f7c640 1 -- start start 2024-07-03T15:42:17.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.664+0000 7f5ce0f7c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cdc1023a0 0x7f5cdc113220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:17.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.664+0000 7f5ce0f7c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cdc102d40 0x7f5cdc113760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:17.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.664+0000 7f5ce0f7c640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cdc113df0 con 0x7f5cdc102d40 2024-07-03T15:42:17.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.664+0000 7f5ce0f7c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cdc10d1a0 con 0x7f5cdc1023a0 2024-07-03T15:42:17.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.664+0000 7f5cd9d9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cdc102d40 0x7f5cdc113760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:17.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.664+0000 7f5cd9d9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cdc102d40 0x7f5cdc113760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:40948/0 (socket says 172.21.15.28:40948) 2024-07-03T15:42:17.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.664+0000 7f5cd9d9b640 1 -- 172.21.15.28:0/3472686803 learned_addr learned my addr 172.21.15.28:0/3472686803 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:42:17.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.664+0000 7f5cda59c640 1 --2- 172.21.15.28:0/3472686803 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cdc1023a0 0x7f5cdc113220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:17.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.664+0000 7f5cd9d9b640 1 -- 172.21.15.28:0/3472686803 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cdc1023a0 msgr2=0x7f5cdc113220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:17.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.664+0000 7f5cd9d9b640 1 --2- 172.21.15.28:0/3472686803 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cdc1023a0 0x7f5cdc113220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:17.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.664+0000 7f5cd9d9b640 1 -- 172.21.15.28:0/3472686803 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc4005040 con 0x7f5cdc102d40 2024-07-03T15:42:17.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.665+0000 7f5cd9d9b640 1 --2- 172.21.15.28:0/3472686803 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cdc102d40 0x7f5cdc113760 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5cc402a530 tx=0x7f5cc4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:42:17.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.665+0000 7f5cd2ffd640 1 -- 172.21.15.28:0/3472686803 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cc4038070 con 0x7f5cdc102d40 2024-07-03T15:42:17.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.665+0000 7f5ce0f7c640 1 -- 172.21.15.28:0/3472686803 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cdc10d420 con 0x7f5cdc102d40 2024-07-03T15:42:17.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.665+0000 7f5cd2ffd640 1 -- 172.21.15.28:0/3472686803 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5cc4008ee0 con 0x7f5cdc102d40 2024-07-03T15:42:17.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.665+0000 7f5cd2ffd640 1 -- 172.21.15.28:0/3472686803 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cc4033730 con 0x7f5cdc102d40 2024-07-03T15:42:17.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.665+0000 7f5ce0f7c640 1 -- 172.21.15.28:0/3472686803 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cdc10d8e0 con 0x7f5cdc102d40 2024-07-03T15:42:17.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.666+0000 7f5ce0f7c640 1 -- 172.21.15.28:0/3472686803 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cdc0655c0 con 0x7f5cdc102d40 2024-07-03T15:42:17.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.667+0000 7f5cd2ffd640 1 -- 172.21.15.28:0/3472686803 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f5cc4030020 con 0x7f5cdc102d40 2024-07-03T15:42:17.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.667+0000 7f5cd2ffd640 1 --2- 172.21.15.28:0/3472686803 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f5cc8067190 0x7f5cc8069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:17.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.667+0000 7f5cd2ffd640 1 -- 172.21.15.28:0/3472686803 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f5cc40a48d0 con 0x7f5cdc102d40 2024-07-03T15:42:17.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.667+0000 7f5cda59c640 1 --2- 172.21.15.28:0/3472686803 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f5cc8067190 0x7f5cc8069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:17.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.668+0000 7f5cda59c640 1 --2- 172.21.15.28:0/3472686803 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f5cc8067190 0x7f5cc8069640 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc006fd0 tx=0x7f5ccc008040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:42:17.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.671+0000 7f5cd2ffd640 1 -- 172.21.15.28:0/3472686803 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5cc4071b70 con 0x7f5cdc102d40 2024-07-03T15:42:17.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:17 smithi028 ceph-mon[29148]: pgmap v191: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-07-03T15:42:17.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.886+0000 7f5ce0f7c640 1 -- 172.21.15.28:0/3472686803 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5cdc1095a0 con 0x7f5cdc102d40 2024-07-03T15:42:17.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.886+0000 7f5cd2ffd640 1 -- 172.21.15.28:0/3472686803 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+83 (secure 0 0 0) 0x7f5cc4075190 con 0x7f5cdc102d40 2024-07-03T15:42:17.891 INFO:teuthology.orchestra.run.smithi028.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-07-03T15:42:17.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.888+0000 7f5ce0f7c640 1 -- 172.21.15.28:0/3472686803 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f5cc8067190 msgr2=0x7f5cc8069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:17.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.888+0000 7f5ce0f7c640 1 --2- 172.21.15.28:0/3472686803 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f5cc8067190 0x7f5cc8069640 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc006fd0 tx=0x7f5ccc008040 comp rx=0 tx=0).stop 2024-07-03T15:42:17.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.888+0000 7f5ce0f7c640 1 -- 172.21.15.28:0/3472686803 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cdc102d40 msgr2=0x7f5cdc113760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:17.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.888+0000 7f5ce0f7c640 1 --2- 172.21.15.28:0/3472686803 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cdc102d40 0x7f5cdc113760 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5cc402a530 tx=0x7f5cc4009da0 comp rx=0 tx=0).stop 2024-07-03T15:42:17.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.888+0000 7f5ce0f7c640 1 -- 172.21.15.28:0/3472686803 shutdown_connections 2024-07-03T15:42:17.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.888+0000 7f5ce0f7c640 1 --2- 172.21.15.28:0/3472686803 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f5cc8067190 0x7f5cc8069640 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:17.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.888+0000 7f5ce0f7c640 1 --2- 172.21.15.28:0/3472686803 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cdc102d40 0x7f5cdc113760 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:17.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.888+0000 7f5ce0f7c640 1 --2- 172.21.15.28:0/3472686803 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cdc1023a0 0x7f5cdc113220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:17.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.888+0000 7f5ce0f7c640 1 -- 172.21.15.28:0/3472686803 >> 172.21.15.28:0/3472686803 conn(0x7f5cdc070af0 msgr2=0x7f5cdc105160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:42:17.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.888+0000 7f5ce0f7c640 1 -- 172.21.15.28:0/3472686803 shutdown_connections 2024-07-03T15:42:17.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:17.888+0000 7f5ce0f7c640 1 -- 172.21.15.28:0/3472686803 wait complete. 2024-07-03T15:42:17.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:17 smithi148 ceph-mon[36056]: pgmap v191: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-07-03T15:42:18.575 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-07-03T15:42:18.575 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-03T15:42:18.575 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2024-07-03T15:42:18.575 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-07-03T15:42:18.576 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-07-03T15:42:18.576 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-07-03T15:42:18.576 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-07-03T15:42:18.576 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-03T15:42:18.576 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-07-03T15:42:18.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-03T15:42:18.576 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2024-07-03T15:42:18.596 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-03T15:42:18.596 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-03T15:42:18.596 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2024-07-03T15:42:18.596 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-07-03T15:42:18.596 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2024-07-03T15:42:18.596 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-07-03T15:42:18.596 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-03T15:42:18.596 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2024-07-03T15:42:18.596 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f0 2024-07-03T15:42:18.597 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-03T15:42:18.597 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2024-07-03T15:42:18.597 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f1 2024-07-03T15:42:18.597 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-03T15:42:18.597 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2024-07-03T15:42:18.597 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-03T15:42:18.597 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2024-07-03T15:42:18.597 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-07-03T15:42:18.597 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 42236sec preferred_lft 42236sec 2024-07-03T15:42:18.597 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::ec4:7aff:fe88:7257/64 scope link 2024-07-03T15:42:18.597 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-07-03T15:42:18.598 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-07-03T15:42:18.598 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-03T15:42:18.598 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-07-03T15:42:18.598 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link add name ceph-brx type bridge 2024-07-03T15:42:18.598 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip addr flush dev ceph-brx 2024-07-03T15:42:18.598 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set ceph-brx up 2024-07-03T15:42:18.598 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-07-03T15:42:18.598 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-07-03T15:42:18.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:42:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-03T15:42:18.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:42:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-03T15:42:18.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-03T15:42:18.832 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-03T15:42:18.853 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:18 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/3472686803' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-07-03T15:42:18.858 INFO:teuthology.orchestra.run.smithi148.stdout:1 2024-07-03T15:42:18.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-03T15:42:18.860 DEBUG:teuthology.orchestra.run.smithi148:> route 2024-07-03T15:42:18.949 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2024-07-03T15:42:18.949 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-03T15:42:18.949 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-07-03T15:42:18.950 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-07-03T15:42:18.950 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-03T15:42:18.950 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-03T15:42:18.950 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-07-03T15:42:18.950 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-07-03T15:42:18.950 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-07-03T15:42:18.950 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-07-03T15:42:18.950 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-07-03T15:42:18.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:18 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/3472686803' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-07-03T15:42:19.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:42:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-03T15:42:19.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:42:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-03T15:42:19.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-03T15:42:19.222 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-07-03T15:42:19.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-03T15:42:19.278 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list-id 2024-07-03T15:42:19.333 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-03T15:42:19.333 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-07-03T15:42:19.333 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-03T15:42:19.334 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-07-03T15:42:19.334 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-07-03T15:42:19.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:42:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-03T15:42:19.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:42:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-03T15:42:19.453 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-07-03T15:42:19.454 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-03T15:42:19.454 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-07-03T15:42:19.454 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-07-03T15:42:19.454 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-07-03T15:42:19.454 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-07-03T15:42:19.454 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-07-03T15:42:19.454 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-07-03T15:42:19.454 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-07-03T15:42:19.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:42:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-03T15:42:19.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:42:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-03T15:42:19.607 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-03T15:42:19.607 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-07-03T15:42:19.607 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.0 up 2024-07-03T15:42:19.607 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set dev brx.0 master ceph-brx 2024-07-03T15:42:19.607 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-07-03T15:42:19.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:42:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-03T15:42:19.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:42:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-03T15:42:19.686 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-03T15:42:19.687 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-07-03T15:42:19.701 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-07-03T15:42:19.701 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-03T15:42:19.702 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-07-03T15:42:19.757 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-07-03T15:42:19.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-03T15:42:19.757 DEBUG:teuthology.orchestra.run.smithi148:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-07-03T15:42:19.851 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-07-03T15:42:19.851 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-07-03T15:42:19.852 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-07-03T15:42:19.876 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:19 smithi148 ceph-mon[36056]: pgmap v192: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-07-03T15:42:19.972 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-07-03T15:42:19.972 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-07-03T15:42:19.972 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-07-03T15:42:19.972 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.148:3300" 2024-07-03T15:42:19.973 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-03T15:42:19.977 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-07-03T15:42:19.977 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-07-03T15:42:19.977 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-07-03T15:42:19.977 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.148:3300" 2024-07-03T15:42:19.977 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-03T15:42:19.978 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.28:6789,172.21.15.148:6789" 2024-07-03T15:42:19.978 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.28:6789,172.21.15.148:6789:/ 2024-07-03T15:42:19.978 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=a85e4be6-3951-11ef-bcab-c7b262605968" will pass to kernel 2024-07-03T15:42:19.978 INFO:teuthology.orchestra.run.smithi148.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-07-03T15:42:19.978 INFO:teuthology.orchestra.run.smithi148.stdout: You just mounted a file system that supports labels which does not 2024-07-03T15:42:19.978 INFO:teuthology.orchestra.run.smithi148.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-07-03T15:42:19.979 INFO:teuthology.orchestra.run.smithi148.stdout: applications will generate AVC messages and not be allowed access to 2024-07-03T15:42:19.979 INFO:teuthology.orchestra.run.smithi148.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-07-03T15:42:19.980 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-07-03T15:42:19.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-03T15:42:19.981 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-07-03T15:42:20.005 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-03T15:42:20.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:42:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-03T15:42:20.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:42:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-03T15:42:20.103 INFO:teuthology.orchestra.run.smithi148.stdout:client14514 2024-07-03T15:42:20.103 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-07-03T15:42:20.103 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-03T15:42:20.120 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-03T15:42:20.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:19 smithi028 ceph-mon[29148]: pgmap v192: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-07-03T15:42:20.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:42:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-03T15:42:20.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:42:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-03T15:42:20.216 INFO:teuthology.orchestra.run.smithi148.stdout:client14514 2024-07-03T15:42:20.216 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-03T15:42:20.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:42:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-03T15:42:20.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:42:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-03T15:42:20.271 INFO:teuthology.orchestra.run.smithi148.stdout:client14514 2024-07-03T15:42:20.272 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/a85e4be6-3951-11ef-bcab-c7b262605968.client14514/status') 2024-07-03T15:42:20.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:42:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-03T15:42:20.330 INFO:teuthology.orchestra.run.smithi148.stdout:instance: client.14514 (3)172.21.15.148:0/4158297337 2024-07-03T15:42:20.330 INFO:teuthology.orchestra.run.smithi148.stdout:blocklisted: false 2024-07-03T15:42:20.330 INFO:teuthology.orchestra.run.smithi148.stderr:0+1 records in 2024-07-03T15:42:20.330 INFO:teuthology.orchestra.run.smithi148.stderr:0+1 records out 2024-07-03T15:42:20.331 INFO:teuthology.orchestra.run.smithi148.stderr:72 bytes copied, 2.0928e-05 s, 3.4 MB/s 2024-07-03T15:42:20.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-07-03T15:42:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-03T15:42:20.337 INFO:teuthology.run_tasks:Running task print... 2024-07-03T15:42:20.347 INFO:teuthology.task.print:**** done client 2024-07-03T15:42:20.347 INFO:teuthology.run_tasks:Running task parallel... 2024-07-03T15:42:20.373 INFO:teuthology.task.parallel:starting parallel... 2024-07-03T15:42:20.374 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-07-03T15:42:20.374 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-07-03T15:42:20.375 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi028.front.sepia.ceph.com 2024-07-03T15:42:20.375 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -e sha1=c00506dc10003928f8cb40769168571f611cbd67 -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs false || true' 2024-07-03T15:42:20.375 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-07-03T15:42:20.375 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-07-03T15:42:20.379 INFO:tasks.workunit:Pulling workunits from ref 47d23ed6cd9bd552d37bff09d8392220d7ba7b11 2024-07-03T15:42:20.380 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-07-03T15:42:20.380 INFO:tasks.workunit:timeout=3h 2024-07-03T15:42:20.380 INFO:tasks.workunit:cleanup=True 2024-07-03T15:42:20.380 DEBUG:teuthology.orchestra.run.smithi028:> stat -- /home/ubuntu/cephtest/mnt.0 2024-07-03T15:42:20.401 INFO:teuthology.orchestra.run.smithi028.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-07-03T15:42:20.401 INFO:teuthology.orchestra.run.smithi028.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-07-03T15:42:20.401 INFO:teuthology.orchestra.run.smithi028.stdout:Device: 98h/152d Inode: 1 Links: 2 2024-07-03T15:42:20.401 INFO:teuthology.orchestra.run.smithi028.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-03T15:42:20.401 INFO:teuthology.orchestra.run.smithi028.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-07-03T15:42:20.401 INFO:teuthology.orchestra.run.smithi028.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-07-03T15:42:20.402 INFO:teuthology.orchestra.run.smithi028.stdout:Modify: 2024-07-03 15:41:45.679827785 +0000 2024-07-03T15:42:20.402 INFO:teuthology.orchestra.run.smithi028.stdout:Change: 2024-07-03 15:42:15.467022061 +0000 2024-07-03T15:42:20.402 INFO:teuthology.orchestra.run.smithi028.stdout: Birth: 2024-07-03 15:41:45.679827785 +0000 2024-07-03T15:42:20.402 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-07-03T15:42:20.402 DEBUG:teuthology.orchestra.run.smithi028:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-07-03T15:42:20.473 DEBUG:teuthology.orchestra.run.smithi148:> stat -- /home/ubuntu/cephtest/mnt.1 2024-07-03T15:42:20.489 INFO:teuthology.orchestra.run.smithi148.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-07-03T15:42:20.489 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 1 Blocks: 0 IO Block: 65536 directory 2024-07-03T15:42:20.489 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 74h/116d Inode: 1 Links: 3 2024-07-03T15:42:20.489 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-03T15:42:20.489 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-07-03T15:42:20.490 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-07-03T15:42:20.490 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-07-03 15:42:20.464881845 +0000 2024-07-03T15:42:20.490 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-07-03 15:42:20.464881845 +0000 2024-07-03T15:42:20.490 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-07-03 15:41:45.679827785 +0000 2024-07-03T15:42:20.490 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-07-03T15:42:20.490 DEBUG:teuthology.orchestra.run.smithi148:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-07-03T15:42:20.559 DEBUG:teuthology.orchestra.run.smithi028:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/guits/ceph /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 47d23ed6cd9bd552d37bff09d8392220d7ba7b11 2024-07-03T15:42:20.560 DEBUG:teuthology.orchestra.run.smithi148:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://github.com/guits/ceph /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout 47d23ed6cd9bd552d37bff09d8392220d7ba7b11 2024-07-03T15:42:20.577 INFO:tasks.workunit.client.0.smithi028.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-07-03T15:42:20.615 INFO:tasks.workunit.client.1.smithi148.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-07-03T15:42:20.662 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:42:21.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:21 smithi148 ceph-mon[36056]: pgmap v193: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-07-03T15:42:22.051 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:21 smithi028 ceph-mon[29148]: pgmap v193: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-07-03T15:42:22.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:22.277+0000 7f0c8181e640 1 -- 172.21.15.28:0/870561663 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0c7c0ff4f0 msgr2=0x7f0c7c1018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:22.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:22.277+0000 7f0c8181e640 1 --2- 172.21.15.28:0/870561663 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0c7c0ff4f0 0x7f0c7c1018d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0c640048d0 tx=0x7f0c64009da0 comp rx=0 tx=0).stop 2024-07-03T15:42:22.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:22.277+0000 7f0c8181e640 1 -- 172.21.15.28:0/870561663 shutdown_connections 2024-07-03T15:42:22.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:22.277+0000 7f0c8181e640 1 --2- 172.21.15.28:0/870561663 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0c7c0ff4f0 0x7f0c7c1018d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:22.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:22.277+0000 7f0c8181e640 1 --2- 172.21.15.28:0/870561663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c7c0659f0 0x7f0c7c0fefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:22.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:22.277+0000 7f0c8181e640 1 -- 172.21.15.28:0/870561663 >> 172.21.15.28:0/870561663 conn(0x7f0c7c0f6a20 msgr2=0x7f0c7c0f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:42:22.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:22.277+0000 7f0c8181e640 1 -- 172.21.15.28:0/870561663 shutdown_connections 2024-07-03T15:42:22.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:22.278+0000 7f0c8181e640 1 -- 172.21.15.28:0/870561663 wait complete. 2024-07-03T15:42:22.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:22.278+0000 7f0c8181e640 1 Processor -- start 2024-07-03T15:42:22.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:22.278+0000 7f0c8181e640 1 -- start start 2024-07-03T15:42:22.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:22.278+0000 7f0c8181e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c7c0659f0 0x7f0c7c10cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:22.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:22.278+0000 7f0c8181e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0c7c0ff4f0 0x7f0c7c10d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:22.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:22.278+0000 7f0c8181e640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7c10db20 con 0x7f0c7c0ff4f0 2024-07-03T15:42:22.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:22.278+0000 7f0c8181e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7c10e2f0 con 0x7f0c7c0659f0 2024-07-03T15:42:22.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:22.279+0000 7f0c73fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0c7c0ff4f0 0x7f0c7c10d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:22.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:22.279+0000 7f0c73fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0c7c0ff4f0 0x7f0c7c10d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52222/0 (socket says 172.21.15.28:52222) 2024-07-03T15:42:22.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:22.279+0000 7f0c7affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c7c0659f0 0x7f0c7c10cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:22.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:22.279+0000 7f0c73fff640 1 -- 172.21.15.28:0/227963165 learned_addr learned my addr 172.21.15.28:0/227963165 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:42:22.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:22.279+0000 7f0c73fff640 1 -- 172.21.15.28:0/227963165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c7c0659f0 msgr2=0x7f0c7c10cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:22.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:22.279+0000 7f0c73fff640 1 --2- 172.21.15.28:0/227963165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c7c0659f0 0x7f0c7c10cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:22.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:22.279+0000 7f0c73fff640 1 -- 172.21.15.28:0/227963165 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c64005040 con 0x7f0c7c0ff4f0 2024-07-03T15:42:22.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:22.279+0000 7f0c7affd640 1 --2- 172.21.15.28:0/227963165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c7c0659f0 0x7f0c7c10cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-03T15:42:22.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:22.279+0000 7f0c73fff640 1 --2- 172.21.15.28:0/227963165 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0c7c0ff4f0 0x7f0c7c10d4d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0c64009f80 tx=0x7f0c64009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:42:22.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:22.279+0000 7f0c737fe640 1 -- 172.21.15.28:0/227963165 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c64038070 con 0x7f0c7c0ff4f0 2024-07-03T15:42:22.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:22.279+0000 7f0c737fe640 1 -- 172.21.15.28:0/227963165 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0c64009720 con 0x7f0c7c0ff4f0 2024-07-03T15:42:22.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:22.279+0000 7f0c8181e640 1 -- 172.21.15.28:0/227963165 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c7c10e520 con 0x7f0c7c0ff4f0 2024-07-03T15:42:22.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:22.279+0000 7f0c737fe640 1 -- 172.21.15.28:0/227963165 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c6403dc80 con 0x7f0c7c0ff4f0 2024-07-03T15:42:22.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:22.279+0000 7f0c8181e640 1 -- 172.21.15.28:0/227963165 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c7c10ea60 con 0x7f0c7c0ff4f0 2024-07-03T15:42:22.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:22.281+0000 7f0c8181e640 1 -- 172.21.15.28:0/227963165 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c7c0fa6a0 con 0x7f0c7c0ff4f0 2024-07-03T15:42:22.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:22.281+0000 7f0c737fe640 1 -- 172.21.15.28:0/227963165 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f0c64030020 con 0x7f0c7c0ff4f0 2024-07-03T15:42:22.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:22.281+0000 7f0c737fe640 1 --2- 172.21.15.28:0/227963165 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f0c68067190 0x7f0c68069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:22.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:22.282+0000 7f0c737fe640 1 -- 172.21.15.28:0/227963165 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f0c640a5670 con 0x7f0c7c0ff4f0 2024-07-03T15:42:22.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:22.282+0000 7f0c7affd640 1 --2- 172.21.15.28:0/227963165 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f0c68067190 0x7f0c68069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:22.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:22.282+0000 7f0c7affd640 1 --2- 172.21.15.28:0/227963165 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f0c68067190 0x7f0c68069640 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c004fd0 tx=0x7f0c6c00b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:42:22.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:22.286+0000 7f0c737fe640 1 -- 172.21.15.28:0/227963165 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0c64072910 con 0x7f0c7c0ff4f0 2024-07-03T15:42:22.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:22.468+0000 7f0c8181e640 1 -- 172.21.15.28:0/227963165 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7f0c7c04b530 con 0x7f0c7c0ff4f0 2024-07-03T15:42:22.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:22.468+0000 7f0c737fe640 1 -- 172.21.15.28:0/227963165 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=-22 unrecognized config option 'mgr/orchestrator/fail_fs' v21)=-22 unrecognized config option 'mgr/orchestrator/fail_fs' v21) v1 ==== 179+0+0 (secure 0 0 0) 0x7f0c64075f30 con 0x7f0c7c0ff4f0 2024-07-03T15:42:22.472 INFO:teuthology.orchestra.run.smithi028.stderr:Error EINVAL: unrecognized config option 'mgr/orchestrator/fail_fs' 2024-07-03T15:42:23.148 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-07-03T15:42:23.149 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi028.front.sepia.ceph.com 2024-07-03T15:42:23.149 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -e sha1=c00506dc10003928f8cb40769168571f611cbd67 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-07-03T15:42:23.474 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:42:23.860 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:23 smithi028 ceph-mon[29148]: pgmap v194: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 853 B/s wr, 3 op/s 2024-07-03T15:42:23.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:23 smithi148 ceph-mon[36056]: pgmap v194: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 853 B/s wr, 3 op/s 2024-07-03T15:42:25.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.253+0000 7f8b9301f640 1 -- 172.21.15.28:0/2738167164 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b8c0fb240 msgr2=0x7f8b8c0fb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:25.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.253+0000 7f8b9301f640 1 --2- 172.21.15.28:0/2738167164 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b8c0fb240 0x7f8b8c0fb650 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8b800048d0 tx=0x7f8b80009da0 comp rx=0 tx=0).stop 2024-07-03T15:42:25.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.253+0000 7f8b9301f640 1 -- 172.21.15.28:0/2738167164 shutdown_connections 2024-07-03T15:42:25.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.253+0000 7f8b9301f640 1 --2- 172.21.15.28:0/2738167164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b8c0fc510 0x7f8b8c0fc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:25.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.253+0000 7f8b9301f640 1 --2- 172.21.15.28:0/2738167164 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b8c0fb240 0x7f8b8c0fb650 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:25.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.253+0000 7f8b9301f640 1 -- 172.21.15.28:0/2738167164 >> 172.21.15.28:0/2738167164 conn(0x7f8b8c0f6a20 msgr2=0x7f8b8c0f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:42:25.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.253+0000 7f8b9301f640 1 -- 172.21.15.28:0/2738167164 shutdown_connections 2024-07-03T15:42:25.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.253+0000 7f8b9301f640 1 -- 172.21.15.28:0/2738167164 wait complete. 2024-07-03T15:42:25.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.253+0000 7f8b9301f640 1 Processor -- start 2024-07-03T15:42:25.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.253+0000 7f8b9301f640 1 -- start start 2024-07-03T15:42:25.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.254+0000 7f8b9301f640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b8c0fb240 0x7f8b8c0ff360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:25.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.254+0000 7f8b9301f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b8c0fc510 0x7f8b8c0fd850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:25.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.254+0000 7f8b9301f640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8c0fdde0 con 0x7f8b8c0fb240 2024-07-03T15:42:25.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.254+0000 7f8b9301f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8c0fdf20 con 0x7f8b8c0fc510 2024-07-03T15:42:25.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.254+0000 7f8b90dbb640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b8c0fb240 0x7f8b8c0ff360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:25.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.254+0000 7f8b90dbb640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b8c0fb240 0x7f8b8c0ff360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52234/0 (socket says 172.21.15.28:52234) 2024-07-03T15:42:25.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.254+0000 7f8b90dbb640 1 -- 172.21.15.28:0/2878467051 learned_addr learned my addr 172.21.15.28:0/2878467051 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:42:25.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.254+0000 7f8b8bfff640 1 --2- 172.21.15.28:0/2878467051 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b8c0fc510 0x7f8b8c0fd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:25.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.254+0000 7f8b90dbb640 1 -- 172.21.15.28:0/2878467051 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b8c0fc510 msgr2=0x7f8b8c0fd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:25.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.254+0000 7f8b90dbb640 1 --2- 172.21.15.28:0/2878467051 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b8c0fc510 0x7f8b8c0fd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:25.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.254+0000 7f8b90dbb640 1 -- 172.21.15.28:0/2878467051 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b80005040 con 0x7f8b8c0fb240 2024-07-03T15:42:25.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.254+0000 7f8b8bfff640 1 --2- 172.21.15.28:0/2878467051 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b8c0fc510 0x7f8b8c0fd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-03T15:42:25.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.254+0000 7f8b90dbb640 1 --2- 172.21.15.28:0/2878467051 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b8c0fb240 0x7f8b8c0ff360 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8b80009f80 tx=0x7f8b80009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:42:25.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.255+0000 7f8b897fa640 1 -- 172.21.15.28:0/2878467051 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b80038070 con 0x7f8b8c0fb240 2024-07-03T15:42:25.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.255+0000 7f8b9301f640 1 -- 172.21.15.28:0/2878467051 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b8c0fe1a0 con 0x7f8b8c0fb240 2024-07-03T15:42:25.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.255+0000 7f8b897fa640 1 -- 172.21.15.28:0/2878467051 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8b80032970 con 0x7f8b8c0fb240 2024-07-03T15:42:25.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.255+0000 7f8b897fa640 1 -- 172.21.15.28:0/2878467051 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b80033860 con 0x7f8b8c0fb240 2024-07-03T15:42:25.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.255+0000 7f8b9301f640 1 -- 172.21.15.28:0/2878467051 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b8c114fd0 con 0x7f8b8c0fb240 2024-07-03T15:42:25.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.256+0000 7f8b9301f640 1 -- 172.21.15.28:0/2878467051 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b8c0fa6a0 con 0x7f8b8c0fb240 2024-07-03T15:42:25.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.257+0000 7f8b897fa640 1 -- 172.21.15.28:0/2878467051 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f8b80030020 con 0x7f8b8c0fb240 2024-07-03T15:42:25.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.257+0000 7f8b897fa640 1 --2- 172.21.15.28:0/2878467051 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f8b7c067190 0x7f8b7c069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:25.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.257+0000 7f8b897fa640 1 -- 172.21.15.28:0/2878467051 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f8b800a4840 con 0x7f8b8c0fb240 2024-07-03T15:42:25.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.257+0000 7f8b8bfff640 1 --2- 172.21.15.28:0/2878467051 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f8b7c067190 0x7f8b7c069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:25.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.258+0000 7f8b8bfff640 1 --2- 172.21.15.28:0/2878467051 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f8b7c067190 0x7f8b7c069640 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8b78004fd0 tx=0x7f8b7800b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:42:25.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.261+0000 7f8b897fa640 1 -- 172.21.15.28:0/2878467051 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8b80071b90 con 0x7f8b8c0fb240 2024-07-03T15:42:25.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.479+0000 7f8b9301f640 1 -- 172.21.15.28:0/2878467051 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f8b8c04b530 con 0x7f8b8c0fb240 2024-07-03T15:42:25.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.479+0000 7f8b897fa640 1 -- 172.21.15.28:0/2878467051 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7f8b800410e0 con 0x7f8b8c0fb240 2024-07-03T15:42:25.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.482+0000 7f8b9301f640 1 -- 172.21.15.28:0/2878467051 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f8b7c067190 msgr2=0x7f8b7c069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:25.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.482+0000 7f8b9301f640 1 --2- 172.21.15.28:0/2878467051 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f8b7c067190 0x7f8b7c069640 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8b78004fd0 tx=0x7f8b7800b040 comp rx=0 tx=0).stop 2024-07-03T15:42:25.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.482+0000 7f8b9301f640 1 -- 172.21.15.28:0/2878467051 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b8c0fb240 msgr2=0x7f8b8c0ff360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:25.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.482+0000 7f8b9301f640 1 --2- 172.21.15.28:0/2878467051 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b8c0fb240 0x7f8b8c0ff360 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8b80009f80 tx=0x7f8b80009da0 comp rx=0 tx=0).stop 2024-07-03T15:42:25.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.482+0000 7f8b9301f640 1 -- 172.21.15.28:0/2878467051 shutdown_connections 2024-07-03T15:42:25.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.482+0000 7f8b9301f640 1 --2- 172.21.15.28:0/2878467051 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f8b7c067190 0x7f8b7c069640 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:25.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.482+0000 7f8b9301f640 1 --2- 172.21.15.28:0/2878467051 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b8c0fc510 0x7f8b8c0fd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:25.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.482+0000 7f8b9301f640 1 --2- 172.21.15.28:0/2878467051 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b8c0fb240 0x7f8b8c0ff360 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:25.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.482+0000 7f8b9301f640 1 -- 172.21.15.28:0/2878467051 >> 172.21.15.28:0/2878467051 conn(0x7f8b8c0f6a20 msgr2=0x7f8b8c0653a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:42:25.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.482+0000 7f8b9301f640 1 -- 172.21.15.28:0/2878467051 shutdown_connections 2024-07-03T15:42:25.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:25.482+0000 7f8b9301f640 1 -- 172.21.15.28:0/2878467051 wait complete. 2024-07-03T15:42:25.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:25 smithi028 ceph-mon[29148]: pgmap v195: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 853 B/s wr, 2 op/s 2024-07-03T15:42:25.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:25 smithi148 ceph-mon[36056]: pgmap v195: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 853 B/s wr, 2 op/s 2024-07-03T15:42:26.174 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -e sha1=c00506dc10003928f8cb40769168571f611cbd67 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-07-03T15:42:26.455 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:42:27.801 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:27 smithi028 ceph-mon[29148]: pgmap v196: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 853 B/s wr, 3 op/s 2024-07-03T15:42:27.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:27 smithi148 ceph-mon[36056]: pgmap v196: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 853 B/s wr, 3 op/s 2024-07-03T15:42:28.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.016+0000 7f3e00c37640 1 -- 172.21.15.28:0/343428899 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3dfc0fb240 msgr2=0x7f3dfc0fb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:28.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.016+0000 7f3e00c37640 1 --2- 172.21.15.28:0/343428899 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3dfc0fb240 0x7f3dfc0fb650 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3dec0048d0 tx=0x7f3dec009da0 comp rx=0 tx=0).stop 2024-07-03T15:42:28.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.016+0000 7f3e00c37640 1 -- 172.21.15.28:0/343428899 shutdown_connections 2024-07-03T15:42:28.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.016+0000 7f3e00c37640 1 --2- 172.21.15.28:0/343428899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dfc0fc510 0x7f3dfc0fc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:28.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.016+0000 7f3e00c37640 1 --2- 172.21.15.28:0/343428899 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3dfc0fb240 0x7f3dfc0fb650 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:28.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.016+0000 7f3e00c37640 1 -- 172.21.15.28:0/343428899 >> 172.21.15.28:0/343428899 conn(0x7f3dfc0f6a20 msgr2=0x7f3dfc0f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:42:28.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.016+0000 7f3e00c37640 1 -- 172.21.15.28:0/343428899 shutdown_connections 2024-07-03T15:42:28.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.016+0000 7f3e00c37640 1 -- 172.21.15.28:0/343428899 wait complete. 2024-07-03T15:42:28.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.017+0000 7f3e00c37640 1 Processor -- start 2024-07-03T15:42:28.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.017+0000 7f3e00c37640 1 -- start start 2024-07-03T15:42:28.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.017+0000 7f3e00c37640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dfc0fb240 0x7f3dfc0ff360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:28.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.017+0000 7f3e00c37640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3dfc0fc510 0x7f3dfc0fd850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:28.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.017+0000 7f3e00c37640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dfc0fdde0 con 0x7f3dfc0fc510 2024-07-03T15:42:28.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.017+0000 7f3e00c37640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dfc0fdf20 con 0x7f3dfc0fb240 2024-07-03T15:42:28.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.017+0000 7f3df9d9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3dfc0fc510 0x7f3dfc0fd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:28.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.017+0000 7f3dfa59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dfc0fb240 0x7f3dfc0ff360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:28.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.017+0000 7f3df9d9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3dfc0fc510 0x7f3dfc0fd850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52240/0 (socket says 172.21.15.28:52240) 2024-07-03T15:42:28.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.017+0000 7f3df9d9b640 1 -- 172.21.15.28:0/2794962895 learned_addr learned my addr 172.21.15.28:0/2794962895 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:42:28.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.017+0000 7f3dfa59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dfc0fb240 0x7f3dfc0ff360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.28:52952/0 (socket says 172.21.15.28:52952) 2024-07-03T15:42:28.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.017+0000 7f3df9d9b640 1 -- 172.21.15.28:0/2794962895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dfc0fb240 msgr2=0x7f3dfc0ff360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:28.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.018+0000 7f3df9d9b640 1 --2- 172.21.15.28:0/2794962895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dfc0fb240 0x7f3dfc0ff360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:28.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.018+0000 7f3df9d9b640 1 -- 172.21.15.28:0/2794962895 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dec005040 con 0x7f3dfc0fc510 2024-07-03T15:42:28.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.018+0000 7f3dfa59c640 1 --2- 172.21.15.28:0/2794962895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dfc0fb240 0x7f3dfc0ff360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-03T15:42:28.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.018+0000 7f3df9d9b640 1 --2- 172.21.15.28:0/2794962895 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3dfc0fc510 0x7f3dfc0fd850 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3df000eb70 tx=0x7f3df000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:42:28.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.018+0000 7f3deaffd640 1 -- 172.21.15.28:0/2794962895 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3df0007c30 con 0x7f3dfc0fc510 2024-07-03T15:42:28.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.018+0000 7f3deaffd640 1 -- 172.21.15.28:0/2794962895 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3df000f040 con 0x7f3dfc0fc510 2024-07-03T15:42:28.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.018+0000 7f3e00c37640 1 -- 172.21.15.28:0/2794962895 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dfc0fe1a0 con 0x7f3dfc0fc510 2024-07-03T15:42:28.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.018+0000 7f3deaffd640 1 -- 172.21.15.28:0/2794962895 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3df00095b0 con 0x7f3dfc0fc510 2024-07-03T15:42:28.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.018+0000 7f3e00c37640 1 -- 172.21.15.28:0/2794962895 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dfc114fb0 con 0x7f3dfc0fc510 2024-07-03T15:42:28.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.019+0000 7f3e00c37640 1 -- 172.21.15.28:0/2794962895 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dfc1045a0 con 0x7f3dfc0fc510 2024-07-03T15:42:28.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.020+0000 7f3deaffd640 1 -- 172.21.15.28:0/2794962895 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f3df0009820 con 0x7f3dfc0fc510 2024-07-03T15:42:28.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.020+0000 7f3deaffd640 1 --2- 172.21.15.28:0/2794962895 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3de4067190 0x7f3de4069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:28.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.020+0000 7f3deaffd640 1 -- 172.21.15.28:0/2794962895 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f3df0084770 con 0x7f3dfc0fc510 2024-07-03T15:42:28.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.020+0000 7f3dfa59c640 1 --2- 172.21.15.28:0/2794962895 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3de4067190 0x7f3de4069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:28.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.021+0000 7f3dfa59c640 1 --2- 172.21.15.28:0/2794962895 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3de4067190 0x7f3de4069640 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3dec009f80 tx=0x7f3dec035040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:42:28.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.025+0000 7f3deaffd640 1 -- 172.21.15.28:0/2794962895 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3df004e250 con 0x7f3dfc0fc510 2024-07-03T15:42:28.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.255+0000 7f3e00c37640 1 -- 172.21.15.28:0/2794962895 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f3dfc0fa6a0 con 0x7f3dfc0fc510 2024-07-03T15:42:28.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.256+0000 7f3deaffd640 1 -- 172.21.15.28:0/2794962895 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7f3df0055550 con 0x7f3dfc0fc510 2024-07-03T15:42:28.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.257+0000 7f3e00c37640 1 -- 172.21.15.28:0/2794962895 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3de4067190 msgr2=0x7f3de4069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:28.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.257+0000 7f3e00c37640 1 --2- 172.21.15.28:0/2794962895 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3de4067190 0x7f3de4069640 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3dec009f80 tx=0x7f3dec035040 comp rx=0 tx=0).stop 2024-07-03T15:42:28.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.257+0000 7f3e00c37640 1 -- 172.21.15.28:0/2794962895 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3dfc0fc510 msgr2=0x7f3dfc0fd850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:28.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.257+0000 7f3e00c37640 1 --2- 172.21.15.28:0/2794962895 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3dfc0fc510 0x7f3dfc0fd850 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3df000eb70 tx=0x7f3df000b040 comp rx=0 tx=0).stop 2024-07-03T15:42:28.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.257+0000 7f3e00c37640 1 -- 172.21.15.28:0/2794962895 shutdown_connections 2024-07-03T15:42:28.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.257+0000 7f3e00c37640 1 --2- 172.21.15.28:0/2794962895 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3de4067190 0x7f3de4069640 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:28.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.257+0000 7f3e00c37640 1 --2- 172.21.15.28:0/2794962895 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3dfc0fc510 0x7f3dfc0fd850 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:28.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.257+0000 7f3e00c37640 1 --2- 172.21.15.28:0/2794962895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dfc0fb240 0x7f3dfc0ff360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:28.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.258+0000 7f3e00c37640 1 -- 172.21.15.28:0/2794962895 >> 172.21.15.28:0/2794962895 conn(0x7f3dfc0f6a20 msgr2=0x7f3dfc065360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:42:28.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.258+0000 7f3e00c37640 1 -- 172.21.15.28:0/2794962895 shutdown_connections 2024-07-03T15:42:28.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:28.258+0000 7f3e00c37640 1 -- 172.21.15.28:0/2794962895 wait complete. 2024-07-03T15:42:28.936 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -e sha1=c00506dc10003928f8cb40769168571f611cbd67 -- bash -c 'ceph config set global log_to_journald false --force' 2024-07-03T15:42:29.217 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:42:29.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:29 smithi028 ceph-mon[29148]: pgmap v197: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 767 B/s wr, 2 op/s 2024-07-03T15:42:29.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:29 smithi148 ceph-mon[36056]: pgmap v197: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 767 B/s wr, 2 op/s 2024-07-03T15:42:30.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:30.916+0000 7f999f3e1640 1 -- 172.21.15.28:0/3661245709 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f99980fc530 msgr2=0x7f99980fc9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:30.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:30.916+0000 7f999f3e1640 1 --2- 172.21.15.28:0/3661245709 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f99980fc530 0x7f99980fc9a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f99880048d0 tx=0x7f9988009da0 comp rx=0 tx=0).stop 2024-07-03T15:42:30.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:30.916+0000 7f999f3e1640 1 -- 172.21.15.28:0/3661245709 shutdown_connections 2024-07-03T15:42:30.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:30.916+0000 7f999f3e1640 1 --2- 172.21.15.28:0/3661245709 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f99980fc530 0x7f99980fc9a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:30.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:30.916+0000 7f999f3e1640 1 --2- 172.21.15.28:0/3661245709 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99980fb260 0x7f99980fb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:30.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:30.916+0000 7f999f3e1640 1 -- 172.21.15.28:0/3661245709 >> 172.21.15.28:0/3661245709 conn(0x7f99980f6a20 msgr2=0x7f99980f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:42:30.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:30.916+0000 7f999f3e1640 1 -- 172.21.15.28:0/3661245709 shutdown_connections 2024-07-03T15:42:30.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:30.916+0000 7f999f3e1640 1 -- 172.21.15.28:0/3661245709 wait complete. 2024-07-03T15:42:30.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:30.916+0000 7f999f3e1640 1 Processor -- start 2024-07-03T15:42:30.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:30.917+0000 7f999f3e1640 1 -- start start 2024-07-03T15:42:30.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:30.917+0000 7f999f3e1640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f99980fb260 0x7f999810d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:30.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:30.917+0000 7f999f3e1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99980fc530 0x7f999810d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:30.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:30.917+0000 7f999f3e1640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f999810dbe0 con 0x7f99980fb260 2024-07-03T15:42:30.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:30.917+0000 7f999f3e1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f999810e400 con 0x7f99980fc530 2024-07-03T15:42:30.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:30.917+0000 7f999d17d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f99980fb260 0x7f999810d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:30.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:30.917+0000 7f999d17d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f99980fb260 0x7f999810d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50868/0 (socket says 172.21.15.28:50868) 2024-07-03T15:42:30.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:30.917+0000 7f999d17d640 1 -- 172.21.15.28:0/585123410 learned_addr learned my addr 172.21.15.28:0/585123410 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:42:30.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:30.917+0000 7f999c97c640 1 --2- 172.21.15.28:0/585123410 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99980fc530 0x7f999810d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:30.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:30.917+0000 7f999d17d640 1 -- 172.21.15.28:0/585123410 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99980fc530 msgr2=0x7f999810d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:30.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:30.917+0000 7f999d17d640 1 --2- 172.21.15.28:0/585123410 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99980fc530 0x7f999810d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:30.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:30.917+0000 7f999d17d640 1 -- 172.21.15.28:0/585123410 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9988005040 con 0x7f99980fb260 2024-07-03T15:42:30.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:30.917+0000 7f999d17d640 1 --2- 172.21.15.28:0/585123410 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f99980fb260 0x7f999810d050 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9994002a50 tx=0x7f9994004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:42:30.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:30.917+0000 7f998dffb640 1 -- 172.21.15.28:0/585123410 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9994007c30 con 0x7f99980fb260 2024-07-03T15:42:30.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:30.918+0000 7f999f3e1640 1 -- 172.21.15.28:0/585123410 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f999810e690 con 0x7f99980fb260 2024-07-03T15:42:30.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:30.918+0000 7f998dffb640 1 -- 172.21.15.28:0/585123410 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f999400f040 con 0x7f99980fb260 2024-07-03T15:42:30.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:30.918+0000 7f998dffb640 1 -- 172.21.15.28:0/585123410 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f999400dad0 con 0x7f99980fb260 2024-07-03T15:42:30.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:30.918+0000 7f999f3e1640 1 -- 172.21.15.28:0/585123410 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f999810ec00 con 0x7f99980fb260 2024-07-03T15:42:30.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:30.919+0000 7f999f3e1640 1 -- 172.21.15.28:0/585123410 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99981045e0 con 0x7f99980fb260 2024-07-03T15:42:30.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:30.919+0000 7f998dffb640 1 -- 172.21.15.28:0/585123410 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f999400dc70 con 0x7f99980fb260 2024-07-03T15:42:30.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:30.919+0000 7f998dffb640 1 --2- 172.21.15.28:0/585123410 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f9984067190 0x7f9984069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:30.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:30.920+0000 7f998dffb640 1 -- 172.21.15.28:0/585123410 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f9994084230 con 0x7f99980fb260 2024-07-03T15:42:30.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:30.920+0000 7f999c97c640 1 --2- 172.21.15.28:0/585123410 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f9984067190 0x7f9984069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:30.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:30.920+0000 7f999c97c640 1 --2- 172.21.15.28:0/585123410 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f9984067190 0x7f9984069640 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f99880043d0 tx=0x7f9988035040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:42:30.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:30.924+0000 7f998dffb640 1 -- 172.21.15.28:0/585123410 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f999404d250 con 0x7f99980fb260 2024-07-03T15:42:30.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:30 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:42:31.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:31.116+0000 7f999f3e1640 1 -- 172.21.15.28:0/585123410 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f999804b5b0 con 0x7f99980fb260 2024-07-03T15:42:31.120 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:30 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:42:31.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:31.117+0000 7f998dffb640 1 -- 172.21.15.28:0/585123410 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7f999804b5b0 con 0x7f99980fb260 2024-07-03T15:42:31.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:31.119+0000 7f999f3e1640 1 -- 172.21.15.28:0/585123410 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f9984067190 msgr2=0x7f9984069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:31.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:31.119+0000 7f999f3e1640 1 --2- 172.21.15.28:0/585123410 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f9984067190 0x7f9984069640 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f99880043d0 tx=0x7f9988035040 comp rx=0 tx=0).stop 2024-07-03T15:42:31.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:31.119+0000 7f999f3e1640 1 -- 172.21.15.28:0/585123410 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f99980fb260 msgr2=0x7f999810d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:31.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:31.119+0000 7f999f3e1640 1 --2- 172.21.15.28:0/585123410 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f99980fb260 0x7f999810d050 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9994002a50 tx=0x7f9994004600 comp rx=0 tx=0).stop 2024-07-03T15:42:31.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:31.119+0000 7f999f3e1640 1 -- 172.21.15.28:0/585123410 shutdown_connections 2024-07-03T15:42:31.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:31.119+0000 7f999f3e1640 1 --2- 172.21.15.28:0/585123410 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f9984067190 0x7f9984069640 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:31.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:31.119+0000 7f999f3e1640 1 --2- 172.21.15.28:0/585123410 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99980fc530 0x7f999810d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:31.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:31.119+0000 7f999f3e1640 1 --2- 172.21.15.28:0/585123410 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f99980fb260 0x7f999810d050 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:31.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:31.119+0000 7f999f3e1640 1 -- 172.21.15.28:0/585123410 >> 172.21.15.28:0/585123410 conn(0x7f99980f6a20 msgr2=0x7f9998065f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:42:31.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:31.119+0000 7f999f3e1640 1 -- 172.21.15.28:0/585123410 shutdown_connections 2024-07-03T15:42:31.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:31.119+0000 7f999f3e1640 1 -- 172.21.15.28:0/585123410 wait complete. 2024-07-03T15:42:31.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:31 smithi028 ceph-mon[29148]: pgmap v198: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 767 B/s wr, 2 op/s 2024-07-03T15:42:31.939 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -e sha1=c00506dc10003928f8cb40769168571f611cbd67 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-07-03T15:42:31.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:31 smithi148 ceph-mon[36056]: pgmap v198: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 767 B/s wr, 2 op/s 2024-07-03T15:42:32.210 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:42:33.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:33 smithi148 ceph-mon[36056]: pgmap v199: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 767 B/s wr, 3 op/s 2024-07-03T15:42:34.032 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:33 smithi028 ceph-mon[29148]: pgmap v199: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 767 B/s wr, 3 op/s 2024-07-03T15:42:34.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.028+0000 7fcace2b6640 1 -- 172.21.15.28:0/2436594060 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcac8066280 msgr2=0x7fcac80fefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:34.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.028+0000 7fcace2b6640 1 --2- 172.21.15.28:0/2436594060 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcac8066280 0x7fcac80fefa0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcab80048d0 tx=0x7fcab8009da0 comp rx=0 tx=0).stop 2024-07-03T15:42:34.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.028+0000 7fcace2b6640 1 -- 172.21.15.28:0/2436594060 shutdown_connections 2024-07-03T15:42:34.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.028+0000 7fcace2b6640 1 --2- 172.21.15.28:0/2436594060 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcac80ff4e0 0x7fcac81018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:34.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.028+0000 7fcace2b6640 1 --2- 172.21.15.28:0/2436594060 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcac8066280 0x7fcac80fefa0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:34.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.028+0000 7fcace2b6640 1 -- 172.21.15.28:0/2436594060 >> 172.21.15.28:0/2436594060 conn(0x7fcac80f6a10 msgr2=0x7fcac80f8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:42:34.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.029+0000 7fcace2b6640 1 -- 172.21.15.28:0/2436594060 shutdown_connections 2024-07-03T15:42:34.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.029+0000 7fcace2b6640 1 -- 172.21.15.28:0/2436594060 wait complete. 2024-07-03T15:42:34.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.029+0000 7fcace2b6640 1 Processor -- start 2024-07-03T15:42:34.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.029+0000 7fcace2b6640 1 -- start start 2024-07-03T15:42:34.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.030+0000 7fcace2b6640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcac8066280 0x7fcac810cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:34.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.030+0000 7fcace2b6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcac80ff4e0 0x7fcac810d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:34.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.030+0000 7fcace2b6640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac810db40 con 0x7fcac8066280 2024-07-03T15:42:34.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.030+0000 7fcace2b6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac810e360 con 0x7fcac80ff4e0 2024-07-03T15:42:34.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.030+0000 7fcac7fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcac8066280 0x7fcac810cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:34.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.030+0000 7fcac7fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcac8066280 0x7fcac810cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50888/0 (socket says 172.21.15.28:50888) 2024-07-03T15:42:34.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.030+0000 7fcac7fff640 1 -- 172.21.15.28:0/3168755734 learned_addr learned my addr 172.21.15.28:0/3168755734 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:42:34.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.030+0000 7fcac77fe640 1 --2- 172.21.15.28:0/3168755734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcac80ff4e0 0x7fcac810d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:34.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.030+0000 7fcac7fff640 1 -- 172.21.15.28:0/3168755734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcac80ff4e0 msgr2=0x7fcac810d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:34.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.030+0000 7fcac7fff640 1 --2- 172.21.15.28:0/3168755734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcac80ff4e0 0x7fcac810d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:34.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.030+0000 7fcac7fff640 1 -- 172.21.15.28:0/3168755734 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab8005040 con 0x7fcac8066280 2024-07-03T15:42:34.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.030+0000 7fcac7fff640 1 --2- 172.21.15.28:0/3168755734 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcac8066280 0x7fcac810cfb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcab802a5e0 tx=0x7fcab8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:42:34.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.030+0000 7fcac4ff9640 1 -- 172.21.15.28:0/3168755734 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcab8038070 con 0x7fcac8066280 2024-07-03T15:42:34.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.030+0000 7fcac4ff9640 1 -- 172.21.15.28:0/3168755734 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcab8009720 con 0x7fcac8066280 2024-07-03T15:42:34.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.031+0000 7fcace2b6640 1 -- 172.21.15.28:0/3168755734 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcac810e590 con 0x7fcac8066280 2024-07-03T15:42:34.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.031+0000 7fcac4ff9640 1 -- 172.21.15.28:0/3168755734 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcab803dc50 con 0x7fcac8066280 2024-07-03T15:42:34.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.031+0000 7fcace2b6640 1 -- 172.21.15.28:0/3168755734 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcac810ead0 con 0x7fcac8066280 2024-07-03T15:42:34.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.032+0000 7fcace2b6640 1 -- 172.21.15.28:0/3168755734 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcac80fa690 con 0x7fcac8066280 2024-07-03T15:42:34.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.032+0000 7fcac4ff9640 1 -- 172.21.15.28:0/3168755734 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7fcab8030020 con 0x7fcac8066280 2024-07-03T15:42:34.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.033+0000 7fcac4ff9640 1 --2- 172.21.15.28:0/3168755734 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fcab0067190 0x7fcab0069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:34.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.033+0000 7fcac4ff9640 1 -- 172.21.15.28:0/3168755734 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fcab80a55d0 con 0x7fcac8066280 2024-07-03T15:42:34.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.033+0000 7fcac77fe640 1 --2- 172.21.15.28:0/3168755734 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fcab0067190 0x7fcab0069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:34.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.033+0000 7fcac77fe640 1 --2- 172.21.15.28:0/3168755734 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fcab0067190 0x7fcab0069640 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcac810cd60 tx=0x7fcabc00b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:42:34.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.037+0000 7fcac4ff9640 1 -- 172.21.15.28:0/3168755734 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fcab8072870 con 0x7fcac8066280 2024-07-03T15:42:34.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.227+0000 7fcace2b6640 1 -- 172.21.15.28:0/3168755734 --> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:c00506dc10003928f8cb40769168571f611cbd67", "target": ["mon-mgr", ""]}) v1 -- 0x7fcac810ee20 con 0x7fcab0067190 2024-07-03T15:42:34.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.233+0000 7fcac4ff9640 1 -- 172.21.15.28:0/3168755734 <== mgr.14210 v2:172.21.15.28:6800/421057178 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7fcac810ee20 con 0x7fcab0067190 2024-07-03T15:42:34.236 INFO:teuthology.orchestra.run.smithi028.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:c00506dc10003928f8cb40769168571f611cbd67 2024-07-03T15:42:34.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.235+0000 7fcace2b6640 1 -- 172.21.15.28:0/3168755734 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fcab0067190 msgr2=0x7fcab0069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:34.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.235+0000 7fcace2b6640 1 --2- 172.21.15.28:0/3168755734 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fcab0067190 0x7fcab0069640 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcac810cd60 tx=0x7fcabc00b040 comp rx=0 tx=0).stop 2024-07-03T15:42:34.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.235+0000 7fcace2b6640 1 -- 172.21.15.28:0/3168755734 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcac8066280 msgr2=0x7fcac810cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:34.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.235+0000 7fcace2b6640 1 --2- 172.21.15.28:0/3168755734 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcac8066280 0x7fcac810cfb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcab802a5e0 tx=0x7fcab8009da0 comp rx=0 tx=0).stop 2024-07-03T15:42:34.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.235+0000 7fcace2b6640 1 -- 172.21.15.28:0/3168755734 shutdown_connections 2024-07-03T15:42:34.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.235+0000 7fcace2b6640 1 --2- 172.21.15.28:0/3168755734 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fcab0067190 0x7fcab0069640 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:34.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.235+0000 7fcace2b6640 1 --2- 172.21.15.28:0/3168755734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcac80ff4e0 0x7fcac810d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:34.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.235+0000 7fcace2b6640 1 --2- 172.21.15.28:0/3168755734 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcac8066280 0x7fcac810cfb0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:34.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.235+0000 7fcace2b6640 1 -- 172.21.15.28:0/3168755734 >> 172.21.15.28:0/3168755734 conn(0x7fcac80f6a10 msgr2=0x7fcac80f6de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:42:34.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.235+0000 7fcace2b6640 1 -- 172.21.15.28:0/3168755734 shutdown_connections 2024-07-03T15:42:34.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:34.235+0000 7fcace2b6640 1 -- 172.21.15.28:0/3168755734 wait complete. 2024-07-03T15:42:34.984 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-07-03T15:42:34.985 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi028.front.sepia.ceph.com 2024-07-03T15:42:34.985 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -e sha1=c00506dc10003928f8cb40769168571f611cbd67 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-07-03T15:42:35.282 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:42:35.608 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:35 smithi028 ceph-mon[29148]: from='client.14534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:c00506dc10003928f8cb40769168571f611cbd67", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:42:35.608 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:35 smithi028 ceph-mon[29148]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:c00506dc10003928f8cb40769168571f611cbd67 2024-07-03T15:42:35.608 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:35 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:42:35.609 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:35 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:42:35.609 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:35 smithi028 ceph-mon[29148]: pgmap v200: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 85 B/s wr, 1 op/s 2024-07-03T15:42:35.609 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:35 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:42:35.609 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:35 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:42:35.609 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:35 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:42:35.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:35 smithi148 ceph-mon[36056]: from='client.14534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:c00506dc10003928f8cb40769168571f611cbd67", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:42:35.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:35 smithi148 ceph-mon[36056]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:c00506dc10003928f8cb40769168571f611cbd67 2024-07-03T15:42:35.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:35 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:42:35.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:35 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:42:35.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:35 smithi148 ceph-mon[36056]: pgmap v200: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 85 B/s wr, 1 op/s 2024-07-03T15:42:35.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:35 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:42:35.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:35 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:42:35.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:35 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:42:36.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:36 smithi148 ceph-mon[36056]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:c00506dc10003928f8cb40769168571f611cbd67 2024-07-03T15:42:36.601 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:36 smithi028 ceph-mon[29148]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:c00506dc10003928f8cb40769168571f611cbd67 2024-07-03T15:42:37.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.203+0000 7fc7872aa640 1 -- 172.21.15.28:0/2092302340 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc7800fc360 msgr2=0x7fc7800fc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:37.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.203+0000 7fc7872aa640 1 --2- 172.21.15.28:0/2092302340 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc7800fc360 0x7fc7800fc770 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc77c0048d0 tx=0x7fc77c009da0 comp rx=0 tx=0).stop 2024-07-03T15:42:37.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.203+0000 7fc7872aa640 1 -- 172.21.15.28:0/2092302340 shutdown_connections 2024-07-03T15:42:37.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.203+0000 7fc7872aa640 1 --2- 172.21.15.28:0/2092302340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7800fd630 0x7fc7800fdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:37.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.203+0000 7fc7872aa640 1 --2- 172.21.15.28:0/2092302340 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc7800fc360 0x7fc7800fc770 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:37.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.203+0000 7fc7872aa640 1 -- 172.21.15.28:0/2092302340 >> 172.21.15.28:0/2092302340 conn(0x7fc7800f7b40 msgr2=0x7fc7800f9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:42:37.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.203+0000 7fc7872aa640 1 -- 172.21.15.28:0/2092302340 shutdown_connections 2024-07-03T15:42:37.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.203+0000 7fc7872aa640 1 -- 172.21.15.28:0/2092302340 wait complete. 2024-07-03T15:42:37.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.203+0000 7fc7872aa640 1 Processor -- start 2024-07-03T15:42:37.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.203+0000 7fc7872aa640 1 -- start start 2024-07-03T15:42:37.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.204+0000 7fc7872aa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7800fc360 0x7fc78010f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:37.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.204+0000 7fc7872aa640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc7800fd630 0x7fc78010f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:37.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.204+0000 7fc7872aa640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78010fcc0 con 0x7fc7800fd630 2024-07-03T15:42:37.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.204+0000 7fc7872aa640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7801104e0 con 0x7fc7800fc360 2024-07-03T15:42:37.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.204+0000 7fc784845640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc7800fd630 0x7fc78010f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:37.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.204+0000 7fc784845640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc7800fd630 0x7fc78010f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50914/0 (socket says 172.21.15.28:50914) 2024-07-03T15:42:37.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.204+0000 7fc784845640 1 -- 172.21.15.28:0/2649804370 learned_addr learned my addr 172.21.15.28:0/2649804370 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:42:37.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.204+0000 7fc785046640 1 --2- 172.21.15.28:0/2649804370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7800fc360 0x7fc78010f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:37.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.204+0000 7fc784845640 1 -- 172.21.15.28:0/2649804370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7800fc360 msgr2=0x7fc78010f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:37.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.204+0000 7fc784845640 1 --2- 172.21.15.28:0/2649804370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7800fc360 0x7fc78010f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:37.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.204+0000 7fc784845640 1 -- 172.21.15.28:0/2649804370 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc77c005040 con 0x7fc7800fd630 2024-07-03T15:42:37.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.204+0000 7fc784845640 1 --2- 172.21.15.28:0/2649804370 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc7800fd630 0x7fc78010f670 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc77000c830 tx=0x7fc770004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:42:37.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.205+0000 7fc775ffb640 1 -- 172.21.15.28:0/2649804370 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc77000a7d0 con 0x7fc7800fd630 2024-07-03T15:42:37.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.205+0000 7fc775ffb640 1 -- 172.21.15.28:0/2649804370 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc77000ae00 con 0x7fc7800fd630 2024-07-03T15:42:37.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.205+0000 7fc7872aa640 1 -- 172.21.15.28:0/2649804370 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc780110770 con 0x7fc7800fd630 2024-07-03T15:42:37.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.205+0000 7fc775ffb640 1 -- 172.21.15.28:0/2649804370 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7700113a0 con 0x7fc7800fd630 2024-07-03T15:42:37.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.205+0000 7fc7872aa640 1 -- 172.21.15.28:0/2649804370 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc780110d10 con 0x7fc7800fd630 2024-07-03T15:42:37.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.206+0000 7fc7872aa640 1 -- 172.21.15.28:0/2649804370 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7800fb7c0 con 0x7fc7800fd630 2024-07-03T15:42:37.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.207+0000 7fc775ffb640 1 -- 172.21.15.28:0/2649804370 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7fc770012400 con 0x7fc7800fd630 2024-07-03T15:42:37.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.207+0000 7fc775ffb640 1 --2- 172.21.15.28:0/2649804370 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fc76c067190 0x7fc76c069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:37.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.208+0000 7fc775ffb640 1 -- 172.21.15.28:0/2649804370 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fc770084b50 con 0x7fc7800fd630 2024-07-03T15:42:37.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.208+0000 7fc785046640 1 --2- 172.21.15.28:0/2649804370 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fc76c067190 0x7fc76c069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:37.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.208+0000 7fc785046640 1 --2- 172.21.15.28:0/2649804370 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fc76c067190 0x7fc76c069640 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc77c000c00 tx=0x7fc77c035040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:42:37.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.212+0000 7fc775ffb640 1 -- 172.21.15.28:0/2649804370 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc770051f10 con 0x7fc7800fd630 2024-07-03T15:42:37.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.477+0000 7fc7872aa640 1 -- 172.21.15.28:0/2649804370 --> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc780105020 con 0x7fc76c067190 2024-07-03T15:42:37.480 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:37 smithi028 ceph-mon[29148]: pgmap v201: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 85 B/s wr, 2 op/s 2024-07-03T15:42:37.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.478+0000 7fc775ffb640 1 -- 172.21.15.28:0/2649804370 <== mgr.14210 v2:172.21.15.28:6800/421057178 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7fc780105020 con 0x7fc76c067190 2024-07-03T15:42:37.481 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:37 smithi148 ceph-mon[36056]: pgmap v201: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 85 B/s wr, 2 op/s 2024-07-03T15:42:37.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.480+0000 7fc7872aa640 1 -- 172.21.15.28:0/2649804370 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fc76c067190 msgr2=0x7fc76c069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:37.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.480+0000 7fc7872aa640 1 --2- 172.21.15.28:0/2649804370 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fc76c067190 0x7fc76c069640 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc77c000c00 tx=0x7fc77c035040 comp rx=0 tx=0).stop 2024-07-03T15:42:37.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.480+0000 7fc7872aa640 1 -- 172.21.15.28:0/2649804370 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc7800fd630 msgr2=0x7fc78010f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:37.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.480+0000 7fc7872aa640 1 --2- 172.21.15.28:0/2649804370 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc7800fd630 0x7fc78010f670 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc77000c830 tx=0x7fc770004430 comp rx=0 tx=0).stop 2024-07-03T15:42:37.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.480+0000 7fc7872aa640 1 -- 172.21.15.28:0/2649804370 shutdown_connections 2024-07-03T15:42:37.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.480+0000 7fc7872aa640 1 --2- 172.21.15.28:0/2649804370 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fc76c067190 0x7fc76c069640 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:37.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.480+0000 7fc7872aa640 1 --2- 172.21.15.28:0/2649804370 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc7800fd630 0x7fc78010f670 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:37.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.481+0000 7fc7872aa640 1 --2- 172.21.15.28:0/2649804370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7800fc360 0x7fc78010f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:37.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.481+0000 7fc7872aa640 1 -- 172.21.15.28:0/2649804370 >> 172.21.15.28:0/2649804370 conn(0x7fc7800f7b40 msgr2=0x7fc7800f90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:42:37.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.481+0000 7fc7872aa640 1 -- 172.21.15.28:0/2649804370 shutdown_connections 2024-07-03T15:42:37.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.481+0000 7fc7872aa640 1 -- 172.21.15.28:0/2649804370 wait complete. 2024-07-03T15:42:37.494 INFO:teuthology.orchestra.run.smithi028.stdout:true 2024-07-03T15:42:37.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.580+0000 7f3ea39b2640 1 -- 172.21.15.28:0/1914863749 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e9c0659f0 msgr2=0x7f3e9c0fefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:37.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.580+0000 7f3ea39b2640 1 --2- 172.21.15.28:0/1914863749 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e9c0659f0 0x7f3e9c0fefb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3e980048d0 tx=0x7f3e98009da0 comp rx=0 tx=0).stop 2024-07-03T15:42:37.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.580+0000 7f3ea39b2640 1 -- 172.21.15.28:0/1914863749 shutdown_connections 2024-07-03T15:42:37.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.580+0000 7f3ea39b2640 1 --2- 172.21.15.28:0/1914863749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e9c0ff4f0 0x7f3e9c1018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:37.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.580+0000 7f3ea39b2640 1 --2- 172.21.15.28:0/1914863749 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e9c0659f0 0x7f3e9c0fefb0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:37.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.580+0000 7f3ea39b2640 1 -- 172.21.15.28:0/1914863749 >> 172.21.15.28:0/1914863749 conn(0x7f3e9c0f6a20 msgr2=0x7f3e9c0f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:42:37.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.581+0000 7f3ea39b2640 1 -- 172.21.15.28:0/1914863749 shutdown_connections 2024-07-03T15:42:37.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.581+0000 7f3ea39b2640 1 -- 172.21.15.28:0/1914863749 wait complete. 2024-07-03T15:42:37.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.581+0000 7f3ea39b2640 1 Processor -- start 2024-07-03T15:42:37.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.581+0000 7f3ea39b2640 1 -- start start 2024-07-03T15:42:37.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.582+0000 7f3ea39b2640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e9c0659f0 0x7f3e9c0fc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:37.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.582+0000 7f3ea39b2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e9c0ff4f0 0x7f3e9c0faeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:37.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.582+0000 7f3ea39b2640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c0fb440 con 0x7f3e9c0659f0 2024-07-03T15:42:37.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.582+0000 7f3ea39b2640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c0fb580 con 0x7f3e9c0ff4f0 2024-07-03T15:42:37.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.582+0000 7f3ea174e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e9c0659f0 0x7f3e9c0fc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:37.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.582+0000 7f3ea174e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e9c0659f0 0x7f3e9c0fc9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50930/0 (socket says 172.21.15.28:50930) 2024-07-03T15:42:37.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.582+0000 7f3ea174e640 1 -- 172.21.15.28:0/525505627 learned_addr learned my addr 172.21.15.28:0/525505627 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:42:37.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.582+0000 7f3ea0f4d640 1 --2- 172.21.15.28:0/525505627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e9c0ff4f0 0x7f3e9c0faeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:37.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.582+0000 7f3ea174e640 1 -- 172.21.15.28:0/525505627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e9c0ff4f0 msgr2=0x7f3e9c0faeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:37.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.582+0000 7f3ea174e640 1 --2- 172.21.15.28:0/525505627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e9c0ff4f0 0x7f3e9c0faeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:37.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.582+0000 7f3ea174e640 1 -- 172.21.15.28:0/525505627 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e98005040 con 0x7f3e9c0659f0 2024-07-03T15:42:37.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.583+0000 7f3ea174e640 1 --2- 172.21.15.28:0/525505627 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e9c0659f0 0x7f3e9c0fc9c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3e9802a5e0 tx=0x7f3e98009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:42:37.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.583+0000 7f3e91ffb640 1 -- 172.21.15.28:0/525505627 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e98038070 con 0x7f3e9c0659f0 2024-07-03T15:42:37.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.583+0000 7f3e91ffb640 1 -- 172.21.15.28:0/525505627 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3e98009720 con 0x7f3e9c0659f0 2024-07-03T15:42:37.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.583+0000 7f3ea39b2640 1 -- 172.21.15.28:0/525505627 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e9c0fb800 con 0x7f3e9c0659f0 2024-07-03T15:42:37.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.583+0000 7f3e91ffb640 1 -- 172.21.15.28:0/525505627 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e9803dc50 con 0x7f3e9c0659f0 2024-07-03T15:42:37.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.583+0000 7f3ea39b2640 1 -- 172.21.15.28:0/525505627 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e9c114ff0 con 0x7f3e9c0659f0 2024-07-03T15:42:37.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.584+0000 7f3ea39b2640 1 -- 172.21.15.28:0/525505627 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e9c0fa6a0 con 0x7f3e9c0659f0 2024-07-03T15:42:37.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.585+0000 7f3e91ffb640 1 -- 172.21.15.28:0/525505627 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f3e98030020 con 0x7f3e9c0659f0 2024-07-03T15:42:37.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.585+0000 7f3e91ffb640 1 --2- 172.21.15.28:0/525505627 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3e88067190 0x7f3e88069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:37.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.585+0000 7f3e91ffb640 1 -- 172.21.15.28:0/525505627 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f3e980a55c0 con 0x7f3e9c0659f0 2024-07-03T15:42:37.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.585+0000 7f3ea0f4d640 1 --2- 172.21.15.28:0/525505627 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3e88067190 0x7f3e88069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:37.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.586+0000 7f3ea0f4d640 1 --2- 172.21.15.28:0/525505627 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3e88067190 0x7f3e88069640 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c004a70 tx=0x7f3e8c00b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:42:37.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.593+0000 7f3e91ffb640 1 -- 172.21.15.28:0/525505627 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3e980728e0 con 0x7f3e9c0659f0 2024-07-03T15:42:37.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.849+0000 7f3ea39b2640 1 -- 172.21.15.28:0/525505627 --> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e9c0f8510 con 0x7f3e88067190 2024-07-03T15:42:37.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.851+0000 7f3e91ffb640 1 -- 172.21.15.28:0/525505627 <== mgr.14210 v2:172.21.15.28:6800/421057178 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f3e9c0f8510 con 0x7f3e88067190 2024-07-03T15:42:37.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.853+0000 7f3ea39b2640 1 -- 172.21.15.28:0/525505627 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3e88067190 msgr2=0x7f3e88069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:37.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.853+0000 7f3ea39b2640 1 --2- 172.21.15.28:0/525505627 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3e88067190 0x7f3e88069640 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c004a70 tx=0x7f3e8c00b040 comp rx=0 tx=0).stop 2024-07-03T15:42:37.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.853+0000 7f3ea39b2640 1 -- 172.21.15.28:0/525505627 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e9c0659f0 msgr2=0x7f3e9c0fc9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:37.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.853+0000 7f3ea39b2640 1 --2- 172.21.15.28:0/525505627 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e9c0659f0 0x7f3e9c0fc9c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3e9802a5e0 tx=0x7f3e98009da0 comp rx=0 tx=0).stop 2024-07-03T15:42:37.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.853+0000 7f3ea39b2640 1 -- 172.21.15.28:0/525505627 shutdown_connections 2024-07-03T15:42:37.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.853+0000 7f3ea39b2640 1 --2- 172.21.15.28:0/525505627 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3e88067190 0x7f3e88069640 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:37.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.853+0000 7f3ea39b2640 1 --2- 172.21.15.28:0/525505627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e9c0ff4f0 0x7f3e9c0faeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:37.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.853+0000 7f3ea39b2640 1 --2- 172.21.15.28:0/525505627 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e9c0659f0 0x7f3e9c0fc9c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:37.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.853+0000 7f3ea39b2640 1 -- 172.21.15.28:0/525505627 >> 172.21.15.28:0/525505627 conn(0x7f3e9c0f6a20 msgr2=0x7f3e9c0f6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:42:37.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.853+0000 7f3ea39b2640 1 -- 172.21.15.28:0/525505627 shutdown_connections 2024-07-03T15:42:37.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.854+0000 7f3ea39b2640 1 -- 172.21.15.28:0/525505627 wait complete. 2024-07-03T15:42:37.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.953+0000 7efe5788e640 1 -- 172.21.15.28:0/3265194782 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe500ff490 msgr2=0x7efe50101870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:37.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.953+0000 7efe5788e640 1 --2- 172.21.15.28:0/3265194782 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe500ff490 0x7efe50101870 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7efe400048d0 tx=0x7efe40009da0 comp rx=0 tx=0).stop 2024-07-03T15:42:37.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.953+0000 7efe5788e640 1 -- 172.21.15.28:0/3265194782 shutdown_connections 2024-07-03T15:42:37.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.953+0000 7efe5788e640 1 --2- 172.21.15.28:0/3265194782 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe500ff490 0x7efe50101870 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:37.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.953+0000 7efe5788e640 1 --2- 172.21.15.28:0/3265194782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe50065a50 0x7efe500fef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:37.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.953+0000 7efe5788e640 1 -- 172.21.15.28:0/3265194782 >> 172.21.15.28:0/3265194782 conn(0x7efe500f6a20 msgr2=0x7efe500f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:42:37.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.954+0000 7efe5788e640 1 -- 172.21.15.28:0/3265194782 shutdown_connections 2024-07-03T15:42:37.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.954+0000 7efe5788e640 1 -- 172.21.15.28:0/3265194782 wait complete. 2024-07-03T15:42:37.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.954+0000 7efe5788e640 1 Processor -- start 2024-07-03T15:42:37.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.954+0000 7efe5788e640 1 -- start start 2024-07-03T15:42:37.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.955+0000 7efe5788e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe50065a50 0x7efe5010cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:37.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.955+0000 7efe5788e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe500ff490 0x7efe5010d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:37.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.955+0000 7efe5788e640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5010db20 con 0x7efe500ff490 2024-07-03T15:42:37.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.955+0000 7efe5788e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5010e2f0 con 0x7efe50065a50 2024-07-03T15:42:37.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.955+0000 7efe54e29640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe500ff490 0x7efe5010d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:37.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.955+0000 7efe5562a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe50065a50 0x7efe5010cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:37.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.955+0000 7efe5562a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe50065a50 0x7efe5010cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.28:49728/0 (socket says 172.21.15.28:49728) 2024-07-03T15:42:37.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.955+0000 7efe5562a640 1 -- 172.21.15.28:0/4080901677 learned_addr learned my addr 172.21.15.28:0/4080901677 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:42:37.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.955+0000 7efe5562a640 1 -- 172.21.15.28:0/4080901677 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe500ff490 msgr2=0x7efe5010d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:37.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.955+0000 7efe5562a640 1 --2- 172.21.15.28:0/4080901677 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe500ff490 0x7efe5010d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:37.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.955+0000 7efe5562a640 1 -- 172.21.15.28:0/4080901677 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe40005040 con 0x7efe50065a50 2024-07-03T15:42:37.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.955+0000 7efe5562a640 1 --2- 172.21.15.28:0/4080901677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe50065a50 0x7efe5010cf90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efe4c00eb70 tx=0x7efe4c00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:42:37.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.956+0000 7efe45ffb640 1 -- 172.21.15.28:0/4080901677 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe4c007c30 con 0x7efe50065a50 2024-07-03T15:42:37.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.956+0000 7efe45ffb640 1 -- 172.21.15.28:0/4080901677 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe4c00f040 con 0x7efe50065a50 2024-07-03T15:42:37.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.956+0000 7efe5788e640 1 -- 172.21.15.28:0/4080901677 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe5010e580 con 0x7efe50065a50 2024-07-03T15:42:37.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.956+0000 7efe45ffb640 1 -- 172.21.15.28:0/4080901677 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe4c00ccb0 con 0x7efe50065a50 2024-07-03T15:42:37.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.956+0000 7efe5788e640 1 -- 172.21.15.28:0/4080901677 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe5010eaf0 con 0x7efe50065a50 2024-07-03T15:42:37.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.957+0000 7efe5788e640 1 -- 172.21.15.28:0/4080901677 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe501056a0 con 0x7efe50065a50 2024-07-03T15:42:37.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.958+0000 7efe45ffb640 1 -- 172.21.15.28:0/4080901677 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7efe4c009110 con 0x7efe50065a50 2024-07-03T15:42:37.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.958+0000 7efe45ffb640 1 --2- 172.21.15.28:0/4080901677 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7efe3c067190 0x7efe3c069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:37.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.959+0000 7efe54e29640 1 --2- 172.21.15.28:0/4080901677 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7efe3c067190 0x7efe3c069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:37.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.959+0000 7efe45ffb640 1 -- 172.21.15.28:0/4080901677 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7efe4c052330 con 0x7efe50065a50 2024-07-03T15:42:37.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.959+0000 7efe54e29640 1 --2- 172.21.15.28:0/4080901677 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7efe3c067190 0x7efe3c069640 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efe40009f80 tx=0x7efe40035040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:42:37.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:37.964+0000 7efe45ffb640 1 -- 172.21.15.28:0/4080901677 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7efe4c051f60 con 0x7efe50065a50 2024-07-03T15:42:38.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.242+0000 7efe5788e640 1 -- 172.21.15.28:0/4080901677 --> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7efe5010ef00 con 0x7efe3c067190 2024-07-03T15:42:38.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.255+0000 7efe45ffb640 1 -- 172.21.15.28:0/4080901677 <== mgr.14210 v2:172.21.15.28:6800/421057178 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7efe5010ef00 con 0x7efe3c067190 2024-07-03T15:42:38.259 INFO:teuthology.orchestra.run.smithi028.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-03T15:42:38.259 INFO:teuthology.orchestra.run.smithi028.stdout:alertmanager.smithi028 smithi028 *:9093,9094 running (4m) 28s ago 6m 24.9M - 0.25.0 c8568f914cd2 b9a1a484a0a2 2024-07-03T15:42:38.259 INFO:teuthology.orchestra.run.smithi028.stdout:crash.smithi028 smithi028 running (6m) 28s ago 6m 6899k - 17.2.7-1047-gd4b063ab 4ca399d9e65d 535660e448f9 2024-07-03T15:42:38.259 INFO:teuthology.orchestra.run.smithi028.stdout:crash.smithi148 smithi148 running (5m) 35s ago 5m 6899k - 17.2.7-1047-gd4b063ab 4ca399d9e65d 4bb7675895e7 2024-07-03T15:42:38.259 INFO:teuthology.orchestra.run.smithi028.stdout:grafana.smithi028 smithi028 *:3000 running (4m) 28s ago 5m 88.5M - 9.4.7 954c08fa6188 07e0ae3c15aa 2024-07-03T15:42:38.259 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi028.rhpvve smithi028 running (53s) 28s ago 53s 15.6M - 17.2.7-1047-gd4b063ab 4ca399d9e65d 3ed5abd02cfb 2024-07-03T15:42:38.259 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi028.wmbxew smithi028 running (48s) 28s ago 47s 14.9M - 17.2.7-1047-gd4b063ab 4ca399d9e65d b3bf736998f9 2024-07-03T15:42:38.259 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi148.szomfl smithi148 running (45s) 35s ago 45s 12.9M - 17.2.7-1047-gd4b063ab 4ca399d9e65d d94f5c95fa8d 2024-07-03T15:42:38.260 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi148.xgpuld smithi148 running (51s) 35s ago 50s 10.7M - 17.2.7-1047-gd4b063ab 4ca399d9e65d fc70360a8de0 2024-07-03T15:42:38.260 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.smithi028.vtqrtz smithi028 *:9283 running (7m) 28s ago 7m 447M - 17.2.7-1047-gd4b063ab 4ca399d9e65d 0d26e74cb154 2024-07-03T15:42:38.260 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.smithi148.rjzsar smithi148 *:8443,9283 running (5m) 35s ago 5m 401M - 17.2.7-1047-gd4b063ab 4ca399d9e65d c637b3ae9fce 2024-07-03T15:42:38.260 INFO:teuthology.orchestra.run.smithi028.stdout:mon.smithi028 smithi028 running (7m) 28s ago 7m 43.2M 2048M 17.2.7-1047-gd4b063ab 4ca399d9e65d 7461f9bef4e0 2024-07-03T15:42:38.260 INFO:teuthology.orchestra.run.smithi028.stdout:mon.smithi148 smithi148 running (5m) 35s ago 5m 32.8M 2048M 17.2.7-1047-gd4b063ab 4ca399d9e65d 4c6c8ed2ab85 2024-07-03T15:42:38.260 INFO:teuthology.orchestra.run.smithi028.stdout:node-exporter.smithi028 smithi028 *:9100 running (6m) 28s ago 6m 16.0M - 1.5.0 0da6a335fe13 68318a0ca520 2024-07-03T15:42:38.260 INFO:teuthology.orchestra.run.smithi028.stdout:node-exporter.smithi148 smithi148 *:9100 running (5m) 35s ago 5m 17.6M - 1.5.0 0da6a335fe13 4994e60bd57c 2024-07-03T15:42:38.260 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0 smithi028 running (4m) 28s ago 4m 48.5M 4289M 17.2.7-1047-gd4b063ab 4ca399d9e65d b18089a439ba 2024-07-03T15:42:38.260 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1 smithi028 running (4m) 28s ago 4m 45.5M 4289M 17.2.7-1047-gd4b063ab 4ca399d9e65d 87d415dde231 2024-07-03T15:42:38.261 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2 smithi028 running (3m) 28s ago 3m 46.3M 4289M 17.2.7-1047-gd4b063ab 4ca399d9e65d b2cbb312212d 2024-07-03T15:42:38.261 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3 smithi148 running (3m) 35s ago 3m 44.4M 5312M 17.2.7-1047-gd4b063ab 4ca399d9e65d 5be69a27df31 2024-07-03T15:42:38.261 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4 smithi148 running (2m) 35s ago 2m 44.3M 5312M 17.2.7-1047-gd4b063ab 4ca399d9e65d 54f7c26c77ca 2024-07-03T15:42:38.261 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5 smithi148 running (2m) 35s ago 2m 41.4M 5312M 17.2.7-1047-gd4b063ab 4ca399d9e65d d185c6905384 2024-07-03T15:42:38.261 INFO:teuthology.orchestra.run.smithi028.stdout:prometheus.smithi028 smithi028 *:9095 running (4m) 28s ago 5m 54.2M - 2.43.0 a07b618ecd1d 97f838e118a3 2024-07-03T15:42:38.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.258+0000 7efe5788e640 1 -- 172.21.15.28:0/4080901677 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7efe3c067190 msgr2=0x7efe3c069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:38.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.258+0000 7efe5788e640 1 --2- 172.21.15.28:0/4080901677 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7efe3c067190 0x7efe3c069640 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efe40009f80 tx=0x7efe40035040 comp rx=0 tx=0).stop 2024-07-03T15:42:38.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.258+0000 7efe5788e640 1 -- 172.21.15.28:0/4080901677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe50065a50 msgr2=0x7efe5010cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:38.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.258+0000 7efe5788e640 1 --2- 172.21.15.28:0/4080901677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe50065a50 0x7efe5010cf90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efe4c00eb70 tx=0x7efe4c00b040 comp rx=0 tx=0).stop 2024-07-03T15:42:38.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.258+0000 7efe5788e640 1 -- 172.21.15.28:0/4080901677 shutdown_connections 2024-07-03T15:42:38.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.258+0000 7efe5788e640 1 --2- 172.21.15.28:0/4080901677 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7efe3c067190 0x7efe3c069640 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:38.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.258+0000 7efe5788e640 1 --2- 172.21.15.28:0/4080901677 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe500ff490 0x7efe5010d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:38.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.258+0000 7efe5788e640 1 --2- 172.21.15.28:0/4080901677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe50065a50 0x7efe5010cf90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:38.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.258+0000 7efe5788e640 1 -- 172.21.15.28:0/4080901677 >> 172.21.15.28:0/4080901677 conn(0x7efe500f6a20 msgr2=0x7efe500f6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:42:38.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.258+0000 7efe5788e640 1 -- 172.21.15.28:0/4080901677 shutdown_connections 2024-07-03T15:42:38.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.258+0000 7efe5788e640 1 -- 172.21.15.28:0/4080901677 wait complete. 2024-07-03T15:42:38.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.357+0000 7f1b676cf640 1 -- 172.21.15.28:0/2343183293 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b601018f0 msgr2=0x7f1b60103cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:38.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.357+0000 7f1b676cf640 1 --2- 172.21.15.28:0/2343183293 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b601018f0 0x7f1b60103cd0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1b500048d0 tx=0x7f1b50009da0 comp rx=0 tx=0).stop 2024-07-03T15:42:38.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.357+0000 7f1b676cf640 1 -- 172.21.15.28:0/2343183293 shutdown_connections 2024-07-03T15:42:38.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.357+0000 7f1b676cf640 1 --2- 172.21.15.28:0/2343183293 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b601018f0 0x7f1b60103cd0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:38.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.357+0000 7f1b676cf640 1 --2- 172.21.15.28:0/2343183293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b600659f0 0x7f1b601013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:38.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.357+0000 7f1b676cf640 1 -- 172.21.15.28:0/2343183293 >> 172.21.15.28:0/2343183293 conn(0x7f1b600f6a20 msgr2=0x7f1b600f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:42:38.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.357+0000 7f1b676cf640 1 -- 172.21.15.28:0/2343183293 shutdown_connections 2024-07-03T15:42:38.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.357+0000 7f1b676cf640 1 -- 172.21.15.28:0/2343183293 wait complete. 2024-07-03T15:42:38.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.358+0000 7f1b676cf640 1 Processor -- start 2024-07-03T15:42:38.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.358+0000 7f1b676cf640 1 -- start start 2024-07-03T15:42:38.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.358+0000 7f1b676cf640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b600659f0 0x7f1b6010f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:38.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.358+0000 7f1b676cf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b601018f0 0x7f1b6010f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:38.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.358+0000 7f1b676cf640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6010fcf0 con 0x7f1b600659f0 2024-07-03T15:42:38.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.358+0000 7f1b676cf640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b60110510 con 0x7f1b601018f0 2024-07-03T15:42:38.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.358+0000 7f1b6546b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b600659f0 0x7f1b6010f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:38.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.359+0000 7f1b64c6a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b601018f0 0x7f1b6010f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:38.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.359+0000 7f1b6546b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b600659f0 0x7f1b6010f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50974/0 (socket says 172.21.15.28:50974) 2024-07-03T15:42:38.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.359+0000 7f1b6546b640 1 -- 172.21.15.28:0/1578838812 learned_addr learned my addr 172.21.15.28:0/1578838812 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:42:38.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.359+0000 7f1b6546b640 1 -- 172.21.15.28:0/1578838812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b601018f0 msgr2=0x7f1b6010f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:38.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.359+0000 7f1b6546b640 1 --2- 172.21.15.28:0/1578838812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b601018f0 0x7f1b6010f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:38.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.359+0000 7f1b6546b640 1 -- 172.21.15.28:0/1578838812 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b50005040 con 0x7f1b600659f0 2024-07-03T15:42:38.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.359+0000 7f1b64c6a640 1 --2- 172.21.15.28:0/1578838812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b601018f0 0x7f1b6010f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-03T15:42:38.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.359+0000 7f1b6546b640 1 --2- 172.21.15.28:0/1578838812 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b600659f0 0x7f1b6010f160 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c00eb70 tx=0x7f1b5c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:42:38.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.359+0000 7f1b55ffb640 1 -- 172.21.15.28:0/1578838812 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b5c007c30 con 0x7f1b600659f0 2024-07-03T15:42:38.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.359+0000 7f1b55ffb640 1 -- 172.21.15.28:0/1578838812 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1b5c00f040 con 0x7f1b600659f0 2024-07-03T15:42:38.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.360+0000 7f1b55ffb640 1 -- 172.21.15.28:0/1578838812 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b5c009730 con 0x7f1b600659f0 2024-07-03T15:42:38.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.360+0000 7f1b676cf640 1 -- 172.21.15.28:0/1578838812 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b601107a0 con 0x7f1b600659f0 2024-07-03T15:42:38.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.360+0000 7f1b676cf640 1 -- 172.21.15.28:0/1578838812 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b60110cf0 con 0x7f1b600659f0 2024-07-03T15:42:38.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.361+0000 7f1b676cf640 1 -- 172.21.15.28:0/1578838812 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b600fa6a0 con 0x7f1b600659f0 2024-07-03T15:42:38.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.362+0000 7f1b55ffb640 1 -- 172.21.15.28:0/1578838812 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f1b5c00c710 con 0x7f1b600659f0 2024-07-03T15:42:38.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.362+0000 7f1b55ffb640 1 --2- 172.21.15.28:0/1578838812 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f1b4c067190 0x7f1b4c069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:38.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.362+0000 7f1b55ffb640 1 -- 172.21.15.28:0/1578838812 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f1b5c084da0 con 0x7f1b600659f0 2024-07-03T15:42:38.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.363+0000 7f1b64c6a640 1 --2- 172.21.15.28:0/1578838812 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f1b4c067190 0x7f1b4c069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:38.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.363+0000 7f1b64c6a640 1 --2- 172.21.15.28:0/1578838812 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f1b4c067190 0x7f1b4c069640 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1b6010ef10 tx=0x7f1b50035040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:42:38.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.370+0000 7f1b55ffb640 1 -- 172.21.15.28:0/1578838812 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f1b5c052040 con 0x7f1b600659f0 2024-07-03T15:42:38.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:38 smithi028 ceph-mon[29148]: from='client.14538 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:42:38.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.700+0000 7f1b676cf640 1 -- 172.21.15.28:0/1578838812 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1b6004b530 con 0x7f1b600659f0 2024-07-03T15:42:38.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.700+0000 7f1b55ffb640 1 -- 172.21.15.28:0/1578838812 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+643 (secure 0 0 0) 0x7f1b5c055660 con 0x7f1b600659f0 2024-07-03T15:42:38.703 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-07-03T15:42:38.704 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": { 2024-07-03T15:42:38.704 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 2 2024-07-03T15:42:38.704 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:42:38.704 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": { 2024-07-03T15:42:38.704 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 2 2024-07-03T15:42:38.704 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:42:38.704 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": { 2024-07-03T15:42:38.704 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 6 2024-07-03T15:42:38.704 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:42:38.704 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": { 2024-07-03T15:42:38.704 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 4 2024-07-03T15:42:38.705 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:42:38.705 INFO:teuthology.orchestra.run.smithi028.stdout: "overall": { 2024-07-03T15:42:38.705 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 14 2024-07-03T15:42:38.705 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-07-03T15:42:38.705 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-07-03T15:42:38.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.702+0000 7f1b676cf640 1 -- 172.21.15.28:0/1578838812 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f1b4c067190 msgr2=0x7f1b4c069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:38.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.702+0000 7f1b676cf640 1 --2- 172.21.15.28:0/1578838812 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f1b4c067190 0x7f1b4c069640 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1b6010ef10 tx=0x7f1b50035040 comp rx=0 tx=0).stop 2024-07-03T15:42:38.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.702+0000 7f1b676cf640 1 -- 172.21.15.28:0/1578838812 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b600659f0 msgr2=0x7f1b6010f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:38.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.702+0000 7f1b676cf640 1 --2- 172.21.15.28:0/1578838812 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b600659f0 0x7f1b6010f160 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c00eb70 tx=0x7f1b5c00b040 comp rx=0 tx=0).stop 2024-07-03T15:42:38.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.702+0000 7f1b676cf640 1 -- 172.21.15.28:0/1578838812 shutdown_connections 2024-07-03T15:42:38.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.703+0000 7f1b676cf640 1 --2- 172.21.15.28:0/1578838812 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f1b4c067190 0x7f1b4c069640 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:38.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.703+0000 7f1b676cf640 1 --2- 172.21.15.28:0/1578838812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b601018f0 0x7f1b6010f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:38.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.703+0000 7f1b676cf640 1 --2- 172.21.15.28:0/1578838812 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b600659f0 0x7f1b6010f160 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:38.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.703+0000 7f1b676cf640 1 -- 172.21.15.28:0/1578838812 >> 172.21.15.28:0/1578838812 conn(0x7f1b600f6a20 msgr2=0x7f1b600f6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:42:38.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.703+0000 7f1b676cf640 1 -- 172.21.15.28:0/1578838812 shutdown_connections 2024-07-03T15:42:38.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.703+0000 7f1b676cf640 1 -- 172.21.15.28:0/1578838812 wait complete. 2024-07-03T15:42:38.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:38 smithi148 ceph-mon[36056]: from='client.14538 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:42:38.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.805+0000 7fe34656c640 1 -- 172.21.15.28:0/1996926113 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3400fe390 msgr2=0x7fe3400fa4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:38.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.805+0000 7fe34656c640 1 --2- 172.21.15.28:0/1996926113 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3400fe390 0x7fe3400fa4f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe3340048d0 tx=0x7fe334009da0 comp rx=0 tx=0).stop 2024-07-03T15:42:38.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.805+0000 7fe34656c640 1 -- 172.21.15.28:0/1996926113 shutdown_connections 2024-07-03T15:42:38.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.805+0000 7fe34656c640 1 --2- 172.21.15.28:0/1996926113 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3400fe390 0x7fe3400fa4f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:38.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.805+0000 7fe34656c640 1 --2- 172.21.15.28:0/1996926113 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3400fda40 0x7fe3400fde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:38.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.805+0000 7fe34656c640 1 -- 172.21.15.28:0/1996926113 >> 172.21.15.28:0/1996926113 conn(0x7fe3400f61a0 msgr2=0x7fe3400f85b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:42:38.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.805+0000 7fe34656c640 1 -- 172.21.15.28:0/1996926113 shutdown_connections 2024-07-03T15:42:38.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.805+0000 7fe34656c640 1 -- 172.21.15.28:0/1996926113 wait complete. 2024-07-03T15:42:38.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.806+0000 7fe34656c640 1 Processor -- start 2024-07-03T15:42:38.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.806+0000 7fe34656c640 1 -- start start 2024-07-03T15:42:38.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.806+0000 7fe34656c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3400fda40 0x7fe34010cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:38.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.806+0000 7fe34656c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3400fe390 0x7fe34010d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:38.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.806+0000 7fe34656c640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34010db60 con 0x7fe3400fda40 2024-07-03T15:42:38.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.806+0000 7fe33ffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3400fda40 0x7fe34010cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:38.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.806+0000 7fe34656c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34010e380 con 0x7fe3400fe390 2024-07-03T15:42:38.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.806+0000 7fe33f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3400fe390 0x7fe34010d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:38.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.806+0000 7fe33ffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3400fda40 0x7fe34010cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50994/0 (socket says 172.21.15.28:50994) 2024-07-03T15:42:38.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.806+0000 7fe33ffff640 1 -- 172.21.15.28:0/1029521708 learned_addr learned my addr 172.21.15.28:0/1029521708 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:42:38.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.807+0000 7fe33f7fe640 1 -- 172.21.15.28:0/1029521708 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3400fda40 msgr2=0x7fe34010cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:38.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.807+0000 7fe33f7fe640 1 --2- 172.21.15.28:0/1029521708 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3400fda40 0x7fe34010cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:38.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.807+0000 7fe33f7fe640 1 -- 172.21.15.28:0/1029521708 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe334005040 con 0x7fe3400fe390 2024-07-03T15:42:38.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.807+0000 7fe33f7fe640 1 --2- 172.21.15.28:0/1029521708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3400fe390 0x7fe34010d510 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe33402a530 tx=0x7fe334009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:42:38.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.807+0000 7fe33cff9640 1 -- 172.21.15.28:0/1029521708 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe334038070 con 0x7fe3400fe390 2024-07-03T15:42:38.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.807+0000 7fe33cff9640 1 -- 172.21.15.28:0/1029521708 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe3340092b0 con 0x7fe3400fe390 2024-07-03T15:42:38.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.807+0000 7fe34656c640 1 -- 172.21.15.28:0/1029521708 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe34010e5b0 con 0x7fe3400fe390 2024-07-03T15:42:38.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.807+0000 7fe33cff9640 1 -- 172.21.15.28:0/1029521708 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe334033660 con 0x7fe3400fe390 2024-07-03T15:42:38.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.807+0000 7fe34656c640 1 -- 172.21.15.28:0/1029521708 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe34010eac0 con 0x7fe3400fe390 2024-07-03T15:42:38.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.809+0000 7fe33cff9640 1 -- 172.21.15.28:0/1029521708 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7fe334030020 con 0x7fe3400fe390 2024-07-03T15:42:38.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.809+0000 7fe34656c640 1 -- 172.21.15.28:0/1029521708 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3400f9e20 con 0x7fe3400fe390 2024-07-03T15:42:38.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.809+0000 7fe33cff9640 1 --2- 172.21.15.28:0/1029521708 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fe328067190 0x7fe328069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:38.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.809+0000 7fe33cff9640 1 -- 172.21.15.28:0/1029521708 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fe3340a47e0 con 0x7fe3400fe390 2024-07-03T15:42:38.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.809+0000 7fe33ffff640 1 --2- 172.21.15.28:0/1029521708 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fe328067190 0x7fe328069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:38.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.810+0000 7fe33ffff640 1 --2- 172.21.15.28:0/1029521708 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fe328067190 0x7fe328069640 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe330004a50 tx=0x7fe33000b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:42:38.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:38.813+0000 7fe33cff9640 1 -- 172.21.15.28:0/1029521708 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe334071bc0 con 0x7fe3400fe390 2024-07-03T15:42:39.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.058+0000 7fe34656c640 1 -- 172.21.15.28:0/1029521708 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe34010ed30 con 0x7fe3400fe390 2024-07-03T15:42:39.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.059+0000 7fe33cff9640 1 -- 172.21.15.28:0/1029521708 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1838 (secure 0 0 0) 0x7fe334041070 con 0x7fe3400fe390 2024-07-03T15:42:39.062 INFO:teuthology.orchestra.run.smithi028.stdout:e17 2024-07-03T15:42:39.062 INFO:teuthology.orchestra.run.smithi028.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-03T15:42:39.062 INFO:teuthology.orchestra.run.smithi028.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:42:39.063 INFO:teuthology.orchestra.run.smithi028.stdout:legacy client fscid: 1 2024-07-03T15:42:39.063 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:42:39.063 INFO:teuthology.orchestra.run.smithi028.stdout:Filesystem 'cephfs' (1) 2024-07-03T15:42:39.063 INFO:teuthology.orchestra.run.smithi028.stdout:fs_name cephfs 2024-07-03T15:42:39.063 INFO:teuthology.orchestra.run.smithi028.stdout:epoch 16 2024-07-03T15:42:39.063 INFO:teuthology.orchestra.run.smithi028.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-03T15:42:39.063 INFO:teuthology.orchestra.run.smithi028.stdout:created 2024-07-03T15:41:40.656298+0000 2024-07-03T15:42:39.063 INFO:teuthology.orchestra.run.smithi028.stdout:modified 2024-07-03T15:41:56.454984+0000 2024-07-03T15:42:39.063 INFO:teuthology.orchestra.run.smithi028.stdout:tableserver 0 2024-07-03T15:42:39.063 INFO:teuthology.orchestra.run.smithi028.stdout:root 0 2024-07-03T15:42:39.063 INFO:teuthology.orchestra.run.smithi028.stdout:session_timeout 60 2024-07-03T15:42:39.063 INFO:teuthology.orchestra.run.smithi028.stdout:session_autoclose 300 2024-07-03T15:42:39.063 INFO:teuthology.orchestra.run.smithi028.stdout:max_file_size 1099511627776 2024-07-03T15:42:39.063 INFO:teuthology.orchestra.run.smithi028.stdout:required_client_features {} 2024-07-03T15:42:39.064 INFO:teuthology.orchestra.run.smithi028.stdout:last_failure 0 2024-07-03T15:42:39.064 INFO:teuthology.orchestra.run.smithi028.stdout:last_failure_osd_epoch 0 2024-07-03T15:42:39.064 INFO:teuthology.orchestra.run.smithi028.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:42:39.064 INFO:teuthology.orchestra.run.smithi028.stdout:max_mds 1 2024-07-03T15:42:39.064 INFO:teuthology.orchestra.run.smithi028.stdout:in 0 2024-07-03T15:42:39.064 INFO:teuthology.orchestra.run.smithi028.stdout:up {0=14450} 2024-07-03T15:42:39.064 INFO:teuthology.orchestra.run.smithi028.stdout:failed 2024-07-03T15:42:39.064 INFO:teuthology.orchestra.run.smithi028.stdout:damaged 2024-07-03T15:42:39.064 INFO:teuthology.orchestra.run.smithi028.stdout:stopped 2024-07-03T15:42:39.064 INFO:teuthology.orchestra.run.smithi028.stdout:data_pools [3] 2024-07-03T15:42:39.064 INFO:teuthology.orchestra.run.smithi028.stdout:metadata_pool 2 2024-07-03T15:42:39.064 INFO:teuthology.orchestra.run.smithi028.stdout:inline_data disabled 2024-07-03T15:42:39.064 INFO:teuthology.orchestra.run.smithi028.stdout:balancer 2024-07-03T15:42:39.065 INFO:teuthology.orchestra.run.smithi028.stdout:standby_count_wanted 1 2024-07-03T15:42:39.065 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi028.rhpvve{0:14450} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.28:6826/1613936042,v1:172.21.15.28:6827/1613936042] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:42:39.065 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi028.wmbxew{0:14460} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.28:6828/1730336821,v1:172.21.15.28:6829/1730336821] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:42:39.065 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:42:39.065 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:42:39.065 INFO:teuthology.orchestra.run.smithi028.stdout:Standby daemons: 2024-07-03T15:42:39.065 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:42:39.065 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi148.szomfl{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/4171231858,v1:172.21.15.148:6827/4171231858] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:42:39.065 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi148.xgpuld{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/2347536398,v1:172.21.15.148:6825/2347536398] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:42:39.066 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 17 2024-07-03T15:42:39.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.061+0000 7fe34656c640 1 -- 172.21.15.28:0/1029521708 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fe328067190 msgr2=0x7fe328069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:39.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.061+0000 7fe34656c640 1 --2- 172.21.15.28:0/1029521708 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fe328067190 0x7fe328069640 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe330004a50 tx=0x7fe33000b040 comp rx=0 tx=0).stop 2024-07-03T15:42:39.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.061+0000 7fe34656c640 1 -- 172.21.15.28:0/1029521708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3400fe390 msgr2=0x7fe34010d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:39.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.061+0000 7fe34656c640 1 --2- 172.21.15.28:0/1029521708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3400fe390 0x7fe34010d510 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe33402a530 tx=0x7fe334009da0 comp rx=0 tx=0).stop 2024-07-03T15:42:39.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.062+0000 7fe34656c640 1 -- 172.21.15.28:0/1029521708 shutdown_connections 2024-07-03T15:42:39.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.062+0000 7fe34656c640 1 --2- 172.21.15.28:0/1029521708 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fe328067190 0x7fe328069640 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:39.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.062+0000 7fe34656c640 1 --2- 172.21.15.28:0/1029521708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3400fe390 0x7fe34010d510 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:39.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.062+0000 7fe34656c640 1 --2- 172.21.15.28:0/1029521708 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3400fda40 0x7fe34010cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:39.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.062+0000 7fe34656c640 1 -- 172.21.15.28:0/1029521708 >> 172.21.15.28:0/1029521708 conn(0x7fe3400f61a0 msgr2=0x7fe3400f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:42:39.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.062+0000 7fe34656c640 1 -- 172.21.15.28:0/1029521708 shutdown_connections 2024-07-03T15:42:39.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.062+0000 7fe34656c640 1 -- 172.21.15.28:0/1029521708 wait complete. 2024-07-03T15:42:39.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.155+0000 7f730eaa2640 1 -- 172.21.15.28:0/1271160052 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73080fc510 msgr2=0x7f73080fc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:39.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.155+0000 7f730eaa2640 1 --2- 172.21.15.28:0/1271160052 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73080fc510 0x7f73080fc980 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f72f40048d0 tx=0x7f72f4009da0 comp rx=0 tx=0).stop 2024-07-03T15:42:39.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.155+0000 7f730eaa2640 1 -- 172.21.15.28:0/1271160052 shutdown_connections 2024-07-03T15:42:39.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.155+0000 7f730eaa2640 1 --2- 172.21.15.28:0/1271160052 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73080fc510 0x7f73080fc980 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:39.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.155+0000 7f730eaa2640 1 --2- 172.21.15.28:0/1271160052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73080fb240 0x7f73080fb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:39.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.155+0000 7f730eaa2640 1 -- 172.21.15.28:0/1271160052 >> 172.21.15.28:0/1271160052 conn(0x7f73080f6a20 msgr2=0x7f73080f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:42:39.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.155+0000 7f730eaa2640 1 -- 172.21.15.28:0/1271160052 shutdown_connections 2024-07-03T15:42:39.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.155+0000 7f730eaa2640 1 -- 172.21.15.28:0/1271160052 wait complete. 2024-07-03T15:42:39.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.156+0000 7f730eaa2640 1 Processor -- start 2024-07-03T15:42:39.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.156+0000 7f730eaa2640 1 -- start start 2024-07-03T15:42:39.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.157+0000 7f730eaa2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73080fb240 0x7f730810fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:39.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.157+0000 7f730eaa2640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73080fc510 0x7f7308112210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:39.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.157+0000 7f730eaa2640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7308065a70 con 0x7f73080fc510 2024-07-03T15:42:39.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.157+0000 7f730c83e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73080fb240 0x7f730810fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:39.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.157+0000 7f730c83e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73080fb240 0x7f730810fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.28:49776/0 (socket says 172.21.15.28:49776) 2024-07-03T15:42:39.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.157+0000 7f730c83e640 1 -- 172.21.15.28:0/1900688386 learned_addr learned my addr 172.21.15.28:0/1900688386 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:42:39.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.157+0000 7f7307fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73080fc510 0x7f7308112210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:39.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.157+0000 7f730eaa2640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7308065be0 con 0x7f73080fb240 2024-07-03T15:42:39.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.158+0000 7f7307fff640 1 -- 172.21.15.28:0/1900688386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73080fb240 msgr2=0x7f730810fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:39.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.158+0000 7f7307fff640 1 --2- 172.21.15.28:0/1900688386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73080fb240 0x7f730810fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:39.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.158+0000 7f7307fff640 1 -- 172.21.15.28:0/1900688386 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f4005040 con 0x7f73080fc510 2024-07-03T15:42:39.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.158+0000 7f730c83e640 1 --2- 172.21.15.28:0/1900688386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73080fb240 0x7f730810fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-03T15:42:39.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.158+0000 7f7307fff640 1 --2- 172.21.15.28:0/1900688386 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73080fc510 0x7f7308112210 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f72f40048a0 tx=0x7f72f4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:42:39.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.158+0000 7f73057fa640 1 -- 172.21.15.28:0/1900688386 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72f4038070 con 0x7f73080fc510 2024-07-03T15:42:39.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.158+0000 7f73057fa640 1 -- 172.21.15.28:0/1900688386 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f72f40092b0 con 0x7f73080fc510 2024-07-03T15:42:39.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.158+0000 7f730eaa2640 1 -- 172.21.15.28:0/1900688386 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7308112750 con 0x7f73080fc510 2024-07-03T15:42:39.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.158+0000 7f73057fa640 1 -- 172.21.15.28:0/1900688386 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72f4033660 con 0x7f73080fc510 2024-07-03T15:42:39.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.158+0000 7f730eaa2640 1 -- 172.21.15.28:0/1900688386 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7308112c60 con 0x7f73080fc510 2024-07-03T15:42:39.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.160+0000 7f730eaa2640 1 -- 172.21.15.28:0/1900688386 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73080fa6a0 con 0x7f73080fc510 2024-07-03T15:42:39.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.160+0000 7f73057fa640 1 -- 172.21.15.28:0/1900688386 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f72f4030020 con 0x7f73080fc510 2024-07-03T15:42:39.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.161+0000 7f73057fa640 1 --2- 172.21.15.28:0/1900688386 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f72f8067190 0x7f72f8069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:39.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.161+0000 7f73057fa640 1 -- 172.21.15.28:0/1900688386 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f72f40a47e0 con 0x7f73080fc510 2024-07-03T15:42:39.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.161+0000 7f730c83e640 1 --2- 172.21.15.28:0/1900688386 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f72f8067190 0x7f72f8069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:39.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.161+0000 7f730c83e640 1 --2- 172.21.15.28:0/1900688386 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f72f8067190 0x7f72f8069640 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7308065760 tx=0x7f72fc00b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:42:39.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.166+0000 7f73057fa640 1 -- 172.21.15.28:0/1900688386 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f72f4071bb0 con 0x7f73080fc510 2024-07-03T15:42:39.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:39 smithi028 ceph-mon[29148]: from='client.14542 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:42:39.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:39 smithi028 ceph-mon[29148]: from='client.24333 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:42:39.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:39 smithi028 ceph-mon[29148]: pgmap v202: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-07-03T15:42:39.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:39 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/1578838812' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:42:39.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:39 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/1029521708' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-03T15:42:39.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.401+0000 7f730eaa2640 1 -- 172.21.15.28:0/1900688386 --> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7308100880 con 0x7f72f8067190 2024-07-03T15:42:39.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.402+0000 7f73057fa640 1 -- 172.21.15.28:0/1900688386 <== mgr.14210 v2:172.21.15.28:6800/421057178 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f7308100880 con 0x7f72f8067190 2024-07-03T15:42:39.406 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-07-03T15:42:39.406 INFO:teuthology.orchestra.run.smithi028.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:c00506dc10003928f8cb40769168571f611cbd67", 2024-07-03T15:42:39.406 INFO:teuthology.orchestra.run.smithi028.stdout: "in_progress": true, 2024-07-03T15:42:39.406 INFO:teuthology.orchestra.run.smithi028.stdout: "which": "Upgrading all daemon types on all hosts", 2024-07-03T15:42:39.406 INFO:teuthology.orchestra.run.smithi028.stdout: "services_complete": [], 2024-07-03T15:42:39.406 INFO:teuthology.orchestra.run.smithi028.stdout: "progress": "", 2024-07-03T15:42:39.406 INFO:teuthology.orchestra.run.smithi028.stdout: "message": "Doing first pull of quay.ceph.io/ceph-ci/ceph:c00506dc10003928f8cb40769168571f611cbd67 image", 2024-07-03T15:42:39.406 INFO:teuthology.orchestra.run.smithi028.stdout: "is_paused": false 2024-07-03T15:42:39.406 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-07-03T15:42:39.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.404+0000 7f730eaa2640 1 -- 172.21.15.28:0/1900688386 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f72f8067190 msgr2=0x7f72f8069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:39.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.404+0000 7f730eaa2640 1 --2- 172.21.15.28:0/1900688386 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f72f8067190 0x7f72f8069640 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7308065760 tx=0x7f72fc00b040 comp rx=0 tx=0).stop 2024-07-03T15:42:39.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.404+0000 7f730eaa2640 1 -- 172.21.15.28:0/1900688386 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73080fc510 msgr2=0x7f7308112210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:39.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.404+0000 7f730eaa2640 1 --2- 172.21.15.28:0/1900688386 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73080fc510 0x7f7308112210 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f72f40048a0 tx=0x7f72f4009da0 comp rx=0 tx=0).stop 2024-07-03T15:42:39.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.404+0000 7f730eaa2640 1 -- 172.21.15.28:0/1900688386 shutdown_connections 2024-07-03T15:42:39.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.405+0000 7f730eaa2640 1 --2- 172.21.15.28:0/1900688386 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f72f8067190 0x7f72f8069640 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:39.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.405+0000 7f730eaa2640 1 --2- 172.21.15.28:0/1900688386 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73080fc510 0x7f7308112210 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:39.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.405+0000 7f730eaa2640 1 --2- 172.21.15.28:0/1900688386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73080fb240 0x7f730810fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:39.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.405+0000 7f730eaa2640 1 -- 172.21.15.28:0/1900688386 >> 172.21.15.28:0/1900688386 conn(0x7f73080f6a20 msgr2=0x7f73080f8140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:42:39.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.405+0000 7f730eaa2640 1 -- 172.21.15.28:0/1900688386 shutdown_connections 2024-07-03T15:42:39.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.405+0000 7f730eaa2640 1 -- 172.21.15.28:0/1900688386 wait complete. 2024-07-03T15:42:39.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.515+0000 7f75c67c1640 1 -- 172.21.15.28:0/1770543775 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75c00fc330 msgr2=0x7f75c00fc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:39.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.515+0000 7f75c67c1640 1 --2- 172.21.15.28:0/1770543775 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75c00fc330 0x7f75c00fc740 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f75b00048d0 tx=0x7f75b0009da0 comp rx=0 tx=0).stop 2024-07-03T15:42:39.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.516+0000 7f75c67c1640 1 -- 172.21.15.28:0/1770543775 shutdown_connections 2024-07-03T15:42:39.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.516+0000 7f75c67c1640 1 --2- 172.21.15.28:0/1770543775 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75c00fd600 0x7f75c00fda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:39.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.516+0000 7f75c67c1640 1 --2- 172.21.15.28:0/1770543775 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75c00fc330 0x7f75c00fc740 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:39.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.516+0000 7f75c67c1640 1 -- 172.21.15.28:0/1770543775 >> 172.21.15.28:0/1770543775 conn(0x7f75c00f7af0 msgr2=0x7f75c00f9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:42:39.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.516+0000 7f75c67c1640 1 -- 172.21.15.28:0/1770543775 shutdown_connections 2024-07-03T15:42:39.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.516+0000 7f75c67c1640 1 -- 172.21.15.28:0/1770543775 wait complete. 2024-07-03T15:42:39.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.516+0000 7f75c67c1640 1 Processor -- start 2024-07-03T15:42:39.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.517+0000 7f75c67c1640 1 -- start start 2024-07-03T15:42:39.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.517+0000 7f75c67c1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75c00fc330 0x7f75c010cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:39.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.517+0000 7f75c67c1640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75c00fd600 0x7f75c010d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:39.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.517+0000 7f75c67c1640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c010dad0 con 0x7f75c00fd600 2024-07-03T15:42:39.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.517+0000 7f75c67c1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c010e2f0 con 0x7f75c00fc330 2024-07-03T15:42:39.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.517+0000 7f75bffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75c00fc330 0x7f75c010cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:39.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.517+0000 7f75bf7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75c00fd600 0x7f75c010d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:39.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.517+0000 7f75bffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75c00fc330 0x7f75c010cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.28:49786/0 (socket says 172.21.15.28:49786) 2024-07-03T15:42:39.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.517+0000 7f75bffff640 1 -- 172.21.15.28:0/553249500 learned_addr learned my addr 172.21.15.28:0/553249500 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:42:39.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.517+0000 7f75bf7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75c00fd600 0x7f75c010d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51040/0 (socket says 172.21.15.28:51040) 2024-07-03T15:42:39.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.517+0000 7f75bffff640 1 -- 172.21.15.28:0/553249500 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75c00fd600 msgr2=0x7f75c010d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:39.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.517+0000 7f75bffff640 1 --2- 172.21.15.28:0/553249500 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75c00fd600 0x7f75c010d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:39.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.517+0000 7f75bffff640 1 -- 172.21.15.28:0/553249500 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b0005040 con 0x7f75c00fc330 2024-07-03T15:42:39.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.518+0000 7f75bffff640 1 --2- 172.21.15.28:0/553249500 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75c00fc330 0x7f75c010cf40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f75b0009f80 tx=0x7f75b0009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:42:39.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.518+0000 7f75bcff9640 1 -- 172.21.15.28:0/553249500 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75b0038070 con 0x7f75c00fc330 2024-07-03T15:42:39.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.518+0000 7f75bcff9640 1 -- 172.21.15.28:0/553249500 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f75b0032970 con 0x7f75c00fc330 2024-07-03T15:42:39.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.518+0000 7f75bcff9640 1 -- 172.21.15.28:0/553249500 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75b0033860 con 0x7f75c00fc330 2024-07-03T15:42:39.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.518+0000 7f75c67c1640 1 -- 172.21.15.28:0/553249500 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75c010e520 con 0x7f75c00fc330 2024-07-03T15:42:39.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.519+0000 7f75c67c1640 1 -- 172.21.15.28:0/553249500 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75c010e9e0 con 0x7f75c00fc330 2024-07-03T15:42:39.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.520+0000 7f75c67c1640 1 -- 172.21.15.28:0/553249500 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75c00fb790 con 0x7f75c00fc330 2024-07-03T15:42:39.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.520+0000 7f75bcff9640 1 -- 172.21.15.28:0/553249500 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f75b0030020 con 0x7f75c00fc330 2024-07-03T15:42:39.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.521+0000 7f75bcff9640 1 --2- 172.21.15.28:0/553249500 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f75a8067190 0x7f75a8069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:42:39.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.521+0000 7f75bcff9640 1 -- 172.21.15.28:0/553249500 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f75b00a48f0 con 0x7f75c00fc330 2024-07-03T15:42:39.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.521+0000 7f75bf7fe640 1 --2- 172.21.15.28:0/553249500 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f75a8067190 0x7f75a8069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:42:39.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.521+0000 7f75bf7fe640 1 --2- 172.21.15.28:0/553249500 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f75a8067190 0x7f75a8069640 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f75c010ccf0 tx=0x7f75b4011040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:42:39.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.525+0000 7f75bcff9640 1 -- 172.21.15.28:0/553249500 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f75b0071cc0 con 0x7f75c00fc330 2024-07-03T15:42:39.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:39 smithi148 ceph-mon[36056]: from='client.14542 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:42:39.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:39 smithi148 ceph-mon[36056]: from='client.24333 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:42:39.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:39 smithi148 ceph-mon[36056]: pgmap v202: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-07-03T15:42:39.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:39 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/1578838812' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:42:39.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:39 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/1029521708' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-03T15:42:39.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.826+0000 7f75c67c1640 1 -- 172.21.15.28:0/553249500 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f75c004b530 con 0x7f75c00fc330 2024-07-03T15:42:39.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.826+0000 7f75bcff9640 1 -- 172.21.15.28:0/553249500 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f75b0041090 con 0x7f75c00fc330 2024-07-03T15:42:39.830 INFO:teuthology.orchestra.run.smithi028.stdout:HEALTH_OK 2024-07-03T15:42:39.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.829+0000 7f75c67c1640 1 -- 172.21.15.28:0/553249500 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f75a8067190 msgr2=0x7f75a8069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:39.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.829+0000 7f75c67c1640 1 --2- 172.21.15.28:0/553249500 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f75a8067190 0x7f75a8069640 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f75c010ccf0 tx=0x7f75b4011040 comp rx=0 tx=0).stop 2024-07-03T15:42:39.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.829+0000 7f75c67c1640 1 -- 172.21.15.28:0/553249500 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75c00fc330 msgr2=0x7f75c010cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:42:39.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.829+0000 7f75c67c1640 1 --2- 172.21.15.28:0/553249500 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75c00fc330 0x7f75c010cf40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f75b0009f80 tx=0x7f75b0009da0 comp rx=0 tx=0).stop 2024-07-03T15:42:39.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.829+0000 7f75c67c1640 1 -- 172.21.15.28:0/553249500 shutdown_connections 2024-07-03T15:42:39.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.829+0000 7f75c67c1640 1 --2- 172.21.15.28:0/553249500 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f75a8067190 0x7f75a8069640 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:39.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.829+0000 7f75c67c1640 1 --2- 172.21.15.28:0/553249500 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75c00fd600 0x7f75c010d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:39.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.829+0000 7f75c67c1640 1 --2- 172.21.15.28:0/553249500 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75c00fc330 0x7f75c010cf40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:42:39.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.829+0000 7f75c67c1640 1 -- 172.21.15.28:0/553249500 >> 172.21.15.28:0/553249500 conn(0x7f75c00f7af0 msgr2=0x7f75c00f91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:42:39.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.829+0000 7f75c67c1640 1 -- 172.21.15.28:0/553249500 shutdown_connections 2024-07-03T15:42:39.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:42:39.829+0000 7f75c67c1640 1 -- 172.21.15.28:0/553249500 wait complete. 2024-07-03T15:42:40.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:40 smithi028 ceph-mon[29148]: from='client.14558 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:42:40.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:40 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/553249500' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-03T15:42:40.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:40 smithi148 ceph-mon[36056]: from='client.14558 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:42:40.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:40 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/553249500' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-03T15:42:41.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:41 smithi028 ceph-mon[29148]: pgmap v203: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:42:41.731 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:41 smithi148 ceph-mon[36056]: pgmap v203: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:42:43.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:43 smithi148 ceph-mon[36056]: pgmap v204: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:42:43.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:43 smithi028 ceph-mon[29148]: pgmap v204: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:42:45.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:45 smithi148 ceph-mon[36056]: pgmap v205: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:42:45.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:45 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:42:45.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:45 smithi028 ceph-mon[29148]: pgmap v205: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:42:45.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:45 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:42:47.461 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:47 smithi148 ceph-mon[36056]: pgmap v206: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:42:47.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:47 smithi028 ceph-mon[29148]: pgmap v206: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:42:49.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:49 smithi148 ceph-mon[36056]: pgmap v207: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:42:49.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:49 smithi028 ceph-mon[29148]: pgmap v207: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:42:51.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:51 smithi148 ceph-mon[36056]: pgmap v208: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:42:51.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:51 smithi028 ceph-mon[29148]: pgmap v208: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:42:53.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:53 smithi148 ceph-mon[36056]: pgmap v209: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:42:53.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:53 smithi028 ceph-mon[29148]: pgmap v209: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:42:55.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:55 smithi148 ceph-mon[36056]: pgmap v210: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:42:55.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:55 smithi028 ceph-mon[29148]: pgmap v210: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:42:57.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:57 smithi148 ceph-mon[36056]: pgmap v211: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 426 B/s wr, 2 op/s 2024-07-03T15:42:57.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:57 smithi028 ceph-mon[29148]: pgmap v211: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 426 B/s wr, 2 op/s 2024-07-03T15:42:59.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:42:59 smithi148 ceph-mon[36056]: pgmap v212: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-07-03T15:42:59.753 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:42:59 smithi028 ceph-mon[29148]: pgmap v212: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-07-03T15:43:00.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:00 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:43:00.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:00 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:43:01.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:01 smithi148 ceph-mon[36056]: pgmap v213: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-07-03T15:43:01.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:01 smithi028 ceph-mon[29148]: pgmap v213: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-07-03T15:43:03.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:03 smithi028 ceph-mon[29148]: pgmap v214: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 597 B/s wr, 3 op/s 2024-07-03T15:43:03.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:03 smithi148 ceph-mon[36056]: pgmap v214: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 597 B/s wr, 3 op/s 2024-07-03T15:43:05.728 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:05 smithi028 ceph-mon[29148]: pgmap v215: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 597 B/s wr, 2 op/s 2024-07-03T15:43:05.783 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:05 smithi148 ceph-mon[36056]: pgmap v215: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 597 B/s wr, 2 op/s 2024-07-03T15:43:07.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:07 smithi148 ceph-mon[36056]: pgmap v216: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 597 B/s wr, 3 op/s 2024-07-03T15:43:07.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:07 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:43:07.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:07 smithi148 ceph-mon[36056]: Upgrade: Target is version 19.0.0-2550-gc00506dc (unknown) 2024-07-03T15:43:07.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:07 smithi148 ceph-mon[36056]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, digests ['quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2'] 2024-07-03T15:43:07.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:07 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:43:07.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:07 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:43:07.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:07 smithi148 ceph-mon[36056]: Upgrade: Need to upgrade myself (mgr.smithi028.vtqrtz) 2024-07-03T15:43:07.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:07 smithi148 ceph-mon[36056]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2 on smithi148 2024-07-03T15:43:07.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:07 smithi028 ceph-mon[29148]: pgmap v216: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 597 B/s wr, 3 op/s 2024-07-03T15:43:07.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:07 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:43:07.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:07 smithi028 ceph-mon[29148]: Upgrade: Target is version 19.0.0-2550-gc00506dc (unknown) 2024-07-03T15:43:07.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:07 smithi028 ceph-mon[29148]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, digests ['quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2'] 2024-07-03T15:43:07.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:07 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:43:07.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:07 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:43:07.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:07 smithi028 ceph-mon[29148]: Upgrade: Need to upgrade myself (mgr.smithi028.vtqrtz) 2024-07-03T15:43:07.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:07 smithi028 ceph-mon[29148]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2 on smithi148 2024-07-03T15:43:08.861 INFO:tasks.workunit.client.1.smithi148.stderr:Note: switching to '47d23ed6cd9bd552d37bff09d8392220d7ba7b11'. 2024-07-03T15:43:08.861 INFO:tasks.workunit.client.1.smithi148.stderr: 2024-07-03T15:43:08.862 INFO:tasks.workunit.client.1.smithi148.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-07-03T15:43:08.862 INFO:tasks.workunit.client.1.smithi148.stderr:changes and commit them, and you can discard any commits you make in this 2024-07-03T15:43:08.862 INFO:tasks.workunit.client.1.smithi148.stderr:state without impacting any branches by switching back to a branch. 2024-07-03T15:43:08.862 INFO:tasks.workunit.client.1.smithi148.stderr: 2024-07-03T15:43:08.862 INFO:tasks.workunit.client.1.smithi148.stderr:If you want to create a new branch to retain commits you create, you may 2024-07-03T15:43:08.862 INFO:tasks.workunit.client.1.smithi148.stderr:do so (now or later) by using -c with the switch command. Example: 2024-07-03T15:43:08.862 INFO:tasks.workunit.client.1.smithi148.stderr: 2024-07-03T15:43:08.862 INFO:tasks.workunit.client.1.smithi148.stderr: git switch -c 2024-07-03T15:43:08.862 INFO:tasks.workunit.client.1.smithi148.stderr: 2024-07-03T15:43:08.862 INFO:tasks.workunit.client.1.smithi148.stderr:Or undo this operation with: 2024-07-03T15:43:08.862 INFO:tasks.workunit.client.1.smithi148.stderr: 2024-07-03T15:43:08.862 INFO:tasks.workunit.client.1.smithi148.stderr: git switch - 2024-07-03T15:43:08.862 INFO:tasks.workunit.client.1.smithi148.stderr: 2024-07-03T15:43:08.863 INFO:tasks.workunit.client.1.smithi148.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-07-03T15:43:08.863 INFO:tasks.workunit.client.1.smithi148.stderr: 2024-07-03T15:43:08.863 INFO:tasks.workunit.client.1.smithi148.stderr:HEAD is now at 47d23ed6cd9 qa: fix log errors for cephadm tests 2024-07-03T15:43:08.866 DEBUG:teuthology.orchestra.run.smithi148:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-07-03T15:43:09.085 INFO:tasks.workunit.client.1.smithi148.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-07-03T15:43:09.088 INFO:tasks.workunit.client.1.smithi148.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-07-03T15:43:09.088 INFO:tasks.workunit.client.1.smithi148.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-07-03T15:43:09.717 INFO:tasks.workunit.client.1.smithi148.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-07-03T15:43:09.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:09 smithi148 ceph-mon[36056]: pgmap v217: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 170 B/s wr, 2 op/s 2024-07-03T15:43:09.771 INFO:tasks.workunit.client.1.smithi148.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-07-03T15:43:09.810 INFO:tasks.workunit.client.1.smithi148.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-07-03T15:43:09.812 INFO:tasks.workunit.client.1.smithi148.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-07-03T15:43:09.812 INFO:tasks.workunit.client.1.smithi148.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-07-03T15:43:09.850 INFO:tasks.workunit.client.1.smithi148.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-07-03T15:43:09.853 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-07-03T15:43:09.854 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-07-03T15:43:09.870 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-07-03T15:43:09.871 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-07-03T15:43:09.872 DEBUG:teuthology.orchestra.run.smithi148:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=47d23ed6cd9bd552d37bff09d8392220d7ba7b11 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-07-03T15:43:09.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:09 smithi028 ceph-mon[29148]: pgmap v217: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 170 B/s wr, 2 op/s 2024-07-03T15:43:09.934 INFO:tasks.workunit.client.1.smithi148.stderr:+ mkdir -p fsstress 2024-07-03T15:43:09.935 INFO:tasks.workunit.client.1.smithi148.stderr:+ pushd fsstress 2024-07-03T15:43:09.936 INFO:tasks.workunit.client.1.smithi148.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-07-03T15:43:09.936 INFO:tasks.workunit.client.1.smithi148.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-07-03T15:43:09.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:09.935+0000 7f090a625640 1 -- 172.21.15.28:0/1871089884 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f090406fdd0 msgr2=0x7f0904070240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:09.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:09.935+0000 7f090a625640 1 --2- 172.21.15.28:0/1871089884 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f090406fdd0 0x7f0904070240 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f08f80048d0 tx=0x7f08f8009da0 comp rx=0 tx=0).stop 2024-07-03T15:43:09.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:09.935+0000 7f090a625640 1 -- 172.21.15.28:0/1871089884 shutdown_connections 2024-07-03T15:43:09.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:09.935+0000 7f090a625640 1 --2- 172.21.15.28:0/1871089884 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f090406fdd0 0x7f0904070240 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:09.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:09.935+0000 7f090a625640 1 --2- 172.21.15.28:0/1871089884 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0904071360 0x7f090406f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:09.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:09.935+0000 7f090a625640 1 -- 172.21.15.28:0/1871089884 >> 172.21.15.28:0/1871089884 conn(0x7f09040f7af0 msgr2=0x7f09040f9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:43:09.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:09.936+0000 7f090a625640 1 -- 172.21.15.28:0/1871089884 shutdown_connections 2024-07-03T15:43:09.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:09.936+0000 7f090a625640 1 -- 172.21.15.28:0/1871089884 wait complete. 2024-07-03T15:43:09.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:09.936+0000 7f090a625640 1 Processor -- start 2024-07-03T15:43:09.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:09.937+0000 7f090a625640 1 -- start start 2024-07-03T15:43:09.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:09.937+0000 7f090a625640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f090406fdd0 0x7f090410f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:43:09.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:09.937+0000 7f090a625640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0904071360 0x7f090410f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:43:09.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:09.937+0000 7f090a625640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090410fd00 con 0x7f090406fdd0 2024-07-03T15:43:09.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:09.937+0000 7f090a625640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0904110520 con 0x7f0904071360 2024-07-03T15:43:09.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:09.937+0000 7f0903fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f090406fdd0 0x7f090410f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:43:09.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:09.937+0000 7f0903fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f090406fdd0 0x7f090410f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39756/0 (socket says 172.21.15.28:39756) 2024-07-03T15:43:09.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:09.937+0000 7f0903fff640 1 -- 172.21.15.28:0/458100511 learned_addr learned my addr 172.21.15.28:0/458100511 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:43:09.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:09.937+0000 7f09037fe640 1 --2- 172.21.15.28:0/458100511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0904071360 0x7f090410f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:43:09.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:09.937+0000 7f0903fff640 1 -- 172.21.15.28:0/458100511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0904071360 msgr2=0x7f090410f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:09.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:09.937+0000 7f0903fff640 1 --2- 172.21.15.28:0/458100511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0904071360 0x7f090410f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:09.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:09.937+0000 7f0903fff640 1 -- 172.21.15.28:0/458100511 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f8005040 con 0x7f090406fdd0 2024-07-03T15:43:09.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:09.937+0000 7f09037fe640 1 --2- 172.21.15.28:0/458100511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0904071360 0x7f090410f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-03T15:43:09.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:09.938+0000 7f0903fff640 1 --2- 172.21.15.28:0/458100511 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f090406fdd0 0x7f090410f170 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f08f400eb70 tx=0x7f08f400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:43:09.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:09.938+0000 7f0900ff9640 1 -- 172.21.15.28:0/458100511 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08f4007c30 con 0x7f090406fdd0 2024-07-03T15:43:09.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:09.938+0000 7f0900ff9640 1 -- 172.21.15.28:0/458100511 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f08f400f040 con 0x7f090406fdd0 2024-07-03T15:43:09.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:09.938+0000 7f090a625640 1 -- 172.21.15.28:0/458100511 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09041107b0 con 0x7f090406fdd0 2024-07-03T15:43:09.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:09.938+0000 7f0900ff9640 1 -- 172.21.15.28:0/458100511 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08f400cc30 con 0x7f090406fdd0 2024-07-03T15:43:09.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:09.938+0000 7f090a625640 1 -- 172.21.15.28:0/458100511 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0904110d50 con 0x7f090406fdd0 2024-07-03T15:43:09.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:09.939+0000 7f090a625640 1 -- 172.21.15.28:0/458100511 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09040fb790 con 0x7f090406fdd0 2024-07-03T15:43:09.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:09.940+0000 7f0900ff9640 1 -- 172.21.15.28:0/458100511 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f08f4009110 con 0x7f090406fdd0 2024-07-03T15:43:09.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:09.940+0000 7f0900ff9640 1 --2- 172.21.15.28:0/458100511 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f08ec067190 0x7f08ec069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:43:09.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:09.940+0000 7f0900ff9640 1 -- 172.21.15.28:0/458100511 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f08f4084d80 con 0x7f090406fdd0 2024-07-03T15:43:09.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:09.941+0000 7f09037fe640 1 --2- 172.21.15.28:0/458100511 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f08ec067190 0x7f08ec069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:43:09.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:09.941+0000 7f09037fe640 1 --2- 172.21.15.28:0/458100511 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f08ec067190 0x7f08ec069640 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f08f8000c00 tx=0x7f08f8035040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:43:09.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:09.945+0000 7f0900ff9640 1 -- 172.21.15.28:0/458100511 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f08f40520a0 con 0x7f090406fdd0 2024-07-03T15:43:10.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.155+0000 7f090a625640 1 -- 172.21.15.28:0/458100511 --> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0904111190 con 0x7f08ec067190 2024-07-03T15:43:10.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.157+0000 7f0900ff9640 1 -- 172.21.15.28:0/458100511 <== mgr.14210 v2:172.21.15.28:6800/421057178 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f0904111190 con 0x7f08ec067190 2024-07-03T15:43:10.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.159+0000 7f090a625640 1 -- 172.21.15.28:0/458100511 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f08ec067190 msgr2=0x7f08ec069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:10.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.159+0000 7f090a625640 1 --2- 172.21.15.28:0/458100511 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f08ec067190 0x7f08ec069640 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f08f8000c00 tx=0x7f08f8035040 comp rx=0 tx=0).stop 2024-07-03T15:43:10.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.159+0000 7f090a625640 1 -- 172.21.15.28:0/458100511 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f090406fdd0 msgr2=0x7f090410f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:10.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.159+0000 7f090a625640 1 --2- 172.21.15.28:0/458100511 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f090406fdd0 0x7f090410f170 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f08f400eb70 tx=0x7f08f400b040 comp rx=0 tx=0).stop 2024-07-03T15:43:10.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.159+0000 7f090a625640 1 -- 172.21.15.28:0/458100511 shutdown_connections 2024-07-03T15:43:10.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.159+0000 7f090a625640 1 --2- 172.21.15.28:0/458100511 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f08ec067190 0x7f08ec069640 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:10.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.159+0000 7f090a625640 1 --2- 172.21.15.28:0/458100511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0904071360 0x7f090410f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:10.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.159+0000 7f090a625640 1 --2- 172.21.15.28:0/458100511 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f090406fdd0 0x7f090410f170 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:10.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.159+0000 7f090a625640 1 -- 172.21.15.28:0/458100511 >> 172.21.15.28:0/458100511 conn(0x7f09040f7af0 msgr2=0x7f09040f91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:43:10.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.159+0000 7f090a625640 1 -- 172.21.15.28:0/458100511 shutdown_connections 2024-07-03T15:43:10.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.159+0000 7f090a625640 1 -- 172.21.15.28:0/458100511 wait complete. 2024-07-03T15:43:10.171 INFO:teuthology.orchestra.run.smithi028.stdout:true 2024-07-03T15:43:10.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.246+0000 7f7844adb640 1 -- 172.21.15.28:0/1958695392 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78400fc510 msgr2=0x7f78400fc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:10.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.246+0000 7f7844adb640 1 --2- 172.21.15.28:0/1958695392 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78400fc510 0x7f78400fc980 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f78340048d0 tx=0x7f7834009da0 comp rx=0 tx=0).stop 2024-07-03T15:43:10.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.246+0000 7f7844adb640 1 -- 172.21.15.28:0/1958695392 shutdown_connections 2024-07-03T15:43:10.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.246+0000 7f7844adb640 1 --2- 172.21.15.28:0/1958695392 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78400fc510 0x7f78400fc980 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:10.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.246+0000 7f7844adb640 1 --2- 172.21.15.28:0/1958695392 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78400fb240 0x7f78400fb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:10.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.246+0000 7f7844adb640 1 -- 172.21.15.28:0/1958695392 >> 172.21.15.28:0/1958695392 conn(0x7f78400f6a20 msgr2=0x7f78400f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:43:10.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.247+0000 7f7844adb640 1 -- 172.21.15.28:0/1958695392 shutdown_connections 2024-07-03T15:43:10.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.247+0000 7f7844adb640 1 -- 172.21.15.28:0/1958695392 wait complete. 2024-07-03T15:43:10.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.247+0000 7f7844adb640 1 Processor -- start 2024-07-03T15:43:10.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.247+0000 7f7844adb640 1 -- start start 2024-07-03T15:43:10.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.248+0000 7f7844adb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78400fb240 0x7f784010f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:43:10.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.248+0000 7f7844adb640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78400fc510 0x7f784010f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:43:10.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.248+0000 7f7844adb640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784010fd80 con 0x7f78400fc510 2024-07-03T15:43:10.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.248+0000 7f783e59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78400fb240 0x7f784010f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:43:10.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.248+0000 7f7844adb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7840110550 con 0x7f78400fb240 2024-07-03T15:43:10.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.248+0000 7f783e59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78400fb240 0x7f784010f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.28:34920/0 (socket says 172.21.15.28:34920) 2024-07-03T15:43:10.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.248+0000 7f783e59c640 1 -- 172.21.15.28:0/473423454 learned_addr learned my addr 172.21.15.28:0/473423454 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:43:10.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.248+0000 7f783dd9b640 1 --2- 172.21.15.28:0/473423454 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78400fc510 0x7f784010f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:43:10.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.249+0000 7f783dd9b640 1 -- 172.21.15.28:0/473423454 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78400fb240 msgr2=0x7f784010f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:10.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.249+0000 7f783dd9b640 1 --2- 172.21.15.28:0/473423454 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78400fb240 0x7f784010f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:10.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.249+0000 7f783dd9b640 1 -- 172.21.15.28:0/473423454 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7834005040 con 0x7f78400fc510 2024-07-03T15:43:10.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.249+0000 7f783dd9b640 1 --2- 172.21.15.28:0/473423454 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78400fc510 0x7f784010f730 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f783402a530 tx=0x7f7834009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:43:10.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.250+0000 7f782effd640 1 -- 172.21.15.28:0/473423454 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7834038070 con 0x7f78400fc510 2024-07-03T15:43:10.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.250+0000 7f782effd640 1 -- 172.21.15.28:0/473423454 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7834032970 con 0x7f78400fc510 2024-07-03T15:43:10.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.250+0000 7f7844adb640 1 -- 172.21.15.28:0/473423454 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7840110780 con 0x7f78400fc510 2024-07-03T15:43:10.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.250+0000 7f782effd640 1 -- 172.21.15.28:0/473423454 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f783403d9d0 con 0x7f78400fc510 2024-07-03T15:43:10.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.250+0000 7f7844adb640 1 -- 172.21.15.28:0/473423454 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7840110cc0 con 0x7f78400fc510 2024-07-03T15:43:10.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.252+0000 7f7844adb640 1 -- 172.21.15.28:0/473423454 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78400fa6a0 con 0x7f78400fc510 2024-07-03T15:43:10.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.252+0000 7f782effd640 1 -- 172.21.15.28:0/473423454 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f7834030020 con 0x7f78400fc510 2024-07-03T15:43:10.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.253+0000 7f782effd640 1 --2- 172.21.15.28:0/473423454 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f7828067190 0x7f7828069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:43:10.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.253+0000 7f782effd640 1 -- 172.21.15.28:0/473423454 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f78340a4850 con 0x7f78400fc510 2024-07-03T15:43:10.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.253+0000 7f783e59c640 1 --2- 172.21.15.28:0/473423454 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f7828067190 0x7f7828069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:43:10.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.254+0000 7f783e59c640 1 --2- 172.21.15.28:0/473423454 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f7828067190 0x7f7828069640 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7830004aa0 tx=0x7f783000b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:43:10.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.260+0000 7f782effd640 1 -- 172.21.15.28:0/473423454 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7834071ba0 con 0x7f78400fc510 2024-07-03T15:43:10.463 INFO:tasks.workunit.client.1.smithi148.stderr:+ tar xzf ltp-full.tgz 2024-07-03T15:43:10.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.518+0000 7f7844adb640 1 -- 172.21.15.28:0/473423454 --> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7840103f60 con 0x7f7828067190 2024-07-03T15:43:10.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.520+0000 7f782effd640 1 -- 172.21.15.28:0/473423454 <== mgr.14210 v2:172.21.15.28:6800/421057178 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f783000f040 con 0x7f7828067190 2024-07-03T15:43:10.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.523+0000 7f7844adb640 1 -- 172.21.15.28:0/473423454 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f7828067190 msgr2=0x7f7828069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:10.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.523+0000 7f7844adb640 1 --2- 172.21.15.28:0/473423454 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f7828067190 0x7f7828069640 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7830004aa0 tx=0x7f783000b040 comp rx=0 tx=0).stop 2024-07-03T15:43:10.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.523+0000 7f7844adb640 1 -- 172.21.15.28:0/473423454 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78400fc510 msgr2=0x7f784010f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:10.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.523+0000 7f7844adb640 1 --2- 172.21.15.28:0/473423454 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78400fc510 0x7f784010f730 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f783402a530 tx=0x7f7834009da0 comp rx=0 tx=0).stop 2024-07-03T15:43:10.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.523+0000 7f7844adb640 1 -- 172.21.15.28:0/473423454 shutdown_connections 2024-07-03T15:43:10.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.523+0000 7f7844adb640 1 --2- 172.21.15.28:0/473423454 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f7828067190 0x7f7828069640 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:10.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.523+0000 7f7844adb640 1 --2- 172.21.15.28:0/473423454 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78400fc510 0x7f784010f730 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:10.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.523+0000 7f7844adb640 1 --2- 172.21.15.28:0/473423454 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78400fb240 0x7f784010f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:10.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.523+0000 7f7844adb640 1 -- 172.21.15.28:0/473423454 >> 172.21.15.28:0/473423454 conn(0x7f78400f6a20 msgr2=0x7f7840065380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:43:10.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.523+0000 7f7844adb640 1 -- 172.21.15.28:0/473423454 shutdown_connections 2024-07-03T15:43:10.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.523+0000 7f7844adb640 1 -- 172.21.15.28:0/473423454 wait complete. 2024-07-03T15:43:10.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.654+0000 7fbab8009640 1 -- 172.21.15.28:0/548599876 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbab006da00 msgr2=0x7fbab006de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:10.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.654+0000 7fbab8009640 1 --2- 172.21.15.28:0/548599876 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbab006da00 0x7fbab006de10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbaac004810 tx=0x7fbaac009da0 comp rx=0 tx=0).stop 2024-07-03T15:43:10.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.655+0000 7fbab8009640 1 -- 172.21.15.28:0/548599876 shutdown_connections 2024-07-03T15:43:10.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.655+0000 7fbab8009640 1 --2- 172.21.15.28:0/548599876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbab006e350 0x7fbab0108540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:10.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.655+0000 7fbab8009640 1 --2- 172.21.15.28:0/548599876 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbab006da00 0x7fbab006de10 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:10.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.655+0000 7fbab8009640 1 -- 172.21.15.28:0/548599876 >> 172.21.15.28:0/548599876 conn(0x7fbab0069370 msgr2=0x7fbab006b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:43:10.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.655+0000 7fbab8009640 1 -- 172.21.15.28:0/548599876 shutdown_connections 2024-07-03T15:43:10.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.655+0000 7fbab8009640 1 -- 172.21.15.28:0/548599876 wait complete. 2024-07-03T15:43:10.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.655+0000 7fbab8009640 1 Processor -- start 2024-07-03T15:43:10.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.656+0000 7fbab8009640 1 -- start start 2024-07-03T15:43:10.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.656+0000 7fbab8009640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbab006da00 0x7fbab011bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:43:10.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.656+0000 7fbab8009640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbab006e350 0x7fbab011c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:43:10.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.656+0000 7fbab8009640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab011c910 con 0x7fbab006e350 2024-07-03T15:43:10.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.656+0000 7fbab8009640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab011d050 con 0x7fbab006da00 2024-07-03T15:43:10.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.656+0000 7fbab55a4640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbab006e350 0x7fbab011c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:43:10.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.656+0000 7fbab55a4640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbab006e350 0x7fbab011c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39792/0 (socket says 172.21.15.28:39792) 2024-07-03T15:43:10.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.656+0000 7fbab55a4640 1 -- 172.21.15.28:0/3280288456 learned_addr learned my addr 172.21.15.28:0/3280288456 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:43:10.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.656+0000 7fbab5da5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbab006da00 0x7fbab011bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:43:10.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.657+0000 7fbab55a4640 1 -- 172.21.15.28:0/3280288456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbab006da00 msgr2=0x7fbab011bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:10.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.657+0000 7fbab55a4640 1 --2- 172.21.15.28:0/3280288456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbab006da00 0x7fbab011bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:10.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.657+0000 7fbab55a4640 1 -- 172.21.15.28:0/3280288456 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaac005040 con 0x7fbab006e350 2024-07-03T15:43:10.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.657+0000 7fbab5da5640 1 --2- 172.21.15.28:0/3280288456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbab006da00 0x7fbab011bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-03T15:43:10.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.657+0000 7fbab55a4640 1 --2- 172.21.15.28:0/3280288456 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbab006e350 0x7fbab011c2c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbaa000eb40 tx=0x7fbaa000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:43:10.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.657+0000 7fbaa67fc640 1 -- 172.21.15.28:0/3280288456 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbaa0007c30 con 0x7fbab006e350 2024-07-03T15:43:10.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.657+0000 7fbaa67fc640 1 -- 172.21.15.28:0/3280288456 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbaa000f040 con 0x7fbab006e350 2024-07-03T15:43:10.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.657+0000 7fbaa67fc640 1 -- 172.21.15.28:0/3280288456 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbaa000cb40 con 0x7fbab006e350 2024-07-03T15:43:10.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.657+0000 7fbab8009640 1 -- 172.21.15.28:0/3280288456 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbab011d280 con 0x7fbab006e350 2024-07-03T15:43:10.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.658+0000 7fbab8009640 1 -- 172.21.15.28:0/3280288456 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbab011d7a0 con 0x7fbab006e350 2024-07-03T15:43:10.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.659+0000 7fbab8009640 1 -- 172.21.15.28:0/3280288456 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbab006cfc0 con 0x7fbab006e350 2024-07-03T15:43:10.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.661+0000 7fbaa67fc640 1 -- 172.21.15.28:0/3280288456 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7fbaa000cca0 con 0x7fbab006e350 2024-07-03T15:43:10.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.662+0000 7fbaa67fc640 1 --2- 172.21.15.28:0/3280288456 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fba9c067320 0x7fba9c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:43:10.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.662+0000 7fbaa67fc640 1 -- 172.21.15.28:0/3280288456 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fbaa0084d30 con 0x7fbab006e350 2024-07-03T15:43:10.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.662+0000 7fbab5da5640 1 --2- 172.21.15.28:0/3280288456 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fba9c067320 0x7fba9c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:43:10.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.663+0000 7fbab5da5640 1 --2- 172.21.15.28:0/3280288456 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fba9c067320 0x7fba9c0697d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbaac000c00 tx=0x7fbaac035040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:43:10.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:10.668+0000 7fbaa67fc640 1 -- 172.21.15.28:0/3280288456 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fbaa00520e0 con 0x7fbab006e350 2024-07-03T15:43:11.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.045+0000 7fbab8009640 1 -- 172.21.15.28:0/3280288456 --> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbab011dc70 con 0x7fba9c067320 2024-07-03T15:43:11.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.064+0000 7fbaa67fc640 1 -- 172.21.15.28:0/3280288456 <== mgr.14210 v2:172.21.15.28:6800/421057178 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7fbab011dc70 con 0x7fba9c067320 2024-07-03T15:43:11.068 INFO:teuthology.orchestra.run.smithi028.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-03T15:43:11.068 INFO:teuthology.orchestra.run.smithi028.stdout:alertmanager.smithi028 smithi028 *:9093,9094 running (5m) 61s ago 6m 24.9M - 0.25.0 c8568f914cd2 b9a1a484a0a2 2024-07-03T15:43:11.068 INFO:teuthology.orchestra.run.smithi028.stdout:crash.smithi028 smithi028 running (6m) 61s ago 6m 6899k - 17.2.7-1047-gd4b063ab 4ca399d9e65d 535660e448f9 2024-07-03T15:43:11.069 INFO:teuthology.orchestra.run.smithi028.stdout:crash.smithi148 smithi148 running (6m) 68s ago 5m 6899k - 17.2.7-1047-gd4b063ab 4ca399d9e65d 4bb7675895e7 2024-07-03T15:43:11.069 INFO:teuthology.orchestra.run.smithi028.stdout:grafana.smithi028 smithi028 *:3000 running (5m) 61s ago 6m 88.5M - 9.4.7 954c08fa6188 07e0ae3c15aa 2024-07-03T15:43:11.069 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi028.rhpvve smithi028 running (86s) 61s ago 85s 15.6M - 17.2.7-1047-gd4b063ab 4ca399d9e65d 3ed5abd02cfb 2024-07-03T15:43:11.069 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi028.wmbxew smithi028 running (81s) 61s ago 80s 14.9M - 17.2.7-1047-gd4b063ab 4ca399d9e65d b3bf736998f9 2024-07-03T15:43:11.069 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi148.szomfl smithi148 running (78s) 68s ago 78s 12.9M - 17.2.7-1047-gd4b063ab 4ca399d9e65d d94f5c95fa8d 2024-07-03T15:43:11.069 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi148.xgpuld smithi148 running (83s) 68s ago 83s 10.7M - 17.2.7-1047-gd4b063ab 4ca399d9e65d fc70360a8de0 2024-07-03T15:43:11.069 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.smithi028.vtqrtz smithi028 *:9283 running (8m) 61s ago 8m 447M - 17.2.7-1047-gd4b063ab 4ca399d9e65d 0d26e74cb154 2024-07-03T15:43:11.069 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.smithi148.rjzsar smithi148 *:8443,9283 running (5m) 68s ago 5m 401M - 17.2.7-1047-gd4b063ab 4ca399d9e65d c637b3ae9fce 2024-07-03T15:43:11.069 INFO:teuthology.orchestra.run.smithi028.stdout:mon.smithi028 smithi028 running (8m) 61s ago 8m 43.2M 2048M 17.2.7-1047-gd4b063ab 4ca399d9e65d 7461f9bef4e0 2024-07-03T15:43:11.069 INFO:teuthology.orchestra.run.smithi028.stdout:mon.smithi148 smithi148 running (5m) 68s ago 5m 32.8M 2048M 17.2.7-1047-gd4b063ab 4ca399d9e65d 4c6c8ed2ab85 2024-07-03T15:43:11.069 INFO:teuthology.orchestra.run.smithi028.stdout:node-exporter.smithi028 smithi028 *:9100 running (6m) 61s ago 6m 16.0M - 1.5.0 0da6a335fe13 68318a0ca520 2024-07-03T15:43:11.069 INFO:teuthology.orchestra.run.smithi028.stdout:node-exporter.smithi148 smithi148 *:9100 running (5m) 68s ago 5m 17.6M - 1.5.0 0da6a335fe13 4994e60bd57c 2024-07-03T15:43:11.069 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0 smithi028 running (5m) 61s ago 5m 48.5M 4289M 17.2.7-1047-gd4b063ab 4ca399d9e65d b18089a439ba 2024-07-03T15:43:11.070 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1 smithi028 running (4m) 61s ago 4m 45.5M 4289M 17.2.7-1047-gd4b063ab 4ca399d9e65d 87d415dde231 2024-07-03T15:43:11.070 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2 smithi028 running (4m) 61s ago 4m 46.3M 4289M 17.2.7-1047-gd4b063ab 4ca399d9e65d b2cbb312212d 2024-07-03T15:43:11.070 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3 smithi148 running (3m) 68s ago 3m 44.4M 5312M 17.2.7-1047-gd4b063ab 4ca399d9e65d 5be69a27df31 2024-07-03T15:43:11.070 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4 smithi148 running (3m) 68s ago 3m 44.3M 5312M 17.2.7-1047-gd4b063ab 4ca399d9e65d 54f7c26c77ca 2024-07-03T15:43:11.070 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5 smithi148 running (2m) 68s ago 2m 41.4M 5312M 17.2.7-1047-gd4b063ab 4ca399d9e65d d185c6905384 2024-07-03T15:43:11.070 INFO:teuthology.orchestra.run.smithi028.stdout:prometheus.smithi028 smithi028 *:9095 running (5m) 61s ago 6m 54.2M - 2.43.0 a07b618ecd1d 97f838e118a3 2024-07-03T15:43:11.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.067+0000 7fbab8009640 1 -- 172.21.15.28:0/3280288456 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fba9c067320 msgr2=0x7fba9c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:11.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.067+0000 7fbab8009640 1 --2- 172.21.15.28:0/3280288456 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fba9c067320 0x7fba9c0697d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbaac000c00 tx=0x7fbaac035040 comp rx=0 tx=0).stop 2024-07-03T15:43:11.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.067+0000 7fbab8009640 1 -- 172.21.15.28:0/3280288456 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbab006e350 msgr2=0x7fbab011c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:11.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.067+0000 7fbab8009640 1 --2- 172.21.15.28:0/3280288456 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbab006e350 0x7fbab011c2c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbaa000eb40 tx=0x7fbaa000b040 comp rx=0 tx=0).stop 2024-07-03T15:43:11.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.068+0000 7fbab8009640 1 -- 172.21.15.28:0/3280288456 shutdown_connections 2024-07-03T15:43:11.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.068+0000 7fbab8009640 1 --2- 172.21.15.28:0/3280288456 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fba9c067320 0x7fba9c0697d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:11.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.068+0000 7fbab8009640 1 --2- 172.21.15.28:0/3280288456 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbab006e350 0x7fbab011c2c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:11.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.068+0000 7fbab8009640 1 --2- 172.21.15.28:0/3280288456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbab006da00 0x7fbab011bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:11.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.068+0000 7fbab8009640 1 -- 172.21.15.28:0/3280288456 >> 172.21.15.28:0/3280288456 conn(0x7fbab0069370 msgr2=0x7fbab0106600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:43:11.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.068+0000 7fbab8009640 1 -- 172.21.15.28:0/3280288456 shutdown_connections 2024-07-03T15:43:11.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.068+0000 7fbab8009640 1 -- 172.21.15.28:0/3280288456 wait complete. 2024-07-03T15:43:11.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.195+0000 7f2c9a944640 1 -- 172.21.15.28:0/3469652575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c940fe550 msgr2=0x7f2c940fe960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:11.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.195+0000 7f2c9a944640 1 --2- 172.21.15.28:0/3469652575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c940fe550 0x7f2c940fe960 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2c840048d0 tx=0x7f2c84009da0 comp rx=0 tx=0).stop 2024-07-03T15:43:11.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.195+0000 7f2c9a944640 1 -- 172.21.15.28:0/3469652575 shutdown_connections 2024-07-03T15:43:11.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.195+0000 7f2c9a944640 1 --2- 172.21.15.28:0/3469652575 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c940ff820 0x7f2c940ffc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:11.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.195+0000 7f2c9a944640 1 --2- 172.21.15.28:0/3469652575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c940fe550 0x7f2c940fe960 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:11.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.195+0000 7f2c9a944640 1 -- 172.21.15.28:0/3469652575 >> 172.21.15.28:0/3469652575 conn(0x7f2c940f9e70 msgr2=0x7f2c940fc250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:43:11.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.195+0000 7f2c9a944640 1 -- 172.21.15.28:0/3469652575 shutdown_connections 2024-07-03T15:43:11.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.195+0000 7f2c9a944640 1 -- 172.21.15.28:0/3469652575 wait complete. 2024-07-03T15:43:11.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.197+0000 7f2c9a944640 1 Processor -- start 2024-07-03T15:43:11.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.197+0000 7f2c9a944640 1 -- start start 2024-07-03T15:43:11.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.197+0000 7f2c9a944640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c940fe550 0x7f2c9410efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:43:11.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.197+0000 7f2c9a944640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c940ff820 0x7f2c9410f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:43:11.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.197+0000 7f2c93fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c940fe550 0x7f2c9410efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:43:11.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.197+0000 7f2c93fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c940fe550 0x7f2c9410efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.28:34960/0 (socket says 172.21.15.28:34960) 2024-07-03T15:43:11.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.198+0000 7f2c8bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c940ff820 0x7f2c9410f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:43:11.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.198+0000 7f2c9a944640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9410fb50 con 0x7f2c940ff820 2024-07-03T15:43:11.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.198+0000 7f2c8bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c940ff820 0x7f2c9410f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39820/0 (socket says 172.21.15.28:39820) 2024-07-03T15:43:11.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.198+0000 7f2c9a944640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c94110320 con 0x7f2c940fe550 2024-07-03T15:43:11.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.198+0000 7f2c8bfff640 1 -- 172.21.15.28:0/1225967754 learned_addr learned my addr 172.21.15.28:0/1225967754 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:43:11.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.198+0000 7f2c93fff640 1 -- 172.21.15.28:0/1225967754 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c940ff820 msgr2=0x7f2c9410f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:11.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.198+0000 7f2c93fff640 1 --2- 172.21.15.28:0/1225967754 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c940ff820 0x7f2c9410f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:11.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.198+0000 7f2c93fff640 1 -- 172.21.15.28:0/1225967754 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c84005040 con 0x7f2c940fe550 2024-07-03T15:43:11.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.199+0000 7f2c93fff640 1 --2- 172.21.15.28:0/1225967754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c940fe550 0x7f2c9410efc0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2c8402a530 tx=0x7f2c84009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:43:11.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.199+0000 7f2c917fa640 1 -- 172.21.15.28:0/1225967754 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c84038070 con 0x7f2c940fe550 2024-07-03T15:43:11.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.199+0000 7f2c9a944640 1 -- 172.21.15.28:0/1225967754 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c94110550 con 0x7f2c940fe550 2024-07-03T15:43:11.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.199+0000 7f2c917fa640 1 -- 172.21.15.28:0/1225967754 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2c84032970 con 0x7f2c940fe550 2024-07-03T15:43:11.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.199+0000 7f2c917fa640 1 -- 172.21.15.28:0/1225967754 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c840339c0 con 0x7f2c940fe550 2024-07-03T15:43:11.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.199+0000 7f2c9a944640 1 -- 172.21.15.28:0/1225967754 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c941109e0 con 0x7f2c940fe550 2024-07-03T15:43:11.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.201+0000 7f2c917fa640 1 -- 172.21.15.28:0/1225967754 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f2c84030020 con 0x7f2c940fe550 2024-07-03T15:43:11.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.202+0000 7f2c917fa640 1 --2- 172.21.15.28:0/1225967754 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f2c80067260 0x7f2c80069710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:43:11.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.202+0000 7f2c9a944640 1 -- 172.21.15.28:0/1225967754 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c941078b0 con 0x7f2c940fe550 2024-07-03T15:43:11.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.202+0000 7f2c917fa640 1 -- 172.21.15.28:0/1225967754 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f2c840a4b20 con 0x7f2c940fe550 2024-07-03T15:43:11.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.202+0000 7f2c8bfff640 1 --2- 172.21.15.28:0/1225967754 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f2c80067260 0x7f2c80069710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:43:11.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.203+0000 7f2c8bfff640 1 --2- 172.21.15.28:0/1225967754 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f2c80067260 0x7f2c80069710 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2c9410ed70 tx=0x7f2c7c00b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:43:11.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.207+0000 7f2c917fa640 1 -- 172.21.15.28:0/1225967754 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2c84071e50 con 0x7f2c940fe550 2024-07-03T15:43:11.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.603+0000 7f2c9a944640 1 -- 172.21.15.28:0/1225967754 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2c9404b530 con 0x7f2c940fe550 2024-07-03T15:43:11.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.603+0000 7f2c917fa640 1 -- 172.21.15.28:0/1225967754 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+643 (secure 0 0 0) 0x7f2c84075470 con 0x7f2c940fe550 2024-07-03T15:43:11.608 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-07-03T15:43:11.608 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": { 2024-07-03T15:43:11.608 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 2 2024-07-03T15:43:11.608 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:43:11.608 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": { 2024-07-03T15:43:11.608 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 2 2024-07-03T15:43:11.608 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:43:11.608 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": { 2024-07-03T15:43:11.608 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 6 2024-07-03T15:43:11.608 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:43:11.608 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": { 2024-07-03T15:43:11.608 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 4 2024-07-03T15:43:11.609 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:43:11.609 INFO:teuthology.orchestra.run.smithi028.stdout: "overall": { 2024-07-03T15:43:11.609 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 14 2024-07-03T15:43:11.609 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-07-03T15:43:11.609 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-07-03T15:43:11.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.606+0000 7f2c9a944640 1 -- 172.21.15.28:0/1225967754 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f2c80067260 msgr2=0x7f2c80069710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:11.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.606+0000 7f2c9a944640 1 --2- 172.21.15.28:0/1225967754 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f2c80067260 0x7f2c80069710 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2c9410ed70 tx=0x7f2c7c00b040 comp rx=0 tx=0).stop 2024-07-03T15:43:11.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.606+0000 7f2c9a944640 1 -- 172.21.15.28:0/1225967754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c940fe550 msgr2=0x7f2c9410efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:11.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.606+0000 7f2c9a944640 1 --2- 172.21.15.28:0/1225967754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c940fe550 0x7f2c9410efc0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2c8402a530 tx=0x7f2c84009da0 comp rx=0 tx=0).stop 2024-07-03T15:43:11.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.607+0000 7f2c9a944640 1 -- 172.21.15.28:0/1225967754 shutdown_connections 2024-07-03T15:43:11.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.607+0000 7f2c9a944640 1 --2- 172.21.15.28:0/1225967754 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f2c80067260 0x7f2c80069710 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:11.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.607+0000 7f2c9a944640 1 --2- 172.21.15.28:0/1225967754 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c940ff820 0x7f2c9410f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:11.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.607+0000 7f2c9a944640 1 --2- 172.21.15.28:0/1225967754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c940fe550 0x7f2c9410efc0 secure :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2c8402a530 tx=0x7f2c84009da0 comp rx=0 tx=0).stop 2024-07-03T15:43:11.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.607+0000 7f2c9a944640 1 -- 172.21.15.28:0/1225967754 >> 172.21.15.28:0/1225967754 conn(0x7f2c940f9e70 msgr2=0x7f2c940fc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:43:11.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.607+0000 7f2c9a944640 1 -- 172.21.15.28:0/1225967754 shutdown_connections 2024-07-03T15:43:11.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.608+0000 7f2c9a944640 1 -- 172.21.15.28:0/1225967754 wait complete. 2024-07-03T15:43:11.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:11 smithi148 ceph-mon[36056]: from='client.14566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:43:11.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:11 smithi148 ceph-mon[36056]: pgmap v218: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-07-03T15:43:11.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:11 smithi148 ceph-mon[36056]: from='client.14570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:43:11.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.740+0000 7fe5f7d31640 1 -- 172.21.15.28:0/790267560 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5f00fc330 msgr2=0x7fe5f00fc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:11.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.740+0000 7fe5f7d31640 1 --2- 172.21.15.28:0/790267560 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5f00fc330 0x7fe5f00fc740 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec0048d0 tx=0x7fe5ec009da0 comp rx=0 tx=0).stop 2024-07-03T15:43:11.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.740+0000 7fe5f7d31640 1 -- 172.21.15.28:0/790267560 shutdown_connections 2024-07-03T15:43:11.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.740+0000 7fe5f7d31640 1 --2- 172.21.15.28:0/790267560 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5f00fd600 0x7fe5f00fda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:11.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.740+0000 7fe5f7d31640 1 --2- 172.21.15.28:0/790267560 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5f00fc330 0x7fe5f00fc740 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:11.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.740+0000 7fe5f7d31640 1 -- 172.21.15.28:0/790267560 >> 172.21.15.28:0/790267560 conn(0x7fe5f00f7af0 msgr2=0x7fe5f00f9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:43:11.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.740+0000 7fe5f7d31640 1 -- 172.21.15.28:0/790267560 shutdown_connections 2024-07-03T15:43:11.744 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:11 smithi028 ceph-mon[29148]: from='client.14566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:43:11.744 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:11 smithi028 ceph-mon[29148]: pgmap v218: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-07-03T15:43:11.745 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:11 smithi028 ceph-mon[29148]: from='client.14570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:43:11.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.740+0000 7fe5f7d31640 1 -- 172.21.15.28:0/790267560 wait complete. 2024-07-03T15:43:11.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.741+0000 7fe5f7d31640 1 Processor -- start 2024-07-03T15:43:11.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.741+0000 7fe5f7d31640 1 -- start start 2024-07-03T15:43:11.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.741+0000 7fe5f7d31640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5f00fc330 0x7fe5f010af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:43:11.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.741+0000 7fe5f7d31640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5f00fd600 0x7fe5f010b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:43:11.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.741+0000 7fe5f7d31640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f010ba60 con 0x7fe5f00fc330 2024-07-03T15:43:11.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.741+0000 7fe5f7d31640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f010c320 con 0x7fe5f00fd600 2024-07-03T15:43:11.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.742+0000 7fe5f5acd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5f00fc330 0x7fe5f010af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:43:11.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.742+0000 7fe5f5acd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5f00fc330 0x7fe5f010af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39848/0 (socket says 172.21.15.28:39848) 2024-07-03T15:43:11.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.742+0000 7fe5f5acd640 1 -- 172.21.15.28:0/1536757007 learned_addr learned my addr 172.21.15.28:0/1536757007 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:43:11.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.742+0000 7fe5f52cc640 1 --2- 172.21.15.28:0/1536757007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5f00fd600 0x7fe5f010b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:43:11.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.742+0000 7fe5f5acd640 1 -- 172.21.15.28:0/1536757007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5f00fd600 msgr2=0x7fe5f010b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:11.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.742+0000 7fe5f5acd640 1 --2- 172.21.15.28:0/1536757007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5f00fd600 0x7fe5f010b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:11.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.742+0000 7fe5f5acd640 1 -- 172.21.15.28:0/1536757007 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e0005040 con 0x7fe5f00fc330 2024-07-03T15:43:11.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.742+0000 7fe5f5acd640 1 --2- 172.21.15.28:0/1536757007 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5f00fc330 0x7fe5f010af20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec0041d0 tx=0x7fe5ec009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:43:11.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.743+0000 7fe5e67fc640 1 -- 172.21.15.28:0/1536757007 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5ec038070 con 0x7fe5f00fc330 2024-07-03T15:43:11.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.743+0000 7fe5f7d31640 1 -- 172.21.15.28:0/1536757007 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5ec005040 con 0x7fe5f00fc330 2024-07-03T15:43:11.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.743+0000 7fe5e67fc640 1 -- 172.21.15.28:0/1536757007 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe5ec032e10 con 0x7fe5f00fc330 2024-07-03T15:43:11.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.743+0000 7fe5e67fc640 1 -- 172.21.15.28:0/1536757007 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5ec009760 con 0x7fe5f00fc330 2024-07-03T15:43:11.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.743+0000 7fe5f7d31640 1 -- 172.21.15.28:0/1536757007 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f010c950 con 0x7fe5f00fc330 2024-07-03T15:43:11.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.744+0000 7fe5f7d31640 1 -- 172.21.15.28:0/1536757007 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5f00fb790 con 0x7fe5f00fc330 2024-07-03T15:43:11.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.748+0000 7fe5e67fc640 1 -- 172.21.15.28:0/1536757007 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7fe5ec030020 con 0x7fe5f00fc330 2024-07-03T15:43:11.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.749+0000 7fe5e67fc640 1 --2- 172.21.15.28:0/1536757007 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fe5dc067320 0x7fe5dc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:43:11.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.749+0000 7fe5e67fc640 1 -- 172.21.15.28:0/1536757007 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fe5ec0a4d80 con 0x7fe5f00fc330 2024-07-03T15:43:11.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.749+0000 7fe5f52cc640 1 --2- 172.21.15.28:0/1536757007 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fe5dc067320 0x7fe5dc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:43:11.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.750+0000 7fe5f52cc640 1 --2- 172.21.15.28:0/1536757007 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fe5dc067320 0x7fe5dc0697d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0007fd0 tx=0x7fe5e0009da0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:43:11.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:11.755+0000 7fe5e67fc640 1 -- 172.21.15.28:0/1536757007 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe5ec072080 con 0x7fe5f00fc330 2024-07-03T15:43:12.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.165+0000 7fe5f7d31640 1 -- 172.21.15.28:0/1536757007 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe5f0104e80 con 0x7fe5f00fc330 2024-07-03T15:43:12.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.165+0000 7fe5e67fc640 1 -- 172.21.15.28:0/1536757007 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1838 (secure 0 0 0) 0x7fe5ec0098c0 con 0x7fe5f00fc330 2024-07-03T15:43:12.169 INFO:teuthology.orchestra.run.smithi028.stdout:e17 2024-07-03T15:43:12.169 INFO:teuthology.orchestra.run.smithi028.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-03T15:43:12.169 INFO:teuthology.orchestra.run.smithi028.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:43:12.169 INFO:teuthology.orchestra.run.smithi028.stdout:legacy client fscid: 1 2024-07-03T15:43:12.169 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:43:12.169 INFO:teuthology.orchestra.run.smithi028.stdout:Filesystem 'cephfs' (1) 2024-07-03T15:43:12.169 INFO:teuthology.orchestra.run.smithi028.stdout:fs_name cephfs 2024-07-03T15:43:12.170 INFO:teuthology.orchestra.run.smithi028.stdout:epoch 16 2024-07-03T15:43:12.170 INFO:teuthology.orchestra.run.smithi028.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-03T15:43:12.170 INFO:teuthology.orchestra.run.smithi028.stdout:created 2024-07-03T15:41:40.656298+0000 2024-07-03T15:43:12.170 INFO:teuthology.orchestra.run.smithi028.stdout:modified 2024-07-03T15:41:56.454984+0000 2024-07-03T15:43:12.170 INFO:teuthology.orchestra.run.smithi028.stdout:tableserver 0 2024-07-03T15:43:12.170 INFO:teuthology.orchestra.run.smithi028.stdout:root 0 2024-07-03T15:43:12.170 INFO:teuthology.orchestra.run.smithi028.stdout:session_timeout 60 2024-07-03T15:43:12.170 INFO:teuthology.orchestra.run.smithi028.stdout:session_autoclose 300 2024-07-03T15:43:12.170 INFO:teuthology.orchestra.run.smithi028.stdout:max_file_size 1099511627776 2024-07-03T15:43:12.170 INFO:teuthology.orchestra.run.smithi028.stdout:required_client_features {} 2024-07-03T15:43:12.170 INFO:teuthology.orchestra.run.smithi028.stdout:last_failure 0 2024-07-03T15:43:12.170 INFO:teuthology.orchestra.run.smithi028.stdout:last_failure_osd_epoch 0 2024-07-03T15:43:12.171 INFO:teuthology.orchestra.run.smithi028.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:43:12.171 INFO:teuthology.orchestra.run.smithi028.stdout:max_mds 1 2024-07-03T15:43:12.171 INFO:teuthology.orchestra.run.smithi028.stdout:in 0 2024-07-03T15:43:12.171 INFO:teuthology.orchestra.run.smithi028.stdout:up {0=14450} 2024-07-03T15:43:12.171 INFO:teuthology.orchestra.run.smithi028.stdout:failed 2024-07-03T15:43:12.171 INFO:teuthology.orchestra.run.smithi028.stdout:damaged 2024-07-03T15:43:12.171 INFO:teuthology.orchestra.run.smithi028.stdout:stopped 2024-07-03T15:43:12.171 INFO:teuthology.orchestra.run.smithi028.stdout:data_pools [3] 2024-07-03T15:43:12.171 INFO:teuthology.orchestra.run.smithi028.stdout:metadata_pool 2 2024-07-03T15:43:12.171 INFO:teuthology.orchestra.run.smithi028.stdout:inline_data disabled 2024-07-03T15:43:12.171 INFO:teuthology.orchestra.run.smithi028.stdout:balancer 2024-07-03T15:43:12.171 INFO:teuthology.orchestra.run.smithi028.stdout:standby_count_wanted 1 2024-07-03T15:43:12.171 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi028.rhpvve{0:14450} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.28:6826/1613936042,v1:172.21.15.28:6827/1613936042] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:43:12.172 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi028.wmbxew{0:14460} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.28:6828/1730336821,v1:172.21.15.28:6829/1730336821] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:43:12.172 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:43:12.172 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:43:12.172 INFO:teuthology.orchestra.run.smithi028.stdout:Standby daemons: 2024-07-03T15:43:12.172 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:43:12.172 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi148.szomfl{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/4171231858,v1:172.21.15.148:6827/4171231858] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:43:12.172 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi148.xgpuld{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/2347536398,v1:172.21.15.148:6825/2347536398] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:43:12.173 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 17 2024-07-03T15:43:12.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.168+0000 7fe5f7d31640 1 -- 172.21.15.28:0/1536757007 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fe5dc067320 msgr2=0x7fe5dc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:12.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.168+0000 7fe5f7d31640 1 --2- 172.21.15.28:0/1536757007 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fe5dc067320 0x7fe5dc0697d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0007fd0 tx=0x7fe5e0009da0 comp rx=0 tx=0).stop 2024-07-03T15:43:12.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.168+0000 7fe5f7d31640 1 -- 172.21.15.28:0/1536757007 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5f00fc330 msgr2=0x7fe5f010af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:12.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.168+0000 7fe5f7d31640 1 --2- 172.21.15.28:0/1536757007 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5f00fc330 0x7fe5f010af20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec0041d0 tx=0x7fe5ec009da0 comp rx=0 tx=0).stop 2024-07-03T15:43:12.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.169+0000 7fe5f7d31640 1 -- 172.21.15.28:0/1536757007 shutdown_connections 2024-07-03T15:43:12.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.169+0000 7fe5f7d31640 1 --2- 172.21.15.28:0/1536757007 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fe5dc067320 0x7fe5dc0697d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:12.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.169+0000 7fe5f7d31640 1 --2- 172.21.15.28:0/1536757007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5f00fd600 0x7fe5f010b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:12.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.169+0000 7fe5f7d31640 1 --2- 172.21.15.28:0/1536757007 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5f00fc330 0x7fe5f010af20 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:12.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.169+0000 7fe5f7d31640 1 -- 172.21.15.28:0/1536757007 >> 172.21.15.28:0/1536757007 conn(0x7fe5f00f7af0 msgr2=0x7fe5f00f9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:43:12.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.169+0000 7fe5f7d31640 1 -- 172.21.15.28:0/1536757007 shutdown_connections 2024-07-03T15:43:12.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.169+0000 7fe5f7d31640 1 -- 172.21.15.28:0/1536757007 wait complete. 2024-07-03T15:43:12.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.293+0000 7fedbc0fa640 1 -- 172.21.15.28:0/318309553 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fedb40fda00 msgr2=0x7fedb40fde10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:12.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.293+0000 7fedbc0fa640 1 --2- 172.21.15.28:0/318309553 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fedb40fda00 0x7fedb40fde10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fedb00048d0 tx=0x7fedb0009da0 comp rx=0 tx=0).stop 2024-07-03T15:43:12.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.293+0000 7fedbc0fa640 1 -- 172.21.15.28:0/318309553 shutdown_connections 2024-07-03T15:43:12.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.293+0000 7fedbc0fa640 1 --2- 172.21.15.28:0/318309553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedb40fe350 0x7fedb40fa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:12.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.293+0000 7fedbc0fa640 1 --2- 172.21.15.28:0/318309553 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fedb40fda00 0x7fedb40fde10 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:12.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.294+0000 7fedbc0fa640 1 -- 172.21.15.28:0/318309553 >> 172.21.15.28:0/318309553 conn(0x7fedb40f61a0 msgr2=0x7fedb40f85b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:43:12.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.294+0000 7fedbc0fa640 1 -- 172.21.15.28:0/318309553 shutdown_connections 2024-07-03T15:43:12.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.294+0000 7fedbc0fa640 1 -- 172.21.15.28:0/318309553 wait complete. 2024-07-03T15:43:12.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.294+0000 7fedbc0fa640 1 Processor -- start 2024-07-03T15:43:12.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.294+0000 7fedbc0fa640 1 -- start start 2024-07-03T15:43:12.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.295+0000 7fedbc0fa640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fedb40fda00 0x7fedb41135c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:43:12.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.295+0000 7fedbc0fa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedb40fe350 0x7fedb4113b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:43:12.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.295+0000 7fedbc0fa640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb4114150 con 0x7fedb40fda00 2024-07-03T15:43:12.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.295+0000 7fedbc0fa640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb4114920 con 0x7fedb40fe350 2024-07-03T15:43:12.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.295+0000 7fedb9695640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedb40fe350 0x7fedb4113b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:43:12.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.295+0000 7fedb9695640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedb40fe350 0x7fedb4113b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.28:35008/0 (socket says 172.21.15.28:35008) 2024-07-03T15:43:12.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.295+0000 7fedb9695640 1 -- 172.21.15.28:0/1281439321 learned_addr learned my addr 172.21.15.28:0/1281439321 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:43:12.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.295+0000 7fedb9e96640 1 --2- 172.21.15.28:0/1281439321 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fedb40fda00 0x7fedb41135c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:43:12.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.295+0000 7fedb9695640 1 -- 172.21.15.28:0/1281439321 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fedb40fda00 msgr2=0x7fedb41135c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:12.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.295+0000 7fedb9695640 1 --2- 172.21.15.28:0/1281439321 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fedb40fda00 0x7fedb41135c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:12.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.295+0000 7fedb9695640 1 -- 172.21.15.28:0/1281439321 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb0005040 con 0x7fedb40fe350 2024-07-03T15:43:12.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.295+0000 7fedb9695640 1 --2- 172.21.15.28:0/1281439321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedb40fe350 0x7fedb4113b00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7feda400eb70 tx=0x7feda400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:43:12.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.296+0000 7fedaa7fc640 1 -- 172.21.15.28:0/1281439321 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feda4007c30 con 0x7fedb40fe350 2024-07-03T15:43:12.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.296+0000 7fedbc0fa640 1 -- 172.21.15.28:0/1281439321 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedb4114b70 con 0x7fedb40fe350 2024-07-03T15:43:12.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.296+0000 7fedaa7fc640 1 -- 172.21.15.28:0/1281439321 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feda400f040 con 0x7fedb40fe350 2024-07-03T15:43:12.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.296+0000 7fedaa7fc640 1 -- 172.21.15.28:0/1281439321 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feda40094d0 con 0x7fedb40fe350 2024-07-03T15:43:12.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.296+0000 7fedbc0fa640 1 -- 172.21.15.28:0/1281439321 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedb4115110 con 0x7fedb40fe350 2024-07-03T15:43:12.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.297+0000 7fedaa7fc640 1 -- 172.21.15.28:0/1281439321 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7feda4009630 con 0x7fedb40fe350 2024-07-03T15:43:12.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.297+0000 7fedbc0fa640 1 -- 172.21.15.28:0/1281439321 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedb40f9e20 con 0x7fedb40fe350 2024-07-03T15:43:12.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.298+0000 7fedaa7fc640 1 --2- 172.21.15.28:0/1281439321 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7feda0067190 0x7feda0069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:43:12.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.298+0000 7fedaa7fc640 1 -- 172.21.15.28:0/1281439321 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7feda4084660 con 0x7fedb40fe350 2024-07-03T15:43:12.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.298+0000 7fedb9e96640 1 --2- 172.21.15.28:0/1281439321 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7feda0067190 0x7feda0069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:43:12.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.299+0000 7fedb9e96640 1 --2- 172.21.15.28:0/1281439321 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7feda0067190 0x7feda0069640 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fedb0000c00 tx=0x7fedb0035040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:43:12.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.303+0000 7fedaa7fc640 1 -- 172.21.15.28:0/1281439321 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7feda404e250 con 0x7fedb40fe350 2024-07-03T15:43:12.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.659+0000 7fedbc0fa640 1 -- 172.21.15.28:0/1281439321 --> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fedb4115380 con 0x7feda0067190 2024-07-03T15:43:12.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.661+0000 7fedaa7fc640 1 -- 172.21.15.28:0/1281439321 <== mgr.14210 v2:172.21.15.28:6800/421057178 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7fedb4115380 con 0x7feda0067190 2024-07-03T15:43:12.665 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-07-03T15:43:12.665 INFO:teuthology.orchestra.run.smithi028.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2", 2024-07-03T15:43:12.665 INFO:teuthology.orchestra.run.smithi028.stdout: "in_progress": true, 2024-07-03T15:43:12.665 INFO:teuthology.orchestra.run.smithi028.stdout: "which": "Upgrading all daemon types on all hosts", 2024-07-03T15:43:12.665 INFO:teuthology.orchestra.run.smithi028.stdout: "services_complete": [], 2024-07-03T15:43:12.665 INFO:teuthology.orchestra.run.smithi028.stdout: "progress": "0/21 daemons upgraded", 2024-07-03T15:43:12.665 INFO:teuthology.orchestra.run.smithi028.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2 image on host smithi148", 2024-07-03T15:43:12.665 INFO:teuthology.orchestra.run.smithi028.stdout: "is_paused": false 2024-07-03T15:43:12.666 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-07-03T15:43:12.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.664+0000 7fedbc0fa640 1 -- 172.21.15.28:0/1281439321 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7feda0067190 msgr2=0x7feda0069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:12.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.664+0000 7fedbc0fa640 1 --2- 172.21.15.28:0/1281439321 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7feda0067190 0x7feda0069640 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fedb0000c00 tx=0x7fedb0035040 comp rx=0 tx=0).stop 2024-07-03T15:43:12.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.664+0000 7fedbc0fa640 1 -- 172.21.15.28:0/1281439321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedb40fe350 msgr2=0x7fedb4113b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:12.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.664+0000 7fedbc0fa640 1 --2- 172.21.15.28:0/1281439321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedb40fe350 0x7fedb4113b00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7feda400eb70 tx=0x7feda400b040 comp rx=0 tx=0).stop 2024-07-03T15:43:12.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.664+0000 7fedbc0fa640 1 -- 172.21.15.28:0/1281439321 shutdown_connections 2024-07-03T15:43:12.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.664+0000 7fedbc0fa640 1 --2- 172.21.15.28:0/1281439321 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7feda0067190 0x7feda0069640 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:12.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.664+0000 7fedbc0fa640 1 --2- 172.21.15.28:0/1281439321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedb40fe350 0x7fedb4113b00 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:12.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.664+0000 7fedbc0fa640 1 --2- 172.21.15.28:0/1281439321 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fedb40fda00 0x7fedb41135c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:12.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.664+0000 7fedbc0fa640 1 -- 172.21.15.28:0/1281439321 >> 172.21.15.28:0/1281439321 conn(0x7fedb40f61a0 msgr2=0x7fedb40f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:43:12.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.664+0000 7fedbc0fa640 1 -- 172.21.15.28:0/1281439321 shutdown_connections 2024-07-03T15:43:12.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.665+0000 7fedbc0fa640 1 -- 172.21.15.28:0/1281439321 wait complete. 2024-07-03T15:43:12.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:12 smithi148 ceph-mon[36056]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:43:12.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:12 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/1225967754' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:43:12.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:12 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/1536757007' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-03T15:43:12.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.785+0000 7f208521a640 1 -- 172.21.15.28:0/1692694423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20800fb230 msgr2=0x7f20800fb640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:12.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.785+0000 7f208521a640 1 --2- 172.21.15.28:0/1692694423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20800fb230 0x7f20800fb640 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2070004810 tx=0x7f2070009da0 comp rx=0 tx=0).stop 2024-07-03T15:43:12.789 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:12 smithi028 ceph-mon[29148]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:43:12.789 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:12 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/1225967754' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:43:12.789 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:12 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/1536757007' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-03T15:43:12.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.786+0000 7f208521a640 1 -- 172.21.15.28:0/1692694423 shutdown_connections 2024-07-03T15:43:12.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.786+0000 7f208521a640 1 --2- 172.21.15.28:0/1692694423 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f20800fc500 0x7f20800fc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:12.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.786+0000 7f208521a640 1 --2- 172.21.15.28:0/1692694423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20800fb230 0x7f20800fb640 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:12.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.786+0000 7f208521a640 1 -- 172.21.15.28:0/1692694423 >> 172.21.15.28:0/1692694423 conn(0x7f20800f6a10 msgr2=0x7f20800f8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:43:12.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.786+0000 7f208521a640 1 -- 172.21.15.28:0/1692694423 shutdown_connections 2024-07-03T15:43:12.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.786+0000 7f208521a640 1 -- 172.21.15.28:0/1692694423 wait complete. 2024-07-03T15:43:12.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.787+0000 7f208521a640 1 Processor -- start 2024-07-03T15:43:12.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.787+0000 7f208521a640 1 -- start start 2024-07-03T15:43:12.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.787+0000 7f208521a640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f20800fb230 0x7f2080106650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:43:12.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.787+0000 7f208521a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20800fc500 0x7f2080106b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:43:12.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.787+0000 7f208521a640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2080107190 con 0x7f20800fb230 2024-07-03T15:43:12.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.787+0000 7f208521a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2080107a50 con 0x7f20800fc500 2024-07-03T15:43:12.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.787+0000 7f2077fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20800fc500 0x7f2080106b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:43:12.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.788+0000 7f2077fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20800fc500 0x7f2080106b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.28:35014/0 (socket says 172.21.15.28:35014) 2024-07-03T15:43:12.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.788+0000 7f2077fff640 1 -- 172.21.15.28:0/1147982558 learned_addr learned my addr 172.21.15.28:0/1147982558 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:43:12.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.788+0000 7f2077fff640 1 -- 172.21.15.28:0/1147982558 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f20800fb230 msgr2=0x7f2080106650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:12.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.788+0000 7f2077fff640 1 --2- 172.21.15.28:0/1147982558 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f20800fb230 0x7f2080106650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:12.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.788+0000 7f2077fff640 1 -- 172.21.15.28:0/1147982558 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2070005040 con 0x7f20800fc500 2024-07-03T15:43:12.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.788+0000 7f2077fff640 1 --2- 172.21.15.28:0/1147982558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20800fc500 0x7f2080106b90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f206800eb40 tx=0x7f206800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:43:12.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.789+0000 7f20777fe640 1 -- 172.21.15.28:0/1147982558 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2068007c30 con 0x7f20800fc500 2024-07-03T15:43:12.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.789+0000 7f20777fe640 1 -- 172.21.15.28:0/1147982558 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f206800f040 con 0x7f20800fc500 2024-07-03T15:43:12.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.789+0000 7f20777fe640 1 -- 172.21.15.28:0/1147982558 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f206800cbd0 con 0x7f20800fc500 2024-07-03T15:43:12.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.789+0000 7f208521a640 1 -- 172.21.15.28:0/1147982558 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2080107c50 con 0x7f20800fc500 2024-07-03T15:43:12.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.790+0000 7f208521a640 1 -- 172.21.15.28:0/1147982558 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20801081f0 con 0x7f20800fc500 2024-07-03T15:43:12.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.792+0000 7f208521a640 1 -- 172.21.15.28:0/1147982558 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f208004b530 con 0x7f20800fc500 2024-07-03T15:43:12.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.792+0000 7f20777fe640 1 -- 172.21.15.28:0/1147982558 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f206800cd30 con 0x7f20800fc500 2024-07-03T15:43:12.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.792+0000 7f20777fe640 1 --2- 172.21.15.28:0/1147982558 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f206c0672a0 0x7f206c069750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:43:12.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.793+0000 7f207ed9d640 1 --2- 172.21.15.28:0/1147982558 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f206c0672a0 0x7f206c069750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:43:12.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.793+0000 7f20777fe640 1 -- 172.21.15.28:0/1147982558 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f2068084670 con 0x7f20800fc500 2024-07-03T15:43:12.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.793+0000 7f207ed9d640 1 --2- 172.21.15.28:0/1147982558 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f206c0672a0 0x7f206c069750 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2070004730 tx=0x7f2070009da0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:43:12.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:12.797+0000 7f20777fe640 1 -- 172.21.15.28:0/1147982558 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f206804e250 con 0x7f20800fc500 2024-07-03T15:43:13.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:13.074+0000 7f208521a640 1 -- 172.21.15.28:0/1147982558 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2080101ef0 con 0x7f20800fc500 2024-07-03T15:43:13.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:13.074+0000 7f20777fe640 1 -- 172.21.15.28:0/1147982558 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f2068055570 con 0x7f20800fc500 2024-07-03T15:43:13.078 INFO:teuthology.orchestra.run.smithi028.stdout:HEALTH_OK 2024-07-03T15:43:13.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:13.077+0000 7f208521a640 1 -- 172.21.15.28:0/1147982558 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f206c0672a0 msgr2=0x7f206c069750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:13.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:13.077+0000 7f208521a640 1 --2- 172.21.15.28:0/1147982558 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f206c0672a0 0x7f206c069750 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2070004730 tx=0x7f2070009da0 comp rx=0 tx=0).stop 2024-07-03T15:43:13.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:13.077+0000 7f208521a640 1 -- 172.21.15.28:0/1147982558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20800fc500 msgr2=0x7f2080106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:13.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:13.077+0000 7f208521a640 1 --2- 172.21.15.28:0/1147982558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20800fc500 0x7f2080106b90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f206800eb40 tx=0x7f206800b040 comp rx=0 tx=0).stop 2024-07-03T15:43:13.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:13.078+0000 7f208521a640 1 -- 172.21.15.28:0/1147982558 shutdown_connections 2024-07-03T15:43:13.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:13.078+0000 7f208521a640 1 --2- 172.21.15.28:0/1147982558 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f206c0672a0 0x7f206c069750 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:13.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:13.078+0000 7f208521a640 1 --2- 172.21.15.28:0/1147982558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20800fc500 0x7f2080106b90 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:13.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:13.078+0000 7f208521a640 1 --2- 172.21.15.28:0/1147982558 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f20800fb230 0x7f2080106650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:13.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:13.078+0000 7f208521a640 1 -- 172.21.15.28:0/1147982558 >> 172.21.15.28:0/1147982558 conn(0x7f20800f6a10 msgr2=0x7f20800f80c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:43:13.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:13.078+0000 7f208521a640 1 -- 172.21.15.28:0/1147982558 shutdown_connections 2024-07-03T15:43:13.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:13.078+0000 7f208521a640 1 -- 172.21.15.28:0/1147982558 wait complete. 2024-07-03T15:43:13.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:13 smithi148 ceph-mon[36056]: pgmap v219: 65 pgs: 65 active+clean; 601 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-07-03T15:43:13.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:13 smithi148 ceph-mon[36056]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:43:13.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:13 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/1147982558' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-03T15:43:13.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:13 smithi028 ceph-mon[29148]: pgmap v219: 65 pgs: 65 active+clean; 601 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-07-03T15:43:13.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:13 smithi028 ceph-mon[29148]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:43:13.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:13 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/1147982558' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-03T15:43:15.281 INFO:tasks.workunit.client.0.smithi028.stderr:Updating files: 96% (10663/11064) Updating files: 97% (10733/11064) Updating files: 98% (10843/11064) Updating files: 99% (10954/11064) Updating files: 100% (11064/11064) Updating files: 100% (11064/11064), done. 2024-07-03T15:43:15.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:15 smithi148 ceph-mon[36056]: pgmap v220: 65 pgs: 65 active+clean; 601 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 1.6 KiB/s wr, 2 op/s 2024-07-03T15:43:15.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:15 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:43:15.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:15 smithi028 ceph-mon[29148]: pgmap v220: 65 pgs: 65 active+clean; 601 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 1.6 KiB/s wr, 2 op/s 2024-07-03T15:43:15.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:15 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:43:16.667 INFO:tasks.workunit.client.0.smithi028.stderr:Note: switching to '47d23ed6cd9bd552d37bff09d8392220d7ba7b11'. 2024-07-03T15:43:16.667 INFO:tasks.workunit.client.0.smithi028.stderr: 2024-07-03T15:43:16.668 INFO:tasks.workunit.client.0.smithi028.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-07-03T15:43:16.668 INFO:tasks.workunit.client.0.smithi028.stderr:changes and commit them, and you can discard any commits you make in this 2024-07-03T15:43:16.668 INFO:tasks.workunit.client.0.smithi028.stderr:state without impacting any branches by switching back to a branch. 2024-07-03T15:43:16.668 INFO:tasks.workunit.client.0.smithi028.stderr: 2024-07-03T15:43:16.668 INFO:tasks.workunit.client.0.smithi028.stderr:If you want to create a new branch to retain commits you create, you may 2024-07-03T15:43:16.668 INFO:tasks.workunit.client.0.smithi028.stderr:do so (now or later) by using -c with the switch command. Example: 2024-07-03T15:43:16.668 INFO:tasks.workunit.client.0.smithi028.stderr: 2024-07-03T15:43:16.668 INFO:tasks.workunit.client.0.smithi028.stderr: git switch -c 2024-07-03T15:43:16.668 INFO:tasks.workunit.client.0.smithi028.stderr: 2024-07-03T15:43:16.668 INFO:tasks.workunit.client.0.smithi028.stderr:Or undo this operation with: 2024-07-03T15:43:16.668 INFO:tasks.workunit.client.0.smithi028.stderr: 2024-07-03T15:43:16.668 INFO:tasks.workunit.client.0.smithi028.stderr: git switch - 2024-07-03T15:43:16.668 INFO:tasks.workunit.client.0.smithi028.stderr: 2024-07-03T15:43:16.669 INFO:tasks.workunit.client.0.smithi028.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-07-03T15:43:16.669 INFO:tasks.workunit.client.0.smithi028.stderr: 2024-07-03T15:43:16.669 INFO:tasks.workunit.client.0.smithi028.stderr:HEAD is now at 47d23ed6cd9 qa: fix log errors for cephadm tests 2024-07-03T15:43:16.673 DEBUG:teuthology.orchestra.run.smithi028:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-07-03T15:43:16.803 INFO:tasks.workunit.client.0.smithi028.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-07-03T15:43:16.806 INFO:tasks.workunit.client.0.smithi028.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-07-03T15:43:16.806 INFO:tasks.workunit.client.0.smithi028.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-07-03T15:43:17.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:17 smithi148 ceph-mon[36056]: pgmap v221: 65 pgs: 65 active+clean; 12 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 683 KiB/s rd, 993 KiB/s wr, 61 op/s 2024-07-03T15:43:17.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:17 smithi028 ceph-mon[29148]: pgmap v221: 65 pgs: 65 active+clean; 12 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 683 KiB/s rd, 993 KiB/s wr, 61 op/s 2024-07-03T15:43:19.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:19 smithi148 ceph-mon[36056]: pgmap v222: 65 pgs: 65 active+clean; 25 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1024 KiB/s rd, 2.1 MiB/s wr, 107 op/s 2024-07-03T15:43:19.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:19 smithi028 ceph-mon[29148]: pgmap v222: 65 pgs: 65 active+clean; 25 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1024 KiB/s rd, 2.1 MiB/s wr, 107 op/s 2024-07-03T15:43:21.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:21 smithi148 ceph-mon[36056]: pgmap v223: 65 pgs: 65 active+clean; 41 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 232 op/s 2024-07-03T15:43:21.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:21 smithi028 ceph-mon[29148]: pgmap v223: 65 pgs: 65 active+clean; 41 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 232 op/s 2024-07-03T15:43:22.989 INFO:tasks.workunit.client.0.smithi028.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-07-03T15:43:23.318 INFO:tasks.workunit.client.0.smithi028.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-07-03T15:43:23.419 INFO:tasks.workunit.client.0.smithi028.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-07-03T15:43:23.421 INFO:tasks.workunit.client.0.smithi028.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-07-03T15:43:23.421 INFO:tasks.workunit.client.0.smithi028.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-07-03T15:43:23.482 INFO:tasks.workunit.client.0.smithi028.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-07-03T15:43:23.486 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-07-03T15:43:23.486 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-07-03T15:43:23.504 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-07-03T15:43:23.505 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-07-03T15:43:23.506 DEBUG:teuthology.orchestra.run.smithi028:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=47d23ed6cd9bd552d37bff09d8392220d7ba7b11 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-07-03T15:43:23.566 INFO:tasks.workunit.client.0.smithi028.stderr:+ mkdir -p fsstress 2024-07-03T15:43:23.568 INFO:tasks.workunit.client.0.smithi028.stderr:+ pushd fsstress 2024-07-03T15:43:23.569 INFO:tasks.workunit.client.0.smithi028.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-07-03T15:43:23.569 INFO:tasks.workunit.client.0.smithi028.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-07-03T15:43:23.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:23 smithi148 ceph-mon[36056]: pgmap v224: 65 pgs: 65 active+clean; 60 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 2.4 MiB/s rd, 5.1 MiB/s wr, 310 op/s 2024-07-03T15:43:23.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:23 smithi028 ceph-mon[29148]: pgmap v224: 65 pgs: 65 active+clean; 60 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 2.4 MiB/s rd, 5.1 MiB/s wr, 310 op/s 2024-07-03T15:43:24.930 INFO:tasks.workunit.client.1.smithi148.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-07-03T15:43:24.930 INFO:tasks.workunit.client.1.smithi148.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-07-03T15:43:24.931 INFO:tasks.workunit.client.1.smithi148.stderr:+ make 2024-07-03T15:43:25.178 INFO:tasks.workunit.client.1.smithi148.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-07-03T15:43:25.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:25 smithi148 ceph-mon[36056]: pgmap v225: 65 pgs: 65 active+clean; 63 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 2.4 MiB/s rd, 5.4 MiB/s wr, 309 op/s 2024-07-03T15:43:25.771 INFO:tasks.workunit.client.1.smithi148.stderr:++ readlink -f fsstress 2024-07-03T15:43:25.772 INFO:tasks.workunit.client.1.smithi148.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-07-03T15:43:25.772 INFO:tasks.workunit.client.1.smithi148.stderr:+ popd 2024-07-03T15:43:25.772 INFO:tasks.workunit.client.1.smithi148.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-07-03T15:43:25.773 INFO:tasks.workunit.client.1.smithi148.stderr:+ popd 2024-07-03T15:43:25.773 INFO:tasks.workunit.client.1.smithi148.stdout:~/cephtest/mnt.1/client.1/tmp 2024-07-03T15:43:25.773 INFO:tasks.workunit.client.1.smithi148.stderr:++ mktemp -d -p . 2024-07-03T15:43:25.774 INFO:tasks.workunit.client.1.smithi148.stderr:+ T=./tmp.T0EhYNVc6z 2024-07-03T15:43:25.774 INFO:tasks.workunit.client.1.smithi148.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.T0EhYNVc6z -l 1 -n 1000 -p 10 -v 2024-07-03T15:43:25.775 INFO:tasks.workunit.client.1.smithi148.stdout:seed = 1720630577 2024-07-03T15:43:25.777 INFO:tasks.workunit.client.1.smithi148.stdout:0/0: truncate - no filename 2024-07-03T15:43:25.777 INFO:tasks.workunit.client.1.smithi148.stdout:0/1: truncate - no filename 2024-07-03T15:43:25.777 INFO:tasks.workunit.client.1.smithi148.stdout:0/2: write - no filename 2024-07-03T15:43:25.777 INFO:tasks.workunit.client.1.smithi148.stdout:0/3: rename - no filename 2024-07-03T15:43:25.778 INFO:tasks.workunit.client.1.smithi148.stdout:0/4: truncate - no filename 2024-07-03T15:43:25.778 INFO:tasks.workunit.client.1.smithi148.stdout:0/5: rename - no filename 2024-07-03T15:43:25.778 INFO:tasks.workunit.client.1.smithi148.stdout:0/6: link - no file 2024-07-03T15:43:25.778 INFO:tasks.workunit.client.1.smithi148.stdout:0/7: fdatasync - no filename 2024-07-03T15:43:25.778 INFO:tasks.workunit.client.1.smithi148.stdout:0/8: read - no filename 2024-07-03T15:43:25.778 INFO:tasks.workunit.client.1.smithi148.stdout:0/9: unlink - no file 2024-07-03T15:43:25.779 INFO:tasks.workunit.client.1.smithi148.stdout:0/10: dwrite - no filename 2024-07-03T15:43:25.782 INFO:tasks.workunit.client.1.smithi148.stdout:1/0: rename - no filename 2024-07-03T15:43:25.782 INFO:tasks.workunit.client.1.smithi148.stdout:1/1: chown . 7789594 1 2024-07-03T15:43:25.782 INFO:tasks.workunit.client.1.smithi148.stdout:1/2: fdatasync - no filename 2024-07-03T15:43:25.783 INFO:tasks.workunit.client.1.smithi148.stdout:0/11: symlink l0 0 2024-07-03T15:43:25.783 INFO:tasks.workunit.client.1.smithi148.stdout:0/12: read - no filename 2024-07-03T15:43:25.783 INFO:tasks.workunit.client.1.smithi148.stdout:2/0: link - no file 2024-07-03T15:43:25.783 INFO:tasks.workunit.client.1.smithi148.stdout:1/3: mkdir d0 0 2024-07-03T15:43:25.783 INFO:tasks.workunit.client.1.smithi148.stdout:1/4: readlink - no filename 2024-07-03T15:43:25.783 INFO:tasks.workunit.client.1.smithi148.stdout:1/5: truncate - no filename 2024-07-03T15:43:25.784 INFO:tasks.workunit.client.1.smithi148.stdout:1/6: fdatasync - no filename 2024-07-03T15:43:25.784 INFO:tasks.workunit.client.1.smithi148.stdout:1/7: stat d0 0 2024-07-03T15:43:25.784 INFO:tasks.workunit.client.1.smithi148.stdout:0/13: rename l0 to l1 0 2024-07-03T15:43:25.784 INFO:tasks.workunit.client.1.smithi148.stdout:0/14: chown l1 14144782 1 2024-07-03T15:43:25.784 INFO:tasks.workunit.client.1.smithi148.stdout:3/0: chown . 146890719 1 2024-07-03T15:43:25.785 INFO:tasks.workunit.client.1.smithi148.stdout:3/1: dwrite - no filename 2024-07-03T15:43:25.785 INFO:tasks.workunit.client.1.smithi148.stdout:2/1: creat f0 x:0 0 0 2024-07-03T15:43:25.785 INFO:tasks.workunit.client.1.smithi148.stdout:2/2: creat f1 x:0 0 0 2024-07-03T15:43:25.785 INFO:tasks.workunit.client.1.smithi148.stdout:2/3: write f1 [46541,22] 0 2024-07-03T15:43:25.785 INFO:tasks.workunit.client.1.smithi148.stdout:2/4: truncate f0 289212 0 2024-07-03T15:43:25.785 INFO:tasks.workunit.client.1.smithi148.stdout:2/5: readlink - no filename 2024-07-03T15:43:25.787 INFO:tasks.workunit.client.1.smithi148.stdout:1/8: mknod d0/c1 0 2024-07-03T15:43:25.787 INFO:tasks.workunit.client.1.smithi148.stdout:0/15: creat f2 x:0 0 0 2024-07-03T15:43:25.787 INFO:tasks.workunit.client.1.smithi148.stdout:4/0: chown . 66 1 2024-07-03T15:43:25.787 INFO:tasks.workunit.client.1.smithi148.stdout:4/1: dwrite - no filename 2024-07-03T15:43:25.788 INFO:tasks.workunit.client.1.smithi148.stdout:4/2: dread - no filename 2024-07-03T15:43:25.788 INFO:tasks.workunit.client.1.smithi148.stdout:4/3: rename - no filename 2024-07-03T15:43:25.788 INFO:tasks.workunit.client.1.smithi148.stdout:4/4: chown . 150459 1 2024-07-03T15:43:25.788 INFO:tasks.workunit.client.1.smithi148.stdout:3/2: creat f0 x:0 0 0 2024-07-03T15:43:25.799 INFO:tasks.workunit.client.1.smithi148.stdout:1/9: mkdir d0/d2 0 2024-07-03T15:43:25.801 INFO:tasks.workunit.client.1.smithi148.stdout:5/0: fsync - no filename 2024-07-03T15:43:25.801 INFO:tasks.workunit.client.1.smithi148.stdout:5/1: dwrite - no filename 2024-07-03T15:43:25.801 INFO:tasks.workunit.client.1.smithi148.stdout:5/2: fdatasync - no filename 2024-07-03T15:43:25.802 INFO:tasks.workunit.client.1.smithi148.stdout:0/16: mknod c3 0 2024-07-03T15:43:25.802 INFO:tasks.workunit.client.1.smithi148.stdout:4/5: mkdir d0 0 2024-07-03T15:43:25.804 INFO:tasks.workunit.client.1.smithi148.stdout:1/10: mknod d0/d2/c3 0 2024-07-03T15:43:25.804 INFO:tasks.workunit.client.1.smithi148.stdout:1/11: dread - no filename 2024-07-03T15:43:25.804 INFO:tasks.workunit.client.1.smithi148.stdout:1/12: dwrite - no filename 2024-07-03T15:43:25.804 INFO:tasks.workunit.client.1.smithi148.stdout:1/13: chown d0/c1 24 1 2024-07-03T15:43:25.804 INFO:tasks.workunit.client.1.smithi148.stdout:1/14: dwrite - no filename 2024-07-03T15:43:25.805 INFO:tasks.workunit.client.1.smithi148.stdout:1/15: write - no filename 2024-07-03T15:43:25.805 INFO:tasks.workunit.client.1.smithi148.stdout:1/16: dwrite - no filename 2024-07-03T15:43:25.805 INFO:tasks.workunit.client.1.smithi148.stdout:5/3: symlink l0 0 2024-07-03T15:43:25.806 INFO:tasks.workunit.client.1.smithi148.stdout:0/17: stat l1 0 2024-07-03T15:43:25.806 INFO:tasks.workunit.client.1.smithi148.stdout:0/18: creat f4 x:0 0 0 2024-07-03T15:43:25.807 INFO:tasks.workunit.client.1.smithi148.stdout:4/6: mknod d0/c1 0 2024-07-03T15:43:25.807 INFO:tasks.workunit.client.1.smithi148.stdout:4/7: write - no filename 2024-07-03T15:43:25.809 INFO:tasks.workunit.client.1.smithi148.stdout:1/17: symlink d0/d2/l4 0 2024-07-03T15:43:25.809 INFO:tasks.workunit.client.1.smithi148.stdout:1/18: fdatasync - no filename 2024-07-03T15:43:25.809 INFO:tasks.workunit.client.1.smithi148.stdout:1/19: readlink d0/d2/l4 0 2024-07-03T15:43:25.812 INFO:tasks.workunit.client.1.smithi148.stdout:6/0: mknod c0 0 2024-07-03T15:43:25.812 INFO:tasks.workunit.client.1.smithi148.stdout:5/4: symlink l1 0 2024-07-03T15:43:25.813 INFO:tasks.workunit.client.1.smithi148.stdout:0/19: rename c3 to c5 0 2024-07-03T15:43:25.816 INFO:tasks.workunit.client.1.smithi148.stdout:6/1: creat f1 x:0 0 0 2024-07-03T15:43:25.816 INFO:tasks.workunit.client.1.smithi148.stdout:6/2: dread - f1 zero size 2024-07-03T15:43:25.817 INFO:tasks.workunit.client.1.smithi148.stdout:6/3: write f1 [260671,49869] 0 2024-07-03T15:43:25.817 INFO:tasks.workunit.client.1.smithi148.stdout:6/4: write f1 [806417,83542] 0 2024-07-03T15:43:25.817 INFO:tasks.workunit.client.1.smithi148.stdout:6/5: stat f1 0 2024-07-03T15:43:25.819 INFO:tasks.workunit.client.1.smithi148.stdout:7/0: creat f0 x:0 0 0 2024-07-03T15:43:25.819 INFO:tasks.workunit.client.1.smithi148.stdout:5/5: creat f2 x:0 0 0 2024-07-03T15:43:25.820 INFO:tasks.workunit.client.1.smithi148.stdout:5/6: creat f3 x:0 0 0 2024-07-03T15:43:25.820 INFO:tasks.workunit.client.1.smithi148.stdout:5/7: creat f4 x:0 0 0 2024-07-03T15:43:25.820 INFO:tasks.workunit.client.1.smithi148.stdout:5/8: dread - f2 zero size 2024-07-03T15:43:25.820 INFO:tasks.workunit.client.1.smithi148.stdout:4/8: rmdir d0 39 2024-07-03T15:43:25.820 INFO:tasks.workunit.client.1.smithi148.stdout:4/9: fdatasync - no filename 2024-07-03T15:43:25.820 INFO:tasks.workunit.client.1.smithi148.stdout:4/10: readlink - no filename 2024-07-03T15:43:25.820 INFO:tasks.workunit.client.1.smithi148.stdout:1/20: creat d0/d2/f5 x:0 0 0 2024-07-03T15:43:25.821 INFO:tasks.workunit.client.1.smithi148.stdout:6/6: mknod c2 0 2024-07-03T15:43:25.822 INFO:tasks.workunit.client.1.smithi148.stdout:7/1: symlink l1 0 2024-07-03T15:43:25.822 INFO:tasks.workunit.client.1.smithi148.stdout:7/2: truncate f0 725948 0 2024-07-03T15:43:25.823 INFO:tasks.workunit.client.1.smithi148.stdout:4/11: symlink d0/l2 0 2024-07-03T15:43:25.823 INFO:tasks.workunit.client.1.smithi148.stdout:4/12: readlink d0/l2 0 2024-07-03T15:43:25.823 INFO:tasks.workunit.client.1.smithi148.stdout:4/13: write - no filename 2024-07-03T15:43:25.825 INFO:tasks.workunit.client.1.smithi148.stdout:4/14: creat d0/f3 x:0 0 0 2024-07-03T15:43:25.825 INFO:tasks.workunit.client.1.smithi148.stdout:4/15: dread - d0/f3 zero size 2024-07-03T15:43:25.828 INFO:tasks.workunit.client.1.smithi148.stdout:8/0: creat f0 x:0 0 0 2024-07-03T15:43:25.828 INFO:tasks.workunit.client.1.smithi148.stdout:9/0: chown . 32 1 2024-07-03T15:43:25.828 INFO:tasks.workunit.client.1.smithi148.stdout:9/1: fdatasync - no filename 2024-07-03T15:43:25.831 INFO:tasks.workunit.client.1.smithi148.stdout:9/2: mkdir d0 0 2024-07-03T15:43:25.831 INFO:tasks.workunit.client.1.smithi148.stdout:9/3: truncate - no filename 2024-07-03T15:43:25.831 INFO:tasks.workunit.client.1.smithi148.stdout:9/4: fdatasync - no filename 2024-07-03T15:43:25.831 INFO:tasks.workunit.client.1.smithi148.stdout:9/5: dwrite - no filename 2024-07-03T15:43:25.831 INFO:tasks.workunit.client.1.smithi148.stdout:9/6: rename d0 to d0/d1 22 2024-07-03T15:43:25.831 INFO:tasks.workunit.client.1.smithi148.stdout:9/7: dwrite - no filename 2024-07-03T15:43:25.831 INFO:tasks.workunit.client.1.smithi148.stdout:9/8: fdatasync - no filename 2024-07-03T15:43:25.831 INFO:tasks.workunit.client.1.smithi148.stdout:9/9: truncate - no filename 2024-07-03T15:43:25.831 INFO:tasks.workunit.client.1.smithi148.stdout:9/10: read - no filename 2024-07-03T15:43:25.832 INFO:tasks.workunit.client.1.smithi148.stdout:9/11: fdatasync - no filename 2024-07-03T15:43:25.832 INFO:tasks.workunit.client.1.smithi148.stdout:9/12: truncate - no filename 2024-07-03T15:43:25.832 INFO:tasks.workunit.client.1.smithi148.stdout:9/13: mknod d0/c2 0 2024-07-03T15:43:25.834 INFO:tasks.workunit.client.1.smithi148.stdout:9/14: creat d0/f3 x:0 0 0 2024-07-03T15:43:25.835 INFO:tasks.workunit.client.1.smithi148.stdout:9/15: rename d0/c2 to d0/c4 0 2024-07-03T15:43:25.837 INFO:tasks.workunit.client.1.smithi148.stdout:9/16: symlink d0/l5 0 2024-07-03T15:43:25.837 INFO:tasks.workunit.client.1.smithi148.stdout:9/17: read - d0/f3 zero size 2024-07-03T15:43:25.837 INFO:tasks.workunit.client.1.smithi148.stdout:9/18: creat d0/f6 x:0 0 0 2024-07-03T15:43:25.856 INFO:tasks.workunit.client.0.smithi028.stderr:+ tar xzf ltp-full.tgz 2024-07-03T15:43:25.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:25 smithi028 ceph-mon[29148]: pgmap v225: 65 pgs: 65 active+clean; 63 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 2.4 MiB/s rd, 5.4 MiB/s wr, 309 op/s 2024-07-03T15:43:26.188 INFO:tasks.workunit.client.1.smithi148.stdout:3/3: dwrite f0 [0,4194304] 0 2024-07-03T15:43:26.195 INFO:tasks.workunit.client.1.smithi148.stdout:1/21: fsync d0/d2/f5 0 2024-07-03T15:43:26.208 INFO:tasks.workunit.client.1.smithi148.stdout:8/1: getdents . 0 2024-07-03T15:43:26.214 INFO:tasks.workunit.client.1.smithi148.stdout:9/19: unlink d0/c4 0 2024-07-03T15:43:26.277 INFO:tasks.workunit.client.1.smithi148.stdout:4/16: dwrite d0/f3 [0,4194304] 0 2024-07-03T15:43:26.277 INFO:tasks.workunit.client.1.smithi148.stdout:4/17: fsync d0/f3 0 2024-07-03T15:43:26.448 INFO:tasks.workunit.client.1.smithi148.stdout:7/3: dwrite f0 [0,4194304] 0 2024-07-03T15:43:26.448 INFO:tasks.workunit.client.1.smithi148.stdout:7/4: readlink l1 0 2024-07-03T15:43:26.448 INFO:tasks.workunit.client.1.smithi148.stdout:7/5: truncate f0 4789360 0 2024-07-03T15:43:26.935 INFO:tasks.workunit.client.1.smithi148.stdout:7/6: dread f0 [0,4194304] 0 2024-07-03T15:43:26.940 INFO:tasks.workunit.client.1.smithi148.stdout:7/7: dread f0 [0,4194304] 0 2024-07-03T15:43:26.940 INFO:tasks.workunit.client.1.smithi148.stdout:7/8: creat f2 x:0 0 0 2024-07-03T15:43:26.940 INFO:tasks.workunit.client.1.smithi148.stdout:7/9: creat f3 x:0 0 0 2024-07-03T15:43:26.941 INFO:tasks.workunit.client.1.smithi148.stdout:7/10: fsync f0 0 2024-07-03T15:43:26.941 INFO:tasks.workunit.client.1.smithi148.stdout:7/11: rmdir - no directory 2024-07-03T15:43:26.941 INFO:tasks.workunit.client.1.smithi148.stdout:7/12: chown f2 3393240 1 2024-07-03T15:43:26.951 INFO:tasks.workunit.client.1.smithi148.stdout:7/13: read f0 [504792,5844] 0 2024-07-03T15:43:26.951 INFO:tasks.workunit.client.1.smithi148.stdout:7/14: chown f3 1692708 1 2024-07-03T15:43:26.951 INFO:tasks.workunit.client.1.smithi148.stdout:7/15: chown f2 16065868 1 2024-07-03T15:43:26.957 INFO:tasks.workunit.client.1.smithi148.stdout:7/16: dread f0 [0,4194304] 0 2024-07-03T15:43:27.040 INFO:tasks.workunit.client.1.smithi148.stdout:6/7: dread f1 [0,4194304] 0 2024-07-03T15:43:27.057 INFO:tasks.workunit.client.1.smithi148.stdout:8/2: dwrite f0 [0,4194304] 0 2024-07-03T15:43:27.058 INFO:tasks.workunit.client.1.smithi148.stdout:4/18: dread d0/f3 [0,4194304] 0 2024-07-03T15:43:27.058 INFO:tasks.workunit.client.1.smithi148.stdout:4/19: stat d0/c1 0 2024-07-03T15:43:27.066 INFO:tasks.workunit.client.1.smithi148.stdout:8/3: truncate f0 4467169 0 2024-07-03T15:43:27.079 INFO:tasks.workunit.client.1.smithi148.stdout:3/4: read f0 [4061093,34486] 0 2024-07-03T15:43:27.079 INFO:tasks.workunit.client.1.smithi148.stdout:3/5: chown f0 0 1 2024-07-03T15:43:27.682 INFO:tasks.workunit.client.1.smithi148.stdout:1/22: mkdir d0/d2/d6 0 2024-07-03T15:43:27.698 INFO:tasks.workunit.client.1.smithi148.stdout:1/23: dwrite d0/d2/f5 [0,4194304] 0 2024-07-03T15:43:27.704 INFO:tasks.workunit.client.1.smithi148.stdout:1/24: dread d0/d2/f5 [0,4194304] 0 2024-07-03T15:43:27.712 INFO:tasks.workunit.client.1.smithi148.stdout:9/20: unlink d0/f3 0 2024-07-03T15:43:27.712 INFO:tasks.workunit.client.1.smithi148.stdout:9/21: write d0/f6 [764996,60562] 0 2024-07-03T15:43:27.712 INFO:tasks.workunit.client.1.smithi148.stdout:9/22: chown d0/l5 240865 1 2024-07-03T15:43:27.717 INFO:tasks.workunit.client.1.smithi148.stdout:1/25: dwrite d0/d2/f5 [0,4194304] 0 2024-07-03T15:43:27.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:27 smithi148 ceph-mon[36056]: pgmap v226: 65 pgs: 65 active+clean; 72 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 3.3 MiB/s rd, 6.2 MiB/s wr, 317 op/s 2024-07-03T15:43:27.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:27 smithi028 ceph-mon[29148]: pgmap v226: 65 pgs: 65 active+clean; 72 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 3.3 MiB/s rd, 6.2 MiB/s wr, 317 op/s 2024-07-03T15:43:29.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:29 smithi148 ceph-mon[36056]: pgmap v227: 65 pgs: 65 active+clean; 79 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.7 MiB/s rd, 5.9 MiB/s wr, 326 op/s 2024-07-03T15:43:29.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:29 smithi028 ceph-mon[29148]: pgmap v227: 65 pgs: 65 active+clean; 79 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.7 MiB/s rd, 5.9 MiB/s wr, 326 op/s 2024-07-03T15:43:30.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:30 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:43:30.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:30 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:43:30.904 INFO:tasks.workunit.client.1.smithi148.stdout:6/8: mkdir d3 0 2024-07-03T15:43:31.001 INFO:tasks.workunit.client.1.smithi148.stdout:8/4: symlink l1 0 2024-07-03T15:43:31.001 INFO:tasks.workunit.client.1.smithi148.stdout:8/5: chown l1 118 1 2024-07-03T15:43:31.144 INFO:tasks.workunit.client.1.smithi148.stdout:9/23: link d0/f6 d0/f7 0 2024-07-03T15:43:31.144 INFO:tasks.workunit.client.1.smithi148.stdout:9/24: readlink d0/l5 0 2024-07-03T15:43:31.144 INFO:tasks.workunit.client.1.smithi148.stdout:9/25: chown d0/l5 4 1 2024-07-03T15:43:31.145 INFO:tasks.workunit.client.1.smithi148.stdout:1/26: getdents d0 0 2024-07-03T15:43:31.152 INFO:tasks.workunit.client.1.smithi148.stdout:1/27: dread d0/d2/f5 [0,4194304] 0 2024-07-03T15:43:31.152 INFO:tasks.workunit.client.1.smithi148.stdout:1/28: stat d0/d2/l4 0 2024-07-03T15:43:31.152 INFO:tasks.workunit.client.1.smithi148.stdout:1/29: chown d0/d2 501189 1 2024-07-03T15:43:31.682 INFO:tasks.workunit.client.1.smithi148.stdout:7/17: fsync f2 0 2024-07-03T15:43:31.683 INFO:tasks.workunit.client.1.smithi148.stdout:7/18: chown f3 2789 1 2024-07-03T15:43:31.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:31 smithi148 ceph-mon[36056]: pgmap v228: 65 pgs: 65 active+clean; 107 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 3.3 MiB/s rd, 7.1 MiB/s wr, 529 op/s 2024-07-03T15:43:31.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:31 smithi028 ceph-mon[29148]: pgmap v228: 65 pgs: 65 active+clean; 107 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 3.3 MiB/s rd, 7.1 MiB/s wr, 529 op/s 2024-07-03T15:43:31.939 INFO:tasks.workunit.client.1.smithi148.stdout:6/9: mknod d3/c4 0 2024-07-03T15:43:31.940 INFO:tasks.workunit.client.1.smithi148.stdout:8/6: rename f0 to f2 0 2024-07-03T15:43:31.941 INFO:tasks.workunit.client.1.smithi148.stdout:9/26: mkdir d0/d8 0 2024-07-03T15:43:31.944 INFO:tasks.workunit.client.1.smithi148.stdout:1/30: link d0/d2/c3 d0/d2/c7 0 2024-07-03T15:43:31.944 INFO:tasks.workunit.client.1.smithi148.stdout:1/31: truncate d0/d2/f5 4441389 0 2024-07-03T15:43:31.944 INFO:tasks.workunit.client.1.smithi148.stdout:1/32: chown d0/d2/d6 13 1 2024-07-03T15:43:31.944 INFO:tasks.workunit.client.1.smithi148.stdout:9/27: dread d0/f7 [0,4194304] 0 2024-07-03T15:43:31.944 INFO:tasks.workunit.client.1.smithi148.stdout:6/10: symlink d3/l5 0 2024-07-03T15:43:31.945 INFO:tasks.workunit.client.1.smithi148.stdout:3/6: truncate f0 2365178 0 2024-07-03T15:43:31.947 INFO:tasks.workunit.client.1.smithi148.stdout:7/19: getdents . 0 2024-07-03T15:43:31.948 INFO:tasks.workunit.client.1.smithi148.stdout:6/11: mknod d3/c6 0 2024-07-03T15:43:31.948 INFO:tasks.workunit.client.1.smithi148.stdout:6/12: chown d3/c6 109 1 2024-07-03T15:43:31.948 INFO:tasks.workunit.client.1.smithi148.stdout:6/13: write f1 [1849084,58363] 0 2024-07-03T15:43:31.948 INFO:tasks.workunit.client.1.smithi148.stdout:6/14: write f1 [396461,78982] 0 2024-07-03T15:43:31.951 INFO:tasks.workunit.client.1.smithi148.stdout:7/20: rename l1 to l4 0 2024-07-03T15:43:31.953 INFO:tasks.workunit.client.1.smithi148.stdout:6/15: fsync f1 0 2024-07-03T15:43:31.953 INFO:tasks.workunit.client.1.smithi148.stdout:6/16: chown c2 0 1 2024-07-03T15:43:31.954 INFO:tasks.workunit.client.1.smithi148.stdout:6/17: write f1 [871100,35229] 0 2024-07-03T15:43:31.957 INFO:tasks.workunit.client.1.smithi148.stdout:7/21: dread f0 [0,4194304] 0 2024-07-03T15:43:31.959 INFO:tasks.workunit.client.1.smithi148.stdout:7/22: mkdir d5 0 2024-07-03T15:43:31.960 INFO:tasks.workunit.client.1.smithi148.stdout:9/28: dwrite d0/f6 [0,4194304] 0 2024-07-03T15:43:31.960 INFO:tasks.workunit.client.1.smithi148.stdout:9/29: symlink d0/d8/l9 0 2024-07-03T15:43:31.961 INFO:tasks.workunit.client.1.smithi148.stdout:9/30: rmdir d0/d8 39 2024-07-03T15:43:31.964 INFO:tasks.workunit.client.1.smithi148.stdout:7/23: dread f0 [0,4194304] 0 2024-07-03T15:43:31.964 INFO:tasks.workunit.client.1.smithi148.stdout:7/24: readlink l4 0 2024-07-03T15:43:31.965 INFO:tasks.workunit.client.1.smithi148.stdout:7/25: read - f3 zero size 2024-07-03T15:43:31.965 INFO:tasks.workunit.client.1.smithi148.stdout:7/26: fsync f0 0 2024-07-03T15:43:31.965 INFO:tasks.workunit.client.1.smithi148.stdout:7/27: truncate f2 571641 0 2024-07-03T15:43:31.967 INFO:tasks.workunit.client.1.smithi148.stdout:7/28: rename f3 to d5/f6 0 2024-07-03T15:43:31.967 INFO:tasks.workunit.client.1.smithi148.stdout:7/29: truncate f2 1537156 0 2024-07-03T15:43:31.968 INFO:tasks.workunit.client.1.smithi148.stdout:7/30: unlink f0 0 2024-07-03T15:43:31.969 INFO:tasks.workunit.client.1.smithi148.stdout:9/31: dread d0/f6 [0,4194304] 0 2024-07-03T15:43:31.969 INFO:tasks.workunit.client.1.smithi148.stdout:7/31: creat d5/f7 x:0 0 0 2024-07-03T15:43:31.970 INFO:tasks.workunit.client.1.smithi148.stdout:9/32: mknod d0/d8/ca 0 2024-07-03T15:43:31.971 INFO:tasks.workunit.client.1.smithi148.stdout:7/32: mkdir d5/d8 0 2024-07-03T15:43:32.705 INFO:tasks.workunit.client.1.smithi148.stdout:8/7: getdents . 0 2024-07-03T15:43:32.706 INFO:tasks.workunit.client.1.smithi148.stdout:8/8: rename l1 to l3 0 2024-07-03T15:43:32.708 INFO:tasks.workunit.client.1.smithi148.stdout:1/33: getdents d0/d2 0 2024-07-03T15:43:32.708 INFO:tasks.workunit.client.1.smithi148.stdout:1/34: fsync d0/d2/f5 0 2024-07-03T15:43:32.708 INFO:tasks.workunit.client.1.smithi148.stdout:1/35: chown d0 12087000 1 2024-07-03T15:43:32.708 INFO:tasks.workunit.client.1.smithi148.stdout:1/36: truncate d0/d2/f5 4841668 0 2024-07-03T15:43:32.709 INFO:tasks.workunit.client.1.smithi148.stdout:1/37: creat d0/f8 x:0 0 0 2024-07-03T15:43:32.710 INFO:tasks.workunit.client.1.smithi148.stdout:1/38: dread d0/d2/f5 [4194304,4194304] 0 2024-07-03T15:43:32.711 INFO:tasks.workunit.client.1.smithi148.stdout:6/18: getdents d3 0 2024-07-03T15:43:32.714 INFO:tasks.workunit.client.1.smithi148.stdout:6/19: creat d3/f7 x:0 0 0 2024-07-03T15:43:32.716 INFO:tasks.workunit.client.1.smithi148.stdout:6/20: write f1 [1310189,3544] 0 2024-07-03T15:43:32.716 INFO:tasks.workunit.client.1.smithi148.stdout:6/21: creat d3/f8 x:0 0 0 2024-07-03T15:43:32.716 INFO:tasks.workunit.client.1.smithi148.stdout:6/22: write f1 [85651,22581] 0 2024-07-03T15:43:32.720 INFO:tasks.workunit.client.1.smithi148.stdout:6/23: link d3/c4 d3/c9 0 2024-07-03T15:43:32.720 INFO:tasks.workunit.client.1.smithi148.stdout:6/24: creat d3/fa x:0 0 0 2024-07-03T15:43:32.722 INFO:tasks.workunit.client.1.smithi148.stdout:9/33: rename d0/d8 to d0/db 0 2024-07-03T15:43:32.725 INFO:tasks.workunit.client.1.smithi148.stdout:8/9: dwrite f2 [0,4194304] 0 2024-07-03T15:43:32.726 INFO:tasks.workunit.client.1.smithi148.stdout:8/10: mkdir d4 0 2024-07-03T15:43:32.727 INFO:tasks.workunit.client.1.smithi148.stdout:8/11: symlink d4/l5 0 2024-07-03T15:43:32.727 INFO:tasks.workunit.client.1.smithi148.stdout:8/12: fdatasync f2 0 2024-07-03T15:43:32.728 INFO:tasks.workunit.client.1.smithi148.stdout:8/13: mkdir d4/d6 0 2024-07-03T15:43:32.729 INFO:tasks.workunit.client.1.smithi148.stdout:1/39: dwrite d0/d2/f5 [4194304,4194304] 0 2024-07-03T15:43:32.731 INFO:tasks.workunit.client.1.smithi148.stdout:1/40: rename d0/d2/l4 to d0/d2/l9 0 2024-07-03T15:43:32.732 INFO:tasks.workunit.client.1.smithi148.stdout:1/41: link d0/f8 d0/d2/d6/fa 0 2024-07-03T15:43:32.735 INFO:tasks.workunit.client.1.smithi148.stdout:1/42: mknod d0/cb 0 2024-07-03T15:43:32.735 INFO:tasks.workunit.client.1.smithi148.stdout:1/43: chown d0/f8 958 1 2024-07-03T15:43:32.737 INFO:tasks.workunit.client.1.smithi148.stdout:6/25: dwrite d3/fa [0,4194304] 0 2024-07-03T15:43:32.737 INFO:tasks.workunit.client.1.smithi148.stdout:1/44: link d0/cb d0/d2/cc 0 2024-07-03T15:43:32.737 INFO:tasks.workunit.client.1.smithi148.stdout:1/45: chown d0/d2/f5 1333670 1 2024-07-03T15:43:32.739 INFO:tasks.workunit.client.1.smithi148.stdout:6/26: mknod d3/cb 0 2024-07-03T15:43:32.740 INFO:tasks.workunit.client.1.smithi148.stdout:6/27: write f1 [746650,131017] 0 2024-07-03T15:43:32.740 INFO:tasks.workunit.client.1.smithi148.stdout:6/28: creat d3/fc x:0 0 0 2024-07-03T15:43:32.741 INFO:tasks.workunit.client.1.smithi148.stdout:1/46: creat d0/d2/fd x:0 0 0 2024-07-03T15:43:32.743 INFO:tasks.workunit.client.1.smithi148.stdout:6/29: symlink d3/ld 0 2024-07-03T15:43:32.743 INFO:tasks.workunit.client.1.smithi148.stdout:6/30: write d3/fc [288254,118535] 0 2024-07-03T15:43:32.743 INFO:tasks.workunit.client.1.smithi148.stdout:6/31: stat d3/ld 0 2024-07-03T15:43:32.743 INFO:tasks.workunit.client.1.smithi148.stdout:6/32: chown c2 11817112 1 2024-07-03T15:43:32.744 INFO:tasks.workunit.client.1.smithi148.stdout:1/47: mkdir d0/de 0 2024-07-03T15:43:32.744 INFO:tasks.workunit.client.1.smithi148.stdout:1/48: readlink d0/d2/l9 0 2024-07-03T15:43:32.745 INFO:tasks.workunit.client.1.smithi148.stdout:1/49: symlink d0/de/lf 0 2024-07-03T15:43:32.746 INFO:tasks.workunit.client.1.smithi148.stdout:8/14: dwrite f2 [0,4194304] 0 2024-07-03T15:43:32.746 INFO:tasks.workunit.client.1.smithi148.stdout:1/50: rename d0/d2/f5 to d0/d2/f10 0 2024-07-03T15:43:32.748 INFO:tasks.workunit.client.1.smithi148.stdout:8/15: link f2 d4/d6/f7 0 2024-07-03T15:43:32.763 INFO:tasks.workunit.client.1.smithi148.stdout:1/51: dwrite d0/f8 [0,4194304] 0 2024-07-03T15:43:32.765 INFO:tasks.workunit.client.1.smithi148.stdout:6/33: dwrite d3/fc [0,4194304] 0 2024-07-03T15:43:32.765 INFO:tasks.workunit.client.1.smithi148.stdout:6/34: write d3/f8 [451982,114062] 0 2024-07-03T15:43:32.766 INFO:tasks.workunit.client.1.smithi148.stdout:1/52: mkdir d0/d2/d11 0 2024-07-03T15:43:32.766 INFO:tasks.workunit.client.1.smithi148.stdout:1/53: fdatasync d0/f8 0 2024-07-03T15:43:32.766 INFO:tasks.workunit.client.1.smithi148.stdout:1/54: creat d0/f12 x:0 0 0 2024-07-03T15:43:32.767 INFO:tasks.workunit.client.1.smithi148.stdout:6/35: mkdir d3/de 0 2024-07-03T15:43:32.769 INFO:tasks.workunit.client.1.smithi148.stdout:6/36: rmdir d3/de 0 2024-07-03T15:43:32.770 INFO:tasks.workunit.client.1.smithi148.stdout:6/37: link d3/ld d3/lf 0 2024-07-03T15:43:32.770 INFO:tasks.workunit.client.1.smithi148.stdout:6/38: write d3/f7 [224019,58755] 0 2024-07-03T15:43:32.781 INFO:tasks.workunit.client.1.smithi148.stdout:1/55: dwrite d0/d2/d6/fa [0,4194304] 0 2024-07-03T15:43:32.781 INFO:tasks.workunit.client.1.smithi148.stdout:1/56: dread - d0/f12 zero size 2024-07-03T15:43:32.798 INFO:tasks.workunit.client.1.smithi148.stdout:6/39: dwrite f1 [0,4194304] 0 2024-07-03T15:43:32.798 INFO:tasks.workunit.client.1.smithi148.stdout:1/57: dwrite d0/d2/d6/fa [0,4194304] 0 2024-07-03T15:43:32.799 INFO:tasks.workunit.client.1.smithi148.stdout:6/40: symlink d3/l10 0 2024-07-03T15:43:32.800 INFO:tasks.workunit.client.1.smithi148.stdout:6/41: read d3/f8 [21591,43970] 0 2024-07-03T15:43:32.802 INFO:tasks.workunit.client.1.smithi148.stdout:6/42: link d3/cb d3/c11 0 2024-07-03T15:43:32.802 INFO:tasks.workunit.client.1.smithi148.stdout:6/43: write d3/f8 [1339108,63954] 0 2024-07-03T15:43:32.847 INFO:tasks.workunit.client.1.smithi148.stdout:2/6: sync 2024-07-03T15:43:32.848 INFO:tasks.workunit.client.1.smithi148.stdout:2/7: truncate f1 1042431 0 2024-07-03T15:43:32.864 INFO:tasks.workunit.client.1.smithi148.stdout:2/8: dwrite f1 [0,4194304] 0 2024-07-03T15:43:32.864 INFO:tasks.workunit.client.1.smithi148.stdout:2/9: write f0 [642789,42525] 0 2024-07-03T15:43:32.864 INFO:tasks.workunit.client.1.smithi148.stdout:2/10: chown f1 141997 1 2024-07-03T15:43:32.865 INFO:tasks.workunit.client.1.smithi148.stdout:2/11: symlink l2 0 2024-07-03T15:43:32.866 INFO:tasks.workunit.client.1.smithi148.stdout:2/12: mknod c3 0 2024-07-03T15:43:32.867 INFO:tasks.workunit.client.1.smithi148.stdout:2/13: symlink l4 0 2024-07-03T15:43:32.867 INFO:tasks.workunit.client.1.smithi148.stdout:2/14: mkdir d5 0 2024-07-03T15:43:32.868 INFO:tasks.workunit.client.1.smithi148.stdout:2/15: truncate f1 5069224 0 2024-07-03T15:43:32.868 INFO:tasks.workunit.client.1.smithi148.stdout:2/16: creat d5/f6 x:0 0 0 2024-07-03T15:43:32.870 INFO:tasks.workunit.client.1.smithi148.stdout:2/17: unlink f0 0 2024-07-03T15:43:32.871 INFO:tasks.workunit.client.1.smithi148.stdout:2/18: mkdir d5/d7 0 2024-07-03T15:43:32.871 INFO:tasks.workunit.client.1.smithi148.stdout:2/19: dread - d5/f6 zero size 2024-07-03T15:43:32.996 INFO:tasks.workunit.client.1.smithi148.stdout:8/16: unlink l3 0 2024-07-03T15:43:32.998 INFO:tasks.workunit.client.1.smithi148.stdout:3/7: write f0 [958024,3382] 0 2024-07-03T15:43:32.998 INFO:tasks.workunit.client.1.smithi148.stdout:3/8: chown f0 1172794644 1 2024-07-03T15:43:32.999 INFO:tasks.workunit.client.1.smithi148.stdout:6/44: fsync d3/f8 0 2024-07-03T15:43:32.999 INFO:tasks.workunit.client.1.smithi148.stdout:6/45: chown d3/c9 13722146 1 2024-07-03T15:43:32.999 INFO:tasks.workunit.client.1.smithi148.stdout:6/46: chown d3/fc 1 1 2024-07-03T15:43:33.001 INFO:tasks.workunit.client.1.smithi148.stdout:9/34: getdents d0 0 2024-07-03T15:43:33.001 INFO:tasks.workunit.client.1.smithi148.stdout:0/20: sync 2024-07-03T15:43:33.007 INFO:tasks.workunit.client.1.smithi148.stdout:8/17: write d4/d6/f7 [2011064,73769] 0 2024-07-03T15:43:33.007 INFO:tasks.workunit.client.1.smithi148.stdout:1/58: rename d0/d2/l9 to d0/d2/d11/l13 0 2024-07-03T15:43:33.008 INFO:tasks.workunit.client.1.smithi148.stdout:6/47: write f1 [3250725,39383] 0 2024-07-03T15:43:33.010 INFO:tasks.workunit.client.1.smithi148.stdout:8/18: creat d4/d6/f8 x:0 0 0 2024-07-03T15:43:33.011 INFO:tasks.workunit.client.1.smithi148.stdout:9/35: read d0/f6 [1465263,31290] 0 2024-07-03T15:43:33.014 INFO:tasks.workunit.client.1.smithi148.stdout:8/19: symlink d4/d6/l9 0 2024-07-03T15:43:33.014 INFO:tasks.workunit.client.1.smithi148.stdout:8/20: write d4/d6/f7 [605522,4549] 0 2024-07-03T15:43:33.020 INFO:tasks.workunit.client.1.smithi148.stdout:6/48: dread d3/f7 [0,4194304] 0 2024-07-03T15:43:33.024 INFO:tasks.workunit.client.1.smithi148.stdout:1/59: dwrite d0/d2/f10 [0,4194304] 0 2024-07-03T15:43:33.032 INFO:tasks.workunit.client.1.smithi148.stdout:6/49: dread d3/fc [0,4194304] 0 2024-07-03T15:43:33.041 INFO:tasks.workunit.client.1.smithi148.stdout:2/20: truncate f1 296559 0 2024-07-03T15:43:33.043 INFO:tasks.workunit.client.1.smithi148.stdout:2/21: creat d5/d7/f8 x:0 0 0 2024-07-03T15:43:33.043 INFO:tasks.workunit.client.1.smithi148.stdout:2/22: truncate d5/f6 1002246 0 2024-07-03T15:43:33.044 INFO:tasks.workunit.client.1.smithi148.stdout:2/23: rename d5/f6 to d5/f9 0 2024-07-03T15:43:33.051 INFO:tasks.workunit.client.1.smithi148.stdout:0/21: dwrite f4 [0,4194304] 0 2024-07-03T15:43:33.055 INFO:tasks.workunit.client.1.smithi148.stdout:9/36: dwrite d0/f7 [4194304,4194304] 0 2024-07-03T15:43:33.055 INFO:tasks.workunit.client.1.smithi148.stdout:9/37: chown d0/db 87 1 2024-07-03T15:43:33.055 INFO:tasks.workunit.client.1.smithi148.stdout:9/38: write d0/f6 [14211,2058] 0 2024-07-03T15:43:33.055 INFO:tasks.workunit.client.1.smithi148.stdout:0/22: unlink f4 0 2024-07-03T15:43:33.055 INFO:tasks.workunit.client.1.smithi148.stdout:0/23: dread - f2 zero size 2024-07-03T15:43:33.055 INFO:tasks.workunit.client.1.smithi148.stdout:1/60: dwrite d0/f12 [0,4194304] 0 2024-07-03T15:43:33.055 INFO:tasks.workunit.client.1.smithi148.stdout:6/50: dwrite d3/fa [4194304,4194304] 0 2024-07-03T15:43:33.058 INFO:tasks.workunit.client.1.smithi148.stdout:9/39: creat d0/db/fc x:0 0 0 2024-07-03T15:43:33.061 INFO:tasks.workunit.client.1.smithi148.stdout:6/51: dread d3/fc [0,4194304] 0 2024-07-03T15:43:33.065 INFO:tasks.workunit.client.1.smithi148.stdout:1/61: mkdir d0/d2/d6/d14 0 2024-07-03T15:43:33.068 INFO:tasks.workunit.client.1.smithi148.stdout:6/52: link d3/lf d3/l12 0 2024-07-03T15:43:33.068 INFO:tasks.workunit.client.1.smithi148.stdout:6/53: fsync d3/f7 0 2024-07-03T15:43:33.068 INFO:tasks.workunit.client.1.smithi148.stdout:6/54: chown d3/lf 41990784 1 2024-07-03T15:43:33.068 INFO:tasks.workunit.client.1.smithi148.stdout:6/55: chown d3/c6 191 1 2024-07-03T15:43:33.085 INFO:tasks.workunit.client.1.smithi148.stdout:9/40: dwrite d0/db/fc [0,4194304] 0 2024-07-03T15:43:33.088 INFO:tasks.workunit.client.1.smithi148.stdout:0/24: dwrite f2 [0,4194304] 0 2024-07-03T15:43:33.088 INFO:tasks.workunit.client.1.smithi148.stdout:0/25: chown c5 1020160233 1 2024-07-03T15:43:33.089 INFO:tasks.workunit.client.1.smithi148.stdout:1/62: dwrite d0/d2/fd [0,4194304] 0 2024-07-03T15:43:33.089 INFO:tasks.workunit.client.1.smithi148.stdout:1/63: fsync d0/d2/d6/fa 0 2024-07-03T15:43:33.091 INFO:tasks.workunit.client.1.smithi148.stdout:6/56: dwrite d3/fc [0,4194304] 0 2024-07-03T15:43:33.091 INFO:tasks.workunit.client.1.smithi148.stdout:6/57: readlink d3/l5 0 2024-07-03T15:43:33.091 INFO:tasks.workunit.client.1.smithi148.stdout:6/58: chown d3/c9 175 1 2024-07-03T15:43:33.091 INFO:tasks.workunit.client.1.smithi148.stdout:6/59: readlink d3/l12 0 2024-07-03T15:43:33.097 INFO:tasks.workunit.client.1.smithi148.stdout:0/26: write f2 [2412976,95570] 0 2024-07-03T15:43:33.100 INFO:tasks.workunit.client.1.smithi148.stdout:6/60: symlink d3/l13 0 2024-07-03T15:43:33.103 INFO:tasks.workunit.client.1.smithi148.stdout:6/61: symlink d3/l14 0 2024-07-03T15:43:33.113 INFO:tasks.workunit.client.1.smithi148.stdout:3/9: truncate f0 1848128 0 2024-07-03T15:43:33.115 INFO:tasks.workunit.client.1.smithi148.stdout:0/27: dwrite f2 [0,4194304] 0 2024-07-03T15:43:33.160 INFO:tasks.workunit.client.1.smithi148.stdout:2/24: dwrite f1 [0,4194304] 0 2024-07-03T15:43:33.162 INFO:tasks.workunit.client.1.smithi148.stdout:2/25: link d5/f9 d5/d7/fa 0 2024-07-03T15:43:33.163 INFO:tasks.workunit.client.1.smithi148.stdout:2/26: stat d5/d7 0 2024-07-03T15:43:33.163 INFO:tasks.workunit.client.1.smithi148.stdout:2/27: read - d5/d7/f8 zero size 2024-07-03T15:43:33.179 INFO:tasks.workunit.client.1.smithi148.stdout:2/28: dwrite d5/d7/fa [0,4194304] 0 2024-07-03T15:43:33.185 INFO:tasks.workunit.client.1.smithi148.stdout:2/29: mkdir d5/d7/db 0 2024-07-03T15:43:33.185 INFO:tasks.workunit.client.1.smithi148.stdout:2/30: chown l2 926 1 2024-07-03T15:43:33.185 INFO:tasks.workunit.client.1.smithi148.stdout:2/31: fsync d5/d7/f8 0 2024-07-03T15:43:33.203 INFO:tasks.workunit.client.1.smithi148.stdout:2/32: dwrite f1 [0,4194304] 0 2024-07-03T15:43:33.211 INFO:tasks.workunit.client.1.smithi148.stdout:9/41: truncate d0/db/fc 210060 0 2024-07-03T15:43:33.225 INFO:tasks.workunit.client.1.smithi148.stdout:2/33: dwrite d5/f9 [0,4194304] 0 2024-07-03T15:43:33.225 INFO:tasks.workunit.client.1.smithi148.stdout:9/42: dwrite d0/f6 [4194304,4194304] 0 2024-07-03T15:43:33.226 INFO:tasks.workunit.client.1.smithi148.stdout:6/62: rmdir d3 39 2024-07-03T15:43:33.227 INFO:tasks.workunit.client.1.smithi148.stdout:2/34: symlink d5/lc 0 2024-07-03T15:43:33.230 INFO:tasks.workunit.client.1.smithi148.stdout:6/63: dread d3/f8 [0,4194304] 0 2024-07-03T15:43:33.236 INFO:tasks.workunit.client.1.smithi148.stdout:6/64: dread d3/fa [0,4194304] 0 2024-07-03T15:43:33.241 INFO:tasks.workunit.client.1.smithi148.stdout:6/65: mknod d3/c15 0 2024-07-03T15:43:33.248 INFO:tasks.workunit.client.1.smithi148.stdout:2/35: dwrite f1 [0,4194304] 0 2024-07-03T15:43:33.254 INFO:tasks.workunit.client.1.smithi148.stdout:6/66: write d3/fc [2426658,46579] 0 2024-07-03T15:43:33.257 INFO:tasks.workunit.client.1.smithi148.stdout:6/67: unlink d3/f8 0 2024-07-03T15:43:33.261 INFO:tasks.workunit.client.1.smithi148.stdout:6/68: dread f1 [0,4194304] 0 2024-07-03T15:43:33.261 INFO:tasks.workunit.client.1.smithi148.stdout:6/69: chown f1 36217 1 2024-07-03T15:43:33.261 INFO:tasks.workunit.client.1.smithi148.stdout:3/10: dwrite f0 [0,4194304] 0 2024-07-03T15:43:33.264 INFO:tasks.workunit.client.1.smithi148.stdout:3/11: symlink l1 0 2024-07-03T15:43:33.267 INFO:tasks.workunit.client.1.smithi148.stdout:6/70: dread d3/fc [0,4194304] 0 2024-07-03T15:43:33.280 INFO:tasks.workunit.client.1.smithi148.stdout:6/71: dwrite d3/fa [4194304,4194304] 0 2024-07-03T15:43:33.286 INFO:tasks.workunit.client.1.smithi148.stdout:6/72: dread d3/fc [0,4194304] 0 2024-07-03T15:43:33.288 INFO:tasks.workunit.client.1.smithi148.stdout:6/73: link c2 d3/c16 0 2024-07-03T15:43:33.301 INFO:tasks.workunit.client.1.smithi148.stdout:2/36: dwrite d5/d7/fa [4194304,4194304] 0 2024-07-03T15:43:33.301 INFO:tasks.workunit.client.1.smithi148.stdout:2/37: getdents d5/d7/db 0 2024-07-03T15:43:33.301 INFO:tasks.workunit.client.1.smithi148.stdout:2/38: chown d5 440805049 1 2024-07-03T15:43:33.315 INFO:tasks.workunit.client.1.smithi148.stdout:9/43: rename d0/db to d0/dd 0 2024-07-03T15:43:33.315 INFO:tasks.workunit.client.1.smithi148.stdout:9/44: write d0/f7 [3561973,73476] 0 2024-07-03T15:43:33.317 INFO:tasks.workunit.client.1.smithi148.stdout:2/39: dwrite f1 [0,4194304] 0 2024-07-03T15:43:33.319 INFO:tasks.workunit.client.1.smithi148.stdout:9/45: read d0/dd/fc [122964,50169] 0 2024-07-03T15:43:33.321 INFO:tasks.workunit.client.1.smithi148.stdout:2/40: rename l4 to d5/d7/db/ld 0 2024-07-03T15:43:33.323 INFO:tasks.workunit.client.1.smithi148.stdout:9/46: creat d0/dd/fe x:0 0 0 2024-07-03T15:43:33.328 INFO:tasks.workunit.client.1.smithi148.stdout:9/47: creat d0/dd/ff x:0 0 0 2024-07-03T15:43:33.356 INFO:tasks.workunit.client.1.smithi148.stdout:3/12: getdents . 0 2024-07-03T15:43:33.356 INFO:tasks.workunit.client.1.smithi148.stdout:3/13: write f0 [4563720,31239] 0 2024-07-03T15:43:33.374 INFO:tasks.workunit.client.1.smithi148.stdout:3/14: dwrite f0 [0,4194304] 0 2024-07-03T15:43:33.382 INFO:tasks.workunit.client.1.smithi148.stdout:3/15: dread f0 [0,4194304] 0 2024-07-03T15:43:33.385 INFO:tasks.workunit.client.1.smithi148.stdout:3/16: mkdir d2 0 2024-07-03T15:43:33.402 INFO:tasks.workunit.client.1.smithi148.stdout:3/17: dwrite f0 [0,4194304] 0 2024-07-03T15:43:33.403 INFO:tasks.workunit.client.1.smithi148.stdout:3/18: rename d2 to d2/d3 22 2024-07-03T15:43:33.414 INFO:tasks.workunit.client.1.smithi148.stdout:2/41: getdents d5/d7/db 0 2024-07-03T15:43:33.416 INFO:tasks.workunit.client.1.smithi148.stdout:2/42: creat d5/d7/db/fe x:0 0 0 2024-07-03T15:43:33.419 INFO:tasks.workunit.client.1.smithi148.stdout:2/43: unlink d5/d7/db/fe 0 2024-07-03T15:43:33.419 INFO:tasks.workunit.client.1.smithi148.stdout:9/48: getdents d0/dd 0 2024-07-03T15:43:33.421 INFO:tasks.workunit.client.1.smithi148.stdout:2/44: rename c3 to d5/d7/cf 0 2024-07-03T15:43:33.433 INFO:tasks.workunit.client.1.smithi148.stdout:9/49: dwrite d0/dd/fe [0,4194304] 0 2024-07-03T15:43:33.436 INFO:tasks.workunit.client.1.smithi148.stdout:9/50: symlink d0/l10 0 2024-07-03T15:43:33.436 INFO:tasks.workunit.client.1.smithi148.stdout:9/51: read - d0/dd/ff zero size 2024-07-03T15:43:33.436 INFO:tasks.workunit.client.1.smithi148.stdout:9/52: stat d0/l5 0 2024-07-03T15:43:33.444 INFO:tasks.workunit.client.1.smithi148.stdout:9/53: dread d0/f6 [4194304,4194304] 0 2024-07-03T15:43:33.444 INFO:tasks.workunit.client.1.smithi148.stdout:9/54: chown d0/dd/fc 100031966 1 2024-07-03T15:43:33.444 INFO:tasks.workunit.client.1.smithi148.stdout:9/55: truncate d0/dd/ff 878562 0 2024-07-03T15:43:33.444 INFO:tasks.workunit.client.1.smithi148.stdout:9/56: stat d0/f6 0 2024-07-03T15:43:33.460 INFO:tasks.workunit.client.1.smithi148.stdout:9/57: dwrite d0/dd/fc [0,4194304] 0 2024-07-03T15:43:33.463 INFO:tasks.workunit.client.1.smithi148.stdout:9/58: truncate d0/f7 5922560 0 2024-07-03T15:43:33.465 INFO:tasks.workunit.client.1.smithi148.stdout:9/59: mkdir d0/dd/d11 0 2024-07-03T15:43:33.465 INFO:tasks.workunit.client.1.smithi148.stdout:9/60: truncate d0/dd/ff 1049446 0 2024-07-03T15:43:33.465 INFO:tasks.workunit.client.1.smithi148.stdout:9/61: write d0/dd/ff [1369710,128051] 0 2024-07-03T15:43:33.466 INFO:tasks.workunit.client.1.smithi148.stdout:9/62: rmdir d0/dd 39 2024-07-03T15:43:33.468 INFO:tasks.workunit.client.1.smithi148.stdout:9/63: creat d0/dd/d11/f12 x:0 0 0 2024-07-03T15:43:33.469 INFO:tasks.workunit.client.1.smithi148.stdout:9/64: write d0/dd/d11/f12 [1045994,32136] 0 2024-07-03T15:43:33.471 INFO:tasks.workunit.client.1.smithi148.stdout:9/65: mknod d0/dd/c13 0 2024-07-03T15:43:33.473 INFO:tasks.workunit.client.1.smithi148.stdout:9/66: creat d0/dd/f14 x:0 0 0 2024-07-03T15:43:33.476 INFO:tasks.workunit.client.1.smithi148.stdout:9/67: mkdir d0/dd/d15 0 2024-07-03T15:43:33.482 INFO:tasks.workunit.client.1.smithi148.stdout:9/68: write d0/dd/fe [3846916,88103] 0 2024-07-03T15:43:33.485 INFO:tasks.workunit.client.1.smithi148.stdout:3/19: truncate f0 1926432 0 2024-07-03T15:43:33.485 INFO:tasks.workunit.client.1.smithi148.stdout:3/20: readlink l1 0 2024-07-03T15:43:33.502 INFO:tasks.workunit.client.1.smithi148.stdout:2/45: rename d5/d7/cf to d5/d7/db/c10 0 2024-07-03T15:43:33.502 INFO:tasks.workunit.client.1.smithi148.stdout:2/46: stat d5/lc 0 2024-07-03T15:43:33.504 INFO:tasks.workunit.client.1.smithi148.stdout:2/47: unlink d5/d7/f8 0 2024-07-03T15:43:33.510 INFO:tasks.workunit.client.1.smithi148.stdout:2/48: write d5/d7/fa [3259134,30035] 0 2024-07-03T15:43:33.527 INFO:tasks.workunit.client.1.smithi148.stdout:2/49: dwrite f1 [0,4194304] 0 2024-07-03T15:43:33.536 INFO:tasks.workunit.client.1.smithi148.stdout:2/50: dread d5/d7/fa [4194304,4194304] 0 2024-07-03T15:43:33.559 INFO:tasks.workunit.client.1.smithi148.stdout:9/69: dread d0/f6 [0,4194304] 0 2024-07-03T15:43:33.563 INFO:tasks.workunit.client.1.smithi148.stdout:9/70: rmdir d0/dd/d11 39 2024-07-03T15:43:33.563 INFO:tasks.workunit.client.1.smithi148.stdout:9/71: chown d0/l5 149 1 2024-07-03T15:43:33.563 INFO:tasks.workunit.client.1.smithi148.stdout:9/72: creat d0/dd/f16 x:0 0 0 2024-07-03T15:43:33.567 INFO:tasks.workunit.client.1.smithi148.stdout:9/73: creat d0/f17 x:0 0 0 2024-07-03T15:43:33.567 INFO:tasks.workunit.client.1.smithi148.stdout:9/74: fsync d0/f6 0 2024-07-03T15:43:33.570 INFO:tasks.workunit.client.1.smithi148.stdout:9/75: mkdir d0/d18 0 2024-07-03T15:43:33.572 INFO:tasks.workunit.client.1.smithi148.stdout:9/76: fsync d0/dd/ff 0 2024-07-03T15:43:33.572 INFO:tasks.workunit.client.1.smithi148.stdout:9/77: stat d0/dd/f16 0 2024-07-03T15:43:33.578 INFO:tasks.workunit.client.1.smithi148.stdout:9/78: write d0/dd/fc [3535989,119436] 0 2024-07-03T15:43:33.586 INFO:tasks.workunit.client.1.smithi148.stdout:3/21: read f0 [1005263,32573] 0 2024-07-03T15:43:33.599 INFO:tasks.workunit.client.1.smithi148.stdout:2/51: rename d5/d7/db/c10 to d5/d7/db/c11 0 2024-07-03T15:43:33.601 INFO:tasks.workunit.client.1.smithi148.stdout:2/52: stat d5/d7/db/c11 0 2024-07-03T15:43:33.607 INFO:tasks.workunit.client.1.smithi148.stdout:3/22: dwrite f0 [0,4194304] 0 2024-07-03T15:43:33.607 INFO:tasks.workunit.client.1.smithi148.stdout:2/53: dread d5/d7/fa [4194304,4194304] 0 2024-07-03T15:43:33.607 INFO:tasks.workunit.client.1.smithi148.stdout:2/54: readlink d5/d7/db/ld 0 2024-07-03T15:43:33.607 INFO:tasks.workunit.client.1.smithi148.stdout:3/23: rmdir d2 0 2024-07-03T15:43:33.609 INFO:tasks.workunit.client.1.smithi148.stdout:2/55: symlink d5/d7/l12 0 2024-07-03T15:43:33.609 INFO:tasks.workunit.client.1.smithi148.stdout:2/56: fdatasync f1 0 2024-07-03T15:43:33.609 INFO:tasks.workunit.client.1.smithi148.stdout:2/57: chown d5/d7/db 16125 1 2024-07-03T15:43:33.609 INFO:tasks.workunit.client.1.smithi148.stdout:2/58: write f1 [4703704,87891] 0 2024-07-03T15:43:33.623 INFO:tasks.workunit.client.1.smithi148.stdout:3/24: dwrite f0 [0,4194304] 0 2024-07-03T15:43:33.631 INFO:tasks.workunit.client.1.smithi148.stdout:3/25: read f0 [3579487,99363] 0 2024-07-03T15:43:33.631 INFO:tasks.workunit.client.1.smithi148.stdout:3/26: chown f0 25484349 1 2024-07-03T15:43:33.632 INFO:tasks.workunit.client.1.smithi148.stdout:3/27: write f0 [570066,60129] 0 2024-07-03T15:43:33.638 INFO:tasks.workunit.client.1.smithi148.stdout:3/28: dread f0 [0,4194304] 0 2024-07-03T15:43:33.668 INFO:tasks.workunit.client.1.smithi148.stdout:9/79: truncate d0/dd/fe 637662 0 2024-07-03T15:43:33.693 INFO:tasks.workunit.client.1.smithi148.stdout:2/59: truncate d5/f9 4291222 0 2024-07-03T15:43:33.714 INFO:tasks.workunit.client.1.smithi148.stdout:3/29: truncate f0 2881592 0 2024-07-03T15:43:33.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:33 smithi148 ceph-mon[36056]: pgmap v229: 65 pgs: 65 active+clean; 158 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 5.5 MiB/s rd, 10 MiB/s wr, 741 op/s 2024-07-03T15:43:33.737 INFO:tasks.workunit.client.1.smithi148.stdout:9/80: rmdir d0 39 2024-07-03T15:43:33.739 INFO:tasks.workunit.client.1.smithi148.stdout:9/81: mkdir d0/dd/d15/d19 0 2024-07-03T15:43:33.744 INFO:tasks.workunit.client.1.smithi148.stdout:9/82: rename d0/f7 to d0/dd/f1a 0 2024-07-03T15:43:33.747 INFO:tasks.workunit.client.1.smithi148.stdout:9/83: truncate d0/dd/f1a 4326908 0 2024-07-03T15:43:33.750 INFO:tasks.workunit.client.1.smithi148.stdout:9/84: mknod d0/d18/c1b 0 2024-07-03T15:43:33.750 INFO:tasks.workunit.client.1.smithi148.stdout:9/85: fdatasync d0/dd/f1a 0 2024-07-03T15:43:33.753 INFO:tasks.workunit.client.1.smithi148.stdout:9/86: rename d0/l5 to d0/dd/l1c 0 2024-07-03T15:43:33.755 INFO:tasks.workunit.client.1.smithi148.stdout:9/87: symlink d0/d18/l1d 0 2024-07-03T15:43:33.755 INFO:tasks.workunit.client.1.smithi148.stdout:9/88: chown d0/d18 1 1 2024-07-03T15:43:33.755 INFO:tasks.workunit.client.1.smithi148.stdout:9/89: stat d0/d18/l1d 0 2024-07-03T15:43:33.756 INFO:tasks.workunit.client.1.smithi148.stdout:9/90: creat d0/dd/d15/d19/f1e x:0 0 0 2024-07-03T15:43:33.758 INFO:tasks.workunit.client.1.smithi148.stdout:9/91: creat d0/dd/f1f x:0 0 0 2024-07-03T15:43:33.758 INFO:tasks.workunit.client.1.smithi148.stdout:9/92: chown d0/dd/d15 3020 1 2024-07-03T15:43:33.765 INFO:tasks.workunit.client.1.smithi148.stdout:2/60: fsync d5/f9 0 2024-07-03T15:43:33.767 INFO:tasks.workunit.client.1.smithi148.stdout:2/61: mkdir d5/d7/db/d13 0 2024-07-03T15:43:33.784 INFO:tasks.workunit.client.1.smithi148.stdout:3/30: fsync f0 0 2024-07-03T15:43:33.784 INFO:tasks.workunit.client.1.smithi148.stdout:3/31: write f0 [2222268,16874] 0 2024-07-03T15:43:33.786 INFO:tasks.workunit.client.1.smithi148.stdout:3/32: creat f4 x:0 0 0 2024-07-03T15:43:33.826 INFO:tasks.workunit.client.1.smithi148.stdout:9/93: getdents d0/dd/d15/d19 0 2024-07-03T15:43:33.840 INFO:tasks.workunit.client.1.smithi148.stdout:9/94: dwrite d0/dd/fe [0,4194304] 0 2024-07-03T15:43:33.842 INFO:tasks.workunit.client.1.smithi148.stdout:9/95: mknod d0/dd/d11/c20 0 2024-07-03T15:43:33.844 INFO:tasks.workunit.client.1.smithi148.stdout:9/96: creat d0/d18/f21 x:0 0 0 2024-07-03T15:43:33.844 INFO:tasks.workunit.client.1.smithi148.stdout:9/97: stat d0/dd/d15 0 2024-07-03T15:43:33.844 INFO:tasks.workunit.client.1.smithi148.stdout:9/98: dread - d0/dd/f1f zero size 2024-07-03T15:43:33.844 INFO:tasks.workunit.client.1.smithi148.stdout:9/99: truncate d0/dd/d15/d19/f1e 523775 0 2024-07-03T15:43:33.844 INFO:tasks.workunit.client.1.smithi148.stdout:9/100: truncate d0/dd/f1f 677095 0 2024-07-03T15:43:33.846 INFO:tasks.workunit.client.1.smithi148.stdout:9/101: creat d0/dd/d15/f22 x:0 0 0 2024-07-03T15:43:33.860 INFO:tasks.workunit.client.1.smithi148.stdout:9/102: dwrite d0/d18/f21 [0,4194304] 0 2024-07-03T15:43:33.865 INFO:tasks.workunit.client.1.smithi148.stdout:9/103: mkdir d0/dd/d23 0 2024-07-03T15:43:33.865 INFO:tasks.workunit.client.1.smithi148.stdout:9/104: write d0/dd/f1a [580973,280] 0 2024-07-03T15:43:33.867 INFO:tasks.workunit.client.1.smithi148.stdout:9/105: mkdir d0/d18/d24 0 2024-07-03T15:43:33.867 INFO:tasks.workunit.client.1.smithi148.stdout:9/106: dread - d0/dd/d15/f22 zero size 2024-07-03T15:43:33.869 INFO:tasks.workunit.client.1.smithi148.stdout:9/107: mkdir d0/dd/d25 0 2024-07-03T15:43:33.869 INFO:tasks.workunit.client.1.smithi148.stdout:9/108: stat d0/dd/f14 0 2024-07-03T15:43:33.869 INFO:tasks.workunit.client.1.smithi148.stdout:9/109: creat d0/d18/f26 x:0 0 0 2024-07-03T15:43:33.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:33 smithi028 ceph-mon[29148]: pgmap v229: 65 pgs: 65 active+clean; 158 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 5.5 MiB/s rd, 10 MiB/s wr, 741 op/s 2024-07-03T15:43:33.894 INFO:tasks.workunit.client.1.smithi148.stdout:2/62: truncate d5/f9 498370 0 2024-07-03T15:43:33.937 INFO:tasks.workunit.client.1.smithi148.stdout:9/110: rmdir d0/dd 39 2024-07-03T15:43:33.937 INFO:tasks.workunit.client.1.smithi148.stdout:9/111: write d0/f17 [330061,65208] 0 2024-07-03T15:43:33.939 INFO:tasks.workunit.client.1.smithi148.stdout:9/112: symlink d0/l27 0 2024-07-03T15:43:33.939 INFO:tasks.workunit.client.1.smithi148.stdout:9/113: getdents d0/d18/d24 0 2024-07-03T15:43:33.964 INFO:tasks.workunit.client.1.smithi148.stdout:2/63: read d5/f9 [410713,110824] 0 2024-07-03T15:43:33.967 INFO:tasks.workunit.client.1.smithi148.stdout:2/64: mkdir d5/d14 0 2024-07-03T15:43:33.967 INFO:tasks.workunit.client.1.smithi148.stdout:2/65: fdatasync d5/f9 0 2024-07-03T15:43:33.967 INFO:tasks.workunit.client.1.smithi148.stdout:2/66: readlink l2 0 2024-07-03T15:43:33.971 INFO:tasks.workunit.client.1.smithi148.stdout:2/67: creat d5/d7/db/d13/f15 x:0 0 0 2024-07-03T15:43:33.971 INFO:tasks.workunit.client.1.smithi148.stdout:2/68: dread - d5/d7/db/d13/f15 zero size 2024-07-03T15:43:33.971 INFO:tasks.workunit.client.1.smithi148.stdout:2/69: creat d5/d7/db/f16 x:0 0 0 2024-07-03T15:43:33.971 INFO:tasks.workunit.client.1.smithi148.stdout:2/70: dread - d5/d7/db/d13/f15 zero size 2024-07-03T15:43:34.032 INFO:tasks.workunit.client.1.smithi148.stdout:9/114: truncate d0/dd/f1a 4037481 0 2024-07-03T15:43:34.033 INFO:tasks.workunit.client.1.smithi148.stdout:9/115: dread d0/dd/f1f [0,4194304] 0 2024-07-03T15:43:34.033 INFO:tasks.workunit.client.1.smithi148.stdout:9/116: write d0/d18/f26 [167562,29459] 0 2024-07-03T15:43:34.035 INFO:tasks.workunit.client.1.smithi148.stdout:9/117: rename d0/dd/d15 to d0/dd/d25/d28 0 2024-07-03T15:43:34.066 INFO:tasks.workunit.client.1.smithi148.stdout:2/71: rmdir d5/d7/db 39 2024-07-03T15:43:34.069 INFO:tasks.workunit.client.1.smithi148.stdout:2/72: mkdir d5/d14/d17 0 2024-07-03T15:43:34.072 INFO:tasks.workunit.client.1.smithi148.stdout:2/73: rename f1 to d5/f18 0 2024-07-03T15:43:34.074 INFO:tasks.workunit.client.1.smithi148.stdout:2/74: symlink d5/d14/l19 0 2024-07-03T15:43:34.074 INFO:tasks.workunit.client.1.smithi148.stdout:2/75: truncate d5/d7/db/f16 469657 0 2024-07-03T15:43:34.076 INFO:tasks.workunit.client.1.smithi148.stdout:2/76: dread d5/f9 [0,4194304] 0 2024-07-03T15:43:34.078 INFO:tasks.workunit.client.1.smithi148.stdout:2/77: creat d5/d7/db/f1a x:0 0 0 2024-07-03T15:43:34.081 INFO:tasks.workunit.client.1.smithi148.stdout:2/78: link d5/d7/fa d5/d7/db/d13/f1b 0 2024-07-03T15:43:34.083 INFO:tasks.workunit.client.1.smithi148.stdout:2/79: mknod d5/d7/db/c1c 0 2024-07-03T15:43:34.101 INFO:tasks.workunit.client.1.smithi148.stdout:2/80: dwrite d5/d7/db/d13/f1b [0,4194304] 0 2024-07-03T15:43:34.103 INFO:tasks.workunit.client.1.smithi148.stdout:2/81: mkdir d5/d7/db/d13/d1d 0 2024-07-03T15:43:34.103 INFO:tasks.workunit.client.1.smithi148.stdout:2/82: write d5/d7/db/d13/f15 [705797,53990] 0 2024-07-03T15:43:34.107 INFO:tasks.workunit.client.1.smithi148.stdout:2/83: dread d5/d7/db/d13/f15 [0,4194304] 0 2024-07-03T15:43:34.107 INFO:tasks.workunit.client.1.smithi148.stdout:2/84: read - d5/d7/db/f1a zero size 2024-07-03T15:43:34.116 INFO:tasks.workunit.client.1.smithi148.stdout:2/85: write d5/d7/db/d13/f1b [1038824,17691] 0 2024-07-03T15:43:34.137 INFO:tasks.workunit.client.1.smithi148.stdout:2/86: dwrite d5/d7/db/d13/f1b [0,4194304] 0 2024-07-03T15:43:34.137 INFO:tasks.workunit.client.1.smithi148.stdout:2/87: readlink d5/lc 0 2024-07-03T15:43:34.137 INFO:tasks.workunit.client.1.smithi148.stdout:2/88: chown d5 13573 1 2024-07-03T15:43:34.147 INFO:tasks.workunit.client.1.smithi148.stdout:2/89: dread d5/f18 [0,4194304] 0 2024-07-03T15:43:34.149 INFO:tasks.workunit.client.1.smithi148.stdout:2/90: dread d5/d7/db/d13/f15 [0,4194304] 0 2024-07-03T15:43:34.165 INFO:tasks.workunit.client.1.smithi148.stdout:2/91: dwrite d5/d7/db/f1a [0,4194304] 0 2024-07-03T15:43:34.172 INFO:tasks.workunit.client.1.smithi148.stdout:2/92: getdents d5/d7 0 2024-07-03T15:43:34.172 INFO:tasks.workunit.client.1.smithi148.stdout:2/93: creat d5/d7/db/f1e x:0 0 0 2024-07-03T15:43:34.172 INFO:tasks.workunit.client.1.smithi148.stdout:2/94: creat d5/f1f x:0 0 0 2024-07-03T15:43:34.191 INFO:tasks.workunit.client.1.smithi148.stdout:2/95: dwrite d5/d7/db/f1a [0,4194304] 0 2024-07-03T15:43:34.191 INFO:tasks.workunit.client.1.smithi148.stdout:2/96: chown d5/d14 258 1 2024-07-03T15:43:34.191 INFO:tasks.workunit.client.1.smithi148.stdout:2/97: stat d5/d7/db/ld 0 2024-07-03T15:43:34.191 INFO:tasks.workunit.client.1.smithi148.stdout:2/98: rename d5/d14 to d5/d14/d17/d20 22 2024-07-03T15:43:34.235 INFO:tasks.workunit.client.1.smithi148.stdout:2/99: rmdir d5 39 2024-07-03T15:43:34.235 INFO:tasks.workunit.client.1.smithi148.stdout:2/100: getdents d5/d14/d17 0 2024-07-03T15:43:35.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:35 smithi148 ceph-mon[36056]: pgmap v230: 65 pgs: 65 active+clean; 174 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 7.1 MiB/s rd, 14 MiB/s wr, 688 op/s 2024-07-03T15:43:35.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:35 smithi028 ceph-mon[29148]: pgmap v230: 65 pgs: 65 active+clean; 174 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 7.1 MiB/s rd, 14 MiB/s wr, 688 op/s 2024-07-03T15:43:37.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:37 smithi148 ceph-mon[36056]: pgmap v231: 65 pgs: 65 active+clean; 229 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 12 MiB/s rd, 22 MiB/s wr, 716 op/s 2024-07-03T15:43:37.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:37 smithi028 ceph-mon[29148]: pgmap v231: 65 pgs: 65 active+clean; 229 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 12 MiB/s rd, 22 MiB/s wr, 716 op/s 2024-07-03T15:43:38.121 INFO:tasks.workunit.client.1.smithi148.stdout:5/9: sync 2024-07-03T15:43:38.147 INFO:tasks.workunit.client.1.smithi148.stdout:5/10: dwrite f3 [0,4194304] 0 2024-07-03T15:43:38.150 INFO:tasks.workunit.client.1.smithi148.stdout:5/11: mknod c5 0 2024-07-03T15:43:38.165 INFO:tasks.workunit.client.1.smithi148.stdout:5/12: dwrite f4 [0,4194304] 0 2024-07-03T15:43:38.167 INFO:tasks.workunit.client.1.smithi148.stdout:5/13: symlink l6 0 2024-07-03T15:43:38.168 INFO:tasks.workunit.client.1.smithi148.stdout:5/14: mkdir d7 0 2024-07-03T15:43:38.168 INFO:tasks.workunit.client.1.smithi148.stdout:5/15: write f2 [73182,45066] 0 2024-07-03T15:43:38.168 INFO:tasks.workunit.client.1.smithi148.stdout:5/16: truncate f3 4573743 0 2024-07-03T15:43:38.168 INFO:tasks.workunit.client.1.smithi148.stdout:5/17: chown l0 3 1 2024-07-03T15:43:38.168 INFO:tasks.workunit.client.1.smithi148.stdout:5/18: readlink l1 0 2024-07-03T15:43:38.169 INFO:tasks.workunit.client.1.smithi148.stdout:5/19: creat d7/f8 x:0 0 0 2024-07-03T15:43:38.170 INFO:tasks.workunit.client.1.smithi148.stdout:5/20: unlink l6 0 2024-07-03T15:43:38.170 INFO:tasks.workunit.client.1.smithi148.stdout:5/21: readlink l1 0 2024-07-03T15:43:38.171 INFO:tasks.workunit.client.1.smithi148.stdout:5/22: mknod d7/c9 0 2024-07-03T15:43:38.171 INFO:tasks.workunit.client.1.smithi148.stdout:5/23: truncate d7/f8 620898 0 2024-07-03T15:43:38.171 INFO:tasks.workunit.client.1.smithi148.stdout:5/24: mkdir d7/da 0 2024-07-03T15:43:39.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:39 smithi148 ceph-mon[36056]: pgmap v232: 65 pgs: 65 active+clean; 244 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 14 MiB/s rd, 24 MiB/s wr, 717 op/s 2024-07-03T15:43:39.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:39 smithi028 ceph-mon[29148]: pgmap v232: 65 pgs: 65 active+clean; 244 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 14 MiB/s rd, 24 MiB/s wr, 717 op/s 2024-07-03T15:43:40.206 INFO:tasks.workunit.client.1.smithi148.stdout:0/28: sync 2024-07-03T15:43:40.206 INFO:tasks.workunit.client.1.smithi148.stdout:7/33: sync 2024-07-03T15:43:40.206 INFO:tasks.workunit.client.1.smithi148.stdout:6/74: sync 2024-07-03T15:43:40.206 INFO:tasks.workunit.client.1.smithi148.stdout:8/21: sync 2024-07-03T15:43:40.206 INFO:tasks.workunit.client.1.smithi148.stdout:3/33: sync 2024-07-03T15:43:40.206 INFO:tasks.workunit.client.1.smithi148.stdout:4/20: sync 2024-07-03T15:43:40.206 INFO:tasks.workunit.client.1.smithi148.stdout:8/22: chown d4/d6/l9 46801 1 2024-07-03T15:43:40.206 INFO:tasks.workunit.client.1.smithi148.stdout:7/34: creat d5/f9 x:0 0 0 2024-07-03T15:43:40.207 INFO:tasks.workunit.client.1.smithi148.stdout:4/21: creat d0/f4 x:0 0 0 2024-07-03T15:43:40.207 INFO:tasks.workunit.client.1.smithi148.stdout:0/29: creat f6 x:0 0 0 2024-07-03T15:43:40.207 INFO:tasks.workunit.client.1.smithi148.stdout:0/30: rmdir - no directory 2024-07-03T15:43:40.207 INFO:tasks.workunit.client.1.smithi148.stdout:3/34: write f0 [2824908,65696] 0 2024-07-03T15:43:40.207 INFO:tasks.workunit.client.1.smithi148.stdout:7/35: chown f2 2001 1 2024-07-03T15:43:40.207 INFO:tasks.workunit.client.1.smithi148.stdout:0/31: fsync f2 0 2024-07-03T15:43:40.207 INFO:tasks.workunit.client.1.smithi148.stdout:3/35: creat f5 x:0 0 0 2024-07-03T15:43:40.207 INFO:tasks.workunit.client.1.smithi148.stdout:0/32: creat f7 x:0 0 0 2024-07-03T15:43:40.207 INFO:tasks.workunit.client.1.smithi148.stdout:2/101: sync 2024-07-03T15:43:40.207 INFO:tasks.workunit.client.1.smithi148.stdout:9/118: sync 2024-07-03T15:43:40.207 INFO:tasks.workunit.client.1.smithi148.stdout:2/102: readlink d5/d7/db/ld 0 2024-07-03T15:43:40.207 INFO:tasks.workunit.client.1.smithi148.stdout:5/25: sync 2024-07-03T15:43:40.207 INFO:tasks.workunit.client.1.smithi148.stdout:2/103: stat d5/d14/d17 0 2024-07-03T15:43:40.207 INFO:tasks.workunit.client.1.smithi148.stdout:1/64: sync 2024-07-03T15:43:40.209 INFO:tasks.workunit.client.1.smithi148.stdout:3/36: fsync f0 0 2024-07-03T15:43:40.209 INFO:tasks.workunit.client.1.smithi148.stdout:3/37: creat f6 x:0 0 0 2024-07-03T15:43:40.209 INFO:tasks.workunit.client.1.smithi148.stdout:3/38: rmdir - no directory 2024-07-03T15:43:40.209 INFO:tasks.workunit.client.1.smithi148.stdout:3/39: dread - f4 zero size 2024-07-03T15:43:40.211 INFO:tasks.workunit.client.1.smithi148.stdout:4/22: write d0/f3 [3907567,120056] 0 2024-07-03T15:43:40.217 INFO:tasks.workunit.client.1.smithi148.stdout:6/75: dread d3/fa [4194304,4194304] 0 2024-07-03T15:43:40.217 INFO:tasks.workunit.client.1.smithi148.stdout:6/76: read d3/fc [322889,101661] 0 2024-07-03T15:43:40.218 INFO:tasks.workunit.client.1.smithi148.stdout:0/33: dread f2 [0,4194304] 0 2024-07-03T15:43:40.230 INFO:tasks.workunit.client.1.smithi148.stdout:9/119: truncate d0/dd/fe 310377 0 2024-07-03T15:43:40.237 INFO:tasks.workunit.client.1.smithi148.stdout:3/40: symlink l7 0 2024-07-03T15:43:40.237 INFO:tasks.workunit.client.1.smithi148.stdout:3/41: dread - f6 zero size 2024-07-03T15:43:40.257 INFO:tasks.workunit.client.1.smithi148.stdout:6/77: creat d3/f17 x:0 0 0 2024-07-03T15:43:40.257 INFO:tasks.workunit.client.1.smithi148.stdout:6/78: truncate d3/f17 627340 0 2024-07-03T15:43:40.257 INFO:tasks.workunit.client.1.smithi148.stdout:6/79: readlink d3/lf 0 2024-07-03T15:43:40.259 INFO:tasks.workunit.client.1.smithi148.stdout:0/34: mknod c8 0 2024-07-03T15:43:40.260 INFO:tasks.workunit.client.1.smithi148.stdout:0/35: readlink l1 0 2024-07-03T15:43:40.278 INFO:tasks.workunit.client.1.smithi148.stdout:6/80: dread d3/f17 [0,4194304] 0 2024-07-03T15:43:40.335 INFO:tasks.workunit.client.1.smithi148.stdout:2/104: dwrite d5/d7/db/d13/f15 [0,4194304] 0 2024-07-03T15:43:40.335 INFO:tasks.workunit.client.1.smithi148.stdout:2/105: fsync d5/f9 0 2024-07-03T15:43:40.336 INFO:tasks.workunit.client.1.smithi148.stdout:2/106: write d5/d7/db/f16 [859457,6126] 0 2024-07-03T15:43:40.338 INFO:tasks.workunit.client.1.smithi148.stdout:5/26: dwrite d7/f8 [0,4194304] 0 2024-07-03T15:43:40.354 INFO:tasks.workunit.client.1.smithi148.stdout:5/27: unlink d7/f8 0 2024-07-03T15:43:40.357 INFO:tasks.workunit.client.1.smithi148.stdout:2/107: write d5/f18 [2980627,119248] 0 2024-07-03T15:43:40.363 INFO:tasks.workunit.client.1.smithi148.stdout:2/108: symlink d5/d14/l21 0 2024-07-03T15:43:40.365 INFO:tasks.workunit.client.1.smithi148.stdout:4/23: dwrite d0/f4 [0,4194304] 0 2024-07-03T15:43:40.365 INFO:tasks.workunit.client.1.smithi148.stdout:4/24: creat d0/f5 x:0 0 0 2024-07-03T15:43:40.365 INFO:tasks.workunit.client.1.smithi148.stdout:4/25: write d0/f5 [793408,17378] 0 2024-07-03T15:43:40.420 INFO:tasks.workunit.client.1.smithi148.stdout:0/36: dwrite f7 [0,4194304] 0 2024-07-03T15:43:40.420 INFO:tasks.workunit.client.1.smithi148.stdout:0/37: chown f2 0 1 2024-07-03T15:43:40.430 INFO:tasks.workunit.client.1.smithi148.stdout:0/38: mknod c9 0 2024-07-03T15:43:40.446 INFO:tasks.workunit.client.1.smithi148.stdout:0/39: mkdir da 0 2024-07-03T15:43:40.452 INFO:tasks.workunit.client.1.smithi148.stdout:0/40: rename c8 to da/cb 0 2024-07-03T15:43:40.452 INFO:tasks.workunit.client.1.smithi148.stdout:0/41: fdatasync f7 0 2024-07-03T15:43:40.454 INFO:tasks.workunit.client.1.smithi148.stdout:4/26: dread d0/f5 [0,4194304] 0 2024-07-03T15:43:40.481 INFO:tasks.workunit.client.1.smithi148.stdout:4/27: write d0/f4 [2878004,12887] 0 2024-07-03T15:43:40.481 INFO:tasks.workunit.client.1.smithi148.stdout:4/28: write d0/f5 [848129,123739] 0 2024-07-03T15:43:40.481 INFO:tasks.workunit.client.1.smithi148.stdout:4/29: chown d0/f5 842 1 2024-07-03T15:43:40.482 INFO:tasks.workunit.client.1.smithi148.stdout:4/30: creat d0/f6 x:0 0 0 2024-07-03T15:43:40.502 INFO:tasks.workunit.client.1.smithi148.stdout:8/23: mkdir d4/da 0 2024-07-03T15:43:40.509 INFO:tasks.workunit.client.1.smithi148.stdout:8/24: truncate d4/d6/f7 3219514 0 2024-07-03T15:43:40.554 INFO:tasks.workunit.client.1.smithi148.stdout:5/28: dwrite f4 [0,4194304] 0 2024-07-03T15:43:40.563 INFO:tasks.workunit.client.1.smithi148.stdout:5/29: creat d7/da/fb x:0 0 0 2024-07-03T15:43:40.584 INFO:tasks.workunit.client.1.smithi148.stdout:3/42: dwrite f4 [0,4194304] 0 2024-07-03T15:43:40.584 INFO:tasks.workunit.client.1.smithi148.stdout:3/43: write f6 [640505,18839] 0 2024-07-03T15:43:40.584 INFO:tasks.workunit.client.1.smithi148.stdout:3/44: write f6 [1336167,68800] 0 2024-07-03T15:43:40.586 INFO:tasks.workunit.client.1.smithi148.stdout:5/30: write f3 [3676132,16416] 0 2024-07-03T15:43:40.586 INFO:tasks.workunit.client.1.smithi148.stdout:5/31: chown d7/da 2 1 2024-07-03T15:43:40.598 INFO:tasks.workunit.client.1.smithi148.stdout:3/45: unlink l1 0 2024-07-03T15:43:40.599 INFO:tasks.workunit.client.1.smithi148.stdout:3/46: chown f6 22 1 2024-07-03T15:43:40.614 INFO:tasks.workunit.client.1.smithi148.stdout:4/31: dwrite d0/f3 [0,4194304] 0 2024-07-03T15:43:40.614 INFO:tasks.workunit.client.1.smithi148.stdout:4/32: creat d0/f7 x:0 0 0 2024-07-03T15:43:40.626 INFO:tasks.workunit.client.1.smithi148.stdout:4/33: mkdir d0/d8 0 2024-07-03T15:43:40.648 INFO:tasks.workunit.client.1.smithi148.stdout:7/36: write f2 [1145993,65660] 0 2024-07-03T15:43:40.648 INFO:tasks.workunit.client.1.smithi148.stdout:7/37: creat d5/fa x:0 0 0 2024-07-03T15:43:40.648 INFO:tasks.workunit.client.1.smithi148.stdout:7/38: write d5/f6 [963506,67859] 0 2024-07-03T15:43:40.653 INFO:tasks.workunit.client.1.smithi148.stdout:7/39: creat d5/d8/fb x:0 0 0 2024-07-03T15:43:40.653 INFO:tasks.workunit.client.1.smithi148.stdout:7/40: write d5/f7 [7402,118542] 0 2024-07-03T15:43:40.653 INFO:tasks.workunit.client.1.smithi148.stdout:7/41: write d5/d8/fb [618265,57726] 0 2024-07-03T15:43:40.655 INFO:tasks.workunit.client.1.smithi148.stdout:7/42: symlink d5/lc 0 2024-07-03T15:43:40.659 INFO:tasks.workunit.client.1.smithi148.stdout:7/43: symlink d5/d8/ld 0 2024-07-03T15:43:40.662 INFO:tasks.workunit.client.1.smithi148.stdout:7/44: mknod d5/d8/ce 0 2024-07-03T15:43:40.662 INFO:tasks.workunit.client.1.smithi148.stdout:7/45: chown d5/f6 33188440 1 2024-07-03T15:43:40.723 INFO:tasks.workunit.client.1.smithi148.stdout:5/32: fsync f3 0 2024-07-03T15:43:40.723 INFO:tasks.workunit.client.1.smithi148.stdout:5/33: write d7/da/fb [234416,16870] 0 2024-07-03T15:43:40.799 INFO:tasks.workunit.client.1.smithi148.stdout:3/47: write f4 [1339402,49755] 0 2024-07-03T15:43:40.799 INFO:tasks.workunit.client.1.smithi148.stdout:3/48: creat f8 x:0 0 0 2024-07-03T15:43:40.881 INFO:tasks.workunit.client.1.smithi148.stdout:0/42: dwrite f2 [0,4194304] 0 2024-07-03T15:43:40.881 INFO:tasks.workunit.client.1.smithi148.stdout:0/43: stat f6 0 2024-07-03T15:43:40.928 INFO:tasks.workunit.client.1.smithi148.stdout:9/120: dwrite d0/dd/fe [0,4194304] 0 2024-07-03T15:43:40.930 INFO:tasks.workunit.client.1.smithi148.stdout:9/121: getdents d0/dd/d25/d28 0 2024-07-03T15:43:41.056 INFO:tasks.workunit.client.1.smithi148.stdout:0/44: dwrite f7 [0,4194304] 0 2024-07-03T15:43:41.061 INFO:tasks.workunit.client.1.smithi148.stdout:0/45: mkdir da/dc 0 2024-07-03T15:43:41.102 INFO:tasks.workunit.client.1.smithi148.stdout:0/46: read f2 [388152,2324] 0 2024-07-03T15:43:41.103 INFO:tasks.workunit.client.1.smithi148.stdout:0/47: readlink l1 0 2024-07-03T15:43:41.105 INFO:tasks.workunit.client.1.smithi148.stdout:0/48: mkdir da/dd 0 2024-07-03T15:43:41.192 INFO:tasks.workunit.client.1.smithi148.stdout:7/46: dwrite d5/f7 [0,4194304] 0 2024-07-03T15:43:41.249 INFO:tasks.workunit.client.1.smithi148.stdout:0/49: dwrite f7 [0,4194304] 0 2024-07-03T15:43:41.380 INFO:tasks.workunit.client.1.smithi148.stdout:7/47: dwrite f2 [0,4194304] 0 2024-07-03T15:43:41.381 INFO:tasks.workunit.client.1.smithi148.stdout:7/48: creat d5/ff x:0 0 0 2024-07-03T15:43:41.387 INFO:tasks.workunit.client.1.smithi148.stdout:7/49: link d5/d8/ld d5/l10 0 2024-07-03T15:43:41.387 INFO:tasks.workunit.client.1.smithi148.stdout:7/50: readlink d5/l10 0 2024-07-03T15:43:41.387 INFO:tasks.workunit.client.1.smithi148.stdout:7/51: dread - d5/fa zero size 2024-07-03T15:43:41.531 INFO:tasks.workunit.client.1.smithi148.stdout:6/81: getdents d3 0 2024-07-03T15:43:41.531 INFO:tasks.workunit.client.1.smithi148.stdout:6/82: fdatasync d3/f17 0 2024-07-03T15:43:41.534 INFO:tasks.workunit.client.1.smithi148.stdout:6/83: mknod d3/c18 0 2024-07-03T15:43:41.665 INFO:tasks.workunit.client.1.smithi148.stdout:9/122: mkdir d0/dd/d29 0 2024-07-03T15:43:41.666 INFO:tasks.workunit.client.1.smithi148.stdout:9/123: dread d0/dd/d25/d28/d19/f1e [0,4194304] 0 2024-07-03T15:43:41.667 INFO:tasks.workunit.client.1.smithi148.stdout:8/25: write f2 [1365153,47822] 0 2024-07-03T15:43:41.667 INFO:tasks.workunit.client.1.smithi148.stdout:8/26: write d4/d6/f8 [158310,79817] 0 2024-07-03T15:43:41.669 INFO:tasks.workunit.client.1.smithi148.stdout:9/124: mknod d0/c2a 0 2024-07-03T15:43:41.669 INFO:tasks.workunit.client.1.smithi148.stdout:2/109: rmdir d5/d7/db/d13 39 2024-07-03T15:43:41.672 INFO:tasks.workunit.client.1.smithi148.stdout:6/84: rmdir d3 39 2024-07-03T15:43:41.672 INFO:tasks.workunit.client.1.smithi148.stdout:6/85: write d3/f17 [1249301,39835] 0 2024-07-03T15:43:41.672 INFO:tasks.workunit.client.1.smithi148.stdout:6/86: write d3/f7 [783452,118072] 0 2024-07-03T15:43:41.672 INFO:tasks.workunit.client.1.smithi148.stdout:6/87: write f1 [4045986,35330] 0 2024-07-03T15:43:41.674 INFO:tasks.workunit.client.1.smithi148.stdout:9/125: creat d0/dd/f2b x:0 0 0 2024-07-03T15:43:41.682 INFO:tasks.workunit.client.1.smithi148.stdout:6/88: rename d3/l10 to d3/l19 0 2024-07-03T15:43:41.684 INFO:tasks.workunit.client.1.smithi148.stdout:9/126: creat d0/dd/d25/d28/f2c x:0 0 0 2024-07-03T15:43:41.686 INFO:tasks.workunit.client.1.smithi148.stdout:9/127: symlink d0/dd/d23/l2d 0 2024-07-03T15:43:41.690 INFO:tasks.workunit.client.1.smithi148.stdout:9/128: link d0/dd/ca d0/dd/d29/c2e 0 2024-07-03T15:43:41.690 INFO:tasks.workunit.client.1.smithi148.stdout:9/129: stat d0/d18/f21 0 2024-07-03T15:43:41.690 INFO:tasks.workunit.client.1.smithi148.stdout:9/130: readlink d0/l10 0 2024-07-03T15:43:41.690 INFO:tasks.workunit.client.1.smithi148.stdout:9/131: chown d0 3036 1 2024-07-03T15:43:41.693 INFO:tasks.workunit.client.1.smithi148.stdout:9/132: mkdir d0/d2f 0 2024-07-03T15:43:41.696 INFO:tasks.workunit.client.1.smithi148.stdout:9/133: creat d0/d18/d24/f30 x:0 0 0 2024-07-03T15:43:41.699 INFO:tasks.workunit.client.1.smithi148.stdout:9/134: creat d0/f31 x:0 0 0 2024-07-03T15:43:41.699 INFO:tasks.workunit.client.1.smithi148.stdout:9/135: rename d0 to d0/dd/d23/d32 22 2024-07-03T15:43:41.702 INFO:tasks.workunit.client.1.smithi148.stdout:9/136: symlink d0/d18/d24/l33 0 2024-07-03T15:43:41.702 INFO:tasks.workunit.client.1.smithi148.stdout:9/137: stat d0/dd/ff 0 2024-07-03T15:43:41.702 INFO:tasks.workunit.client.1.smithi148.stdout:9/138: read - d0/f31 zero size 2024-07-03T15:43:41.702 INFO:tasks.workunit.client.1.smithi148.stdout:9/139: dread - d0/dd/d25/d28/f2c zero size 2024-07-03T15:43:41.715 INFO:tasks.workunit.client.1.smithi148.stdout:9/140: dwrite d0/f6 [4194304,4194304] 0 2024-07-03T15:43:41.822 INFO:tasks.workunit.client.1.smithi148.stdout:1/65: sync 2024-07-03T15:43:41.822 INFO:tasks.workunit.client.1.smithi148.stdout:1/66: truncate d0/f12 4861841 0 2024-07-03T15:43:41.823 INFO:tasks.workunit.client.1.smithi148.stdout:1/67: rmdir d0/de 39 2024-07-03T15:43:41.827 INFO:tasks.workunit.client.1.smithi148.stdout:1/68: creat d0/d2/d6/d14/f15 x:0 0 0 2024-07-03T15:43:41.833 INFO:tasks.workunit.client.1.smithi148.stdout:1/69: read d0/d2/fd [3887422,115512] 0 2024-07-03T15:43:41.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:41 smithi028 ceph-mon[29148]: pgmap v233: 65 pgs: 65 active+clean; 252 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 15 MiB/s rd, 24 MiB/s wr, 655 op/s 2024-07-03T15:43:41.878 INFO:tasks.workunit.client.1.smithi148.stdout:2/110: dwrite d5/d7/db/d13/f1b [0,4194304] 0 2024-07-03T15:43:41.878 INFO:tasks.workunit.client.1.smithi148.stdout:2/111: stat d5/d14/d17 0 2024-07-03T15:43:41.881 INFO:tasks.workunit.client.1.smithi148.stdout:1/70: write d0/d2/d6/fa [509762,34146] 0 2024-07-03T15:43:41.881 INFO:tasks.workunit.client.1.smithi148.stdout:2/112: mkdir d5/d14/d22 0 2024-07-03T15:43:41.883 INFO:tasks.workunit.client.1.smithi148.stdout:9/141: dwrite d0/dd/fc [0,4194304] 0 2024-07-03T15:43:41.883 INFO:tasks.workunit.client.1.smithi148.stdout:9/142: fdatasync d0/dd/f14 0 2024-07-03T15:43:41.885 INFO:tasks.workunit.client.1.smithi148.stdout:1/71: mknod d0/d2/d6/d14/c16 0 2024-07-03T15:43:41.887 INFO:tasks.workunit.client.1.smithi148.stdout:2/113: mknod d5/c23 0 2024-07-03T15:43:41.889 INFO:tasks.workunit.client.1.smithi148.stdout:9/143: link d0/dd/ca d0/c34 0 2024-07-03T15:43:41.890 INFO:tasks.workunit.client.1.smithi148.stdout:1/72: link d0/d2/cc d0/d2/d6/c17 0 2024-07-03T15:43:41.894 INFO:tasks.workunit.client.1.smithi148.stdout:1/73: mknod d0/c18 0 2024-07-03T15:43:41.898 INFO:tasks.workunit.client.1.smithi148.stdout:2/114: dread d5/f9 [0,4194304] 0 2024-07-03T15:43:41.901 INFO:tasks.workunit.client.1.smithi148.stdout:2/115: symlink d5/d7/l24 0 2024-07-03T15:43:41.901 INFO:tasks.workunit.client.1.smithi148.stdout:2/116: getdents d5/d7/db/d13/d1d 0 2024-07-03T15:43:41.902 INFO:tasks.workunit.client.1.smithi148.stdout:9/144: dwrite d0/dd/f2b [0,4194304] 0 2024-07-03T15:43:41.903 INFO:tasks.workunit.client.1.smithi148.stdout:3/49: getdents . 0 2024-07-03T15:43:41.906 INFO:tasks.workunit.client.1.smithi148.stdout:2/117: write d5/d7/fa [1581798,29831] 0 2024-07-03T15:43:41.909 INFO:tasks.workunit.client.1.smithi148.stdout:9/145: creat d0/dd/d11/f35 x:0 0 0 2024-07-03T15:43:41.910 INFO:tasks.workunit.client.1.smithi148.stdout:2/118: mknod d5/d14/d22/c25 0 2024-07-03T15:43:41.912 INFO:tasks.workunit.client.1.smithi148.stdout:9/146: mknod d0/c36 0 2024-07-03T15:43:41.912 INFO:tasks.workunit.client.1.smithi148.stdout:9/147: truncate d0/dd/ff 1626381 0 2024-07-03T15:43:41.914 INFO:tasks.workunit.client.1.smithi148.stdout:9/148: mkdir d0/dd/d25/d37 0 2024-07-03T15:43:41.914 INFO:tasks.workunit.client.1.smithi148.stdout:9/149: dread - d0/dd/d11/f35 zero size 2024-07-03T15:43:41.919 INFO:tasks.workunit.client.1.smithi148.stdout:9/150: dread d0/f17 [0,4194304] 0 2024-07-03T15:43:41.922 INFO:tasks.workunit.client.1.smithi148.stdout:9/151: link d0/d18/l1d d0/dd/d25/d37/l38 0 2024-07-03T15:43:41.922 INFO:tasks.workunit.client.1.smithi148.stdout:9/152: write d0/dd/f1a [3930307,70422] 0 2024-07-03T15:43:41.927 INFO:tasks.workunit.client.1.smithi148.stdout:1/74: dwrite d0/d2/f10 [0,4194304] 0 2024-07-03T15:43:41.927 INFO:tasks.workunit.client.1.smithi148.stdout:1/75: stat d0/d2/d6/d14 0 2024-07-03T15:43:41.937 INFO:tasks.workunit.client.1.smithi148.stdout:3/50: dwrite f6 [0,4194304] 0 2024-07-03T15:43:41.941 INFO:tasks.workunit.client.1.smithi148.stdout:2/119: dwrite d5/d7/db/f16 [0,4194304] 0 2024-07-03T15:43:41.942 INFO:tasks.workunit.client.1.smithi148.stdout:9/153: dwrite d0/dd/ff [0,4194304] 0 2024-07-03T15:43:41.942 INFO:tasks.workunit.client.1.smithi148.stdout:9/154: truncate d0/d18/d24/f30 1017197 0 2024-07-03T15:43:41.942 INFO:tasks.workunit.client.1.smithi148.stdout:9/155: dread - d0/dd/d11/f35 zero size 2024-07-03T15:43:41.942 INFO:tasks.workunit.client.1.smithi148.stdout:9/156: chown d0/l27 747876 1 2024-07-03T15:43:41.942 INFO:tasks.workunit.client.1.smithi148.stdout:3/51: rename f6 to f9 0 2024-07-03T15:43:41.946 INFO:tasks.workunit.client.1.smithi148.stdout:2/120: truncate d5/d7/db/f1a 633476 0 2024-07-03T15:43:41.946 INFO:tasks.workunit.client.1.smithi148.stdout:1/76: dread d0/f8 [0,4194304] 0 2024-07-03T15:43:41.947 INFO:tasks.workunit.client.1.smithi148.stdout:1/77: fdatasync d0/d2/f10 0 2024-07-03T15:43:41.947 INFO:tasks.workunit.client.1.smithi148.stdout:1/78: chown d0/d2/f10 5 1 2024-07-03T15:43:41.947 INFO:tasks.workunit.client.1.smithi148.stdout:1/79: dread d0/f12 [4194304,4194304] 0 2024-07-03T15:43:41.949 INFO:tasks.workunit.client.1.smithi148.stdout:3/52: creat fa x:0 0 0 2024-07-03T15:43:41.950 INFO:tasks.workunit.client.1.smithi148.stdout:9/157: read d0/dd/fe [375075,3831] 0 2024-07-03T15:43:41.953 INFO:tasks.workunit.client.1.smithi148.stdout:1/80: unlink d0/d2/d6/d14/c16 0 2024-07-03T15:43:41.957 INFO:tasks.workunit.client.1.smithi148.stdout:9/158: mkdir d0/dd/d25/d39 0 2024-07-03T15:43:41.957 INFO:tasks.workunit.client.1.smithi148.stdout:9/159: stat d0/d18/c1b 0 2024-07-03T15:43:41.960 INFO:tasks.workunit.client.1.smithi148.stdout:1/81: write d0/d2/f10 [6438832,90475] 0 2024-07-03T15:43:41.960 INFO:tasks.workunit.client.1.smithi148.stdout:1/82: readlink d0/de/lf 0 2024-07-03T15:43:41.960 INFO:tasks.workunit.client.1.smithi148.stdout:9/160: link d0/c34 d0/dd/d25/d39/c3a 0 2024-07-03T15:43:41.964 INFO:tasks.workunit.client.1.smithi148.stdout:9/161: mknod d0/c3b 0 2024-07-03T15:43:41.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:41 smithi148 ceph-mon[36056]: pgmap v233: 65 pgs: 65 active+clean; 252 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 15 MiB/s rd, 24 MiB/s wr, 655 op/s 2024-07-03T15:43:42.136 INFO:tasks.workunit.client.1.smithi148.stdout:0/50: getdents da 0 2024-07-03T15:43:42.138 INFO:tasks.workunit.client.1.smithi148.stdout:0/51: creat da/dc/fe x:0 0 0 2024-07-03T15:43:42.138 INFO:tasks.workunit.client.1.smithi148.stdout:0/52: creat da/dc/ff x:0 0 0 2024-07-03T15:43:42.145 INFO:tasks.workunit.client.1.smithi148.stdout:0/53: write f7 [4093909,59635] 0 2024-07-03T15:43:42.152 INFO:tasks.workunit.client.1.smithi148.stdout:0/54: dread f2 [0,4194304] 0 2024-07-03T15:43:42.152 INFO:tasks.workunit.client.1.smithi148.stdout:0/55: dread - f6 zero size 2024-07-03T15:43:42.152 INFO:tasks.workunit.client.1.smithi148.stdout:0/56: write da/dc/ff [592849,80064] 0 2024-07-03T15:43:42.152 INFO:tasks.workunit.client.1.smithi148.stdout:0/57: write da/dc/ff [888839,107457] 0 2024-07-03T15:43:42.169 INFO:tasks.workunit.client.1.smithi148.stdout:0/58: dwrite f2 [0,4194304] 0 2024-07-03T15:43:42.185 INFO:tasks.workunit.client.1.smithi148.stdout:0/59: dwrite f6 [0,4194304] 0 2024-07-03T15:43:42.205 INFO:tasks.workunit.client.1.smithi148.stdout:0/60: dwrite f7 [0,4194304] 0 2024-07-03T15:43:42.213 INFO:tasks.workunit.client.1.smithi148.stdout:0/61: dread f7 [0,4194304] 0 2024-07-03T15:43:42.654 INFO:tasks.workunit.client.1.smithi148.stdout:6/89: truncate d3/f7 395886 0 2024-07-03T15:43:42.657 INFO:tasks.workunit.client.1.smithi148.stdout:6/90: creat d3/f1a x:0 0 0 2024-07-03T15:43:42.659 INFO:tasks.workunit.client.1.smithi148.stdout:6/91: mknod d3/c1b 0 2024-07-03T15:43:42.659 INFO:tasks.workunit.client.1.smithi148.stdout:6/92: write d3/fc [3482573,70133] 0 2024-07-03T15:43:42.663 INFO:tasks.workunit.client.1.smithi148.stdout:6/93: unlink d3/c11 0 2024-07-03T15:43:42.665 INFO:tasks.workunit.client.1.smithi148.stdout:6/94: link d3/fa d3/f1c 0 2024-07-03T15:43:42.665 INFO:tasks.workunit.client.1.smithi148.stdout:6/95: fsync d3/f17 0 2024-07-03T15:43:42.667 INFO:tasks.workunit.client.1.smithi148.stdout:6/96: mkdir d3/d1d 0 2024-07-03T15:43:42.670 INFO:tasks.workunit.client.1.smithi148.stdout:6/97: write d3/f17 [112873,108709] 0 2024-07-03T15:43:42.670 INFO:tasks.workunit.client.1.smithi148.stdout:6/98: creat d3/f1e x:0 0 0 2024-07-03T15:43:42.674 INFO:tasks.workunit.client.1.smithi148.stdout:6/99: unlink d3/fc 0 2024-07-03T15:43:42.676 INFO:tasks.workunit.client.1.smithi148.stdout:6/100: mknod d3/d1d/c1f 0 2024-07-03T15:43:42.676 INFO:tasks.workunit.client.1.smithi148.stdout:6/101: write d3/f1e [1007864,47638] 0 2024-07-03T15:43:42.676 INFO:tasks.workunit.client.1.smithi148.stdout:6/102: creat d3/f20 x:0 0 0 2024-07-03T15:43:42.676 INFO:tasks.workunit.client.1.smithi148.stdout:6/103: write d3/f20 [290926,7127] 0 2024-07-03T15:43:42.701 INFO:tasks.workunit.client.1.smithi148.stdout:6/104: dwrite d3/f20 [0,4194304] 0 2024-07-03T15:43:42.880 INFO:tasks.workunit.client.1.smithi148.stdout:3/53: getdents . 0 2024-07-03T15:43:42.880 INFO:tasks.workunit.client.1.smithi148.stdout:3/54: chown f5 342602113 1 2024-07-03T15:43:42.882 INFO:tasks.workunit.client.1.smithi148.stdout:3/55: symlink lb 0 2024-07-03T15:43:42.882 INFO:tasks.workunit.client.1.smithi148.stdout:2/121: dread d5/d7/db/f1a [0,4194304] 0 2024-07-03T15:43:42.885 INFO:tasks.workunit.client.1.smithi148.stdout:3/56: symlink lc 0 2024-07-03T15:43:42.887 INFO:tasks.workunit.client.1.smithi148.stdout:2/122: mkdir d5/d26 0 2024-07-03T15:43:42.887 INFO:tasks.workunit.client.1.smithi148.stdout:2/123: chown d5/d7/fa 4405845 1 2024-07-03T15:43:42.888 INFO:tasks.workunit.client.1.smithi148.stdout:3/57: mkdir dd 0 2024-07-03T15:43:42.890 INFO:tasks.workunit.client.1.smithi148.stdout:2/124: symlink d5/d7/db/l27 0 2024-07-03T15:43:42.890 INFO:tasks.workunit.client.1.smithi148.stdout:2/125: write d5/f9 [1723249,114110] 0 2024-07-03T15:43:42.892 INFO:tasks.workunit.client.1.smithi148.stdout:3/58: dread f4 [0,4194304] 0 2024-07-03T15:43:42.893 INFO:tasks.workunit.client.1.smithi148.stdout:2/126: mknod d5/c28 0 2024-07-03T15:43:42.894 INFO:tasks.workunit.client.1.smithi148.stdout:3/59: symlink dd/le 0 2024-07-03T15:43:42.894 INFO:tasks.workunit.client.1.smithi148.stdout:3/60: dread - fa zero size 2024-07-03T15:43:42.896 INFO:tasks.workunit.client.1.smithi148.stdout:2/127: creat d5/d7/db/d13/f29 x:0 0 0 2024-07-03T15:43:42.901 INFO:tasks.workunit.client.1.smithi148.stdout:3/61: dread f9 [0,4194304] 0 2024-07-03T15:43:42.901 INFO:tasks.workunit.client.1.smithi148.stdout:3/62: chown f4 5 1 2024-07-03T15:43:42.903 INFO:tasks.workunit.client.1.smithi148.stdout:3/63: truncate f0 2139312 0 2024-07-03T15:43:42.915 INFO:tasks.workunit.client.1.smithi148.stdout:2/128: dwrite d5/d7/db/d13/f15 [0,4194304] 0 2024-07-03T15:43:42.919 INFO:tasks.workunit.client.1.smithi148.stdout:2/129: symlink d5/d7/db/l2a 0 2024-07-03T15:43:42.920 INFO:tasks.workunit.client.1.smithi148.stdout:2/130: write d5/d7/db/d13/f29 [325755,115412] 0 2024-07-03T15:43:43.108 INFO:tasks.workunit.client.1.smithi148.stdout:0/62: fsync da/dc/fe 0 2024-07-03T15:43:43.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.174+0000 7f73cf065640 1 -- 172.21.15.28:0/25773676 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73c80fc510 msgr2=0x7f73c80fc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:43.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.174+0000 7f73cf065640 1 --2- 172.21.15.28:0/25773676 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73c80fc510 0x7f73c80fc980 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f73b40048d0 tx=0x7f73b4009da0 comp rx=0 tx=0).stop 2024-07-03T15:43:43.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.174+0000 7f73cf065640 1 -- 172.21.15.28:0/25773676 shutdown_connections 2024-07-03T15:43:43.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.174+0000 7f73cf065640 1 --2- 172.21.15.28:0/25773676 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73c80fc510 0x7f73c80fc980 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:43.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.174+0000 7f73cf065640 1 --2- 172.21.15.28:0/25773676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73c80fb240 0x7f73c80fb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:43.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.174+0000 7f73cf065640 1 -- 172.21.15.28:0/25773676 >> 172.21.15.28:0/25773676 conn(0x7f73c80f6a20 msgr2=0x7f73c80f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:43:43.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.174+0000 7f73cf065640 1 -- 172.21.15.28:0/25773676 shutdown_connections 2024-07-03T15:43:43.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.174+0000 7f73cf065640 1 -- 172.21.15.28:0/25773676 wait complete. 2024-07-03T15:43:43.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.174+0000 7f73cf065640 1 Processor -- start 2024-07-03T15:43:43.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.175+0000 7f73cf065640 1 -- start start 2024-07-03T15:43:43.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.175+0000 7f73cf065640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73c80fb240 0x7f73c810cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:43:43.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.175+0000 7f73cf065640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73c80fc510 0x7f73c810d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:43:43.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.175+0000 7f73cf065640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c810db10 con 0x7f73c80fc510 2024-07-03T15:43:43.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.175+0000 7f73cf065640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c810e330 con 0x7f73c80fb240 2024-07-03T15:43:43.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.175+0000 7f73c7fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73c80fc510 0x7f73c810d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:43:43.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.175+0000 7f73cce01640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73c80fb240 0x7f73c810cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:43:43.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.175+0000 7f73c7fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73c80fc510 0x7f73c810d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53126/0 (socket says 172.21.15.28:53126) 2024-07-03T15:43:43.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.175+0000 7f73c7fff640 1 -- 172.21.15.28:0/385808089 learned_addr learned my addr 172.21.15.28:0/385808089 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:43:43.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.176+0000 7f73cce01640 1 -- 172.21.15.28:0/385808089 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73c80fc510 msgr2=0x7f73c810d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:43.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.176+0000 7f73cce01640 1 --2- 172.21.15.28:0/385808089 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73c80fc510 0x7f73c810d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:43.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.176+0000 7f73cce01640 1 -- 172.21.15.28:0/385808089 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b4005040 con 0x7f73c80fb240 2024-07-03T15:43:43.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.176+0000 7f73cce01640 1 --2- 172.21.15.28:0/385808089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73c80fb240 0x7f73c810cf80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f73bc0027a0 tx=0x7f73bc009f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:43:43.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.176+0000 7f73c57fa640 1 -- 172.21.15.28:0/385808089 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73bc002d90 con 0x7f73c80fb240 2024-07-03T15:43:43.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.176+0000 7f73cf065640 1 -- 172.21.15.28:0/385808089 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73c810e5c0 con 0x7f73c80fb240 2024-07-03T15:43:43.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.176+0000 7f73c57fa640 1 -- 172.21.15.28:0/385808089 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f73bc00f040 con 0x7f73c80fb240 2024-07-03T15:43:43.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.176+0000 7f73c57fa640 1 -- 172.21.15.28:0/385808089 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73bc00da40 con 0x7f73c80fb240 2024-07-03T15:43:43.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.176+0000 7f73cf065640 1 -- 172.21.15.28:0/385808089 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73c810eb60 con 0x7f73c80fb240 2024-07-03T15:43:43.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.177+0000 7f73cf065640 1 -- 172.21.15.28:0/385808089 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73c80fa6a0 con 0x7f73c80fb240 2024-07-03T15:43:43.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.178+0000 7f73c57fa640 1 -- 172.21.15.28:0/385808089 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f73bc00dd00 con 0x7f73c80fb240 2024-07-03T15:43:43.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.179+0000 7f73c57fa640 1 --2- 172.21.15.28:0/385808089 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f73b8067190 0x7f73b8069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:43:43.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.179+0000 7f73c57fa640 1 -- 172.21.15.28:0/385808089 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f73bc0848c0 con 0x7f73c80fb240 2024-07-03T15:43:43.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.179+0000 7f73c7fff640 1 --2- 172.21.15.28:0/385808089 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f73b8067190 0x7f73b8069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:43:43.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.179+0000 7f73c7fff640 1 --2- 172.21.15.28:0/385808089 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f73b8067190 0x7f73b8069640 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f73b4007ac0 tx=0x7f73b4035040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:43:43.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.183+0000 7f73c57fa640 1 -- 172.21.15.28:0/385808089 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f73bc051b60 con 0x7f73c80fb240 2024-07-03T15:43:43.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.413+0000 7f73cf065640 1 -- 172.21.15.28:0/385808089 --> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f73c802aad0 con 0x7f73b8067190 2024-07-03T15:43:43.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.414+0000 7f73c57fa640 1 -- 172.21.15.28:0/385808089 <== mgr.14210 v2:172.21.15.28:6800/421057178 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7f73c802aad0 con 0x7f73b8067190 2024-07-03T15:43:43.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.416+0000 7f73cf065640 1 -- 172.21.15.28:0/385808089 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f73b8067190 msgr2=0x7f73b8069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:43.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.416+0000 7f73cf065640 1 --2- 172.21.15.28:0/385808089 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f73b8067190 0x7f73b8069640 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f73b4007ac0 tx=0x7f73b4035040 comp rx=0 tx=0).stop 2024-07-03T15:43:43.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.416+0000 7f73cf065640 1 -- 172.21.15.28:0/385808089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73c80fb240 msgr2=0x7f73c810cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:43.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.417+0000 7f73cf065640 1 --2- 172.21.15.28:0/385808089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73c80fb240 0x7f73c810cf80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f73bc0027a0 tx=0x7f73bc009f30 comp rx=0 tx=0).stop 2024-07-03T15:43:43.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.417+0000 7f73cf065640 1 -- 172.21.15.28:0/385808089 shutdown_connections 2024-07-03T15:43:43.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.417+0000 7f73cf065640 1 --2- 172.21.15.28:0/385808089 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f73b8067190 0x7f73b8069640 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:43.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.417+0000 7f73cf065640 1 --2- 172.21.15.28:0/385808089 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73c80fc510 0x7f73c810d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:43.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.417+0000 7f73cf065640 1 --2- 172.21.15.28:0/385808089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73c80fb240 0x7f73c810cf80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:43.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.417+0000 7f73cf065640 1 -- 172.21.15.28:0/385808089 >> 172.21.15.28:0/385808089 conn(0x7f73c80f6a20 msgr2=0x7f73c8065400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:43:43.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.417+0000 7f73cf065640 1 -- 172.21.15.28:0/385808089 shutdown_connections 2024-07-03T15:43:43.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.417+0000 7f73cf065640 1 -- 172.21.15.28:0/385808089 wait complete. 2024-07-03T15:43:43.433 INFO:teuthology.orchestra.run.smithi028.stdout:true 2024-07-03T15:43:43.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.527+0000 7fb8ea350640 1 -- 172.21.15.28:0/4156003802 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8e40659f0 msgr2=0x7fb8e41013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:43.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.527+0000 7fb8ea350640 1 --2- 172.21.15.28:0/4156003802 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8e40659f0 0x7fb8e41013b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb8d40048d0 tx=0x7fb8d4009da0 comp rx=0 tx=0).stop 2024-07-03T15:43:43.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.527+0000 7fb8ea350640 1 -- 172.21.15.28:0/4156003802 shutdown_connections 2024-07-03T15:43:43.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.527+0000 7fb8ea350640 1 --2- 172.21.15.28:0/4156003802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8e41018f0 0x7fb8e4103cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:43.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.527+0000 7fb8ea350640 1 --2- 172.21.15.28:0/4156003802 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8e40659f0 0x7fb8e41013b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:43.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.527+0000 7fb8ea350640 1 -- 172.21.15.28:0/4156003802 >> 172.21.15.28:0/4156003802 conn(0x7fb8e40f6a20 msgr2=0x7fb8e40f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:43:43.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.527+0000 7fb8ea350640 1 -- 172.21.15.28:0/4156003802 shutdown_connections 2024-07-03T15:43:43.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.527+0000 7fb8ea350640 1 -- 172.21.15.28:0/4156003802 wait complete. 2024-07-03T15:43:43.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.528+0000 7fb8ea350640 1 Processor -- start 2024-07-03T15:43:43.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.528+0000 7fb8ea350640 1 -- start start 2024-07-03T15:43:43.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.528+0000 7fb8ea350640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8e40659f0 0x7fb8e410f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:43:43.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.528+0000 7fb8ea350640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8e41018f0 0x7fb8e410f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:43:43.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.528+0000 7fb8ea350640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e410fd10 con 0x7fb8e40659f0 2024-07-03T15:43:43.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.528+0000 7fb8ea350640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e4110530 con 0x7fb8e41018f0 2024-07-03T15:43:43.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.529+0000 7fb8e3fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8e40659f0 0x7fb8e410f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:43:43.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.529+0000 7fb8e3fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8e40659f0 0x7fb8e410f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53140/0 (socket says 172.21.15.28:53140) 2024-07-03T15:43:43.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.529+0000 7fb8e3fff640 1 -- 172.21.15.28:0/589491492 learned_addr learned my addr 172.21.15.28:0/589491492 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:43:43.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.529+0000 7fb8dbfff640 1 --2- 172.21.15.28:0/589491492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8e41018f0 0x7fb8e410f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:43:43.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.529+0000 7fb8e3fff640 1 -- 172.21.15.28:0/589491492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8e41018f0 msgr2=0x7fb8e410f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:43.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.529+0000 7fb8e3fff640 1 --2- 172.21.15.28:0/589491492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8e41018f0 0x7fb8e410f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:43.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.529+0000 7fb8e3fff640 1 -- 172.21.15.28:0/589491492 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d4005040 con 0x7fb8e40659f0 2024-07-03T15:43:43.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.529+0000 7fb8e3fff640 1 --2- 172.21.15.28:0/589491492 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8e40659f0 0x7fb8e410f180 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb8d40048a0 tx=0x7fb8d4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:43:43.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.529+0000 7fb8e17fa640 1 -- 172.21.15.28:0/589491492 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8d4038070 con 0x7fb8e40659f0 2024-07-03T15:43:43.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.530+0000 7fb8e17fa640 1 -- 172.21.15.28:0/589491492 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb8d4009720 con 0x7fb8e40659f0 2024-07-03T15:43:43.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.530+0000 7fb8ea350640 1 -- 172.21.15.28:0/589491492 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8e4110760 con 0x7fb8e40659f0 2024-07-03T15:43:43.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.530+0000 7fb8e17fa640 1 -- 172.21.15.28:0/589491492 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8d403dbf0 con 0x7fb8e40659f0 2024-07-03T15:43:43.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.530+0000 7fb8ea350640 1 -- 172.21.15.28:0/589491492 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8e4110ca0 con 0x7fb8e40659f0 2024-07-03T15:43:43.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.531+0000 7fb8ea350640 1 -- 172.21.15.28:0/589491492 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8e40fa6a0 con 0x7fb8e40659f0 2024-07-03T15:43:43.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.532+0000 7fb8e17fa640 1 -- 172.21.15.28:0/589491492 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7fb8d4030020 con 0x7fb8e40659f0 2024-07-03T15:43:43.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.532+0000 7fb8e17fa640 1 --2- 172.21.15.28:0/589491492 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fb8d00671e0 0x7fb8d0069690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:43:43.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.532+0000 7fb8e17fa640 1 -- 172.21.15.28:0/589491492 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fb8d40a5780 con 0x7fb8e40659f0 2024-07-03T15:43:43.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.532+0000 7fb8dbfff640 1 --2- 172.21.15.28:0/589491492 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fb8d00671e0 0x7fb8d0069690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:43:43.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.532+0000 7fb8dbfff640 1 --2- 172.21.15.28:0/589491492 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fb8d00671e0 0x7fb8d0069690 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb8e410ef30 tx=0x7fb8cc00b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:43:43.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.539+0000 7fb8e17fa640 1 -- 172.21.15.28:0/589491492 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb8d4072a40 con 0x7fb8e40659f0 2024-07-03T15:43:43.558 INFO:tasks.workunit.client.1.smithi148.stdout:6/105: read d3/f7 [331520,5180] 0 2024-07-03T15:43:43.558 INFO:tasks.workunit.client.1.smithi148.stdout:6/106: chown d3/l19 3296 1 2024-07-03T15:43:43.568 INFO:tasks.workunit.client.1.smithi148.stdout:8/27: truncate d4/d6/f7 3021582 0 2024-07-03T15:43:43.573 INFO:tasks.workunit.client.1.smithi148.stdout:8/28: creat d4/da/fb x:0 0 0 2024-07-03T15:43:43.573 INFO:tasks.workunit.client.1.smithi148.stdout:8/29: creat d4/d6/fc x:0 0 0 2024-07-03T15:43:43.574 INFO:tasks.workunit.client.1.smithi148.stdout:6/107: dwrite f1 [4194304,4194304] 0 2024-07-03T15:43:43.577 INFO:tasks.workunit.client.1.smithi148.stdout:8/30: mknod d4/cd 0 2024-07-03T15:43:43.577 INFO:tasks.workunit.client.1.smithi148.stdout:8/31: write d4/da/fb [309502,76809] 0 2024-07-03T15:43:43.580 INFO:tasks.workunit.client.1.smithi148.stdout:6/108: rmdir d3/d1d 39 2024-07-03T15:43:43.582 INFO:tasks.workunit.client.1.smithi148.stdout:6/109: dread d3/f7 [0,4194304] 0 2024-07-03T15:43:43.592 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:43 smithi148 ceph-mon[36056]: pgmap v234: 65 pgs: 65 active+clean; 286 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 17 MiB/s rd, 26 MiB/s wr, 586 op/s 2024-07-03T15:43:43.592 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:43 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:43:43.592 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:43 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi148.rjzsar", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-03T15:43:43.593 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:43 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-03T15:43:43.593 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:43 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:43:43.597 INFO:tasks.workunit.client.1.smithi148.stdout:6/110: dwrite d3/fa [4194304,4194304] 0 2024-07-03T15:43:43.597 INFO:tasks.workunit.client.1.smithi148.stdout:6/111: write d3/f17 [1622668,53872] 0 2024-07-03T15:43:43.613 INFO:tasks.workunit.client.1.smithi148.stdout:6/112: dwrite d3/f1a [0,4194304] 0 2024-07-03T15:43:43.613 INFO:tasks.workunit.client.1.smithi148.stdout:6/113: chown d3/f17 1 1 2024-07-03T15:43:43.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.809+0000 7fb8ea350640 1 -- 172.21.15.28:0/589491492 --> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb8e4111000 con 0x7fb8d00671e0 2024-07-03T15:43:43.813 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:43 smithi028 ceph-mon[29148]: pgmap v234: 65 pgs: 65 active+clean; 286 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 17 MiB/s rd, 26 MiB/s wr, 586 op/s 2024-07-03T15:43:43.813 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:43 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:43:43.813 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:43 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi148.rjzsar", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-03T15:43:43.813 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:43 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-03T15:43:43.813 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:43 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:43:43.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.810+0000 7fb8e17fa640 1 -- 172.21.15.28:0/589491492 <== mgr.14210 v2:172.21.15.28:6800/421057178 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7fb8e4111000 con 0x7fb8d00671e0 2024-07-03T15:43:43.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.812+0000 7fb8ea350640 1 -- 172.21.15.28:0/589491492 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fb8d00671e0 msgr2=0x7fb8d0069690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:43.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.812+0000 7fb8ea350640 1 --2- 172.21.15.28:0/589491492 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fb8d00671e0 0x7fb8d0069690 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb8e410ef30 tx=0x7fb8cc00b040 comp rx=0 tx=0).stop 2024-07-03T15:43:43.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.812+0000 7fb8ea350640 1 -- 172.21.15.28:0/589491492 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8e40659f0 msgr2=0x7fb8e410f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:43.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.812+0000 7fb8ea350640 1 --2- 172.21.15.28:0/589491492 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8e40659f0 0x7fb8e410f180 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb8d40048a0 tx=0x7fb8d4009da0 comp rx=0 tx=0).stop 2024-07-03T15:43:43.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.812+0000 7fb8ea350640 1 -- 172.21.15.28:0/589491492 shutdown_connections 2024-07-03T15:43:43.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.812+0000 7fb8ea350640 1 --2- 172.21.15.28:0/589491492 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fb8d00671e0 0x7fb8d0069690 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:43.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.812+0000 7fb8ea350640 1 --2- 172.21.15.28:0/589491492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8e41018f0 0x7fb8e410f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:43.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.812+0000 7fb8ea350640 1 --2- 172.21.15.28:0/589491492 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8e40659f0 0x7fb8e410f180 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:43.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.812+0000 7fb8ea350640 1 -- 172.21.15.28:0/589491492 >> 172.21.15.28:0/589491492 conn(0x7fb8e40f6a20 msgr2=0x7fb8e40f6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:43:43.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.812+0000 7fb8ea350640 1 -- 172.21.15.28:0/589491492 shutdown_connections 2024-07-03T15:43:43.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.812+0000 7fb8ea350640 1 -- 172.21.15.28:0/589491492 wait complete. 2024-07-03T15:43:43.891 INFO:tasks.workunit.client.1.smithi148.stdout:2/131: rename d5/d7/db to d5/d14/d22/d2b 0 2024-07-03T15:43:43.895 INFO:tasks.workunit.client.1.smithi148.stdout:0/63: rename da/dc/fe to da/dd/f10 0 2024-07-03T15:43:43.895 INFO:tasks.workunit.client.1.smithi148.stdout:0/64: readlink l1 0 2024-07-03T15:43:43.896 INFO:tasks.workunit.client.1.smithi148.stdout:2/132: mknod d5/d14/d22/c2c 0 2024-07-03T15:43:43.899 INFO:tasks.workunit.client.1.smithi148.stdout:6/114: rename d3/f20 to d3/d1d/f21 0 2024-07-03T15:43:43.901 INFO:tasks.workunit.client.1.smithi148.stdout:0/65: creat da/f11 x:0 0 0 2024-07-03T15:43:43.905 INFO:tasks.workunit.client.1.smithi148.stdout:6/115: mknod d3/c22 0 2024-07-03T15:43:43.908 INFO:tasks.workunit.client.1.smithi148.stdout:0/66: dread da/dc/ff [0,4194304] 0 2024-07-03T15:43:43.908 INFO:tasks.workunit.client.1.smithi148.stdout:0/67: creat da/dc/f12 x:0 0 0 2024-07-03T15:43:43.911 INFO:tasks.workunit.client.1.smithi148.stdout:2/133: rmdir d5/d14/d17 0 2024-07-03T15:43:43.912 INFO:tasks.workunit.client.1.smithi148.stdout:2/134: chown d5/d14/d22/d2b/ld 198890 1 2024-07-03T15:43:43.916 INFO:tasks.workunit.client.1.smithi148.stdout:0/68: mkdir da/dc/d13 0 2024-07-03T15:43:43.916 INFO:tasks.workunit.client.1.smithi148.stdout:0/69: dread - da/dd/f10 zero size 2024-07-03T15:43:43.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.914+0000 7fa71d0e6640 1 -- 172.21.15.28:0/1076179746 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7180ff510 msgr2=0x7fa7181018f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:43.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.914+0000 7fa71d0e6640 1 --2- 172.21.15.28:0/1076179746 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7180ff510 0x7fa7181018f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa7000048d0 tx=0x7fa700009da0 comp rx=0 tx=0).stop 2024-07-03T15:43:43.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.914+0000 7fa71d0e6640 1 -- 172.21.15.28:0/1076179746 shutdown_connections 2024-07-03T15:43:43.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.914+0000 7fa71d0e6640 1 --2- 172.21.15.28:0/1076179746 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7180ff510 0x7fa7181018f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:43.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.914+0000 7fa71d0e6640 1 --2- 172.21.15.28:0/1076179746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7180659d0 0x7fa7180fefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:43.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.914+0000 7fa71d0e6640 1 -- 172.21.15.28:0/1076179746 >> 172.21.15.28:0/1076179746 conn(0x7fa7180f6a20 msgr2=0x7fa7180f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:43:43.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.914+0000 7fa71d0e6640 1 -- 172.21.15.28:0/1076179746 shutdown_connections 2024-07-03T15:43:43.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.914+0000 7fa71d0e6640 1 -- 172.21.15.28:0/1076179746 wait complete. 2024-07-03T15:43:43.919 INFO:tasks.workunit.client.1.smithi148.stdout:2/135: mkdir d5/d14/d22/d2b/d13/d2d 0 2024-07-03T15:43:43.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.915+0000 7fa71d0e6640 1 Processor -- start 2024-07-03T15:43:43.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.915+0000 7fa71d0e6640 1 -- start start 2024-07-03T15:43:43.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.915+0000 7fa71d0e6640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7180659d0 0x7fa71810cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:43:43.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.915+0000 7fa71d0e6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7180ff510 0x7fa71810d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:43:43.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.915+0000 7fa71d0e6640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71810db50 con 0x7fa7180659d0 2024-07-03T15:43:43.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.915+0000 7fa71d0e6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71810e320 con 0x7fa7180ff510 2024-07-03T15:43:43.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.916+0000 7fa70ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7180ff510 0x7fa71810d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:43:43.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.916+0000 7fa716d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7180659d0 0x7fa71810cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:43:43.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.916+0000 7fa716d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7180659d0 0x7fa71810cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53162/0 (socket says 172.21.15.28:53162) 2024-07-03T15:43:43.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.916+0000 7fa716d9d640 1 -- 172.21.15.28:0/2539328738 learned_addr learned my addr 172.21.15.28:0/2539328738 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:43:43.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.916+0000 7fa716d9d640 1 -- 172.21.15.28:0/2539328738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7180ff510 msgr2=0x7fa71810d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:43.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.916+0000 7fa716d9d640 1 --2- 172.21.15.28:0/2539328738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7180ff510 0x7fa71810d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:43.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.916+0000 7fa716d9d640 1 -- 172.21.15.28:0/2539328738 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa700005040 con 0x7fa7180659d0 2024-07-03T15:43:43.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.916+0000 7fa70ffff640 1 --2- 172.21.15.28:0/2539328738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7180ff510 0x7fa71810d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-03T15:43:43.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.916+0000 7fa716d9d640 1 --2- 172.21.15.28:0/2539328738 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7180659d0 0x7fa71810cfc0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa70800db00 tx=0x7fa708004e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:43:43.921 INFO:tasks.workunit.client.1.smithi148.stdout:0/70: mkdir da/dc/d13/d14 0 2024-07-03T15:43:43.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.917+0000 7fa70f7fe640 1 -- 172.21.15.28:0/2539328738 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa70800b910 con 0x7fa7180659d0 2024-07-03T15:43:43.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.917+0000 7fa71d0e6640 1 -- 172.21.15.28:0/2539328738 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa71810e5b0 con 0x7fa7180659d0 2024-07-03T15:43:43.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.917+0000 7fa70f7fe640 1 -- 172.21.15.28:0/2539328738 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa70800f040 con 0x7fa7180659d0 2024-07-03T15:43:43.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.917+0000 7fa70f7fe640 1 -- 172.21.15.28:0/2539328738 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa708012390 con 0x7fa7180659d0 2024-07-03T15:43:43.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.917+0000 7fa71d0e6640 1 -- 172.21.15.28:0/2539328738 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa71810eb20 con 0x7fa7180659d0 2024-07-03T15:43:43.923 INFO:tasks.workunit.client.1.smithi148.stdout:6/116: dwrite d3/f1a [0,4194304] 0 2024-07-03T15:43:43.923 INFO:tasks.workunit.client.1.smithi148.stdout:2/136: creat d5/d14/d22/d2b/d13/d2d/f2e x:0 0 0 2024-07-03T15:43:43.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.918+0000 7fa71d0e6640 1 -- 172.21.15.28:0/2539328738 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7181056d0 con 0x7fa7180659d0 2024-07-03T15:43:43.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.919+0000 7fa70f7fe640 1 -- 172.21.15.28:0/2539328738 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7fa708013430 con 0x7fa7180659d0 2024-07-03T15:43:43.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.919+0000 7fa70f7fe640 1 --2- 172.21.15.28:0/2539328738 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fa7040671e0 0x7fa704069690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:43:43.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.919+0000 7fa70f7fe640 1 -- 172.21.15.28:0/2539328738 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fa708084fb0 con 0x7fa7180659d0 2024-07-03T15:43:43.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.920+0000 7fa70ffff640 1 --2- 172.21.15.28:0/2539328738 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fa7040671e0 0x7fa704069690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:43:43.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.920+0000 7fa70ffff640 1 --2- 172.21.15.28:0/2539328738 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fa7040671e0 0x7fa704069690 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa7000043d0 tx=0x7fa700035040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:43:43.926 INFO:tasks.workunit.client.1.smithi148.stdout:6/117: mknod d3/c23 0 2024-07-03T15:43:43.928 INFO:tasks.workunit.client.1.smithi148.stdout:2/137: link d5/d14/d22/d2b/d13/d2d/f2e d5/f2f 0 2024-07-03T15:43:43.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:43.925+0000 7fa70f7fe640 1 -- 172.21.15.28:0/2539328738 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fa708052250 con 0x7fa7180659d0 2024-07-03T15:43:43.930 INFO:tasks.workunit.client.1.smithi148.stdout:6/118: creat d3/d1d/f24 x:0 0 0 2024-07-03T15:43:43.932 INFO:tasks.workunit.client.1.smithi148.stdout:2/138: symlink d5/l30 0 2024-07-03T15:43:43.934 INFO:tasks.workunit.client.1.smithi148.stdout:2/139: unlink d5/d7/fa 0 2024-07-03T15:43:43.934 INFO:tasks.workunit.client.1.smithi148.stdout:2/140: readlink d5/d14/d22/d2b/l27 0 2024-07-03T15:43:43.935 INFO:tasks.workunit.client.1.smithi148.stdout:6/119: dread d3/f1e [0,4194304] 0 2024-07-03T15:43:43.935 INFO:tasks.workunit.client.1.smithi148.stdout:6/120: creat d3/d1d/f25 x:0 0 0 2024-07-03T15:43:43.936 INFO:tasks.workunit.client.1.smithi148.stdout:2/141: symlink d5/l31 0 2024-07-03T15:43:43.936 INFO:tasks.workunit.client.1.smithi148.stdout:2/142: rename d5 to d5/d32 22 2024-07-03T15:43:43.936 INFO:tasks.workunit.client.1.smithi148.stdout:2/143: chown d5/d14/d22/d2b/f1e 318905309 1 2024-07-03T15:43:43.938 INFO:tasks.workunit.client.1.smithi148.stdout:6/121: mknod d3/d1d/c26 0 2024-07-03T15:43:43.939 INFO:tasks.workunit.client.1.smithi148.stdout:6/122: creat d3/d1d/f27 x:0 0 0 2024-07-03T15:43:43.940 INFO:tasks.workunit.client.1.smithi148.stdout:2/144: unlink d5/f18 0 2024-07-03T15:43:43.945 INFO:tasks.workunit.client.1.smithi148.stdout:2/145: rename d5/d14/d22/d2b/ld to d5/d26/l33 0 2024-07-03T15:43:43.945 INFO:tasks.workunit.client.1.smithi148.stdout:2/146: fdatasync d5/d14/d22/d2b/f1a 0 2024-07-03T15:43:43.945 INFO:tasks.workunit.client.1.smithi148.stdout:2/147: write d5/d14/d22/d2b/d13/d2d/f2e [180764,43728] 0 2024-07-03T15:43:43.949 INFO:tasks.workunit.client.1.smithi148.stdout:2/148: read d5/d14/d22/d2b/d13/f1b [442029,112381] 0 2024-07-03T15:43:43.972 INFO:tasks.workunit.client.1.smithi148.stdout:2/149: dwrite d5/d14/d22/d2b/d13/f1b [0,4194304] 0 2024-07-03T15:43:43.974 INFO:tasks.workunit.client.1.smithi148.stdout:2/150: rename d5/l30 to d5/d14/d22/d2b/d13/d2d/l34 0 2024-07-03T15:43:43.976 INFO:tasks.workunit.client.1.smithi148.stdout:2/151: creat d5/d7/f35 x:0 0 0 2024-07-03T15:43:44.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.164+0000 7fa71d0e6640 1 -- 172.21.15.28:0/2539328738 --> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa71810eeb0 con 0x7fa7040671e0 2024-07-03T15:43:44.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.173+0000 7fa70f7fe640 1 -- 172.21.15.28:0/2539328738 <== mgr.14210 v2:172.21.15.28:6800/421057178 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3432 (secure 0 0 0) 0x7fa71810eeb0 con 0x7fa7040671e0 2024-07-03T15:43:44.177 INFO:teuthology.orchestra.run.smithi028.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-03T15:43:44.177 INFO:teuthology.orchestra.run.smithi028.stdout:alertmanager.smithi028 smithi028 *:9093,9094 running (5m) 94s ago 7m 24.9M - 0.25.0 c8568f914cd2 b9a1a484a0a2 2024-07-03T15:43:44.177 INFO:teuthology.orchestra.run.smithi028.stdout:crash.smithi028 smithi028 running (7m) 94s ago 7m 6899k - 17.2.7-1047-gd4b063ab 4ca399d9e65d 535660e448f9 2024-07-03T15:43:44.177 INFO:teuthology.orchestra.run.smithi028.stdout:crash.smithi148 smithi148 running (6m) 101s ago 6m 6899k - 17.2.7-1047-gd4b063ab 4ca399d9e65d 4bb7675895e7 2024-07-03T15:43:44.177 INFO:teuthology.orchestra.run.smithi028.stdout:grafana.smithi028 smithi028 *:3000 running (5m) 94s ago 6m 88.5M - 9.4.7 954c08fa6188 07e0ae3c15aa 2024-07-03T15:43:44.177 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi028.rhpvve smithi028 running (119s) 94s ago 119s 15.6M - 17.2.7-1047-gd4b063ab 4ca399d9e65d 3ed5abd02cfb 2024-07-03T15:43:44.177 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi028.wmbxew smithi028 running (114s) 94s ago 113s 14.9M - 17.2.7-1047-gd4b063ab 4ca399d9e65d b3bf736998f9 2024-07-03T15:43:44.177 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi148.szomfl smithi148 running (111s) 101s ago 111s 12.9M - 17.2.7-1047-gd4b063ab 4ca399d9e65d d94f5c95fa8d 2024-07-03T15:43:44.177 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi148.xgpuld smithi148 running (117s) 101s ago 116s 10.7M - 17.2.7-1047-gd4b063ab 4ca399d9e65d fc70360a8de0 2024-07-03T15:43:44.177 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.smithi028.vtqrtz smithi028 *:9283 running (8m) 94s ago 8m 447M - 17.2.7-1047-gd4b063ab 4ca399d9e65d 0d26e74cb154 2024-07-03T15:43:44.177 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.smithi148.rjzsar smithi148 *:8443,9283 running (6m) 101s ago 6m 401M - 17.2.7-1047-gd4b063ab 4ca399d9e65d c637b3ae9fce 2024-07-03T15:43:44.177 INFO:teuthology.orchestra.run.smithi028.stdout:mon.smithi028 smithi028 running (8m) 94s ago 8m 43.2M 2048M 17.2.7-1047-gd4b063ab 4ca399d9e65d 7461f9bef4e0 2024-07-03T15:43:44.178 INFO:teuthology.orchestra.run.smithi028.stdout:mon.smithi148 smithi148 running (6m) 101s ago 6m 32.8M 2048M 17.2.7-1047-gd4b063ab 4ca399d9e65d 4c6c8ed2ab85 2024-07-03T15:43:44.178 INFO:teuthology.orchestra.run.smithi028.stdout:node-exporter.smithi028 smithi028 *:9100 running (7m) 94s ago 7m 16.0M - 1.5.0 0da6a335fe13 68318a0ca520 2024-07-03T15:43:44.178 INFO:teuthology.orchestra.run.smithi028.stdout:node-exporter.smithi148 smithi148 *:9100 running (6m) 101s ago 6m 17.6M - 1.5.0 0da6a335fe13 4994e60bd57c 2024-07-03T15:43:44.178 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0 smithi028 running (5m) 94s ago 5m 48.5M 4289M 17.2.7-1047-gd4b063ab 4ca399d9e65d b18089a439ba 2024-07-03T15:43:44.178 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1 smithi028 running (5m) 94s ago 5m 45.5M 4289M 17.2.7-1047-gd4b063ab 4ca399d9e65d 87d415dde231 2024-07-03T15:43:44.178 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2 smithi028 running (4m) 94s ago 4m 46.3M 4289M 17.2.7-1047-gd4b063ab 4ca399d9e65d b2cbb312212d 2024-07-03T15:43:44.178 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3 smithi148 running (4m) 101s ago 4m 44.4M 5312M 17.2.7-1047-gd4b063ab 4ca399d9e65d 5be69a27df31 2024-07-03T15:43:44.178 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4 smithi148 running (3m) 101s ago 3m 44.3M 5312M 17.2.7-1047-gd4b063ab 4ca399d9e65d 54f7c26c77ca 2024-07-03T15:43:44.178 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5 smithi148 running (3m) 101s ago 3m 41.4M 5312M 17.2.7-1047-gd4b063ab 4ca399d9e65d d185c6905384 2024-07-03T15:43:44.178 INFO:teuthology.orchestra.run.smithi028.stdout:prometheus.smithi028 smithi028 *:9095 running (5m) 94s ago 6m 54.2M - 2.43.0 a07b618ecd1d 97f838e118a3 2024-07-03T15:43:44.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.176+0000 7fa71d0e6640 1 -- 172.21.15.28:0/2539328738 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fa7040671e0 msgr2=0x7fa704069690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:44.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.176+0000 7fa71d0e6640 1 --2- 172.21.15.28:0/2539328738 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fa7040671e0 0x7fa704069690 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa7000043d0 tx=0x7fa700035040 comp rx=0 tx=0).stop 2024-07-03T15:43:44.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.176+0000 7fa71d0e6640 1 -- 172.21.15.28:0/2539328738 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7180659d0 msgr2=0x7fa71810cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:44.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.176+0000 7fa71d0e6640 1 --2- 172.21.15.28:0/2539328738 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7180659d0 0x7fa71810cfc0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa70800db00 tx=0x7fa708004e30 comp rx=0 tx=0).stop 2024-07-03T15:43:44.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.176+0000 7fa71d0e6640 1 -- 172.21.15.28:0/2539328738 shutdown_connections 2024-07-03T15:43:44.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.176+0000 7fa71d0e6640 1 --2- 172.21.15.28:0/2539328738 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fa7040671e0 0x7fa704069690 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:44.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.176+0000 7fa71d0e6640 1 --2- 172.21.15.28:0/2539328738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7180ff510 0x7fa71810d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:44.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.176+0000 7fa71d0e6640 1 --2- 172.21.15.28:0/2539328738 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7180659d0 0x7fa71810cfc0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:44.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.176+0000 7fa71d0e6640 1 -- 172.21.15.28:0/2539328738 >> 172.21.15.28:0/2539328738 conn(0x7fa7180f6a20 msgr2=0x7fa7180f6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:43:44.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.176+0000 7fa71d0e6640 1 -- 172.21.15.28:0/2539328738 shutdown_connections 2024-07-03T15:43:44.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.176+0000 7fa71d0e6640 1 -- 172.21.15.28:0/2539328738 wait complete. 2024-07-03T15:43:44.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.260+0000 7fc6a0b0d640 1 -- 172.21.15.28:0/3334644476 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc69c0fb290 msgr2=0x7fc69c0fb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:44.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.260+0000 7fc6a0b0d640 1 --2- 172.21.15.28:0/3334644476 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc69c0fb290 0x7fc69c0fb6a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc68c0048d0 tx=0x7fc68c009da0 comp rx=0 tx=0).stop 2024-07-03T15:43:44.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.260+0000 7fc6a0b0d640 1 -- 172.21.15.28:0/3334644476 shutdown_connections 2024-07-03T15:43:44.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.260+0000 7fc6a0b0d640 1 --2- 172.21.15.28:0/3334644476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc69c0fc560 0x7fc69c0fc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:44.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.260+0000 7fc6a0b0d640 1 --2- 172.21.15.28:0/3334644476 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc69c0fb290 0x7fc69c0fb6a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:44.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.260+0000 7fc6a0b0d640 1 -- 172.21.15.28:0/3334644476 >> 172.21.15.28:0/3334644476 conn(0x7fc69c0f6a50 msgr2=0x7fc69c0f8e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:43:44.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.260+0000 7fc6a0b0d640 1 -- 172.21.15.28:0/3334644476 shutdown_connections 2024-07-03T15:43:44.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.260+0000 7fc6a0b0d640 1 -- 172.21.15.28:0/3334644476 wait complete. 2024-07-03T15:43:44.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.261+0000 7fc6a0b0d640 1 Processor -- start 2024-07-03T15:43:44.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.261+0000 7fc6a0b0d640 1 -- start start 2024-07-03T15:43:44.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.261+0000 7fc6a0b0d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc69c0fb290 0x7fc69c10d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:43:44.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.261+0000 7fc6a0b0d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc69c0fc560 0x7fc69c10d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:43:44.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.261+0000 7fc6a0b0d640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c10e420 con 0x7fc69c0fb290 2024-07-03T15:43:44.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.261+0000 7fc6a0b0d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c10e590 con 0x7fc69c0fc560 2024-07-03T15:43:44.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.261+0000 7fc69a59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc69c0fb290 0x7fc69c10d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:43:44.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.261+0000 7fc69a59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc69c0fb290 0x7fc69c10d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53182/0 (socket says 172.21.15.28:53182) 2024-07-03T15:43:44.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.261+0000 7fc69a59c640 1 -- 172.21.15.28:0/2968271659 learned_addr learned my addr 172.21.15.28:0/2968271659 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:43:44.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.262+0000 7fc699d9b640 1 --2- 172.21.15.28:0/2968271659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc69c0fc560 0x7fc69c10d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:43:44.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.262+0000 7fc69a59c640 1 -- 172.21.15.28:0/2968271659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc69c0fc560 msgr2=0x7fc69c10d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:44.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.262+0000 7fc69a59c640 1 --2- 172.21.15.28:0/2968271659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc69c0fc560 0x7fc69c10d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:44.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.262+0000 7fc69a59c640 1 -- 172.21.15.28:0/2968271659 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc68c005040 con 0x7fc69c0fb290 2024-07-03T15:43:44.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.262+0000 7fc699d9b640 1 --2- 172.21.15.28:0/2968271659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc69c0fc560 0x7fc69c10d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-03T15:43:44.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.262+0000 7fc69a59c640 1 --2- 172.21.15.28:0/2968271659 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc69c0fb290 0x7fc69c10d0c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc68c0048a0 tx=0x7fc68c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:43:44.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.262+0000 7fc68affd640 1 -- 172.21.15.28:0/2968271659 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc68c038070 con 0x7fc69c0fb290 2024-07-03T15:43:44.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.262+0000 7fc68affd640 1 -- 172.21.15.28:0/2968271659 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc68c032970 con 0x7fc69c0fb290 2024-07-03T15:43:44.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.262+0000 7fc6a0b0d640 1 -- 172.21.15.28:0/2968271659 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc69c10e810 con 0x7fc69c0fb290 2024-07-03T15:43:44.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.262+0000 7fc68affd640 1 -- 172.21.15.28:0/2968271659 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc68c03d8c0 con 0x7fc69c0fb290 2024-07-03T15:43:44.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.262+0000 7fc6a0b0d640 1 -- 172.21.15.28:0/2968271659 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc69c10ecd0 con 0x7fc69c0fb290 2024-07-03T15:43:44.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.263+0000 7fc6a0b0d640 1 -- 172.21.15.28:0/2968271659 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc69c104670 con 0x7fc69c0fb290 2024-07-03T15:43:44.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.264+0000 7fc68affd640 1 -- 172.21.15.28:0/2968271659 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7fc68c030020 con 0x7fc69c0fb290 2024-07-03T15:43:44.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.264+0000 7fc68affd640 1 --2- 172.21.15.28:0/2968271659 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fc684067190 0x7fc684069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:43:44.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.264+0000 7fc68affd640 1 -- 172.21.15.28:0/2968271659 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fc68c0a5140 con 0x7fc69c0fb290 2024-07-03T15:43:44.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.264+0000 7fc699d9b640 1 --2- 172.21.15.28:0/2968271659 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fc684067190 0x7fc684069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:43:44.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.265+0000 7fc699d9b640 1 --2- 172.21.15.28:0/2968271659 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fc684067190 0x7fc684069640 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc690006fd0 tx=0x7fc690008040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:43:44.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.268+0000 7fc68affd640 1 -- 172.21.15.28:0/2968271659 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc68c072520 con 0x7fc69c0fb290 2024-07-03T15:43:44.424 INFO:tasks.workunit.client.1.smithi148.stdout:8/32: dread f2 [0,4194304] 0 2024-07-03T15:43:44.435 INFO:tasks.workunit.client.1.smithi148.stdout:7/52: sync 2024-07-03T15:43:44.435 INFO:tasks.workunit.client.1.smithi148.stdout:1/83: sync 2024-07-03T15:43:44.435 INFO:tasks.workunit.client.1.smithi148.stdout:9/162: sync 2024-07-03T15:43:44.435 INFO:tasks.workunit.client.1.smithi148.stdout:5/34: sync 2024-07-03T15:43:44.435 INFO:tasks.workunit.client.1.smithi148.stdout:4/34: sync 2024-07-03T15:43:44.435 INFO:tasks.workunit.client.1.smithi148.stdout:9/163: write d0/dd/f14 [738015,96213] 0 2024-07-03T15:43:44.435 INFO:tasks.workunit.client.1.smithi148.stdout:1/84: getdents d0/de 0 2024-07-03T15:43:44.436 INFO:tasks.workunit.client.1.smithi148.stdout:4/35: symlink d0/d8/l9 0 2024-07-03T15:43:44.438 INFO:tasks.workunit.client.1.smithi148.stdout:1/85: rmdir d0/d2 39 2024-07-03T15:43:44.438 INFO:tasks.workunit.client.1.smithi148.stdout:9/164: creat d0/dd/d25/f3c x:0 0 0 2024-07-03T15:43:44.439 INFO:tasks.workunit.client.1.smithi148.stdout:4/36: truncate d0/f5 961336 0 2024-07-03T15:43:44.439 INFO:tasks.workunit.client.1.smithi148.stdout:4/37: write d0/f4 [209268,120661] 0 2024-07-03T15:43:44.442 INFO:tasks.workunit.client.1.smithi148.stdout:1/86: link d0/f8 d0/d2/d6/f19 0 2024-07-03T15:43:44.442 INFO:tasks.workunit.client.1.smithi148.stdout:1/87: fsync d0/d2/fd 0 2024-07-03T15:43:44.443 INFO:tasks.workunit.client.1.smithi148.stdout:1/88: mkdir d0/de/d1a 0 2024-07-03T15:43:44.443 INFO:tasks.workunit.client.1.smithi148.stdout:1/89: stat d0/f8 0 2024-07-03T15:43:44.444 INFO:tasks.workunit.client.1.smithi148.stdout:1/90: write d0/d2/d6/d14/f15 [656326,2236] 0 2024-07-03T15:43:44.444 INFO:tasks.workunit.client.1.smithi148.stdout:5/35: dread f3 [0,4194304] 0 2024-07-03T15:43:44.445 INFO:tasks.workunit.client.1.smithi148.stdout:7/53: write d5/f7 [4164577,13769] 0 2024-07-03T15:43:44.456 INFO:tasks.workunit.client.1.smithi148.stdout:1/91: write d0/d2/f10 [3286402,102181] 0 2024-07-03T15:43:44.457 INFO:tasks.workunit.client.1.smithi148.stdout:1/92: read d0/d2/d6/f19 [4021143,69753] 0 2024-07-03T15:43:44.457 INFO:tasks.workunit.client.1.smithi148.stdout:7/54: write f2 [3668879,18648] 0 2024-07-03T15:43:44.457 INFO:tasks.workunit.client.1.smithi148.stdout:7/55: creat d5/f11 x:0 0 0 2024-07-03T15:43:44.457 INFO:tasks.workunit.client.1.smithi148.stdout:7/56: creat d5/d8/f12 x:0 0 0 2024-07-03T15:43:44.457 INFO:tasks.workunit.client.1.smithi148.stdout:7/57: write d5/f9 [620114,102522] 0 2024-07-03T15:43:44.457 INFO:tasks.workunit.client.1.smithi148.stdout:7/58: rename d5 to d5/d8/d13 22 2024-07-03T15:43:44.457 INFO:tasks.workunit.client.1.smithi148.stdout:1/93: rmdir d0/d2/d11 39 2024-07-03T15:43:44.465 INFO:tasks.workunit.client.1.smithi148.stdout:5/36: dread f4 [0,4194304] 0 2024-07-03T15:43:44.467 INFO:tasks.workunit.client.1.smithi148.stdout:4/38: dwrite d0/f6 [0,4194304] 0 2024-07-03T15:43:44.467 INFO:tasks.workunit.client.1.smithi148.stdout:9/165: dwrite d0/dd/d25/f3c [0,4194304] 0 2024-07-03T15:43:44.474 INFO:tasks.workunit.client.1.smithi148.stdout:9/166: link d0/f6 d0/d2f/f3d 0 2024-07-03T15:43:44.474 INFO:tasks.workunit.client.1.smithi148.stdout:9/167: chown d0/dd/d25/d28/f22 8295559 1 2024-07-03T15:43:44.491 INFO:tasks.workunit.client.1.smithi148.stdout:1/94: dwrite d0/d2/d6/f19 [0,4194304] 0 2024-07-03T15:43:44.492 INFO:tasks.workunit.client.1.smithi148.stdout:9/168: dwrite d0/d18/f21 [4194304,4194304] 0 2024-07-03T15:43:44.492 INFO:tasks.workunit.client.1.smithi148.stdout:5/37: dwrite f2 [0,4194304] 0 2024-07-03T15:43:44.494 INFO:tasks.workunit.client.1.smithi148.stdout:1/95: fsync d0/d2/d6/d14/f15 0 2024-07-03T15:43:44.497 INFO:tasks.workunit.client.1.smithi148.stdout:4/39: dread d0/f4 [0,4194304] 0 2024-07-03T15:43:44.497 INFO:tasks.workunit.client.1.smithi148.stdout:4/40: chown d0/f4 1385 1 2024-07-03T15:43:44.497 INFO:tasks.workunit.client.1.smithi148.stdout:9/169: mknod d0/dd/d25/d28/d19/c3e 0 2024-07-03T15:43:44.498 INFO:tasks.workunit.client.1.smithi148.stdout:5/38: mknod d7/da/cc 0 2024-07-03T15:43:44.499 INFO:tasks.workunit.client.1.smithi148.stdout:1/96: mknod d0/d2/d6/d14/c1b 0 2024-07-03T15:43:44.501 INFO:tasks.workunit.client.1.smithi148.stdout:4/41: mkdir d0/da 0 2024-07-03T15:43:44.505 INFO:tasks.workunit.client.1.smithi148.stdout:1/97: dread d0/d2/fd [0,4194304] 0 2024-07-03T15:43:44.507 INFO:tasks.workunit.client.1.smithi148.stdout:9/170: write d0/d2f/f3d [7441192,32193] 0 2024-07-03T15:43:44.508 INFO:tasks.workunit.client.1.smithi148.stdout:1/98: mknod d0/c1c 0 2024-07-03T15:43:44.510 INFO:tasks.workunit.client.1.smithi148.stdout:9/171: symlink d0/d18/l3f 0 2024-07-03T15:43:44.512 INFO:tasks.workunit.client.1.smithi148.stdout:1/99: rename d0/d2/d6/d14/c1b to d0/de/c1d 0 2024-07-03T15:43:44.513 INFO:tasks.workunit.client.1.smithi148.stdout:9/172: mkdir d0/dd/d25/d37/d40 0 2024-07-03T15:43:44.517 INFO:tasks.workunit.client.1.smithi148.stdout:9/173: creat d0/d2f/f41 x:0 0 0 2024-07-03T15:43:44.517 INFO:tasks.workunit.client.1.smithi148.stdout:9/174: chown d0/l27 3090498 1 2024-07-03T15:43:44.517 INFO:tasks.workunit.client.1.smithi148.stdout:9/175: stat d0/dd/d23 0 2024-07-03T15:43:44.524 INFO:tasks.workunit.client.1.smithi148.stdout:5/39: dwrite f4 [0,4194304] 0 2024-07-03T15:43:44.533 INFO:tasks.workunit.client.1.smithi148.stdout:4/42: dwrite d0/f6 [0,4194304] 0 2024-07-03T15:43:44.537 INFO:tasks.workunit.client.1.smithi148.stdout:5/40: dread f2 [0,4194304] 0 2024-07-03T15:43:44.542 INFO:tasks.workunit.client.1.smithi148.stdout:0/71: link da/dd/f10 da/dc/d13/d14/f15 0 2024-07-03T15:43:44.547 INFO:tasks.workunit.client.1.smithi148.stdout:1/100: dwrite d0/f12 [4194304,4194304] 0 2024-07-03T15:43:44.547 INFO:tasks.workunit.client.1.smithi148.stdout:9/176: dwrite d0/dd/d11/f35 [0,4194304] 0 2024-07-03T15:43:44.547 INFO:tasks.workunit.client.1.smithi148.stdout:4/43: dread d0/f3 [0,4194304] 0 2024-07-03T15:43:44.550 INFO:tasks.workunit.client.1.smithi148.stdout:9/177: mkdir d0/dd/d25/d42 0 2024-07-03T15:43:44.550 INFO:tasks.workunit.client.1.smithi148.stdout:9/178: chown d0/dd/d11/c20 0 1 2024-07-03T15:43:44.551 INFO:tasks.workunit.client.1.smithi148.stdout:9/179: truncate d0/f31 818584 0 2024-07-03T15:43:44.551 INFO:tasks.workunit.client.1.smithi148.stdout:4/44: mknod d0/cb 0 2024-07-03T15:43:44.552 INFO:tasks.workunit.client.1.smithi148.stdout:4/45: symlink d0/lc 0 2024-07-03T15:43:44.554 INFO:tasks.workunit.client.1.smithi148.stdout:4/46: creat d0/d8/fd x:0 0 0 2024-07-03T15:43:44.554 INFO:tasks.workunit.client.1.smithi148.stdout:4/47: write d0/f4 [4129890,103974] 0 2024-07-03T15:43:44.554 INFO:tasks.workunit.client.1.smithi148.stdout:4/48: chown d0/c1 168867847 1 2024-07-03T15:43:44.560 INFO:tasks.workunit.client.1.smithi148.stdout:6/123: getdents d3/d1d 0 2024-07-03T15:43:44.560 INFO:tasks.workunit.client.1.smithi148.stdout:6/124: mkdir d3/d1d/d28 0 2024-07-03T15:43:44.562 INFO:tasks.workunit.client.1.smithi148.stdout:6/125: mknod d3/d1d/c29 0 2024-07-03T15:43:44.573 INFO:tasks.workunit.client.1.smithi148.stdout:5/41: dwrite f3 [0,4194304] 0 2024-07-03T15:43:44.573 INFO:tasks.workunit.client.1.smithi148.stdout:4/49: dwrite d0/f3 [0,4194304] 0 2024-07-03T15:43:44.575 INFO:tasks.workunit.client.1.smithi148.stdout:4/50: rmdir d0/da 0 2024-07-03T15:43:44.575 INFO:tasks.workunit.client.1.smithi148.stdout:4/51: chown d0/f6 3729 1 2024-07-03T15:43:44.581 INFO:tasks.workunit.client.1.smithi148.stdout:6/126: dwrite d3/d1d/f25 [0,4194304] 0 2024-07-03T15:43:44.583 INFO:tasks.workunit.client.1.smithi148.stdout:5/42: dread f3 [0,4194304] 0 2024-07-03T15:43:44.584 INFO:tasks.workunit.client.1.smithi148.stdout:6/127: creat d3/d1d/d28/f2a x:0 0 0 2024-07-03T15:43:44.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.585+0000 7fc6a0b0d640 1 -- 172.21.15.28:0/2968271659 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc69c04b530 con 0x7fc69c0fb290 2024-07-03T15:43:44.589 INFO:tasks.workunit.client.1.smithi148.stdout:6/128: dread d3/f1c [0,4194304] 0 2024-07-03T15:43:44.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.586+0000 7fc68affd640 1 -- 172.21.15.28:0/2968271659 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+643 (secure 0 0 0) 0x7fc68c07b070 con 0x7fc69c0fb290 2024-07-03T15:43:44.590 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-07-03T15:43:44.590 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": { 2024-07-03T15:43:44.590 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 2 2024-07-03T15:43:44.590 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:43:44.590 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": { 2024-07-03T15:43:44.590 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 2 2024-07-03T15:43:44.590 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:43:44.590 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": { 2024-07-03T15:43:44.591 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 6 2024-07-03T15:43:44.591 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:43:44.591 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": { 2024-07-03T15:43:44.591 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 4 2024-07-03T15:43:44.591 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:43:44.591 INFO:teuthology.orchestra.run.smithi028.stdout: "overall": { 2024-07-03T15:43:44.591 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 14 2024-07-03T15:43:44.591 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-07-03T15:43:44.591 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-07-03T15:43:44.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.589+0000 7fc6a0b0d640 1 -- 172.21.15.28:0/2968271659 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fc684067190 msgr2=0x7fc684069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:44.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.589+0000 7fc6a0b0d640 1 --2- 172.21.15.28:0/2968271659 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fc684067190 0x7fc684069640 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc690006fd0 tx=0x7fc690008040 comp rx=0 tx=0).stop 2024-07-03T15:43:44.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.589+0000 7fc6a0b0d640 1 -- 172.21.15.28:0/2968271659 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc69c0fb290 msgr2=0x7fc69c10d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:44.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.589+0000 7fc6a0b0d640 1 --2- 172.21.15.28:0/2968271659 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc69c0fb290 0x7fc69c10d0c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc68c0048a0 tx=0x7fc68c009da0 comp rx=0 tx=0).stop 2024-07-03T15:43:44.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.589+0000 7fc6a0b0d640 1 -- 172.21.15.28:0/2968271659 shutdown_connections 2024-07-03T15:43:44.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.589+0000 7fc6a0b0d640 1 --2- 172.21.15.28:0/2968271659 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fc684067190 0x7fc684069640 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:44.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.589+0000 7fc6a0b0d640 1 --2- 172.21.15.28:0/2968271659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc69c0fc560 0x7fc69c10d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:44.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.589+0000 7fc6a0b0d640 1 --2- 172.21.15.28:0/2968271659 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc69c0fb290 0x7fc69c10d0c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:44.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.589+0000 7fc6a0b0d640 1 -- 172.21.15.28:0/2968271659 >> 172.21.15.28:0/2968271659 conn(0x7fc69c0f6a50 msgr2=0x7fc69c065eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:43:44.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.589+0000 7fc6a0b0d640 1 -- 172.21.15.28:0/2968271659 shutdown_connections 2024-07-03T15:43:44.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.589+0000 7fc6a0b0d640 1 -- 172.21.15.28:0/2968271659 wait complete. 2024-07-03T15:43:44.605 INFO:tasks.workunit.client.1.smithi148.stdout:5/43: dwrite f4 [0,4194304] 0 2024-07-03T15:43:44.605 INFO:tasks.workunit.client.1.smithi148.stdout:5/44: chown d7/da/cc 13751 1 2024-07-03T15:43:44.607 INFO:tasks.workunit.client.1.smithi148.stdout:5/45: dread d7/da/fb [0,4194304] 0 2024-07-03T15:43:44.607 INFO:tasks.workunit.client.1.smithi148.stdout:5/46: readlink l0 0 2024-07-03T15:43:44.607 INFO:tasks.workunit.client.1.smithi148.stdout:5/47: stat l0 0 2024-07-03T15:43:44.607 INFO:tasks.workunit.client.1.smithi148.stdout:6/129: dwrite f1 [0,4194304] 0 2024-07-03T15:43:44.610 INFO:tasks.workunit.client.1.smithi148.stdout:3/64: truncate f0 16325 0 2024-07-03T15:43:44.611 INFO:tasks.workunit.client.1.smithi148.stdout:3/65: mkdir dd/df 0 2024-07-03T15:43:44.611 INFO:tasks.workunit.client.1.smithi148.stdout:3/66: truncate f9 4679229 0 2024-07-03T15:43:44.612 INFO:tasks.workunit.client.1.smithi148.stdout:3/67: unlink f8 0 2024-07-03T15:43:44.614 INFO:tasks.workunit.client.1.smithi148.stdout:6/130: write d3/f1c [3073649,42208] 0 2024-07-03T15:43:44.639 INFO:tasks.workunit.client.1.smithi148.stdout:5/48: dwrite f4 [0,4194304] 0 2024-07-03T15:43:44.639 INFO:tasks.workunit.client.1.smithi148.stdout:3/68: dwrite fa [0,4194304] 0 2024-07-03T15:43:44.639 INFO:tasks.workunit.client.1.smithi148.stdout:6/131: dwrite d3/d1d/f24 [0,4194304] 0 2024-07-03T15:43:44.639 INFO:tasks.workunit.client.1.smithi148.stdout:6/132: write d3/f7 [306894,123119] 0 2024-07-03T15:43:44.639 INFO:tasks.workunit.client.1.smithi148.stdout:6/133: chown d3/f1e 254 1 2024-07-03T15:43:44.639 INFO:tasks.workunit.client.1.smithi148.stdout:3/69: symlink dd/df/l10 0 2024-07-03T15:43:44.641 INFO:tasks.workunit.client.1.smithi148.stdout:6/134: rename d3/c9 to d3/c2b 0 2024-07-03T15:43:44.642 INFO:tasks.workunit.client.1.smithi148.stdout:6/135: write d3/f1e [452804,129338] 0 2024-07-03T15:43:44.642 INFO:tasks.workunit.client.1.smithi148.stdout:6/136: creat d3/f2c x:0 0 0 2024-07-03T15:43:44.644 INFO:tasks.workunit.client.1.smithi148.stdout:6/137: symlink d3/d1d/d28/l2d 0 2024-07-03T15:43:44.645 INFO:tasks.workunit.client.1.smithi148.stdout:6/138: creat d3/d1d/d28/f2e x:0 0 0 2024-07-03T15:43:44.645 INFO:tasks.workunit.client.1.smithi148.stdout:6/139: creat d3/f2f x:0 0 0 2024-07-03T15:43:44.647 INFO:tasks.workunit.client.1.smithi148.stdout:5/49: write f4 [1172839,41924] 0 2024-07-03T15:43:44.648 INFO:tasks.workunit.client.1.smithi148.stdout:5/50: fdatasync f3 0 2024-07-03T15:43:44.648 INFO:tasks.workunit.client.1.smithi148.stdout:5/51: write f4 [708635,1637] 0 2024-07-03T15:43:44.659 INFO:tasks.workunit.client.1.smithi148.stdout:3/70: dwrite fa [0,4194304] 0 2024-07-03T15:43:44.659 INFO:tasks.workunit.client.1.smithi148.stdout:6/140: dwrite d3/d1d/f27 [0,4194304] 0 2024-07-03T15:43:44.664 INFO:tasks.workunit.client.1.smithi148.stdout:3/71: creat dd/df/f11 x:0 0 0 2024-07-03T15:43:44.666 INFO:tasks.workunit.client.1.smithi148.stdout:6/141: link d3/ld d3/d1d/d28/l30 0 2024-07-03T15:43:44.667 INFO:tasks.workunit.client.1.smithi148.stdout:6/142: mkdir d3/d1d/d28/d31 0 2024-07-03T15:43:44.668 INFO:tasks.workunit.client.1.smithi148.stdout:6/143: symlink d3/d1d/d28/d31/l32 0 2024-07-03T15:43:44.668 INFO:tasks.workunit.client.1.smithi148.stdout:6/144: write d3/d1d/d28/f2e [12745,44879] 0 2024-07-03T15:43:44.678 INFO:tasks.workunit.client.1.smithi148.stdout:0/72: truncate f7 2590722 0 2024-07-03T15:43:44.678 INFO:tasks.workunit.client.1.smithi148.stdout:0/73: write da/dd/f10 [365230,55158] 0 2024-07-03T15:43:44.679 INFO:tasks.workunit.client.1.smithi148.stdout:0/74: creat da/f16 x:0 0 0 2024-07-03T15:43:44.696 INFO:tasks.workunit.client.1.smithi148.stdout:0/75: dwrite f6 [4194304,4194304] 0 2024-07-03T15:43:44.696 INFO:tasks.workunit.client.1.smithi148.stdout:0/76: read - da/dc/f12 zero size 2024-07-03T15:43:44.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.693+0000 7f38c279a640 1 -- 172.21.15.28:0/2480636177 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38bc06fdf0 msgr2=0x7f38bc070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:44.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.693+0000 7f38c279a640 1 --2- 172.21.15.28:0/2480636177 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38bc06fdf0 0x7f38bc070260 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f38b00048d0 tx=0x7f38b0009da0 comp rx=0 tx=0).stop 2024-07-03T15:43:44.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.693+0000 7f38c279a640 1 -- 172.21.15.28:0/2480636177 shutdown_connections 2024-07-03T15:43:44.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.693+0000 7f38c279a640 1 --2- 172.21.15.28:0/2480636177 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38bc06fdf0 0x7f38bc070260 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:44.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.693+0000 7f38c279a640 1 --2- 172.21.15.28:0/2480636177 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38bc071380 0x7f38bc06f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:44.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.693+0000 7f38c279a640 1 -- 172.21.15.28:0/2480636177 >> 172.21.15.28:0/2480636177 conn(0x7f38bc0f7b40 msgr2=0x7f38bc0f9f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:43:44.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.694+0000 7f38c279a640 1 -- 172.21.15.28:0/2480636177 shutdown_connections 2024-07-03T15:43:44.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.694+0000 7f38c279a640 1 -- 172.21.15.28:0/2480636177 wait complete. 2024-07-03T15:43:44.698 INFO:tasks.workunit.client.1.smithi148.stdout:1/101: fsync d0/d2/d6/fa 0 2024-07-03T15:43:44.699 INFO:tasks.workunit.client.1.smithi148.stdout:1/102: write d0/d2/fd [3390034,100315] 0 2024-07-03T15:43:44.699 INFO:tasks.workunit.client.1.smithi148.stdout:1/103: getdents d0/de/d1a 0 2024-07-03T15:43:44.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.694+0000 7f38c279a640 1 Processor -- start 2024-07-03T15:43:44.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.694+0000 7f38c279a640 1 -- start start 2024-07-03T15:43:44.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.695+0000 7f38c279a640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38bc06fdf0 0x7f38bc10f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:43:44.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.695+0000 7f38c279a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38bc071380 0x7f38bc10f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:43:44.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.695+0000 7f38c279a640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38bc10fcf0 con 0x7f38bc06fdf0 2024-07-03T15:43:44.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.695+0000 7f38c279a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38bc110510 con 0x7f38bc071380 2024-07-03T15:43:44.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.695+0000 7f38bbfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38bc06fdf0 0x7f38bc10f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:43:44.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.695+0000 7f38bbfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38bc06fdf0 0x7f38bc10f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53208/0 (socket says 172.21.15.28:53208) 2024-07-03T15:43:44.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.695+0000 7f38bbfff640 1 -- 172.21.15.28:0/1821863728 learned_addr learned my addr 172.21.15.28:0/1821863728 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:43:44.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.696+0000 7f38bb7fe640 1 --2- 172.21.15.28:0/1821863728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38bc071380 0x7f38bc10f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:43:44.701 INFO:tasks.workunit.client.1.smithi148.stdout:0/77: mkdir da/d17 0 2024-07-03T15:43:44.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.696+0000 7f38bb7fe640 1 -- 172.21.15.28:0/1821863728 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38bc06fdf0 msgr2=0x7f38bc10f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:44.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.696+0000 7f38bb7fe640 1 --2- 172.21.15.28:0/1821863728 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38bc06fdf0 0x7f38bc10f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:44.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.696+0000 7f38bb7fe640 1 -- 172.21.15.28:0/1821863728 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b0005040 con 0x7f38bc071380 2024-07-03T15:43:44.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.697+0000 7f38bbfff640 1 --2- 172.21.15.28:0/1821863728 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38bc06fdf0 0x7f38bc10f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-03T15:43:44.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.697+0000 7f38bb7fe640 1 --2- 172.21.15.28:0/1821863728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38bc071380 0x7f38bc10f6a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f38b00048a0 tx=0x7f38b0009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:43:44.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.697+0000 7f38b8ff9640 1 -- 172.21.15.28:0/1821863728 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38b0038070 con 0x7f38bc071380 2024-07-03T15:43:44.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.697+0000 7f38c279a640 1 -- 172.21.15.28:0/1821863728 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38bc110740 con 0x7f38bc071380 2024-07-03T15:43:44.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.697+0000 7f38b8ff9640 1 -- 172.21.15.28:0/1821863728 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f38b00092b0 con 0x7f38bc071380 2024-07-03T15:43:44.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.697+0000 7f38b8ff9640 1 -- 172.21.15.28:0/1821863728 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38b0033660 con 0x7f38bc071380 2024-07-03T15:43:44.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.697+0000 7f38c279a640 1 -- 172.21.15.28:0/1821863728 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38bc110c50 con 0x7f38bc071380 2024-07-03T15:43:44.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.699+0000 7f38c279a640 1 -- 172.21.15.28:0/1821863728 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38bc0fb7e0 con 0x7f38bc071380 2024-07-03T15:43:44.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.699+0000 7f38b8ff9640 1 -- 172.21.15.28:0/1821863728 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f38b0030020 con 0x7f38bc071380 2024-07-03T15:43:44.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.700+0000 7f38b8ff9640 1 --2- 172.21.15.28:0/1821863728 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f38a4067190 0x7f38a4069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:43:44.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.700+0000 7f38b8ff9640 1 -- 172.21.15.28:0/1821863728 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f38b00a4830 con 0x7f38bc071380 2024-07-03T15:43:44.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.700+0000 7f38bbfff640 1 --2- 172.21.15.28:0/1821863728 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f38a4067190 0x7f38a4069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:43:44.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.700+0000 7f38bbfff640 1 --2- 172.21.15.28:0/1821863728 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f38a4067190 0x7f38a4069640 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f38ac007e20 tx=0x7f38ac00b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:43:44.704 INFO:tasks.workunit.client.1.smithi148.stdout:7/59: truncate d5/f7 2873003 0 2024-07-03T15:43:44.705 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:44 smithi148 ceph-mon[36056]: Upgrade: Updating mgr.smithi148.rjzsar 2024-07-03T15:43:44.705 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:44 smithi148 ceph-mon[36056]: Deploying daemon mgr.smithi148.rjzsar on smithi148 2024-07-03T15:43:44.705 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:44 smithi148 ceph-mon[36056]: from='client.24381 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:43:44.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:44.705+0000 7f38b8ff9640 1 -- 172.21.15.28:0/1821863728 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f38b0071bd0 con 0x7f38bc071380 2024-07-03T15:43:44.710 INFO:tasks.workunit.client.1.smithi148.stdout:7/60: getdents d5 0 2024-07-03T15:43:44.715 INFO:tasks.workunit.client.1.smithi148.stdout:7/61: symlink d5/l14 0 2024-07-03T15:43:44.717 INFO:tasks.workunit.client.1.smithi148.stdout:9/180: rename d0/dd/d25/d28 to d0/dd/d23/d43 0 2024-07-03T15:43:44.719 INFO:tasks.workunit.client.1.smithi148.stdout:7/62: rmdir d5 39 2024-07-03T15:43:44.723 INFO:tasks.workunit.client.1.smithi148.stdout:5/52: rename f2 to d7/da/fd 0 2024-07-03T15:43:44.725 INFO:tasks.workunit.client.1.smithi148.stdout:9/181: creat d0/dd/d25/d37/d40/f44 x:0 0 0 2024-07-03T15:43:44.727 INFO:tasks.workunit.client.1.smithi148.stdout:7/63: mkdir d5/d15 0 2024-07-03T15:43:44.729 INFO:tasks.workunit.client.1.smithi148.stdout:5/53: mknod d7/ce 0 2024-07-03T15:43:44.730 INFO:tasks.workunit.client.1.smithi148.stdout:5/54: dread d7/da/fb [0,4194304] 0 2024-07-03T15:43:44.732 INFO:tasks.workunit.client.1.smithi148.stdout:9/182: symlink d0/dd/d29/l45 0 2024-07-03T15:43:44.732 INFO:tasks.workunit.client.1.smithi148.stdout:9/183: readlink d0/dd/l9 0 2024-07-03T15:43:44.735 INFO:tasks.workunit.client.1.smithi148.stdout:7/64: rename d5/d15 to d5/d16 0 2024-07-03T15:43:44.744 INFO:tasks.workunit.client.1.smithi148.stdout:8/33: truncate f2 1650258 0 2024-07-03T15:43:44.746 INFO:tasks.workunit.client.1.smithi148.stdout:7/65: getdents d5/d8 0 2024-07-03T15:43:44.746 INFO:tasks.workunit.client.1.smithi148.stdout:7/66: fsync d5/f11 0 2024-07-03T15:43:44.750 INFO:tasks.workunit.client.1.smithi148.stdout:9/184: dwrite d0/dd/f1a [0,4194304] 0 2024-07-03T15:43:44.750 INFO:tasks.workunit.client.1.smithi148.stdout:9/185: write d0/dd/f1f [1232182,23065] 0 2024-07-03T15:43:44.753 INFO:tasks.workunit.client.1.smithi148.stdout:9/186: mkdir d0/dd/d46 0 2024-07-03T15:43:44.753 INFO:tasks.workunit.client.1.smithi148.stdout:9/187: chown d0/d18/c1b 4683 1 2024-07-03T15:43:44.754 INFO:tasks.workunit.client.1.smithi148.stdout:9/188: read d0/d18/f26 [98408,5504] 0 2024-07-03T15:43:44.754 INFO:tasks.workunit.client.1.smithi148.stdout:9/189: chown d0/dd/d23/d43/f22 3870935 1 2024-07-03T15:43:44.756 INFO:tasks.workunit.client.1.smithi148.stdout:3/72: truncate f0 890369 0 2024-07-03T15:43:44.756 INFO:tasks.workunit.client.1.smithi148.stdout:3/73: stat l7 0 2024-07-03T15:43:44.756 INFO:tasks.workunit.client.1.smithi148.stdout:2/152: sync 2024-07-03T15:43:44.756 INFO:tasks.workunit.client.1.smithi148.stdout:9/190: mkdir d0/d18/d24/d47 0 2024-07-03T15:43:44.757 INFO:tasks.workunit.client.1.smithi148.stdout:9/191: readlink d0/l27 0 2024-07-03T15:43:44.760 INFO:tasks.workunit.client.1.smithi148.stdout:9/192: unlink d0/f17 0 2024-07-03T15:43:44.770 INFO:tasks.workunit.client.1.smithi148.stdout:5/55: truncate f4 1983155 0 2024-07-03T15:43:44.772 INFO:tasks.workunit.client.1.smithi148.stdout:5/56: mkdir d7/da/df 0 2024-07-03T15:43:44.780 INFO:tasks.workunit.client.1.smithi148.stdout:6/145: getdents d3/d1d/d28/d31 0 2024-07-03T15:43:44.782 INFO:tasks.workunit.client.1.smithi148.stdout:0/78: fsync da/f16 0 2024-07-03T15:43:44.782 INFO:tasks.workunit.client.1.smithi148.stdout:0/79: chown f7 7973 1 2024-07-03T15:43:44.783 INFO:tasks.workunit.client.1.smithi148.stdout:6/146: stat d3/l12 0 2024-07-03T15:43:44.785 INFO:tasks.workunit.client.1.smithi148.stdout:0/80: link da/dc/f12 da/dc/d13/f18 0 2024-07-03T15:43:44.786 INFO:tasks.workunit.client.1.smithi148.stdout:0/81: getdents da/d17 0 2024-07-03T15:43:44.786 INFO:tasks.workunit.client.1.smithi148.stdout:0/82: creat da/dc/f19 x:0 0 0 2024-07-03T15:43:44.786 INFO:tasks.workunit.client.1.smithi148.stdout:0/83: truncate da/f16 815787 0 2024-07-03T15:43:44.787 INFO:tasks.workunit.client.1.smithi148.stdout:6/147: rename d3/c1b to d3/d1d/d28/c33 0 2024-07-03T15:43:44.787 INFO:tasks.workunit.client.1.smithi148.stdout:6/148: creat d3/f34 x:0 0 0 2024-07-03T15:43:44.788 INFO:tasks.workunit.client.1.smithi148.stdout:6/149: fsync d3/f17 0 2024-07-03T15:43:44.788 INFO:tasks.workunit.client.1.smithi148.stdout:6/150: dread - d3/d1d/d28/f2a zero size 2024-07-03T15:43:44.789 INFO:tasks.workunit.client.1.smithi148.stdout:5/57: dwrite d7/da/fb [0,4194304] 0 2024-07-03T15:43:44.789 INFO:tasks.workunit.client.1.smithi148.stdout:5/58: fdatasync f4 0 2024-07-03T15:43:44.790 INFO:tasks.workunit.client.1.smithi148.stdout:0/84: symlink da/dc/d13/l1a 0 2024-07-03T15:43:44.794 INFO:tasks.workunit.client.1.smithi148.stdout:7/67: write d5/f7 [923336,20616] 0 2024-07-03T15:43:44.795 INFO:tasks.workunit.client.1.smithi148.stdout:5/59: creat d7/da/df/f10 x:0 0 0 2024-07-03T15:43:44.797 INFO:tasks.workunit.client.1.smithi148.stdout:0/85: unlink da/dc/f12 0 2024-07-03T15:43:44.798 INFO:tasks.workunit.client.1.smithi148.stdout:6/151: link d3/l5 d3/d1d/l35 0 2024-07-03T15:43:44.798 INFO:tasks.workunit.client.1.smithi148.stdout:6/152: write d3/fa [627640,54530] 0 2024-07-03T15:43:44.801 INFO:tasks.workunit.client.1.smithi148.stdout:7/68: mknod d5/d16/c17 0 2024-07-03T15:43:44.801 INFO:tasks.workunit.client.1.smithi148.stdout:7/69: write d5/f6 [1797167,54766] 0 2024-07-03T15:43:44.801 INFO:tasks.workunit.client.1.smithi148.stdout:7/70: chown d5/f7 437401 1 2024-07-03T15:43:44.802 INFO:tasks.workunit.client.1.smithi148.stdout:5/60: symlink d7/da/l11 0 2024-07-03T15:43:44.802 INFO:tasks.workunit.client.1.smithi148.stdout:5/61: dread - d7/da/df/f10 zero size 2024-07-03T15:43:44.803 INFO:tasks.workunit.client.1.smithi148.stdout:0/86: truncate f2 1791561 0 2024-07-03T15:43:44.807 INFO:tasks.workunit.client.1.smithi148.stdout:0/87: dread da/dc/d13/d14/f15 [0,4194304] 0 2024-07-03T15:43:44.812 INFO:tasks.workunit.client.1.smithi148.stdout:7/71: rename d5/l10 to d5/d8/l18 0 2024-07-03T15:43:44.815 INFO:tasks.workunit.client.1.smithi148.stdout:0/88: creat da/dc/d13/f1b x:0 0 0 2024-07-03T15:43:44.817 INFO:tasks.workunit.client.1.smithi148.stdout:7/72: creat d5/f19 x:0 0 0 2024-07-03T15:43:44.823 INFO:tasks.workunit.client.1.smithi148.stdout:5/62: dwrite d7/da/fb [0,4194304] 0 2024-07-03T15:43:44.824 INFO:tasks.workunit.client.1.smithi148.stdout:5/63: write d7/da/df/f10 [816599,78285] 0 2024-07-03T15:43:44.826 INFO:tasks.workunit.client.1.smithi148.stdout:7/73: dread d5/f7 [0,4194304] 0 2024-07-03T15:43:44.829 INFO:tasks.workunit.client.1.smithi148.stdout:8/34: write d4/d6/f7 [1567334,127934] 0 2024-07-03T15:43:44.829 INFO:tasks.workunit.client.1.smithi148.stdout:8/35: chown d4/da/fb 30 1 2024-07-03T15:43:44.829 INFO:tasks.workunit.client.1.smithi148.stdout:8/36: readlink d4/l5 0 2024-07-03T15:43:44.829 INFO:tasks.workunit.client.1.smithi148.stdout:8/37: write f2 [670248,120105] 0 2024-07-03T15:43:44.829 INFO:tasks.workunit.client.1.smithi148.stdout:8/38: readlink d4/d6/l9 0 2024-07-03T15:43:44.829 INFO:tasks.workunit.client.1.smithi148.stdout:8/39: write d4/d6/f8 [163243,68798] 0 2024-07-03T15:43:44.830 INFO:tasks.workunit.client.1.smithi148.stdout:7/74: rmdir d5/d8 39 2024-07-03T15:43:44.830 INFO:tasks.workunit.client.1.smithi148.stdout:4/52: sync 2024-07-03T15:43:44.830 INFO:tasks.workunit.client.1.smithi148.stdout:4/53: fdatasync d0/d8/fd 0 2024-07-03T15:43:44.833 INFO:tasks.workunit.client.1.smithi148.stdout:8/40: unlink d4/cd 0 2024-07-03T15:43:44.834 INFO:tasks.workunit.client.1.smithi148.stdout:8/41: creat d4/d6/fe x:0 0 0 2024-07-03T15:43:44.835 INFO:tasks.workunit.client.1.smithi148.stdout:6/153: dwrite d3/d1d/f21 [0,4194304] 0 2024-07-03T15:43:44.835 INFO:tasks.workunit.client.1.smithi148.stdout:4/54: symlink d0/d8/le 0 2024-07-03T15:43:44.835 INFO:tasks.workunit.client.1.smithi148.stdout:4/55: readlink d0/d8/le 0 2024-07-03T15:43:44.835 INFO:tasks.workunit.client.1.smithi148.stdout:4/56: stat d0/f6 0 2024-07-03T15:43:44.835 INFO:tasks.workunit.client.1.smithi148.stdout:4/57: fsync d0/f3 0 2024-07-03T15:43:44.835 INFO:tasks.workunit.client.1.smithi148.stdout:4/58: creat d0/d8/ff x:0 0 0 2024-07-03T15:43:44.835 INFO:tasks.workunit.client.1.smithi148.stdout:4/59: fsync d0/f3 0 2024-07-03T15:43:44.835 INFO:tasks.workunit.client.1.smithi148.stdout:4/60: creat d0/d8/f10 x:0 0 0 2024-07-03T15:43:44.835 INFO:tasks.workunit.client.1.smithi148.stdout:4/61: fdatasync d0/f3 0 2024-07-03T15:43:44.835 INFO:tasks.workunit.client.1.smithi148.stdout:4/62: chown d0/l2 624 1 2024-07-03T15:43:44.836 INFO:tasks.workunit.client.1.smithi148.stdout:7/75: dread f2 [0,4194304] 0 2024-07-03T15:43:44.837 INFO:tasks.workunit.client.1.smithi148.stdout:6/154: mknod d3/d1d/d28/c36 0 2024-07-03T15:43:44.837 INFO:tasks.workunit.client.1.smithi148.stdout:6/155: creat d3/f37 x:0 0 0 2024-07-03T15:43:44.839 INFO:tasks.workunit.client.1.smithi148.stdout:3/74: truncate f0 1446971 0 2024-07-03T15:43:44.839 INFO:tasks.workunit.client.1.smithi148.stdout:3/75: write f5 [715716,78982] 0 2024-07-03T15:43:44.842 INFO:tasks.workunit.client.1.smithi148.stdout:0/89: dwrite da/dc/f19 [0,4194304] 0 2024-07-03T15:43:44.844 INFO:tasks.workunit.client.1.smithi148.stdout:4/63: mkdir d0/d8/d11 0 2024-07-03T15:43:44.845 INFO:tasks.workunit.client.1.smithi148.stdout:7/76: rename d5/d8/fb to d5/d16/f1a 0 2024-07-03T15:43:44.847 INFO:tasks.workunit.client.1.smithi148.stdout:9/193: truncate d0/dd/f1a 1559297 0 2024-07-03T15:43:44.853 INFO:tasks.workunit.client.1.smithi148.stdout:4/64: write d0/f6 [2690835,26066] 0 2024-07-03T15:43:44.853 INFO:tasks.workunit.client.1.smithi148.stdout:7/77: dread d5/f6 [0,4194304] 0 2024-07-03T15:43:44.856 INFO:tasks.workunit.client.1.smithi148.stdout:0/90: mknod da/c1c 0 2024-07-03T15:43:44.856 INFO:tasks.workunit.client.1.smithi148.stdout:3/76: rmdir dd/df 39 2024-07-03T15:43:44.858 INFO:tasks.workunit.client.1.smithi148.stdout:4/65: mknod d0/d8/d11/c12 0 2024-07-03T15:43:44.860 INFO:tasks.workunit.client.1.smithi148.stdout:6/156: dwrite d3/f1c [0,4194304] 0 2024-07-03T15:43:44.860 INFO:tasks.workunit.client.1.smithi148.stdout:6/157: readlink d3/d1d/d28/l30 0 2024-07-03T15:43:44.861 INFO:tasks.workunit.client.1.smithi148.stdout:7/78: link l4 d5/l1b 0 2024-07-03T15:43:44.864 INFO:tasks.workunit.client.1.smithi148.stdout:0/91: rename da/dd to da/dc/d13/d1d 0 2024-07-03T15:43:44.864 INFO:tasks.workunit.client.1.smithi148.stdout:0/92: chown da/dc/d13/d14 13 1 2024-07-03T15:43:44.864 INFO:tasks.workunit.client.1.smithi148.stdout:0/93: truncate da/dc/d13/d1d/f10 750810 0 2024-07-03T15:43:44.868 INFO:tasks.workunit.client.1.smithi148.stdout:6/158: stat d3/cb 0 2024-07-03T15:43:44.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:44 smithi028 ceph-mon[29148]: Upgrade: Updating mgr.smithi148.rjzsar 2024-07-03T15:43:44.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:44 smithi028 ceph-mon[29148]: Deploying daemon mgr.smithi148.rjzsar on smithi148 2024-07-03T15:43:44.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:44 smithi028 ceph-mon[29148]: from='client.24381 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:43:44.873 INFO:tasks.workunit.client.1.smithi148.stdout:7/79: unlink d5/f19 0 2024-07-03T15:43:44.883 INFO:tasks.workunit.client.1.smithi148.stdout:6/159: read d3/f1c [1051208,29278] 0 2024-07-03T15:43:44.885 INFO:tasks.workunit.client.1.smithi148.stdout:8/42: dwrite f2 [0,4194304] 0 2024-07-03T15:43:44.887 INFO:tasks.workunit.client.1.smithi148.stdout:6/160: creat d3/d1d/f38 x:0 0 0 2024-07-03T15:43:44.887 INFO:tasks.workunit.client.1.smithi148.stdout:6/161: read - d3/d1d/d28/f2a zero size 2024-07-03T15:43:44.888 INFO:tasks.workunit.client.1.smithi148.stdout:0/94: dwrite da/dc/d13/f1b [0,4194304] 0 2024-07-03T15:43:44.889 INFO:tasks.workunit.client.1.smithi148.stdout:3/77: dwrite f9 [0,4194304] 0 2024-07-03T15:43:44.889 INFO:tasks.workunit.client.1.smithi148.stdout:5/64: dwrite f4 [0,4194304] 0 2024-07-03T15:43:44.899 INFO:tasks.workunit.client.1.smithi148.stdout:3/78: dread f9 [0,4194304] 0 2024-07-03T15:43:44.900 INFO:tasks.workunit.client.1.smithi148.stdout:3/79: dread - dd/df/f11 zero size 2024-07-03T15:43:44.900 INFO:tasks.workunit.client.1.smithi148.stdout:3/80: fsync f9 0 2024-07-03T15:43:44.909 INFO:tasks.workunit.client.1.smithi148.stdout:5/65: mkdir d7/da/d12 0 2024-07-03T15:43:44.910 INFO:tasks.workunit.client.1.smithi148.stdout:8/43: getdents d4/da 0 2024-07-03T15:43:44.913 INFO:tasks.workunit.client.1.smithi148.stdout:3/81: write f0 [1760550,50879] 0 2024-07-03T15:43:44.917 INFO:tasks.workunit.client.1.smithi148.stdout:5/66: mkdir d7/d13 0 2024-07-03T15:43:44.917 INFO:tasks.workunit.client.1.smithi148.stdout:5/67: truncate d7/da/fb 4897425 0 2024-07-03T15:43:44.919 INFO:tasks.workunit.client.1.smithi148.stdout:3/82: fsync f5 0 2024-07-03T15:43:44.919 INFO:tasks.workunit.client.1.smithi148.stdout:5/68: symlink d7/da/l14 0 2024-07-03T15:43:44.920 INFO:tasks.workunit.client.1.smithi148.stdout:7/80: getdents d5/d8 0 2024-07-03T15:43:44.923 INFO:tasks.workunit.client.1.smithi148.stdout:3/83: creat dd/f12 x:0 0 0 2024-07-03T15:43:44.925 INFO:tasks.workunit.client.1.smithi148.stdout:8/44: dwrite d4/d6/f8 [0,4194304] 0 2024-07-03T15:43:44.925 INFO:tasks.workunit.client.1.smithi148.stdout:8/45: fdatasync d4/da/fb 0 2024-07-03T15:43:44.925 INFO:tasks.workunit.client.1.smithi148.stdout:9/194: fsync d0/dd/f1a 0 2024-07-03T15:43:44.926 INFO:tasks.workunit.client.1.smithi148.stdout:7/81: mkdir d5/d16/d1c 0 2024-07-03T15:43:44.928 INFO:tasks.workunit.client.1.smithi148.stdout:5/69: dread d7/da/df/f10 [0,4194304] 0 2024-07-03T15:43:44.930 INFO:tasks.workunit.client.1.smithi148.stdout:8/46: symlink d4/lf 0 2024-07-03T15:43:44.930 INFO:tasks.workunit.client.1.smithi148.stdout:3/84: dread f0 [0,4194304] 0 2024-07-03T15:43:44.930 INFO:tasks.workunit.client.1.smithi148.stdout:3/85: dread - dd/df/f11 zero size 2024-07-03T15:43:44.930 INFO:tasks.workunit.client.1.smithi148.stdout:3/86: truncate f0 2254313 0 2024-07-03T15:43:44.932 INFO:tasks.workunit.client.1.smithi148.stdout:9/195: creat d0/dd/d23/d43/d19/f48 x:0 0 0 2024-07-03T15:43:44.936 INFO:tasks.workunit.client.1.smithi148.stdout:8/47: symlink d4/d6/l10 0 2024-07-03T15:43:44.940 INFO:tasks.workunit.client.1.smithi148.stdout:4/66: rename d0/d8/d11 to d0/d8/d13 0 2024-07-03T15:43:44.941 INFO:tasks.workunit.client.1.smithi148.stdout:8/48: creat d4/f11 x:0 0 0 2024-07-03T15:43:44.941 INFO:tasks.workunit.client.1.smithi148.stdout:5/70: dread d7/da/fb [0,4194304] 0 2024-07-03T15:43:44.942 INFO:tasks.workunit.client.1.smithi148.stdout:4/67: dread d0/f5 [0,4194304] 0 2024-07-03T15:43:44.951 INFO:tasks.workunit.client.1.smithi148.stdout:4/68: mknod d0/c14 0 2024-07-03T15:43:44.953 INFO:tasks.workunit.client.1.smithi148.stdout:0/95: unlink da/dc/d13/d14/f15 0 2024-07-03T15:43:44.953 INFO:tasks.workunit.client.1.smithi148.stdout:0/96: fsync f6 0 2024-07-03T15:43:44.962 INFO:tasks.workunit.client.1.smithi148.stdout:6/162: rmdir d3 39 2024-07-03T15:43:44.962 INFO:tasks.workunit.client.1.smithi148.stdout:6/163: dread - d3/d1d/f38 zero size 2024-07-03T15:43:44.964 INFO:tasks.workunit.client.1.smithi148.stdout:6/164: symlink d3/l39 0 2024-07-03T15:43:44.964 INFO:tasks.workunit.client.1.smithi148.stdout:6/165: write d3/f17 [2494244,73797] 0 2024-07-03T15:43:44.975 INFO:tasks.workunit.client.1.smithi148.stdout:6/166: write d3/d1d/f24 [3458740,81276] 0 2024-07-03T15:43:44.975 INFO:tasks.workunit.client.1.smithi148.stdout:5/71: dwrite f4 [0,4194304] 0 2024-07-03T15:43:44.980 INFO:tasks.workunit.client.1.smithi148.stdout:5/72: mkdir d7/d15 0 2024-07-03T15:43:44.989 INFO:tasks.workunit.client.1.smithi148.stdout:6/167: dwrite d3/f1c [4194304,4194304] 0 2024-07-03T15:43:44.989 INFO:tasks.workunit.client.1.smithi148.stdout:6/168: truncate d3/f34 546166 0 2024-07-03T15:43:44.989 INFO:tasks.workunit.client.1.smithi148.stdout:6/169: dread - d3/f2c zero size 2024-07-03T15:43:44.995 INFO:tasks.workunit.client.1.smithi148.stdout:6/170: symlink d3/d1d/l3a 0 2024-07-03T15:43:44.998 INFO:tasks.workunit.client.1.smithi148.stdout:6/171: getdents d3/d1d/d28 0 2024-07-03T15:43:44.999 INFO:tasks.workunit.client.1.smithi148.stdout:6/172: dread d3/f1e [0,4194304] 0 2024-07-03T15:43:45.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.043+0000 7f38c279a640 1 -- 172.21.15.28:0/1821863728 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f38bc04b530 con 0x7f38bc071380 2024-07-03T15:43:45.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.044+0000 7f38b8ff9640 1 -- 172.21.15.28:0/1821863728 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1838 (secure 0 0 0) 0x7f38b0041740 con 0x7f38bc071380 2024-07-03T15:43:45.050 INFO:teuthology.orchestra.run.smithi028.stdout:e17 2024-07-03T15:43:45.050 INFO:teuthology.orchestra.run.smithi028.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-03T15:43:45.050 INFO:teuthology.orchestra.run.smithi028.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:43:45.051 INFO:teuthology.orchestra.run.smithi028.stdout:legacy client fscid: 1 2024-07-03T15:43:45.051 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:43:45.051 INFO:teuthology.orchestra.run.smithi028.stdout:Filesystem 'cephfs' (1) 2024-07-03T15:43:45.051 INFO:teuthology.orchestra.run.smithi028.stdout:fs_name cephfs 2024-07-03T15:43:45.051 INFO:teuthology.orchestra.run.smithi028.stdout:epoch 16 2024-07-03T15:43:45.054 INFO:teuthology.orchestra.run.smithi028.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-03T15:43:45.054 INFO:teuthology.orchestra.run.smithi028.stdout:created 2024-07-03T15:41:40.656298+0000 2024-07-03T15:43:45.054 INFO:teuthology.orchestra.run.smithi028.stdout:modified 2024-07-03T15:41:56.454984+0000 2024-07-03T15:43:45.054 INFO:teuthology.orchestra.run.smithi028.stdout:tableserver 0 2024-07-03T15:43:45.055 INFO:teuthology.orchestra.run.smithi028.stdout:root 0 2024-07-03T15:43:45.055 INFO:teuthology.orchestra.run.smithi028.stdout:session_timeout 60 2024-07-03T15:43:45.055 INFO:teuthology.orchestra.run.smithi028.stdout:session_autoclose 300 2024-07-03T15:43:45.055 INFO:teuthology.orchestra.run.smithi028.stdout:max_file_size 1099511627776 2024-07-03T15:43:45.056 INFO:teuthology.orchestra.run.smithi028.stdout:required_client_features {} 2024-07-03T15:43:45.056 INFO:teuthology.orchestra.run.smithi028.stdout:last_failure 0 2024-07-03T15:43:45.056 INFO:teuthology.orchestra.run.smithi028.stdout:last_failure_osd_epoch 0 2024-07-03T15:43:45.056 INFO:teuthology.orchestra.run.smithi028.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:43:45.056 INFO:teuthology.orchestra.run.smithi028.stdout:max_mds 1 2024-07-03T15:43:45.056 INFO:teuthology.orchestra.run.smithi028.stdout:in 0 2024-07-03T15:43:45.056 INFO:teuthology.orchestra.run.smithi028.stdout:up {0=14450} 2024-07-03T15:43:45.057 INFO:teuthology.orchestra.run.smithi028.stdout:failed 2024-07-03T15:43:45.057 INFO:teuthology.orchestra.run.smithi028.stdout:damaged 2024-07-03T15:43:45.057 INFO:teuthology.orchestra.run.smithi028.stdout:stopped 2024-07-03T15:43:45.057 INFO:teuthology.orchestra.run.smithi028.stdout:data_pools [3] 2024-07-03T15:43:45.057 INFO:teuthology.orchestra.run.smithi028.stdout:metadata_pool 2 2024-07-03T15:43:45.058 INFO:teuthology.orchestra.run.smithi028.stdout:inline_data disabled 2024-07-03T15:43:45.058 INFO:teuthology.orchestra.run.smithi028.stdout:balancer 2024-07-03T15:43:45.058 INFO:teuthology.orchestra.run.smithi028.stdout:standby_count_wanted 1 2024-07-03T15:43:45.059 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi028.rhpvve{0:14450} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.28:6826/1613936042,v1:172.21.15.28:6827/1613936042] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:43:45.059 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi028.wmbxew{0:14460} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.28:6828/1730336821,v1:172.21.15.28:6829/1730336821] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:43:45.059 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:43:45.059 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:43:45.059 INFO:teuthology.orchestra.run.smithi028.stdout:Standby daemons: 2024-07-03T15:43:45.060 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:43:45.060 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi148.szomfl{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/4171231858,v1:172.21.15.148:6827/4171231858] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:43:45.060 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi148.xgpuld{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/2347536398,v1:172.21.15.148:6825/2347536398] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:43:45.063 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 17 2024-07-03T15:43:45.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.046+0000 7f38c279a640 1 -- 172.21.15.28:0/1821863728 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f38a4067190 msgr2=0x7f38a4069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:45.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.046+0000 7f38c279a640 1 --2- 172.21.15.28:0/1821863728 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f38a4067190 0x7f38a4069640 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f38ac007e20 tx=0x7f38ac00b040 comp rx=0 tx=0).stop 2024-07-03T15:43:45.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.046+0000 7f38c279a640 1 -- 172.21.15.28:0/1821863728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38bc071380 msgr2=0x7f38bc10f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:45.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.046+0000 7f38c279a640 1 --2- 172.21.15.28:0/1821863728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38bc071380 0x7f38bc10f6a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f38b00048a0 tx=0x7f38b0009da0 comp rx=0 tx=0).stop 2024-07-03T15:43:45.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.047+0000 7f38c279a640 1 -- 172.21.15.28:0/1821863728 shutdown_connections 2024-07-03T15:43:45.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.047+0000 7f38c279a640 1 --2- 172.21.15.28:0/1821863728 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f38a4067190 0x7f38a4069640 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:45.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.047+0000 7f38c279a640 1 --2- 172.21.15.28:0/1821863728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38bc071380 0x7f38bc10f6a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:45.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.047+0000 7f38c279a640 1 --2- 172.21.15.28:0/1821863728 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38bc06fdf0 0x7f38bc10f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:45.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.047+0000 7f38c279a640 1 -- 172.21.15.28:0/1821863728 >> 172.21.15.28:0/1821863728 conn(0x7f38bc0f7b40 msgr2=0x7f38bc0f9190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:43:45.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.047+0000 7f38c279a640 1 -- 172.21.15.28:0/1821863728 shutdown_connections 2024-07-03T15:43:45.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.047+0000 7f38c279a640 1 -- 172.21.15.28:0/1821863728 wait complete. 2024-07-03T15:43:45.073 INFO:tasks.workunit.client.1.smithi148.stdout:7/82: dwrite f2 [0,4194304] 0 2024-07-03T15:43:45.074 INFO:tasks.workunit.client.1.smithi148.stdout:7/83: dread - d5/fa zero size 2024-07-03T15:43:45.074 INFO:tasks.workunit.client.1.smithi148.stdout:7/84: dread d5/d16/f1a [0,4194304] 0 2024-07-03T15:43:45.075 INFO:tasks.workunit.client.1.smithi148.stdout:7/85: write d5/fa [344633,20836] 0 2024-07-03T15:43:45.075 INFO:tasks.workunit.client.1.smithi148.stdout:4/69: dwrite d0/f3 [0,4194304] 0 2024-07-03T15:43:45.075 INFO:tasks.workunit.client.1.smithi148.stdout:4/70: dread d0/f5 [0,4194304] 0 2024-07-03T15:43:45.075 INFO:tasks.workunit.client.1.smithi148.stdout:7/86: link d5/l1b d5/d16/d1c/l1d 0 2024-07-03T15:43:45.075 INFO:tasks.workunit.client.1.smithi148.stdout:6/173: dwrite f1 [0,4194304] 0 2024-07-03T15:43:45.085 INFO:tasks.workunit.client.1.smithi148.stdout:4/71: dwrite d0/d8/fd [0,4194304] 0 2024-07-03T15:43:45.086 INFO:tasks.workunit.client.1.smithi148.stdout:7/87: dwrite d5/f7 [0,4194304] 0 2024-07-03T15:43:45.087 INFO:tasks.workunit.client.1.smithi148.stdout:9/196: rename d0/dd/d23 to d0/d49 0 2024-07-03T15:43:45.089 INFO:tasks.workunit.client.1.smithi148.stdout:7/88: fdatasync d5/fa 0 2024-07-03T15:43:45.089 INFO:tasks.workunit.client.1.smithi148.stdout:8/49: rmdir d4/d6 39 2024-07-03T15:43:45.089 INFO:tasks.workunit.client.1.smithi148.stdout:6/174: mknod d3/d1d/d28/d31/c3b 0 2024-07-03T15:43:45.092 INFO:tasks.workunit.client.1.smithi148.stdout:6/175: dread d3/fa [0,4194304] 0 2024-07-03T15:43:45.092 INFO:tasks.workunit.client.1.smithi148.stdout:6/176: read - d3/f37 zero size 2024-07-03T15:43:45.095 INFO:tasks.workunit.client.1.smithi148.stdout:6/177: dread d3/d1d/d28/f2e [0,4194304] 0 2024-07-03T15:43:45.095 INFO:tasks.workunit.client.1.smithi148.stdout:6/178: readlink d3/l13 0 2024-07-03T15:43:45.095 INFO:tasks.workunit.client.1.smithi148.stdout:6/179: write d3/f17 [3242580,41055] 0 2024-07-03T15:43:45.102 INFO:tasks.workunit.client.1.smithi148.stdout:4/72: dwrite d0/f3 [4194304,4194304] 0 2024-07-03T15:43:45.108 INFO:tasks.workunit.client.1.smithi148.stdout:1/104: sync 2024-07-03T15:43:45.108 INFO:tasks.workunit.client.1.smithi148.stdout:2/153: sync 2024-07-03T15:43:45.108 INFO:tasks.workunit.client.1.smithi148.stdout:0/97: rename f2 to da/d17/f1e 0 2024-07-03T15:43:45.111 INFO:tasks.workunit.client.1.smithi148.stdout:9/197: rmdir d0/d49/d43 39 2024-07-03T15:43:45.116 INFO:tasks.workunit.client.1.smithi148.stdout:6/180: dwrite d3/d1d/f25 [0,4194304] 0 2024-07-03T15:43:45.117 INFO:tasks.workunit.client.1.smithi148.stdout:1/105: write d0/d2/d6/fa [2498317,108332] 0 2024-07-03T15:43:45.120 INFO:tasks.workunit.client.1.smithi148.stdout:7/89: rmdir d5/d8 39 2024-07-03T15:43:45.126 INFO:tasks.workunit.client.1.smithi148.stdout:2/154: dwrite d5/d14/d22/d2b/d13/f1b [0,4194304] 0 2024-07-03T15:43:45.126 INFO:tasks.workunit.client.1.smithi148.stdout:2/155: write d5/f9 [4984036,86998] 0 2024-07-03T15:43:45.131 INFO:tasks.workunit.client.1.smithi148.stdout:8/50: symlink d4/l12 0 2024-07-03T15:43:45.132 INFO:tasks.workunit.client.1.smithi148.stdout:8/51: readlink d4/d6/l10 0 2024-07-03T15:43:45.132 INFO:tasks.workunit.client.1.smithi148.stdout:8/52: chown d4/l12 1179 1 2024-07-03T15:43:45.132 INFO:tasks.workunit.client.1.smithi148.stdout:8/53: write d4/d6/fe [934530,16871] 0 2024-07-03T15:43:45.132 INFO:tasks.workunit.client.1.smithi148.stdout:8/54: write d4/f11 [784546,63713] 0 2024-07-03T15:43:45.132 INFO:tasks.workunit.client.1.smithi148.stdout:8/55: chown d4/l12 63 1 2024-07-03T15:43:45.132 INFO:tasks.workunit.client.1.smithi148.stdout:8/56: stat d4/l12 0 2024-07-03T15:43:45.138 INFO:tasks.workunit.client.1.smithi148.stdout:4/73: symlink d0/d8/d13/l15 0 2024-07-03T15:43:45.138 INFO:tasks.workunit.client.1.smithi148.stdout:4/74: creat d0/f16 x:0 0 0 2024-07-03T15:43:45.138 INFO:tasks.workunit.client.1.smithi148.stdout:4/75: creat d0/f17 x:0 0 0 2024-07-03T15:43:45.144 INFO:tasks.workunit.client.1.smithi148.stdout:5/73: rename d7/da/d12 to d7/da/df/d16 0 2024-07-03T15:43:45.147 INFO:tasks.workunit.client.1.smithi148.stdout:0/98: creat da/dc/d13/d14/f1f x:0 0 0 2024-07-03T15:43:45.149 INFO:tasks.workunit.client.1.smithi148.stdout:9/198: stat d0/dd/d29/c2e 0 2024-07-03T15:43:45.149 INFO:tasks.workunit.client.1.smithi148.stdout:9/199: chown d0/dd/d11 1091730 1 2024-07-03T15:43:45.155 INFO:tasks.workunit.client.1.smithi148.stdout:6/181: unlink d3/d1d/c29 0 2024-07-03T15:43:45.159 INFO:tasks.workunit.client.1.smithi148.stdout:6/182: dread d3/d1d/f24 [0,4194304] 0 2024-07-03T15:43:45.161 INFO:tasks.workunit.client.1.smithi148.stdout:7/90: stat d5/d8/ld 0 2024-07-03T15:43:45.166 INFO:tasks.workunit.client.1.smithi148.stdout:2/156: symlink d5/l36 0 2024-07-03T15:43:45.168 INFO:tasks.workunit.client.1.smithi148.stdout:8/57: mknod d4/d6/c13 0 2024-07-03T15:43:45.172 INFO:tasks.workunit.client.1.smithi148.stdout:2/157: read d5/f9 [1931996,125802] 0 2024-07-03T15:43:45.172 INFO:tasks.workunit.client.1.smithi148.stdout:8/58: fdatasync d4/f11 0 2024-07-03T15:43:45.177 INFO:tasks.workunit.client.1.smithi148.stdout:4/76: mknod d0/d8/c18 0 2024-07-03T15:43:45.180 INFO:tasks.workunit.client.1.smithi148.stdout:1/106: rename d0/de/d1a to d0/d2/d11/d1e 0 2024-07-03T15:43:45.181 INFO:tasks.workunit.client.1.smithi148.stdout:5/74: truncate f4 1096799 0 2024-07-03T15:43:45.183 INFO:tasks.workunit.client.1.smithi148.stdout:0/99: symlink da/dc/l20 0 2024-07-03T15:43:45.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.187+0000 7fc0479c6640 1 -- 172.21.15.28:0/3155900681 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc0400662e0 msgr2=0x7fc0400fef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:45.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.187+0000 7fc0479c6640 1 --2- 172.21.15.28:0/3155900681 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc0400662e0 0x7fc0400fef40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc03c0048d0 tx=0x7fc03c009da0 comp rx=0 tx=0).stop 2024-07-03T15:43:45.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.187+0000 7fc0479c6640 1 -- 172.21.15.28:0/3155900681 shutdown_connections 2024-07-03T15:43:45.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.187+0000 7fc0479c6640 1 --2- 172.21.15.28:0/3155900681 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0400ff480 0x7fc040101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:45.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.187+0000 7fc0479c6640 1 --2- 172.21.15.28:0/3155900681 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc0400662e0 0x7fc0400fef40 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:45.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.187+0000 7fc0479c6640 1 -- 172.21.15.28:0/3155900681 >> 172.21.15.28:0/3155900681 conn(0x7fc0400f6a10 msgr2=0x7fc0400f8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:43:45.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.188+0000 7fc0479c6640 1 -- 172.21.15.28:0/3155900681 shutdown_connections 2024-07-03T15:43:45.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.188+0000 7fc0479c6640 1 -- 172.21.15.28:0/3155900681 wait complete. 2024-07-03T15:43:45.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.189+0000 7fc0479c6640 1 Processor -- start 2024-07-03T15:43:45.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.189+0000 7fc0479c6640 1 -- start start 2024-07-03T15:43:45.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.189+0000 7fc0479c6640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc0400662e0 0x7fc0400fc970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:43:45.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.189+0000 7fc0479c6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0400ff480 0x7fc0400fae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:43:45.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.189+0000 7fc0479c6640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc040110c10 con 0x7fc0400662e0 2024-07-03T15:43:45.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.189+0000 7fc0479c6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc040110d80 con 0x7fc0400ff480 2024-07-03T15:43:45.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.189+0000 7fc045762640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc0400662e0 0x7fc0400fc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:43:45.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.189+0000 7fc045762640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc0400662e0 0x7fc0400fc970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53228/0 (socket says 172.21.15.28:53228) 2024-07-03T15:43:45.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.189+0000 7fc045762640 1 -- 172.21.15.28:0/3850104521 learned_addr learned my addr 172.21.15.28:0/3850104521 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:43:45.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.189+0000 7fc044f61640 1 --2- 172.21.15.28:0/3850104521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0400ff480 0x7fc0400fae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:43:45.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.189+0000 7fc045762640 1 -- 172.21.15.28:0/3850104521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0400ff480 msgr2=0x7fc0400fae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:45.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.189+0000 7fc045762640 1 --2- 172.21.15.28:0/3850104521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0400ff480 0x7fc0400fae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:45.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.189+0000 7fc045762640 1 -- 172.21.15.28:0/3850104521 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc03c005040 con 0x7fc0400662e0 2024-07-03T15:43:45.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.189+0000 7fc044f61640 1 --2- 172.21.15.28:0/3850104521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0400ff480 0x7fc0400fae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-03T15:43:45.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.190+0000 7fc045762640 1 --2- 172.21.15.28:0/3850104521 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc0400662e0 0x7fc0400fc970 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc03c00b020 tx=0x7fc03c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:43:45.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.190+0000 7fc035ffb640 1 -- 172.21.15.28:0/3850104521 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc03c038070 con 0x7fc0400662e0 2024-07-03T15:43:45.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.190+0000 7fc0479c6640 1 -- 172.21.15.28:0/3850104521 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0400fb400 con 0x7fc0400662e0 2024-07-03T15:43:45.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.190+0000 7fc0479c6640 1 -- 172.21.15.28:0/3850104521 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0400fb9a0 con 0x7fc0400662e0 2024-07-03T15:43:45.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.190+0000 7fc035ffb640 1 -- 172.21.15.28:0/3850104521 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc03c0092b0 con 0x7fc0400662e0 2024-07-03T15:43:45.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.190+0000 7fc035ffb640 1 -- 172.21.15.28:0/3850104521 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc03c033660 con 0x7fc0400662e0 2024-07-03T15:43:45.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.191+0000 7fc0479c6640 1 -- 172.21.15.28:0/3850104521 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc024005240 con 0x7fc0400662e0 2024-07-03T15:43:45.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.192+0000 7fc035ffb640 1 -- 172.21.15.28:0/3850104521 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7fc03c030020 con 0x7fc0400662e0 2024-07-03T15:43:45.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.192+0000 7fc035ffb640 1 --2- 172.21.15.28:0/3850104521 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fc02c067190 0x7fc02c069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:43:45.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.193+0000 7fc035ffb640 1 -- 172.21.15.28:0/3850104521 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fc03c0a4650 con 0x7fc0400662e0 2024-07-03T15:43:45.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.193+0000 7fc044f61640 1 --2- 172.21.15.28:0/3850104521 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fc02c067190 0x7fc02c069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:43:45.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.193+0000 7fc044f61640 1 --2- 172.21.15.28:0/3850104521 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fc02c067190 0x7fc02c069640 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc0400fc110 tx=0x7fc03000b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:43:45.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.197+0000 7fc035ffb640 1 -- 172.21.15.28:0/3850104521 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc03c071a20 con 0x7fc0400662e0 2024-07-03T15:43:45.204 INFO:tasks.workunit.client.1.smithi148.stdout:6/183: creat d3/d1d/d28/d31/f3c x:0 0 0 2024-07-03T15:43:45.204 INFO:tasks.workunit.client.1.smithi148.stdout:6/184: write d3/f2c [5517,44992] 0 2024-07-03T15:43:45.204 INFO:tasks.workunit.client.1.smithi148.stdout:7/91: symlink d5/d16/d1c/l1e 0 2024-07-03T15:43:45.204 INFO:tasks.workunit.client.1.smithi148.stdout:7/92: creat d5/f1f x:0 0 0 2024-07-03T15:43:45.204 INFO:tasks.workunit.client.1.smithi148.stdout:7/93: dread - d5/ff zero size 2024-07-03T15:43:45.204 INFO:tasks.workunit.client.1.smithi148.stdout:8/59: mkdir d4/da/d14 0 2024-07-03T15:43:45.204 INFO:tasks.workunit.client.1.smithi148.stdout:8/60: chown d4/d6/l9 318861 1 2024-07-03T15:43:45.204 INFO:tasks.workunit.client.1.smithi148.stdout:2/158: symlink d5/d7/l37 0 2024-07-03T15:43:45.204 INFO:tasks.workunit.client.1.smithi148.stdout:2/159: truncate d5/d14/d22/d2b/d13/d2d/f2e 734270 0 2024-07-03T15:43:45.204 INFO:tasks.workunit.client.1.smithi148.stdout:2/160: creat d5/d7/f38 x:0 0 0 2024-07-03T15:43:45.205 INFO:tasks.workunit.client.1.smithi148.stdout:4/77: mkdir d0/d8/d19 0 2024-07-03T15:43:45.205 INFO:tasks.workunit.client.1.smithi148.stdout:4/78: fdatasync d0/d8/f10 0 2024-07-03T15:43:45.205 INFO:tasks.workunit.client.1.smithi148.stdout:4/79: chown d0/f17 124 1 2024-07-03T15:43:45.207 INFO:tasks.workunit.client.1.smithi148.stdout:9/200: rename d0/d18/c1b to d0/d18/d24/c4a 0 2024-07-03T15:43:45.210 INFO:tasks.workunit.client.1.smithi148.stdout:1/107: creat d0/f1f x:0 0 0 2024-07-03T15:43:45.210 INFO:tasks.workunit.client.1.smithi148.stdout:8/61: dread d4/d6/f7 [0,4194304] 0 2024-07-03T15:43:45.210 INFO:tasks.workunit.client.1.smithi148.stdout:8/62: chown d4/d6/fe 454131 1 2024-07-03T15:43:45.210 INFO:tasks.workunit.client.1.smithi148.stdout:8/63: write d4/d6/f7 [4469748,97690] 0 2024-07-03T15:43:45.210 INFO:tasks.workunit.client.1.smithi148.stdout:8/64: stat d4/da/fb 0 2024-07-03T15:43:45.213 INFO:tasks.workunit.client.1.smithi148.stdout:0/100: write da/dc/d13/f1b [4348810,80783] 0 2024-07-03T15:43:45.214 INFO:tasks.workunit.client.1.smithi148.stdout:6/185: symlink d3/d1d/d28/l3d 0 2024-07-03T15:43:45.215 INFO:tasks.workunit.client.1.smithi148.stdout:6/186: chown d3/d1d/d28/d31/c3b 142 1 2024-07-03T15:43:45.218 INFO:tasks.workunit.client.1.smithi148.stdout:7/94: mknod d5/c20 0 2024-07-03T15:43:45.219 INFO:tasks.workunit.client.1.smithi148.stdout:2/161: dwrite d5/d14/d22/d2b/f1a [0,4194304] 0 2024-07-03T15:43:45.219 INFO:tasks.workunit.client.1.smithi148.stdout:2/162: truncate d5/d14/d22/d2b/d13/d2d/f2e 1612631 0 2024-07-03T15:43:45.223 INFO:tasks.workunit.client.1.smithi148.stdout:9/201: unlink d0/d2f/f3d 0 2024-07-03T15:43:45.225 INFO:tasks.workunit.client.1.smithi148.stdout:1/108: creat d0/d2/d6/f20 x:0 0 0 2024-07-03T15:43:45.228 INFO:tasks.workunit.client.1.smithi148.stdout:7/95: write d5/f7 [3843476,104584] 0 2024-07-03T15:43:45.228 INFO:tasks.workunit.client.1.smithi148.stdout:8/65: mknod d4/d6/c15 0 2024-07-03T15:43:45.232 INFO:tasks.workunit.client.1.smithi148.stdout:6/187: mkdir d3/d1d/d28/d3e 0 2024-07-03T15:43:45.233 INFO:tasks.workunit.client.1.smithi148.stdout:2/163: write d5/d14/d22/d2b/f16 [2564433,110543] 0 2024-07-03T15:43:45.233 INFO:tasks.workunit.client.1.smithi148.stdout:2/164: fdatasync d5/d14/d22/d2b/d13/d2d/f2e 0 2024-07-03T15:43:45.233 INFO:tasks.workunit.client.1.smithi148.stdout:2/165: chown d5/d14/d22/d2b/l27 1207 1 2024-07-03T15:43:45.236 INFO:tasks.workunit.client.1.smithi148.stdout:9/202: rename d0/dd/d25/d39/c3a to d0/d18/d24/d47/c4b 0 2024-07-03T15:43:45.236 INFO:tasks.workunit.client.1.smithi148.stdout:9/203: chown d0/d18/l1d 446194711 1 2024-07-03T15:43:45.237 INFO:tasks.workunit.client.1.smithi148.stdout:8/66: dread d4/d6/f7 [0,4194304] 0 2024-07-03T15:43:45.237 INFO:tasks.workunit.client.1.smithi148.stdout:8/67: readlink d4/d6/l9 0 2024-07-03T15:43:45.237 INFO:tasks.workunit.client.1.smithi148.stdout:8/68: chown d4/da/fb 278959 1 2024-07-03T15:43:45.239 INFO:tasks.workunit.client.1.smithi148.stdout:1/109: mknod d0/d2/d11/c21 0 2024-07-03T15:43:45.246 INFO:tasks.workunit.client.1.smithi148.stdout:7/96: dwrite d5/d16/f1a [0,4194304] 0 2024-07-03T15:43:45.247 INFO:tasks.workunit.client.1.smithi148.stdout:6/188: rename c0 to d3/c3f 0 2024-07-03T15:43:45.247 INFO:tasks.workunit.client.1.smithi148.stdout:6/189: chown d3/c15 243 1 2024-07-03T15:43:45.249 INFO:tasks.workunit.client.1.smithi148.stdout:9/204: link d0/l27 d0/d49/d43/l4c 0 2024-07-03T15:43:45.249 INFO:tasks.workunit.client.1.smithi148.stdout:9/205: read d0/d18/f26 [134142,118182] 0 2024-07-03T15:43:45.249 INFO:tasks.workunit.client.1.smithi148.stdout:9/206: dread - d0/d49/d43/f22 zero size 2024-07-03T15:43:45.251 INFO:tasks.workunit.client.1.smithi148.stdout:8/69: creat d4/da/d14/f16 x:0 0 0 2024-07-03T15:43:45.252 INFO:tasks.workunit.client.1.smithi148.stdout:2/166: dwrite d5/f2f [0,4194304] 0 2024-07-03T15:43:45.252 INFO:tasks.workunit.client.1.smithi148.stdout:1/110: symlink d0/l22 0 2024-07-03T15:43:45.253 INFO:tasks.workunit.client.1.smithi148.stdout:9/207: dread d0/dd/f1f [0,4194304] 0 2024-07-03T15:43:45.259 INFO:tasks.workunit.client.1.smithi148.stdout:7/97: mkdir d5/d8/d21 0 2024-07-03T15:43:45.262 INFO:tasks.workunit.client.1.smithi148.stdout:6/190: rename d3/c4 to d3/d1d/d28/d31/c40 0 2024-07-03T15:43:45.266 INFO:tasks.workunit.client.1.smithi148.stdout:8/70: mkdir d4/da/d14/d17 0 2024-07-03T15:43:45.269 INFO:tasks.workunit.client.1.smithi148.stdout:6/191: write d3/d1d/f27 [3156519,14897] 0 2024-07-03T15:43:45.269 INFO:tasks.workunit.client.1.smithi148.stdout:6/192: read d3/d1d/d28/f2e [36475,115626] 0 2024-07-03T15:43:45.273 INFO:tasks.workunit.client.1.smithi148.stdout:2/167: creat d5/d14/d22/d2b/d13/d2d/f39 x:0 0 0 2024-07-03T15:43:45.274 INFO:tasks.workunit.client.1.smithi148.stdout:1/111: mknod d0/d2/d6/d14/c23 0 2024-07-03T15:43:45.275 INFO:tasks.workunit.client.1.smithi148.stdout:2/168: fsync d5/f9 0 2024-07-03T15:43:45.276 INFO:tasks.workunit.client.1.smithi148.stdout:9/208: mkdir d0/d18/d24/d47/d4d 0 2024-07-03T15:43:45.277 INFO:tasks.workunit.client.1.smithi148.stdout:9/209: truncate d0/dd/f14 1491285 0 2024-07-03T15:43:45.281 INFO:tasks.workunit.client.1.smithi148.stdout:7/98: rename d5/f11 to d5/d8/f22 0 2024-07-03T15:43:45.281 INFO:tasks.workunit.client.1.smithi148.stdout:7/99: write d5/d8/f12 [598008,10054] 0 2024-07-03T15:43:45.283 INFO:tasks.workunit.client.1.smithi148.stdout:8/71: mkdir d4/d6/d18 0 2024-07-03T15:43:45.286 INFO:tasks.workunit.client.1.smithi148.stdout:2/169: creat d5/d14/d22/d2b/f3a x:0 0 0 2024-07-03T15:43:45.286 INFO:tasks.workunit.client.1.smithi148.stdout:2/170: creat d5/d14/d22/d2b/d13/d2d/f3b x:0 0 0 2024-07-03T15:43:45.289 INFO:tasks.workunit.client.1.smithi148.stdout:6/193: rename d3/l19 to d3/d1d/d28/d3e/l41 0 2024-07-03T15:43:45.289 INFO:tasks.workunit.client.1.smithi148.stdout:8/72: rename d4/da to d4/da/d14/d19 22 2024-07-03T15:43:45.289 INFO:tasks.workunit.client.1.smithi148.stdout:6/194: chown d3/d1d/d28/l2d 634 1 2024-07-03T15:43:45.289 INFO:tasks.workunit.client.1.smithi148.stdout:6/195: fdatasync d3/d1d/d28/d31/f3c 0 2024-07-03T15:43:45.291 INFO:tasks.workunit.client.1.smithi148.stdout:7/100: mkdir d5/d16/d1c/d23 0 2024-07-03T15:43:45.294 INFO:tasks.workunit.client.1.smithi148.stdout:2/171: symlink d5/d14/d22/d2b/d13/l3c 0 2024-07-03T15:43:45.296 INFO:tasks.workunit.client.1.smithi148.stdout:1/112: dwrite d0/d2/fd [0,4194304] 0 2024-07-03T15:43:45.299 INFO:tasks.workunit.client.1.smithi148.stdout:9/210: dwrite d0/dd/f14 [0,4194304] 0 2024-07-03T15:43:45.301 INFO:tasks.workunit.client.1.smithi148.stdout:6/196: symlink d3/l42 0 2024-07-03T15:43:45.302 INFO:tasks.workunit.client.1.smithi148.stdout:7/101: truncate d5/f9 421484 0 2024-07-03T15:43:45.317 INFO:tasks.workunit.client.1.smithi148.stdout:7/102: symlink d5/d16/l24 0 2024-07-03T15:43:45.317 INFO:tasks.workunit.client.1.smithi148.stdout:7/103: chown f2 293616970 1 2024-07-03T15:43:45.318 INFO:tasks.workunit.client.1.smithi148.stdout:0/101: fsync da/d17/f1e 0 2024-07-03T15:43:45.318 INFO:tasks.workunit.client.1.smithi148.stdout:2/172: dwrite d5/f1f [0,4194304] 0 2024-07-03T15:43:45.319 INFO:tasks.workunit.client.1.smithi148.stdout:2/173: chown d5/d26/l33 1361784 1 2024-07-03T15:43:45.319 INFO:tasks.workunit.client.1.smithi148.stdout:2/174: dread - d5/d14/d22/d2b/d13/d2d/f3b zero size 2024-07-03T15:43:45.319 INFO:tasks.workunit.client.1.smithi148.stdout:2/175: chown d5/d14/d22/d2b/l2a 307 1 2024-07-03T15:43:45.319 INFO:tasks.workunit.client.1.smithi148.stdout:2/176: dread - d5/d14/d22/d2b/d13/d2d/f39 zero size 2024-07-03T15:43:45.319 INFO:tasks.workunit.client.1.smithi148.stdout:2/177: creat d5/d7/f3d x:0 0 0 2024-07-03T15:43:45.319 INFO:tasks.workunit.client.1.smithi148.stdout:7/104: mknod d5/c25 0 2024-07-03T15:43:45.320 INFO:tasks.workunit.client.1.smithi148.stdout:6/197: dwrite d3/f7 [0,4194304] 0 2024-07-03T15:43:45.321 INFO:tasks.workunit.client.1.smithi148.stdout:1/113: dwrite d0/d2/d6/f20 [0,4194304] 0 2024-07-03T15:43:45.321 INFO:tasks.workunit.client.1.smithi148.stdout:1/114: creat d0/f24 x:0 0 0 2024-07-03T15:43:45.328 INFO:tasks.workunit.client.1.smithi148.stdout:7/105: rename d5/d16/f1a to d5/d8/f26 0 2024-07-03T15:43:45.329 INFO:tasks.workunit.client.1.smithi148.stdout:7/106: truncate d5/f1f 30246 0 2024-07-03T15:43:45.330 INFO:tasks.workunit.client.1.smithi148.stdout:6/198: dread d3/d1d/f27 [0,4194304] 0 2024-07-03T15:43:45.332 INFO:tasks.workunit.client.1.smithi148.stdout:9/211: dwrite d0/d18/d24/f30 [0,4194304] 0 2024-07-03T15:43:45.332 INFO:tasks.workunit.client.1.smithi148.stdout:9/212: stat d0/dd/d11/f12 0 2024-07-03T15:43:45.334 INFO:tasks.workunit.client.1.smithi148.stdout:8/73: truncate d4/d6/f8 478349 0 2024-07-03T15:43:45.338 INFO:tasks.workunit.client.1.smithi148.stdout:7/107: read f2 [4056351,13641] 0 2024-07-03T15:43:45.340 INFO:tasks.workunit.client.1.smithi148.stdout:6/199: dread f1 [4194304,4194304] 0 2024-07-03T15:43:45.342 INFO:tasks.workunit.client.1.smithi148.stdout:0/102: creat da/dc/d13/d1d/f21 x:0 0 0 2024-07-03T15:43:45.346 INFO:tasks.workunit.client.1.smithi148.stdout:9/213: rename d0/dd/fc to d0/d49/f4e 0 2024-07-03T15:43:45.348 INFO:tasks.workunit.client.1.smithi148.stdout:7/108: mkdir d5/d16/d27 0 2024-07-03T15:43:45.350 INFO:tasks.workunit.client.1.smithi148.stdout:5/75: getdents d7/da 0 2024-07-03T15:43:45.350 INFO:tasks.workunit.client.1.smithi148.stdout:5/76: chown d7/da/df/d16 38165 1 2024-07-03T15:43:45.351 INFO:tasks.workunit.client.1.smithi148.stdout:2/178: dwrite d5/d14/d22/d2b/d13/f29 [0,4194304] 0 2024-07-03T15:43:45.351 INFO:tasks.workunit.client.1.smithi148.stdout:2/179: stat d5/d7 0 2024-07-03T15:43:45.352 INFO:tasks.workunit.client.1.smithi148.stdout:9/214: dread d0/dd/fe [0,4194304] 0 2024-07-03T15:43:45.353 INFO:tasks.workunit.client.1.smithi148.stdout:0/103: symlink da/dc/l22 0 2024-07-03T15:43:45.355 INFO:tasks.workunit.client.1.smithi148.stdout:7/109: creat d5/d16/d27/f28 x:0 0 0 2024-07-03T15:43:45.355 INFO:tasks.workunit.client.1.smithi148.stdout:7/110: dread - d5/d8/f22 zero size 2024-07-03T15:43:45.360 INFO:tasks.workunit.client.1.smithi148.stdout:5/77: mknod d7/d13/c17 0 2024-07-03T15:43:45.360 INFO:tasks.workunit.client.1.smithi148.stdout:2/180: truncate d5/d14/d22/d2b/d13/f15 2998418 0 2024-07-03T15:43:45.364 INFO:tasks.workunit.client.1.smithi148.stdout:9/215: creat d0/dd/d25/d42/f4f x:0 0 0 2024-07-03T15:43:45.364 INFO:tasks.workunit.client.1.smithi148.stdout:9/216: dread - d0/d49/d43/d19/f48 zero size 2024-07-03T15:43:45.364 INFO:tasks.workunit.client.1.smithi148.stdout:9/217: write d0/dd/d11/f12 [1345356,118696] 0 2024-07-03T15:43:45.364 INFO:tasks.workunit.client.1.smithi148.stdout:6/200: dwrite d3/f7 [0,4194304] 0 2024-07-03T15:43:45.364 INFO:tasks.workunit.client.1.smithi148.stdout:8/74: dwrite d4/d6/fe [0,4194304] 0 2024-07-03T15:43:45.364 INFO:tasks.workunit.client.1.smithi148.stdout:8/75: fdatasync d4/da/fb 0 2024-07-03T15:43:45.367 INFO:tasks.workunit.client.1.smithi148.stdout:0/104: symlink da/dc/d13/d1d/l23 0 2024-07-03T15:43:45.369 INFO:tasks.workunit.client.1.smithi148.stdout:7/111: creat d5/d8/d21/f29 x:0 0 0 2024-07-03T15:43:45.369 INFO:tasks.workunit.client.1.smithi148.stdout:7/112: truncate d5/d16/d27/f28 703877 0 2024-07-03T15:43:45.369 INFO:tasks.workunit.client.1.smithi148.stdout:7/113: fdatasync d5/f6 0 2024-07-03T15:43:45.369 INFO:tasks.workunit.client.1.smithi148.stdout:7/114: chown d5/d8/l18 741375671 1 2024-07-03T15:43:45.370 INFO:tasks.workunit.client.1.smithi148.stdout:2/181: write d5/f2f [3796810,38943] 0 2024-07-03T15:43:45.371 INFO:tasks.workunit.client.1.smithi148.stdout:5/78: symlink d7/d15/l18 0 2024-07-03T15:43:45.373 INFO:tasks.workunit.client.1.smithi148.stdout:9/218: creat d0/dd/d25/d37/f50 x:0 0 0 2024-07-03T15:43:45.376 INFO:tasks.workunit.client.1.smithi148.stdout:2/182: read d5/d14/d22/d2b/f1a [3325929,57199] 0 2024-07-03T15:43:45.385 INFO:tasks.workunit.client.1.smithi148.stdout:4/80: fsync d0/f17 0 2024-07-03T15:43:45.385 INFO:tasks.workunit.client.1.smithi148.stdout:4/81: write d0/f6 [4132712,55394] 0 2024-07-03T15:43:45.387 INFO:tasks.workunit.client.1.smithi148.stdout:5/79: dwrite d7/da/df/f10 [0,4194304] 0 2024-07-03T15:43:45.387 INFO:tasks.workunit.client.1.smithi148.stdout:5/80: chown d7/d13/c17 2252728 1 2024-07-03T15:43:45.388 INFO:tasks.workunit.client.1.smithi148.stdout:7/115: mknod d5/d16/d1c/c2a 0 2024-07-03T15:43:45.388 INFO:tasks.workunit.client.1.smithi148.stdout:7/116: write d5/f6 [1908975,108022] 0 2024-07-03T15:43:45.395 INFO:tasks.workunit.client.1.smithi148.stdout:9/219: rename d0/dd/d11/c20 to d0/d18/d24/d47/c51 0 2024-07-03T15:43:45.402 INFO:tasks.workunit.client.1.smithi148.stdout:2/183: creat d5/d26/f3e x:0 0 0 2024-07-03T15:43:45.411 INFO:tasks.workunit.client.1.smithi148.stdout:4/82: unlink d0/f7 0 2024-07-03T15:43:45.417 INFO:tasks.workunit.client.1.smithi148.stdout:7/117: creat d5/d16/d1c/d23/f2b x:0 0 0 2024-07-03T15:43:45.417 INFO:tasks.workunit.client.1.smithi148.stdout:7/118: readlink d5/l1b 0 2024-07-03T15:43:45.420 INFO:tasks.workunit.client.1.smithi148.stdout:3/87: sync 2024-07-03T15:43:45.421 INFO:tasks.workunit.client.1.smithi148.stdout:3/88: dread - dd/df/f11 zero size 2024-07-03T15:43:45.421 INFO:tasks.workunit.client.1.smithi148.stdout:5/81: rename d7/da/fd to d7/f19 0 2024-07-03T15:43:45.424 INFO:tasks.workunit.client.1.smithi148.stdout:5/82: write d7/f19 [233259,109932] 0 2024-07-03T15:43:45.425 INFO:tasks.workunit.client.1.smithi148.stdout:4/83: dwrite d0/d8/fd [0,4194304] 0 2024-07-03T15:43:45.425 INFO:tasks.workunit.client.1.smithi148.stdout:4/84: write d0/f6 [2965698,18126] 0 2024-07-03T15:43:45.425 INFO:tasks.workunit.client.1.smithi148.stdout:2/184: symlink d5/d14/l3f 0 2024-07-03T15:43:45.426 INFO:tasks.workunit.client.1.smithi148.stdout:7/119: mknod d5/d16/c2c 0 2024-07-03T15:43:45.426 INFO:tasks.workunit.client.1.smithi148.stdout:7/120: stat d5/d16 0 2024-07-03T15:43:45.428 INFO:tasks.workunit.client.1.smithi148.stdout:3/89: truncate f0 2037285 0 2024-07-03T15:43:45.428 INFO:tasks.workunit.client.1.smithi148.stdout:3/90: stat dd/df 0 2024-07-03T15:43:45.432 INFO:tasks.workunit.client.1.smithi148.stdout:5/83: mkdir d7/da/df/d16/d1a 0 2024-07-03T15:43:45.438 INFO:tasks.workunit.client.1.smithi148.stdout:4/85: creat d0/d8/d13/f1a x:0 0 0 2024-07-03T15:43:45.440 INFO:tasks.workunit.client.1.smithi148.stdout:4/86: dread d0/f5 [0,4194304] 0 2024-07-03T15:43:45.440 INFO:tasks.workunit.client.1.smithi148.stdout:4/87: dread - d0/d8/ff zero size 2024-07-03T15:43:45.440 INFO:tasks.workunit.client.1.smithi148.stdout:4/88: fdatasync d0/f17 0 2024-07-03T15:43:45.440 INFO:tasks.workunit.client.1.smithi148.stdout:3/91: link f9 dd/df/f13 0 2024-07-03T15:43:45.446 INFO:tasks.workunit.client.1.smithi148.stdout:5/84: write d7/da/df/f10 [2305645,31296] 0 2024-07-03T15:43:45.453 INFO:tasks.workunit.client.1.smithi148.stdout:7/121: dwrite f2 [0,4194304] 0 2024-07-03T15:43:45.454 INFO:tasks.workunit.client.1.smithi148.stdout:2/185: dwrite d5/d14/d22/d2b/d13/d2d/f2e [0,4194304] 0 2024-07-03T15:43:45.458 INFO:tasks.workunit.client.1.smithi148.stdout:4/89: mknod d0/d8/d19/c1b 0 2024-07-03T15:43:45.461 INFO:tasks.workunit.client.1.smithi148.stdout:3/92: dwrite dd/df/f13 [4194304,4194304] 0 2024-07-03T15:43:45.461 INFO:tasks.workunit.client.1.smithi148.stdout:3/93: write dd/df/f11 [974991,94282] 0 2024-07-03T15:43:45.461 INFO:tasks.workunit.client.1.smithi148.stdout:3/94: creat dd/f14 x:0 0 0 2024-07-03T15:43:45.461 INFO:tasks.workunit.client.1.smithi148.stdout:5/85: creat d7/da/df/d16/f1b x:0 0 0 2024-07-03T15:43:45.464 INFO:tasks.workunit.client.1.smithi148.stdout:7/122: creat d5/d16/d1c/f2d x:0 0 0 2024-07-03T15:43:45.467 INFO:tasks.workunit.client.1.smithi148.stdout:2/186: dwrite d5/d14/d22/d2b/d13/d2d/f3b [0,4194304] 0 2024-07-03T15:43:45.468 INFO:tasks.workunit.client.1.smithi148.stdout:2/187: truncate d5/d26/f3e 522376 0 2024-07-03T15:43:45.469 INFO:tasks.workunit.client.1.smithi148.stdout:4/90: symlink d0/d8/d13/l1c 0 2024-07-03T15:43:45.469 INFO:tasks.workunit.client.1.smithi148.stdout:4/91: write d0/f5 [1851888,45491] 0 2024-07-03T15:43:45.475 INFO:tasks.workunit.client.1.smithi148.stdout:3/95: rename f9 to dd/f15 0 2024-07-03T15:43:45.475 INFO:tasks.workunit.client.1.smithi148.stdout:3/96: read - dd/f12 zero size 2024-07-03T15:43:45.475 INFO:tasks.workunit.client.1.smithi148.stdout:3/97: truncate f4 4905825 0 2024-07-03T15:43:45.477 INFO:tasks.workunit.client.1.smithi148.stdout:5/86: mknod d7/c1c 0 2024-07-03T15:43:45.484 INFO:tasks.workunit.client.1.smithi148.stdout:4/92: dwrite d0/f16 [0,4194304] 0 2024-07-03T15:43:45.485 INFO:tasks.workunit.client.1.smithi148.stdout:9/220: getdents d0/d49/d43 0 2024-07-03T15:43:45.486 INFO:tasks.workunit.client.1.smithi148.stdout:2/188: dwrite d5/d14/d22/d2b/f3a [0,4194304] 0 2024-07-03T15:43:45.487 INFO:tasks.workunit.client.1.smithi148.stdout:2/189: readlink d5/d14/d22/d2b/d13/d2d/l34 0 2024-07-03T15:43:45.487 INFO:tasks.workunit.client.1.smithi148.stdout:2/190: dread - d5/d14/d22/d2b/f1e zero size 2024-07-03T15:43:45.487 INFO:tasks.workunit.client.1.smithi148.stdout:2/191: stat d5/d14/d22/d2b/f1e 0 2024-07-03T15:43:45.487 INFO:tasks.workunit.client.1.smithi148.stdout:9/221: dread d0/d49/d43/d19/f1e [0,4194304] 0 2024-07-03T15:43:45.487 INFO:tasks.workunit.client.1.smithi148.stdout:9/222: readlink d0/d49/l2d 0 2024-07-03T15:43:45.487 INFO:tasks.workunit.client.1.smithi148.stdout:3/98: rename dd/f14 to dd/df/f16 0 2024-07-03T15:43:45.489 INFO:tasks.workunit.client.1.smithi148.stdout:5/87: mkdir d7/da/df/d1d 0 2024-07-03T15:43:45.492 INFO:tasks.workunit.client.1.smithi148.stdout:5/88: dread f4 [0,4194304] 0 2024-07-03T15:43:45.492 INFO:tasks.workunit.client.1.smithi148.stdout:5/89: fdatasync d7/f19 0 2024-07-03T15:43:45.493 INFO:tasks.workunit.client.1.smithi148.stdout:6/201: rmdir d3/d1d/d28/d31 39 2024-07-03T15:43:45.494 INFO:tasks.workunit.client.1.smithi148.stdout:6/202: readlink d3/l39 0 2024-07-03T15:43:45.494 INFO:tasks.workunit.client.1.smithi148.stdout:4/93: mknod d0/d8/d13/c1d 0 2024-07-03T15:43:45.495 INFO:tasks.workunit.client.1.smithi148.stdout:3/99: dread dd/f15 [4194304,4194304] 0 2024-07-03T15:43:45.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.495+0000 7fc0479c6640 1 -- 172.21.15.28:0/3850104521 --> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc024000bf0 con 0x7fc02c067190 2024-07-03T15:43:45.500 INFO:tasks.workunit.client.1.smithi148.stdout:2/192: creat d5/f40 x:0 0 0 2024-07-03T15:43:45.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.496+0000 7fc035ffb640 1 -- 172.21.15.28:0/3850104521 <== mgr.14210 v2:172.21.15.28:6800/421057178 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7fc024000bf0 con 0x7fc02c067190 2024-07-03T15:43:45.500 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-07-03T15:43:45.500 INFO:teuthology.orchestra.run.smithi028.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2", 2024-07-03T15:43:45.500 INFO:teuthology.orchestra.run.smithi028.stdout: "in_progress": true, 2024-07-03T15:43:45.501 INFO:teuthology.orchestra.run.smithi028.stdout: "which": "Upgrading all daemon types on all hosts", 2024-07-03T15:43:45.501 INFO:teuthology.orchestra.run.smithi028.stdout: "services_complete": [], 2024-07-03T15:43:45.501 INFO:teuthology.orchestra.run.smithi028.stdout: "progress": "0/21 daemons upgraded", 2024-07-03T15:43:45.501 INFO:teuthology.orchestra.run.smithi028.stdout: "message": "Currently upgrading mgr daemons", 2024-07-03T15:43:45.501 INFO:teuthology.orchestra.run.smithi028.stdout: "is_paused": false 2024-07-03T15:43:45.501 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-07-03T15:43:45.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.499+0000 7fc0479c6640 1 -- 172.21.15.28:0/3850104521 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fc02c067190 msgr2=0x7fc02c069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:45.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.499+0000 7fc0479c6640 1 --2- 172.21.15.28:0/3850104521 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fc02c067190 0x7fc02c069640 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc0400fc110 tx=0x7fc03000b040 comp rx=0 tx=0).stop 2024-07-03T15:43:45.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.499+0000 7fc0479c6640 1 -- 172.21.15.28:0/3850104521 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc0400662e0 msgr2=0x7fc0400fc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:45.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.499+0000 7fc0479c6640 1 --2- 172.21.15.28:0/3850104521 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc0400662e0 0x7fc0400fc970 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc03c00b020 tx=0x7fc03c009da0 comp rx=0 tx=0).stop 2024-07-03T15:43:45.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.499+0000 7fc0479c6640 1 -- 172.21.15.28:0/3850104521 shutdown_connections 2024-07-03T15:43:45.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.499+0000 7fc0479c6640 1 --2- 172.21.15.28:0/3850104521 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7fc02c067190 0x7fc02c069640 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:45.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.499+0000 7fc0479c6640 1 --2- 172.21.15.28:0/3850104521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0400ff480 0x7fc0400fae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:45.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.499+0000 7fc0479c6640 1 --2- 172.21.15.28:0/3850104521 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc0400662e0 0x7fc0400fc970 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:45.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.499+0000 7fc0479c6640 1 -- 172.21.15.28:0/3850104521 >> 172.21.15.28:0/3850104521 conn(0x7fc0400f6a10 msgr2=0x7fc0400f6de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:43:45.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.499+0000 7fc0479c6640 1 -- 172.21.15.28:0/3850104521 shutdown_connections 2024-07-03T15:43:45.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.499+0000 7fc0479c6640 1 -- 172.21.15.28:0/3850104521 wait complete. 2024-07-03T15:43:45.505 INFO:tasks.workunit.client.1.smithi148.stdout:5/90: mkdir d7/da/df/d1e 0 2024-07-03T15:43:45.506 INFO:tasks.workunit.client.1.smithi148.stdout:9/223: dread d0/dd/f1a [0,4194304] 0 2024-07-03T15:43:45.507 INFO:tasks.workunit.client.1.smithi148.stdout:1/115: rmdir d0/d2/d6 39 2024-07-03T15:43:45.508 INFO:tasks.workunit.client.1.smithi148.stdout:6/203: creat d3/d1d/d28/d31/f43 x:0 0 0 2024-07-03T15:43:45.510 INFO:tasks.workunit.client.1.smithi148.stdout:5/91: dread d7/f19 [0,4194304] 0 2024-07-03T15:43:45.513 INFO:tasks.workunit.client.1.smithi148.stdout:6/204: dread d3/f17 [0,4194304] 0 2024-07-03T15:43:45.514 INFO:tasks.workunit.client.1.smithi148.stdout:9/224: dread d0/dd/f2b [0,4194304] 0 2024-07-03T15:43:45.518 INFO:tasks.workunit.client.1.smithi148.stdout:4/94: truncate d0/f3 1224587 0 2024-07-03T15:43:45.523 INFO:tasks.workunit.client.1.smithi148.stdout:3/100: mknod dd/df/c17 0 2024-07-03T15:43:45.527 INFO:tasks.workunit.client.1.smithi148.stdout:2/193: mkdir d5/d14/d22/d2b/d13/d41 0 2024-07-03T15:43:45.527 INFO:tasks.workunit.client.1.smithi148.stdout:2/194: read - d5/d7/f35 zero size 2024-07-03T15:43:45.527 INFO:tasks.workunit.client.1.smithi148.stdout:2/195: creat d5/d14/d22/d2b/f42 x:0 0 0 2024-07-03T15:43:45.527 INFO:tasks.workunit.client.1.smithi148.stdout:2/196: fsync d5/d7/f38 0 2024-07-03T15:43:45.529 INFO:tasks.workunit.client.1.smithi148.stdout:6/205: rename d3/d1d/d28/l2d to d3/d1d/d28/d31/l44 0 2024-07-03T15:43:45.529 INFO:tasks.workunit.client.1.smithi148.stdout:6/206: chown d3/d1d/f38 4 1 2024-07-03T15:43:45.530 INFO:tasks.workunit.client.1.smithi148.stdout:9/225: symlink d0/d49/d43/l52 0 2024-07-03T15:43:45.533 INFO:tasks.workunit.client.1.smithi148.stdout:2/197: creat d5/d14/d22/d2b/d13/d1d/f43 x:0 0 0 2024-07-03T15:43:45.534 INFO:tasks.workunit.client.1.smithi148.stdout:6/207: stat d3/d1d/d28/d31/c40 0 2024-07-03T15:43:45.536 INFO:tasks.workunit.client.1.smithi148.stdout:9/226: link d0/dd/f1f d0/f53 0 2024-07-03T15:43:45.537 INFO:tasks.workunit.client.1.smithi148.stdout:2/198: mknod d5/d14/d22/d2b/d13/d41/c44 0 2024-07-03T15:43:45.537 INFO:tasks.workunit.client.1.smithi148.stdout:2/199: write d5/d14/d22/d2b/f16 [1994174,57437] 0 2024-07-03T15:43:45.542 INFO:tasks.workunit.client.1.smithi148.stdout:9/227: symlink d0/dd/d25/d39/l54 0 2024-07-03T15:43:45.542 INFO:tasks.workunit.client.1.smithi148.stdout:9/228: dread - d0/dd/f16 zero size 2024-07-03T15:43:45.542 INFO:tasks.workunit.client.1.smithi148.stdout:9/229: readlink d0/l10 0 2024-07-03T15:43:45.542 INFO:tasks.workunit.client.1.smithi148.stdout:9/230: write d0/d2f/f41 [618934,86283] 0 2024-07-03T15:43:45.542 INFO:tasks.workunit.client.1.smithi148.stdout:9/231: truncate d0/dd/f1f 1365782 0 2024-07-03T15:43:45.546 INFO:tasks.workunit.client.1.smithi148.stdout:9/232: creat d0/dd/d29/f55 x:0 0 0 2024-07-03T15:43:45.546 INFO:tasks.workunit.client.1.smithi148.stdout:1/116: dwrite d0/f1f [0,4194304] 0 2024-07-03T15:43:45.546 INFO:tasks.workunit.client.1.smithi148.stdout:9/233: write d0/dd/d25/d37/f50 [184079,1434] 0 2024-07-03T15:43:45.546 INFO:tasks.workunit.client.1.smithi148.stdout:9/234: chown d0/f31 217 1 2024-07-03T15:43:45.546 INFO:tasks.workunit.client.1.smithi148.stdout:1/117: creat d0/f25 x:0 0 0 2024-07-03T15:43:45.546 INFO:tasks.workunit.client.1.smithi148.stdout:4/95: dwrite d0/d8/f10 [0,4194304] 0 2024-07-03T15:43:45.546 INFO:tasks.workunit.client.1.smithi148.stdout:1/118: getdents d0/d2/d11/d1e 0 2024-07-03T15:43:45.546 INFO:tasks.workunit.client.1.smithi148.stdout:5/92: dwrite d7/f19 [4194304,4194304] 0 2024-07-03T15:43:45.555 INFO:tasks.workunit.client.1.smithi148.stdout:4/96: dread d0/d8/f10 [0,4194304] 0 2024-07-03T15:43:45.559 INFO:tasks.workunit.client.1.smithi148.stdout:5/93: dwrite f4 [0,4194304] 0 2024-07-03T15:43:45.559 INFO:tasks.workunit.client.1.smithi148.stdout:5/94: creat d7/da/df/f1f x:0 0 0 2024-07-03T15:43:45.559 INFO:tasks.workunit.client.1.smithi148.stdout:5/95: chown l1 252 1 2024-07-03T15:43:45.559 INFO:tasks.workunit.client.1.smithi148.stdout:5/96: chown d7/ce 134089835 1 2024-07-03T15:43:45.560 INFO:tasks.workunit.client.1.smithi148.stdout:5/97: readlink d7/da/l14 0 2024-07-03T15:43:45.565 INFO:tasks.workunit.client.1.smithi148.stdout:9/235: mknod d0/dd/d25/d37/d40/c56 0 2024-07-03T15:43:45.565 INFO:tasks.workunit.client.1.smithi148.stdout:9/236: write d0/dd/fe [3616174,105169] 0 2024-07-03T15:43:45.565 INFO:tasks.workunit.client.1.smithi148.stdout:9/237: truncate d0/dd/d29/f55 855598 0 2024-07-03T15:43:45.568 INFO:tasks.workunit.client.1.smithi148.stdout:4/97: mknod d0/d8/d13/c1e 0 2024-07-03T15:43:45.568 INFO:tasks.workunit.client.1.smithi148.stdout:4/98: write d0/d8/fd [4500425,52594] 0 2024-07-03T15:43:45.571 INFO:tasks.workunit.client.1.smithi148.stdout:9/238: read d0/dd/d25/f3c [1312633,89878] 0 2024-07-03T15:43:45.571 INFO:tasks.workunit.client.1.smithi148.stdout:9/239: fsync d0/dd/f16 0 2024-07-03T15:43:45.573 INFO:tasks.workunit.client.1.smithi148.stdout:5/98: creat d7/da/df/d16/f20 x:0 0 0 2024-07-03T15:43:45.573 INFO:tasks.workunit.client.1.smithi148.stdout:5/99: read - d7/da/df/d16/f20 zero size 2024-07-03T15:43:45.575 INFO:tasks.workunit.client.1.smithi148.stdout:4/99: mkdir d0/d8/d1f 0 2024-07-03T15:43:45.577 INFO:tasks.workunit.client.1.smithi148.stdout:9/240: symlink d0/d18/d24/d47/d4d/l57 0 2024-07-03T15:43:45.577 INFO:tasks.workunit.client.1.smithi148.stdout:9/241: creat d0/dd/d25/d42/f58 x:0 0 0 2024-07-03T15:43:45.579 INFO:tasks.workunit.client.1.smithi148.stdout:5/100: mkdir d7/da/d21 0 2024-07-03T15:43:45.579 INFO:tasks.workunit.client.1.smithi148.stdout:5/101: truncate d7/da/fb 5199127 0 2024-07-03T15:43:45.579 INFO:tasks.workunit.client.1.smithi148.stdout:5/102: dread - d7/da/df/f1f zero size 2024-07-03T15:43:45.579 INFO:tasks.workunit.client.1.smithi148.stdout:5/103: dread - d7/da/df/d16/f20 zero size 2024-07-03T15:43:45.582 INFO:tasks.workunit.client.1.smithi148.stdout:5/104: mknod d7/da/df/d16/d1a/c22 0 2024-07-03T15:43:45.582 INFO:tasks.workunit.client.1.smithi148.stdout:5/105: chown d7/da/df/d1e 90 1 2024-07-03T15:43:45.585 INFO:tasks.workunit.client.1.smithi148.stdout:5/106: mkdir d7/d13/d23 0 2024-07-03T15:43:45.586 INFO:tasks.workunit.client.1.smithi148.stdout:9/242: read d0/d18/d24/f30 [1757623,18687] 0 2024-07-03T15:43:45.593 INFO:tasks.workunit.client.1.smithi148.stdout:5/107: read f4 [2263759,110736] 0 2024-07-03T15:43:45.595 INFO:tasks.workunit.client.1.smithi148.stdout:9/243: dread d0/dd/d11/f35 [0,4194304] 0 2024-07-03T15:43:45.600 INFO:tasks.workunit.client.1.smithi148.stdout:5/108: dread d7/da/df/f10 [0,4194304] 0 2024-07-03T15:43:45.600 INFO:tasks.workunit.client.1.smithi148.stdout:5/109: chown d7/da/df/f1f 0 1 2024-07-03T15:43:45.602 INFO:tasks.workunit.client.1.smithi148.stdout:0/105: rmdir da 39 2024-07-03T15:43:45.605 INFO:tasks.workunit.client.1.smithi148.stdout:0/106: fdatasync da/dc/d13/f18 0 2024-07-03T15:43:45.606 INFO:tasks.workunit.client.1.smithi148.stdout:7/123: rmdir d5/d8/d21 39 2024-07-03T15:43:45.606 INFO:tasks.workunit.client.1.smithi148.stdout:7/124: write d5/fa [1253853,112230] 0 2024-07-03T15:43:45.606 INFO:tasks.workunit.client.1.smithi148.stdout:7/125: chown d5/d16/d1c 11962 1 2024-07-03T15:43:45.620 INFO:tasks.workunit.client.1.smithi148.stdout:9/244: dwrite d0/d18/f26 [0,4194304] 0 2024-07-03T15:43:45.621 INFO:tasks.workunit.client.1.smithi148.stdout:0/107: write da/dc/f19 [3974947,35009] 0 2024-07-03T15:43:45.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.620+0000 7f39916e0640 1 -- 172.21.15.28:0/3340166038 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f398c0ff850 msgr2=0x7f398c0ffcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:45.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.620+0000 7f39916e0640 1 --2- 172.21.15.28:0/3340166038 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f398c0ff850 0x7f398c0ffcc0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f39800048d0 tx=0x7f3980009da0 comp rx=0 tx=0).stop 2024-07-03T15:43:45.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.620+0000 7f39916e0640 1 -- 172.21.15.28:0/3340166038 shutdown_connections 2024-07-03T15:43:45.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.620+0000 7f39916e0640 1 --2- 172.21.15.28:0/3340166038 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f398c0ff850 0x7f398c0ffcc0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:45.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.620+0000 7f39916e0640 1 --2- 172.21.15.28:0/3340166038 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f398c0fe580 0x7f398c0fe990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:45.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.620+0000 7f39916e0640 1 -- 172.21.15.28:0/3340166038 >> 172.21.15.28:0/3340166038 conn(0x7f398c0f9e70 msgr2=0x7f398c0fc250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:43:45.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.620+0000 7f39916e0640 1 -- 172.21.15.28:0/3340166038 shutdown_connections 2024-07-03T15:43:45.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.621+0000 7f39916e0640 1 -- 172.21.15.28:0/3340166038 wait complete. 2024-07-03T15:43:45.625 INFO:tasks.workunit.client.1.smithi148.stdout:5/110: dwrite f4 [4194304,4194304] 0 2024-07-03T15:43:45.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.621+0000 7f39916e0640 1 Processor -- start 2024-07-03T15:43:45.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.621+0000 7f39916e0640 1 -- start start 2024-07-03T15:43:45.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.622+0000 7f39916e0640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f398c0fe580 0x7f398c06d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:43:45.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.622+0000 7f39916e0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f398c0ff850 0x7f398c06de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:43:45.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.622+0000 7f39916e0640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c06f520 con 0x7f398c0fe580 2024-07-03T15:43:45.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.622+0000 7f39916e0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c075980 con 0x7f398c0ff850 2024-07-03T15:43:45.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.622+0000 7f398affd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f398c0fe580 0x7f398c06d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:43:45.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.622+0000 7f398a7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f398c0ff850 0x7f398c06de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:43:45.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.622+0000 7f398affd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f398c0fe580 0x7f398c06d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53240/0 (socket says 172.21.15.28:53240) 2024-07-03T15:43:45.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.622+0000 7f398affd640 1 -- 172.21.15.28:0/83022458 learned_addr learned my addr 172.21.15.28:0/83022458 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:43:45.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.622+0000 7f398affd640 1 -- 172.21.15.28:0/83022458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f398c0ff850 msgr2=0x7f398c06de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:45.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.622+0000 7f398affd640 1 --2- 172.21.15.28:0/83022458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f398c0ff850 0x7f398c06de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:45.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.622+0000 7f398affd640 1 -- 172.21.15.28:0/83022458 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3980005040 con 0x7f398c0fe580 2024-07-03T15:43:45.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.623+0000 7f398affd640 1 --2- 172.21.15.28:0/83022458 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f398c0fe580 0x7f398c06d8c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f397c00b790 tx=0x7f397c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:43:45.628 INFO:tasks.workunit.client.1.smithi148.stdout:0/108: dread da/d17/f1e [0,4194304] 0 2024-07-03T15:43:45.628 INFO:tasks.workunit.client.1.smithi148.stdout:0/109: chown da/c1c 21984 1 2024-07-03T15:43:45.628 INFO:tasks.workunit.client.1.smithi148.stdout:0/110: write f7 [3307308,99312] 0 2024-07-03T15:43:45.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.623+0000 7f39737fe640 1 -- 172.21.15.28:0/83022458 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f397c00c700 con 0x7f398c0fe580 2024-07-03T15:43:45.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.623+0000 7f39916e0640 1 -- 172.21.15.28:0/83022458 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f398c06e340 con 0x7f398c0fe580 2024-07-03T15:43:45.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.623+0000 7f39916e0640 1 -- 172.21.15.28:0/83022458 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f398c121db0 con 0x7f398c0fe580 2024-07-03T15:43:45.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.623+0000 7f39737fe640 1 -- 172.21.15.28:0/83022458 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f397c00cd30 con 0x7f398c0fe580 2024-07-03T15:43:45.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.623+0000 7f39737fe640 1 -- 172.21.15.28:0/83022458 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f397c013370 con 0x7f398c0fe580 2024-07-03T15:43:45.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.624+0000 7f39916e0640 1 -- 172.21.15.28:0/83022458 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3978005240 con 0x7f398c0fe580 2024-07-03T15:43:45.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.625+0000 7f39737fe640 1 -- 172.21.15.28:0/83022458 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) v1 ==== 85125+0+0 (secure 0 0 0) 0x7f397c012070 con 0x7f398c0fe580 2024-07-03T15:43:45.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.626+0000 7f39737fe640 1 --2- 172.21.15.28:0/83022458 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3974067190 0x7f3974069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:43:45.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.626+0000 7f39737fe640 1 -- 172.21.15.28:0/83022458 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f397c0846c0 con 0x7f398c0fe580 2024-07-03T15:43:45.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.626+0000 7f398a7fc640 1 --2- 172.21.15.28:0/83022458 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3974067190 0x7f3974069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:43:45.631 INFO:tasks.workunit.client.1.smithi148.stdout:7/126: dwrite d5/f6 [0,4194304] 0 2024-07-03T15:43:45.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.627+0000 7f398a7fc640 1 --2- 172.21.15.28:0/83022458 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3974067190 0x7f3974069640 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3980000c00 tx=0x7f3980035040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:43:45.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.630+0000 7f39737fe640 1 -- 172.21.15.28:0/83022458 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f397c051960 con 0x7f398c0fe580 2024-07-03T15:43:45.635 INFO:tasks.workunit.client.1.smithi148.stdout:5/111: rmdir d7/da/df/d1e 0 2024-07-03T15:43:45.635 INFO:tasks.workunit.client.1.smithi148.stdout:5/112: write d7/da/df/f10 [815927,72092] 0 2024-07-03T15:43:45.637 INFO:tasks.workunit.client.1.smithi148.stdout:7/127: write f2 [1598820,115082] 0 2024-07-03T15:43:45.638 INFO:tasks.workunit.client.1.smithi148.stdout:0/111: mkdir da/dc/d24 0 2024-07-03T15:43:45.640 INFO:tasks.workunit.client.1.smithi148.stdout:0/112: dread da/dc/ff [0,4194304] 0 2024-07-03T15:43:45.640 INFO:tasks.workunit.client.1.smithi148.stdout:0/113: dread - da/f11 zero size 2024-07-03T15:43:45.650 INFO:tasks.workunit.client.1.smithi148.stdout:0/114: mknod da/dc/d24/c25 0 2024-07-03T15:43:45.674 INFO:tasks.workunit.client.1.smithi148.stdout:7/128: rmdir d5 39 2024-07-03T15:43:45.674 INFO:tasks.workunit.client.1.smithi148.stdout:7/129: readlink l4 0 2024-07-03T15:43:45.675 INFO:tasks.workunit.client.1.smithi148.stdout:7/130: read d5/fa [450563,50454] 0 2024-07-03T15:43:45.675 INFO:tasks.workunit.client.1.smithi148.stdout:7/131: write d5/d16/d1c/d23/f2b [202904,30759] 0 2024-07-03T15:43:45.675 INFO:tasks.workunit.client.1.smithi148.stdout:7/132: stat d5/d16/d1c/d23 0 2024-07-03T15:43:45.675 INFO:tasks.workunit.client.1.smithi148.stdout:7/133: fsync d5/ff 0 2024-07-03T15:43:45.676 INFO:tasks.workunit.client.1.smithi148.stdout:7/134: link d5/d16/d1c/l1d d5/d16/d1c/l2e 0 2024-07-03T15:43:45.678 INFO:tasks.workunit.client.1.smithi148.stdout:7/135: symlink d5/d8/d21/l2f 0 2024-07-03T15:43:45.680 INFO:tasks.workunit.client.1.smithi148.stdout:7/136: dread d5/d8/f12 [0,4194304] 0 2024-07-03T15:43:45.682 INFO:tasks.workunit.client.1.smithi148.stdout:7/137: rmdir d5/d8 39 2024-07-03T15:43:45.682 INFO:tasks.workunit.client.1.smithi148.stdout:7/138: truncate d5/f1f 263820 0 2024-07-03T15:43:45.682 INFO:tasks.workunit.client.1.smithi148.stdout:7/139: read - d5/d16/d1c/f2d zero size 2024-07-03T15:43:45.683 INFO:tasks.workunit.client.1.smithi148.stdout:7/140: creat d5/d8/f30 x:0 0 0 2024-07-03T15:43:45.684 INFO:tasks.workunit.client.1.smithi148.stdout:7/141: read - d5/d8/f30 zero size 2024-07-03T15:43:45.685 INFO:tasks.workunit.client.1.smithi148.stdout:7/142: rmdir d5/d16/d1c/d23 39 2024-07-03T15:43:45.687 INFO:tasks.workunit.client.1.smithi148.stdout:7/143: mkdir d5/d8/d21/d31 0 2024-07-03T15:43:45.702 INFO:tasks.workunit.client.1.smithi148.stdout:7/144: dwrite d5/d16/d1c/d23/f2b [0,4194304] 0 2024-07-03T15:43:45.703 INFO:tasks.workunit.client.1.smithi148.stdout:7/145: write f2 [5161080,80891] 0 2024-07-03T15:43:45.713 INFO:tasks.workunit.client.1.smithi148.stdout:4/100: dread d0/f3 [0,4194304] 0 2024-07-03T15:43:45.715 INFO:tasks.workunit.client.1.smithi148.stdout:6/208: rename d3/d1d/d28/d31/l44 to d3/d1d/d28/l45 0 2024-07-03T15:43:45.718 INFO:tasks.workunit.client.1.smithi148.stdout:5/113: rename d7/d13/c17 to d7/d13/d23/c24 0 2024-07-03T15:43:45.724 INFO:tasks.workunit.client.1.smithi148.stdout:5/114: dread d7/da/fb [0,4194304] 0 2024-07-03T15:43:45.724 INFO:tasks.workunit.client.1.smithi148.stdout:5/115: read - d7/da/df/f1f zero size 2024-07-03T15:43:45.724 INFO:tasks.workunit.client.1.smithi148.stdout:5/116: chown d7/da/l11 2734417 1 2024-07-03T15:43:45.724 INFO:tasks.workunit.client.1.smithi148.stdout:5/117: chown d7/d13 7836827 1 2024-07-03T15:43:45.725 INFO:tasks.workunit.client.1.smithi148.stdout:0/115: rename f6 to da/dc/f26 0 2024-07-03T15:43:45.728 INFO:tasks.workunit.client.1.smithi148.stdout:4/101: dwrite d0/d8/f10 [0,4194304] 0 2024-07-03T15:43:45.730 INFO:tasks.workunit.client.1.smithi148.stdout:5/118: link d7/da/cc d7/da/df/d1d/c25 0 2024-07-03T15:43:45.731 INFO:tasks.workunit.client.1.smithi148.stdout:7/146: rename d5/d8/f30 to d5/d8/d21/f32 0 2024-07-03T15:43:45.732 INFO:tasks.workunit.client.1.smithi148.stdout:7/147: write d5/f9 [791963,90922] 0 2024-07-03T15:43:45.732 INFO:tasks.workunit.client.1.smithi148.stdout:7/148: readlink d5/d16/l24 0 2024-07-03T15:43:45.732 INFO:tasks.workunit.client.1.smithi148.stdout:7/149: write d5/d8/f26 [4336161,90415] 0 2024-07-03T15:43:45.733 INFO:tasks.workunit.client.1.smithi148.stdout:0/116: creat da/dc/d13/f27 x:0 0 0 2024-07-03T15:43:45.738 INFO:tasks.workunit.client.1.smithi148.stdout:4/102: unlink d0/d8/fd 0 2024-07-03T15:43:45.739 INFO:tasks.workunit.client.1.smithi148.stdout:5/119: rmdir d7/d15 39 2024-07-03T15:43:45.739 INFO:tasks.workunit.client.1.smithi148.stdout:5/120: read - d7/da/df/d16/f20 zero size 2024-07-03T15:43:45.744 INFO:tasks.workunit.client.1.smithi148.stdout:7/150: mkdir d5/d16/d1c/d33 0 2024-07-03T15:43:45.750 INFO:tasks.workunit.client.1.smithi148.stdout:1/119: write d0/f1f [4846102,8244] 0 2024-07-03T15:43:45.752 INFO:tasks.workunit.client.1.smithi148.stdout:7/151: creat d5/d16/f34 x:0 0 0 2024-07-03T15:43:45.752 INFO:tasks.workunit.client.1.smithi148.stdout:7/152: fsync d5/d16/d27/f28 0 2024-07-03T15:43:45.753 INFO:tasks.workunit.client.1.smithi148.stdout:0/117: dwrite da/dc/d13/d1d/f21 [0,4194304] 0 2024-07-03T15:43:45.753 INFO:tasks.workunit.client.1.smithi148.stdout:1/120: dread d0/d2/f10 [4194304,4194304] 0 2024-07-03T15:43:45.753 INFO:tasks.workunit.client.1.smithi148.stdout:7/153: dread d5/d8/f12 [0,4194304] 0 2024-07-03T15:43:45.756 INFO:tasks.workunit.client.1.smithi148.stdout:7/154: dread d5/fa [0,4194304] 0 2024-07-03T15:43:45.756 INFO:tasks.workunit.client.1.smithi148.stdout:7/155: truncate d5/d8/f22 569999 0 2024-07-03T15:43:45.762 INFO:tasks.workunit.client.1.smithi148.stdout:1/121: rename d0/f12 to d0/d2/d6/f26 0 2024-07-03T15:43:45.762 INFO:tasks.workunit.client.1.smithi148.stdout:1/122: write d0/f8 [4734605,74954] 0 2024-07-03T15:43:45.765 INFO:tasks.workunit.client.1.smithi148.stdout:7/156: rename d5/l1b to d5/d16/d1c/d33/l35 0 2024-07-03T15:43:45.765 INFO:tasks.workunit.client.1.smithi148.stdout:7/157: chown l4 229530218 1 2024-07-03T15:43:45.766 INFO:tasks.workunit.client.1.smithi148.stdout:1/123: rmdir d0/d2/d11 39 2024-07-03T15:43:45.775 INFO:tasks.workunit.client.1.smithi148.stdout:9/245: rename d0/dd/d25/d42 to d0/dd/d25/d59 0 2024-07-03T15:43:45.778 INFO:tasks.workunit.client.1.smithi148.stdout:9/246: mkdir d0/d18/d24/d5a 0 2024-07-03T15:43:45.778 INFO:tasks.workunit.client.1.smithi148.stdout:9/247: fdatasync d0/dd/f1a 0 2024-07-03T15:43:45.778 INFO:tasks.workunit.client.1.smithi148.stdout:9/248: chown d0/dd/f14 1171683582 1 2024-07-03T15:43:45.790 INFO:tasks.workunit.client.1.smithi148.stdout:9/249: dwrite d0/dd/d11/f12 [0,4194304] 0 2024-07-03T15:43:45.798 INFO:tasks.workunit.client.1.smithi148.stdout:9/250: read d0/dd/d11/f12 [2219434,71458] 0 2024-07-03T15:43:45.800 INFO:tasks.workunit.client.1.smithi148.stdout:9/251: unlink d0/dd/d25/d37/d40/f44 0 2024-07-03T15:43:45.800 INFO:tasks.workunit.client.1.smithi148.stdout:9/252: chown d0/d18/f21 7 1 2024-07-03T15:43:45.801 INFO:tasks.workunit.client.1.smithi148.stdout:9/253: fdatasync d0/dd/d29/f55 0 2024-07-03T15:43:45.801 INFO:tasks.workunit.client.1.smithi148.stdout:9/254: chown d0/dd/ff 5083843 1 2024-07-03T15:43:45.803 INFO:tasks.workunit.client.1.smithi148.stdout:9/255: truncate d0/dd/f1a 1397701 0 2024-07-03T15:43:45.804 INFO:tasks.workunit.client.1.smithi148.stdout:9/256: write d0/f31 [145495,86063] 0 2024-07-03T15:43:45.804 INFO:tasks.workunit.client.1.smithi148.stdout:9/257: stat d0/d49/d43/d19 0 2024-07-03T15:43:45.806 INFO:tasks.workunit.client.1.smithi148.stdout:9/258: creat d0/d49/d43/f5b x:0 0 0 2024-07-03T15:43:45.812 INFO:tasks.workunit.client.1.smithi148.stdout:9/259: write d0/dd/d11/f35 [2727780,31135] 0 2024-07-03T15:43:45.815 INFO:tasks.workunit.client.1.smithi148.stdout:9/260: write d0/dd/f2b [811326,74897] 0 2024-07-03T15:43:45.819 INFO:tasks.workunit.client.1.smithi148.stdout:9/261: mknod d0/d49/c5c 0 2024-07-03T15:43:45.819 INFO:tasks.workunit.client.1.smithi148.stdout:9/262: stat d0/dd/d11/f12 0 2024-07-03T15:43:45.821 INFO:tasks.workunit.client.1.smithi148.stdout:9/263: symlink d0/dd/d29/l5d 0 2024-07-03T15:43:45.822 INFO:tasks.workunit.client.1.smithi148.stdout:9/264: truncate d0/d49/f4e 4343725 0 2024-07-03T15:43:45.825 INFO:tasks.workunit.client.1.smithi148.stdout:9/265: unlink d0/c3b 0 2024-07-03T15:43:45.825 INFO:tasks.workunit.client.1.smithi148.stdout:9/266: chown d0/d18/d24/c4a 153 1 2024-07-03T15:43:45.825 INFO:tasks.workunit.client.1.smithi148.stdout:9/267: chown d0/d49/d43/l4c 5359052 1 2024-07-03T15:43:45.825 INFO:tasks.workunit.client.1.smithi148.stdout:9/268: truncate d0/dd/d25/d37/f50 1144846 0 2024-07-03T15:43:45.825 INFO:tasks.workunit.client.1.smithi148.stdout:9/269: dread - d0/d49/d43/f22 zero size 2024-07-03T15:43:45.825 INFO:tasks.workunit.client.1.smithi148.stdout:9/270: creat d0/dd/d29/f5e x:0 0 0 2024-07-03T15:43:45.825 INFO:tasks.workunit.client.1.smithi148.stdout:9/271: readlink d0/d49/d43/l52 0 2024-07-03T15:43:45.825 INFO:tasks.workunit.client.1.smithi148.stdout:9/272: write d0/f53 [364367,37903] 0 2024-07-03T15:43:45.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:45 smithi028 ceph-mon[29148]: from='client.14596 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:43:45.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:45 smithi028 ceph-mon[29148]: from='client.14600 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:43:45.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:45 smithi028 ceph-mon[29148]: pgmap v235: 65 pgs: 65 active+clean; 317 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 15 MiB/s rd, 24 MiB/s wr, 426 op/s 2024-07-03T15:43:45.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:45 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/2968271659' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:43:45.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:45 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/1821863728' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-03T15:43:45.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:45 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:43:45.874 INFO:tasks.workunit.client.1.smithi148.stdout:6/209: rmdir d3/d1d/d28/d31 39 2024-07-03T15:43:45.874 INFO:tasks.workunit.client.1.smithi148.stdout:6/210: write d3/f1c [764013,7256] 0 2024-07-03T15:43:45.874 INFO:tasks.workunit.client.1.smithi148.stdout:6/211: write d3/d1d/d28/f2e [432452,58687] 0 2024-07-03T15:43:45.877 INFO:tasks.workunit.client.1.smithi148.stdout:5/121: link d7/d13/d23/c24 d7/da/df/d16/d1a/c26 0 2024-07-03T15:43:45.880 INFO:tasks.workunit.client.1.smithi148.stdout:5/122: mknod d7/d13/d23/c27 0 2024-07-03T15:43:45.881 INFO:tasks.workunit.client.1.smithi148.stdout:5/123: dread - d7/da/df/f1f zero size 2024-07-03T15:43:45.883 INFO:tasks.workunit.client.1.smithi148.stdout:6/212: write d3/d1d/f21 [2980896,20024] 0 2024-07-03T15:43:45.889 INFO:tasks.workunit.client.1.smithi148.stdout:6/213: getdents d3/d1d/d28/d31 0 2024-07-03T15:43:45.899 INFO:tasks.workunit.client.1.smithi148.stdout:5/124: dwrite d7/da/fb [0,4194304] 0 2024-07-03T15:43:45.899 INFO:tasks.workunit.client.1.smithi148.stdout:6/214: dread d3/d1d/f21 [0,4194304] 0 2024-07-03T15:43:45.903 INFO:tasks.workunit.client.1.smithi148.stdout:6/215: getdents d3/d1d/d28 0 2024-07-03T15:43:45.903 INFO:tasks.workunit.client.1.smithi148.stdout:6/216: getdents d3/d1d/d28/d31 0 2024-07-03T15:43:45.918 INFO:tasks.workunit.client.1.smithi148.stdout:1/124: truncate d0/d2/d6/f19 831363 0 2024-07-03T15:43:45.919 INFO:tasks.workunit.client.1.smithi148.stdout:1/125: chown d0/f24 5858 1 2024-07-03T15:43:45.922 INFO:tasks.workunit.client.1.smithi148.stdout:1/126: getdents d0 0 2024-07-03T15:43:45.924 INFO:tasks.workunit.client.1.smithi148.stdout:1/127: symlink d0/d2/d11/d1e/l27 0 2024-07-03T15:43:45.924 INFO:tasks.workunit.client.1.smithi148.stdout:1/128: fsync d0/d2/d6/f20 0 2024-07-03T15:43:45.924 INFO:tasks.workunit.client.1.smithi148.stdout:1/129: readlink d0/l22 0 2024-07-03T15:43:45.924 INFO:tasks.workunit.client.1.smithi148.stdout:8/76: sync 2024-07-03T15:43:45.924 INFO:tasks.workunit.client.1.smithi148.stdout:2/200: sync 2024-07-03T15:43:45.925 INFO:tasks.workunit.client.1.smithi148.stdout:3/101: sync 2024-07-03T15:43:45.925 INFO:tasks.workunit.client.1.smithi148.stdout:3/102: chown f0 128 1 2024-07-03T15:43:45.925 INFO:tasks.workunit.client.1.smithi148.stdout:6/217: dwrite d3/d1d/d28/f2e [0,4194304] 0 2024-07-03T15:43:45.926 INFO:tasks.workunit.client.1.smithi148.stdout:8/77: mkdir d4/da/d14/d1a 0 2024-07-03T15:43:45.926 INFO:tasks.workunit.client.1.smithi148.stdout:8/78: write d4/da/d14/f16 [890454,122009] 0 2024-07-03T15:43:45.928 INFO:tasks.workunit.client.1.smithi148.stdout:3/103: getdents dd/df 0 2024-07-03T15:43:45.935 INFO:tasks.workunit.client.1.smithi148.stdout:2/201: write d5/d14/d22/d2b/d13/f15 [1202715,128412] 0 2024-07-03T15:43:45.935 INFO:tasks.workunit.client.1.smithi148.stdout:2/202: chown l2 2045 1 2024-07-03T15:43:45.937 INFO:tasks.workunit.client.1.smithi148.stdout:1/130: write d0/d2/d6/f20 [2505675,70441] 0 2024-07-03T15:43:45.939 INFO:tasks.workunit.client.1.smithi148.stdout:2/203: symlink d5/d14/l45 0 2024-07-03T15:43:45.939 INFO:tasks.workunit.client.1.smithi148.stdout:2/204: chown d5/d14/d22/c25 367 1 2024-07-03T15:43:45.943 INFO:tasks.workunit.client.1.smithi148.stdout:2/205: link d5/d14/d22/d2b/c1c d5/d14/c46 0 2024-07-03T15:43:45.947 INFO:tasks.workunit.client.1.smithi148.stdout:2/206: mknod d5/d14/d22/d2b/d13/d41/c47 0 2024-07-03T15:43:45.947 INFO:tasks.workunit.client.1.smithi148.stdout:2/207: creat d5/d26/f48 x:0 0 0 2024-07-03T15:43:45.951 INFO:tasks.workunit.client.1.smithi148.stdout:2/208: dread d5/d14/d22/d2b/f1a [0,4194304] 0 2024-07-03T15:43:45.951 INFO:tasks.workunit.client.1.smithi148.stdout:2/209: write d5/f9 [2645310,120850] 0 2024-07-03T15:43:45.951 INFO:tasks.workunit.client.1.smithi148.stdout:1/131: dwrite d0/f25 [0,4194304] 0 2024-07-03T15:43:45.952 INFO:tasks.workunit.client.1.smithi148.stdout:3/104: dwrite f4 [0,4194304] 0 2024-07-03T15:43:45.952 INFO:tasks.workunit.client.1.smithi148.stdout:3/105: chown dd 3333537 1 2024-07-03T15:43:45.952 INFO:tasks.workunit.client.1.smithi148.stdout:3/106: readlink l7 0 2024-07-03T15:43:45.957 INFO:tasks.workunit.client.1.smithi148.stdout:2/210: creat d5/d14/d22/d2b/d13/d41/f49 x:0 0 0 2024-07-03T15:43:45.957 INFO:tasks.workunit.client.1.smithi148.stdout:2/211: dread - d5/d26/f48 zero size 2024-07-03T15:43:45.958 INFO:tasks.workunit.client.1.smithi148.stdout:9/273: dwrite d0/dd/d11/f12 [4194304,4194304] 0 2024-07-03T15:43:45.958 INFO:tasks.workunit.client.1.smithi148.stdout:9/274: write d0/f53 [2159665,105105] 0 2024-07-03T15:43:45.959 INFO:tasks.workunit.client.1.smithi148.stdout:3/107: creat dd/df/f18 x:0 0 0 2024-07-03T15:43:45.960 INFO:tasks.workunit.client.1.smithi148.stdout:3/108: write f0 [1993214,107836] 0 2024-07-03T15:43:45.960 INFO:tasks.workunit.client.1.smithi148.stdout:3/109: creat dd/f19 x:0 0 0 2024-07-03T15:43:45.961 INFO:tasks.workunit.client.1.smithi148.stdout:2/212: mknod d5/d14/c4a 0 2024-07-03T15:43:45.962 INFO:tasks.workunit.client.1.smithi148.stdout:2/213: chown d5/d14/l21 22871926 1 2024-07-03T15:43:45.962 INFO:tasks.workunit.client.1.smithi148.stdout:2/214: chown d5/l36 9 1 2024-07-03T15:43:45.962 INFO:tasks.workunit.client.1.smithi148.stdout:2/215: chown d5/d14/d22/d2b/d13/d41/f49 1008315709 1 2024-07-03T15:43:45.962 INFO:tasks.workunit.client.1.smithi148.stdout:3/110: write f0 [1740689,94458] 0 2024-07-03T15:43:45.970 INFO:tasks.workunit.client.1.smithi148.stdout:2/216: dread d5/f1f [0,4194304] 0 2024-07-03T15:43:45.970 INFO:tasks.workunit.client.1.smithi148.stdout:2/217: chown d5/d14/l21 253968685 1 2024-07-03T15:43:45.970 INFO:tasks.workunit.client.1.smithi148.stdout:2/218: creat d5/d14/d22/d2b/d13/d41/f4b x:0 0 0 2024-07-03T15:43:45.970 INFO:tasks.workunit.client.1.smithi148.stdout:9/275: read d0/d18/f21 [7818545,92948] 0 2024-07-03T15:43:45.971 INFO:tasks.workunit.client.1.smithi148.stdout:3/111: truncate dd/df/f11 569265 0 2024-07-03T15:43:45.971 INFO:tasks.workunit.client.1.smithi148.stdout:3/112: write dd/df/f18 [13752,128850] 0 2024-07-03T15:43:45.975 INFO:tasks.workunit.client.1.smithi148.stdout:2/219: mkdir d5/d14/d22/d4c 0 2024-07-03T15:43:45.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.971+0000 7f39916e0640 1 -- 172.21.15.28:0/83022458 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f39780054f0 con 0x7f398c0fe580 2024-07-03T15:43:45.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.971+0000 7f39737fe640 1 -- 172.21.15.28:0/83022458 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f397c054f80 con 0x7f398c0fe580 2024-07-03T15:43:45.975 INFO:teuthology.orchestra.run.smithi028.stdout:HEALTH_OK 2024-07-03T15:43:45.977 INFO:tasks.workunit.client.1.smithi148.stdout:9/276: unlink d0/f31 0 2024-07-03T15:43:45.977 INFO:tasks.workunit.client.1.smithi148.stdout:9/277: write d0/f53 [1465218,3928] 0 2024-07-03T15:43:45.977 INFO:tasks.workunit.client.1.smithi148.stdout:9/278: chown d0/d49/d43/l4c 98809269 1 2024-07-03T15:43:45.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.974+0000 7f39916e0640 1 -- 172.21.15.28:0/83022458 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3974067190 msgr2=0x7f3974069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:45.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.974+0000 7f39916e0640 1 --2- 172.21.15.28:0/83022458 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3974067190 0x7f3974069640 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3980000c00 tx=0x7f3980035040 comp rx=0 tx=0).stop 2024-07-03T15:43:45.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.974+0000 7f39916e0640 1 -- 172.21.15.28:0/83022458 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f398c0fe580 msgr2=0x7f398c06d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:43:45.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.974+0000 7f39916e0640 1 --2- 172.21.15.28:0/83022458 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f398c0fe580 0x7f398c06d8c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f397c00b790 tx=0x7f397c0091d0 comp rx=0 tx=0).stop 2024-07-03T15:43:45.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.974+0000 7f39916e0640 1 -- 172.21.15.28:0/83022458 shutdown_connections 2024-07-03T15:43:45.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.974+0000 7f39916e0640 1 --2- 172.21.15.28:0/83022458 >> [v2:172.21.15.28:6800/421057178,v1:172.21.15.28:6801/421057178] conn(0x7f3974067190 0x7f3974069640 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:45.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.974+0000 7f39916e0640 1 --2- 172.21.15.28:0/83022458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f398c0ff850 0x7f398c06de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:45.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.974+0000 7f39916e0640 1 --2- 172.21.15.28:0/83022458 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f398c0fe580 0x7f398c06d8c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:43:45.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.974+0000 7f39916e0640 1 -- 172.21.15.28:0/83022458 >> 172.21.15.28:0/83022458 conn(0x7f398c0f9e70 msgr2=0x7f398c0fc540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:43:45.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.974+0000 7f39916e0640 1 -- 172.21.15.28:0/83022458 shutdown_connections 2024-07-03T15:43:45.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:43:45.974+0000 7f39916e0640 1 -- 172.21.15.28:0/83022458 wait complete. 2024-07-03T15:43:45.979 INFO:tasks.workunit.client.1.smithi148.stdout:3/113: symlink dd/df/l1a 0 2024-07-03T15:43:45.980 INFO:tasks.workunit.client.1.smithi148.stdout:9/279: dread d0/f6 [0,4194304] 0 2024-07-03T15:43:45.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:45 smithi148 ceph-mon[36056]: from='client.14596 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:43:45.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:45 smithi148 ceph-mon[36056]: from='client.14600 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:43:45.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:45 smithi148 ceph-mon[36056]: pgmap v235: 65 pgs: 65 active+clean; 317 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 15 MiB/s rd, 24 MiB/s wr, 426 op/s 2024-07-03T15:43:45.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:45 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/2968271659' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:43:45.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:45 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/1821863728' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-03T15:43:45.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:45 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:43:45.982 INFO:tasks.workunit.client.1.smithi148.stdout:3/114: mknod dd/c1b 0 2024-07-03T15:43:45.983 INFO:tasks.workunit.client.1.smithi148.stdout:2/220: dread d5/f2f [0,4194304] 0 2024-07-03T15:43:45.984 INFO:tasks.workunit.client.1.smithi148.stdout:9/280: creat d0/d18/d24/d5a/f5f x:0 0 0 2024-07-03T15:43:45.984 INFO:tasks.workunit.client.1.smithi148.stdout:9/281: chown d0/d18/d24/d5a 9188115 1 2024-07-03T15:43:45.985 INFO:tasks.workunit.client.1.smithi148.stdout:3/115: mkdir dd/d1c 0 2024-07-03T15:43:45.989 INFO:tasks.workunit.client.1.smithi148.stdout:9/282: mkdir d0/d18/d24/d60 0 2024-07-03T15:43:45.992 INFO:tasks.workunit.client.1.smithi148.stdout:3/116: rename f4 to dd/d1c/f1d 0 2024-07-03T15:43:45.992 INFO:tasks.workunit.client.1.smithi148.stdout:9/283: dread d0/dd/f2b [0,4194304] 0 2024-07-03T15:43:45.992 INFO:tasks.workunit.client.1.smithi148.stdout:3/117: dread - dd/df/f16 zero size 2024-07-03T15:43:45.992 INFO:tasks.workunit.client.1.smithi148.stdout:9/284: write d0/dd/f16 [633047,58628] 0 2024-07-03T15:43:45.993 INFO:tasks.workunit.client.1.smithi148.stdout:2/221: getdents d5/d7 0 2024-07-03T15:43:45.997 INFO:tasks.workunit.client.1.smithi148.stdout:3/118: truncate dd/df/f13 1048100 0 2024-07-03T15:43:45.997 INFO:tasks.workunit.client.1.smithi148.stdout:3/119: write dd/f12 [289418,112070] 0 2024-07-03T15:43:45.998 INFO:tasks.workunit.client.1.smithi148.stdout:9/285: symlink d0/dd/l61 0 2024-07-03T15:43:45.998 INFO:tasks.workunit.client.1.smithi148.stdout:9/286: truncate d0/dd/d25/d59/f58 461195 0 2024-07-03T15:43:46.001 INFO:tasks.workunit.client.1.smithi148.stdout:2/222: dread d5/d14/d22/d2b/d13/d2d/f3b [0,4194304] 0 2024-07-03T15:43:46.001 INFO:tasks.workunit.client.1.smithi148.stdout:2/223: dread - d5/d14/d22/d2b/d13/d1d/f43 zero size 2024-07-03T15:43:46.004 INFO:tasks.workunit.client.1.smithi148.stdout:2/224: dread d5/f9 [0,4194304] 0 2024-07-03T15:43:46.005 INFO:tasks.workunit.client.1.smithi148.stdout:2/225: chown d5/d14/l3f 222 1 2024-07-03T15:43:46.005 INFO:tasks.workunit.client.1.smithi148.stdout:9/287: read d0/d49/f4e [774325,95166] 0 2024-07-03T15:43:46.005 INFO:tasks.workunit.client.1.smithi148.stdout:9/288: creat d0/dd/d25/d37/d40/f62 x:0 0 0 2024-07-03T15:43:46.005 INFO:tasks.workunit.client.1.smithi148.stdout:9/289: chown d0/d49/d43 11760347 1 2024-07-03T15:43:46.008 INFO:tasks.workunit.client.1.smithi148.stdout:3/120: mknod dd/df/c1e 0 2024-07-03T15:43:46.010 INFO:tasks.workunit.client.1.smithi148.stdout:2/226: rename d5/d14/d22/d2b/l2a to d5/d14/d22/d4c/l4d 0 2024-07-03T15:43:46.037 INFO:tasks.workunit.client.1.smithi148.stdout:9/290: dwrite d0/dd/f2b [0,4194304] 0 2024-07-03T15:43:46.037 INFO:tasks.workunit.client.1.smithi148.stdout:3/121: dwrite dd/d1c/f1d [0,4194304] 0 2024-07-03T15:43:46.038 INFO:tasks.workunit.client.1.smithi148.stdout:2/227: dwrite d5/d14/d22/d2b/d13/d41/f4b [0,4194304] 0 2024-07-03T15:43:46.038 INFO:tasks.workunit.client.1.smithi148.stdout:2/228: creat d5/d14/d22/d2b/d13/d2d/f4e x:0 0 0 2024-07-03T15:43:46.038 INFO:tasks.workunit.client.1.smithi148.stdout:2/229: chown d5/d26/f48 3287590 1 2024-07-03T15:43:46.038 INFO:tasks.workunit.client.1.smithi148.stdout:2/230: stat d5/d14/d22/d2b/d13/d2d/f39 0 2024-07-03T15:43:46.038 INFO:tasks.workunit.client.1.smithi148.stdout:2/231: dread - d5/d14/d22/d2b/d13/d1d/f43 zero size 2024-07-03T15:43:46.038 INFO:tasks.workunit.client.1.smithi148.stdout:9/291: dread d0/d49/d43/d19/f1e [0,4194304] 0 2024-07-03T15:43:46.039 INFO:tasks.workunit.client.1.smithi148.stdout:9/292: chown d0/d18/d24/d60 1784 1 2024-07-03T15:43:46.039 INFO:tasks.workunit.client.1.smithi148.stdout:9/293: readlink d0/dd/d29/l5d 0 2024-07-03T15:43:46.048 INFO:tasks.workunit.client.1.smithi148.stdout:2/232: symlink d5/d14/d22/d2b/d13/d41/l4f 0 2024-07-03T15:43:46.048 INFO:tasks.workunit.client.1.smithi148.stdout:6/218: truncate d3/d1d/f21 1876703 0 2024-07-03T15:43:46.050 INFO:tasks.workunit.client.1.smithi148.stdout:2/233: creat d5/d7/f50 x:0 0 0 2024-07-03T15:43:46.051 INFO:tasks.workunit.client.1.smithi148.stdout:6/219: symlink d3/d1d/d28/l46 0 2024-07-03T15:43:46.051 INFO:tasks.workunit.client.1.smithi148.stdout:6/220: truncate d3/d1d/d28/f2a 371417 0 2024-07-03T15:43:46.051 INFO:tasks.workunit.client.1.smithi148.stdout:6/221: truncate d3/f37 484786 0 2024-07-03T15:43:46.053 INFO:tasks.workunit.client.1.smithi148.stdout:2/234: rename d5/d14/d22/d2b/d13/d2d/f39 to d5/d14/d22/d2b/d13/d2d/f51 0 2024-07-03T15:43:46.053 INFO:tasks.workunit.client.1.smithi148.stdout:2/235: chown d5/d14/d22/d2b/d13/d41/c47 59416 1 2024-07-03T15:43:46.053 INFO:tasks.workunit.client.1.smithi148.stdout:2/236: fdatasync d5/f1f 0 2024-07-03T15:43:46.054 INFO:tasks.workunit.client.1.smithi148.stdout:2/237: symlink d5/d14/d22/d2b/d13/l52 0 2024-07-03T15:43:46.056 INFO:tasks.workunit.client.1.smithi148.stdout:2/238: creat d5/d14/d22/d4c/f53 x:0 0 0 2024-07-03T15:43:46.060 INFO:tasks.workunit.client.1.smithi148.stdout:9/294: dwrite d0/d18/d24/f30 [0,4194304] 0 2024-07-03T15:43:46.060 INFO:tasks.workunit.client.1.smithi148.stdout:9/295: readlink d0/dd/d25/d39/l54 0 2024-07-03T15:43:46.061 INFO:tasks.workunit.client.1.smithi148.stdout:9/296: write d0/dd/f2b [4947441,117015] 0 2024-07-03T15:43:46.065 INFO:tasks.workunit.client.1.smithi148.stdout:2/239: write d5/d14/d22/d2b/d13/f29 [1410933,108204] 0 2024-07-03T15:43:46.069 INFO:tasks.workunit.client.1.smithi148.stdout:9/297: mkdir d0/d63 0 2024-07-03T15:43:46.069 INFO:tasks.workunit.client.1.smithi148.stdout:9/298: write d0/dd/d29/f55 [1532577,86826] 0 2024-07-03T15:43:46.072 INFO:tasks.workunit.client.1.smithi148.stdout:9/299: rename d0/dd/d29/l5d to d0/d18/d24/d47/l64 0 2024-07-03T15:43:46.074 INFO:tasks.workunit.client.1.smithi148.stdout:2/240: dread d5/d14/d22/d2b/f3a [0,4194304] 0 2024-07-03T15:43:46.075 INFO:tasks.workunit.client.1.smithi148.stdout:9/300: unlink d0/dd/d25/d39/l54 0 2024-07-03T15:43:46.075 INFO:tasks.workunit.client.1.smithi148.stdout:4/103: sync 2024-07-03T15:43:46.075 INFO:tasks.workunit.client.1.smithi148.stdout:0/118: sync 2024-07-03T15:43:46.075 INFO:tasks.workunit.client.1.smithi148.stdout:7/158: sync 2024-07-03T15:43:46.075 INFO:tasks.workunit.client.1.smithi148.stdout:4/104: truncate d0/d8/d13/f1a 186644 0 2024-07-03T15:43:46.075 INFO:tasks.workunit.client.1.smithi148.stdout:4/105: chown d0/d8/d19/c1b 4 1 2024-07-03T15:43:46.077 INFO:tasks.workunit.client.1.smithi148.stdout:9/301: symlink d0/d49/l65 0 2024-07-03T15:43:46.078 INFO:tasks.workunit.client.1.smithi148.stdout:0/119: truncate da/dc/d13/d1d/f10 962014 0 2024-07-03T15:43:46.079 INFO:tasks.workunit.client.1.smithi148.stdout:8/79: rmdir d4 39 2024-07-03T15:43:46.079 INFO:tasks.workunit.client.1.smithi148.stdout:8/80: write f2 [4563493,96574] 0 2024-07-03T15:43:46.080 INFO:tasks.workunit.client.1.smithi148.stdout:8/81: write f2 [5267174,120326] 0 2024-07-03T15:43:46.080 INFO:tasks.workunit.client.1.smithi148.stdout:7/159: link d5/d8/ce d5/d16/c36 0 2024-07-03T15:43:46.082 INFO:tasks.workunit.client.1.smithi148.stdout:4/106: getdents d0/d8/d19 0 2024-07-03T15:43:46.082 INFO:tasks.workunit.client.1.smithi148.stdout:4/107: write d0/d8/f10 [5105982,105465] 0 2024-07-03T15:43:46.082 INFO:tasks.workunit.client.1.smithi148.stdout:4/108: chown d0/f5 3 1 2024-07-03T15:43:46.082 INFO:tasks.workunit.client.1.smithi148.stdout:4/109: chown d0/d8/c18 174922 1 2024-07-03T15:43:46.085 INFO:tasks.workunit.client.1.smithi148.stdout:9/302: unlink d0/dd/d29/f55 0 2024-07-03T15:43:46.085 INFO:tasks.workunit.client.1.smithi148.stdout:9/303: creat d0/dd/d11/f66 x:0 0 0 2024-07-03T15:43:46.087 INFO:tasks.workunit.client.1.smithi148.stdout:0/120: creat da/f28 x:0 0 0 2024-07-03T15:43:46.089 INFO:tasks.workunit.client.1.smithi148.stdout:8/82: mknod d4/d6/d18/c1b 0 2024-07-03T15:43:46.090 INFO:tasks.workunit.client.1.smithi148.stdout:7/160: creat d5/d16/f37 x:0 0 0 2024-07-03T15:43:46.093 INFO:tasks.workunit.client.1.smithi148.stdout:4/110: mkdir d0/d8/d1f/d20 0 2024-07-03T15:43:46.096 INFO:tasks.workunit.client.1.smithi148.stdout:9/304: mknod d0/dd/d25/d37/d40/c67 0 2024-07-03T15:43:46.098 INFO:tasks.workunit.client.1.smithi148.stdout:0/121: rename da/dc/d13/f1b to da/dc/d24/f29 0 2024-07-03T15:43:46.099 INFO:tasks.workunit.client.1.smithi148.stdout:8/83: symlink d4/da/l1c 0 2024-07-03T15:43:46.101 INFO:tasks.workunit.client.1.smithi148.stdout:7/161: getdents d5/d8/d21 0 2024-07-03T15:43:46.102 INFO:tasks.workunit.client.1.smithi148.stdout:9/305: symlink d0/dd/d25/d59/l68 0 2024-07-03T15:43:46.102 INFO:tasks.workunit.client.1.smithi148.stdout:9/306: creat d0/d18/f69 x:0 0 0 2024-07-03T15:43:46.104 INFO:tasks.workunit.client.1.smithi148.stdout:0/122: mkdir da/d17/d2a 0 2024-07-03T15:43:46.106 INFO:tasks.workunit.client.1.smithi148.stdout:8/84: unlink d4/d6/fe 0 2024-07-03T15:43:46.108 INFO:tasks.workunit.client.1.smithi148.stdout:7/162: unlink d5/d8/f22 0 2024-07-03T15:43:46.108 INFO:tasks.workunit.client.1.smithi148.stdout:7/163: write d5/f1f [1089707,12191] 0 2024-07-03T15:43:46.108 INFO:tasks.workunit.client.1.smithi148.stdout:7/164: write d5/f7 [3342656,88717] 0 2024-07-03T15:43:46.108 INFO:tasks.workunit.client.1.smithi148.stdout:7/165: write d5/d8/d21/f29 [667519,69242] 0 2024-07-03T15:43:46.109 INFO:tasks.workunit.client.1.smithi148.stdout:4/111: dwrite d0/f6 [0,4194304] 0 2024-07-03T15:43:46.111 INFO:tasks.workunit.client.1.smithi148.stdout:9/307: mkdir d0/dd/d25/d6a 0 2024-07-03T15:43:46.114 INFO:tasks.workunit.client.1.smithi148.stdout:0/123: unlink da/dc/d13/d1d/f10 0 2024-07-03T15:43:46.115 INFO:tasks.workunit.client.1.smithi148.stdout:8/85: dread f2 [0,4194304] 0 2024-07-03T15:43:46.115 INFO:tasks.workunit.client.1.smithi148.stdout:8/86: write d4/d6/f8 [1324595,16815] 0 2024-07-03T15:43:46.116 INFO:tasks.workunit.client.1.smithi148.stdout:8/87: write d4/d6/f8 [542713,69937] 0 2024-07-03T15:43:46.118 INFO:tasks.workunit.client.1.smithi148.stdout:7/166: mknod d5/d16/d1c/d33/c38 0 2024-07-03T15:43:46.118 INFO:tasks.workunit.client.1.smithi148.stdout:7/167: stat d5/d8/d21/d31 0 2024-07-03T15:43:46.118 INFO:tasks.workunit.client.1.smithi148.stdout:7/168: chown d5/ff 1660350972 1 2024-07-03T15:43:46.121 INFO:tasks.workunit.client.1.smithi148.stdout:9/308: symlink d0/dd/d25/d37/d40/l6b 0 2024-07-03T15:43:46.121 INFO:tasks.workunit.client.1.smithi148.stdout:9/309: write d0/d18/f26 [5159583,54324] 0 2024-07-03T15:43:46.124 INFO:tasks.workunit.client.1.smithi148.stdout:8/88: unlink d4/l12 0 2024-07-03T15:43:46.125 INFO:tasks.workunit.client.1.smithi148.stdout:7/169: mkdir d5/d8/d21/d39 0 2024-07-03T15:43:46.129 INFO:tasks.workunit.client.1.smithi148.stdout:8/89: symlink d4/da/l1d 0 2024-07-03T15:43:46.129 INFO:tasks.workunit.client.1.smithi148.stdout:9/310: write d0/dd/f2b [3395054,117433] 0 2024-07-03T15:43:46.131 INFO:tasks.workunit.client.1.smithi148.stdout:8/90: creat d4/d6/d18/f1e x:0 0 0 2024-07-03T15:43:46.132 INFO:tasks.workunit.client.1.smithi148.stdout:9/311: write d0/f6 [1296959,37408] 0 2024-07-03T15:43:46.134 INFO:tasks.workunit.client.1.smithi148.stdout:9/312: symlink d0/d18/d24/d47/l6c 0 2024-07-03T15:43:46.134 INFO:tasks.workunit.client.1.smithi148.stdout:9/313: fdatasync d0/dd/f1f 0 2024-07-03T15:43:46.136 INFO:tasks.workunit.client.1.smithi148.stdout:8/91: dread d4/da/fb [0,4194304] 0 2024-07-03T15:43:46.136 INFO:tasks.workunit.client.1.smithi148.stdout:0/124: dwrite da/dc/d24/f29 [4194304,4194304] 0 2024-07-03T15:43:46.140 INFO:tasks.workunit.client.1.smithi148.stdout:4/112: dwrite d0/f4 [0,4194304] 0 2024-07-03T15:43:46.140 INFO:tasks.workunit.client.1.smithi148.stdout:3/122: write dd/f15 [1301260,33752] 0 2024-07-03T15:43:46.144 INFO:tasks.workunit.client.1.smithi148.stdout:7/170: dwrite d5/d16/f37 [0,4194304] 0 2024-07-03T15:43:46.144 INFO:tasks.workunit.client.1.smithi148.stdout:7/171: write d5/f1f [1292576,21227] 0 2024-07-03T15:43:46.146 INFO:tasks.workunit.client.1.smithi148.stdout:4/113: creat d0/d8/d1f/d20/f21 x:0 0 0 2024-07-03T15:43:46.155 INFO:tasks.workunit.client.1.smithi148.stdout:7/172: dread d5/f7 [0,4194304] 0 2024-07-03T15:43:46.168 INFO:tasks.workunit.client.1.smithi148.stdout:7/173: read d5/d16/f37 [3038836,28549] 0 2024-07-03T15:43:46.168 INFO:tasks.workunit.client.1.smithi148.stdout:7/174: chown d5/d8/d21 138700264 1 2024-07-03T15:43:46.169 INFO:tasks.workunit.client.1.smithi148.stdout:3/123: dwrite dd/f15 [0,4194304] 0 2024-07-03T15:43:46.169 INFO:tasks.workunit.client.1.smithi148.stdout:3/124: fsync dd/df/f11 0 2024-07-03T15:43:46.169 INFO:tasks.workunit.client.1.smithi148.stdout:3/125: chown dd/f19 32251 1 2024-07-03T15:43:46.169 INFO:tasks.workunit.client.1.smithi148.stdout:4/114: dwrite d0/f17 [0,4194304] 0 2024-07-03T15:43:46.170 INFO:tasks.workunit.client.1.smithi148.stdout:7/175: rename d5/d8/ce to d5/d8/d21/d39/c3a 0 2024-07-03T15:43:46.172 INFO:tasks.workunit.client.1.smithi148.stdout:7/176: dread d5/d8/f12 [0,4194304] 0 2024-07-03T15:43:46.172 INFO:tasks.workunit.client.1.smithi148.stdout:7/177: chown d5/d8/d21 109140456 1 2024-07-03T15:43:46.172 INFO:tasks.workunit.client.1.smithi148.stdout:7/178: dread - d5/d16/f34 zero size 2024-07-03T15:43:46.172 INFO:tasks.workunit.client.1.smithi148.stdout:7/179: chown d5/d8/l18 2025 1 2024-07-03T15:43:46.172 INFO:tasks.workunit.client.1.smithi148.stdout:7/180: write d5/d16/f37 [1023235,102415] 0 2024-07-03T15:43:46.182 INFO:tasks.workunit.client.1.smithi148.stdout:7/181: write d5/f6 [2588054,97655] 0 2024-07-03T15:43:46.182 INFO:tasks.workunit.client.1.smithi148.stdout:7/182: fsync d5/d16/f34 0 2024-07-03T15:43:46.183 INFO:tasks.workunit.client.1.smithi148.stdout:7/183: dread d5/d8/f12 [0,4194304] 0 2024-07-03T15:43:46.184 INFO:tasks.workunit.client.1.smithi148.stdout:7/184: fdatasync d5/fa 0 2024-07-03T15:43:46.184 INFO:tasks.workunit.client.1.smithi148.stdout:4/115: dwrite d0/d8/d1f/d20/f21 [0,4194304] 0 2024-07-03T15:43:46.186 INFO:tasks.workunit.client.1.smithi148.stdout:7/185: link f2 d5/f3b 0 2024-07-03T15:43:46.189 INFO:tasks.workunit.client.1.smithi148.stdout:5/125: sync 2024-07-03T15:43:46.191 INFO:tasks.workunit.client.1.smithi148.stdout:7/186: rename d5/ff to d5/d8/d21/f3c 0 2024-07-03T15:43:46.191 INFO:tasks.workunit.client.1.smithi148.stdout:6/222: dread d3/d1d/f21 [0,4194304] 0 2024-07-03T15:43:46.192 INFO:tasks.workunit.client.1.smithi148.stdout:2/241: rmdir d5/d14/d22/d2b/d13/d2d 39 2024-07-03T15:43:46.193 INFO:tasks.workunit.client.1.smithi148.stdout:4/116: write d0/f16 [442377,52043] 0 2024-07-03T15:43:46.194 INFO:tasks.workunit.client.1.smithi148.stdout:4/117: fsync d0/d8/d13/f1a 0 2024-07-03T15:43:46.194 INFO:tasks.workunit.client.1.smithi148.stdout:4/118: stat d0/f17 0 2024-07-03T15:43:46.196 INFO:tasks.workunit.client.1.smithi148.stdout:7/187: unlink d5/lc 0 2024-07-03T15:43:46.196 INFO:tasks.workunit.client.1.smithi148.stdout:7/188: chown d5/d16/c36 2012171 1 2024-07-03T15:43:46.196 INFO:tasks.workunit.client.1.smithi148.stdout:7/189: chown f2 3970 1 2024-07-03T15:43:46.205 INFO:tasks.workunit.client.1.smithi148.stdout:6/223: write d3/f1a [751090,130124] 0 2024-07-03T15:43:46.205 INFO:tasks.workunit.client.1.smithi148.stdout:7/190: dread d5/d16/d1c/d23/f2b [0,4194304] 0 2024-07-03T15:43:46.208 INFO:tasks.workunit.client.1.smithi148.stdout:7/191: getdents d5/d16/d27 0 2024-07-03T15:43:46.209 INFO:tasks.workunit.client.1.smithi148.stdout:6/224: dread d3/f17 [0,4194304] 0 2024-07-03T15:43:46.209 INFO:tasks.workunit.client.1.smithi148.stdout:1/132: sync 2024-07-03T15:43:46.210 INFO:tasks.workunit.client.1.smithi148.stdout:3/126: truncate dd/d1c/f1d 435764 0 2024-07-03T15:43:46.210 INFO:tasks.workunit.client.1.smithi148.stdout:7/192: unlink d5/d8/l18 0 2024-07-03T15:43:46.211 INFO:tasks.workunit.client.1.smithi148.stdout:7/193: write d5/d16/f34 [1021495,62693] 0 2024-07-03T15:43:46.211 INFO:tasks.workunit.client.1.smithi148.stdout:4/119: dwrite d0/d8/d13/f1a [0,4194304] 0 2024-07-03T15:43:46.214 INFO:tasks.workunit.client.1.smithi148.stdout:1/133: mknod d0/d2/d11/d1e/c28 0 2024-07-03T15:43:46.216 INFO:tasks.workunit.client.1.smithi148.stdout:3/127: creat dd/d1c/f1f x:0 0 0 2024-07-03T15:43:46.217 INFO:tasks.workunit.client.1.smithi148.stdout:7/194: mknod d5/d8/d21/d31/c3d 0 2024-07-03T15:43:46.220 INFO:tasks.workunit.client.1.smithi148.stdout:9/314: getdents d0/dd/d25/d39 0 2024-07-03T15:43:46.221 INFO:tasks.workunit.client.1.smithi148.stdout:9/315: fdatasync d0/dd/fe 0 2024-07-03T15:43:46.221 INFO:tasks.workunit.client.1.smithi148.stdout:9/316: write d0/dd/d11/f66 [699115,62792] 0 2024-07-03T15:43:46.222 INFO:tasks.workunit.client.1.smithi148.stdout:1/134: write d0/f1f [439059,104019] 0 2024-07-03T15:43:46.223 INFO:tasks.workunit.client.1.smithi148.stdout:6/225: rename d3/d1d/c1f to d3/d1d/d28/d31/c47 0 2024-07-03T15:43:46.223 INFO:tasks.workunit.client.1.smithi148.stdout:6/226: write d3/f17 [7957,17277] 0 2024-07-03T15:43:46.223 INFO:tasks.workunit.client.1.smithi148.stdout:3/128: write f5 [392465,25635] 0 2024-07-03T15:43:46.225 INFO:tasks.workunit.client.1.smithi148.stdout:7/195: creat d5/d8/d21/d39/f3e x:0 0 0 2024-07-03T15:43:46.225 INFO:tasks.workunit.client.1.smithi148.stdout:7/196: write d5/d8/d21/f32 [569885,62819] 0 2024-07-03T15:43:46.226 INFO:tasks.workunit.client.1.smithi148.stdout:9/317: mknod d0/d18/d24/d47/c6d 0 2024-07-03T15:43:46.226 INFO:tasks.workunit.client.1.smithi148.stdout:9/318: write d0/d18/f69 [644101,119361] 0 2024-07-03T15:43:46.226 INFO:tasks.workunit.client.1.smithi148.stdout:9/319: truncate d0/d49/d43/f22 664857 0 2024-07-03T15:43:46.230 INFO:tasks.workunit.client.1.smithi148.stdout:1/135: mkdir d0/d2/d6/d14/d29 0 2024-07-03T15:43:46.230 INFO:tasks.workunit.client.1.smithi148.stdout:1/136: write d0/f1f [3901644,12105] 0 2024-07-03T15:43:46.234 INFO:tasks.workunit.client.1.smithi148.stdout:6/227: mknod d3/d1d/d28/d3e/c48 0 2024-07-03T15:43:46.234 INFO:tasks.workunit.client.1.smithi148.stdout:6/228: chown d3/d1d/f24 52006957 1 2024-07-03T15:43:46.236 INFO:tasks.workunit.client.1.smithi148.stdout:9/320: creat d0/dd/d25/d6a/f6e x:0 0 0 2024-07-03T15:43:46.236 INFO:tasks.workunit.client.1.smithi148.stdout:9/321: write d0/d18/f69 [1666365,91550] 0 2024-07-03T15:43:46.238 INFO:tasks.workunit.client.1.smithi148.stdout:6/229: symlink d3/d1d/d28/d3e/l49 0 2024-07-03T15:43:46.239 INFO:tasks.workunit.client.1.smithi148.stdout:6/230: fsync d3/fa 0 2024-07-03T15:43:46.239 INFO:tasks.workunit.client.1.smithi148.stdout:6/231: fdatasync d3/f34 0 2024-07-03T15:43:46.248 INFO:tasks.workunit.client.1.smithi148.stdout:6/232: rename d3/d1d/d28/d31/c3b to d3/d1d/c4a 0 2024-07-03T15:43:46.248 INFO:tasks.workunit.client.1.smithi148.stdout:6/233: chown d3/lf 9 1 2024-07-03T15:43:46.248 INFO:tasks.workunit.client.1.smithi148.stdout:6/234: chown d3/l14 58 1 2024-07-03T15:43:46.258 INFO:tasks.workunit.client.1.smithi148.stdout:1/137: dread d0/d2/f10 [0,4194304] 0 2024-07-03T15:43:46.264 INFO:tasks.workunit.client.1.smithi148.stdout:1/138: unlink d0/cb 0 2024-07-03T15:43:46.264 INFO:tasks.workunit.client.1.smithi148.stdout:1/139: write d0/d2/d6/f26 [8627708,123912] 0 2024-07-03T15:43:46.264 INFO:tasks.workunit.client.1.smithi148.stdout:1/140: fsync d0/d2/d6/d14/f15 0 2024-07-03T15:43:46.265 INFO:tasks.workunit.client.1.smithi148.stdout:2/242: dwrite d5/d14/d22/d2b/f1a [0,4194304] 0 2024-07-03T15:43:46.265 INFO:tasks.workunit.client.1.smithi148.stdout:1/141: symlink d0/d2/d11/l2a 0 2024-07-03T15:43:46.268 INFO:tasks.workunit.client.1.smithi148.stdout:8/92: rmdir d4/d6 39 2024-07-03T15:43:46.268 INFO:tasks.workunit.client.1.smithi148.stdout:6/235: dwrite d3/d1d/d28/d31/f43 [0,4194304] 0 2024-07-03T15:43:46.272 INFO:tasks.workunit.client.1.smithi148.stdout:8/93: mkdir d4/d6/d18/d1f 0 2024-07-03T15:43:46.279 INFO:tasks.workunit.client.1.smithi148.stdout:6/236: rename d3/f1a to d3/d1d/d28/d31/f4b 0 2024-07-03T15:43:46.279 INFO:tasks.workunit.client.1.smithi148.stdout:6/237: truncate d3/d1d/d28/f2a 666132 0 2024-07-03T15:43:46.279 INFO:tasks.workunit.client.1.smithi148.stdout:6/238: fdatasync d3/f37 0 2024-07-03T15:43:46.279 INFO:tasks.workunit.client.1.smithi148.stdout:6/239: fdatasync d3/d1d/f27 0 2024-07-03T15:43:46.279 INFO:tasks.workunit.client.1.smithi148.stdout:0/125: truncate da/dc/d24/f29 5983674 0 2024-07-03T15:43:46.280 INFO:tasks.workunit.client.1.smithi148.stdout:0/126: write da/f28 [1001428,71996] 0 2024-07-03T15:43:46.280 INFO:tasks.workunit.client.1.smithi148.stdout:0/127: chown da/dc/l22 139818124 1 2024-07-03T15:43:46.280 INFO:tasks.workunit.client.1.smithi148.stdout:8/94: dread d4/d6/f7 [0,4194304] 0 2024-07-03T15:43:46.282 INFO:tasks.workunit.client.1.smithi148.stdout:6/240: mkdir d3/d1d/d28/d31/d4c 0 2024-07-03T15:43:46.282 INFO:tasks.workunit.client.1.smithi148.stdout:6/241: fsync d3/f37 0 2024-07-03T15:43:46.283 INFO:tasks.workunit.client.1.smithi148.stdout:0/128: write f7 [2333889,100441] 0 2024-07-03T15:43:46.285 INFO:tasks.workunit.client.1.smithi148.stdout:6/242: mknod d3/d1d/d28/d31/c4d 0 2024-07-03T15:43:46.289 INFO:tasks.workunit.client.1.smithi148.stdout:2/243: dwrite d5/d14/d22/d2b/d13/d2d/f51 [0,4194304] 0 2024-07-03T15:43:46.289 INFO:tasks.workunit.client.1.smithi148.stdout:2/244: fsync d5/d14/d22/d2b/d13/d2d/f51 0 2024-07-03T15:43:46.289 INFO:tasks.workunit.client.1.smithi148.stdout:1/142: dwrite d0/d2/fd [0,4194304] 0 2024-07-03T15:43:46.289 INFO:tasks.workunit.client.1.smithi148.stdout:1/143: readlink d0/de/lf 0 2024-07-03T15:43:46.290 INFO:tasks.workunit.client.1.smithi148.stdout:6/243: creat d3/d1d/f4e x:0 0 0 2024-07-03T15:43:46.305 INFO:tasks.workunit.client.1.smithi148.stdout:8/95: dwrite d4/f11 [0,4194304] 0 2024-07-03T15:43:46.316 INFO:tasks.workunit.client.1.smithi148.stdout:8/96: mkdir d4/d6/d18/d20 0 2024-07-03T15:43:46.318 INFO:tasks.workunit.client.1.smithi148.stdout:2/245: dwrite d5/d14/d22/d2b/f16 [0,4194304] 0 2024-07-03T15:43:46.318 INFO:tasks.workunit.client.1.smithi148.stdout:8/97: rmdir d4/d6/d18 39 2024-07-03T15:43:46.318 INFO:tasks.workunit.client.1.smithi148.stdout:8/98: truncate d4/da/fb 669475 0 2024-07-03T15:43:46.318 INFO:tasks.workunit.client.1.smithi148.stdout:6/244: dwrite d3/d1d/d28/f2a [0,4194304] 0 2024-07-03T15:43:46.319 INFO:tasks.workunit.client.1.smithi148.stdout:0/129: dwrite da/dc/f19 [0,4194304] 0 2024-07-03T15:43:46.323 INFO:tasks.workunit.client.1.smithi148.stdout:2/246: unlink d5/d7/f50 0 2024-07-03T15:43:46.327 INFO:tasks.workunit.client.1.smithi148.stdout:8/99: creat d4/d6/d18/d20/f21 x:0 0 0 2024-07-03T15:43:46.333 INFO:tasks.workunit.client.1.smithi148.stdout:6/245: rename d3/l5 to d3/d1d/d28/l4f 0 2024-07-03T15:43:46.334 INFO:tasks.workunit.client.1.smithi148.stdout:6/246: chown d3/f7 4842 1 2024-07-03T15:43:46.339 INFO:tasks.workunit.client.1.smithi148.stdout:2/247: rename d5/d7/l37 to d5/d7/l54 0 2024-07-03T15:43:46.340 INFO:tasks.workunit.client.1.smithi148.stdout:0/130: dwrite da/dc/d13/f27 [0,4194304] 0 2024-07-03T15:43:46.340 INFO:tasks.workunit.client.1.smithi148.stdout:0/131: dread - da/f11 zero size 2024-07-03T15:43:46.341 INFO:tasks.workunit.client.1.smithi148.stdout:7/197: getdents d5 0 2024-07-03T15:43:46.341 INFO:tasks.workunit.client.1.smithi148.stdout:7/198: rename d5 to d5/d16/d27/d3f 22 2024-07-03T15:43:46.341 INFO:tasks.workunit.client.1.smithi148.stdout:7/199: chown d5/d16/d27 428994589 1 2024-07-03T15:43:46.341 INFO:tasks.workunit.client.1.smithi148.stdout:7/200: chown d5/d16/d1c/d33 1339723855 1 2024-07-03T15:43:46.341 INFO:tasks.workunit.client.1.smithi148.stdout:7/201: chown d5/d16/c36 9 1 2024-07-03T15:43:46.343 INFO:tasks.workunit.client.1.smithi148.stdout:8/100: link d4/d6/c15 d4/d6/c22 0 2024-07-03T15:43:46.343 INFO:tasks.workunit.client.1.smithi148.stdout:8/101: read d4/d6/f8 [197565,79081] 0 2024-07-03T15:43:46.345 INFO:tasks.workunit.client.1.smithi148.stdout:8/102: dread d4/da/d14/f16 [0,4194304] 0 2024-07-03T15:43:46.345 INFO:tasks.workunit.client.1.smithi148.stdout:8/103: fsync d4/d6/f7 0 2024-07-03T15:43:46.345 INFO:tasks.workunit.client.1.smithi148.stdout:6/247: mkdir d3/d50 0 2024-07-03T15:43:46.346 INFO:tasks.workunit.client.1.smithi148.stdout:6/248: fsync d3/d1d/f27 0 2024-07-03T15:43:46.347 INFO:tasks.workunit.client.1.smithi148.stdout:2/248: mknod d5/d14/c55 0 2024-07-03T15:43:46.348 INFO:tasks.workunit.client.1.smithi148.stdout:2/249: write d5/d14/d22/d2b/d13/f1b [3543883,51186] 0 2024-07-03T15:43:46.348 INFO:tasks.workunit.client.1.smithi148.stdout:2/250: chown d5/f40 98 1 2024-07-03T15:43:46.352 INFO:tasks.workunit.client.1.smithi148.stdout:7/202: mknod d5/d16/d27/c40 0 2024-07-03T15:43:46.352 INFO:tasks.workunit.client.1.smithi148.stdout:7/203: readlink d5/d8/ld 0 2024-07-03T15:43:46.353 INFO:tasks.workunit.client.1.smithi148.stdout:6/249: write d3/d1d/f25 [2613267,87265] 0 2024-07-03T15:43:46.355 INFO:tasks.workunit.client.1.smithi148.stdout:8/104: symlink d4/d6/l23 0 2024-07-03T15:43:46.355 INFO:tasks.workunit.client.1.smithi148.stdout:8/105: chown d4/d6/c15 1114 1 2024-07-03T15:43:46.355 INFO:tasks.workunit.client.1.smithi148.stdout:8/106: write d4/d6/fc [145744,83847] 0 2024-07-03T15:43:46.360 INFO:tasks.workunit.client.1.smithi148.stdout:3/129: write dd/d1c/f1d [328083,102563] 0 2024-07-03T15:43:46.360 INFO:tasks.workunit.client.1.smithi148.stdout:3/130: chown fa 283806 1 2024-07-03T15:43:46.360 INFO:tasks.workunit.client.1.smithi148.stdout:3/131: write f0 [1823509,38234] 0 2024-07-03T15:43:46.361 INFO:tasks.workunit.client.1.smithi148.stdout:2/251: symlink d5/d14/d22/d2b/d13/d41/l56 0 2024-07-03T15:43:46.361 INFO:tasks.workunit.client.1.smithi148.stdout:2/252: creat d5/d26/f57 x:0 0 0 2024-07-03T15:43:46.361 INFO:tasks.workunit.client.1.smithi148.stdout:2/253: write d5/d14/d22/d2b/d13/f1b [5286270,117787] 0 2024-07-03T15:43:46.364 INFO:tasks.workunit.client.1.smithi148.stdout:7/204: unlink d5/d16/c2c 0 2024-07-03T15:43:46.364 INFO:tasks.workunit.client.1.smithi148.stdout:7/205: truncate d5/fa 2231493 0 2024-07-03T15:43:46.364 INFO:tasks.workunit.client.1.smithi148.stdout:7/206: chown d5/d16/l24 1 1 2024-07-03T15:43:46.370 INFO:tasks.workunit.client.1.smithi148.stdout:3/132: rename dd/df/f16 to dd/d1c/f20 0 2024-07-03T15:43:46.376 INFO:tasks.workunit.client.1.smithi148.stdout:8/107: dwrite d4/d6/f8 [0,4194304] 0 2024-07-03T15:43:46.376 INFO:tasks.workunit.client.1.smithi148.stdout:8/108: truncate d4/d6/d18/f1e 277307 0 2024-07-03T15:43:46.378 INFO:tasks.workunit.client.1.smithi148.stdout:2/254: link d5/d14/d22/d2b/d13/f15 d5/d14/f58 0 2024-07-03T15:43:46.385 INFO:tasks.workunit.client.1.smithi148.stdout:4/120: truncate d0/f4 1964565 0 2024-07-03T15:43:46.388 INFO:tasks.workunit.client.1.smithi148.stdout:7/207: rename f2 to d5/d16/d1c/f41 0 2024-07-03T15:43:46.388 INFO:tasks.workunit.client.1.smithi148.stdout:7/208: truncate d5/d8/d21/d39/f3e 617073 0 2024-07-03T15:43:46.389 INFO:tasks.workunit.client.1.smithi148.stdout:7/209: chown l4 260101 1 2024-07-03T15:43:46.389 INFO:tasks.workunit.client.1.smithi148.stdout:7/210: chown d5/d16/l24 1 1 2024-07-03T15:43:46.393 INFO:tasks.workunit.client.1.smithi148.stdout:3/133: dwrite f5 [0,4194304] 0 2024-07-03T15:43:46.394 INFO:tasks.workunit.client.1.smithi148.stdout:8/109: rmdir d4/da 39 2024-07-03T15:43:46.395 INFO:tasks.workunit.client.1.smithi148.stdout:4/121: write d0/d8/d1f/d20/f21 [3436572,87670] 0 2024-07-03T15:43:46.396 INFO:tasks.workunit.client.1.smithi148.stdout:2/255: rename d5/d14/c4a to d5/c59 0 2024-07-03T15:43:46.396 INFO:tasks.workunit.client.1.smithi148.stdout:2/256: dread - d5/d14/d22/d2b/d13/d2d/f4e zero size 2024-07-03T15:43:46.396 INFO:tasks.workunit.client.1.smithi148.stdout:2/257: write d5/d26/f57 [2910,81608] 0 2024-07-03T15:43:46.398 INFO:tasks.workunit.client.1.smithi148.stdout:4/122: dread d0/f3 [0,4194304] 0 2024-07-03T15:43:46.398 INFO:tasks.workunit.client.1.smithi148.stdout:7/211: unlink d5/c25 0 2024-07-03T15:43:46.401 INFO:tasks.workunit.client.1.smithi148.stdout:8/110: link d4/da/l1c d4/d6/d18/l24 0 2024-07-03T15:43:46.403 INFO:tasks.workunit.client.1.smithi148.stdout:7/212: dread d5/f7 [0,4194304] 0 2024-07-03T15:43:46.405 INFO:tasks.workunit.client.1.smithi148.stdout:2/258: creat d5/d14/f5a x:0 0 0 2024-07-03T15:43:46.405 INFO:tasks.workunit.client.1.smithi148.stdout:2/259: dread - d5/d7/f35 zero size 2024-07-03T15:43:46.408 INFO:tasks.workunit.client.1.smithi148.stdout:7/213: creat d5/d16/d1c/d33/f42 x:0 0 0 2024-07-03T15:43:46.409 INFO:tasks.workunit.client.1.smithi148.stdout:3/134: dwrite dd/d1c/f1f [0,4194304] 0 2024-07-03T15:43:46.419 INFO:tasks.workunit.client.1.smithi148.stdout:4/123: rename d0/d8/d1f/d20 to d0/d8/d22 0 2024-07-03T15:43:46.422 INFO:tasks.workunit.client.1.smithi148.stdout:4/124: mknod d0/d8/d19/c23 0 2024-07-03T15:43:46.432 INFO:tasks.workunit.client.1.smithi148.stdout:3/135: dwrite dd/d1c/f1d [0,4194304] 0 2024-07-03T15:43:46.439 INFO:tasks.workunit.client.1.smithi148.stdout:0/132: write da/dc/d24/f29 [6069573,108250] 0 2024-07-03T15:43:46.439 INFO:tasks.workunit.client.1.smithi148.stdout:7/214: dwrite d5/f6 [0,4194304] 0 2024-07-03T15:43:46.440 INFO:tasks.workunit.client.1.smithi148.stdout:4/125: dwrite d0/f6 [0,4194304] 0 2024-07-03T15:43:46.440 INFO:tasks.workunit.client.1.smithi148.stdout:4/126: truncate d0/f6 4666225 0 2024-07-03T15:43:46.443 INFO:tasks.workunit.client.1.smithi148.stdout:3/136: dread dd/d1c/f1d [0,4194304] 0 2024-07-03T15:43:46.457 INFO:tasks.workunit.client.1.smithi148.stdout:7/215: rmdir d5/d16/d1c/d23 39 2024-07-03T15:43:46.458 INFO:tasks.workunit.client.1.smithi148.stdout:7/216: dread - d5/d8/d21/f3c zero size 2024-07-03T15:43:46.459 INFO:tasks.workunit.client.1.smithi148.stdout:4/127: mkdir d0/d8/d19/d24 0 2024-07-03T15:43:46.459 INFO:tasks.workunit.client.1.smithi148.stdout:4/128: dread - d0/d8/ff zero size 2024-07-03T15:43:46.465 INFO:tasks.workunit.client.1.smithi148.stdout:0/133: dwrite f7 [0,4194304] 0 2024-07-03T15:43:46.474 INFO:tasks.workunit.client.1.smithi148.stdout:0/134: mknod da/dc/d13/c2b 0 2024-07-03T15:43:46.474 INFO:tasks.workunit.client.1.smithi148.stdout:0/135: symlink da/d17/d2a/l2c 0 2024-07-03T15:43:46.476 INFO:tasks.workunit.client.1.smithi148.stdout:0/136: creat da/d17/d2a/f2d x:0 0 0 2024-07-03T15:43:46.476 INFO:tasks.workunit.client.1.smithi148.stdout:0/137: creat da/d17/d2a/f2e x:0 0 0 2024-07-03T15:43:46.481 INFO:tasks.workunit.client.1.smithi148.stdout:0/138: unlink f7 0 2024-07-03T15:43:46.485 INFO:tasks.workunit.client.1.smithi148.stdout:3/137: dwrite dd/f12 [0,4194304] 0 2024-07-03T15:43:46.485 INFO:tasks.workunit.client.1.smithi148.stdout:3/138: readlink dd/le 0 2024-07-03T15:43:46.486 INFO:tasks.workunit.client.1.smithi148.stdout:4/129: dwrite d0/f5 [0,4194304] 0 2024-07-03T15:43:46.488 INFO:tasks.workunit.client.1.smithi148.stdout:4/130: dread d0/f3 [0,4194304] 0 2024-07-03T15:43:46.490 INFO:tasks.workunit.client.1.smithi148.stdout:3/139: unlink dd/df/c1e 0 2024-07-03T15:43:46.490 INFO:tasks.workunit.client.1.smithi148.stdout:3/140: truncate dd/f12 4223265 0 2024-07-03T15:43:46.495 INFO:tasks.workunit.client.1.smithi148.stdout:0/139: dwrite da/f28 [0,4194304] 0 2024-07-03T15:43:46.495 INFO:tasks.workunit.client.1.smithi148.stdout:0/140: chown da/d17/d2a 23 1 2024-07-03T15:43:46.499 INFO:tasks.workunit.client.1.smithi148.stdout:8/111: getdents d4/d6 0 2024-07-03T15:43:46.500 INFO:tasks.workunit.client.1.smithi148.stdout:8/112: write d4/d6/d18/f1e [8971,128150] 0 2024-07-03T15:43:46.506 INFO:tasks.workunit.client.1.smithi148.stdout:4/131: dwrite d0/f16 [0,4194304] 0 2024-07-03T15:43:46.509 INFO:tasks.workunit.client.1.smithi148.stdout:0/141: dread da/f28 [0,4194304] 0 2024-07-03T15:43:46.509 INFO:tasks.workunit.client.1.smithi148.stdout:0/142: stat c9 0 2024-07-03T15:43:46.509 INFO:tasks.workunit.client.1.smithi148.stdout:0/143: write da/d17/f1e [1821960,115216] 0 2024-07-03T15:43:46.510 INFO:tasks.workunit.client.1.smithi148.stdout:0/144: chown da/dc/ff 731775 1 2024-07-03T15:43:46.510 INFO:tasks.workunit.client.1.smithi148.stdout:0/145: chown da/d17 1 1 2024-07-03T15:43:46.510 INFO:tasks.workunit.client.1.smithi148.stdout:0/146: dread - da/d17/d2a/f2e zero size 2024-07-03T15:43:46.511 INFO:tasks.workunit.client.1.smithi148.stdout:2/260: fsync d5/d26/f57 0 2024-07-03T15:43:46.511 INFO:tasks.workunit.client.1.smithi148.stdout:2/261: fsync d5/d14/d22/d2b/d13/d2d/f51 0 2024-07-03T15:43:46.513 INFO:tasks.workunit.client.1.smithi148.stdout:2/262: symlink d5/d26/l5b 0 2024-07-03T15:43:46.517 INFO:tasks.workunit.client.1.smithi148.stdout:2/263: mknod d5/d14/d22/d2b/d13/c5c 0 2024-07-03T15:43:46.528 INFO:tasks.workunit.client.1.smithi148.stdout:8/113: dwrite f2 [4194304,4194304] 0 2024-07-03T15:43:46.528 INFO:tasks.workunit.client.1.smithi148.stdout:8/114: chown d4/da/l1d 116955223 1 2024-07-03T15:43:46.528 INFO:tasks.workunit.client.1.smithi148.stdout:8/115: getdents d4/da/d14/d17 0 2024-07-03T15:43:46.530 INFO:tasks.workunit.client.1.smithi148.stdout:8/116: mkdir d4/d6/d18/d25 0 2024-07-03T15:43:46.532 INFO:tasks.workunit.client.1.smithi148.stdout:0/147: fdatasync da/dc/d24/f29 0 2024-07-03T15:43:46.532 INFO:tasks.workunit.client.1.smithi148.stdout:2/264: dwrite d5/d14/d22/d2b/d13/d41/f49 [0,4194304] 0 2024-07-03T15:43:46.532 INFO:tasks.workunit.client.1.smithi148.stdout:4/132: dwrite d0/d8/f10 [0,4194304] 0 2024-07-03T15:43:46.533 INFO:tasks.workunit.client.1.smithi148.stdout:8/117: unlink d4/da/l1d 0 2024-07-03T15:43:46.535 INFO:tasks.workunit.client.1.smithi148.stdout:0/148: mkdir da/d17/d2a/d2f 0 2024-07-03T15:43:46.535 INFO:tasks.workunit.client.1.smithi148.stdout:0/149: read - da/dc/d13/d14/f1f zero size 2024-07-03T15:43:46.535 INFO:tasks.workunit.client.1.smithi148.stdout:8/118: fsync d4/d6/d18/f1e 0 2024-07-03T15:43:46.539 INFO:tasks.workunit.client.1.smithi148.stdout:2/265: mknod d5/d14/d22/d4c/c5d 0 2024-07-03T15:43:46.544 INFO:tasks.workunit.client.1.smithi148.stdout:4/133: rename d0/d8/d13/c1d to d0/c25 0 2024-07-03T15:43:46.544 INFO:tasks.workunit.client.1.smithi148.stdout:4/134: dread - d0/d8/ff zero size 2024-07-03T15:43:46.547 INFO:tasks.workunit.client.1.smithi148.stdout:0/150: creat da/dc/d24/f30 x:0 0 0 2024-07-03T15:43:46.548 INFO:tasks.workunit.client.1.smithi148.stdout:3/141: truncate dd/d1c/f1f 1177065 0 2024-07-03T15:43:46.548 INFO:tasks.workunit.client.1.smithi148.stdout:3/142: creat dd/f21 x:0 0 0 2024-07-03T15:43:46.549 INFO:tasks.workunit.client.1.smithi148.stdout:3/143: readlink dd/le 0 2024-07-03T15:43:46.553 INFO:tasks.workunit.client.1.smithi148.stdout:4/135: rename d0/c14 to d0/d8/d13/c26 0 2024-07-03T15:43:46.557 INFO:tasks.workunit.client.1.smithi148.stdout:8/119: dwrite d4/d6/f7 [0,4194304] 0 2024-07-03T15:43:46.557 INFO:tasks.workunit.client.1.smithi148.stdout:8/120: readlink d4/d6/l9 0 2024-07-03T15:43:46.568 INFO:tasks.workunit.client.1.smithi148.stdout:0/151: dwrite da/dc/d24/f30 [0,4194304] 0 2024-07-03T15:43:46.568 INFO:tasks.workunit.client.1.smithi148.stdout:4/136: dread d0/d8/d13/f1a [0,4194304] 0 2024-07-03T15:43:46.568 INFO:tasks.workunit.client.1.smithi148.stdout:4/137: read d0/d8/d22/f21 [755988,106934] 0 2024-07-03T15:43:46.574 INFO:tasks.workunit.client.1.smithi148.stdout:3/144: dwrite dd/df/f13 [4194304,4194304] 0 2024-07-03T15:43:46.577 INFO:tasks.workunit.client.1.smithi148.stdout:0/152: creat da/dc/f31 x:0 0 0 2024-07-03T15:43:46.578 INFO:tasks.workunit.client.1.smithi148.stdout:4/138: creat d0/d8/d19/d24/f27 x:0 0 0 2024-07-03T15:43:46.579 INFO:tasks.workunit.client.1.smithi148.stdout:8/121: dwrite d4/f11 [0,4194304] 0 2024-07-03T15:43:46.580 INFO:tasks.workunit.client.1.smithi148.stdout:0/153: unlink c9 0 2024-07-03T15:43:46.582 INFO:tasks.workunit.client.1.smithi148.stdout:4/139: mkdir d0/d28 0 2024-07-03T15:43:46.582 INFO:tasks.workunit.client.1.smithi148.stdout:4/140: fdatasync d0/f16 0 2024-07-03T15:43:46.583 INFO:tasks.workunit.client.1.smithi148.stdout:3/145: dread f5 [0,4194304] 0 2024-07-03T15:43:46.584 INFO:tasks.workunit.client.1.smithi148.stdout:3/146: creat dd/f22 x:0 0 0 2024-07-03T15:43:46.585 INFO:tasks.workunit.client.1.smithi148.stdout:0/154: link da/dc/f31 da/dc/d13/d14/f32 0 2024-07-03T15:43:46.585 INFO:tasks.workunit.client.1.smithi148.stdout:0/155: chown da/dc/d13/d1d 106836 1 2024-07-03T15:43:46.585 INFO:tasks.workunit.client.1.smithi148.stdout:6/250: sync 2024-07-03T15:43:46.586 INFO:tasks.workunit.client.1.smithi148.stdout:1/144: sync 2024-07-03T15:43:46.586 INFO:tasks.workunit.client.1.smithi148.stdout:9/322: sync 2024-07-03T15:43:46.586 INFO:tasks.workunit.client.1.smithi148.stdout:5/126: sync 2024-07-03T15:43:46.586 INFO:tasks.workunit.client.1.smithi148.stdout:9/323: truncate d0/d18/d24/d5a/f5f 96643 0 2024-07-03T15:43:46.586 INFO:tasks.workunit.client.1.smithi148.stdout:9/324: fsync d0/dd/fe 0 2024-07-03T15:43:46.587 INFO:tasks.workunit.client.1.smithi148.stdout:4/141: dread d0/f4 [0,4194304] 0 2024-07-03T15:43:46.588 INFO:tasks.workunit.client.1.smithi148.stdout:7/217: truncate d5/f6 3303415 0 2024-07-03T15:43:46.589 INFO:tasks.workunit.client.1.smithi148.stdout:6/251: creat d3/d1d/d28/f51 x:0 0 0 2024-07-03T15:43:46.590 INFO:tasks.workunit.client.1.smithi148.stdout:5/127: creat d7/d13/f28 x:0 0 0 2024-07-03T15:43:46.591 INFO:tasks.workunit.client.1.smithi148.stdout:5/128: fdatasync d7/da/fb 0 2024-07-03T15:43:46.591 INFO:tasks.workunit.client.1.smithi148.stdout:5/129: readlink l1 0 2024-07-03T15:43:46.591 INFO:tasks.workunit.client.1.smithi148.stdout:8/122: dread d4/d6/f7 [4194304,4194304] 0 2024-07-03T15:43:46.591 INFO:tasks.workunit.client.1.smithi148.stdout:8/123: dread - d4/d6/d18/d20/f21 zero size 2024-07-03T15:43:46.592 INFO:tasks.workunit.client.1.smithi148.stdout:1/145: symlink d0/d2/d6/d14/d29/l2b 0 2024-07-03T15:43:46.592 INFO:tasks.workunit.client.1.smithi148.stdout:1/146: dread - d0/f24 zero size 2024-07-03T15:43:46.594 INFO:tasks.workunit.client.1.smithi148.stdout:4/142: symlink d0/d8/l29 0 2024-07-03T15:43:46.595 INFO:tasks.workunit.client.1.smithi148.stdout:0/156: write da/dc/d13/d1d/f21 [1410053,54057] 0 2024-07-03T15:43:46.595 INFO:tasks.workunit.client.1.smithi148.stdout:0/157: read - da/dc/d13/f18 zero size 2024-07-03T15:43:46.596 INFO:tasks.workunit.client.1.smithi148.stdout:7/218: write d5/d16/d1c/f41 [4344518,81644] 0 2024-07-03T15:43:46.596 INFO:tasks.workunit.client.1.smithi148.stdout:5/130: symlink d7/d15/l29 0 2024-07-03T15:43:46.596 INFO:tasks.workunit.client.1.smithi148.stdout:5/131: chown f4 245 1 2024-07-03T15:43:46.598 INFO:tasks.workunit.client.1.smithi148.stdout:8/124: creat d4/da/d14/d17/f26 x:0 0 0 2024-07-03T15:43:46.598 INFO:tasks.workunit.client.1.smithi148.stdout:8/125: write d4/f11 [4331611,96319] 0 2024-07-03T15:43:46.600 INFO:tasks.workunit.client.1.smithi148.stdout:1/147: mknod d0/d2/d6/d14/c2c 0 2024-07-03T15:43:46.603 INFO:tasks.workunit.client.1.smithi148.stdout:4/143: mkdir d0/d28/d2a 0 2024-07-03T15:43:46.605 INFO:tasks.workunit.client.1.smithi148.stdout:6/252: link d3/d1d/c26 d3/d1d/d28/c52 0 2024-07-03T15:43:46.607 INFO:tasks.workunit.client.1.smithi148.stdout:5/132: dread f4 [4194304,4194304] 0 2024-07-03T15:43:46.607 INFO:tasks.workunit.client.1.smithi148.stdout:5/133: creat d7/d13/f2a x:0 0 0 2024-07-03T15:43:46.609 INFO:tasks.workunit.client.1.smithi148.stdout:1/148: dread d0/f1f [0,4194304] 0 2024-07-03T15:43:46.609 INFO:tasks.workunit.client.1.smithi148.stdout:1/149: write d0/f24 [617321,129273] 0 2024-07-03T15:43:46.610 INFO:tasks.workunit.client.1.smithi148.stdout:7/219: symlink d5/d8/d21/d39/l43 0 2024-07-03T15:43:46.611 INFO:tasks.workunit.client.1.smithi148.stdout:3/147: dwrite dd/f15 [0,4194304] 0 2024-07-03T15:43:46.611 INFO:tasks.workunit.client.1.smithi148.stdout:9/325: dwrite d0/d49/d43/d19/f48 [0,4194304] 0 2024-07-03T15:43:46.613 INFO:tasks.workunit.client.1.smithi148.stdout:3/148: dread dd/df/f11 [0,4194304] 0 2024-07-03T15:43:46.613 INFO:tasks.workunit.client.1.smithi148.stdout:8/126: link d4/da/fb d4/d6/d18/d25/f27 0 2024-07-03T15:43:46.613 INFO:tasks.workunit.client.1.smithi148.stdout:8/127: readlink d4/d6/l9 0 2024-07-03T15:43:46.614 INFO:tasks.workunit.client.1.smithi148.stdout:4/144: mknod d0/d8/c2b 0 2024-07-03T15:43:46.617 INFO:tasks.workunit.client.1.smithi148.stdout:6/253: unlink d3/d1d/d28/d3e/c48 0 2024-07-03T15:43:46.621 INFO:tasks.workunit.client.1.smithi148.stdout:0/158: rename da/dc/d13/l1a to da/dc/d13/d14/l33 0 2024-07-03T15:43:46.622 INFO:tasks.workunit.client.1.smithi148.stdout:1/150: symlink d0/l2d 0 2024-07-03T15:43:46.626 INFO:tasks.workunit.client.1.smithi148.stdout:9/326: symlink d0/d49/d43/l6f 0 2024-07-03T15:43:46.629 INFO:tasks.workunit.client.1.smithi148.stdout:6/254: dread d3/d1d/d28/f2a [0,4194304] 0 2024-07-03T15:43:46.629 INFO:tasks.workunit.client.1.smithi148.stdout:6/255: chown d3/d1d/d28/d31/c47 6 1 2024-07-03T15:43:46.629 INFO:tasks.workunit.client.1.smithi148.stdout:4/145: rename d0/d8/d19/d24/f27 to d0/d8/d19/d24/f2c 0 2024-07-03T15:43:46.631 INFO:tasks.workunit.client.1.smithi148.stdout:8/128: rename d4/f11 to d4/da/f28 0 2024-07-03T15:43:46.631 INFO:tasks.workunit.client.1.smithi148.stdout:8/129: chown d4 1378184 1 2024-07-03T15:43:46.631 INFO:tasks.workunit.client.1.smithi148.stdout:8/130: chown d4/d6 2950605 1 2024-07-03T15:43:46.632 INFO:tasks.workunit.client.1.smithi148.stdout:0/159: creat da/dc/d13/d1d/f34 x:0 0 0 2024-07-03T15:43:46.632 INFO:tasks.workunit.client.1.smithi148.stdout:7/220: dwrite d5/d16/f34 [0,4194304] 0 2024-07-03T15:43:46.632 INFO:tasks.workunit.client.1.smithi148.stdout:7/221: read d5/f3b [2857148,97692] 0 2024-07-03T15:43:46.633 INFO:tasks.workunit.client.1.smithi148.stdout:1/151: rmdir d0/de 39 2024-07-03T15:43:46.634 INFO:tasks.workunit.client.1.smithi148.stdout:5/134: dwrite d7/f19 [4194304,4194304] 0 2024-07-03T15:43:46.636 INFO:tasks.workunit.client.1.smithi148.stdout:9/327: unlink d0/d49/l2d 0 2024-07-03T15:43:46.638 INFO:tasks.workunit.client.1.smithi148.stdout:6/256: mkdir d3/d1d/d28/d53 0 2024-07-03T15:43:46.638 INFO:tasks.workunit.client.1.smithi148.stdout:6/257: truncate d3/d1d/f4e 968098 0 2024-07-03T15:43:46.639 INFO:tasks.workunit.client.1.smithi148.stdout:4/146: rename d0/d8/ff to d0/d8/f2d 0 2024-07-03T15:43:46.641 INFO:tasks.workunit.client.1.smithi148.stdout:8/131: creat d4/da/f29 x:0 0 0 2024-07-03T15:43:46.641 INFO:tasks.workunit.client.1.smithi148.stdout:8/132: stat d4 0 2024-07-03T15:43:46.641 INFO:tasks.workunit.client.1.smithi148.stdout:8/133: creat d4/da/d14/d17/f2a x:0 0 0 2024-07-03T15:43:46.642 INFO:tasks.workunit.client.1.smithi148.stdout:5/135: write f3 [4122553,115133] 0 2024-07-03T15:43:46.642 INFO:tasks.workunit.client.1.smithi148.stdout:5/136: chown d7/da/df/d16 2044713790 1 2024-07-03T15:43:46.644 INFO:tasks.workunit.client.1.smithi148.stdout:7/222: rmdir d5/d16/d27 39 2024-07-03T15:43:46.648 INFO:tasks.workunit.client.1.smithi148.stdout:0/160: dwrite da/f11 [0,4194304] 0 2024-07-03T15:43:46.648 INFO:tasks.workunit.client.1.smithi148.stdout:6/258: stat d3/c3f 0 2024-07-03T15:43:46.649 INFO:tasks.workunit.client.1.smithi148.stdout:6/259: chown d3/d1d/d28/c36 106867931 1 2024-07-03T15:43:46.650 INFO:tasks.workunit.client.1.smithi148.stdout:4/147: write d0/f6 [1970198,61216] 0 2024-07-03T15:43:46.653 INFO:tasks.workunit.client.1.smithi148.stdout:5/137: creat d7/da/f2b x:0 0 0 2024-07-03T15:43:46.656 INFO:tasks.workunit.client.1.smithi148.stdout:7/223: symlink d5/l44 0 2024-07-03T15:43:46.658 INFO:tasks.workunit.client.1.smithi148.stdout:1/152: mkdir d0/d2/d2e 0 2024-07-03T15:43:46.658 INFO:tasks.workunit.client.1.smithi148.stdout:1/153: write d0/d2/d6/fa [550964,45874] 0 2024-07-03T15:43:46.661 INFO:tasks.workunit.client.1.smithi148.stdout:9/328: dwrite d0/dd/f1a [0,4194304] 0 2024-07-03T15:43:46.662 INFO:tasks.workunit.client.1.smithi148.stdout:8/134: dwrite d4/d6/f7 [0,4194304] 0 2024-07-03T15:43:46.662 INFO:tasks.workunit.client.1.smithi148.stdout:9/329: dread d0/d49/d43/f22 [0,4194304] 0 2024-07-03T15:43:46.662 INFO:tasks.workunit.client.1.smithi148.stdout:9/330: chown d0/dd/d25/d37/d40 4397 1 2024-07-03T15:43:46.664 INFO:tasks.workunit.client.1.smithi148.stdout:0/161: mkdir da/dc/d13/d14/d35 0 2024-07-03T15:43:46.666 INFO:tasks.workunit.client.1.smithi148.stdout:6/260: creat d3/d1d/d28/d31/d4c/f54 x:0 0 0 2024-07-03T15:43:46.667 INFO:tasks.workunit.client.1.smithi148.stdout:4/148: rmdir d0/d8/d19 39 2024-07-03T15:43:46.669 INFO:tasks.workunit.client.1.smithi148.stdout:5/138: link d7/da/df/d16/d1a/c22 d7/da/df/d16/c2c 0 2024-07-03T15:43:46.669 INFO:tasks.workunit.client.1.smithi148.stdout:5/139: rename d7/da/df/d1d to d7/da/df/d1d/d2d 22 2024-07-03T15:43:46.674 INFO:tasks.workunit.client.1.smithi148.stdout:9/331: mknod d0/d2f/c70 0 2024-07-03T15:43:46.674 INFO:tasks.workunit.client.1.smithi148.stdout:9/332: write d0/d18/f21 [5108252,54873] 0 2024-07-03T15:43:46.679 INFO:tasks.workunit.client.1.smithi148.stdout:0/162: truncate da/dc/f26 4404145 0 2024-07-03T15:43:46.679 INFO:tasks.workunit.client.1.smithi148.stdout:0/163: chown da/dc/d13/d14 42650341 1 2024-07-03T15:43:46.679 INFO:tasks.workunit.client.1.smithi148.stdout:0/164: truncate da/dc/d13/d1d/f34 579395 0 2024-07-03T15:43:46.679 INFO:tasks.workunit.client.1.smithi148.stdout:0/165: chown da/dc/d13/d14/d35 12448 1 2024-07-03T15:43:46.681 INFO:tasks.workunit.client.1.smithi148.stdout:6/261: creat d3/f55 x:0 0 0 2024-07-03T15:43:46.683 INFO:tasks.workunit.client.1.smithi148.stdout:8/135: dwrite d4/da/d14/d17/f26 [0,4194304] 0 2024-07-03T15:43:46.683 INFO:tasks.workunit.client.1.smithi148.stdout:8/136: write d4/da/d14/d17/f2a [1015364,106164] 0 2024-07-03T15:43:46.683 INFO:tasks.workunit.client.1.smithi148.stdout:8/137: creat d4/d6/d18/d20/f2b x:0 0 0 2024-07-03T15:43:46.683 INFO:tasks.workunit.client.1.smithi148.stdout:4/149: getdents d0/d8/d1f 0 2024-07-03T15:43:46.683 INFO:tasks.workunit.client.1.smithi148.stdout:4/150: creat d0/d8/d19/d24/f2e x:0 0 0 2024-07-03T15:43:46.685 INFO:tasks.workunit.client.1.smithi148.stdout:5/140: truncate d7/da/fb 2258961 0 2024-07-03T15:43:46.686 INFO:tasks.workunit.client.1.smithi148.stdout:9/333: mknod d0/dd/d29/c71 0 2024-07-03T15:43:46.689 INFO:tasks.workunit.client.1.smithi148.stdout:0/166: mkdir da/d36 0 2024-07-03T15:43:46.693 INFO:tasks.workunit.client.1.smithi148.stdout:0/167: write da/d17/f1e [1521000,73502] 0 2024-07-03T15:43:46.697 INFO:tasks.workunit.client.1.smithi148.stdout:8/138: rmdir d4/da/d14 39 2024-07-03T15:43:46.699 INFO:tasks.workunit.client.1.smithi148.stdout:8/139: dread d4/d6/fc [0,4194304] 0 2024-07-03T15:43:46.700 INFO:tasks.workunit.client.1.smithi148.stdout:4/151: mkdir d0/d8/d22/d2f 0 2024-07-03T15:43:46.703 INFO:tasks.workunit.client.1.smithi148.stdout:5/141: creat d7/da/df/d1d/f2e x:0 0 0 2024-07-03T15:43:46.704 INFO:tasks.workunit.client.1.smithi148.stdout:5/142: dread - d7/d13/f28 zero size 2024-07-03T15:43:46.704 INFO:tasks.workunit.client.1.smithi148.stdout:5/143: read f3 [2389182,124566] 0 2024-07-03T15:43:46.705 INFO:tasks.workunit.client.1.smithi148.stdout:0/168: creat da/d17/d2a/d2f/f37 x:0 0 0 2024-07-03T15:43:46.706 INFO:tasks.workunit.client.1.smithi148.stdout:8/140: mknod d4/d6/d18/c2c 0 2024-07-03T15:43:46.706 INFO:tasks.workunit.client.1.smithi148.stdout:8/141: rename d4 to d4/da/d2d 22 2024-07-03T15:43:46.710 INFO:tasks.workunit.client.1.smithi148.stdout:5/144: mkdir d7/da/df/d16/d2f 0 2024-07-03T15:43:46.710 INFO:tasks.workunit.client.1.smithi148.stdout:5/145: dread - d7/d13/f28 zero size 2024-07-03T15:43:46.710 INFO:tasks.workunit.client.1.smithi148.stdout:5/146: chown d7/da/l14 873641213 1 2024-07-03T15:43:46.711 INFO:tasks.workunit.client.1.smithi148.stdout:3/149: fsync dd/f22 0 2024-07-03T15:43:46.711 INFO:tasks.workunit.client.1.smithi148.stdout:3/150: chown dd/d1c/f1d 253566 1 2024-07-03T15:43:46.711 INFO:tasks.workunit.client.1.smithi148.stdout:3/151: write dd/f22 [725626,104859] 0 2024-07-03T15:43:46.711 INFO:tasks.workunit.client.1.smithi148.stdout:3/152: creat dd/f23 x:0 0 0 2024-07-03T15:43:46.711 INFO:tasks.workunit.client.1.smithi148.stdout:9/334: dwrite d0/d18/f26 [4194304,4194304] 0 2024-07-03T15:43:46.712 INFO:tasks.workunit.client.1.smithi148.stdout:8/142: mknod d4/d6/d18/d20/c2e 0 2024-07-03T15:43:46.716 INFO:tasks.workunit.client.1.smithi148.stdout:0/169: write da/dc/d24/f29 [1710961,46870] 0 2024-07-03T15:43:46.720 INFO:tasks.workunit.client.1.smithi148.stdout:9/335: mkdir d0/dd/d25/d37/d40/d72 0 2024-07-03T15:43:46.723 INFO:tasks.workunit.client.1.smithi148.stdout:8/143: symlink d4/da/d14/d17/l2f 0 2024-07-03T15:43:46.725 INFO:tasks.workunit.client.1.smithi148.stdout:9/336: link d0/d49/d43/d19/f1e d0/d49/f73 0 2024-07-03T15:43:46.726 INFO:tasks.workunit.client.1.smithi148.stdout:7/224: dread d5/f6 [0,4194304] 0 2024-07-03T15:43:46.727 INFO:tasks.workunit.client.1.smithi148.stdout:9/337: dread d0/d49/f73 [0,4194304] 0 2024-07-03T15:43:46.727 INFO:tasks.workunit.client.1.smithi148.stdout:9/338: creat d0/d2f/f74 x:0 0 0 2024-07-03T15:43:46.727 INFO:tasks.workunit.client.1.smithi148.stdout:9/339: truncate d0/dd/f1a 5039464 0 2024-07-03T15:43:46.728 INFO:tasks.workunit.client.1.smithi148.stdout:8/144: link d4/d6/f8 d4/da/d14/d17/f30 0 2024-07-03T15:43:46.731 INFO:tasks.workunit.client.1.smithi148.stdout:7/225: creat d5/d8/d21/d31/f45 x:0 0 0 2024-07-03T15:43:46.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:46 smithi148 ceph-mon[36056]: from='client.14612 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:43:46.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:46 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/83022458' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-03T15:43:46.733 INFO:tasks.workunit.client.1.smithi148.stdout:9/340: getdents d0/d18/d24/d47 0 2024-07-03T15:43:46.735 INFO:tasks.workunit.client.1.smithi148.stdout:8/145: symlink d4/da/d14/l31 0 2024-07-03T15:43:46.735 INFO:tasks.workunit.client.1.smithi148.stdout:8/146: dread - d4/da/f29 zero size 2024-07-03T15:43:46.736 INFO:tasks.workunit.client.1.smithi148.stdout:3/153: dwrite f5 [0,4194304] 0 2024-07-03T15:43:46.736 INFO:tasks.workunit.client.1.smithi148.stdout:3/154: write f0 [414310,109661] 0 2024-07-03T15:43:46.737 INFO:tasks.workunit.client.1.smithi148.stdout:7/226: symlink d5/d16/l46 0 2024-07-03T15:43:46.752 INFO:tasks.workunit.client.1.smithi148.stdout:3/155: dwrite dd/df/f18 [0,4194304] 0 2024-07-03T15:43:46.752 INFO:tasks.workunit.client.1.smithi148.stdout:3/156: fdatasync dd/df/f18 0 2024-07-03T15:43:46.752 INFO:tasks.workunit.client.1.smithi148.stdout:3/157: write dd/f22 [1708942,60069] 0 2024-07-03T15:43:46.754 INFO:tasks.workunit.client.1.smithi148.stdout:7/227: mkdir d5/d8/d21/d47 0 2024-07-03T15:43:46.754 INFO:tasks.workunit.client.1.smithi148.stdout:7/228: creat d5/d16/d1c/d33/f48 x:0 0 0 2024-07-03T15:43:46.754 INFO:tasks.workunit.client.1.smithi148.stdout:7/229: read d5/d8/d21/f32 [627888,81433] 0 2024-07-03T15:43:46.760 INFO:tasks.workunit.client.1.smithi148.stdout:3/158: read dd/df/f18 [123291,120475] 0 2024-07-03T15:43:46.771 INFO:tasks.workunit.client.1.smithi148.stdout:7/230: dwrite d5/d8/f26 [0,4194304] 0 2024-07-03T15:43:46.772 INFO:tasks.workunit.client.1.smithi148.stdout:3/159: mknod dd/d1c/c24 0 2024-07-03T15:43:46.772 INFO:tasks.workunit.client.1.smithi148.stdout:3/160: readlink l7 0 2024-07-03T15:43:46.789 INFO:tasks.workunit.client.1.smithi148.stdout:7/231: dwrite d5/d16/f34 [0,4194304] 0 2024-07-03T15:43:46.789 INFO:tasks.workunit.client.1.smithi148.stdout:7/232: truncate d5/d16/d1c/f2d 143496 0 2024-07-03T15:43:46.789 INFO:tasks.workunit.client.1.smithi148.stdout:7/233: chown d5/d16/d27/f28 3 1 2024-07-03T15:43:46.798 INFO:tasks.workunit.client.1.smithi148.stdout:7/234: write d5/d8/f26 [3596514,118069] 0 2024-07-03T15:43:46.798 INFO:tasks.workunit.client.1.smithi148.stdout:7/235: write d5/d8/f26 [661374,2450] 0 2024-07-03T15:43:46.799 INFO:tasks.workunit.client.1.smithi148.stdout:7/236: readlink d5/d16/l46 0 2024-07-03T15:43:46.802 INFO:tasks.workunit.client.1.smithi148.stdout:7/237: rename d5/f7 to d5/d16/d1c/d23/f49 0 2024-07-03T15:43:46.803 INFO:tasks.workunit.client.1.smithi148.stdout:7/238: read d5/d16/f37 [2040221,47148] 0 2024-07-03T15:43:46.803 INFO:tasks.workunit.client.1.smithi148.stdout:7/239: truncate d5/d8/d21/d31/f45 508837 0 2024-07-03T15:43:46.808 INFO:tasks.workunit.client.1.smithi148.stdout:7/240: mkdir d5/d16/d1c/d4a 0 2024-07-03T15:43:46.808 INFO:tasks.workunit.client.1.smithi148.stdout:7/241: chown d5/d16/f37 7832565 1 2024-07-03T15:43:46.809 INFO:tasks.workunit.client.1.smithi148.stdout:7/242: creat d5/d16/f4b x:0 0 0 2024-07-03T15:43:46.825 INFO:tasks.workunit.client.1.smithi148.stdout:9/341: rmdir d0/dd/d25 39 2024-07-03T15:43:46.825 INFO:tasks.workunit.client.1.smithi148.stdout:9/342: write d0/dd/d25/d6a/f6e [548987,106963] 0 2024-07-03T15:43:46.828 INFO:tasks.workunit.client.1.smithi148.stdout:9/343: truncate d0/d18/f26 674139 0 2024-07-03T15:43:46.830 INFO:tasks.workunit.client.1.smithi148.stdout:9/344: mknod d0/dd/d46/c75 0 2024-07-03T15:43:46.832 INFO:tasks.workunit.client.1.smithi148.stdout:8/147: getdents d4/da/d14/d17 0 2024-07-03T15:43:46.834 INFO:tasks.workunit.client.1.smithi148.stdout:8/148: mkdir d4/da/d14/d1a/d32 0 2024-07-03T15:43:46.836 INFO:tasks.workunit.client.1.smithi148.stdout:8/149: dread d4/d6/d18/f1e [0,4194304] 0 2024-07-03T15:43:46.838 INFO:tasks.workunit.client.1.smithi148.stdout:8/150: symlink d4/d6/d18/l33 0 2024-07-03T15:43:46.840 INFO:tasks.workunit.client.1.smithi148.stdout:8/151: rename d4/d6/fc to d4/d6/f34 0 2024-07-03T15:43:46.845 INFO:tasks.workunit.client.1.smithi148.stdout:9/345: dwrite d0/dd/f16 [0,4194304] 0 2024-07-03T15:43:46.845 INFO:tasks.workunit.client.1.smithi148.stdout:9/346: read - d0/dd/d29/f5e zero size 2024-07-03T15:43:46.845 INFO:tasks.workunit.client.1.smithi148.stdout:9/347: write d0/dd/d25/d6a/f6e [696917,66332] 0 2024-07-03T15:43:46.850 INFO:tasks.workunit.client.1.smithi148.stdout:9/348: truncate d0/dd/d11/f35 765060 0 2024-07-03T15:43:46.853 INFO:tasks.workunit.client.1.smithi148.stdout:7/243: rename d5/d8 to d5/d16/d1c/d33/d4c 0 2024-07-03T15:43:46.853 INFO:tasks.workunit.client.1.smithi148.stdout:7/244: write d5/d16/d1c/d33/d4c/d21/f3c [234579,49646] 0 2024-07-03T15:43:46.854 INFO:tasks.workunit.client.1.smithi148.stdout:8/152: dwrite d4/da/d14/d17/f30 [0,4194304] 0 2024-07-03T15:43:46.855 INFO:tasks.workunit.client.1.smithi148.stdout:7/245: creat d5/d16/f4d x:0 0 0 2024-07-03T15:43:46.855 INFO:tasks.workunit.client.1.smithi148.stdout:7/246: dread - d5/d16/f4b zero size 2024-07-03T15:43:46.856 INFO:tasks.workunit.client.1.smithi148.stdout:8/153: symlink d4/l35 0 2024-07-03T15:43:46.858 INFO:tasks.workunit.client.1.smithi148.stdout:7/247: mknod d5/c4e 0 2024-07-03T15:43:46.861 INFO:tasks.workunit.client.1.smithi148.stdout:3/161: getdents dd/d1c 0 2024-07-03T15:43:46.861 INFO:tasks.workunit.client.1.smithi148.stdout:3/162: readlink dd/le 0 2024-07-03T15:43:46.864 INFO:tasks.workunit.client.1.smithi148.stdout:3/163: creat dd/d1c/f25 x:0 0 0 2024-07-03T15:43:46.864 INFO:tasks.workunit.client.1.smithi148.stdout:3/164: chown dd/df 2733046 1 2024-07-03T15:43:46.864 INFO:tasks.workunit.client.1.smithi148.stdout:3/165: creat dd/f26 x:0 0 0 2024-07-03T15:43:46.867 INFO:tasks.workunit.client.1.smithi148.stdout:8/154: write d4/da/d14/d17/f30 [1223371,47099] 0 2024-07-03T15:43:46.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:46 smithi028 ceph-mon[29148]: from='client.14612 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:43:46.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:46 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/83022458' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-03T15:43:46.876 INFO:tasks.workunit.client.1.smithi148.stdout:3/166: write fa [2607781,65008] 0 2024-07-03T15:43:46.881 INFO:tasks.workunit.client.1.smithi148.stdout:3/167: symlink dd/df/l27 0 2024-07-03T15:43:46.882 INFO:tasks.workunit.client.1.smithi148.stdout:3/168: link f5 dd/f28 0 2024-07-03T15:43:46.882 INFO:tasks.workunit.client.1.smithi148.stdout:3/169: chown dd/f26 264259 1 2024-07-03T15:43:46.884 INFO:tasks.workunit.client.1.smithi148.stdout:3/170: symlink dd/l29 0 2024-07-03T15:43:46.908 INFO:tasks.workunit.client.1.smithi148.stdout:3/171: dwrite f5 [0,4194304] 0 2024-07-03T15:43:46.908 INFO:tasks.workunit.client.1.smithi148.stdout:3/172: dread - dd/f23 zero size 2024-07-03T15:43:46.911 INFO:tasks.workunit.client.1.smithi148.stdout:3/173: write dd/d1c/f1d [17191,6262] 0 2024-07-03T15:43:46.945 INFO:tasks.workunit.client.1.smithi148.stdout:9/349: dwrite d0/dd/d11/f35 [0,4194304] 0 2024-07-03T15:43:46.946 INFO:tasks.workunit.client.1.smithi148.stdout:3/174: rmdir dd/d1c 39 2024-07-03T15:43:46.952 INFO:tasks.workunit.client.1.smithi148.stdout:3/175: rename f0 to dd/f2a 0 2024-07-03T15:43:46.961 INFO:tasks.workunit.client.1.smithi148.stdout:9/350: dwrite d0/dd/ff [0,4194304] 0 2024-07-03T15:43:46.970 INFO:tasks.workunit.client.1.smithi148.stdout:9/351: mknod d0/dd/d25/d37/d40/d72/c76 0 2024-07-03T15:43:46.970 INFO:tasks.workunit.client.1.smithi148.stdout:9/352: readlink d0/d18/d24/l33 0 2024-07-03T15:43:46.972 INFO:tasks.workunit.client.1.smithi148.stdout:3/176: dwrite fa [0,4194304] 0 2024-07-03T15:43:46.975 INFO:tasks.workunit.client.1.smithi148.stdout:5/147: truncate d7/da/fb 467287 0 2024-07-03T15:43:46.975 INFO:tasks.workunit.client.1.smithi148.stdout:5/148: fdatasync d7/da/f2b 0 2024-07-03T15:43:46.980 INFO:tasks.workunit.client.1.smithi148.stdout:5/149: link d7/c9 d7/c30 0 2024-07-03T15:43:46.982 INFO:tasks.workunit.client.1.smithi148.stdout:5/150: rename d7/da/df/d16/d1a/c26 to d7/da/d21/c31 0 2024-07-03T15:43:46.982 INFO:tasks.workunit.client.1.smithi148.stdout:5/151: stat d7 0 2024-07-03T15:43:46.982 INFO:tasks.workunit.client.1.smithi148.stdout:5/152: chown d7 11 1 2024-07-03T15:43:46.998 INFO:tasks.workunit.client.1.smithi148.stdout:9/353: dwrite d0/d49/d43/f22 [0,4194304] 0 2024-07-03T15:43:47.001 INFO:tasks.workunit.client.1.smithi148.stdout:3/177: dwrite dd/f23 [0,4194304] 0 2024-07-03T15:43:47.001 INFO:tasks.workunit.client.1.smithi148.stdout:3/178: write dd/f21 [603307,39728] 0 2024-07-03T15:43:47.001 INFO:tasks.workunit.client.1.smithi148.stdout:3/179: creat dd/f2b x:0 0 0 2024-07-03T15:43:47.001 INFO:tasks.workunit.client.1.smithi148.stdout:5/153: dwrite d7/d13/f28 [0,4194304] 0 2024-07-03T15:43:47.007 INFO:tasks.workunit.client.1.smithi148.stdout:9/354: dread d0/dd/d11/f35 [0,4194304] 0 2024-07-03T15:43:47.007 INFO:tasks.workunit.client.1.smithi148.stdout:9/355: truncate d0/d18/f26 1293238 0 2024-07-03T15:43:47.007 INFO:tasks.workunit.client.1.smithi148.stdout:9/356: chown d0/dd/d11/f66 727 1 2024-07-03T15:43:47.019 INFO:tasks.workunit.client.1.smithi148.stdout:9/357: dread d0/d18/d24/f30 [0,4194304] 0 2024-07-03T15:43:47.021 INFO:tasks.workunit.client.1.smithi148.stdout:5/154: dwrite d7/da/f2b [0,4194304] 0 2024-07-03T15:43:47.021 INFO:tasks.workunit.client.1.smithi148.stdout:5/155: truncate d7/da/df/d1d/f2e 644299 0 2024-07-03T15:43:47.021 INFO:tasks.workunit.client.1.smithi148.stdout:5/156: fdatasync f4 0 2024-07-03T15:43:47.021 INFO:tasks.workunit.client.1.smithi148.stdout:5/157: chown d7/da/df/d16 95971553 1 2024-07-03T15:43:47.022 INFO:tasks.workunit.client.1.smithi148.stdout:9/358: mkdir d0/dd/d25/d77 0 2024-07-03T15:43:47.026 INFO:tasks.workunit.client.1.smithi148.stdout:5/158: mknod d7/da/df/d1d/c32 0 2024-07-03T15:43:47.027 INFO:tasks.workunit.client.1.smithi148.stdout:3/180: dwrite dd/f28 [4194304,4194304] 0 2024-07-03T15:43:47.027 INFO:tasks.workunit.client.1.smithi148.stdout:3/181: creat dd/df/f2c x:0 0 0 2024-07-03T15:43:47.029 INFO:tasks.workunit.client.1.smithi148.stdout:9/359: rename d0/d18/d24/d47/d4d/l57 to d0/dd/d29/l78 0 2024-07-03T15:43:47.034 INFO:tasks.workunit.client.1.smithi148.stdout:3/182: unlink dd/f26 0 2024-07-03T15:43:47.035 INFO:tasks.workunit.client.1.smithi148.stdout:5/159: read d7/f19 [8275440,119999] 0 2024-07-03T15:43:47.038 INFO:tasks.workunit.client.1.smithi148.stdout:9/360: truncate d0/dd/d25/f3c 1293581 0 2024-07-03T15:43:47.038 INFO:tasks.workunit.client.1.smithi148.stdout:9/361: chown d0/dd/ca 17270847 1 2024-07-03T15:43:47.040 INFO:tasks.workunit.client.1.smithi148.stdout:5/160: rename d7/d13/f28 to d7/da/df/f33 0 2024-07-03T15:43:47.049 INFO:tasks.workunit.client.1.smithi148.stdout:9/362: write d0/dd/d11/f35 [399565,35876] 0 2024-07-03T15:43:47.053 INFO:tasks.workunit.client.1.smithi148.stdout:9/363: truncate d0/f6 4748438 0 2024-07-03T15:43:47.053 INFO:tasks.workunit.client.1.smithi148.stdout:9/364: creat d0/dd/d11/f79 x:0 0 0 2024-07-03T15:43:47.054 INFO:tasks.workunit.client.1.smithi148.stdout:9/365: write d0/d49/f73 [842542,45564] 0 2024-07-03T15:43:47.054 INFO:tasks.workunit.client.1.smithi148.stdout:9/366: write d0/dd/d25/d6a/f6e [879188,67109] 0 2024-07-03T15:43:47.054 INFO:tasks.workunit.client.1.smithi148.stdout:9/367: write d0/dd/fe [1548472,52363] 0 2024-07-03T15:43:47.054 INFO:tasks.workunit.client.1.smithi148.stdout:9/368: fsync d0/dd/f2b 0 2024-07-03T15:43:47.054 INFO:tasks.workunit.client.1.smithi148.stdout:9/369: write d0/d49/f4e [2859257,46344] 0 2024-07-03T15:43:47.057 INFO:tasks.workunit.client.1.smithi148.stdout:9/370: mkdir d0/d49/d43/d19/d7a 0 2024-07-03T15:43:47.057 INFO:tasks.workunit.client.1.smithi148.stdout:9/371: stat d0/dd/d11/f66 0 2024-07-03T15:43:47.059 INFO:tasks.workunit.client.1.smithi148.stdout:9/372: link d0/l10 d0/dd/d29/l7b 0 2024-07-03T15:43:47.061 INFO:tasks.workunit.client.1.smithi148.stdout:9/373: mknod d0/dd/d25/d39/c7c 0 2024-07-03T15:43:47.070 INFO:tasks.workunit.client.1.smithi148.stdout:3/183: dwrite dd/f28 [4194304,4194304] 0 2024-07-03T15:43:47.074 INFO:tasks.workunit.client.1.smithi148.stdout:3/184: truncate dd/f15 655674 0 2024-07-03T15:43:47.077 INFO:tasks.workunit.client.1.smithi148.stdout:3/185: mknod dd/d1c/c2d 0 2024-07-03T15:43:47.078 INFO:tasks.workunit.client.1.smithi148.stdout:7/248: sync 2024-07-03T15:43:47.078 INFO:tasks.workunit.client.1.smithi148.stdout:0/170: sync 2024-07-03T15:43:47.078 INFO:tasks.workunit.client.1.smithi148.stdout:6/262: sync 2024-07-03T15:43:47.078 INFO:tasks.workunit.client.1.smithi148.stdout:2/266: sync 2024-07-03T15:43:47.078 INFO:tasks.workunit.client.1.smithi148.stdout:1/154: sync 2024-07-03T15:43:47.078 INFO:tasks.workunit.client.1.smithi148.stdout:4/152: sync 2024-07-03T15:43:47.078 INFO:tasks.workunit.client.1.smithi148.stdout:5/161: dwrite f3 [0,4194304] 0 2024-07-03T15:43:47.078 INFO:tasks.workunit.client.1.smithi148.stdout:9/374: dwrite d0/dd/f1f [0,4194304] 0 2024-07-03T15:43:47.079 INFO:tasks.workunit.client.1.smithi148.stdout:1/155: dread d0/d2/d6/f19 [0,4194304] 0 2024-07-03T15:43:47.079 INFO:tasks.workunit.client.1.smithi148.stdout:3/186: truncate dd/d1c/f1d 543869 0 2024-07-03T15:43:47.081 INFO:tasks.workunit.client.1.smithi148.stdout:0/171: mkdir da/d38 0 2024-07-03T15:43:47.085 INFO:tasks.workunit.client.1.smithi148.stdout:2/267: symlink d5/d14/d22/d2b/d13/d2d/l5e 0 2024-07-03T15:43:47.085 INFO:tasks.workunit.client.1.smithi148.stdout:7/249: creat d5/d16/d1c/d23/f4f x:0 0 0 2024-07-03T15:43:47.087 INFO:tasks.workunit.client.1.smithi148.stdout:4/153: rename d0/f3 to d0/f30 0 2024-07-03T15:43:47.088 INFO:tasks.workunit.client.1.smithi148.stdout:5/162: creat d7/da/df/d16/d1a/f34 x:0 0 0 2024-07-03T15:43:47.089 INFO:tasks.workunit.client.1.smithi148.stdout:9/375: symlink d0/dd/d25/d6a/l7d 0 2024-07-03T15:43:47.091 INFO:tasks.workunit.client.1.smithi148.stdout:6/263: read d3/d1d/d28/d31/f43 [3518863,19433] 0 2024-07-03T15:43:47.091 INFO:tasks.workunit.client.1.smithi148.stdout:6/264: creat d3/f56 x:0 0 0 2024-07-03T15:43:47.091 INFO:tasks.workunit.client.1.smithi148.stdout:6/265: creat d3/d1d/d28/d31/d4c/f57 x:0 0 0 2024-07-03T15:43:47.092 INFO:tasks.workunit.client.1.smithi148.stdout:1/156: mkdir d0/de/d2f 0 2024-07-03T15:43:47.092 INFO:tasks.workunit.client.1.smithi148.stdout:1/157: fsync d0/f8 0 2024-07-03T15:43:47.092 INFO:tasks.workunit.client.1.smithi148.stdout:1/158: truncate d0/d2/d6/f19 1565342 0 2024-07-03T15:43:47.093 INFO:tasks.workunit.client.1.smithi148.stdout:2/268: write d5/d14/d22/d2b/d13/d41/f49 [3444530,73948] 0 2024-07-03T15:43:47.093 INFO:tasks.workunit.client.1.smithi148.stdout:2/269: fdatasync d5/d14/d22/d2b/d13/d2d/f3b 0 2024-07-03T15:43:47.096 INFO:tasks.workunit.client.1.smithi148.stdout:0/172: rmdir da/dc/d13/d14 39 2024-07-03T15:43:47.098 INFO:tasks.workunit.client.1.smithi148.stdout:0/173: dread da/dc/ff [0,4194304] 0 2024-07-03T15:43:47.098 INFO:tasks.workunit.client.1.smithi148.stdout:4/154: creat d0/d28/f31 x:0 0 0 2024-07-03T15:43:47.101 INFO:tasks.workunit.client.1.smithi148.stdout:7/250: dwrite d5/d16/d1c/d33/d4c/f26 [0,4194304] 0 2024-07-03T15:43:47.103 INFO:tasks.workunit.client.1.smithi148.stdout:5/163: creat d7/da/df/d16/d2f/f35 x:0 0 0 2024-07-03T15:43:47.103 INFO:tasks.workunit.client.1.smithi148.stdout:5/164: truncate d7/da/df/d16/f1b 876309 0 2024-07-03T15:43:47.103 INFO:tasks.workunit.client.1.smithi148.stdout:5/165: read - d7/da/df/d16/d1a/f34 zero size 2024-07-03T15:43:47.103 INFO:tasks.workunit.client.1.smithi148.stdout:5/166: fdatasync d7/da/df/d16/f1b 0 2024-07-03T15:43:47.103 INFO:tasks.workunit.client.1.smithi148.stdout:0/174: dread da/dc/d24/f29 [4194304,4194304] 0 2024-07-03T15:43:47.104 INFO:tasks.workunit.client.1.smithi148.stdout:9/376: mknod d0/dd/d25/d59/c7e 0 2024-07-03T15:43:47.104 INFO:tasks.workunit.client.1.smithi148.stdout:9/377: creat d0/d2f/f7f x:0 0 0 2024-07-03T15:43:47.104 INFO:tasks.workunit.client.1.smithi148.stdout:8/155: sync 2024-07-03T15:43:47.104 INFO:tasks.workunit.client.1.smithi148.stdout:8/156: chown d4/l35 40 1 2024-07-03T15:43:47.110 INFO:tasks.workunit.client.1.smithi148.stdout:6/266: creat d3/d1d/d28/d3e/f58 x:0 0 0 2024-07-03T15:43:47.113 INFO:tasks.workunit.client.1.smithi148.stdout:0/175: write da/dc/f26 [1140929,26884] 0 2024-07-03T15:43:47.113 INFO:tasks.workunit.client.1.smithi148.stdout:0/176: stat da/d17/d2a/d2f/f37 0 2024-07-03T15:43:47.113 INFO:tasks.workunit.client.1.smithi148.stdout:2/270: getdents d5/d14/d22/d2b/d13 0 2024-07-03T15:43:47.123 INFO:tasks.workunit.client.1.smithi148.stdout:7/251: dwrite d5/d16/f34 [4194304,4194304] 0 2024-07-03T15:43:47.124 INFO:tasks.workunit.client.1.smithi148.stdout:8/157: dwrite f2 [8388608,4194304] 0 2024-07-03T15:43:47.126 INFO:tasks.workunit.client.1.smithi148.stdout:5/167: mknod d7/da/df/d1d/c36 0 2024-07-03T15:43:47.139 INFO:tasks.workunit.client.1.smithi148.stdout:8/158: dwrite d4/d6/f34 [0,4194304] 0 2024-07-03T15:43:47.139 INFO:tasks.workunit.client.1.smithi148.stdout:8/159: fdatasync d4/da/f29 0 2024-07-03T15:43:47.145 INFO:tasks.workunit.client.1.smithi148.stdout:5/168: dwrite d7/da/df/d16/f20 [0,4194304] 0 2024-07-03T15:43:47.145 INFO:tasks.workunit.client.1.smithi148.stdout:5/169: chown f4 389 1 2024-07-03T15:43:47.146 INFO:tasks.workunit.client.1.smithi148.stdout:9/378: link d0/dd/d25/d39/c7c d0/dd/d25/d37/d40/c80 0 2024-07-03T15:43:47.146 INFO:tasks.workunit.client.1.smithi148.stdout:9/379: read - d0/dd/d25/d37/d40/f62 zero size 2024-07-03T15:43:47.146 INFO:tasks.workunit.client.1.smithi148.stdout:9/380: truncate d0/dd/d25/d6a/f6e 1587524 0 2024-07-03T15:43:47.149 INFO:tasks.workunit.client.1.smithi148.stdout:6/267: creat d3/d1d/d28/d53/f59 x:0 0 0 2024-07-03T15:43:47.151 INFO:tasks.workunit.client.1.smithi148.stdout:0/177: mkdir da/d17/d39 0 2024-07-03T15:43:47.153 INFO:tasks.workunit.client.1.smithi148.stdout:2/271: mkdir d5/d5f 0 2024-07-03T15:43:47.157 INFO:tasks.workunit.client.1.smithi148.stdout:3/187: rename dd/df to dd/d1c/d2e 0 2024-07-03T15:43:47.158 INFO:tasks.workunit.client.1.smithi148.stdout:8/160: dwrite d4/d6/d18/d20/f2b [0,4194304] 0 2024-07-03T15:43:47.158 INFO:tasks.workunit.client.1.smithi148.stdout:8/161: chown d4/da/l1c 1158 1 2024-07-03T15:43:47.158 INFO:tasks.workunit.client.1.smithi148.stdout:8/162: write d4/d6/f8 [533313,109764] 0 2024-07-03T15:43:47.162 INFO:tasks.workunit.client.1.smithi148.stdout:5/170: dwrite d7/da/df/d16/d1a/f34 [0,4194304] 0 2024-07-03T15:43:47.165 INFO:tasks.workunit.client.1.smithi148.stdout:6/268: truncate d3/d1d/d28/f2e 270528 0 2024-07-03T15:43:47.166 INFO:tasks.workunit.client.1.smithi148.stdout:6/269: dread d3/f1e [0,4194304] 0 2024-07-03T15:43:47.177 INFO:tasks.workunit.client.1.smithi148.stdout:1/159: rename d0/d2/d11/l13 to d0/d2/d2e/l30 0 2024-07-03T15:43:47.179 INFO:tasks.workunit.client.1.smithi148.stdout:0/178: dwrite da/dc/d13/d14/f1f [0,4194304] 0 2024-07-03T15:43:47.180 INFO:tasks.workunit.client.1.smithi148.stdout:3/188: symlink dd/d1c/l2f 0 2024-07-03T15:43:47.184 INFO:tasks.workunit.client.1.smithi148.stdout:3/189: dread dd/f2a [0,4194304] 0 2024-07-03T15:43:47.184 INFO:tasks.workunit.client.1.smithi148.stdout:3/190: chown dd/d1c/c24 1092786 1 2024-07-03T15:43:47.185 INFO:tasks.workunit.client.1.smithi148.stdout:8/163: truncate d4/da/fb 456910 0 2024-07-03T15:43:47.185 INFO:tasks.workunit.client.1.smithi148.stdout:8/164: readlink d4/l35 0 2024-07-03T15:43:47.185 INFO:tasks.workunit.client.1.smithi148.stdout:8/165: read d4/d6/f8 [4002023,117173] 0 2024-07-03T15:43:47.189 INFO:tasks.workunit.client.1.smithi148.stdout:0/179: dread da/dc/f19 [0,4194304] 0 2024-07-03T15:43:47.189 INFO:tasks.workunit.client.1.smithi148.stdout:0/180: creat da/f3a x:0 0 0 2024-07-03T15:43:47.189 INFO:tasks.workunit.client.1.smithi148.stdout:5/171: mknod d7/da/df/c37 0 2024-07-03T15:43:47.191 INFO:tasks.workunit.client.1.smithi148.stdout:6/270: mknod d3/d50/c5a 0 2024-07-03T15:43:47.195 INFO:tasks.workunit.client.1.smithi148.stdout:7/252: rename d5/l44 to d5/d16/d27/l50 0 2024-07-03T15:43:47.197 INFO:tasks.workunit.client.1.smithi148.stdout:3/191: dwrite dd/d1c/d2e/f13 [0,4194304] 0 2024-07-03T15:43:47.198 INFO:tasks.workunit.client.1.smithi148.stdout:3/192: write dd/d1c/d2e/f18 [1114921,13828] 0 2024-07-03T15:43:47.204 INFO:tasks.workunit.client.1.smithi148.stdout:8/166: mknod d4/d6/c36 0 2024-07-03T15:43:47.210 INFO:tasks.workunit.client.1.smithi148.stdout:3/193: read dd/f12 [3293026,79807] 0 2024-07-03T15:43:47.210 INFO:tasks.workunit.client.1.smithi148.stdout:3/194: fsync dd/f21 0 2024-07-03T15:43:47.211 INFO:tasks.workunit.client.1.smithi148.stdout:5/172: rmdir d7/d15 39 2024-07-03T15:43:47.211 INFO:tasks.workunit.client.1.smithi148.stdout:5/173: fsync d7/da/fb 0 2024-07-03T15:43:47.211 INFO:tasks.workunit.client.1.smithi148.stdout:5/174: creat d7/da/df/d16/d2f/f38 x:0 0 0 2024-07-03T15:43:47.212 INFO:tasks.workunit.client.1.smithi148.stdout:6/271: mkdir d3/d5b 0 2024-07-03T15:43:47.214 INFO:tasks.workunit.client.1.smithi148.stdout:2/272: rename d5/d14/d22/d4c to d5/d14/d22/d60 0 2024-07-03T15:43:47.214 INFO:tasks.workunit.client.1.smithi148.stdout:2/273: fdatasync d5/d14/d22/d2b/d13/d41/f4b 0 2024-07-03T15:43:47.214 INFO:tasks.workunit.client.1.smithi148.stdout:2/274: write d5/d14/d22/d2b/f1e [764586,30753] 0 2024-07-03T15:43:47.218 INFO:tasks.workunit.client.1.smithi148.stdout:3/195: write fa [245219,889] 0 2024-07-03T15:43:47.218 INFO:tasks.workunit.client.1.smithi148.stdout:1/160: dwrite d0/d2/fd [0,4194304] 0 2024-07-03T15:43:47.218 INFO:tasks.workunit.client.1.smithi148.stdout:7/253: dwrite d5/f3b [0,4194304] 0 2024-07-03T15:43:47.221 INFO:tasks.workunit.client.1.smithi148.stdout:8/167: rename d4/d6/d18/f1e to d4/da/d14/d1a/f37 0 2024-07-03T15:43:47.222 INFO:tasks.workunit.client.1.smithi148.stdout:5/175: write d7/da/df/f33 [320652,101840] 0 2024-07-03T15:43:47.222 INFO:tasks.workunit.client.1.smithi148.stdout:5/176: creat d7/da/f39 x:0 0 0 2024-07-03T15:43:47.222 INFO:tasks.workunit.client.1.smithi148.stdout:6/272: write d3/f7 [2385285,57849] 0 2024-07-03T15:43:47.225 INFO:tasks.workunit.client.1.smithi148.stdout:0/181: rmdir da/dc/d13 39 2024-07-03T15:43:47.231 INFO:tasks.workunit.client.1.smithi148.stdout:2/275: mknod d5/d14/d22/d2b/d13/d2d/c61 0 2024-07-03T15:43:47.232 INFO:tasks.workunit.client.1.smithi148.stdout:8/168: write d4/d6/d18/d20/f2b [432451,88118] 0 2024-07-03T15:43:47.234 INFO:tasks.workunit.client.1.smithi148.stdout:1/161: symlink d0/de/l31 0 2024-07-03T15:43:47.234 INFO:tasks.workunit.client.1.smithi148.stdout:1/162: fsync d0/d2/d6/f19 0 2024-07-03T15:43:47.236 INFO:tasks.workunit.client.1.smithi148.stdout:3/196: creat dd/d1c/f30 x:0 0 0 2024-07-03T15:43:47.236 INFO:tasks.workunit.client.1.smithi148.stdout:3/197: chown lb 6111929 1 2024-07-03T15:43:47.236 INFO:tasks.workunit.client.1.smithi148.stdout:3/198: stat dd/f21 0 2024-07-03T15:43:47.236 INFO:tasks.workunit.client.1.smithi148.stdout:3/199: creat dd/f31 x:0 0 0 2024-07-03T15:43:47.240 INFO:tasks.workunit.client.1.smithi148.stdout:7/254: dwrite d5/d16/f4b [0,4194304] 0 2024-07-03T15:43:47.240 INFO:tasks.workunit.client.1.smithi148.stdout:7/255: chown d5/d16/d1c/d33/d4c/d21/f3c 42 1 2024-07-03T15:43:47.240 INFO:tasks.workunit.client.1.smithi148.stdout:2/276: dread d5/d14/d22/d2b/d13/d2d/f3b [0,4194304] 0 2024-07-03T15:43:47.240 INFO:tasks.workunit.client.1.smithi148.stdout:2/277: dread - d5/d14/d22/d2b/d13/d1d/f43 zero size 2024-07-03T15:43:47.241 INFO:tasks.workunit.client.1.smithi148.stdout:7/256: write d5/d16/d1c/d33/f42 [377579,84245] 0 2024-07-03T15:43:47.249 INFO:tasks.workunit.client.1.smithi148.stdout:5/177: mknod d7/da/df/d16/c3a 0 2024-07-03T15:43:47.250 INFO:tasks.workunit.client.1.smithi148.stdout:6/273: mknod d3/d5b/c5c 0 2024-07-03T15:43:47.250 INFO:tasks.workunit.client.1.smithi148.stdout:6/274: dread - d3/d1d/d28/d31/d4c/f54 zero size 2024-07-03T15:43:47.250 INFO:tasks.workunit.client.1.smithi148.stdout:6/275: creat d3/d1d/d28/d31/d4c/f5d x:0 0 0 2024-07-03T15:43:47.251 INFO:tasks.workunit.client.1.smithi148.stdout:0/182: mknod da/d17/d2a/c3b 0 2024-07-03T15:43:47.252 INFO:tasks.workunit.client.1.smithi148.stdout:3/200: dwrite dd/d1c/d2e/f11 [0,4194304] 0 2024-07-03T15:43:47.252 INFO:tasks.workunit.client.1.smithi148.stdout:3/201: dread - dd/d1c/f20 zero size 2024-07-03T15:43:47.254 INFO:tasks.workunit.client.1.smithi148.stdout:8/169: mkdir d4/da/d14/d1a/d32/d38 0 2024-07-03T15:43:47.255 INFO:tasks.workunit.client.1.smithi148.stdout:1/163: rename d0/l22 to d0/d2/d6/d14/d29/l32 0 2024-07-03T15:43:47.257 INFO:tasks.workunit.client.1.smithi148.stdout:0/183: dread da/dc/f26 [0,4194304] 0 2024-07-03T15:43:47.257 INFO:tasks.workunit.client.1.smithi148.stdout:0/184: dread - da/dc/d13/f18 zero size 2024-07-03T15:43:47.258 INFO:tasks.workunit.client.1.smithi148.stdout:1/164: dread d0/f8 [0,4194304] 0 2024-07-03T15:43:47.258 INFO:tasks.workunit.client.1.smithi148.stdout:1/165: read d0/d2/d6/f20 [3622656,12997] 0 2024-07-03T15:43:47.267 INFO:tasks.workunit.client.1.smithi148.stdout:3/202: dwrite dd/d1c/d2e/f13 [0,4194304] 0 2024-07-03T15:43:47.269 INFO:tasks.workunit.client.1.smithi148.stdout:6/276: mknod d3/d1d/d28/c5e 0 2024-07-03T15:43:47.274 INFO:tasks.workunit.client.1.smithi148.stdout:8/170: creat d4/d6/d18/d1f/f39 x:0 0 0 2024-07-03T15:43:47.277 INFO:tasks.workunit.client.1.smithi148.stdout:5/178: rename d7/da/f39 to d7/d13/d23/f3b 0 2024-07-03T15:43:47.279 INFO:tasks.workunit.client.1.smithi148.stdout:0/185: symlink da/d17/l3c 0 2024-07-03T15:43:47.280 INFO:tasks.workunit.client.1.smithi148.stdout:0/186: read da/dc/f26 [1548189,30710] 0 2024-07-03T15:43:47.281 INFO:tasks.workunit.client.1.smithi148.stdout:1/166: mknod d0/de/c33 0 2024-07-03T15:43:47.283 INFO:tasks.workunit.client.1.smithi148.stdout:7/257: getdents d5/d16/d1c/d33 0 2024-07-03T15:43:47.288 INFO:tasks.workunit.client.1.smithi148.stdout:3/203: truncate dd/f23 748484 0 2024-07-03T15:43:47.288 INFO:tasks.workunit.client.1.smithi148.stdout:3/204: creat dd/d1c/f32 x:0 0 0 2024-07-03T15:43:47.288 INFO:tasks.workunit.client.1.smithi148.stdout:3/205: fdatasync dd/d1c/f20 0 2024-07-03T15:43:47.289 INFO:tasks.workunit.client.1.smithi148.stdout:6/277: stat d3/c16 0 2024-07-03T15:43:47.292 INFO:tasks.workunit.client.1.smithi148.stdout:8/171: dwrite d4/d6/f8 [0,4194304] 0 2024-07-03T15:43:47.293 INFO:tasks.workunit.client.1.smithi148.stdout:0/187: symlink da/d38/l3d 0 2024-07-03T15:43:47.294 INFO:tasks.workunit.client.1.smithi148.stdout:1/167: rmdir d0/d2/d11/d1e 39 2024-07-03T15:43:47.298 INFO:tasks.workunit.client.1.smithi148.stdout:3/206: mkdir dd/d33 0 2024-07-03T15:43:47.298 INFO:tasks.workunit.client.1.smithi148.stdout:7/258: write d5/d16/f34 [1944698,64667] 0 2024-07-03T15:43:47.298 INFO:tasks.workunit.client.1.smithi148.stdout:7/259: read d5/d16/f34 [2559204,8174] 0 2024-07-03T15:43:47.298 INFO:tasks.workunit.client.1.smithi148.stdout:7/260: readlink d5/d16/d1c/l1d 0 2024-07-03T15:43:47.299 INFO:tasks.workunit.client.1.smithi148.stdout:5/179: rmdir d7/da/df/d1d 39 2024-07-03T15:43:47.299 INFO:tasks.workunit.client.1.smithi148.stdout:5/180: write d7/da/df/f33 [4396707,91022] 0 2024-07-03T15:43:47.300 INFO:tasks.workunit.client.1.smithi148.stdout:5/181: dread d7/da/df/d1d/f2e [0,4194304] 0 2024-07-03T15:43:47.300 INFO:tasks.workunit.client.1.smithi148.stdout:5/182: read d7/da/df/f33 [1327800,76956] 0 2024-07-03T15:43:47.303 INFO:tasks.workunit.client.1.smithi148.stdout:8/172: mkdir d4/da/d14/d3a 0 2024-07-03T15:43:47.304 INFO:tasks.workunit.client.1.smithi148.stdout:0/188: mkdir da/dc/d13/d1d/d3e 0 2024-07-03T15:43:47.304 INFO:tasks.workunit.client.1.smithi148.stdout:4/155: sync 2024-07-03T15:43:47.305 INFO:tasks.workunit.client.1.smithi148.stdout:4/156: stat d0/f5 0 2024-07-03T15:43:47.305 INFO:tasks.workunit.client.1.smithi148.stdout:4/157: write d0/f5 [4567178,67467] 0 2024-07-03T15:43:47.305 INFO:tasks.workunit.client.1.smithi148.stdout:6/278: dwrite d3/f7 [0,4194304] 0 2024-07-03T15:43:47.306 INFO:tasks.workunit.client.1.smithi148.stdout:1/168: symlink d0/l34 0 2024-07-03T15:43:47.309 INFO:tasks.workunit.client.1.smithi148.stdout:3/207: rename dd/f12 to dd/f34 0 2024-07-03T15:43:47.311 INFO:tasks.workunit.client.1.smithi148.stdout:7/261: symlink d5/d16/d1c/d33/l51 0 2024-07-03T15:43:47.313 INFO:tasks.workunit.client.1.smithi148.stdout:5/183: symlink d7/da/df/d16/l3c 0 2024-07-03T15:43:47.313 INFO:tasks.workunit.client.1.smithi148.stdout:5/184: chown d7/da/df/d1d 314 1 2024-07-03T15:43:47.317 INFO:tasks.workunit.client.1.smithi148.stdout:0/189: mknod da/d38/c3f 0 2024-07-03T15:43:47.317 INFO:tasks.workunit.client.1.smithi148.stdout:0/190: write da/dc/f31 [1005233,101615] 0 2024-07-03T15:43:47.322 INFO:tasks.workunit.client.1.smithi148.stdout:3/208: mkdir dd/d35 0 2024-07-03T15:43:47.323 INFO:tasks.workunit.client.1.smithi148.stdout:3/209: truncate dd/d1c/f32 223507 0 2024-07-03T15:43:47.323 INFO:tasks.workunit.client.1.smithi148.stdout:3/210: dread - dd/f2b zero size 2024-07-03T15:43:47.323 INFO:tasks.workunit.client.1.smithi148.stdout:5/185: dread d7/da/f2b [0,4194304] 0 2024-07-03T15:43:47.323 INFO:tasks.workunit.client.1.smithi148.stdout:5/186: creat d7/da/df/f3d x:0 0 0 2024-07-03T15:43:47.323 INFO:tasks.workunit.client.1.smithi148.stdout:5/187: dread - d7/da/df/f3d zero size 2024-07-03T15:43:47.327 INFO:tasks.workunit.client.1.smithi148.stdout:9/381: sync 2024-07-03T15:43:47.327 INFO:tasks.workunit.client.1.smithi148.stdout:9/382: read d0/dd/f2b [2140394,117176] 0 2024-07-03T15:43:47.327 INFO:tasks.workunit.client.1.smithi148.stdout:9/383: chown d0/dd/d46 73 1 2024-07-03T15:43:47.327 INFO:tasks.workunit.client.1.smithi148.stdout:3/211: dread dd/f34 [0,4194304] 0 2024-07-03T15:43:47.327 INFO:tasks.workunit.client.1.smithi148.stdout:0/191: link da/dc/d13/c2b da/d38/c40 0 2024-07-03T15:43:47.329 INFO:tasks.workunit.client.1.smithi148.stdout:4/158: dwrite d0/d8/d22/f21 [0,4194304] 0 2024-07-03T15:43:47.330 INFO:tasks.workunit.client.1.smithi148.stdout:4/159: dread - d0/d8/f2d zero size 2024-07-03T15:43:47.330 INFO:tasks.workunit.client.1.smithi148.stdout:4/160: creat d0/d8/f32 x:0 0 0 2024-07-03T15:43:47.330 INFO:tasks.workunit.client.1.smithi148.stdout:4/161: getdents d0/d28/d2a 0 2024-07-03T15:43:47.330 INFO:tasks.workunit.client.1.smithi148.stdout:4/162: chown d0/d8/f10 2 1 2024-07-03T15:43:47.330 INFO:tasks.workunit.client.1.smithi148.stdout:4/163: truncate d0/d8/d19/d24/f2c 227771 0 2024-07-03T15:43:47.330 INFO:tasks.workunit.client.1.smithi148.stdout:8/173: dwrite d4/d6/f34 [0,4194304] 0 2024-07-03T15:43:47.330 INFO:tasks.workunit.client.1.smithi148.stdout:8/174: dread - d4/d6/d18/d1f/f39 zero size 2024-07-03T15:43:47.330 INFO:tasks.workunit.client.1.smithi148.stdout:1/169: truncate d0/d2/fd 1157707 0 2024-07-03T15:43:47.330 INFO:tasks.workunit.client.1.smithi148.stdout:1/170: read d0/d2/d6/f20 [2656942,98606] 0 2024-07-03T15:43:47.330 INFO:tasks.workunit.client.1.smithi148.stdout:1/171: rename d0 to d0/de/d35 22 2024-07-03T15:43:47.332 INFO:tasks.workunit.client.1.smithi148.stdout:5/188: symlink d7/d15/l3e 0 2024-07-03T15:43:47.333 INFO:tasks.workunit.client.1.smithi148.stdout:5/189: dread d7/da/df/d1d/f2e [0,4194304] 0 2024-07-03T15:43:47.335 INFO:tasks.workunit.client.1.smithi148.stdout:9/384: creat d0/d49/f81 x:0 0 0 2024-07-03T15:43:47.335 INFO:tasks.workunit.client.1.smithi148.stdout:9/385: write d0/dd/fe [1891065,55810] 0 2024-07-03T15:43:47.336 INFO:tasks.workunit.client.1.smithi148.stdout:9/386: chown d0/dd/d25/d37/f50 5174799 1 2024-07-03T15:43:47.336 INFO:tasks.workunit.client.1.smithi148.stdout:9/387: dread - d0/d2f/f7f zero size 2024-07-03T15:43:47.336 INFO:tasks.workunit.client.1.smithi148.stdout:9/388: fdatasync d0/dd/f1a 0 2024-07-03T15:43:47.336 INFO:tasks.workunit.client.1.smithi148.stdout:9/389: chown d0/dd/l61 6 1 2024-07-03T15:43:47.338 INFO:tasks.workunit.client.1.smithi148.stdout:4/164: dread d0/f16 [0,4194304] 0 2024-07-03T15:43:47.345 INFO:tasks.workunit.client.1.smithi148.stdout:7/262: truncate d5/d16/f4b 3240063 0 2024-07-03T15:43:47.345 INFO:tasks.workunit.client.1.smithi148.stdout:7/263: fdatasync d5/d16/d1c/d23/f4f 0 2024-07-03T15:43:47.348 INFO:tasks.workunit.client.1.smithi148.stdout:7/264: dread d5/f9 [0,4194304] 0 2024-07-03T15:43:47.348 INFO:tasks.workunit.client.1.smithi148.stdout:7/265: write d5/d16/d1c/d33/d4c/d21/d31/f45 [1021540,99264] 0 2024-07-03T15:43:47.348 INFO:tasks.workunit.client.1.smithi148.stdout:7/266: chown d5/d16/d1c/d33/l35 0 1 2024-07-03T15:43:47.350 INFO:tasks.workunit.client.1.smithi148.stdout:7/267: dread d5/d16/d1c/d33/d4c/d21/f3c [0,4194304] 0 2024-07-03T15:43:47.350 INFO:tasks.workunit.client.1.smithi148.stdout:7/268: fdatasync d5/d16/f37 0 2024-07-03T15:43:47.353 INFO:tasks.workunit.client.1.smithi148.stdout:8/175: mknod d4/d6/d18/d25/c3b 0 2024-07-03T15:43:47.354 INFO:tasks.workunit.client.1.smithi148.stdout:4/165: dwrite d0/f4 [0,4194304] 0 2024-07-03T15:43:47.354 INFO:tasks.workunit.client.1.smithi148.stdout:4/166: dread - d0/d8/f32 zero size 2024-07-03T15:43:47.360 INFO:tasks.workunit.client.1.smithi148.stdout:9/390: creat d0/dd/d25/d6a/f82 x:0 0 0 2024-07-03T15:43:47.368 INFO:tasks.workunit.client.1.smithi148.stdout:7/269: mkdir d5/d16/d1c/d4a/d52 0 2024-07-03T15:43:47.368 INFO:tasks.workunit.client.1.smithi148.stdout:7/270: write d5/f3b [6126287,29009] 0 2024-07-03T15:43:47.370 INFO:tasks.workunit.client.1.smithi148.stdout:8/176: mknod d4/d6/d18/c3c 0 2024-07-03T15:43:47.372 INFO:tasks.workunit.client.1.smithi148.stdout:4/167: dwrite d0/f16 [0,4194304] 0 2024-07-03T15:43:47.372 INFO:tasks.workunit.client.1.smithi148.stdout:5/190: rmdir d7/d13 39 2024-07-03T15:43:47.372 INFO:tasks.workunit.client.1.smithi148.stdout:5/191: chown d7/da/df/d16/d2f 58884801 1 2024-07-03T15:43:47.375 INFO:tasks.workunit.client.1.smithi148.stdout:7/271: link d5/d16/d1c/d33/l51 d5/d16/d1c/d33/d4c/d21/d39/l53 0 2024-07-03T15:43:47.377 INFO:tasks.workunit.client.1.smithi148.stdout:8/177: creat d4/d6/d18/d25/f3d x:0 0 0 2024-07-03T15:43:47.377 INFO:tasks.workunit.client.1.smithi148.stdout:8/178: creat d4/da/f3e x:0 0 0 2024-07-03T15:43:47.379 INFO:tasks.workunit.client.1.smithi148.stdout:1/172: dwrite d0/d2/f10 [8388608,4194304] 0 2024-07-03T15:43:47.380 INFO:tasks.workunit.client.1.smithi148.stdout:4/168: creat d0/d8/d22/d2f/f33 x:0 0 0 2024-07-03T15:43:47.382 INFO:tasks.workunit.client.1.smithi148.stdout:5/192: symlink d7/da/df/d16/d1a/l3f 0 2024-07-03T15:43:47.382 INFO:tasks.workunit.client.1.smithi148.stdout:5/193: fdatasync d7/da/df/d16/d1a/f34 0 2024-07-03T15:43:47.383 INFO:tasks.workunit.client.1.smithi148.stdout:3/212: rmdir dd/d1c 39 2024-07-03T15:43:47.384 INFO:tasks.workunit.client.1.smithi148.stdout:3/213: creat dd/f36 x:0 0 0 2024-07-03T15:43:47.386 INFO:tasks.workunit.client.1.smithi148.stdout:7/272: rename d5/d16/d1c/c2a to d5/d16/d1c/d4a/c54 0 2024-07-03T15:43:47.390 INFO:tasks.workunit.client.1.smithi148.stdout:8/179: mknod d4/d6/d18/c3f 0 2024-07-03T15:43:47.390 INFO:tasks.workunit.client.1.smithi148.stdout:8/180: chown d4/da/d14/d1a 2036587986 1 2024-07-03T15:43:47.399 INFO:tasks.workunit.client.1.smithi148.stdout:1/173: link d0/d2/f10 d0/d2/d2e/f36 0 2024-07-03T15:43:47.402 INFO:tasks.workunit.client.1.smithi148.stdout:5/194: stat d7/da/cc 0 2024-07-03T15:43:47.405 INFO:tasks.workunit.client.1.smithi148.stdout:3/214: mkdir dd/d33/d37 0 2024-07-03T15:43:47.405 INFO:tasks.workunit.client.1.smithi148.stdout:3/215: chown dd/d1c/f32 7668 1 2024-07-03T15:43:47.407 INFO:tasks.workunit.client.1.smithi148.stdout:7/273: mkdir d5/d16/d27/d55 0 2024-07-03T15:43:47.410 INFO:tasks.workunit.client.1.smithi148.stdout:8/181: unlink d4/da/d14/d17/l2f 0 2024-07-03T15:43:47.411 INFO:tasks.workunit.client.1.smithi148.stdout:8/182: dread d4/d6/d18/d25/f27 [0,4194304] 0 2024-07-03T15:43:47.411 INFO:tasks.workunit.client.1.smithi148.stdout:8/183: creat d4/d6/d18/d20/f40 x:0 0 0 2024-07-03T15:43:47.412 INFO:tasks.workunit.client.1.smithi148.stdout:1/174: mknod d0/d2/d6/d14/c37 0 2024-07-03T15:43:47.413 INFO:tasks.workunit.client.1.smithi148.stdout:0/192: rmdir da/dc 39 2024-07-03T15:43:47.413 INFO:tasks.workunit.client.1.smithi148.stdout:0/193: fsync da/d17/d2a/d2f/f37 0 2024-07-03T15:43:47.413 INFO:tasks.workunit.client.1.smithi148.stdout:0/194: creat da/d17/d2a/d2f/f41 x:0 0 0 2024-07-03T15:43:47.414 INFO:tasks.workunit.client.1.smithi148.stdout:3/216: dread dd/f28 [0,4194304] 0 2024-07-03T15:43:47.421 INFO:tasks.workunit.client.1.smithi148.stdout:5/195: mkdir d7/d13/d23/d40 0 2024-07-03T15:43:47.421 INFO:tasks.workunit.client.1.smithi148.stdout:5/196: getdents d7/d13/d23/d40 0 2024-07-03T15:43:47.429 INFO:tasks.workunit.client.1.smithi148.stdout:5/197: read d7/da/df/d16/d1a/f34 [1034953,71382] 0 2024-07-03T15:43:47.429 INFO:tasks.workunit.client.1.smithi148.stdout:5/198: dread - d7/da/df/f3d zero size 2024-07-03T15:43:47.430 INFO:tasks.workunit.client.1.smithi148.stdout:3/217: unlink dd/d1c/f1d 0 2024-07-03T15:43:47.432 INFO:tasks.workunit.client.1.smithi148.stdout:4/169: rmdir d0/d8/d1f 0 2024-07-03T15:43:47.432 INFO:tasks.workunit.client.1.smithi148.stdout:4/170: fdatasync d0/f30 0 2024-07-03T15:43:47.434 INFO:tasks.workunit.client.1.smithi148.stdout:5/199: dread d7/f19 [4194304,4194304] 0 2024-07-03T15:43:47.437 INFO:tasks.workunit.client.1.smithi148.stdout:8/184: link d4/d6/d18/d20/f40 d4/da/d14/d3a/f41 0 2024-07-03T15:43:47.437 INFO:tasks.workunit.client.1.smithi148.stdout:8/185: creat d4/d6/d18/d1f/f42 x:0 0 0 2024-07-03T15:43:47.437 INFO:tasks.workunit.client.1.smithi148.stdout:8/186: write d4/da/f3e [795582,104373] 0 2024-07-03T15:43:47.437 INFO:tasks.workunit.client.1.smithi148.stdout:8/187: chown d4/l35 0 1 2024-07-03T15:43:47.439 INFO:tasks.workunit.client.1.smithi148.stdout:0/195: link da/dc/d24/c25 da/dc/d13/c42 0 2024-07-03T15:43:47.443 INFO:tasks.workunit.client.1.smithi148.stdout:5/200: rmdir d7/d15 39 2024-07-03T15:43:47.446 INFO:tasks.workunit.client.1.smithi148.stdout:0/196: write da/dc/d24/f30 [248782,11975] 0 2024-07-03T15:43:47.448 INFO:tasks.workunit.client.1.smithi148.stdout:8/188: mknod d4/d6/d18/c43 0 2024-07-03T15:43:47.448 INFO:tasks.workunit.client.1.smithi148.stdout:9/391: fsync d0/dd/d25/d6a/f82 0 2024-07-03T15:43:47.455 INFO:tasks.workunit.client.1.smithi148.stdout:8/189: dread d4/da/d14/d17/f30 [0,4194304] 0 2024-07-03T15:43:47.456 INFO:tasks.workunit.client.1.smithi148.stdout:8/190: chown d4/da/d14/d1a/d32/d38 27856 1 2024-07-03T15:43:47.457 INFO:tasks.workunit.client.1.smithi148.stdout:5/201: mknod d7/da/d21/c41 0 2024-07-03T15:43:47.462 INFO:tasks.workunit.client.1.smithi148.stdout:0/197: dwrite da/d17/d2a/f2e [0,4194304] 0 2024-07-03T15:43:47.463 INFO:tasks.workunit.client.1.smithi148.stdout:9/392: unlink d0/d49/d43/l6f 0 2024-07-03T15:43:47.463 INFO:tasks.workunit.client.1.smithi148.stdout:9/393: getdents d0/d18/d24/d47 0 2024-07-03T15:43:47.470 INFO:tasks.workunit.client.1.smithi148.stdout:9/394: mknod d0/d18/d24/d5a/c83 0 2024-07-03T15:43:47.470 INFO:tasks.workunit.client.1.smithi148.stdout:9/395: truncate d0/d49/f81 502569 0 2024-07-03T15:43:47.470 INFO:tasks.workunit.client.1.smithi148.stdout:9/396: readlink d0/dd/l9 0 2024-07-03T15:43:47.470 INFO:tasks.workunit.client.1.smithi148.stdout:9/397: readlink d0/d49/d43/l4c 0 2024-07-03T15:43:47.472 INFO:tasks.workunit.client.1.smithi148.stdout:9/398: symlink d0/d18/l84 0 2024-07-03T15:43:47.472 INFO:tasks.workunit.client.1.smithi148.stdout:8/191: dwrite d4/da/d14/d17/f30 [0,4194304] 0 2024-07-03T15:43:47.473 INFO:tasks.workunit.client.1.smithi148.stdout:8/192: read d4/da/f3e [732789,84902] 0 2024-07-03T15:43:47.475 INFO:tasks.workunit.client.1.smithi148.stdout:0/198: dread da/f11 [0,4194304] 0 2024-07-03T15:43:47.476 INFO:tasks.workunit.client.1.smithi148.stdout:0/199: getdents da/d17/d39 0 2024-07-03T15:43:47.478 INFO:tasks.workunit.client.1.smithi148.stdout:0/200: mkdir da/dc/d13/d1d/d3e/d43 0 2024-07-03T15:43:47.478 INFO:tasks.workunit.client.1.smithi148.stdout:0/201: stat da/dc/d13/d14/f1f 0 2024-07-03T15:43:47.478 INFO:tasks.workunit.client.1.smithi148.stdout:2/278: sync 2024-07-03T15:43:47.478 INFO:tasks.workunit.client.1.smithi148.stdout:2/279: write d5/f9 [4115705,19120] 0 2024-07-03T15:43:47.481 INFO:tasks.workunit.client.1.smithi148.stdout:0/202: rmdir da/dc/d13/d1d/d3e 39 2024-07-03T15:43:47.500 INFO:tasks.workunit.client.1.smithi148.stdout:8/193: dwrite d4/da/d14/d17/f26 [4194304,4194304] 0 2024-07-03T15:43:47.500 INFO:tasks.workunit.client.1.smithi148.stdout:2/280: dwrite d5/d14/d22/d2b/f1a [0,4194304] 0 2024-07-03T15:43:47.500 INFO:tasks.workunit.client.1.smithi148.stdout:2/281: write d5/d26/f48 [808421,75067] 0 2024-07-03T15:43:47.500 INFO:tasks.workunit.client.1.smithi148.stdout:2/282: chown d5/d14/d22/d2b/c11 720779 1 2024-07-03T15:43:47.503 INFO:tasks.workunit.client.1.smithi148.stdout:8/194: mknod d4/d6/d18/d25/c44 0 2024-07-03T15:43:47.505 INFO:tasks.workunit.client.1.smithi148.stdout:2/283: creat d5/d14/d22/f62 x:0 0 0 2024-07-03T15:43:47.524 INFO:tasks.workunit.client.1.smithi148.stdout:2/284: dwrite d5/d26/f48 [0,4194304] 0 2024-07-03T15:43:47.525 INFO:tasks.workunit.client.1.smithi148.stdout:2/285: creat d5/d14/d22/d2b/d13/d41/f63 x:0 0 0 2024-07-03T15:43:47.529 INFO:tasks.workunit.client.1.smithi148.stdout:2/286: mkdir d5/d7/d64 0 2024-07-03T15:43:47.542 INFO:tasks.workunit.client.1.smithi148.stdout:2/287: dwrite d5/d14/d22/d2b/d13/d2d/f51 [0,4194304] 0 2024-07-03T15:43:47.545 INFO:tasks.workunit.client.1.smithi148.stdout:2/288: creat d5/d14/d22/d60/f65 x:0 0 0 2024-07-03T15:43:47.545 INFO:tasks.workunit.client.1.smithi148.stdout:2/289: rename d5/d14/d22/d2b to d5/d14/d22/d2b/d13/d1d/d66 22 2024-07-03T15:43:47.545 INFO:tasks.workunit.client.1.smithi148.stdout:2/290: fdatasync d5/d14/d22/d2b/d13/d41/f49 0 2024-07-03T15:43:47.558 INFO:tasks.workunit.client.1.smithi148.stdout:2/291: dwrite d5/d7/f3d [0,4194304] 0 2024-07-03T15:43:47.559 INFO:tasks.workunit.client.1.smithi148.stdout:2/292: readlink d5/l36 0 2024-07-03T15:43:47.559 INFO:tasks.workunit.client.1.smithi148.stdout:2/293: dread d5/d26/f3e [0,4194304] 0 2024-07-03T15:43:47.643 INFO:tasks.workunit.client.1.smithi148.stdout:7/274: dwrite d5/d16/d1c/d33/d4c/f26 [0,4194304] 0 2024-07-03T15:43:47.643 INFO:tasks.workunit.client.1.smithi148.stdout:7/275: chown d5/d16/d1c/d33/d4c/d21/d31/f45 1413 1 2024-07-03T15:43:47.646 INFO:tasks.workunit.client.1.smithi148.stdout:9/399: dwrite d0/d18/f69 [0,4194304] 0 2024-07-03T15:43:47.646 INFO:tasks.workunit.client.1.smithi148.stdout:8/195: dwrite d4/da/d14/d3a/f41 [0,4194304] 0 2024-07-03T15:43:47.654 INFO:tasks.workunit.client.1.smithi148.stdout:7/276: rename d5/d16/l46 to d5/d16/d1c/d33/d4c/d21/d39/l56 0 2024-07-03T15:43:47.657 INFO:tasks.workunit.client.1.smithi148.stdout:1/175: getdents d0/d2/d6/d14 0 2024-07-03T15:43:47.662 INFO:tasks.workunit.client.1.smithi148.stdout:9/400: dwrite d0/dd/f14 [4194304,4194304] 0 2024-07-03T15:43:47.663 INFO:tasks.workunit.client.1.smithi148.stdout:8/196: mknod d4/d6/c45 0 2024-07-03T15:43:47.665 INFO:tasks.workunit.client.1.smithi148.stdout:9/401: dread d0/dd/d11/f12 [0,4194304] 0 2024-07-03T15:43:47.665 INFO:tasks.workunit.client.1.smithi148.stdout:7/277: creat d5/f57 x:0 0 0 2024-07-03T15:43:47.665 INFO:tasks.workunit.client.1.smithi148.stdout:7/278: creat d5/f58 x:0 0 0 2024-07-03T15:43:47.666 INFO:tasks.workunit.client.1.smithi148.stdout:7/279: chown d5/d16/d1c/d33/d4c/ld 144 1 2024-07-03T15:43:47.667 INFO:tasks.workunit.client.1.smithi148.stdout:1/176: mkdir d0/d2/d11/d38 0 2024-07-03T15:43:47.667 INFO:tasks.workunit.client.1.smithi148.stdout:1/177: getdents d0/de/d2f 0 2024-07-03T15:43:47.670 INFO:tasks.workunit.client.1.smithi148.stdout:3/218: rmdir dd 39 2024-07-03T15:43:47.670 INFO:tasks.workunit.client.1.smithi148.stdout:3/219: dread - dd/f2b zero size 2024-07-03T15:43:47.672 INFO:tasks.workunit.client.1.smithi148.stdout:8/197: rename d4/d6/f34 to d4/d6/d18/d25/f46 0 2024-07-03T15:43:47.674 INFO:tasks.workunit.client.1.smithi148.stdout:1/178: dread d0/f25 [0,4194304] 0 2024-07-03T15:43:47.674 INFO:tasks.workunit.client.1.smithi148.stdout:1/179: fsync d0/f24 0 2024-07-03T15:43:47.675 INFO:tasks.workunit.client.1.smithi148.stdout:9/402: link d0/dd/d25/d37/d40/d72/c76 d0/dd/d25/d37/d40/c85 0 2024-07-03T15:43:47.675 INFO:tasks.workunit.client.1.smithi148.stdout:9/403: chown d0/dd/d25/d37/d40/d72 3 1 2024-07-03T15:43:47.677 INFO:tasks.workunit.client.1.smithi148.stdout:8/198: fsync d4/d6/d18/d20/f40 0 2024-07-03T15:43:47.680 INFO:tasks.workunit.client.1.smithi148.stdout:7/280: rmdir d5/d16/d27/d55 0 2024-07-03T15:43:47.681 INFO:tasks.workunit.client.1.smithi148.stdout:6/279: sync 2024-07-03T15:43:47.682 INFO:tasks.workunit.client.1.smithi148.stdout:6/280: creat d3/d1d/d28/f5f x:0 0 0 2024-07-03T15:43:47.683 INFO:tasks.workunit.client.1.smithi148.stdout:1/180: mkdir d0/d2/d2e/d39 0 2024-07-03T15:43:47.684 INFO:tasks.workunit.client.1.smithi148.stdout:9/404: symlink d0/d18/d24/d47/l86 0 2024-07-03T15:43:47.684 INFO:tasks.workunit.client.1.smithi148.stdout:9/405: chown d0/d49/f81 124061 1 2024-07-03T15:43:47.684 INFO:tasks.workunit.client.1.smithi148.stdout:9/406: chown d0/dd/d25/d37/d40/c80 0 1 2024-07-03T15:43:47.686 INFO:tasks.workunit.client.1.smithi148.stdout:8/199: mkdir d4/da/d47 0 2024-07-03T15:43:47.687 INFO:tasks.workunit.client.1.smithi148.stdout:3/220: dwrite dd/d1c/d2e/f2c [0,4194304] 0 2024-07-03T15:43:47.687 INFO:tasks.workunit.client.1.smithi148.stdout:7/281: rmdir d5/d16/d27 39 2024-07-03T15:43:47.687 INFO:tasks.workunit.client.1.smithi148.stdout:7/282: chown d5/d16/c36 643171 1 2024-07-03T15:43:47.689 INFO:tasks.workunit.client.1.smithi148.stdout:5/202: getdents d7/da/d21 0 2024-07-03T15:43:47.692 INFO:tasks.workunit.client.1.smithi148.stdout:1/181: mkdir d0/d2/d11/d3a 0 2024-07-03T15:43:47.692 INFO:tasks.workunit.client.1.smithi148.stdout:9/407: write d0/d18/d24/f30 [60273,49660] 0 2024-07-03T15:43:47.694 INFO:tasks.workunit.client.1.smithi148.stdout:1/182: dread d0/f8 [0,4194304] 0 2024-07-03T15:43:47.695 INFO:tasks.workunit.client.1.smithi148.stdout:9/408: dread d0/d2f/f41 [0,4194304] 0 2024-07-03T15:43:47.695 INFO:tasks.workunit.client.1.smithi148.stdout:8/200: creat d4/da/d14/d17/f48 x:0 0 0 2024-07-03T15:43:47.695 INFO:tasks.workunit.client.1.smithi148.stdout:9/409: truncate d0/dd/d25/d37/d40/f62 7612 0 2024-07-03T15:43:47.695 INFO:tasks.workunit.client.1.smithi148.stdout:9/410: creat d0/d49/d43/f87 x:0 0 0 2024-07-03T15:43:47.695 INFO:tasks.workunit.client.1.smithi148.stdout:9/411: chown d0/dd/d25/d39 311957718 1 2024-07-03T15:43:47.695 INFO:tasks.workunit.client.1.smithi148.stdout:9/412: dread - d0/dd/d29/f5e zero size 2024-07-03T15:43:47.700 INFO:tasks.workunit.client.1.smithi148.stdout:9/413: dread d0/dd/d11/f35 [0,4194304] 0 2024-07-03T15:43:47.701 INFO:tasks.workunit.client.1.smithi148.stdout:7/283: dwrite d5/d16/f34 [4194304,4194304] 0 2024-07-03T15:43:47.703 INFO:tasks.workunit.client.1.smithi148.stdout:3/221: truncate dd/f22 1690046 0 2024-07-03T15:43:47.705 INFO:tasks.workunit.client.1.smithi148.stdout:5/203: mknod d7/d15/c42 0 2024-07-03T15:43:47.708 INFO:tasks.workunit.client.1.smithi148.stdout:6/281: symlink d3/d1d/l60 0 2024-07-03T15:43:47.708 INFO:tasks.workunit.client.1.smithi148.stdout:6/282: chown d3/f56 2 1 2024-07-03T15:43:47.710 INFO:tasks.workunit.client.1.smithi148.stdout:1/183: link d0/c18 d0/d2/d6/c3b 0 2024-07-03T15:43:47.712 INFO:tasks.workunit.client.1.smithi148.stdout:8/201: symlink d4/d6/d18/d25/l49 0 2024-07-03T15:43:47.712 INFO:tasks.workunit.client.1.smithi148.stdout:8/202: fdatasync d4/da/f3e 0 2024-07-03T15:43:47.714 INFO:tasks.workunit.client.1.smithi148.stdout:9/414: stat d0/dd/d29/l7b 0 2024-07-03T15:43:47.714 INFO:tasks.workunit.client.1.smithi148.stdout:9/415: write d0/dd/fe [2688614,2570] 0 2024-07-03T15:43:47.716 INFO:tasks.workunit.client.1.smithi148.stdout:7/284: rmdir d5/d16/d1c/d33 39 2024-07-03T15:43:47.716 INFO:tasks.workunit.client.1.smithi148.stdout:7/285: write d5/f9 [893533,117244] 0 2024-07-03T15:43:47.717 INFO:tasks.workunit.client.1.smithi148.stdout:5/204: dwrite d7/d13/d23/f3b [0,4194304] 0 2024-07-03T15:43:47.717 INFO:tasks.workunit.client.1.smithi148.stdout:7/286: write d5/d16/d1c/d33/d4c/d21/f3c [117723,21969] 0 2024-07-03T15:43:47.717 INFO:tasks.workunit.client.1.smithi148.stdout:7/287: chown d5/d16/d1c/d33/f42 136642 1 2024-07-03T15:43:47.717 INFO:tasks.workunit.client.1.smithi148.stdout:7/288: chown d5/d16/d1c/d33/d4c/d21/l2f 0 1 2024-07-03T15:43:47.717 INFO:tasks.workunit.client.1.smithi148.stdout:7/289: write d5/f58 [683081,60627] 0 2024-07-03T15:43:47.718 INFO:tasks.workunit.client.1.smithi148.stdout:3/222: unlink dd/f28 0 2024-07-03T15:43:47.720 INFO:tasks.workunit.client.1.smithi148.stdout:6/283: mkdir d3/d1d/d28/d31/d61 0 2024-07-03T15:43:47.721 INFO:tasks.workunit.client.1.smithi148.stdout:1/184: creat d0/d2/d11/d1e/f3c x:0 0 0 2024-07-03T15:43:47.724 INFO:tasks.workunit.client.1.smithi148.stdout:9/416: symlink d0/dd/l88 0 2024-07-03T15:43:47.724 INFO:tasks.workunit.client.1.smithi148.stdout:9/417: truncate d0/d2f/f74 946958 0 2024-07-03T15:43:47.724 INFO:tasks.workunit.client.1.smithi148.stdout:9/418: stat d0/dd/d29/l45 0 2024-07-03T15:43:47.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:47 smithi148 ceph-mon[36056]: pgmap v236: 65 pgs: 65 active+clean; 429 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 27 MiB/s rd, 38 MiB/s wr, 641 op/s 2024-07-03T15:43:47.732 INFO:tasks.workunit.client.1.smithi148.stdout:9/419: write d0/dd/f16 [2705193,106754] 0 2024-07-03T15:43:47.734 INFO:tasks.workunit.client.1.smithi148.stdout:3/223: symlink dd/d1c/l38 0 2024-07-03T15:43:47.734 INFO:tasks.workunit.client.1.smithi148.stdout:3/224: dread - dd/f31 zero size 2024-07-03T15:43:47.746 INFO:tasks.workunit.client.1.smithi148.stdout:9/420: creat d0/dd/d25/d77/f89 x:0 0 0 2024-07-03T15:43:47.748 INFO:tasks.workunit.client.1.smithi148.stdout:5/205: dwrite d7/da/df/d16/d2f/f35 [0,4194304] 0 2024-07-03T15:43:47.753 INFO:tasks.workunit.client.1.smithi148.stdout:5/206: stat d7/da/d21/c31 0 2024-07-03T15:43:47.755 INFO:tasks.workunit.client.1.smithi148.stdout:5/207: getdents d7/da/df/d16 0 2024-07-03T15:43:47.757 INFO:tasks.workunit.client.1.smithi148.stdout:5/208: rmdir d7/da/df/d16 39 2024-07-03T15:43:47.757 INFO:tasks.workunit.client.1.smithi148.stdout:5/209: chown d7/da/df/d1d 0 1 2024-07-03T15:43:47.757 INFO:tasks.workunit.client.1.smithi148.stdout:9/421: write d0/dd/f1f [2771520,79174] 0 2024-07-03T15:43:47.759 INFO:tasks.workunit.client.1.smithi148.stdout:5/210: rename d7/d13/d23/c27 to d7/d13/d23/c43 0 2024-07-03T15:43:47.760 INFO:tasks.workunit.client.1.smithi148.stdout:7/290: dwrite d5/d16/f4d [0,4194304] 0 2024-07-03T15:43:47.760 INFO:tasks.workunit.client.1.smithi148.stdout:6/284: dwrite d3/d1d/d28/f2e [0,4194304] 0 2024-07-03T15:43:47.761 INFO:tasks.workunit.client.1.smithi148.stdout:3/225: dwrite dd/f15 [4194304,4194304] 0 2024-07-03T15:43:47.761 INFO:tasks.workunit.client.1.smithi148.stdout:3/226: getdents dd/d35 0 2024-07-03T15:43:47.761 INFO:tasks.workunit.client.1.smithi148.stdout:3/227: fdatasync f5 0 2024-07-03T15:43:47.762 INFO:tasks.workunit.client.1.smithi148.stdout:9/422: symlink d0/dd/d11/l8a 0 2024-07-03T15:43:47.769 INFO:tasks.workunit.client.1.smithi148.stdout:3/228: dread dd/f15 [0,4194304] 0 2024-07-03T15:43:47.771 INFO:tasks.workunit.client.1.smithi148.stdout:6/285: mknod d3/d1d/d28/d3e/c62 0 2024-07-03T15:43:47.771 INFO:tasks.workunit.client.1.smithi148.stdout:6/286: chown d3/d1d/d28/c5e 12083 1 2024-07-03T15:43:47.771 INFO:tasks.workunit.client.1.smithi148.stdout:6/287: fdatasync d3/d1d/d28/d31/d4c/f57 0 2024-07-03T15:43:47.772 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:47 smithi028 ceph-mon[29148]: pgmap v236: 65 pgs: 65 active+clean; 429 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 27 MiB/s rd, 38 MiB/s wr, 641 op/s 2024-07-03T15:43:47.775 INFO:tasks.workunit.client.1.smithi148.stdout:9/423: mkdir d0/d8b 0 2024-07-03T15:43:47.777 INFO:tasks.workunit.client.1.smithi148.stdout:9/424: dread d0/d49/f73 [0,4194304] 0 2024-07-03T15:43:47.777 INFO:tasks.workunit.client.1.smithi148.stdout:9/425: write d0/dd/d25/d37/d40/f62 [782450,80361] 0 2024-07-03T15:43:47.778 INFO:tasks.workunit.client.1.smithi148.stdout:5/211: dwrite d7/da/df/f33 [0,4194304] 0 2024-07-03T15:43:47.778 INFO:tasks.workunit.client.1.smithi148.stdout:9/426: write d0/dd/d25/d6a/f82 [811762,12119] 0 2024-07-03T15:43:47.778 INFO:tasks.workunit.client.1.smithi148.stdout:5/212: chown d7/da/l14 1779003984 1 2024-07-03T15:43:47.783 INFO:tasks.workunit.client.1.smithi148.stdout:7/291: getdents d5/d16/d1c/d23 0 2024-07-03T15:43:47.784 INFO:tasks.workunit.client.1.smithi148.stdout:3/229: creat dd/d33/f39 x:0 0 0 2024-07-03T15:43:47.784 INFO:tasks.workunit.client.1.smithi148.stdout:3/230: creat dd/d33/f3a x:0 0 0 2024-07-03T15:43:47.785 INFO:tasks.workunit.client.1.smithi148.stdout:5/213: write d7/da/df/f33 [1831817,816] 0 2024-07-03T15:43:47.785 INFO:tasks.workunit.client.1.smithi148.stdout:6/288: symlink d3/d5b/l63 0 2024-07-03T15:43:47.789 INFO:tasks.workunit.client.1.smithi148.stdout:9/427: rmdir d0/d18/d24/d47 39 2024-07-03T15:43:47.791 INFO:tasks.workunit.client.1.smithi148.stdout:7/292: mkdir d5/d16/d1c/d59 0 2024-07-03T15:43:47.791 INFO:tasks.workunit.client.1.smithi148.stdout:7/293: write d5/d16/f4b [3788783,28828] 0 2024-07-03T15:43:47.792 INFO:tasks.workunit.client.1.smithi148.stdout:2/294: rmdir d5 39 2024-07-03T15:43:47.792 INFO:tasks.workunit.client.1.smithi148.stdout:2/295: readlink d5/d14/d22/d2b/d13/d41/l56 0 2024-07-03T15:43:47.796 INFO:tasks.workunit.client.1.smithi148.stdout:3/231: symlink dd/d33/d37/l3b 0 2024-07-03T15:43:47.797 INFO:tasks.workunit.client.1.smithi148.stdout:5/214: mkdir d7/da/df/d16/d2f/d44 0 2024-07-03T15:43:47.799 INFO:tasks.workunit.client.1.smithi148.stdout:9/428: unlink d0/d2f/c70 0 2024-07-03T15:43:47.801 INFO:tasks.workunit.client.1.smithi148.stdout:2/296: getdents d5/d14/d22/d2b 0 2024-07-03T15:43:47.807 INFO:tasks.workunit.client.1.smithi148.stdout:5/215: mknod d7/da/df/d16/d2f/d44/c45 0 2024-07-03T15:43:47.810 INFO:tasks.workunit.client.1.smithi148.stdout:9/429: write d0/dd/d25/d37/d40/f62 [236670,27246] 0 2024-07-03T15:43:47.811 INFO:tasks.workunit.client.1.smithi148.stdout:7/294: dwrite d5/d16/d1c/d33/d4c/d21/d39/f3e [0,4194304] 0 2024-07-03T15:43:47.814 INFO:tasks.workunit.client.1.smithi148.stdout:6/289: dwrite d3/d1d/f21 [0,4194304] 0 2024-07-03T15:43:47.814 INFO:tasks.workunit.client.1.smithi148.stdout:9/430: mkdir d0/d49/d43/d19/d8c 0 2024-07-03T15:43:47.818 INFO:tasks.workunit.client.1.smithi148.stdout:9/431: dread d0/d2f/f74 [0,4194304] 0 2024-07-03T15:43:47.819 INFO:tasks.workunit.client.1.smithi148.stdout:6/290: truncate d3/d1d/d28/f2a 1566141 0 2024-07-03T15:43:47.819 INFO:tasks.workunit.client.1.smithi148.stdout:6/291: readlink d3/d1d/d28/l46 0 2024-07-03T15:43:47.821 INFO:tasks.workunit.client.1.smithi148.stdout:7/295: dread d5/d16/f34 [4194304,4194304] 0 2024-07-03T15:43:47.821 INFO:tasks.workunit.client.1.smithi148.stdout:7/296: write d5/f58 [884365,102962] 0 2024-07-03T15:43:47.822 INFO:tasks.workunit.client.1.smithi148.stdout:7/297: write d5/f9 [171707,25960] 0 2024-07-03T15:43:47.824 INFO:tasks.workunit.client.1.smithi148.stdout:2/297: dwrite d5/d26/f57 [0,4194304] 0 2024-07-03T15:43:47.828 INFO:tasks.workunit.client.1.smithi148.stdout:5/216: dwrite d7/f19 [0,4194304] 0 2024-07-03T15:43:47.831 INFO:tasks.workunit.client.1.smithi148.stdout:5/217: dread d7/da/df/d16/d1a/f34 [0,4194304] 0 2024-07-03T15:43:47.880 INFO:tasks.workunit.client.1.smithi148.stdout:6/292: rmdir d3/d5b 39 2024-07-03T15:43:47.880 INFO:tasks.workunit.client.1.smithi148.stdout:9/432: creat d0/dd/d25/d37/d40/d72/f8d x:0 0 0 2024-07-03T15:43:47.881 INFO:tasks.workunit.client.1.smithi148.stdout:7/298: unlink d5/d16/d1c/d23/f4f 0 2024-07-03T15:43:47.881 INFO:tasks.workunit.client.1.smithi148.stdout:2/298: mkdir d5/d67 0 2024-07-03T15:43:47.881 INFO:tasks.workunit.client.1.smithi148.stdout:5/218: symlink d7/d13/d23/l46 0 2024-07-03T15:43:47.881 INFO:tasks.workunit.client.1.smithi148.stdout:2/299: fdatasync d5/d14/d22/d2b/d13/f15 0 2024-07-03T15:43:47.881 INFO:tasks.workunit.client.1.smithi148.stdout:5/219: truncate d7/da/df/d16/d2f/f38 96114 0 2024-07-03T15:43:47.881 INFO:tasks.workunit.client.1.smithi148.stdout:6/293: creat d3/d1d/d28/d3e/f64 x:0 0 0 2024-07-03T15:43:47.885 INFO:tasks.workunit.client.1.smithi148.stdout:9/433: rename d0/dd/d25/d6a/l7d to d0/l8e 0 2024-07-03T15:43:47.887 INFO:tasks.workunit.client.1.smithi148.stdout:7/299: mknod d5/d16/d27/c5a 0 2024-07-03T15:43:47.890 INFO:tasks.workunit.client.1.smithi148.stdout:2/300: truncate d5/d14/d22/d2b/d13/f29 648514 0 2024-07-03T15:43:47.890 INFO:tasks.workunit.client.1.smithi148.stdout:2/301: read - d5/d14/d22/d2b/d13/d2d/f4e zero size 2024-07-03T15:43:47.892 INFO:tasks.workunit.client.1.smithi148.stdout:5/220: link d7/da/df/d16/l3c d7/da/df/d16/d1a/l47 0 2024-07-03T15:43:47.893 INFO:tasks.workunit.client.1.smithi148.stdout:5/221: write d7/da/df/d1d/f2e [559172,95706] 0 2024-07-03T15:43:47.895 INFO:tasks.workunit.client.1.smithi148.stdout:6/294: symlink d3/d1d/d28/d31/l65 0 2024-07-03T15:43:47.895 INFO:tasks.workunit.client.1.smithi148.stdout:6/295: chown d3/d1d/d28/d3e/l41 22 1 2024-07-03T15:43:47.896 INFO:tasks.workunit.client.1.smithi148.stdout:7/300: symlink d5/d16/d1c/d33/d4c/d21/d31/l5b 0 2024-07-03T15:43:47.899 INFO:tasks.workunit.client.1.smithi148.stdout:2/302: mkdir d5/d14/d22/d2b/d68 0 2024-07-03T15:43:47.901 INFO:tasks.workunit.client.1.smithi148.stdout:5/222: link l0 d7/d13/l48 0 2024-07-03T15:43:47.903 INFO:tasks.workunit.client.1.smithi148.stdout:6/296: symlink d3/l66 0 2024-07-03T15:43:47.903 INFO:tasks.workunit.client.1.smithi148.stdout:6/297: dread - d3/d1d/d28/f5f zero size 2024-07-03T15:43:47.904 INFO:tasks.workunit.client.1.smithi148.stdout:6/298: fdatasync d3/f2c 0 2024-07-03T15:43:47.904 INFO:tasks.workunit.client.1.smithi148.stdout:9/434: dwrite d0/d18/d24/d5a/f5f [0,4194304] 0 2024-07-03T15:43:47.905 INFO:tasks.workunit.client.1.smithi148.stdout:2/303: mkdir d5/d14/d22/d2b/d13/d1d/d69 0 2024-07-03T15:43:47.907 INFO:tasks.workunit.client.1.smithi148.stdout:5/223: symlink d7/d13/d23/l49 0 2024-07-03T15:43:47.910 INFO:tasks.workunit.client.1.smithi148.stdout:6/299: mkdir d3/d1d/d28/d31/d4c/d67 0 2024-07-03T15:43:47.912 INFO:tasks.workunit.client.1.smithi148.stdout:9/435: dread d0/dd/ff [0,4194304] 0 2024-07-03T15:43:47.913 INFO:tasks.workunit.client.1.smithi148.stdout:9/436: stat d0/dd 0 2024-07-03T15:43:47.914 INFO:tasks.workunit.client.1.smithi148.stdout:7/301: dwrite d5/d16/d1c/d33/f42 [0,4194304] 0 2024-07-03T15:43:47.915 INFO:tasks.workunit.client.1.smithi148.stdout:2/304: mkdir d5/d26/d6a 0 2024-07-03T15:43:47.922 INFO:tasks.workunit.client.1.smithi148.stdout:7/302: mknod d5/d16/d1c/d23/c5c 0 2024-07-03T15:43:47.927 INFO:tasks.workunit.client.1.smithi148.stdout:7/303: creat d5/d16/d1c/d33/d4c/d21/d47/f5d x:0 0 0 2024-07-03T15:43:47.940 INFO:tasks.workunit.client.1.smithi148.stdout:6/300: dwrite d3/d1d/f38 [0,4194304] 0 2024-07-03T15:43:47.940 INFO:tasks.workunit.client.1.smithi148.stdout:6/301: readlink d3/ld 0 2024-07-03T15:43:47.940 INFO:tasks.workunit.client.1.smithi148.stdout:6/302: write d3/d1d/d28/d3e/f58 [988141,74025] 0 2024-07-03T15:43:47.940 INFO:tasks.workunit.client.1.smithi148.stdout:6/303: dread - d3/d1d/d28/f5f zero size 2024-07-03T15:43:47.940 INFO:tasks.workunit.client.1.smithi148.stdout:2/305: dwrite d5/d14/d22/d2b/d13/d2d/f51 [0,4194304] 0 2024-07-03T15:43:47.941 INFO:tasks.workunit.client.1.smithi148.stdout:7/304: fsync d5/f57 0 2024-07-03T15:43:47.943 INFO:tasks.workunit.client.1.smithi148.stdout:7/305: dread d5/fa [0,4194304] 0 2024-07-03T15:43:47.946 INFO:tasks.workunit.client.1.smithi148.stdout:2/306: rename d5/c23 to d5/d5f/c6b 0 2024-07-03T15:43:47.946 INFO:tasks.workunit.client.1.smithi148.stdout:7/306: write d5/d16/f34 [7584640,39144] 0 2024-07-03T15:43:47.947 INFO:tasks.workunit.client.1.smithi148.stdout:7/307: fsync d5/d16/d1c/d33/d4c/d21/d31/f45 0 2024-07-03T15:43:47.948 INFO:tasks.workunit.client.1.smithi148.stdout:6/304: dread d3/d1d/f24 [0,4194304] 0 2024-07-03T15:43:47.949 INFO:tasks.workunit.client.1.smithi148.stdout:2/307: truncate d5/d14/d22/d2b/f3a 2050908 0 2024-07-03T15:43:47.950 INFO:tasks.workunit.client.1.smithi148.stdout:7/308: mknod d5/d16/d27/c5e 0 2024-07-03T15:43:47.950 INFO:tasks.workunit.client.1.smithi148.stdout:7/309: fsync d5/fa 0 2024-07-03T15:43:47.952 INFO:tasks.workunit.client.1.smithi148.stdout:7/310: creat d5/d16/d1c/d4a/d52/f5f x:0 0 0 2024-07-03T15:43:47.954 INFO:tasks.workunit.client.1.smithi148.stdout:9/437: rmdir d0/d18/d24 39 2024-07-03T15:43:47.954 INFO:tasks.workunit.client.1.smithi148.stdout:9/438: truncate d0/dd/d25/d37/f50 1247341 0 2024-07-03T15:43:47.973 INFO:tasks.workunit.client.1.smithi148.stdout:3/232: dread dd/f22 [0,4194304] 0 2024-07-03T15:43:47.973 INFO:tasks.workunit.client.1.smithi148.stdout:3/233: chown dd/f2a 0 1 2024-07-03T15:43:47.973 INFO:tasks.workunit.client.1.smithi148.stdout:6/305: dwrite d3/d1d/d28/d31/d4c/f57 [0,4194304] 0 2024-07-03T15:43:47.974 INFO:tasks.workunit.client.1.smithi148.stdout:2/308: dwrite d5/f9 [4194304,4194304] 0 2024-07-03T15:43:47.983 INFO:tasks.workunit.client.1.smithi148.stdout:2/309: rmdir d5/d14/d22/d2b/d13/d1d/d69 0 2024-07-03T15:43:47.984 INFO:tasks.workunit.client.1.smithi148.stdout:6/306: dread d3/f1c [4194304,4194304] 0 2024-07-03T15:43:47.986 INFO:tasks.workunit.client.1.smithi148.stdout:2/310: creat d5/d14/d22/d60/f6c x:0 0 0 2024-07-03T15:43:47.988 INFO:tasks.workunit.client.1.smithi148.stdout:6/307: rmdir d3/d1d/d28/d31/d4c/d67 0 2024-07-03T15:43:47.989 INFO:tasks.workunit.client.1.smithi148.stdout:3/234: dwrite dd/f2a [0,4194304] 0 2024-07-03T15:43:47.989 INFO:tasks.workunit.client.1.smithi148.stdout:2/311: stat d5/d14/c46 0 2024-07-03T15:43:48.011 INFO:tasks.workunit.client.1.smithi148.stdout:2/312: dwrite d5/f1f [0,4194304] 0 2024-07-03T15:43:48.011 INFO:tasks.workunit.client.1.smithi148.stdout:3/235: dwrite dd/f15 [0,4194304] 0 2024-07-03T15:43:48.016 INFO:tasks.workunit.client.1.smithi148.stdout:2/313: creat d5/d14/d22/d2b/f6d x:0 0 0 2024-07-03T15:43:48.019 INFO:tasks.workunit.client.1.smithi148.stdout:4/171: sync 2024-07-03T15:43:48.020 INFO:tasks.workunit.client.1.smithi148.stdout:0/203: sync 2024-07-03T15:43:48.021 INFO:tasks.workunit.client.1.smithi148.stdout:3/236: creat dd/d1c/f3c x:0 0 0 2024-07-03T15:43:48.022 INFO:tasks.workunit.client.1.smithi148.stdout:2/314: symlink d5/d14/d22/d2b/d13/d41/l6e 0 2024-07-03T15:43:48.023 INFO:tasks.workunit.client.1.smithi148.stdout:2/315: fsync d5/d14/d22/f62 0 2024-07-03T15:43:48.025 INFO:tasks.workunit.client.1.smithi148.stdout:0/204: rename da/dc to da/d44 0 2024-07-03T15:43:48.027 INFO:tasks.workunit.client.1.smithi148.stdout:4/172: symlink d0/d8/d19/l34 0 2024-07-03T15:43:48.029 INFO:tasks.workunit.client.1.smithi148.stdout:2/316: dread d5/d14/d22/d2b/f16 [0,4194304] 0 2024-07-03T15:43:48.029 INFO:tasks.workunit.client.1.smithi148.stdout:2/317: chown d5/d14/d22/d2b/f1a 9484 1 2024-07-03T15:43:48.029 INFO:tasks.workunit.client.1.smithi148.stdout:2/318: fsync d5/d14/d22/d2b/f1e 0 2024-07-03T15:43:48.033 INFO:tasks.workunit.client.1.smithi148.stdout:3/237: rename dd/d1c/f25 to dd/f3d 0 2024-07-03T15:43:48.033 INFO:tasks.workunit.client.1.smithi148.stdout:3/238: truncate dd/d1c/f20 87853 0 2024-07-03T15:43:48.035 INFO:tasks.workunit.client.1.smithi148.stdout:0/205: creat da/d17/d39/f45 x:0 0 0 2024-07-03T15:43:48.036 INFO:tasks.workunit.client.1.smithi148.stdout:2/319: write d5/d14/d22/d2b/d13/d41/f4b [668637,1860] 0 2024-07-03T15:43:48.036 INFO:tasks.workunit.client.1.smithi148.stdout:4/173: mknod d0/d28/c35 0 2024-07-03T15:43:48.038 INFO:tasks.workunit.client.1.smithi148.stdout:3/239: rename dd/f23 to dd/d33/f3e 0 2024-07-03T15:43:48.038 INFO:tasks.workunit.client.1.smithi148.stdout:3/240: write dd/d33/f3a [193953,57655] 0 2024-07-03T15:43:48.038 INFO:tasks.workunit.client.1.smithi148.stdout:3/241: chown dd/d1c/d2e/f11 8605575 1 2024-07-03T15:43:48.039 INFO:tasks.workunit.client.1.smithi148.stdout:4/174: truncate d0/f17 3903461 0 2024-07-03T15:43:48.041 INFO:tasks.workunit.client.1.smithi148.stdout:2/320: read d5/d14/d22/d2b/d13/d2d/f3b [750412,21006] 0 2024-07-03T15:43:48.041 INFO:tasks.workunit.client.1.smithi148.stdout:2/321: readlink d5/lc 0 2024-07-03T15:43:48.041 INFO:tasks.workunit.client.1.smithi148.stdout:2/322: write d5/d7/f35 [608143,3298] 0 2024-07-03T15:43:48.041 INFO:tasks.workunit.client.1.smithi148.stdout:2/323: write d5/d7/f38 [849756,20527] 0 2024-07-03T15:43:48.041 INFO:tasks.workunit.client.1.smithi148.stdout:3/242: rmdir dd/d1c/d2e 39 2024-07-03T15:43:48.041 INFO:tasks.workunit.client.1.smithi148.stdout:2/324: stat d5/d14/d22/d2b/d13/c5c 0 2024-07-03T15:43:48.042 INFO:tasks.workunit.client.1.smithi148.stdout:0/206: dread da/d44/d24/f29 [0,4194304] 0 2024-07-03T15:43:48.042 INFO:tasks.workunit.client.1.smithi148.stdout:4/175: mkdir d0/d28/d2a/d36 0 2024-07-03T15:43:48.042 INFO:tasks.workunit.client.1.smithi148.stdout:4/176: fdatasync d0/d8/f32 0 2024-07-03T15:43:48.043 INFO:tasks.workunit.client.1.smithi148.stdout:2/325: mknod d5/d14/d22/d2b/c6f 0 2024-07-03T15:43:48.044 INFO:tasks.workunit.client.1.smithi148.stdout:3/243: creat dd/d35/f3f x:0 0 0 2024-07-03T15:43:48.045 INFO:tasks.workunit.client.1.smithi148.stdout:3/244: stat dd/d1c/d2e/f13 0 2024-07-03T15:43:48.047 INFO:tasks.workunit.client.1.smithi148.stdout:0/207: creat da/d44/d13/f46 x:0 0 0 2024-07-03T15:43:48.050 INFO:tasks.workunit.client.1.smithi148.stdout:4/177: symlink d0/d28/d2a/d36/l37 0 2024-07-03T15:43:48.050 INFO:tasks.workunit.client.1.smithi148.stdout:4/178: write d0/d8/f2d [299690,37337] 0 2024-07-03T15:43:48.050 INFO:tasks.workunit.client.1.smithi148.stdout:3/245: mkdir dd/d1c/d2e/d40 0 2024-07-03T15:43:48.053 INFO:tasks.workunit.client.1.smithi148.stdout:4/179: rename d0/d8/d22/f21 to d0/d8/f38 0 2024-07-03T15:43:48.053 INFO:tasks.workunit.client.1.smithi148.stdout:4/180: write d0/d8/f2d [445607,32598] 0 2024-07-03T15:43:48.064 INFO:tasks.workunit.client.1.smithi148.stdout:4/181: dread d0/d8/f38 [0,4194304] 0 2024-07-03T15:43:48.066 INFO:tasks.workunit.client.1.smithi148.stdout:2/326: dwrite d5/d14/d22/d2b/d13/f1b [4194304,4194304] 0 2024-07-03T15:43:48.069 INFO:tasks.workunit.client.1.smithi148.stdout:0/208: dwrite da/d44/f26 [0,4194304] 0 2024-07-03T15:43:48.070 INFO:tasks.workunit.client.1.smithi148.stdout:2/327: symlink d5/d14/d22/d2b/d13/d41/l70 0 2024-07-03T15:43:48.072 INFO:tasks.workunit.client.1.smithi148.stdout:0/209: rmdir da/d44/d13/d1d/d3e/d43 0 2024-07-03T15:43:48.073 INFO:tasks.workunit.client.1.smithi148.stdout:0/210: dread da/d44/d13/d1d/f34 [0,4194304] 0 2024-07-03T15:43:48.073 INFO:tasks.workunit.client.1.smithi148.stdout:2/328: creat d5/d7/d64/f71 x:0 0 0 2024-07-03T15:43:48.078 INFO:tasks.workunit.client.1.smithi148.stdout:0/211: rename da/d44/d13/d1d/f34 to da/d17/f47 0 2024-07-03T15:43:48.081 INFO:tasks.workunit.client.1.smithi148.stdout:0/212: creat da/d44/d13/d14/f48 x:0 0 0 2024-07-03T15:43:48.083 INFO:tasks.workunit.client.1.smithi148.stdout:8/203: sync 2024-07-03T15:43:48.084 INFO:tasks.workunit.client.1.smithi148.stdout:0/213: dread da/d17/f1e [0,4194304] 0 2024-07-03T15:43:48.085 INFO:tasks.workunit.client.1.smithi148.stdout:8/204: truncate d4/da/d14/d3a/f41 3493194 0 2024-07-03T15:43:48.094 INFO:tasks.workunit.client.1.smithi148.stdout:8/205: rename d4/d6/d18/d25 to d4/da/d14/d1a/d32/d38/d4a 0 2024-07-03T15:43:48.144 INFO:tasks.workunit.client.1.smithi148.stdout:2/329: dread d5/d14/d22/d2b/d13/f29 [0,4194304] 0 2024-07-03T15:43:48.145 INFO:tasks.workunit.client.1.smithi148.stdout:2/330: fdatasync d5/d14/d22/d2b/d13/d41/f63 0 2024-07-03T15:43:48.145 INFO:tasks.workunit.client.1.smithi148.stdout:2/331: fsync d5/d7/f3d 0 2024-07-03T15:43:48.153 INFO:tasks.workunit.client.1.smithi148.stdout:1/185: sync 2024-07-03T15:43:48.153 INFO:tasks.workunit.client.1.smithi148.stdout:1/186: fdatasync d0/f25 0 2024-07-03T15:43:48.155 INFO:tasks.workunit.client.1.smithi148.stdout:1/187: dread d0/d2/d6/d14/f15 [0,4194304] 0 2024-07-03T15:43:48.157 INFO:tasks.workunit.client.1.smithi148.stdout:1/188: mkdir d0/d2/d2e/d39/d3d 0 2024-07-03T15:43:48.157 INFO:tasks.workunit.client.1.smithi148.stdout:1/189: truncate d0/f24 1420348 0 2024-07-03T15:43:48.157 INFO:tasks.workunit.client.1.smithi148.stdout:1/190: write d0/d2/d2e/f36 [1920566,49585] 0 2024-07-03T15:43:48.157 INFO:tasks.workunit.client.1.smithi148.stdout:1/191: fsync d0/d2/d11/d1e/f3c 0 2024-07-03T15:43:48.159 INFO:tasks.workunit.client.1.smithi148.stdout:1/192: rmdir d0/d2/d6 39 2024-07-03T15:43:48.160 INFO:tasks.workunit.client.1.smithi148.stdout:2/332: dwrite d5/d7/f38 [0,4194304] 0 2024-07-03T15:43:48.160 INFO:tasks.workunit.client.1.smithi148.stdout:2/333: write d5/d7/d64/f71 [395207,20390] 0 2024-07-03T15:43:48.163 INFO:tasks.workunit.client.1.smithi148.stdout:1/193: rename d0/d2/d2e/d39/d3d to d0/d2/d11/d38/d3e 0 2024-07-03T15:43:48.169 INFO:tasks.workunit.client.1.smithi148.stdout:2/334: dread d5/d14/d22/d2b/d13/d41/f49 [0,4194304] 0 2024-07-03T15:43:48.171 INFO:tasks.workunit.client.1.smithi148.stdout:2/335: rmdir d5/d14/d22/d2b/d13/d1d 39 2024-07-03T15:43:48.173 INFO:tasks.workunit.client.1.smithi148.stdout:2/336: dread d5/d14/d22/d2b/d13/f29 [0,4194304] 0 2024-07-03T15:43:48.173 INFO:tasks.workunit.client.1.smithi148.stdout:2/337: dread d5/d26/f3e [0,4194304] 0 2024-07-03T15:43:48.173 INFO:tasks.workunit.client.1.smithi148.stdout:2/338: chown d5/d14/d22/d60 59 1 2024-07-03T15:43:48.173 INFO:tasks.workunit.client.1.smithi148.stdout:2/339: stat d5/d14/d22/d2b/d13/f1b 0 2024-07-03T15:43:48.181 INFO:tasks.workunit.client.1.smithi148.stdout:1/194: dwrite d0/d2/fd [0,4194304] 0 2024-07-03T15:43:48.182 INFO:tasks.workunit.client.1.smithi148.stdout:1/195: symlink d0/l3f 0 2024-07-03T15:43:48.185 INFO:tasks.workunit.client.1.smithi148.stdout:2/340: write d5/d14/d22/d2b/f1a [2018698,79555] 0 2024-07-03T15:43:48.188 INFO:tasks.workunit.client.1.smithi148.stdout:1/196: write d0/f25 [826091,105507] 0 2024-07-03T15:43:48.188 INFO:tasks.workunit.client.1.smithi148.stdout:2/341: rename d5/d14/d22/d2b/f16 to d5/d14/d22/f72 0 2024-07-03T15:43:48.188 INFO:tasks.workunit.client.1.smithi148.stdout:2/342: read - d5/d14/d22/d60/f53 zero size 2024-07-03T15:43:48.190 INFO:tasks.workunit.client.1.smithi148.stdout:7/311: getdents d5/d16/d27 0 2024-07-03T15:43:48.192 INFO:tasks.workunit.client.1.smithi148.stdout:1/197: mkdir d0/de/d2f/d40 0 2024-07-03T15:43:48.193 INFO:tasks.workunit.client.1.smithi148.stdout:2/343: link d5/d14/c55 d5/d7/c73 0 2024-07-03T15:43:48.197 INFO:tasks.workunit.client.1.smithi148.stdout:2/344: creat d5/d14/d22/d2b/d13/d41/f74 x:0 0 0 2024-07-03T15:43:48.197 INFO:tasks.workunit.client.1.smithi148.stdout:2/345: chown d5/d14/d22/d2b/f6d 119379 1 2024-07-03T15:43:48.199 INFO:tasks.workunit.client.1.smithi148.stdout:7/312: write d5/d16/f4d [3361988,129379] 0 2024-07-03T15:43:48.203 INFO:tasks.workunit.client.1.smithi148.stdout:7/313: mknod d5/d16/d1c/d33/d4c/d21/d31/c60 0 2024-07-03T15:43:48.205 INFO:tasks.workunit.client.1.smithi148.stdout:1/198: dread d0/d2/f10 [4194304,4194304] 0 2024-07-03T15:43:48.205 INFO:tasks.workunit.client.1.smithi148.stdout:1/199: chown d0 15002 1 2024-07-03T15:43:48.206 INFO:tasks.workunit.client.1.smithi148.stdout:7/314: symlink d5/d16/d1c/d33/d4c/d21/d39/l61 0 2024-07-03T15:43:48.206 INFO:tasks.workunit.client.1.smithi148.stdout:7/315: chown d5/d16/c17 895754 1 2024-07-03T15:43:48.206 INFO:tasks.workunit.client.1.smithi148.stdout:7/316: chown d5/d16/d1c/d33/d4c/d21/f32 565 1 2024-07-03T15:43:48.206 INFO:tasks.workunit.client.1.smithi148.stdout:7/317: fsync d5/d16/d1c/d33/d4c/d21/f32 0 2024-07-03T15:43:48.206 INFO:tasks.workunit.client.1.smithi148.stdout:7/318: chown d5/d16/d1c/d33/d4c/d21/d31/c3d 801108667 1 2024-07-03T15:43:48.209 INFO:tasks.workunit.client.1.smithi148.stdout:7/319: read d5/d16/d1c/d23/f2b [2892084,29410] 0 2024-07-03T15:43:48.209 INFO:tasks.workunit.client.1.smithi148.stdout:7/320: write d5/d16/d1c/d33/d4c/d21/d31/f45 [2137567,93751] 0 2024-07-03T15:43:48.214 INFO:tasks.workunit.client.1.smithi148.stdout:2/346: dwrite d5/d14/d22/d60/f53 [0,4194304] 0 2024-07-03T15:43:48.215 INFO:tasks.workunit.client.1.smithi148.stdout:7/321: creat d5/d16/d1c/d4a/f62 x:0 0 0 2024-07-03T15:43:48.229 INFO:tasks.workunit.client.1.smithi148.stdout:7/322: mkdir d5/d16/d1c/d59/d63 0 2024-07-03T15:43:48.229 INFO:tasks.workunit.client.1.smithi148.stdout:6/308: rmdir d3 39 2024-07-03T15:43:48.229 INFO:tasks.workunit.client.1.smithi148.stdout:6/309: chown d3/d1d/f25 16177 1 2024-07-03T15:43:48.232 INFO:tasks.workunit.client.1.smithi148.stdout:6/310: rename d3/f7 to d3/d1d/d28/d3e/f68 0 2024-07-03T15:43:48.234 INFO:tasks.workunit.client.1.smithi148.stdout:2/347: dwrite d5/d14/d22/d2b/d13/d41/f74 [0,4194304] 0 2024-07-03T15:43:48.234 INFO:tasks.workunit.client.1.smithi148.stdout:1/200: dwrite d0/d2/d6/f26 [4194304,4194304] 0 2024-07-03T15:43:48.235 INFO:tasks.workunit.client.1.smithi148.stdout:6/311: creat d3/d1d/d28/f69 x:0 0 0 2024-07-03T15:43:48.237 INFO:tasks.workunit.client.1.smithi148.stdout:7/323: read d5/d16/d1c/f41 [3219087,70826] 0 2024-07-03T15:43:48.238 INFO:tasks.workunit.client.1.smithi148.stdout:1/201: dread d0/d2/d6/f19 [0,4194304] 0 2024-07-03T15:43:48.239 INFO:tasks.workunit.client.1.smithi148.stdout:1/202: dread d0/d2/d6/f19 [0,4194304] 0 2024-07-03T15:43:48.241 INFO:tasks.workunit.client.1.smithi148.stdout:6/312: symlink d3/d50/l6a 0 2024-07-03T15:43:48.244 INFO:tasks.workunit.client.1.smithi148.stdout:7/324: getdents d5/d16/d1c/d33/d4c/d21 0 2024-07-03T15:43:48.244 INFO:tasks.workunit.client.1.smithi148.stdout:7/325: chown d5/d16/f37 1 1 2024-07-03T15:43:48.244 INFO:tasks.workunit.client.1.smithi148.stdout:7/326: getdents d5/d16/d1c/d33/d4c/d21 0 2024-07-03T15:43:48.245 INFO:tasks.workunit.client.1.smithi148.stdout:7/327: write d5/f1f [263434,127666] 0 2024-07-03T15:43:48.245 INFO:tasks.workunit.client.1.smithi148.stdout:1/203: mkdir d0/d2/d2e/d39/d41 0 2024-07-03T15:43:48.247 INFO:tasks.workunit.client.1.smithi148.stdout:6/313: rename d3/d1d/d28/d31/f3c to d3/d1d/d28/d31/d61/f6b 0 2024-07-03T15:43:48.250 INFO:tasks.workunit.client.1.smithi148.stdout:7/328: rmdir d5/d16/d1c/d59/d63 0 2024-07-03T15:43:48.250 INFO:tasks.workunit.client.1.smithi148.stdout:7/329: write d5/d16/d1c/f41 [7169540,130622] 0 2024-07-03T15:43:48.250 INFO:tasks.workunit.client.1.smithi148.stdout:2/348: dwrite d5/d14/d22/f62 [0,4194304] 0 2024-07-03T15:43:48.251 INFO:tasks.workunit.client.1.smithi148.stdout:1/204: mkdir d0/d2/d2e/d39/d42 0 2024-07-03T15:43:48.256 INFO:tasks.workunit.client.1.smithi148.stdout:7/330: rename d5/d16/l24 to d5/d16/d1c/d33/d4c/d21/d31/l64 0 2024-07-03T15:43:48.266 INFO:tasks.workunit.client.1.smithi148.stdout:1/205: creat d0/d2/d2e/d39/d41/f43 x:0 0 0 2024-07-03T15:43:48.269 INFO:tasks.workunit.client.1.smithi148.stdout:7/331: mkdir d5/d16/d27/d65 0 2024-07-03T15:43:48.282 INFO:tasks.workunit.client.1.smithi148.stdout:7/332: dwrite d5/d16/d1c/d33/d4c/d21/d47/f5d [0,4194304] 0 2024-07-03T15:43:48.287 INFO:tasks.workunit.client.1.smithi148.stdout:7/333: mkdir d5/d16/d1c/d4a/d66 0 2024-07-03T15:43:48.287 INFO:tasks.workunit.client.1.smithi148.stdout:7/334: write d5/d16/d1c/d4a/f62 [282130,3494] 0 2024-07-03T15:43:48.288 INFO:tasks.workunit.client.1.smithi148.stdout:7/335: chown d5/d16 24 1 2024-07-03T15:43:48.290 INFO:tasks.workunit.client.1.smithi148.stdout:7/336: truncate d5/d16/f34 4554922 0 2024-07-03T15:43:48.291 INFO:tasks.workunit.client.1.smithi148.stdout:7/337: read d5/d16/d1c/d33/d4c/f12 [111139,51073] 0 2024-07-03T15:43:48.293 INFO:tasks.workunit.client.1.smithi148.stdout:7/338: creat d5/d16/d1c/d33/f67 x:0 0 0 2024-07-03T15:43:48.297 INFO:tasks.workunit.client.1.smithi148.stdout:7/339: creat d5/d16/d1c/d23/f68 x:0 0 0 2024-07-03T15:43:48.297 INFO:tasks.workunit.client.1.smithi148.stdout:7/340: readlink d5/d16/d1c/d33/d4c/d21/d31/l64 0 2024-07-03T15:43:48.314 INFO:tasks.workunit.client.1.smithi148.stdout:7/341: dwrite d5/d16/d1c/d33/f67 [0,4194304] 0 2024-07-03T15:43:48.318 INFO:tasks.workunit.client.1.smithi148.stdout:0/214: fsync da/d17/f47 0 2024-07-03T15:43:48.320 INFO:tasks.workunit.client.1.smithi148.stdout:7/342: truncate d5/d16/f37 406096 0 2024-07-03T15:43:48.321 INFO:tasks.workunit.client.1.smithi148.stdout:0/215: unlink da/d44/d24/f30 0 2024-07-03T15:43:48.322 INFO:tasks.workunit.client.1.smithi148.stdout:0/216: truncate da/d44/d13/d1d/f21 4829664 0 2024-07-03T15:43:48.324 INFO:tasks.workunit.client.1.smithi148.stdout:7/343: dread d5/d16/d1c/d4a/f62 [0,4194304] 0 2024-07-03T15:43:48.324 INFO:tasks.workunit.client.1.smithi148.stdout:0/217: rename da/d44/d13/d14/f48 to da/d44/d13/d14/f49 0 2024-07-03T15:43:48.327 INFO:tasks.workunit.client.1.smithi148.stdout:7/344: mknod d5/d16/d1c/d4a/d66/c69 0 2024-07-03T15:43:48.330 INFO:tasks.workunit.client.1.smithi148.stdout:0/218: mkdir da/d44/d4a 0 2024-07-03T15:43:48.337 INFO:tasks.workunit.client.1.smithi148.stdout:5/224: sync 2024-07-03T15:43:48.339 INFO:tasks.workunit.client.1.smithi148.stdout:8/206: dwrite d4/da/d14/d3a/f41 [0,4194304] 0 2024-07-03T15:43:48.339 INFO:tasks.workunit.client.1.smithi148.stdout:8/207: creat d4/d6/d18/d1f/f4b x:0 0 0 2024-07-03T15:43:48.339 INFO:tasks.workunit.client.1.smithi148.stdout:8/208: chown d4/lf 3 1 2024-07-03T15:43:48.340 INFO:tasks.workunit.client.1.smithi148.stdout:5/225: mknod d7/d13/d23/c4a 0 2024-07-03T15:43:48.344 INFO:tasks.workunit.client.1.smithi148.stdout:8/209: unlink d4/da/d14/d1a/d32/d38/d4a/f46 0 2024-07-03T15:43:48.348 INFO:tasks.workunit.client.1.smithi148.stdout:8/210: creat d4/da/d14/f4c x:0 0 0 2024-07-03T15:43:48.349 INFO:tasks.workunit.client.1.smithi148.stdout:8/211: truncate d4/da/d14/d17/f2a 1796665 0 2024-07-03T15:43:48.349 INFO:tasks.workunit.client.1.smithi148.stdout:8/212: creat d4/da/d14/f4d x:0 0 0 2024-07-03T15:43:48.349 INFO:tasks.workunit.client.1.smithi148.stdout:8/213: write d4/da/fb [1004455,17008] 0 2024-07-03T15:43:48.349 INFO:tasks.workunit.client.1.smithi148.stdout:8/214: dread - d4/da/f29 zero size 2024-07-03T15:43:48.349 INFO:tasks.workunit.client.1.smithi148.stdout:8/215: truncate d4/d6/d18/d20/f21 19971 0 2024-07-03T15:43:48.349 INFO:tasks.workunit.client.1.smithi148.stdout:5/226: write f3 [437744,56837] 0 2024-07-03T15:43:48.349 INFO:tasks.workunit.client.1.smithi148.stdout:0/219: dwrite da/f11 [4194304,4194304] 0 2024-07-03T15:43:48.349 INFO:tasks.workunit.client.1.smithi148.stdout:0/220: fdatasync da/f3a 0 2024-07-03T15:43:48.349 INFO:tasks.workunit.client.1.smithi148.stdout:5/227: write d7/da/df/f3d [54646,54646] 0 2024-07-03T15:43:48.349 INFO:tasks.workunit.client.1.smithi148.stdout:5/228: write d7/da/df/f10 [4418828,63169] 0 2024-07-03T15:43:48.351 INFO:tasks.workunit.client.1.smithi148.stdout:8/216: mknod d4/da/d14/d17/c4e 0 2024-07-03T15:43:48.355 INFO:tasks.workunit.client.1.smithi148.stdout:0/221: truncate da/d44/d13/f27 3131352 0 2024-07-03T15:43:48.355 INFO:tasks.workunit.client.1.smithi148.stdout:0/222: truncate da/d44/ff 1548331 0 2024-07-03T15:43:48.357 INFO:tasks.workunit.client.1.smithi148.stdout:5/229: creat d7/d15/f4b x:0 0 0 2024-07-03T15:43:48.426 INFO:tasks.workunit.client.1.smithi148.stdout:1/206: rename d0/d2/d11/d38/d3e to d0/de/d44 0 2024-07-03T15:43:48.428 INFO:tasks.workunit.client.1.smithi148.stdout:1/207: creat d0/f45 x:0 0 0 2024-07-03T15:43:48.430 INFO:tasks.workunit.client.1.smithi148.stdout:1/208: mknod d0/d2/d6/c46 0 2024-07-03T15:43:48.444 INFO:tasks.workunit.client.1.smithi148.stdout:1/209: dwrite d0/d2/d6/d14/f15 [0,4194304] 0 2024-07-03T15:43:48.450 INFO:tasks.workunit.client.1.smithi148.stdout:1/210: rename d0/f8 to d0/d2/d11/f47 0 2024-07-03T15:43:48.456 INFO:tasks.workunit.client.1.smithi148.stdout:1/211: read d0/d2/d6/d14/f15 [3984886,25501] 0 2024-07-03T15:43:48.461 INFO:tasks.workunit.client.1.smithi148.stdout:2/349: getdents d5/d7 0 2024-07-03T15:43:48.461 INFO:tasks.workunit.client.1.smithi148.stdout:2/350: truncate d5/d14/d22/d2b/f42 693640 0 2024-07-03T15:43:48.464 INFO:tasks.workunit.client.1.smithi148.stdout:2/351: symlink d5/d7/d64/l75 0 2024-07-03T15:43:48.464 INFO:tasks.workunit.client.1.smithi148.stdout:2/352: write d5/d14/f5a [336320,77621] 0 2024-07-03T15:43:48.464 INFO:tasks.workunit.client.1.smithi148.stdout:2/353: getdents d5/d7 0 2024-07-03T15:43:48.464 INFO:tasks.workunit.client.1.smithi148.stdout:2/354: fsync d5/d14/d22/d60/f53 0 2024-07-03T15:43:48.466 INFO:tasks.workunit.client.1.smithi148.stdout:2/355: mkdir d5/d14/d22/d60/d76 0 2024-07-03T15:43:48.466 INFO:tasks.workunit.client.1.smithi148.stdout:2/356: dread - d5/d14/d22/d60/f65 zero size 2024-07-03T15:43:48.469 INFO:tasks.workunit.client.1.smithi148.stdout:2/357: creat d5/d26/d6a/f77 x:0 0 0 2024-07-03T15:43:48.469 INFO:tasks.workunit.client.1.smithi148.stdout:2/358: chown d5/d7/f38 49101 1 2024-07-03T15:43:48.469 INFO:tasks.workunit.client.1.smithi148.stdout:2/359: chown d5/d7/f35 1983 1 2024-07-03T15:43:48.481 INFO:tasks.workunit.client.1.smithi148.stdout:1/212: dwrite d0/d2/d2e/d39/d41/f43 [0,4194304] 0 2024-07-03T15:43:48.486 INFO:tasks.workunit.client.1.smithi148.stdout:2/360: dwrite d5/d14/d22/d2b/f3a [0,4194304] 0 2024-07-03T15:43:48.487 INFO:tasks.workunit.client.1.smithi148.stdout:9/439: sync 2024-07-03T15:43:48.489 INFO:tasks.workunit.client.1.smithi148.stdout:1/213: dread d0/f25 [0,4194304] 0 2024-07-03T15:43:48.493 INFO:tasks.workunit.client.1.smithi148.stdout:1/214: dread d0/f25 [0,4194304] 0 2024-07-03T15:43:48.493 INFO:tasks.workunit.client.1.smithi148.stdout:1/215: read d0/f1f [3560813,5659] 0 2024-07-03T15:43:48.495 INFO:tasks.workunit.client.1.smithi148.stdout:2/361: dread d5/d14/d22/d2b/f3a [0,4194304] 0 2024-07-03T15:43:48.497 INFO:tasks.workunit.client.1.smithi148.stdout:9/440: mknod d0/d49/d43/d19/d8c/c8f 0 2024-07-03T15:43:48.503 INFO:tasks.workunit.client.1.smithi148.stdout:1/216: dread d0/d2/d2e/f36 [0,4194304] 0 2024-07-03T15:43:48.514 INFO:tasks.workunit.client.1.smithi148.stdout:9/441: dwrite d0/f53 [0,4194304] 0 2024-07-03T15:43:48.514 INFO:tasks.workunit.client.1.smithi148.stdout:2/362: dwrite d5/f1f [0,4194304] 0 2024-07-03T15:43:48.518 INFO:tasks.workunit.client.1.smithi148.stdout:2/363: dread d5/d14/d22/d2b/d13/d41/f49 [0,4194304] 0 2024-07-03T15:43:48.518 INFO:tasks.workunit.client.1.smithi148.stdout:3/246: sync 2024-07-03T15:43:48.519 INFO:tasks.workunit.client.1.smithi148.stdout:9/442: link d0/dd/d25/d37/d40/d72/f8d d0/dd/f90 0 2024-07-03T15:43:48.521 INFO:tasks.workunit.client.1.smithi148.stdout:3/247: rmdir dd 39 2024-07-03T15:43:48.521 INFO:tasks.workunit.client.1.smithi148.stdout:3/248: write dd/f19 [451768,70256] 0 2024-07-03T15:43:48.522 INFO:tasks.workunit.client.1.smithi148.stdout:4/182: sync 2024-07-03T15:43:48.522 INFO:tasks.workunit.client.1.smithi148.stdout:4/183: chown d0/f16 759600196 1 2024-07-03T15:43:48.522 INFO:tasks.workunit.client.1.smithi148.stdout:4/184: truncate d0/d8/d19/d24/f2e 864556 0 2024-07-03T15:43:48.524 INFO:tasks.workunit.client.1.smithi148.stdout:3/249: mkdir dd/d35/d41 0 2024-07-03T15:43:48.540 INFO:tasks.workunit.client.1.smithi148.stdout:9/443: dwrite d0/dd/f90 [0,4194304] 0 2024-07-03T15:43:48.540 INFO:tasks.workunit.client.1.smithi148.stdout:9/444: truncate d0/dd/d25/d37/d40/f62 957322 0 2024-07-03T15:43:48.541 INFO:tasks.workunit.client.1.smithi148.stdout:4/185: dwrite d0/f17 [0,4194304] 0 2024-07-03T15:43:48.541 INFO:tasks.workunit.client.1.smithi148.stdout:4/186: stat d0/d8/d13 0 2024-07-03T15:43:48.547 INFO:tasks.workunit.client.1.smithi148.stdout:4/187: symlink d0/d8/d13/l39 0 2024-07-03T15:43:48.552 INFO:tasks.workunit.client.1.smithi148.stdout:4/188: getdents d0/d8 0 2024-07-03T15:43:48.552 INFO:tasks.workunit.client.1.smithi148.stdout:4/189: truncate d0/d8/f10 5650905 0 2024-07-03T15:43:48.554 INFO:tasks.workunit.client.1.smithi148.stdout:9/445: dwrite d0/d18/d24/d5a/f5f [4194304,4194304] 0 2024-07-03T15:43:48.554 INFO:tasks.workunit.client.1.smithi148.stdout:9/446: write d0/dd/d25/d59/f4f [188995,102] 0 2024-07-03T15:43:48.554 INFO:tasks.workunit.client.1.smithi148.stdout:9/447: fsync d0/dd/f2b 0 2024-07-03T15:43:48.556 INFO:tasks.workunit.client.1.smithi148.stdout:4/190: truncate d0/f5 1918007 0 2024-07-03T15:43:48.556 INFO:tasks.workunit.client.1.smithi148.stdout:9/448: dread d0/dd/d25/d37/d40/f62 [0,4194304] 0 2024-07-03T15:43:48.556 INFO:tasks.workunit.client.1.smithi148.stdout:9/449: truncate d0/dd/f14 9072381 0 2024-07-03T15:43:48.573 INFO:tasks.workunit.client.1.smithi148.stdout:4/191: dwrite d0/d28/f31 [0,4194304] 0 2024-07-03T15:43:48.573 INFO:tasks.workunit.client.1.smithi148.stdout:4/192: write d0/d8/d19/d24/f2c [543462,9159] 0 2024-07-03T15:43:48.574 INFO:tasks.workunit.client.1.smithi148.stdout:4/193: fdatasync d0/d8/d13/f1a 0 2024-07-03T15:43:48.574 INFO:tasks.workunit.client.1.smithi148.stdout:9/450: dwrite d0/dd/d29/f5e [0,4194304] 0 2024-07-03T15:43:48.574 INFO:tasks.workunit.client.1.smithi148.stdout:9/451: stat d0/d18/d24/f30 0 2024-07-03T15:43:48.574 INFO:tasks.workunit.client.1.smithi148.stdout:9/452: write d0/d18/d24/f30 [20668,2602] 0 2024-07-03T15:43:48.574 INFO:tasks.workunit.client.1.smithi148.stdout:4/194: write d0/d8/f2d [22051,1150] 0 2024-07-03T15:43:48.574 INFO:tasks.workunit.client.1.smithi148.stdout:4/195: fsync d0/d8/f10 0 2024-07-03T15:43:48.584 INFO:tasks.workunit.client.1.smithi148.stdout:9/453: rename d0/dd/f1a to d0/d18/d24/f91 0 2024-07-03T15:43:48.592 INFO:tasks.workunit.client.1.smithi148.stdout:4/196: rename d0/d28/d2a to d0/d8/d3a 0 2024-07-03T15:43:48.592 INFO:tasks.workunit.client.1.smithi148.stdout:4/197: fsync d0/d8/d19/d24/f2e 0 2024-07-03T15:43:48.601 INFO:tasks.workunit.client.1.smithi148.stdout:9/454: dwrite d0/d49/d43/f2c [0,4194304] 0 2024-07-03T15:43:48.608 INFO:tasks.workunit.client.1.smithi148.stdout:4/198: rename d0/d8/l9 to d0/d8/d19/l3b 0 2024-07-03T15:43:48.614 INFO:tasks.workunit.client.1.smithi148.stdout:9/455: symlink d0/d49/d43/d19/d7a/l92 0 2024-07-03T15:43:48.615 INFO:tasks.workunit.client.1.smithi148.stdout:4/199: dread d0/f16 [0,4194304] 0 2024-07-03T15:43:48.625 INFO:tasks.workunit.client.1.smithi148.stdout:9/456: write d0/dd/d29/f5e [2119420,113770] 0 2024-07-03T15:43:48.625 INFO:tasks.workunit.client.1.smithi148.stdout:9/457: chown d0/l27 871 1 2024-07-03T15:43:48.627 INFO:tasks.workunit.client.1.smithi148.stdout:0/223: dread da/d44/d13/f27 [0,4194304] 0 2024-07-03T15:43:48.627 INFO:tasks.workunit.client.1.smithi148.stdout:0/224: fsync da/d44/ff 0 2024-07-03T15:43:48.627 INFO:tasks.workunit.client.1.smithi148.stdout:0/225: chown da/d36 5 1 2024-07-03T15:43:48.627 INFO:tasks.workunit.client.1.smithi148.stdout:0/226: creat da/d17/d2a/f4b x:0 0 0 2024-07-03T15:43:48.631 INFO:tasks.workunit.client.1.smithi148.stdout:9/458: unlink d0/dd/d29/c2e 0 2024-07-03T15:43:48.631 INFO:tasks.workunit.client.1.smithi148.stdout:9/459: write d0/d2f/f41 [811311,55971] 0 2024-07-03T15:43:48.632 INFO:tasks.workunit.client.1.smithi148.stdout:9/460: write d0/d2f/f74 [701226,111776] 0 2024-07-03T15:43:48.634 INFO:tasks.workunit.client.1.smithi148.stdout:9/461: dread d0/d18/f26 [0,4194304] 0 2024-07-03T15:43:48.634 INFO:tasks.workunit.client.1.smithi148.stdout:9/462: creat d0/dd/d11/f93 x:0 0 0 2024-07-03T15:43:48.635 INFO:tasks.workunit.client.1.smithi148.stdout:9/463: chown d0/dd/d29/l45 6602061 1 2024-07-03T15:43:48.635 INFO:tasks.workunit.client.1.smithi148.stdout:0/227: mknod da/d38/c4c 0 2024-07-03T15:43:48.639 INFO:tasks.workunit.client.1.smithi148.stdout:0/228: dread da/d17/f1e [0,4194304] 0 2024-07-03T15:43:48.647 INFO:tasks.workunit.client.1.smithi148.stdout:0/229: getdents da/d44/d24 0 2024-07-03T15:43:48.648 INFO:tasks.workunit.client.1.smithi148.stdout:0/230: chown da/d38/c40 23 1 2024-07-03T15:43:48.648 INFO:tasks.workunit.client.1.smithi148.stdout:0/231: write da/d44/d13/f46 [196940,107022] 0 2024-07-03T15:43:48.648 INFO:tasks.workunit.client.1.smithi148.stdout:0/232: dread - da/d17/d2a/d2f/f37 zero size 2024-07-03T15:43:48.648 INFO:tasks.workunit.client.1.smithi148.stdout:0/233: creat da/d17/d39/f4d x:0 0 0 2024-07-03T15:43:48.648 INFO:tasks.workunit.client.1.smithi148.stdout:0/234: readlink da/d17/l3c 0 2024-07-03T15:43:48.648 INFO:tasks.workunit.client.1.smithi148.stdout:0/235: write da/d17/f47 [671369,34903] 0 2024-07-03T15:43:48.651 INFO:tasks.workunit.client.1.smithi148.stdout:9/464: dwrite d0/f53 [0,4194304] 0 2024-07-03T15:43:48.651 INFO:tasks.workunit.client.1.smithi148.stdout:9/465: readlink d0/l10 0 2024-07-03T15:43:48.651 INFO:tasks.workunit.client.1.smithi148.stdout:9/466: write d0/dd/fe [4257453,129929] 0 2024-07-03T15:43:48.651 INFO:tasks.workunit.client.1.smithi148.stdout:9/467: creat d0/d2f/f94 x:0 0 0 2024-07-03T15:43:48.651 INFO:tasks.workunit.client.1.smithi148.stdout:9/468: write d0/dd/d25/d77/f89 [393608,116351] 0 2024-07-03T15:43:48.652 INFO:tasks.workunit.client.1.smithi148.stdout:0/236: rename da/f3a to da/d38/f4e 0 2024-07-03T15:43:48.652 INFO:tasks.workunit.client.1.smithi148.stdout:0/237: chown da/d44/d13/f46 96 1 2024-07-03T15:43:48.657 INFO:tasks.workunit.client.1.smithi148.stdout:0/238: write da/d44/f19 [4086579,77819] 0 2024-07-03T15:43:48.660 INFO:tasks.workunit.client.1.smithi148.stdout:0/239: dread da/d44/d13/d14/f32 [0,4194304] 0 2024-07-03T15:43:48.666 INFO:tasks.workunit.client.1.smithi148.stdout:9/469: creat d0/d49/d43/d19/d7a/f95 x:0 0 0 2024-07-03T15:43:48.668 INFO:tasks.workunit.client.1.smithi148.stdout:9/470: dread d0/d49/d43/d19/f1e [0,4194304] 0 2024-07-03T15:43:48.668 INFO:tasks.workunit.client.1.smithi148.stdout:9/471: creat d0/dd/d25/d77/f96 x:0 0 0 2024-07-03T15:43:48.670 INFO:tasks.workunit.client.1.smithi148.stdout:0/240: creat da/d17/f4f x:0 0 0 2024-07-03T15:43:48.672 INFO:tasks.workunit.client.1.smithi148.stdout:9/472: truncate d0/dd/f16 2045125 0 2024-07-03T15:43:48.675 INFO:tasks.workunit.client.1.smithi148.stdout:0/241: mkdir da/d44/d13/d14/d50 0 2024-07-03T15:43:48.681 INFO:tasks.workunit.client.1.smithi148.stdout:9/473: write d0/d49/d43/f2c [3622379,118455] 0 2024-07-03T15:43:48.683 INFO:tasks.workunit.client.1.smithi148.stdout:0/242: dread da/d17/d2a/f2e [0,4194304] 0 2024-07-03T15:43:48.684 INFO:tasks.workunit.client.1.smithi148.stdout:9/474: truncate d0/d18/f69 771866 0 2024-07-03T15:43:48.687 INFO:tasks.workunit.client.1.smithi148.stdout:9/475: rename d0/d49/d43/d19/d8c/c8f to d0/d49/c97 0 2024-07-03T15:43:48.690 INFO:tasks.workunit.client.1.smithi148.stdout:9/476: dread d0/dd/d25/d6a/f6e [0,4194304] 0 2024-07-03T15:43:48.692 INFO:tasks.workunit.client.1.smithi148.stdout:9/477: dread d0/dd/d25/d37/d40/f62 [0,4194304] 0 2024-07-03T15:43:48.693 INFO:tasks.workunit.client.1.smithi148.stdout:0/243: read da/d44/d13/d14/f1f [3614843,83773] 0 2024-07-03T15:43:48.694 INFO:tasks.workunit.client.1.smithi148.stdout:9/478: creat d0/d63/f98 x:0 0 0 2024-07-03T15:43:48.696 INFO:tasks.workunit.client.1.smithi148.stdout:9/479: dread d0/d18/f26 [0,4194304] 0 2024-07-03T15:43:48.697 INFO:tasks.workunit.client.1.smithi148.stdout:0/244: unlink da/d44/d13/d1d/f21 0 2024-07-03T15:43:48.697 INFO:tasks.workunit.client.1.smithi148.stdout:0/245: write da/d44/d13/d14/f1f [943923,102684] 0 2024-07-03T15:43:48.698 INFO:tasks.workunit.client.1.smithi148.stdout:9/480: mknod d0/dd/d25/d77/c99 0 2024-07-03T15:43:48.698 INFO:tasks.workunit.client.1.smithi148.stdout:9/481: fsync d0/dd/ff 0 2024-07-03T15:43:48.698 INFO:tasks.workunit.client.1.smithi148.stdout:9/482: dread - d0/dd/d11/f79 zero size 2024-07-03T15:43:48.703 INFO:tasks.workunit.client.1.smithi148.stdout:0/246: rename da/d17/l3c to da/d44/d13/l51 0 2024-07-03T15:43:48.705 INFO:tasks.workunit.client.1.smithi148.stdout:9/483: write d0/d18/d24/d5a/f5f [1085696,40704] 0 2024-07-03T15:43:48.705 INFO:tasks.workunit.client.1.smithi148.stdout:0/247: creat da/d44/d24/f52 x:0 0 0 2024-07-03T15:43:48.709 INFO:tasks.workunit.client.1.smithi148.stdout:0/248: creat da/d44/d13/d14/d50/f53 x:0 0 0 2024-07-03T15:43:48.713 INFO:tasks.workunit.client.1.smithi148.stdout:0/249: write da/f28 [4006704,25475] 0 2024-07-03T15:43:48.719 INFO:tasks.workunit.client.1.smithi148.stdout:6/314: sync 2024-07-03T15:43:48.722 INFO:tasks.workunit.client.1.smithi148.stdout:6/315: mknod d3/c6c 0 2024-07-03T15:43:48.722 INFO:tasks.workunit.client.1.smithi148.stdout:6/316: write d3/d1d/d28/f2e [4762487,50517] 0 2024-07-03T15:43:48.728 INFO:tasks.workunit.client.1.smithi148.stdout:6/317: mkdir d3/d1d/d28/d53/d6d 0 2024-07-03T15:43:48.730 INFO:tasks.workunit.client.1.smithi148.stdout:6/318: creat d3/d1d/d28/d53/f6e x:0 0 0 2024-07-03T15:43:48.733 INFO:tasks.workunit.client.1.smithi148.stdout:6/319: unlink d3/c15 0 2024-07-03T15:43:48.733 INFO:tasks.workunit.client.1.smithi148.stdout:6/320: stat d3/f55 0 2024-07-03T15:43:48.746 INFO:tasks.workunit.client.1.smithi148.stdout:8/217: sync 2024-07-03T15:43:48.747 INFO:tasks.workunit.client.1.smithi148.stdout:7/345: sync 2024-07-03T15:43:48.747 INFO:tasks.workunit.client.1.smithi148.stdout:5/230: sync 2024-07-03T15:43:48.747 INFO:tasks.workunit.client.1.smithi148.stdout:5/231: creat d7/da/f4c x:0 0 0 2024-07-03T15:43:48.747 INFO:tasks.workunit.client.1.smithi148.stdout:5/232: creat d7/da/df/f4d x:0 0 0 2024-07-03T15:43:48.749 INFO:tasks.workunit.client.1.smithi148.stdout:7/346: link d5/d16/d1c/d33/d4c/d21/d47/f5d d5/d16/f6a 0 2024-07-03T15:43:48.753 INFO:tasks.workunit.client.1.smithi148.stdout:5/233: creat d7/da/d21/f4e x:0 0 0 2024-07-03T15:43:48.755 INFO:tasks.workunit.client.1.smithi148.stdout:7/347: dread d5/d16/f6a [0,4194304] 0 2024-07-03T15:43:48.756 INFO:tasks.workunit.client.1.smithi148.stdout:7/348: creat d5/d16/d1c/d4a/d52/f6b x:0 0 0 2024-07-03T15:43:48.756 INFO:tasks.workunit.client.1.smithi148.stdout:7/349: fdatasync d5/d16/d1c/f41 0 2024-07-03T15:43:48.756 INFO:tasks.workunit.client.1.smithi148.stdout:7/350: write d5/d16/d1c/d33/d4c/d21/d39/f3e [5136124,17226] 0 2024-07-03T15:43:48.756 INFO:tasks.workunit.client.1.smithi148.stdout:7/351: write d5/d16/d1c/d33/d4c/f12 [245625,126489] 0 2024-07-03T15:43:48.761 INFO:tasks.workunit.client.1.smithi148.stdout:5/234: write d7/f19 [2925485,38034] 0 2024-07-03T15:43:48.772 INFO:tasks.workunit.client.1.smithi148.stdout:3/250: getdents dd/d35 0 2024-07-03T15:43:48.774 INFO:tasks.workunit.client.1.smithi148.stdout:3/251: symlink dd/d33/d37/l42 0 2024-07-03T15:43:48.775 INFO:tasks.workunit.client.1.smithi148.stdout:3/252: stat dd/c1b 0 2024-07-03T15:43:48.775 INFO:tasks.workunit.client.1.smithi148.stdout:3/253: stat dd/d1c/d2e 0 2024-07-03T15:43:48.780 INFO:tasks.workunit.client.1.smithi148.stdout:5/235: dwrite d7/da/df/f33 [0,4194304] 0 2024-07-03T15:43:48.780 INFO:tasks.workunit.client.1.smithi148.stdout:5/236: chown d7/da/df/d1d 22 1 2024-07-03T15:43:48.784 INFO:tasks.workunit.client.1.smithi148.stdout:3/254: write dd/f15 [1420528,44989] 0 2024-07-03T15:43:48.788 INFO:tasks.workunit.client.1.smithi148.stdout:3/255: mknod dd/d35/c43 0 2024-07-03T15:43:48.793 INFO:tasks.workunit.client.1.smithi148.stdout:3/256: dread dd/f34 [0,4194304] 0 2024-07-03T15:43:48.793 INFO:tasks.workunit.client.1.smithi148.stdout:3/257: chown dd/d35/d41 273 1 2024-07-03T15:43:48.795 INFO:tasks.workunit.client.1.smithi148.stdout:3/258: creat dd/f44 x:0 0 0 2024-07-03T15:43:48.798 INFO:tasks.workunit.client.1.smithi148.stdout:3/259: mkdir dd/d1c/d2e/d40/d45 0 2024-07-03T15:43:48.798 INFO:tasks.workunit.client.1.smithi148.stdout:3/260: dread - dd/f3d zero size 2024-07-03T15:43:48.798 INFO:tasks.workunit.client.1.smithi148.stdout:3/261: truncate dd/d1c/f20 89121 0 2024-07-03T15:43:48.798 INFO:tasks.workunit.client.1.smithi148.stdout:3/262: chown dd/f34 13 1 2024-07-03T15:43:48.798 INFO:tasks.workunit.client.1.smithi148.stdout:3/263: chown dd/d1c/d2e/f13 5023453 1 2024-07-03T15:43:48.801 INFO:tasks.workunit.client.1.smithi148.stdout:3/264: creat dd/d33/d37/f46 x:0 0 0 2024-07-03T15:43:48.803 INFO:tasks.workunit.client.1.smithi148.stdout:3/265: creat dd/d1c/d2e/d40/f47 x:0 0 0 2024-07-03T15:43:48.805 INFO:tasks.workunit.client.1.smithi148.stdout:3/266: mknod dd/d35/d41/c48 0 2024-07-03T15:43:48.807 INFO:tasks.workunit.client.1.smithi148.stdout:3/267: mkdir dd/d35/d49 0 2024-07-03T15:43:48.822 INFO:tasks.workunit.client.1.smithi148.stdout:3/268: dwrite dd/d1c/f1f [0,4194304] 0 2024-07-03T15:43:48.822 INFO:tasks.workunit.client.1.smithi148.stdout:3/269: write dd/d1c/f3c [192240,89757] 0 2024-07-03T15:43:48.822 INFO:tasks.workunit.client.1.smithi148.stdout:3/270: readlink dd/d33/d37/l3b 0 2024-07-03T15:43:48.822 INFO:tasks.workunit.client.1.smithi148.stdout:3/271: truncate dd/f2b 51114 0 2024-07-03T15:43:48.825 INFO:tasks.workunit.client.1.smithi148.stdout:3/272: symlink dd/d35/d49/l4a 0 2024-07-03T15:43:48.890 INFO:tasks.workunit.client.1.smithi148.stdout:9/484: rmdir d0/d63 39 2024-07-03T15:43:48.907 INFO:tasks.workunit.client.1.smithi148.stdout:6/321: truncate d3/d1d/d28/f2e 2103347 0 2024-07-03T15:43:48.907 INFO:tasks.workunit.client.1.smithi148.stdout:6/322: dread d3/f37 [0,4194304] 0 2024-07-03T15:43:48.908 INFO:tasks.workunit.client.1.smithi148.stdout:6/323: dread - d3/d1d/d28/f69 zero size 2024-07-03T15:43:48.910 INFO:tasks.workunit.client.1.smithi148.stdout:6/324: mknod d3/d1d/c6f 0 2024-07-03T15:43:48.910 INFO:tasks.workunit.client.1.smithi148.stdout:6/325: chown d3/d1d/d28/d3e/l41 2831397 1 2024-07-03T15:43:48.913 INFO:tasks.workunit.client.1.smithi148.stdout:9/485: dwrite d0/dd/d25/d6a/f6e [0,4194304] 0 2024-07-03T15:43:48.924 INFO:tasks.workunit.client.1.smithi148.stdout:7/352: getdents d5/d16 0 2024-07-03T15:43:48.924 INFO:tasks.workunit.client.1.smithi148.stdout:7/353: stat d5/d16/d1c/d23 0 2024-07-03T15:43:48.924 INFO:tasks.workunit.client.1.smithi148.stdout:7/354: chown d5/d16/d1c/d33/d4c/ld 20 1 2024-07-03T15:43:48.929 INFO:tasks.workunit.client.1.smithi148.stdout:7/355: rmdir d5/d16/d1c/d33/d4c/d21/d47 39 2024-07-03T15:43:48.931 INFO:tasks.workunit.client.1.smithi148.stdout:7/356: unlink d5/d16/d1c/d33/d4c/d21/d31/c3d 0 2024-07-03T15:43:48.933 INFO:tasks.workunit.client.1.smithi148.stdout:6/326: dwrite d3/d1d/f27 [0,4194304] 0 2024-07-03T15:43:48.933 INFO:tasks.workunit.client.1.smithi148.stdout:9/486: dwrite d0/d18/f21 [8388608,4194304] 0 2024-07-03T15:43:48.936 INFO:tasks.workunit.client.1.smithi148.stdout:7/357: dread d5/d16/d1c/d33/d4c/d21/d31/f45 [0,4194304] 0 2024-07-03T15:43:48.940 INFO:tasks.workunit.client.1.smithi148.stdout:6/327: creat d3/d5b/f70 x:0 0 0 2024-07-03T15:43:48.941 INFO:tasks.workunit.client.1.smithi148.stdout:6/328: write d3/f34 [21668,37878] 0 2024-07-03T15:43:48.941 INFO:tasks.workunit.client.1.smithi148.stdout:6/329: write d3/d1d/f4e [1668411,126351] 0 2024-07-03T15:43:48.943 INFO:tasks.workunit.client.1.smithi148.stdout:9/487: mknod d0/d18/d24/d60/c9a 0 2024-07-03T15:43:48.943 INFO:tasks.workunit.client.1.smithi148.stdout:9/488: creat d0/d49/d43/d19/d7a/f9b x:0 0 0 2024-07-03T15:43:48.943 INFO:tasks.workunit.client.1.smithi148.stdout:9/489: creat d0/dd/d11/f9c x:0 0 0 2024-07-03T15:43:48.943 INFO:tasks.workunit.client.1.smithi148.stdout:9/490: creat d0/dd/d25/d37/d40/d72/f9d x:0 0 0 2024-07-03T15:43:48.954 INFO:tasks.workunit.client.1.smithi148.stdout:9/491: link d0/l8e d0/dd/d25/d59/l9e 0 2024-07-03T15:43:48.954 INFO:tasks.workunit.client.1.smithi148.stdout:9/492: write d0/d49/f73 [877351,109587] 0 2024-07-03T15:43:48.956 INFO:tasks.workunit.client.1.smithi148.stdout:9/493: truncate d0/dd/d11/f35 2540126 0 2024-07-03T15:43:48.958 INFO:tasks.workunit.client.1.smithi148.stdout:9/494: dread d0/d2f/f41 [0,4194304] 0 2024-07-03T15:43:48.961 INFO:tasks.workunit.client.1.smithi148.stdout:6/330: dwrite d3/d1d/d28/d53/f59 [0,4194304] 0 2024-07-03T15:43:48.968 INFO:tasks.workunit.client.1.smithi148.stdout:6/331: creat d3/d1d/d28/d31/d61/f71 x:0 0 0 2024-07-03T15:43:48.969 INFO:tasks.workunit.client.1.smithi148.stdout:6/332: chown d3/d1d/d28/d31/d4c/f5d 125268 1 2024-07-03T15:43:48.969 INFO:tasks.workunit.client.1.smithi148.stdout:9/495: dread d0/f6 [0,4194304] 0 2024-07-03T15:43:48.970 INFO:tasks.workunit.client.1.smithi148.stdout:9/496: dread d0/d18/f26 [0,4194304] 0 2024-07-03T15:43:48.971 INFO:tasks.workunit.client.1.smithi148.stdout:6/333: link d3/d1d/d28/d3e/f58 d3/d50/f72 0 2024-07-03T15:43:48.974 INFO:tasks.workunit.client.1.smithi148.stdout:9/497: mkdir d0/dd/d9f 0 2024-07-03T15:43:48.977 INFO:tasks.workunit.client.1.smithi148.stdout:6/334: link d3/d1d/d28/d31/f43 d3/d1d/d28/f73 0 2024-07-03T15:43:48.979 INFO:tasks.workunit.client.1.smithi148.stdout:9/498: symlink d0/d49/la0 0 2024-07-03T15:43:48.979 INFO:tasks.workunit.client.1.smithi148.stdout:9/499: write d0/dd/d25/d77/f96 [138240,4939] 0 2024-07-03T15:43:48.979 INFO:tasks.workunit.client.1.smithi148.stdout:9/500: write d0/d18/f69 [1789703,4900] 0 2024-07-03T15:43:48.980 INFO:tasks.workunit.client.1.smithi148.stdout:6/335: mkdir d3/d5b/d74 0 2024-07-03T15:43:48.980 INFO:tasks.workunit.client.1.smithi148.stdout:6/336: fdatasync d3/d1d/f27 0 2024-07-03T15:43:48.985 INFO:tasks.workunit.client.1.smithi148.stdout:9/501: creat d0/dd/d25/d59/fa1 x:0 0 0 2024-07-03T15:43:48.986 INFO:tasks.workunit.client.1.smithi148.stdout:9/502: stat d0/f6 0 2024-07-03T15:43:48.992 INFO:tasks.workunit.client.1.smithi148.stdout:9/503: rename d0/d18/l3f to d0/d49/d43/d19/d7a/la2 0 2024-07-03T15:43:48.995 INFO:tasks.workunit.client.1.smithi148.stdout:9/504: mkdir d0/dd/d25/d37/d40/da3 0 2024-07-03T15:43:48.996 INFO:tasks.workunit.client.1.smithi148.stdout:9/505: chown d0/dd/d25/d77/f89 112356 1 2024-07-03T15:43:48.996 INFO:tasks.workunit.client.1.smithi148.stdout:9/506: fsync d0/d49/d43/d19/f48 0 2024-07-03T15:43:48.999 INFO:tasks.workunit.client.1.smithi148.stdout:9/507: unlink d0/dd/d25/d59/fa1 0 2024-07-03T15:43:49.004 INFO:tasks.workunit.client.1.smithi148.stdout:9/508: unlink d0/l8e 0 2024-07-03T15:43:49.004 INFO:tasks.workunit.client.1.smithi148.stdout:9/509: fsync d0/d18/f69 0 2024-07-03T15:43:49.021 INFO:tasks.workunit.client.1.smithi148.stdout:9/510: dwrite d0/d49/d43/f5b [0,4194304] 0 2024-07-03T15:43:49.038 INFO:tasks.workunit.client.1.smithi148.stdout:9/511: dwrite d0/dd/d25/d59/f58 [0,4194304] 0 2024-07-03T15:43:49.048 INFO:tasks.workunit.client.1.smithi148.stdout:9/512: dread d0/dd/f90 [0,4194304] 0 2024-07-03T15:43:49.049 INFO:tasks.workunit.client.1.smithi148.stdout:9/513: symlink d0/d49/d43/d19/d7a/la4 0 2024-07-03T15:43:49.052 INFO:tasks.workunit.client.1.smithi148.stdout:9/514: rename d0/dd/d25/d59/f58 to d0/d18/d24/d47/fa5 0 2024-07-03T15:43:49.054 INFO:tasks.workunit.client.1.smithi148.stdout:9/515: mkdir d0/d63/da6 0 2024-07-03T15:43:49.057 INFO:tasks.workunit.client.1.smithi148.stdout:9/516: truncate d0/dd/f1f 1105597 0 2024-07-03T15:43:49.057 INFO:tasks.workunit.client.1.smithi148.stdout:9/517: chown d0/dd/d25/d37/d40/c85 5413 1 2024-07-03T15:43:49.060 INFO:tasks.workunit.client.1.smithi148.stdout:9/518: creat d0/d18/d24/d47/d4d/fa7 x:0 0 0 2024-07-03T15:43:49.062 INFO:tasks.workunit.client.1.smithi148.stdout:9/519: mkdir d0/dd/d25/d6a/da8 0 2024-07-03T15:43:49.067 INFO:tasks.workunit.client.1.smithi148.stdout:9/520: write d0/d18/d24/f91 [795360,122028] 0 2024-07-03T15:43:49.067 INFO:tasks.workunit.client.1.smithi148.stdout:9/521: chown d0/dd/d25/d6a/da8 629 1 2024-07-03T15:43:49.067 INFO:tasks.workunit.client.1.smithi148.stdout:9/522: write d0/dd/d25/d37/f50 [1524261,104928] 0 2024-07-03T15:43:49.069 INFO:tasks.workunit.client.1.smithi148.stdout:9/523: symlink d0/dd/d25/d77/la9 0 2024-07-03T15:43:49.090 INFO:tasks.workunit.client.1.smithi148.stdout:9/524: dwrite d0/f6 [0,4194304] 0 2024-07-03T15:43:49.090 INFO:tasks.workunit.client.1.smithi148.stdout:9/525: write d0/dd/d25/d37/f50 [1952157,101662] 0 2024-07-03T15:43:49.090 INFO:tasks.workunit.client.1.smithi148.stdout:9/526: chown d0/dd/d29/l78 6212926 1 2024-07-03T15:43:49.090 INFO:tasks.workunit.client.1.smithi148.stdout:9/527: chown d0/d18/d24/d47/l6c 4339733 1 2024-07-03T15:43:49.090 INFO:tasks.workunit.client.1.smithi148.stdout:9/528: write d0/dd/d11/f12 [9219533,37781] 0 2024-07-03T15:43:49.099 INFO:tasks.workunit.client.1.smithi148.stdout:9/529: mkdir d0/d49/d43/d19/d7a/daa 0 2024-07-03T15:43:49.103 INFO:tasks.workunit.client.1.smithi148.stdout:9/530: symlink d0/dd/d11/lab 0 2024-07-03T15:43:49.105 INFO:tasks.workunit.client.1.smithi148.stdout:9/531: mknod d0/dd/d25/d37/d40/d72/cac 0 2024-07-03T15:43:49.114 INFO:tasks.workunit.client.1.smithi148.stdout:4/200: truncate d0/f5 1069469 0 2024-07-03T15:43:49.117 INFO:tasks.workunit.client.1.smithi148.stdout:4/201: unlink d0/d8/d19/c1b 0 2024-07-03T15:43:49.119 INFO:tasks.workunit.client.1.smithi148.stdout:4/202: mknod d0/d8/d19/d24/c3c 0 2024-07-03T15:43:49.121 INFO:tasks.workunit.client.1.smithi148.stdout:4/203: dread d0/d8/d19/d24/f2c [0,4194304] 0 2024-07-03T15:43:49.121 INFO:tasks.workunit.client.1.smithi148.stdout:4/204: chown d0/d8/d22 102 1 2024-07-03T15:43:49.124 INFO:tasks.workunit.client.1.smithi148.stdout:9/532: dwrite d0/dd/d25/d37/f50 [0,4194304] 0 2024-07-03T15:43:49.127 INFO:tasks.workunit.client.1.smithi148.stdout:4/205: link d0/f16 d0/f3d 0 2024-07-03T15:43:49.127 INFO:tasks.workunit.client.1.smithi148.stdout:4/206: dread - d0/d8/d22/d2f/f33 zero size 2024-07-03T15:43:49.129 INFO:tasks.workunit.client.1.smithi148.stdout:9/533: mknod d0/d18/d24/d5a/cad 0 2024-07-03T15:43:49.129 INFO:tasks.workunit.client.1.smithi148.stdout:9/534: chown d0/d49/d43 0 1 2024-07-03T15:43:49.131 INFO:tasks.workunit.client.1.smithi148.stdout:4/207: creat d0/d8/d3a/d36/f3e x:0 0 0 2024-07-03T15:43:49.134 INFO:tasks.workunit.client.1.smithi148.stdout:9/535: mkdir d0/d18/d24/d47/d4d/dae 0 2024-07-03T15:43:49.134 INFO:tasks.workunit.client.1.smithi148.stdout:9/536: dread - d0/dd/d11/f9c zero size 2024-07-03T15:43:49.134 INFO:tasks.workunit.client.1.smithi148.stdout:9/537: write d0/d2f/f7f [365649,59555] 0 2024-07-03T15:43:49.138 INFO:tasks.workunit.client.1.smithi148.stdout:9/538: unlink d0/d49/d43/l4c 0 2024-07-03T15:43:49.138 INFO:tasks.workunit.client.1.smithi148.stdout:9/539: fsync d0/d18/f26 0 2024-07-03T15:43:49.146 INFO:tasks.workunit.client.1.smithi148.stdout:4/208: dwrite d0/d8/d19/d24/f2e [0,4194304] 0 2024-07-03T15:43:49.163 INFO:tasks.workunit.client.1.smithi148.stdout:4/209: dwrite d0/d8/f2d [0,4194304] 0 2024-07-03T15:43:49.163 INFO:tasks.workunit.client.1.smithi148.stdout:4/210: creat d0/d8/d19/d24/f3f x:0 0 0 2024-07-03T15:43:49.168 INFO:tasks.workunit.client.1.smithi148.stdout:4/211: mknod d0/d28/c40 0 2024-07-03T15:43:49.169 INFO:tasks.workunit.client.1.smithi148.stdout:4/212: truncate d0/d8/d3a/d36/f3e 119992 0 2024-07-03T15:43:49.171 INFO:tasks.workunit.client.1.smithi148.stdout:4/213: mkdir d0/d8/d22/d2f/d41 0 2024-07-03T15:43:49.185 INFO:tasks.workunit.client.1.smithi148.stdout:4/214: dwrite d0/f6 [0,4194304] 0 2024-07-03T15:43:49.186 INFO:tasks.workunit.client.1.smithi148.stdout:4/215: getdents d0/d8/d22 0 2024-07-03T15:43:49.188 INFO:tasks.workunit.client.1.smithi148.stdout:4/216: symlink d0/d8/d22/d2f/d41/l42 0 2024-07-03T15:43:49.189 INFO:tasks.workunit.client.1.smithi148.stdout:4/217: fdatasync d0/d8/f38 0 2024-07-03T15:43:49.195 INFO:tasks.workunit.client.1.smithi148.stdout:4/218: dread d0/f6 [0,4194304] 0 2024-07-03T15:43:49.197 INFO:tasks.workunit.client.1.smithi148.stdout:4/219: symlink d0/d28/l43 0 2024-07-03T15:43:49.197 INFO:tasks.workunit.client.1.smithi148.stdout:4/220: chown d0/d8/d22/d2f/f33 3595850 1 2024-07-03T15:43:49.199 INFO:tasks.workunit.client.1.smithi148.stdout:4/221: mkdir d0/d8/d44 0 2024-07-03T15:43:49.201 INFO:tasks.workunit.client.1.smithi148.stdout:4/222: unlink d0/d28/f31 0 2024-07-03T15:43:49.203 INFO:tasks.workunit.client.1.smithi148.stdout:4/223: mkdir d0/d8/d19/d45 0 2024-07-03T15:43:49.205 INFO:tasks.workunit.client.1.smithi148.stdout:4/224: write d0/f30 [978887,15036] 0 2024-07-03T15:43:49.214 INFO:tasks.workunit.client.1.smithi148.stdout:4/225: write d0/d8/f10 [1919170,83038] 0 2024-07-03T15:43:49.256 INFO:tasks.workunit.client.1.smithi148.stdout:8/218: sync 2024-07-03T15:43:49.257 INFO:tasks.workunit.client.1.smithi148.stdout:1/217: sync 2024-07-03T15:43:49.257 INFO:tasks.workunit.client.1.smithi148.stdout:5/237: sync 2024-07-03T15:43:49.257 INFO:tasks.workunit.client.1.smithi148.stdout:2/364: sync 2024-07-03T15:43:49.257 INFO:tasks.workunit.client.1.smithi148.stdout:3/273: sync 2024-07-03T15:43:49.257 INFO:tasks.workunit.client.1.smithi148.stdout:0/250: sync 2024-07-03T15:43:49.257 INFO:tasks.workunit.client.1.smithi148.stdout:8/219: dread - d4/da/f29 zero size 2024-07-03T15:43:49.257 INFO:tasks.workunit.client.1.smithi148.stdout:0/251: truncate da/d44/d13/d14/f1f 4204538 0 2024-07-03T15:43:49.257 INFO:tasks.workunit.client.1.smithi148.stdout:5/238: write f4 [2978588,112670] 0 2024-07-03T15:43:49.257 INFO:tasks.workunit.client.1.smithi148.stdout:3/274: creat dd/d33/f4b x:0 0 0 2024-07-03T15:43:49.257 INFO:tasks.workunit.client.1.smithi148.stdout:5/239: write d7/d13/f2a [47821,54597] 0 2024-07-03T15:43:49.257 INFO:tasks.workunit.client.1.smithi148.stdout:7/358: sync 2024-07-03T15:43:49.257 INFO:tasks.workunit.client.1.smithi148.stdout:5/240: truncate d7/d13/f2a 713138 0 2024-07-03T15:43:49.258 INFO:tasks.workunit.client.1.smithi148.stdout:7/359: fdatasync d5/d16/f4b 0 2024-07-03T15:43:49.258 INFO:tasks.workunit.client.1.smithi148.stdout:6/337: sync 2024-07-03T15:43:49.258 INFO:tasks.workunit.client.1.smithi148.stdout:6/338: write d3/d1d/d28/f51 [738046,117443] 0 2024-07-03T15:43:49.258 INFO:tasks.workunit.client.1.smithi148.stdout:6/339: stat d3/d1d/d28/d3e/f64 0 2024-07-03T15:43:49.259 INFO:tasks.workunit.client.1.smithi148.stdout:1/218: creat d0/d2/d2e/d39/d42/f48 x:0 0 0 2024-07-03T15:43:49.259 INFO:tasks.workunit.client.1.smithi148.stdout:1/219: truncate d0/d2/d2e/d39/d42/f48 567126 0 2024-07-03T15:43:49.259 INFO:tasks.workunit.client.1.smithi148.stdout:1/220: stat d0 0 2024-07-03T15:43:49.259 INFO:tasks.workunit.client.1.smithi148.stdout:1/221: creat d0/d2/d2e/d39/d42/f49 x:0 0 0 2024-07-03T15:43:49.259 INFO:tasks.workunit.client.1.smithi148.stdout:1/222: stat d0/d2/d11/d1e 0 2024-07-03T15:43:49.264 INFO:tasks.workunit.client.1.smithi148.stdout:0/252: unlink da/d17/f4f 0 2024-07-03T15:43:49.267 INFO:tasks.workunit.client.1.smithi148.stdout:3/275: rename dd/d1c/f20 to dd/d1c/d2e/d40/f4c 0 2024-07-03T15:43:49.267 INFO:tasks.workunit.client.1.smithi148.stdout:3/276: stat dd/d1c/l2f 0 2024-07-03T15:43:49.270 INFO:tasks.workunit.client.1.smithi148.stdout:6/340: mknod d3/d1d/d28/d53/c75 0 2024-07-03T15:43:49.271 INFO:tasks.workunit.client.1.smithi148.stdout:8/220: write d4/da/d14/d17/f26 [5869665,112796] 0 2024-07-03T15:43:49.275 INFO:tasks.workunit.client.1.smithi148.stdout:1/223: link d0/c18 d0/c4a 0 2024-07-03T15:43:49.275 INFO:tasks.workunit.client.1.smithi148.stdout:1/224: write d0/f24 [2384795,76198] 0 2024-07-03T15:43:49.275 INFO:tasks.workunit.client.1.smithi148.stdout:5/241: dwrite d7/da/df/d16/d2f/f38 [0,4194304] 0 2024-07-03T15:43:49.276 INFO:tasks.workunit.client.1.smithi148.stdout:2/365: dwrite d5/d14/d22/d2b/d13/d1d/f43 [0,4194304] 0 2024-07-03T15:43:49.276 INFO:tasks.workunit.client.1.smithi148.stdout:2/366: write d5/d26/d6a/f77 [874262,104201] 0 2024-07-03T15:43:49.280 INFO:tasks.workunit.client.1.smithi148.stdout:0/253: rename da/d44/f31 to da/d44/d13/d14/f54 0 2024-07-03T15:43:49.280 INFO:tasks.workunit.client.1.smithi148.stdout:0/254: chown c5 47582 1 2024-07-03T15:43:49.280 INFO:tasks.workunit.client.1.smithi148.stdout:0/255: readlink da/d44/l20 0 2024-07-03T15:43:49.281 INFO:tasks.workunit.client.1.smithi148.stdout:6/341: mkdir d3/d5b/d76 0 2024-07-03T15:43:49.282 INFO:tasks.workunit.client.1.smithi148.stdout:3/277: write dd/d1c/d2e/f11 [1823953,59992] 0 2024-07-03T15:43:49.282 INFO:tasks.workunit.client.1.smithi148.stdout:8/221: mknod d4/da/d47/c4f 0 2024-07-03T15:43:49.289 INFO:tasks.workunit.client.1.smithi148.stdout:0/256: dread da/d44/f26 [0,4194304] 0 2024-07-03T15:43:49.289 INFO:tasks.workunit.client.1.smithi148.stdout:0/257: fdatasync da/d17/f47 0 2024-07-03T15:43:49.293 INFO:tasks.workunit.client.1.smithi148.stdout:2/367: dwrite d5/d14/d22/d2b/f1a [4194304,4194304] 0 2024-07-03T15:43:49.294 INFO:tasks.workunit.client.1.smithi148.stdout:5/242: link d7/da/f2b d7/f4f 0 2024-07-03T15:43:49.300 INFO:tasks.workunit.client.1.smithi148.stdout:1/225: rename d0/d2/d11/d38 to d0/d2/d6/d14/d29/d4b 0 2024-07-03T15:43:49.301 INFO:tasks.workunit.client.1.smithi148.stdout:6/342: dwrite d3/d1d/d28/d53/f59 [0,4194304] 0 2024-07-03T15:43:49.303 INFO:tasks.workunit.client.1.smithi148.stdout:8/222: mkdir d4/da/d50 0 2024-07-03T15:43:49.304 INFO:tasks.workunit.client.1.smithi148.stdout:8/223: fsync d4/d6/d18/d20/f2b 0 2024-07-03T15:43:49.307 INFO:tasks.workunit.client.1.smithi148.stdout:2/368: mknod d5/d26/d6a/c78 0 2024-07-03T15:43:49.310 INFO:tasks.workunit.client.1.smithi148.stdout:5/243: link d7/da/df/d16/d1a/f34 d7/da/df/d16/d1a/f50 0 2024-07-03T15:43:49.313 INFO:tasks.workunit.client.1.smithi148.stdout:3/278: rename dd/d1c/d2e/l1a to dd/d1c/l4d 0 2024-07-03T15:43:49.315 INFO:tasks.workunit.client.1.smithi148.stdout:1/226: mknod d0/de/d2f/d40/c4c 0 2024-07-03T15:43:49.319 INFO:tasks.workunit.client.1.smithi148.stdout:5/244: dread d7/da/df/d16/f20 [0,4194304] 0 2024-07-03T15:43:49.323 INFO:tasks.workunit.client.1.smithi148.stdout:6/343: mkdir d3/d50/d77 0 2024-07-03T15:43:49.324 INFO:tasks.workunit.client.1.smithi148.stdout:1/227: dread d0/d2/d2e/d39/d41/f43 [0,4194304] 0 2024-07-03T15:43:49.326 INFO:tasks.workunit.client.1.smithi148.stdout:6/344: dread d3/f34 [0,4194304] 0 2024-07-03T15:43:49.326 INFO:tasks.workunit.client.1.smithi148.stdout:2/369: mkdir d5/d5f/d79 0 2024-07-03T15:43:49.326 INFO:tasks.workunit.client.1.smithi148.stdout:2/370: fsync d5/d26/f48 0 2024-07-03T15:43:49.328 INFO:tasks.workunit.client.1.smithi148.stdout:0/258: rename da/d17/d2a/l2c to da/d44/d13/d14/d50/l55 0 2024-07-03T15:43:49.330 INFO:tasks.workunit.client.1.smithi148.stdout:1/228: write d0/d2/d2e/d39/d41/f43 [1601374,79578] 0 2024-07-03T15:43:49.330 INFO:tasks.workunit.client.1.smithi148.stdout:1/229: chown d0/d2/d2e/f36 260747 1 2024-07-03T15:43:49.330 INFO:tasks.workunit.client.1.smithi148.stdout:1/230: chown d0/d2/d2e/d39/d41 0 1 2024-07-03T15:43:49.330 INFO:tasks.workunit.client.1.smithi148.stdout:1/231: getdents d0/de/d44 0 2024-07-03T15:43:49.331 INFO:tasks.workunit.client.1.smithi148.stdout:5/245: creat d7/da/df/d1d/f51 x:0 0 0 2024-07-03T15:43:49.331 INFO:tasks.workunit.client.1.smithi148.stdout:5/246: write d7/da/df/d16/d1a/f50 [4325372,64837] 0 2024-07-03T15:43:49.331 INFO:tasks.workunit.client.1.smithi148.stdout:5/247: truncate d7/da/df/f1f 301434 0 2024-07-03T15:43:49.331 INFO:tasks.workunit.client.1.smithi148.stdout:5/248: write d7/da/df/d16/f1b [1894261,84764] 0 2024-07-03T15:43:49.331 INFO:tasks.workunit.client.1.smithi148.stdout:5/249: stat d7/da/d21 0 2024-07-03T15:43:49.331 INFO:tasks.workunit.client.1.smithi148.stdout:6/345: creat d3/d1d/d28/d31/d4c/f78 x:0 0 0 2024-07-03T15:43:49.332 INFO:tasks.workunit.client.1.smithi148.stdout:3/279: dwrite dd/d33/d37/f46 [0,4194304] 0 2024-07-03T15:43:49.332 INFO:tasks.workunit.client.1.smithi148.stdout:3/280: fsync dd/f2a 0 2024-07-03T15:43:49.333 INFO:tasks.workunit.client.1.smithi148.stdout:2/371: creat d5/d14/f7a x:0 0 0 2024-07-03T15:43:49.337 INFO:tasks.workunit.client.1.smithi148.stdout:1/232: write d0/d2/fd [2744282,21415] 0 2024-07-03T15:43:49.342 INFO:tasks.workunit.client.1.smithi148.stdout:5/250: mkdir d7/da/df/d16/d2f/d44/d52 0 2024-07-03T15:43:49.342 INFO:tasks.workunit.client.1.smithi148.stdout:5/251: stat d7/c1c 0 2024-07-03T15:43:49.342 INFO:tasks.workunit.client.1.smithi148.stdout:5/252: fdatasync f3 0 2024-07-03T15:43:49.343 INFO:tasks.workunit.client.1.smithi148.stdout:6/346: rmdir d3/d1d 39 2024-07-03T15:43:49.343 INFO:tasks.workunit.client.1.smithi148.stdout:6/347: fdatasync d3/f1e 0 2024-07-03T15:43:49.349 INFO:tasks.workunit.client.1.smithi148.stdout:3/281: getdents dd/d35/d41 0 2024-07-03T15:43:49.350 INFO:tasks.workunit.client.1.smithi148.stdout:2/372: dwrite d5/d26/d6a/f77 [0,4194304] 0 2024-07-03T15:43:49.350 INFO:tasks.workunit.client.1.smithi148.stdout:2/373: truncate d5/d14/d22/d2b/d13/f1b 9228123 0 2024-07-03T15:43:49.350 INFO:tasks.workunit.client.1.smithi148.stdout:2/374: chown d5/d14/d22/d2b/d13/l52 0 1 2024-07-03T15:43:49.352 INFO:tasks.workunit.client.1.smithi148.stdout:8/224: rename d4/d6 to d4/da/d51 0 2024-07-03T15:43:49.353 INFO:tasks.workunit.client.1.smithi148.stdout:0/259: symlink da/d44/d13/l56 0 2024-07-03T15:43:49.356 INFO:tasks.workunit.client.1.smithi148.stdout:1/233: mkdir d0/d2/d11/d4d 0 2024-07-03T15:43:49.357 INFO:tasks.workunit.client.1.smithi148.stdout:1/234: write d0/d2/d2e/d39/d42/f49 [728910,90588] 0 2024-07-03T15:43:49.357 INFO:tasks.workunit.client.1.smithi148.stdout:2/375: read d5/d14/d22/f72 [1306082,43950] 0 2024-07-03T15:43:49.357 INFO:tasks.workunit.client.1.smithi148.stdout:2/376: write d5/d14/d22/d2b/d13/d41/f49 [5040558,7708] 0 2024-07-03T15:43:49.357 INFO:tasks.workunit.client.1.smithi148.stdout:2/377: fsync d5/d14/f5a 0 2024-07-03T15:43:49.359 INFO:tasks.workunit.client.1.smithi148.stdout:6/348: symlink d3/d50/l79 0 2024-07-03T15:43:49.359 INFO:tasks.workunit.client.1.smithi148.stdout:6/349: creat d3/d1d/d28/d31/d61/f7a x:0 0 0 2024-07-03T15:43:49.359 INFO:tasks.workunit.client.1.smithi148.stdout:6/350: chown d3/d1d/d28/l45 20054881 1 2024-07-03T15:43:49.360 INFO:tasks.workunit.client.1.smithi148.stdout:6/351: truncate d3/d1d/d28/d3e/f64 298648 0 2024-07-03T15:43:49.360 INFO:tasks.workunit.client.1.smithi148.stdout:6/352: chown d3/d1d/d28/d31/d61/f71 19436376 1 2024-07-03T15:43:49.360 INFO:tasks.workunit.client.1.smithi148.stdout:6/353: write d3/d1d/d28/d3e/f58 [1817183,73347] 0 2024-07-03T15:43:49.361 INFO:tasks.workunit.client.1.smithi148.stdout:3/282: creat dd/d35/f4e x:0 0 0 2024-07-03T15:43:49.361 INFO:tasks.workunit.client.1.smithi148.stdout:3/283: creat dd/d35/f4f x:0 0 0 2024-07-03T15:43:49.361 INFO:tasks.workunit.client.1.smithi148.stdout:3/284: creat dd/d33/f50 x:0 0 0 2024-07-03T15:43:49.361 INFO:tasks.workunit.client.1.smithi148.stdout:3/285: chown dd/d33/f3e 8084504 1 2024-07-03T15:43:49.362 INFO:tasks.workunit.client.1.smithi148.stdout:8/225: mknod d4/da/d51/d18/d1f/c52 0 2024-07-03T15:43:49.364 INFO:tasks.workunit.client.1.smithi148.stdout:0/260: rename da/d44/d13/f46 to da/d38/f57 0 2024-07-03T15:43:49.364 INFO:tasks.workunit.client.1.smithi148.stdout:0/261: dread - da/d38/f4e zero size 2024-07-03T15:43:49.365 INFO:tasks.workunit.client.1.smithi148.stdout:1/235: creat d0/de/d2f/f4e x:0 0 0 2024-07-03T15:43:49.365 INFO:tasks.workunit.client.1.smithi148.stdout:1/236: write d0/d2/d2e/d39/d42/f48 [1445338,25297] 0 2024-07-03T15:43:49.369 INFO:tasks.workunit.client.1.smithi148.stdout:5/253: getdents d7/da 0 2024-07-03T15:43:49.369 INFO:tasks.workunit.client.1.smithi148.stdout:5/254: fsync d7/d15/f4b 0 2024-07-03T15:43:49.371 INFO:tasks.workunit.client.1.smithi148.stdout:1/237: dread d0/d2/d6/d14/f15 [0,4194304] 0 2024-07-03T15:43:49.372 INFO:tasks.workunit.client.1.smithi148.stdout:6/354: creat d3/d1d/d28/d3e/f7b x:0 0 0 2024-07-03T15:43:49.376 INFO:tasks.workunit.client.1.smithi148.stdout:3/286: symlink dd/d35/d41/l51 0 2024-07-03T15:43:49.381 INFO:tasks.workunit.client.1.smithi148.stdout:0/262: mkdir da/d58 0 2024-07-03T15:43:49.384 INFO:tasks.workunit.client.1.smithi148.stdout:5/255: unlink d7/da/df/d16/c3a 0 2024-07-03T15:43:49.386 INFO:tasks.workunit.client.1.smithi148.stdout:6/355: mknod d3/d1d/d28/d3e/c7c 0 2024-07-03T15:43:49.388 INFO:tasks.workunit.client.1.smithi148.stdout:3/287: mkdir dd/d33/d52 0 2024-07-03T15:43:49.389 INFO:tasks.workunit.client.1.smithi148.stdout:3/288: creat dd/d33/d37/f53 x:0 0 0 2024-07-03T15:43:49.400 INFO:tasks.workunit.client.1.smithi148.stdout:6/356: creat d3/d50/f7d x:0 0 0 2024-07-03T15:43:49.400 INFO:tasks.workunit.client.1.smithi148.stdout:6/357: write d3/d1d/d28/d31/d61/f7a [778089,79624] 0 2024-07-03T15:43:49.419 INFO:tasks.workunit.client.1.smithi148.stdout:5/256: dwrite d7/da/df/d16/f1b [0,4194304] 0 2024-07-03T15:43:49.419 INFO:tasks.workunit.client.1.smithi148.stdout:3/289: dwrite dd/f2b [0,4194304] 0 2024-07-03T15:43:49.420 INFO:tasks.workunit.client.1.smithi148.stdout:5/257: mknod d7/da/df/c53 0 2024-07-03T15:43:49.420 INFO:tasks.workunit.client.1.smithi148.stdout:5/258: chown d7/d13/d23 126908 1 2024-07-03T15:43:49.425 INFO:tasks.workunit.client.1.smithi148.stdout:3/290: mknod dd/d1c/d2e/c54 0 2024-07-03T15:43:49.427 INFO:tasks.workunit.client.1.smithi148.stdout:3/291: symlink dd/l55 0 2024-07-03T15:43:49.428 INFO:tasks.workunit.client.1.smithi148.stdout:3/292: write dd/d33/f4b [575489,10119] 0 2024-07-03T15:43:49.428 INFO:tasks.workunit.client.1.smithi148.stdout:3/293: creat dd/d35/f56 x:0 0 0 2024-07-03T15:43:49.428 INFO:tasks.workunit.client.1.smithi148.stdout:3/294: write dd/d1c/f32 [1028079,90850] 0 2024-07-03T15:43:49.432 INFO:tasks.workunit.client.1.smithi148.stdout:3/295: mkdir dd/d33/d52/d57 0 2024-07-03T15:43:49.433 INFO:tasks.workunit.client.1.smithi148.stdout:5/259: dwrite d7/d15/f4b [0,4194304] 0 2024-07-03T15:43:49.455 INFO:tasks.workunit.client.1.smithi148.stdout:2/378: rename d5/d26 to d5/d14/d22/d2b/d13/d41/d7b 0 2024-07-03T15:43:49.456 INFO:tasks.workunit.client.1.smithi148.stdout:2/379: dread d5/d14/d22/d2b/d13/d41/d7b/f3e [0,4194304] 0 2024-07-03T15:43:49.458 INFO:tasks.workunit.client.1.smithi148.stdout:1/238: rename d0/d2/d6/f19 to d0/d2/d6/d14/d29/d4b/f4f 0 2024-07-03T15:43:49.458 INFO:tasks.workunit.client.1.smithi148.stdout:1/239: chown d0/d2/d6/d14/c23 1 1 2024-07-03T15:43:49.461 INFO:tasks.workunit.client.1.smithi148.stdout:6/358: rename d3/d1d/d28/d3e/c62 to d3/d1d/d28/d53/d6d/c7e 0 2024-07-03T15:43:49.461 INFO:tasks.workunit.client.1.smithi148.stdout:6/359: getdents d3/d50/d77 0 2024-07-03T15:43:49.462 INFO:tasks.workunit.client.1.smithi148.stdout:1/240: creat d0/de/d2f/d40/f50 x:0 0 0 2024-07-03T15:43:49.465 INFO:tasks.workunit.client.1.smithi148.stdout:3/296: rename dd/d1c/d2e/d40/f47 to dd/d33/d52/f58 0 2024-07-03T15:43:49.465 INFO:tasks.workunit.client.1.smithi148.stdout:3/297: write dd/f3d [721191,68464] 0 2024-07-03T15:43:49.465 INFO:tasks.workunit.client.1.smithi148.stdout:3/298: dread - dd/d33/f50 zero size 2024-07-03T15:43:49.465 INFO:tasks.workunit.client.1.smithi148.stdout:2/380: write d5/d14/d22/d2b/d13/d41/d7b/f48 [3691616,67285] 0 2024-07-03T15:43:49.467 INFO:tasks.workunit.client.1.smithi148.stdout:6/360: rmdir d3/d5b/d76 0 2024-07-03T15:43:49.469 INFO:tasks.workunit.client.1.smithi148.stdout:1/241: dread d0/d2/d2e/d39/d41/f43 [0,4194304] 0 2024-07-03T15:43:49.473 INFO:tasks.workunit.client.1.smithi148.stdout:0/263: unlink da/d44/d13/d14/d50/l55 0 2024-07-03T15:43:49.473 INFO:tasks.workunit.client.1.smithi148.stdout:0/264: chown da/f11 880994449 1 2024-07-03T15:43:49.474 INFO:tasks.workunit.client.1.smithi148.stdout:5/260: rename d7/d15 to d7/d13/d54 0 2024-07-03T15:43:49.475 INFO:tasks.workunit.client.1.smithi148.stdout:6/361: write d3/d1d/f27 [2137949,87562] 0 2024-07-03T15:43:49.477 INFO:tasks.workunit.client.1.smithi148.stdout:1/242: rename d0/d2/d2e/f36 to d0/d2/d11/d1e/f51 0 2024-07-03T15:43:49.480 INFO:tasks.workunit.client.1.smithi148.stdout:5/261: unlink d7/da/d21/f4e 0 2024-07-03T15:43:49.480 INFO:tasks.workunit.client.1.smithi148.stdout:5/262: write d7/da/df/d16/d1a/f50 [1557292,2695] 0 2024-07-03T15:43:49.480 INFO:tasks.workunit.client.1.smithi148.stdout:2/381: dread d5/f1f [0,4194304] 0 2024-07-03T15:43:49.481 INFO:tasks.workunit.client.1.smithi148.stdout:6/362: rmdir d3/d1d 39 2024-07-03T15:43:49.482 INFO:tasks.workunit.client.1.smithi148.stdout:6/363: chown d3/d1d/d28/l46 613 1 2024-07-03T15:43:49.494 INFO:tasks.workunit.client.1.smithi148.stdout:8/226: getdents d4/da 0 2024-07-03T15:43:49.495 INFO:tasks.workunit.client.1.smithi148.stdout:3/299: dwrite dd/d33/d37/f53 [0,4194304] 0 2024-07-03T15:43:49.498 INFO:tasks.workunit.client.1.smithi148.stdout:5/263: creat d7/da/df/d16/d2f/d44/d52/f55 x:0 0 0 2024-07-03T15:43:49.499 INFO:tasks.workunit.client.1.smithi148.stdout:6/364: mkdir d3/d1d/d28/d7f 0 2024-07-03T15:43:49.499 INFO:tasks.workunit.client.1.smithi148.stdout:6/365: creat d3/d1d/d28/d53/f80 x:0 0 0 2024-07-03T15:43:49.502 INFO:tasks.workunit.client.1.smithi148.stdout:8/227: mkdir d4/da/d51/d53 0 2024-07-03T15:43:49.502 INFO:tasks.workunit.client.1.smithi148.stdout:8/228: chown d4/da/d51/d18/d1f/f39 23 1 2024-07-03T15:43:49.502 INFO:tasks.workunit.client.1.smithi148.stdout:2/382: dwrite d5/d7/f35 [0,4194304] 0 2024-07-03T15:43:49.508 INFO:tasks.workunit.client.1.smithi148.stdout:3/300: link dd/d1c/d2e/d40/f4c dd/d33/f59 0 2024-07-03T15:43:49.513 INFO:tasks.workunit.client.1.smithi148.stdout:8/229: unlink d4/da/d51/c36 0 2024-07-03T15:43:49.517 INFO:tasks.workunit.client.1.smithi148.stdout:3/301: symlink dd/d33/d52/l5a 0 2024-07-03T15:43:49.517 INFO:tasks.workunit.client.1.smithi148.stdout:3/302: dread - dd/d33/d52/f58 zero size 2024-07-03T15:43:49.518 INFO:tasks.workunit.client.1.smithi148.stdout:8/230: mkdir d4/d54 0 2024-07-03T15:43:49.518 INFO:tasks.workunit.client.1.smithi148.stdout:8/231: write d4/da/d14/f4d [977702,60953] 0 2024-07-03T15:43:49.520 INFO:tasks.workunit.client.1.smithi148.stdout:3/303: symlink dd/d1c/l5b 0 2024-07-03T15:43:49.520 INFO:tasks.workunit.client.1.smithi148.stdout:3/304: readlink dd/d1c/d2e/l27 0 2024-07-03T15:43:49.520 INFO:tasks.workunit.client.1.smithi148.stdout:3/305: chown dd/d1c/d2e 26474 1 2024-07-03T15:43:49.520 INFO:tasks.workunit.client.1.smithi148.stdout:3/306: creat dd/d1c/f5c x:0 0 0 2024-07-03T15:43:49.520 INFO:tasks.workunit.client.1.smithi148.stdout:3/307: creat dd/d1c/d2e/d40/f5d x:0 0 0 2024-07-03T15:43:49.526 INFO:tasks.workunit.client.1.smithi148.stdout:8/232: rmdir d4/da/d14/d17 39 2024-07-03T15:43:49.526 INFO:tasks.workunit.client.1.smithi148.stdout:8/233: chown d4/lf 52781 1 2024-07-03T15:43:49.528 INFO:tasks.workunit.client.1.smithi148.stdout:3/308: write dd/d1c/d2e/f13 [6539057,7514] 0 2024-07-03T15:43:49.528 INFO:tasks.workunit.client.1.smithi148.stdout:3/309: chown dd/f2a 0 1 2024-07-03T15:43:49.528 INFO:tasks.workunit.client.1.smithi148.stdout:3/310: chown dd/d35/f4f 0 1 2024-07-03T15:43:49.528 INFO:tasks.workunit.client.1.smithi148.stdout:3/311: truncate dd/d35/f3f 877788 0 2024-07-03T15:43:49.529 INFO:tasks.workunit.client.1.smithi148.stdout:8/234: symlink d4/da/d14/d3a/l55 0 2024-07-03T15:43:49.529 INFO:tasks.workunit.client.1.smithi148.stdout:8/235: fsync d4/da/d14/d3a/f41 0 2024-07-03T15:43:49.531 INFO:tasks.workunit.client.1.smithi148.stdout:3/312: creat dd/d35/d49/f5e x:0 0 0 2024-07-03T15:43:49.533 INFO:tasks.workunit.client.1.smithi148.stdout:3/313: write dd/f22 [462352,72400] 0 2024-07-03T15:43:49.533 INFO:tasks.workunit.client.1.smithi148.stdout:3/314: chown dd/f22 14 1 2024-07-03T15:43:49.534 INFO:tasks.workunit.client.1.smithi148.stdout:8/236: mknod d4/da/d14/d3a/c56 0 2024-07-03T15:43:49.536 INFO:tasks.workunit.client.1.smithi148.stdout:3/315: link dd/c1b dd/d1c/d2e/c5f 0 2024-07-03T15:43:49.537 INFO:tasks.workunit.client.1.smithi148.stdout:8/237: creat d4/da/d51/f57 x:0 0 0 2024-07-03T15:43:49.539 INFO:tasks.workunit.client.1.smithi148.stdout:3/316: creat dd/d33/d52/f60 x:0 0 0 2024-07-03T15:43:49.541 INFO:tasks.workunit.client.1.smithi148.stdout:8/238: mkdir d4/da/d14/d1a/d32/d38/d58 0 2024-07-03T15:43:49.543 INFO:tasks.workunit.client.1.smithi148.stdout:8/239: symlink d4/da/d51/l59 0 2024-07-03T15:43:49.544 INFO:tasks.workunit.client.1.smithi148.stdout:8/240: write d4/da/d14/f4d [1049592,63623] 0 2024-07-03T15:43:49.546 INFO:tasks.workunit.client.1.smithi148.stdout:8/241: unlink d4/da/d14/d1a/d32/d38/d4a/f3d 0 2024-07-03T15:43:49.570 INFO:tasks.workunit.client.1.smithi148.stdout:4/226: sync 2024-07-03T15:43:49.570 INFO:tasks.workunit.client.1.smithi148.stdout:9/540: sync 2024-07-03T15:43:49.570 INFO:tasks.workunit.client.1.smithi148.stdout:7/360: sync 2024-07-03T15:43:49.570 INFO:tasks.workunit.client.1.smithi148.stdout:4/227: chown d0/d28/c35 0 1 2024-07-03T15:43:49.573 INFO:tasks.workunit.client.1.smithi148.stdout:9/541: symlink d0/d18/laf 0 2024-07-03T15:43:49.574 INFO:tasks.workunit.client.1.smithi148.stdout:4/228: symlink d0/d8/d19/d45/l46 0 2024-07-03T15:43:49.575 INFO:tasks.workunit.client.1.smithi148.stdout:8/242: dread d4/da/d51/d18/d20/f21 [0,4194304] 0 2024-07-03T15:43:49.575 INFO:tasks.workunit.client.1.smithi148.stdout:1/243: dwrite d0/d2/d2e/d39/d42/f49 [0,4194304] 0 2024-07-03T15:43:49.576 INFO:tasks.workunit.client.1.smithi148.stdout:7/361: truncate d5/d16/d1c/d33/d4c/d21/f3c 68432 0 2024-07-03T15:43:49.577 INFO:tasks.workunit.client.1.smithi148.stdout:7/362: write d5/d16/f34 [5093543,125420] 0 2024-07-03T15:43:49.579 INFO:tasks.workunit.client.1.smithi148.stdout:9/542: chown d0/dd/d25/d59/l9e 10 1 2024-07-03T15:43:49.586 INFO:tasks.workunit.client.1.smithi148.stdout:8/243: dread d4/da/d51/f8 [0,4194304] 0 2024-07-03T15:43:49.587 INFO:tasks.workunit.client.1.smithi148.stdout:9/543: mkdir d0/d49/d43/db0 0 2024-07-03T15:43:49.589 INFO:tasks.workunit.client.1.smithi148.stdout:8/244: creat d4/da/d51/d18/d20/f5a x:0 0 0 2024-07-03T15:43:49.589 INFO:tasks.workunit.client.1.smithi148.stdout:8/245: fsync d4/da/d14/d1a/d32/d38/d4a/f27 0 2024-07-03T15:43:49.589 INFO:tasks.workunit.client.1.smithi148.stdout:8/246: write d4/da/d14/d1a/f37 [626356,108309] 0 2024-07-03T15:43:49.590 INFO:tasks.workunit.client.1.smithi148.stdout:6/366: dwrite d3/d1d/d28/d31/d4c/f78 [0,4194304] 0 2024-07-03T15:43:49.591 INFO:tasks.workunit.client.1.smithi148.stdout:9/544: creat d0/d49/d43/d19/d7a/daa/fb1 x:0 0 0 2024-07-03T15:43:49.592 INFO:tasks.workunit.client.1.smithi148.stdout:8/247: dread d4/da/fb [0,4194304] 0 2024-07-03T15:43:49.593 INFO:tasks.workunit.client.1.smithi148.stdout:5/264: dwrite d7/da/df/f4d [0,4194304] 0 2024-07-03T15:43:49.593 INFO:tasks.workunit.client.1.smithi148.stdout:5/265: write d7/da/df/f10 [3309539,88490] 0 2024-07-03T15:43:49.594 INFO:tasks.workunit.client.1.smithi148.stdout:9/545: dread d0/dd/d25/d6a/f82 [0,4194304] 0 2024-07-03T15:43:49.594 INFO:tasks.workunit.client.1.smithi148.stdout:7/363: dwrite d5/d16/d1c/d4a/d52/f6b [0,4194304] 0 2024-07-03T15:43:49.595 INFO:tasks.workunit.client.1.smithi148.stdout:7/364: read - d5/d16/d1c/d23/f68 zero size 2024-07-03T15:43:49.595 INFO:tasks.workunit.client.1.smithi148.stdout:7/365: chown d5/c20 99 1 2024-07-03T15:43:49.597 INFO:tasks.workunit.client.1.smithi148.stdout:7/366: dread d5/f58 [0,4194304] 0 2024-07-03T15:43:49.597 INFO:tasks.workunit.client.1.smithi148.stdout:7/367: write d5/d16/d1c/f41 [2404970,56838] 0 2024-07-03T15:43:49.597 INFO:tasks.workunit.client.1.smithi148.stdout:6/367: symlink d3/d50/l81 0 2024-07-03T15:43:49.601 INFO:tasks.workunit.client.1.smithi148.stdout:8/248: creat d4/d54/f5b x:0 0 0 2024-07-03T15:43:49.601 INFO:tasks.workunit.client.1.smithi148.stdout:8/249: readlink d4/l5 0 2024-07-03T15:43:49.601 INFO:tasks.workunit.client.1.smithi148.stdout:8/250: read d4/da/d14/d1a/d32/d38/d4a/f27 [27475,8719] 0 2024-07-03T15:43:49.601 INFO:tasks.workunit.client.1.smithi148.stdout:8/251: creat d4/d54/f5c x:0 0 0 2024-07-03T15:43:49.609 INFO:tasks.workunit.client.1.smithi148.stdout:5/266: rename d7/da/df/d16/d2f/f35 to d7/d13/d23/d40/f56 0 2024-07-03T15:43:49.613 INFO:tasks.workunit.client.1.smithi148.stdout:9/546: symlink d0/d49/d43/lb2 0 2024-07-03T15:43:49.621 INFO:tasks.workunit.client.1.smithi148.stdout:0/265: rmdir da 39 2024-07-03T15:43:49.621 INFO:tasks.workunit.client.1.smithi148.stdout:0/266: fdatasync da/d44/d13/f27 0 2024-07-03T15:43:49.635 INFO:tasks.workunit.client.1.smithi148.stdout:9/547: dwrite d0/dd/f90 [4194304,4194304] 0 2024-07-03T15:43:49.637 INFO:tasks.workunit.client.1.smithi148.stdout:0/267: dwrite da/d17/d2a/f2d [0,4194304] 0 2024-07-03T15:43:49.637 INFO:tasks.workunit.client.1.smithi148.stdout:0/268: write da/d44/d13/d14/d50/f53 [575999,124899] 0 2024-07-03T15:43:49.638 INFO:tasks.workunit.client.1.smithi148.stdout:0/269: readlink da/d44/d13/d1d/l23 0 2024-07-03T15:43:49.638 INFO:tasks.workunit.client.1.smithi148.stdout:7/368: symlink d5/d16/d1c/d59/l6c 0 2024-07-03T15:43:49.638 INFO:tasks.workunit.client.1.smithi148.stdout:7/369: chown l4 45 1 2024-07-03T15:43:49.641 INFO:tasks.workunit.client.1.smithi148.stdout:0/270: dread da/f11 [0,4194304] 0 2024-07-03T15:43:49.641 INFO:tasks.workunit.client.1.smithi148.stdout:0/271: readlink da/d44/d13/d1d/l23 0 2024-07-03T15:43:49.642 INFO:tasks.workunit.client.1.smithi148.stdout:8/252: mknod d4/da/d14/d3a/c5d 0 2024-07-03T15:43:49.642 INFO:tasks.workunit.client.1.smithi148.stdout:8/253: fdatasync d4/da/d51/f57 0 2024-07-03T15:43:49.642 INFO:tasks.workunit.client.1.smithi148.stdout:8/254: write d4/da/d14/d1a/d32/d38/d4a/f27 [89139,56016] 0 2024-07-03T15:43:49.645 INFO:tasks.workunit.client.1.smithi148.stdout:9/548: getdents d0/dd/d25/d59 0 2024-07-03T15:43:49.645 INFO:tasks.workunit.client.1.smithi148.stdout:9/549: chown d0/dd/d29/f5e 2 1 2024-07-03T15:43:49.645 INFO:tasks.workunit.client.1.smithi148.stdout:3/317: fsync dd/d33/f59 0 2024-07-03T15:43:49.651 INFO:tasks.workunit.client.1.smithi148.stdout:7/370: link d5/f6 d5/d16/f6d 0 2024-07-03T15:43:49.656 INFO:tasks.workunit.client.1.smithi148.stdout:8/255: creat d4/da/d14/d1a/d32/f5e x:0 0 0 2024-07-03T15:43:49.656 INFO:tasks.workunit.client.1.smithi148.stdout:8/256: chown d4/da/d51/l10 19617017 1 2024-07-03T15:43:49.656 INFO:tasks.workunit.client.1.smithi148.stdout:0/272: dwrite da/d17/f1e [0,4194304] 0 2024-07-03T15:43:49.658 INFO:tasks.workunit.client.1.smithi148.stdout:3/318: mkdir dd/d1c/d2e/d61 0 2024-07-03T15:43:49.660 INFO:tasks.workunit.client.1.smithi148.stdout:7/371: symlink d5/d16/d1c/d59/l6e 0 2024-07-03T15:43:49.660 INFO:tasks.workunit.client.1.smithi148.stdout:8/257: dread d4/da/d14/d1a/f37 [0,4194304] 0 2024-07-03T15:43:49.662 INFO:tasks.workunit.client.1.smithi148.stdout:0/273: dread da/d44/f26 [0,4194304] 0 2024-07-03T15:43:49.662 INFO:tasks.workunit.client.1.smithi148.stdout:3/319: dread dd/f15 [0,4194304] 0 2024-07-03T15:43:49.662 INFO:tasks.workunit.client.1.smithi148.stdout:3/320: truncate dd/f44 518177 0 2024-07-03T15:43:49.662 INFO:tasks.workunit.client.1.smithi148.stdout:3/321: chown dd/d1c/d2e/d40/f4c 331249 1 2024-07-03T15:43:49.662 INFO:tasks.workunit.client.1.smithi148.stdout:3/322: creat dd/f62 x:0 0 0 2024-07-03T15:43:49.663 INFO:tasks.workunit.client.1.smithi148.stdout:7/372: mknod d5/d16/d1c/d33/c6f 0 2024-07-03T15:43:49.664 INFO:tasks.workunit.client.1.smithi148.stdout:9/550: dwrite d0/d18/d24/f91 [0,4194304] 0 2024-07-03T15:43:49.664 INFO:tasks.workunit.client.1.smithi148.stdout:9/551: creat d0/dd/d25/d6a/fb3 x:0 0 0 2024-07-03T15:43:49.664 INFO:tasks.workunit.client.1.smithi148.stdout:8/258: truncate d4/da/d14/d3a/f41 266473 0 2024-07-03T15:43:49.666 INFO:tasks.workunit.client.1.smithi148.stdout:0/274: creat da/d17/d2a/f59 x:0 0 0 2024-07-03T15:43:49.666 INFO:tasks.workunit.client.1.smithi148.stdout:0/275: fdatasync da/d17/f1e 0 2024-07-03T15:43:49.666 INFO:tasks.workunit.client.1.smithi148.stdout:9/552: write d0/dd/f1f [703370,68412] 0 2024-07-03T15:43:49.669 INFO:tasks.workunit.client.1.smithi148.stdout:3/323: truncate dd/d1c/d2e/f2c 1336966 0 2024-07-03T15:43:49.675 INFO:tasks.workunit.client.1.smithi148.stdout:8/259: mknod d4/da/d51/d18/d1f/c5f 0 2024-07-03T15:43:49.675 INFO:tasks.workunit.client.1.smithi148.stdout:8/260: getdents d4/da/d50 0 2024-07-03T15:43:49.676 INFO:tasks.workunit.client.1.smithi148.stdout:8/261: chown d4/da/d14/d3a/l55 509 1 2024-07-03T15:43:49.678 INFO:tasks.workunit.client.1.smithi148.stdout:8/262: read d4/da/fb [972541,9516] 0 2024-07-03T15:43:49.680 INFO:tasks.workunit.client.1.smithi148.stdout:3/324: creat dd/d1c/d2e/d61/f63 x:0 0 0 2024-07-03T15:43:49.680 INFO:tasks.workunit.client.1.smithi148.stdout:3/325: truncate dd/d33/f3e 984616 0 2024-07-03T15:43:49.680 INFO:tasks.workunit.client.1.smithi148.stdout:3/326: fdatasync dd/d1c/d2e/d40/f4c 0 2024-07-03T15:43:49.681 INFO:tasks.workunit.client.1.smithi148.stdout:3/327: chown dd/f2b 0 1 2024-07-03T15:43:49.681 INFO:tasks.workunit.client.1.smithi148.stdout:3/328: write dd/d33/f39 [963103,48968] 0 2024-07-03T15:43:49.683 INFO:tasks.workunit.client.1.smithi148.stdout:8/263: mkdir d4/d60 0 2024-07-03T15:43:49.683 INFO:tasks.workunit.client.1.smithi148.stdout:9/553: read d0/dd/d25/d37/f50 [2464914,95221] 0 2024-07-03T15:43:49.683 INFO:tasks.workunit.client.1.smithi148.stdout:9/554: stat d0/d18/laf 0 2024-07-03T15:43:49.684 INFO:tasks.workunit.client.1.smithi148.stdout:9/555: fsync d0/dd/d25/f3c 0 2024-07-03T15:43:49.692 INFO:tasks.workunit.client.1.smithi148.stdout:7/373: dwrite d5/d16/d1c/d33/d4c/d21/d31/f45 [0,4194304] 0 2024-07-03T15:43:49.692 INFO:tasks.workunit.client.1.smithi148.stdout:0/276: dwrite da/d17/f1e [0,4194304] 0 2024-07-03T15:43:49.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:49 smithi148 ceph-mon[36056]: pgmap v237: 65 pgs: 65 active+clean; 469 MiB data, 3.1 GiB used, 533 GiB / 536 GiB avail; 29 MiB/s rd, 35 MiB/s wr, 629 op/s 2024-07-03T15:43:49.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:49 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:43:49.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:49 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:43:49.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:49 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:43:49.695 INFO:tasks.workunit.client.1.smithi148.stdout:7/374: symlink d5/d16/d1c/l70 0 2024-07-03T15:43:49.695 INFO:tasks.workunit.client.1.smithi148.stdout:7/375: write d5/f9 [479201,110829] 0 2024-07-03T15:43:49.711 INFO:tasks.workunit.client.1.smithi148.stdout:7/376: truncate d5/d16/f6d 2111611 0 2024-07-03T15:43:49.713 INFO:tasks.workunit.client.1.smithi148.stdout:0/277: mkdir da/d44/d13/d5a 0 2024-07-03T15:43:49.716 INFO:tasks.workunit.client.1.smithi148.stdout:0/278: mkdir da/d36/d5b 0 2024-07-03T15:43:49.733 INFO:tasks.workunit.client.1.smithi148.stdout:6/368: getdents d3/d50 0 2024-07-03T15:43:49.733 INFO:tasks.workunit.client.1.smithi148.stdout:6/369: write d3/d1d/d28/f69 [1040369,17317] 0 2024-07-03T15:43:49.733 INFO:tasks.workunit.client.1.smithi148.stdout:6/370: fdatasync d3/f1c 0 2024-07-03T15:43:49.737 INFO:tasks.workunit.client.1.smithi148.stdout:0/279: dwrite da/d17/d39/f45 [0,4194304] 0 2024-07-03T15:43:49.737 INFO:tasks.workunit.client.1.smithi148.stdout:0/280: write da/d17/d2a/d2f/f37 [448101,106246] 0 2024-07-03T15:43:49.737 INFO:tasks.workunit.client.1.smithi148.stdout:0/281: write da/d44/d13/f18 [624026,12984] 0 2024-07-03T15:43:49.737 INFO:tasks.workunit.client.1.smithi148.stdout:0/282: chown da/d38/f4e 259992 1 2024-07-03T15:43:49.737 INFO:tasks.workunit.client.1.smithi148.stdout:7/377: dwrite d5/d16/d1c/d33/d4c/d21/d39/f3e [0,4194304] 0 2024-07-03T15:43:49.746 INFO:tasks.workunit.client.1.smithi148.stdout:0/283: creat da/d44/d13/d1d/d3e/f5c x:0 0 0 2024-07-03T15:43:49.746 INFO:tasks.workunit.client.1.smithi148.stdout:0/284: chown da/c1c 15 1 2024-07-03T15:43:49.748 INFO:tasks.workunit.client.1.smithi148.stdout:7/378: creat d5/d16/d1c/f71 x:0 0 0 2024-07-03T15:43:49.754 INFO:tasks.workunit.client.1.smithi148.stdout:0/285: creat da/d44/d13/d14/d50/f5d x:0 0 0 2024-07-03T15:43:49.754 INFO:tasks.workunit.client.1.smithi148.stdout:0/286: write da/d44/d24/f52 [585864,99044] 0 2024-07-03T15:43:49.756 INFO:tasks.workunit.client.1.smithi148.stdout:7/379: rename d5/d16/d27/l50 to d5/d16/d1c/l72 0 2024-07-03T15:43:49.757 INFO:tasks.workunit.client.1.smithi148.stdout:0/287: creat da/d44/d13/d14/d35/f5e x:0 0 0 2024-07-03T15:43:49.757 INFO:tasks.workunit.client.1.smithi148.stdout:0/288: chown da/cb 395 1 2024-07-03T15:43:49.758 INFO:tasks.workunit.client.1.smithi148.stdout:7/380: mknod d5/d16/d1c/d4a/c73 0 2024-07-03T15:43:49.760 INFO:tasks.workunit.client.1.smithi148.stdout:0/289: rename da/d44/ff to da/d17/d2a/d2f/f5f 0 2024-07-03T15:43:49.762 INFO:tasks.workunit.client.1.smithi148.stdout:7/381: symlink d5/d16/d1c/d33/l74 0 2024-07-03T15:43:49.762 INFO:tasks.workunit.client.1.smithi148.stdout:7/382: write d5/d16/f34 [2595100,109506] 0 2024-07-03T15:43:49.767 INFO:tasks.workunit.client.1.smithi148.stdout:7/383: rmdir d5/d16/d1c/d33/d4c/d21/d47 39 2024-07-03T15:43:49.773 INFO:tasks.workunit.client.1.smithi148.stdout:0/290: dwrite da/d17/f47 [0,4194304] 0 2024-07-03T15:43:49.790 INFO:tasks.workunit.client.1.smithi148.stdout:0/291: dwrite da/d44/d13/d1d/d3e/f5c [0,4194304] 0 2024-07-03T15:43:49.796 INFO:tasks.workunit.client.1.smithi148.stdout:0/292: truncate da/d44/f19 1396174 0 2024-07-03T15:43:49.796 INFO:tasks.workunit.client.1.smithi148.stdout:0/293: chown da/f16 4846 1 2024-07-03T15:43:49.798 INFO:tasks.workunit.client.1.smithi148.stdout:0/294: dread da/d44/d13/d14/f32 [0,4194304] 0 2024-07-03T15:43:49.801 INFO:tasks.workunit.client.1.smithi148.stdout:0/295: rmdir da/d36 39 2024-07-03T15:43:49.801 INFO:tasks.workunit.client.1.smithi148.stdout:0/296: getdents da/d36/d5b 0 2024-07-03T15:43:49.801 INFO:tasks.workunit.client.1.smithi148.stdout:0/297: stat da/d17/d39/f45 0 2024-07-03T15:43:49.805 INFO:tasks.workunit.client.1.smithi148.stdout:3/329: getdents dd 0 2024-07-03T15:43:49.805 INFO:tasks.workunit.client.1.smithi148.stdout:3/330: creat dd/d1c/d2e/d61/f64 x:0 0 0 2024-07-03T15:43:49.805 INFO:tasks.workunit.client.1.smithi148.stdout:3/331: fdatasync dd/f2a 0 2024-07-03T15:43:49.822 INFO:tasks.workunit.client.1.smithi148.stdout:9/556: truncate d0/d18/d24/f91 548829 0 2024-07-03T15:43:49.830 INFO:tasks.workunit.client.1.smithi148.stdout:8/264: dwrite d4/da/d14/d3a/f41 [0,4194304] 0 2024-07-03T15:43:49.830 INFO:tasks.workunit.client.1.smithi148.stdout:0/298: dwrite da/d44/d24/f29 [4194304,4194304] 0 2024-07-03T15:43:49.835 INFO:tasks.workunit.client.1.smithi148.stdout:3/332: dwrite dd/f3d [0,4194304] 0 2024-07-03T15:43:49.838 INFO:tasks.workunit.client.1.smithi148.stdout:8/265: dread d4/da/d14/d1a/f37 [0,4194304] 0 2024-07-03T15:43:49.840 INFO:tasks.workunit.client.1.smithi148.stdout:3/333: dread dd/d33/f39 [0,4194304] 0 2024-07-03T15:43:49.840 INFO:tasks.workunit.client.1.smithi148.stdout:3/334: chown dd/d33/d52/f58 41162 1 2024-07-03T15:43:49.840 INFO:tasks.workunit.client.1.smithi148.stdout:3/335: truncate dd/d1c/f5c 843202 0 2024-07-03T15:43:49.846 INFO:tasks.workunit.client.1.smithi148.stdout:8/266: mkdir d4/da/d14/d1a/d32/d38/d4a/d61 0 2024-07-03T15:43:49.847 INFO:tasks.workunit.client.1.smithi148.stdout:3/336: stat dd/d1c/l4d 0 2024-07-03T15:43:49.851 INFO:tasks.workunit.client.1.smithi148.stdout:8/267: unlink d4/da/l1c 0 2024-07-03T15:43:49.853 INFO:tasks.workunit.client.1.smithi148.stdout:3/337: mkdir dd/d1c/d2e/d40/d45/d65 0 2024-07-03T15:43:49.855 INFO:tasks.workunit.client.1.smithi148.stdout:3/338: symlink dd/d1c/d2e/d40/d45/l66 0 2024-07-03T15:43:49.855 INFO:tasks.workunit.client.1.smithi148.stdout:3/339: write dd/d33/f50 [76395,8850] 0 2024-07-03T15:43:49.855 INFO:tasks.workunit.client.1.smithi148.stdout:3/340: write dd/d1c/f3c [731473,12469] 0 2024-07-03T15:43:49.855 INFO:tasks.workunit.client.1.smithi148.stdout:3/341: stat dd/d35/d41/l51 0 2024-07-03T15:43:49.855 INFO:tasks.workunit.client.1.smithi148.stdout:3/342: stat dd/d1c/d2e/d40/d45 0 2024-07-03T15:43:49.857 INFO:tasks.workunit.client.1.smithi148.stdout:3/343: mkdir dd/d1c/d67 0 2024-07-03T15:43:49.861 INFO:tasks.workunit.client.1.smithi148.stdout:7/384: write d5/f6 [356762,12959] 0 2024-07-03T15:43:49.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:49 smithi028 ceph-mon[29148]: pgmap v237: 65 pgs: 65 active+clean; 469 MiB data, 3.1 GiB used, 533 GiB / 536 GiB avail; 29 MiB/s rd, 35 MiB/s wr, 629 op/s 2024-07-03T15:43:49.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:49 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:43:49.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:49 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:43:49.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:49 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:43:49.908 INFO:tasks.workunit.client.1.smithi148.stdout:7/385: getdents d5/d16/d1c/d33 0 2024-07-03T15:43:49.909 INFO:tasks.workunit.client.1.smithi148.stdout:7/386: write d5/f9 [318932,128161] 0 2024-07-03T15:43:49.966 INFO:tasks.workunit.client.1.smithi148.stdout:5/267: sync 2024-07-03T15:43:49.967 INFO:tasks.workunit.client.1.smithi148.stdout:1/244: sync 2024-07-03T15:43:49.967 INFO:tasks.workunit.client.1.smithi148.stdout:4/229: sync 2024-07-03T15:43:49.967 INFO:tasks.workunit.client.1.smithi148.stdout:2/383: sync 2024-07-03T15:43:49.967 INFO:tasks.workunit.client.1.smithi148.stdout:1/245: chown d0/d2/d6/d14/d29 674334749 1 2024-07-03T15:43:49.968 INFO:tasks.workunit.client.1.smithi148.stdout:5/268: mknod d7/da/df/d1d/c57 0 2024-07-03T15:43:49.970 INFO:tasks.workunit.client.1.smithi148.stdout:4/230: truncate d0/d8/f10 4516599 0 2024-07-03T15:43:49.970 INFO:tasks.workunit.client.1.smithi148.stdout:4/231: dread - d0/d8/d19/d24/f3f zero size 2024-07-03T15:43:49.970 INFO:tasks.workunit.client.1.smithi148.stdout:4/232: chown d0/d8 335437 1 2024-07-03T15:43:49.973 INFO:tasks.workunit.client.1.smithi148.stdout:5/269: mkdir d7/da/df/d16/d1a/d58 0 2024-07-03T15:43:49.973 INFO:tasks.workunit.client.1.smithi148.stdout:5/270: chown d7/d13/d23/l46 198489650 1 2024-07-03T15:43:49.973 INFO:tasks.workunit.client.1.smithi148.stdout:5/271: chown d7/da/df/d16/d2f/d44 0 1 2024-07-03T15:43:49.973 INFO:tasks.workunit.client.1.smithi148.stdout:5/272: write d7/da/df/d16/d2f/f38 [4691347,12934] 0 2024-07-03T15:43:49.974 INFO:tasks.workunit.client.1.smithi148.stdout:5/273: readlink d7/da/df/d16/d1a/l47 0 2024-07-03T15:43:49.974 INFO:tasks.workunit.client.1.smithi148.stdout:5/274: readlink d7/d13/d23/l49 0 2024-07-03T15:43:49.974 INFO:tasks.workunit.client.1.smithi148.stdout:5/275: truncate d7/da/f4c 100691 0 2024-07-03T15:43:49.974 INFO:tasks.workunit.client.1.smithi148.stdout:5/276: write d7/da/df/d1d/f2e [739166,32095] 0 2024-07-03T15:43:49.974 INFO:tasks.workunit.client.1.smithi148.stdout:4/233: link d0/d8/d22/d2f/f33 d0/d8/d3a/f47 0 2024-07-03T15:43:49.976 INFO:tasks.workunit.client.1.smithi148.stdout:2/384: write d5/d14/d22/d2b/d13/f1b [4348317,80351] 0 2024-07-03T15:43:49.976 INFO:tasks.workunit.client.1.smithi148.stdout:5/277: mkdir d7/d59 0 2024-07-03T15:43:49.979 INFO:tasks.workunit.client.1.smithi148.stdout:4/234: truncate d0/d8/f2d 1944892 0 2024-07-03T15:43:49.983 INFO:tasks.workunit.client.1.smithi148.stdout:5/278: link d7/d13/d23/c43 d7/da/df/d16/d1a/c5a 0 2024-07-03T15:43:49.986 INFO:tasks.workunit.client.1.smithi148.stdout:2/385: getdents d5/d7/d64 0 2024-07-03T15:43:49.987 INFO:tasks.workunit.client.1.smithi148.stdout:2/386: getdents d5/d14/d22/d2b/d13 0 2024-07-03T15:43:49.987 INFO:tasks.workunit.client.1.smithi148.stdout:1/246: dwrite d0/d2/d11/d1e/f51 [4194304,4194304] 0 2024-07-03T15:43:49.989 INFO:tasks.workunit.client.1.smithi148.stdout:3/344: rename dd/d1c to dd/d33/d37/d68 0 2024-07-03T15:43:49.989 INFO:tasks.workunit.client.1.smithi148.stdout:3/345: write dd/d33/f39 [2054745,130138] 0 2024-07-03T15:43:49.989 INFO:tasks.workunit.client.1.smithi148.stdout:3/346: stat dd/d33/d37/f46 0 2024-07-03T15:43:49.991 INFO:tasks.workunit.client.1.smithi148.stdout:1/247: getdents d0/d2/d6/d14/d29 0 2024-07-03T15:43:49.991 INFO:tasks.workunit.client.1.smithi148.stdout:5/279: dread f4 [0,4194304] 0 2024-07-03T15:43:49.992 INFO:tasks.workunit.client.1.smithi148.stdout:5/280: truncate d7/d13/d23/d40/f56 4983282 0 2024-07-03T15:43:49.993 INFO:tasks.workunit.client.1.smithi148.stdout:7/387: rename d5/d16/d1c/d23/c5c to d5/d16/d1c/d33/d4c/c75 0 2024-07-03T15:43:49.996 INFO:tasks.workunit.client.1.smithi148.stdout:3/347: mknod dd/d33/d37/d68/d2e/d40/d45/d65/c69 0 2024-07-03T15:43:49.996 INFO:tasks.workunit.client.1.smithi148.stdout:3/348: dread - dd/f62 zero size 2024-07-03T15:43:49.997 INFO:tasks.workunit.client.1.smithi148.stdout:3/349: dread dd/d33/f3a [0,4194304] 0 2024-07-03T15:43:50.000 INFO:tasks.workunit.client.1.smithi148.stdout:7/388: creat d5/d16/d1c/d59/f76 x:0 0 0 2024-07-03T15:43:50.005 INFO:tasks.workunit.client.1.smithi148.stdout:7/389: creat d5/d16/d1c/d33/d4c/d21/d39/f77 x:0 0 0 2024-07-03T15:43:50.006 INFO:tasks.workunit.client.1.smithi148.stdout:4/235: rename d0/d8/d19/d24 to d0/d8/d13/d48 0 2024-07-03T15:43:50.008 INFO:tasks.workunit.client.1.smithi148.stdout:7/390: link d5/d16/d1c/d4a/c73 d5/d16/d1c/d23/c78 0 2024-07-03T15:43:50.008 INFO:tasks.workunit.client.1.smithi148.stdout:7/391: write d5/d16/d1c/f71 [497574,34569] 0 2024-07-03T15:43:50.010 INFO:tasks.workunit.client.1.smithi148.stdout:2/387: rename d5/d14/d22/d2b/f1a to d5/d14/d22/d2b/d68/f7c 0 2024-07-03T15:43:50.010 INFO:tasks.workunit.client.1.smithi148.stdout:2/388: readlink d5/d14/l3f 0 2024-07-03T15:43:50.011 INFO:tasks.workunit.client.1.smithi148.stdout:4/236: mkdir d0/d8/d19/d45/d49 0 2024-07-03T15:43:50.011 INFO:tasks.workunit.client.1.smithi148.stdout:4/237: stat d0/d8 0 2024-07-03T15:43:50.014 INFO:tasks.workunit.client.1.smithi148.stdout:1/248: rename d0/de/d2f/d40/c4c to d0/d2/d2e/c52 0 2024-07-03T15:43:50.016 INFO:tasks.workunit.client.1.smithi148.stdout:2/389: link d5/d14/c55 d5/d14/d22/d2b/d13/c7d 0 2024-07-03T15:43:50.016 INFO:tasks.workunit.client.1.smithi148.stdout:2/390: write d5/d7/d64/f71 [644099,129915] 0 2024-07-03T15:43:50.017 INFO:tasks.workunit.client.1.smithi148.stdout:7/392: dread d5/d16/d1c/d4a/d52/f6b [0,4194304] 0 2024-07-03T15:43:50.017 INFO:tasks.workunit.client.1.smithi148.stdout:1/249: symlink d0/d2/d6/l53 0 2024-07-03T15:43:50.018 INFO:tasks.workunit.client.1.smithi148.stdout:4/238: read d0/f6 [3213128,93723] 0 2024-07-03T15:43:50.023 INFO:tasks.workunit.client.1.smithi148.stdout:1/250: unlink d0/d2/d6/f20 0 2024-07-03T15:43:50.024 INFO:tasks.workunit.client.1.smithi148.stdout:1/251: write d0/d2/d6/fa [2146934,123873] 0 2024-07-03T15:43:50.024 INFO:tasks.workunit.client.1.smithi148.stdout:1/252: write d0/f45 [601128,33941] 0 2024-07-03T15:43:50.024 INFO:tasks.workunit.client.1.smithi148.stdout:4/239: mknod d0/d8/d13/c4a 0 2024-07-03T15:43:50.030 INFO:tasks.workunit.client.1.smithi148.stdout:4/240: creat d0/d8/f4b x:0 0 0 2024-07-03T15:43:50.034 INFO:tasks.workunit.client.1.smithi148.stdout:4/241: symlink d0/d8/d19/d45/d49/l4c 0 2024-07-03T15:43:50.036 INFO:tasks.workunit.client.1.smithi148.stdout:2/391: dwrite d5/d14/d22/d2b/f3a [4194304,4194304] 0 2024-07-03T15:43:50.036 INFO:tasks.workunit.client.1.smithi148.stdout:2/392: write d5/d14/d22/d60/f65 [207687,38155] 0 2024-07-03T15:43:50.041 INFO:tasks.workunit.client.1.smithi148.stdout:1/253: dwrite d0/d2/d6/d14/f15 [4194304,4194304] 0 2024-07-03T15:43:50.045 INFO:tasks.workunit.client.1.smithi148.stdout:2/393: write d5/d14/d22/d2b/d13/d41/f74 [2061048,94071] 0 2024-07-03T15:43:50.046 INFO:tasks.workunit.client.1.smithi148.stdout:2/394: chown d5/d14/d22/d2b/d13/d41/f4b 51 1 2024-07-03T15:43:50.049 INFO:tasks.workunit.client.1.smithi148.stdout:4/242: dwrite d0/f16 [0,4194304] 0 2024-07-03T15:43:50.049 INFO:tasks.workunit.client.1.smithi148.stdout:2/395: mkdir d5/d5f/d7e 0 2024-07-03T15:43:50.051 INFO:tasks.workunit.client.1.smithi148.stdout:1/254: dread d0/d2/d6/d14/f15 [4194304,4194304] 0 2024-07-03T15:43:50.052 INFO:tasks.workunit.client.1.smithi148.stdout:4/243: creat d0/d8/d22/d2f/d41/f4d x:0 0 0 2024-07-03T15:43:50.054 INFO:tasks.workunit.client.1.smithi148.stdout:2/396: creat d5/d14/d22/d2b/d13/d1d/f7f x:0 0 0 2024-07-03T15:43:50.057 INFO:tasks.workunit.client.1.smithi148.stdout:4/244: symlink d0/d8/d19/l4e 0 2024-07-03T15:43:50.059 INFO:tasks.workunit.client.1.smithi148.stdout:2/397: mkdir d5/d14/d22/d2b/d13/d41/d7b/d6a/d80 0 2024-07-03T15:43:50.061 INFO:tasks.workunit.client.1.smithi148.stdout:4/245: rename d0/d8/f32 to d0/f4f 0 2024-07-03T15:43:50.061 INFO:tasks.workunit.client.1.smithi148.stdout:4/246: chown d0/f16 0 1 2024-07-03T15:43:50.061 INFO:tasks.workunit.client.1.smithi148.stdout:4/247: fsync d0/d8/f38 0 2024-07-03T15:43:50.061 INFO:tasks.workunit.client.1.smithi148.stdout:4/248: creat d0/f50 x:0 0 0 2024-07-03T15:43:50.067 INFO:tasks.workunit.client.1.smithi148.stdout:1/255: dwrite d0/d2/d6/fa [0,4194304] 0 2024-07-03T15:43:50.089 INFO:tasks.workunit.client.1.smithi148.stdout:1/256: dwrite d0/d2/d2e/d39/d42/f48 [0,4194304] 0 2024-07-03T15:43:50.089 INFO:tasks.workunit.client.1.smithi148.stdout:1/257: fdatasync d0/d2/d6/d14/d29/d4b/f4f 0 2024-07-03T15:43:50.103 INFO:tasks.workunit.client.1.smithi148.stdout:2/398: symlink d5/d14/d22/d60/d76/l81 0 2024-07-03T15:43:50.103 INFO:tasks.workunit.client.1.smithi148.stdout:2/399: chown d5/d14/d22/d2b/d13/d41/l70 1343568 1 2024-07-03T15:43:50.103 INFO:tasks.workunit.client.1.smithi148.stdout:2/400: fdatasync d5/d14/d22/d2b/f42 0 2024-07-03T15:43:50.115 INFO:tasks.workunit.client.1.smithi148.stdout:4/249: mknod d0/d8/d19/d45/d49/c51 0 2024-07-03T15:43:50.145 INFO:tasks.workunit.client.1.smithi148.stdout:5/281: getdents d7/da/df/d1d 0 2024-07-03T15:43:50.146 INFO:tasks.workunit.client.1.smithi148.stdout:5/282: creat d7/da/df/d16/d2f/d44/d52/f5b x:0 0 0 2024-07-03T15:43:50.146 INFO:tasks.workunit.client.1.smithi148.stdout:5/283: truncate d7/da/df/d16/d2f/d44/d52/f55 403828 0 2024-07-03T15:43:50.154 INFO:tasks.workunit.client.1.smithi148.stdout:2/401: creat d5/d5f/d7e/f82 x:0 0 0 2024-07-03T15:43:50.154 INFO:tasks.workunit.client.1.smithi148.stdout:5/284: read d7/da/df/d16/f1b [2355717,67680] 0 2024-07-03T15:43:50.157 INFO:tasks.workunit.client.1.smithi148.stdout:4/250: mkdir d0/d8/d22/d2f/d52 0 2024-07-03T15:43:50.157 INFO:tasks.workunit.client.1.smithi148.stdout:4/251: write d0/d8/d22/d2f/d41/f4d [26023,33022] 0 2024-07-03T15:43:50.158 INFO:tasks.workunit.client.1.smithi148.stdout:1/258: getdents d0/d2/d2e/d39/d42 0 2024-07-03T15:43:50.162 INFO:tasks.workunit.client.1.smithi148.stdout:3/350: getdents dd 0 2024-07-03T15:43:50.166 INFO:tasks.workunit.client.1.smithi148.stdout:5/285: creat d7/d13/d23/d40/f5c x:0 0 0 2024-07-03T15:43:50.166 INFO:tasks.workunit.client.1.smithi148.stdout:5/286: fsync d7/d13/f2a 0 2024-07-03T15:43:50.168 INFO:tasks.workunit.client.1.smithi148.stdout:9/557: sync 2024-07-03T15:43:50.168 INFO:tasks.workunit.client.1.smithi148.stdout:6/371: sync 2024-07-03T15:43:50.168 INFO:tasks.workunit.client.1.smithi148.stdout:9/558: chown d0/f53 6633 1 2024-07-03T15:43:50.169 INFO:tasks.workunit.client.1.smithi148.stdout:6/372: write d3/d1d/d28/d3e/f64 [1068632,95033] 0 2024-07-03T15:43:50.169 INFO:tasks.workunit.client.1.smithi148.stdout:6/373: dread - d3/d1d/d28/d31/d61/f6b zero size 2024-07-03T15:43:50.169 INFO:tasks.workunit.client.1.smithi148.stdout:6/374: chown d3/d1d/d28/d31/c40 189652501 1 2024-07-03T15:43:50.169 INFO:tasks.workunit.client.1.smithi148.stdout:4/252: link d0/l2 d0/d8/d22/d2f/l53 0 2024-07-03T15:43:50.169 INFO:tasks.workunit.client.1.smithi148.stdout:4/253: fdatasync d0/d8/d13/f1a 0 2024-07-03T15:43:50.169 INFO:tasks.workunit.client.1.smithi148.stdout:4/254: write d0/f30 [496276,125888] 0 2024-07-03T15:43:50.169 INFO:tasks.workunit.client.1.smithi148.stdout:4/255: rename d0/d8/d22 to d0/d8/d22/d2f/d41/d54 22 2024-07-03T15:43:50.169 INFO:tasks.workunit.client.1.smithi148.stdout:8/268: sync 2024-07-03T15:43:50.169 INFO:tasks.workunit.client.1.smithi148.stdout:0/299: sync 2024-07-03T15:43:50.169 INFO:tasks.workunit.client.1.smithi148.stdout:0/300: truncate da/d44/d13/d14/f49 1032312 0 2024-07-03T15:43:50.169 INFO:tasks.workunit.client.1.smithi148.stdout:0/301: creat da/d44/d13/d14/d50/f60 x:0 0 0 2024-07-03T15:43:50.170 INFO:tasks.workunit.client.1.smithi148.stdout:1/259: mkdir d0/de/d44/d54 0 2024-07-03T15:43:50.171 INFO:tasks.workunit.client.1.smithi148.stdout:3/351: mknod dd/d33/d37/d68/d2e/d61/c6a 0 2024-07-03T15:43:50.182 INFO:tasks.workunit.client.1.smithi148.stdout:9/559: rmdir d0/d18/d24/d60 39 2024-07-03T15:43:50.182 INFO:tasks.workunit.client.1.smithi148.stdout:9/560: fsync d0/f6 0 2024-07-03T15:43:50.184 INFO:tasks.workunit.client.1.smithi148.stdout:6/375: link d3/c3f d3/d1d/d28/d31/d61/c82 0 2024-07-03T15:43:50.186 INFO:tasks.workunit.client.1.smithi148.stdout:0/302: dwrite da/d17/d2a/f2d [0,4194304] 0 2024-07-03T15:43:50.187 INFO:tasks.workunit.client.1.smithi148.stdout:4/256: rename d0/d8/d22/d2f/l53 to d0/d8/d19/d45/l55 0 2024-07-03T15:43:50.187 INFO:tasks.workunit.client.1.smithi148.stdout:4/257: write d0/d8/f10 [3553881,53927] 0 2024-07-03T15:43:50.195 INFO:tasks.workunit.client.1.smithi148.stdout:3/352: symlink dd/d33/d37/d68/d2e/l6b 0 2024-07-03T15:43:50.195 INFO:tasks.workunit.client.1.smithi148.stdout:3/353: dread - dd/d35/f4f zero size 2024-07-03T15:43:50.195 INFO:tasks.workunit.client.1.smithi148.stdout:3/354: write dd/f34 [1479077,16092] 0 2024-07-03T15:43:50.195 INFO:tasks.workunit.client.1.smithi148.stdout:3/355: creat dd/d33/f6c x:0 0 0 2024-07-03T15:43:50.195 INFO:tasks.workunit.client.1.smithi148.stdout:3/356: truncate dd/f21 1601865 0 2024-07-03T15:43:50.199 INFO:tasks.workunit.client.1.smithi148.stdout:5/287: getdents d7/da/df/d16 0 2024-07-03T15:43:50.199 INFO:tasks.workunit.client.1.smithi148.stdout:9/561: dwrite d0/dd/d11/f79 [0,4194304] 0 2024-07-03T15:43:50.206 INFO:tasks.workunit.client.1.smithi148.stdout:4/258: creat d0/d8/d19/d45/d49/f56 x:0 0 0 2024-07-03T15:43:50.206 INFO:tasks.workunit.client.1.smithi148.stdout:4/259: rename d0/d8/d44 to d0/d8/d44/d57 22 2024-07-03T15:43:50.207 INFO:tasks.workunit.client.1.smithi148.stdout:8/269: dwrite d4/da/d51/d18/d1f/f42 [0,4194304] 0 2024-07-03T15:43:50.210 INFO:tasks.workunit.client.1.smithi148.stdout:3/357: mkdir dd/d33/d37/d68/d2e/d40/d45/d6d 0 2024-07-03T15:43:50.211 INFO:tasks.workunit.client.1.smithi148.stdout:3/358: truncate dd/d33/f6c 708428 0 2024-07-03T15:43:50.211 INFO:tasks.workunit.client.1.smithi148.stdout:3/359: creat dd/d33/d52/f6e x:0 0 0 2024-07-03T15:43:50.216 INFO:tasks.workunit.client.1.smithi148.stdout:5/288: creat d7/d13/d54/f5d x:0 0 0 2024-07-03T15:43:50.216 INFO:tasks.workunit.client.1.smithi148.stdout:5/289: truncate d7/d13/d23/d40/f5c 15701 0 2024-07-03T15:43:50.216 INFO:tasks.workunit.client.1.smithi148.stdout:5/290: creat d7/da/d21/f5e x:0 0 0 2024-07-03T15:43:50.223 INFO:tasks.workunit.client.1.smithi148.stdout:6/376: rmdir d3/d1d/d28/d31/d4c 39 2024-07-03T15:43:50.225 INFO:tasks.workunit.client.1.smithi148.stdout:0/303: getdents da/d44/d13/d14/d35 0 2024-07-03T15:43:50.229 INFO:tasks.workunit.client.1.smithi148.stdout:4/260: dwrite d0/d8/f38 [0,4194304] 0 2024-07-03T15:43:50.229 INFO:tasks.workunit.client.1.smithi148.stdout:4/261: chown d0/d8/d22/d2f/d41/f4d 10 1 2024-07-03T15:43:50.229 INFO:tasks.workunit.client.1.smithi148.stdout:3/360: dwrite dd/f44 [0,4194304] 0 2024-07-03T15:43:50.229 INFO:tasks.workunit.client.1.smithi148.stdout:3/361: write dd/d33/d37/f46 [5153992,13216] 0 2024-07-03T15:43:50.230 INFO:tasks.workunit.client.1.smithi148.stdout:8/270: truncate d4/da/d51/d18/d20/f5a 353932 0 2024-07-03T15:43:50.237 INFO:tasks.workunit.client.1.smithi148.stdout:6/377: mkdir d3/d1d/d28/d31/d83 0 2024-07-03T15:43:50.240 INFO:tasks.workunit.client.1.smithi148.stdout:0/304: creat da/d44/d13/d14/f61 x:0 0 0 2024-07-03T15:43:50.241 INFO:tasks.workunit.client.1.smithi148.stdout:3/362: dwrite dd/d33/d37/d68/f30 [0,4194304] 0 2024-07-03T15:43:50.248 INFO:tasks.workunit.client.1.smithi148.stdout:4/262: mkdir d0/d8/d44/d58 0 2024-07-03T15:43:50.250 INFO:tasks.workunit.client.1.smithi148.stdout:8/271: symlink d4/da/d14/d1a/d32/l62 0 2024-07-03T15:43:50.250 INFO:tasks.workunit.client.1.smithi148.stdout:8/272: chown d4 10043 1 2024-07-03T15:43:50.253 INFO:tasks.workunit.client.1.smithi148.stdout:6/378: creat d3/d5b/d74/f84 x:0 0 0 2024-07-03T15:43:50.256 INFO:tasks.workunit.client.1.smithi148.stdout:0/305: unlink da/d38/c4c 0 2024-07-03T15:43:50.265 INFO:tasks.workunit.client.1.smithi148.stdout:3/363: mknod dd/d33/c6f 0 2024-07-03T15:43:50.265 INFO:tasks.workunit.client.1.smithi148.stdout:3/364: fsync dd/d35/f4e 0 2024-07-03T15:43:50.265 INFO:tasks.workunit.client.1.smithi148.stdout:3/365: fdatasync dd/f62 0 2024-07-03T15:43:50.271 INFO:tasks.workunit.client.1.smithi148.stdout:3/366: symlink dd/d33/d37/d68/d67/l70 0 2024-07-03T15:43:50.274 INFO:tasks.workunit.client.1.smithi148.stdout:3/367: symlink dd/d33/d37/d68/d2e/d40/l71 0 2024-07-03T15:43:50.283 INFO:tasks.workunit.client.1.smithi148.stdout:8/273: dwrite d4/da/d14/d1a/f37 [0,4194304] 0 2024-07-03T15:43:50.283 INFO:tasks.workunit.client.1.smithi148.stdout:8/274: truncate d4/da/f3e 1356666 0 2024-07-03T15:43:50.283 INFO:tasks.workunit.client.1.smithi148.stdout:8/275: creat d4/d54/f63 x:0 0 0 2024-07-03T15:43:50.283 INFO:tasks.workunit.client.1.smithi148.stdout:4/263: dwrite d0/f17 [0,4194304] 0 2024-07-03T15:43:50.285 INFO:tasks.workunit.client.1.smithi148.stdout:6/379: dwrite d3/d1d/d28/d3e/f58 [0,4194304] 0 2024-07-03T15:43:50.288 INFO:tasks.workunit.client.1.smithi148.stdout:4/264: dread d0/f5 [0,4194304] 0 2024-07-03T15:43:50.290 INFO:tasks.workunit.client.1.smithi148.stdout:8/276: symlink d4/da/d47/l64 0 2024-07-03T15:43:50.290 INFO:tasks.workunit.client.1.smithi148.stdout:8/277: write d4/da/d51/d18/d20/f5a [485831,30953] 0 2024-07-03T15:43:50.299 INFO:tasks.workunit.client.1.smithi148.stdout:8/278: symlink d4/da/d14/d1a/d32/d38/d58/l65 0 2024-07-03T15:43:50.300 INFO:tasks.workunit.client.1.smithi148.stdout:6/380: dwrite d3/f1e [0,4194304] 0 2024-07-03T15:43:50.301 INFO:tasks.workunit.client.1.smithi148.stdout:6/381: truncate d3/d1d/d28/f2a 2067890 0 2024-07-03T15:43:50.304 INFO:tasks.workunit.client.1.smithi148.stdout:8/279: dread d4/da/d51/d18/d20/f5a [0,4194304] 0 2024-07-03T15:43:50.305 INFO:tasks.workunit.client.1.smithi148.stdout:6/382: truncate d3/d1d/d28/f73 3132609 0 2024-07-03T15:43:50.307 INFO:tasks.workunit.client.1.smithi148.stdout:8/280: unlink d4/da/d51/c15 0 2024-07-03T15:43:50.316 INFO:tasks.workunit.client.1.smithi148.stdout:8/281: dread d4/da/f28 [0,4194304] 0 2024-07-03T15:43:50.316 INFO:tasks.workunit.client.1.smithi148.stdout:8/282: creat d4/da/d51/d18/d20/f66 x:0 0 0 2024-07-03T15:43:50.320 INFO:tasks.workunit.client.1.smithi148.stdout:8/283: chown d4/da/d51/d18/c3c 141 1 2024-07-03T15:43:50.320 INFO:tasks.workunit.client.1.smithi148.stdout:8/284: readlink d4/da/d14/d1a/d32/l62 0 2024-07-03T15:43:50.320 INFO:tasks.workunit.client.1.smithi148.stdout:8/285: fsync d4/da/d14/f16 0 2024-07-03T15:43:50.324 INFO:tasks.workunit.client.1.smithi148.stdout:6/383: dwrite f1 [4194304,4194304] 0 2024-07-03T15:43:50.328 INFO:tasks.workunit.client.1.smithi148.stdout:6/384: mkdir d3/d1d/d28/d7f/d85 0 2024-07-03T15:43:50.329 INFO:tasks.workunit.client.1.smithi148.stdout:8/286: write d4/da/d14/d3a/f41 [3206537,57335] 0 2024-07-03T15:43:50.336 INFO:tasks.workunit.client.1.smithi148.stdout:9/562: rmdir d0/d18/d24 39 2024-07-03T15:43:50.338 INFO:tasks.workunit.client.1.smithi148.stdout:9/563: mkdir d0/dd/d11/db4 0 2024-07-03T15:43:50.345 INFO:tasks.workunit.client.1.smithi148.stdout:8/287: dwrite d4/da/d14/d1a/f37 [0,4194304] 0 2024-07-03T15:43:50.351 INFO:tasks.workunit.client.1.smithi148.stdout:8/288: truncate d4/da/f28 355064 0 2024-07-03T15:43:50.368 INFO:tasks.workunit.client.1.smithi148.stdout:8/289: dwrite d4/da/d51/f7 [0,4194304] 0 2024-07-03T15:43:50.369 INFO:tasks.workunit.client.1.smithi148.stdout:8/290: write d4/da/d14/d1a/d32/d38/d4a/f27 [425635,7435] 0 2024-07-03T15:43:50.387 INFO:tasks.workunit.client.1.smithi148.stdout:2/402: creat d5/d7/f83 x:0 0 0 2024-07-03T15:43:50.390 INFO:tasks.workunit.client.1.smithi148.stdout:2/403: creat d5/d14/d22/d2b/d68/f84 x:0 0 0 2024-07-03T15:43:50.492 INFO:tasks.workunit.client.1.smithi148.stdout:5/291: creat d7/da/f5f x:0 0 0 2024-07-03T15:43:50.492 INFO:tasks.workunit.client.1.smithi148.stdout:5/292: fdatasync f4 0 2024-07-03T15:43:50.495 INFO:tasks.workunit.client.1.smithi148.stdout:5/293: unlink d7/da/df/d16/c2c 0 2024-07-03T15:43:50.497 INFO:tasks.workunit.client.1.smithi148.stdout:5/294: mkdir d7/da/df/d16/d2f/d44/d52/d60 0 2024-07-03T15:43:50.497 INFO:tasks.workunit.client.1.smithi148.stdout:5/295: fsync d7/d13/d54/f4b 0 2024-07-03T15:43:50.498 INFO:tasks.workunit.client.1.smithi148.stdout:5/296: symlink d7/da/df/d16/d2f/d44/d52/l61 0 2024-07-03T15:43:50.500 INFO:tasks.workunit.client.1.smithi148.stdout:5/297: symlink d7/d59/l62 0 2024-07-03T15:43:50.509 INFO:tasks.workunit.client.1.smithi148.stdout:0/306: rmdir da 39 2024-07-03T15:43:50.511 INFO:tasks.workunit.client.1.smithi148.stdout:0/307: symlink da/d44/d13/d14/l62 0 2024-07-03T15:43:50.511 INFO:tasks.workunit.client.1.smithi148.stdout:0/308: dread - da/d44/d13/d14/d50/f5d zero size 2024-07-03T15:43:50.515 INFO:tasks.workunit.client.1.smithi148.stdout:0/309: dread da/d44/d13/d14/f1f [0,4194304] 0 2024-07-03T15:43:50.529 INFO:tasks.workunit.client.1.smithi148.stdout:6/385: creat d3/f86 x:0 0 0 2024-07-03T15:43:50.529 INFO:tasks.workunit.client.1.smithi148.stdout:6/386: chown d3/d1d/d28/d31/d61/c82 3 1 2024-07-03T15:43:50.532 INFO:tasks.workunit.client.1.smithi148.stdout:8/291: rmdir d4/d60 0 2024-07-03T15:43:50.534 INFO:tasks.workunit.client.1.smithi148.stdout:8/292: unlink d4/da/d14/d3a/l55 0 2024-07-03T15:43:50.536 INFO:tasks.workunit.client.1.smithi148.stdout:8/293: write d4/da/d51/d18/d20/f40 [2633935,22182] 0 2024-07-03T15:43:50.537 INFO:tasks.workunit.client.1.smithi148.stdout:8/294: creat d4/da/d14/f67 x:0 0 0 2024-07-03T15:43:50.538 INFO:tasks.workunit.client.1.smithi148.stdout:8/295: read d4/da/d51/d18/d20/f5a [111127,69068] 0 2024-07-03T15:43:50.538 INFO:tasks.workunit.client.1.smithi148.stdout:8/296: stat d4/da/d47/c4f 0 2024-07-03T15:43:50.538 INFO:tasks.workunit.client.1.smithi148.stdout:8/297: creat d4/da/d14/f68 x:0 0 0 2024-07-03T15:43:50.542 INFO:tasks.workunit.client.1.smithi148.stdout:8/298: chown d4/da/d51/d53 10 1 2024-07-03T15:43:50.544 INFO:tasks.workunit.client.1.smithi148.stdout:6/387: dwrite d3/d1d/d28/d53/f80 [0,4194304] 0 2024-07-03T15:43:50.557 INFO:tasks.workunit.client.1.smithi148.stdout:3/368: rename dd/d33/d52/f6e to dd/f72 0 2024-07-03T15:43:50.561 INFO:tasks.workunit.client.1.smithi148.stdout:9/564: rename d0/dd/d25/d59/f4f to d0/d2f/fb5 0 2024-07-03T15:43:50.561 INFO:tasks.workunit.client.1.smithi148.stdout:9/565: creat d0/d49/d43/d19/d7a/fb6 x:0 0 0 2024-07-03T15:43:50.562 INFO:tasks.workunit.client.1.smithi148.stdout:3/369: mkdir dd/d33/d37/d68/d2e/d40/d45/d73 0 2024-07-03T15:43:50.562 INFO:tasks.workunit.client.1.smithi148.stdout:6/388: dwrite d3/d1d/d28/d31/d61/f6b [0,4194304] 0 2024-07-03T15:43:50.562 INFO:tasks.workunit.client.1.smithi148.stdout:6/389: read d3/d1d/d28/d3e/f64 [2632,105229] 0 2024-07-03T15:43:50.562 INFO:tasks.workunit.client.1.smithi148.stdout:6/390: write d3/d1d/f25 [2534778,84214] 0 2024-07-03T15:43:50.563 INFO:tasks.workunit.client.1.smithi148.stdout:5/298: rename d7/da/df/d16/d1a/l47 to d7/da/d21/l63 0 2024-07-03T15:43:50.563 INFO:tasks.workunit.client.1.smithi148.stdout:5/299: creat d7/da/df/d16/d1a/f64 x:0 0 0 2024-07-03T15:43:50.566 INFO:tasks.workunit.client.1.smithi148.stdout:3/370: mkdir dd/d35/d41/d74 0 2024-07-03T15:43:50.566 INFO:tasks.workunit.client.1.smithi148.stdout:3/371: readlink dd/d33/d37/l42 0 2024-07-03T15:43:50.566 INFO:tasks.workunit.client.1.smithi148.stdout:6/391: dread d3/d1d/d28/d31/f43 [0,4194304] 0 2024-07-03T15:43:50.566 INFO:tasks.workunit.client.1.smithi148.stdout:6/392: chown d3/d1d/d28/d31/d61/f71 1867 1 2024-07-03T15:43:50.570 INFO:tasks.workunit.client.1.smithi148.stdout:0/310: rename da/d17/d2a/d2f/f41 to da/d36/f63 0 2024-07-03T15:43:50.573 INFO:tasks.workunit.client.1.smithi148.stdout:5/300: mknod d7/d13/c65 0 2024-07-03T15:43:50.574 INFO:tasks.workunit.client.1.smithi148.stdout:6/393: write d3/d1d/d28/d31/d4c/f57 [2576593,72487] 0 2024-07-03T15:43:50.575 INFO:tasks.workunit.client.1.smithi148.stdout:3/372: symlink dd/d33/d37/d68/d2e/d40/d45/d65/l75 0 2024-07-03T15:43:50.575 INFO:tasks.workunit.client.1.smithi148.stdout:3/373: getdents dd/d33/d37/d68/d2e/d40/d45/d73 0 2024-07-03T15:43:50.577 INFO:tasks.workunit.client.1.smithi148.stdout:9/566: rename d0/dd/d11/l8a to d0/d18/d24/d47/d4d/dae/lb7 0 2024-07-03T15:43:50.577 INFO:tasks.workunit.client.1.smithi148.stdout:9/567: dread - d0/d49/d43/d19/d7a/f9b zero size 2024-07-03T15:43:50.578 INFO:tasks.workunit.client.1.smithi148.stdout:0/311: mknod da/d36/d5b/c64 0 2024-07-03T15:43:50.579 INFO:tasks.workunit.client.1.smithi148.stdout:7/393: sync 2024-07-03T15:43:50.583 INFO:tasks.workunit.client.1.smithi148.stdout:1/260: rmdir d0/d2/d6 39 2024-07-03T15:43:50.583 INFO:tasks.workunit.client.1.smithi148.stdout:1/261: fdatasync d0/d2/d6/fa 0 2024-07-03T15:43:50.583 INFO:tasks.workunit.client.1.smithi148.stdout:1/262: fsync d0/d2/d11/d1e/f51 0 2024-07-03T15:43:50.583 INFO:tasks.workunit.client.1.smithi148.stdout:1/263: creat d0/de/d2f/f55 x:0 0 0 2024-07-03T15:43:50.583 INFO:tasks.workunit.client.1.smithi148.stdout:1/264: chown d0/d2/d6/d14/f15 255035103 1 2024-07-03T15:43:50.583 INFO:tasks.workunit.client.1.smithi148.stdout:1/265: dread - d0/de/d2f/f55 zero size 2024-07-03T15:43:50.584 INFO:tasks.workunit.client.1.smithi148.stdout:3/374: creat dd/d33/d37/d68/d2e/d61/f76 x:0 0 0 2024-07-03T15:43:50.586 INFO:tasks.workunit.client.1.smithi148.stdout:0/312: symlink da/d44/d13/d14/l65 0 2024-07-03T15:43:50.587 INFO:tasks.workunit.client.1.smithi148.stdout:7/394: getdents d5/d16/d1c/d4a/d66 0 2024-07-03T15:43:50.587 INFO:tasks.workunit.client.1.smithi148.stdout:1/266: dread d0/f45 [0,4194304] 0 2024-07-03T15:43:50.589 INFO:tasks.workunit.client.1.smithi148.stdout:9/568: dread d0/d49/d43/f22 [0,4194304] 0 2024-07-03T15:43:50.592 INFO:tasks.workunit.client.1.smithi148.stdout:0/313: truncate da/d17/d39/f45 3319683 0 2024-07-03T15:43:50.594 INFO:tasks.workunit.client.1.smithi148.stdout:7/395: symlink d5/d16/d1c/d33/d4c/d21/d47/l79 0 2024-07-03T15:43:50.595 INFO:tasks.workunit.client.1.smithi148.stdout:1/267: rmdir d0/d2/d2e/d39/d42 39 2024-07-03T15:43:50.597 INFO:tasks.workunit.client.1.smithi148.stdout:9/569: mkdir d0/dd/d25/d37/d40/d72/db8 0 2024-07-03T15:43:50.597 INFO:tasks.workunit.client.1.smithi148.stdout:9/570: truncate d0/d18/d24/d47/fa5 4591982 0 2024-07-03T15:43:50.598 INFO:tasks.workunit.client.1.smithi148.stdout:9/571: read d0/dd/d25/d77/f89 [72423,95512] 0 2024-07-03T15:43:50.599 INFO:tasks.workunit.client.1.smithi148.stdout:0/314: link da/d36/d5b/c64 da/d36/c66 0 2024-07-03T15:43:50.600 INFO:tasks.workunit.client.1.smithi148.stdout:2/404: mkdir d5/d14/d22/d2b/d85 0 2024-07-03T15:43:50.601 INFO:tasks.workunit.client.1.smithi148.stdout:7/396: mknod d5/c7a 0 2024-07-03T15:43:50.601 INFO:tasks.workunit.client.1.smithi148.stdout:7/397: read d5/d16/d1c/d23/f49 [1526080,53917] 0 2024-07-03T15:43:50.602 INFO:tasks.workunit.client.1.smithi148.stdout:7/398: dread d5/d16/d1c/f2d [0,4194304] 0 2024-07-03T15:43:50.603 INFO:tasks.workunit.client.1.smithi148.stdout:1/268: rmdir d0/d2/d11/d4d 0 2024-07-03T15:43:50.605 INFO:tasks.workunit.client.1.smithi148.stdout:6/394: dwrite d3/d1d/d28/d31/d4c/f57 [0,4194304] 0 2024-07-03T15:43:50.606 INFO:tasks.workunit.client.1.smithi148.stdout:9/572: unlink d0/d49/c5c 0 2024-07-03T15:43:50.606 INFO:tasks.workunit.client.1.smithi148.stdout:9/573: write d0/dd/d25/d37/d40/f62 [1862568,94560] 0 2024-07-03T15:43:50.608 INFO:tasks.workunit.client.1.smithi148.stdout:3/375: dwrite dd/d33/d52/f60 [0,4194304] 0 2024-07-03T15:43:50.608 INFO:tasks.workunit.client.1.smithi148.stdout:2/405: mkdir d5/d14/d86 0 2024-07-03T15:43:50.610 INFO:tasks.workunit.client.1.smithi148.stdout:1/269: write d0/d2/d6/d14/f15 [6124689,65923] 0 2024-07-03T15:43:50.610 INFO:tasks.workunit.client.1.smithi148.stdout:5/301: dwrite d7/da/df/f4d [0,4194304] 0 2024-07-03T15:43:50.611 INFO:tasks.workunit.client.1.smithi148.stdout:7/399: rename d5/d16/f6d to d5/f7b 0 2024-07-03T15:43:50.611 INFO:tasks.workunit.client.1.smithi148.stdout:7/400: truncate d5/f1f 2282770 0 2024-07-03T15:43:50.613 INFO:tasks.workunit.client.1.smithi148.stdout:9/574: getdents d0/d49/d43/d19/d8c 0 2024-07-03T15:43:50.618 INFO:tasks.workunit.client.1.smithi148.stdout:2/406: mknod d5/d14/d22/d2b/d13/d41/c87 0 2024-07-03T15:43:50.619 INFO:tasks.workunit.client.1.smithi148.stdout:0/315: dwrite da/d44/f26 [0,4194304] 0 2024-07-03T15:43:50.621 INFO:tasks.workunit.client.1.smithi148.stdout:5/302: stat l0 0 2024-07-03T15:43:50.621 INFO:tasks.workunit.client.1.smithi148.stdout:5/303: getdents d7/da/df/d1d 0 2024-07-03T15:43:50.621 INFO:tasks.workunit.client.1.smithi148.stdout:5/304: fdatasync d7/da/df/d16/d2f/f38 0 2024-07-03T15:43:50.622 INFO:tasks.workunit.client.1.smithi148.stdout:9/575: dread d0/dd/f14 [4194304,4194304] 0 2024-07-03T15:43:50.622 INFO:tasks.workunit.client.1.smithi148.stdout:9/576: write d0/dd/d29/f5e [1863646,111121] 0 2024-07-03T15:43:50.622 INFO:tasks.workunit.client.1.smithi148.stdout:6/395: rename d3/ld to d3/d1d/d28/d3e/l87 0 2024-07-03T15:43:50.624 INFO:tasks.workunit.client.1.smithi148.stdout:7/401: link d5/d16/d1c/d23/f2b d5/d16/d1c/d59/f7c 0 2024-07-03T15:43:50.624 INFO:tasks.workunit.client.1.smithi148.stdout:4/265: mknod d0/d8/c59 0 2024-07-03T15:43:50.626 INFO:tasks.workunit.client.1.smithi148.stdout:2/407: unlink d5/d14/d22/d2b/d13/d41/d7b/d6a/c78 0 2024-07-03T15:43:50.627 INFO:tasks.workunit.client.1.smithi148.stdout:7/402: dread d5/d16/d1c/d33/d4c/d21/f3c [0,4194304] 0 2024-07-03T15:43:50.627 INFO:tasks.workunit.client.1.smithi148.stdout:0/316: mknod da/d36/d5b/c67 0 2024-07-03T15:43:50.627 INFO:tasks.workunit.client.1.smithi148.stdout:7/403: write d5/d16/d27/f28 [144132,124330] 0 2024-07-03T15:43:50.628 INFO:tasks.workunit.client.1.smithi148.stdout:7/404: write d5/d16/d1c/d23/f68 [833399,76741] 0 2024-07-03T15:43:50.629 INFO:tasks.workunit.client.1.smithi148.stdout:2/408: dread d5/d7/d64/f71 [0,4194304] 0 2024-07-03T15:43:50.631 INFO:tasks.workunit.client.1.smithi148.stdout:9/577: link d0/d49/la0 d0/dd/d25/d6a/da8/lb9 0 2024-07-03T15:43:50.636 INFO:tasks.workunit.client.1.smithi148.stdout:3/376: rename dd/d33/d37/d68/d2e/d40/d45/d6d to dd/d33/d37/d68/d2e/d40/d77 0 2024-07-03T15:43:50.636 INFO:tasks.workunit.client.1.smithi148.stdout:3/377: read - dd/d35/f56 zero size 2024-07-03T15:43:50.638 INFO:tasks.workunit.client.1.smithi148.stdout:7/405: mkdir d5/d7d 0 2024-07-03T15:43:50.638 INFO:tasks.workunit.client.1.smithi148.stdout:7/406: write d5/d16/d1c/d4a/f62 [1322852,82188] 0 2024-07-03T15:43:50.642 INFO:tasks.workunit.client.1.smithi148.stdout:2/409: creat d5/d5f/d79/f88 x:0 0 0 2024-07-03T15:43:50.642 INFO:tasks.workunit.client.1.smithi148.stdout:2/410: chown d5/d14/d22/d2b/d13/d41/c87 495072696 1 2024-07-03T15:43:50.653 INFO:tasks.workunit.client.1.smithi148.stdout:5/305: dwrite f4 [0,4194304] 0 2024-07-03T15:43:50.657 INFO:tasks.workunit.client.1.smithi148.stdout:1/270: rename d0/c18 to d0/de/d44/c56 0 2024-07-03T15:43:50.657 INFO:tasks.workunit.client.1.smithi148.stdout:2/411: write d5/d14/d22/f62 [3883762,4677] 0 2024-07-03T15:43:50.658 INFO:tasks.workunit.client.1.smithi148.stdout:2/412: stat d5/d67 0 2024-07-03T15:43:50.658 INFO:tasks.workunit.client.1.smithi148.stdout:9/578: dwrite d0/d49/d43/f5b [0,4194304] 0 2024-07-03T15:43:50.658 INFO:tasks.workunit.client.1.smithi148.stdout:7/407: symlink d5/d16/d1c/d23/l7e 0 2024-07-03T15:43:50.658 INFO:tasks.workunit.client.1.smithi148.stdout:7/408: write d5/d16/d1c/d4a/d52/f5f [67980,16993] 0 2024-07-03T15:43:50.659 INFO:tasks.workunit.client.1.smithi148.stdout:4/266: dwrite d0/d8/d22/d2f/d41/f4d [0,4194304] 0 2024-07-03T15:43:50.659 INFO:tasks.workunit.client.1.smithi148.stdout:0/317: dwrite da/d17/f47 [0,4194304] 0 2024-07-03T15:43:50.659 INFO:tasks.workunit.client.1.smithi148.stdout:0/318: creat da/d44/d13/d14/f68 x:0 0 0 2024-07-03T15:43:50.662 INFO:tasks.workunit.client.1.smithi148.stdout:5/306: mknod d7/da/df/c66 0 2024-07-03T15:43:50.662 INFO:tasks.workunit.client.1.smithi148.stdout:5/307: write d7/da/fb [1303006,79336] 0 2024-07-03T15:43:50.663 INFO:tasks.workunit.client.1.smithi148.stdout:0/319: write da/d44/d13/d14/f54 [832987,40905] 0 2024-07-03T15:43:50.668 INFO:tasks.workunit.client.1.smithi148.stdout:2/413: mkdir d5/d14/d22/d60/d89 0 2024-07-03T15:43:50.669 INFO:tasks.workunit.client.1.smithi148.stdout:7/409: write d5/d16/d1c/d33/f42 [846284,96942] 0 2024-07-03T15:43:50.673 INFO:tasks.workunit.client.1.smithi148.stdout:9/579: symlink d0/dd/d11/db4/lba 0 2024-07-03T15:43:50.673 INFO:tasks.workunit.client.1.smithi148.stdout:9/580: write d0/d49/d43/d19/d7a/f9b [647706,6843] 0 2024-07-03T15:43:50.674 INFO:tasks.workunit.client.1.smithi148.stdout:9/581: truncate d0/d2f/f94 113699 0 2024-07-03T15:43:50.680 INFO:tasks.workunit.client.1.smithi148.stdout:2/414: write d5/d14/d22/d60/f53 [1420841,104616] 0 2024-07-03T15:43:50.684 INFO:tasks.workunit.client.1.smithi148.stdout:1/271: dwrite d0/f25 [0,4194304] 0 2024-07-03T15:43:50.684 INFO:tasks.workunit.client.1.smithi148.stdout:4/267: dwrite d0/f5 [0,4194304] 0 2024-07-03T15:43:50.689 INFO:tasks.workunit.client.1.smithi148.stdout:0/320: symlink da/d44/d4a/l69 0 2024-07-03T15:43:50.689 INFO:tasks.workunit.client.1.smithi148.stdout:0/321: creat da/d44/d13/d14/f6a x:0 0 0 2024-07-03T15:43:50.694 INFO:tasks.workunit.client.1.smithi148.stdout:9/582: rename d0/d18/d24/d5a/f5f to d0/fbb 0 2024-07-03T15:43:50.694 INFO:tasks.workunit.client.1.smithi148.stdout:9/583: chown d0/d18/l84 296637650 1 2024-07-03T15:43:50.698 INFO:tasks.workunit.client.1.smithi148.stdout:2/415: symlink d5/d14/d22/d2b/d13/d41/d7b/l8a 0 2024-07-03T15:43:50.698 INFO:tasks.workunit.client.1.smithi148.stdout:1/272: read d0/d2/d2e/d39/d42/f48 [2136817,43263] 0 2024-07-03T15:43:50.698 INFO:tasks.workunit.client.1.smithi148.stdout:1/273: chown d0/d2/d2e/d39 424 1 2024-07-03T15:43:50.698 INFO:tasks.workunit.client.1.smithi148.stdout:2/416: dread - d5/d7/f83 zero size 2024-07-03T15:43:50.698 INFO:tasks.workunit.client.1.smithi148.stdout:7/410: dwrite d5/f3b [0,4194304] 0 2024-07-03T15:43:50.698 INFO:tasks.workunit.client.1.smithi148.stdout:2/417: write d5/d14/f58 [2685270,35184] 0 2024-07-03T15:43:50.699 INFO:tasks.workunit.client.1.smithi148.stdout:2/418: stat d5/d14/d22/d2b/d13/d41/f74 0 2024-07-03T15:43:50.699 INFO:tasks.workunit.client.1.smithi148.stdout:2/419: write d5/d7/d64/f71 [1401664,124988] 0 2024-07-03T15:43:50.705 INFO:tasks.workunit.client.1.smithi148.stdout:1/274: dread d0/d2/d2e/d39/d41/f43 [0,4194304] 0 2024-07-03T15:43:50.705 INFO:tasks.workunit.client.1.smithi148.stdout:1/275: dread - d0/de/d2f/f55 zero size 2024-07-03T15:43:50.705 INFO:tasks.workunit.client.1.smithi148.stdout:1/276: write d0/d2/f10 [672353,77491] 0 2024-07-03T15:43:50.706 INFO:tasks.workunit.client.1.smithi148.stdout:9/584: symlink d0/dd/d11/lbc 0 2024-07-03T15:43:50.708 INFO:tasks.workunit.client.1.smithi148.stdout:4/268: dwrite d0/f4 [0,4194304] 0 2024-07-03T15:43:50.711 INFO:tasks.workunit.client.1.smithi148.stdout:7/411: mkdir d5/d16/d1c/d33/d4c/d21/d31/d7f 0 2024-07-03T15:43:50.715 INFO:tasks.workunit.client.1.smithi148.stdout:2/420: rename d5/d14/d22/d2b/d85 to d5/d14/d8b 0 2024-07-03T15:43:50.715 INFO:tasks.workunit.client.1.smithi148.stdout:2/421: stat d5/d14/d22/d2b/c6f 0 2024-07-03T15:43:50.715 INFO:tasks.workunit.client.1.smithi148.stdout:2/422: chown d5/d14/d22/d2b/d13/d41/d7b/l33 378143445 1 2024-07-03T15:43:50.719 INFO:tasks.workunit.client.1.smithi148.stdout:9/585: mknod d0/d18/d24/d47/cbd 0 2024-07-03T15:43:50.719 INFO:tasks.workunit.client.1.smithi148.stdout:0/322: dwrite da/d44/f26 [0,4194304] 0 2024-07-03T15:43:50.719 INFO:tasks.workunit.client.1.smithi148.stdout:9/586: dread - d0/d18/d24/d47/d4d/fa7 zero size 2024-07-03T15:43:50.728 INFO:tasks.workunit.client.1.smithi148.stdout:0/323: write da/d44/f26 [258850,20392] 0 2024-07-03T15:43:50.728 INFO:tasks.workunit.client.1.smithi148.stdout:0/324: fsync da/d44/d13/d14/f49 0 2024-07-03T15:43:50.743 INFO:tasks.workunit.client.1.smithi148.stdout:7/412: dwrite d5/d16/d1c/d59/f76 [0,4194304] 0 2024-07-03T15:43:50.744 INFO:tasks.workunit.client.1.smithi148.stdout:2/423: dwrite d5/d14/d22/d2b/d13/d1d/f7f [0,4194304] 0 2024-07-03T15:43:50.744 INFO:tasks.workunit.client.1.smithi148.stdout:2/424: write d5/d14/d22/d60/f6c [693267,43835] 0 2024-07-03T15:43:50.745 INFO:tasks.workunit.client.1.smithi148.stdout:2/425: creat d5/d14/d22/d2b/d13/d41/f8c x:0 0 0 2024-07-03T15:43:50.745 INFO:tasks.workunit.client.1.smithi148.stdout:1/277: dwrite d0/d2/d11/d1e/f3c [0,4194304] 0 2024-07-03T15:43:50.745 INFO:tasks.workunit.client.1.smithi148.stdout:9/587: dwrite d0/f53 [0,4194304] 0 2024-07-03T15:43:50.747 INFO:tasks.workunit.client.1.smithi148.stdout:7/413: dread d5/f7b [0,4194304] 0 2024-07-03T15:43:50.747 INFO:tasks.workunit.client.1.smithi148.stdout:4/269: symlink d0/l5a 0 2024-07-03T15:43:50.748 INFO:tasks.workunit.client.1.smithi148.stdout:2/426: dread d5/f9 [4194304,4194304] 0 2024-07-03T15:43:50.754 INFO:tasks.workunit.client.1.smithi148.stdout:0/325: getdents da/d44/d13 0 2024-07-03T15:43:50.754 INFO:tasks.workunit.client.1.smithi148.stdout:0/326: chown da/d44/d13/d14/f6a 0 1 2024-07-03T15:43:50.762 INFO:tasks.workunit.client.1.smithi148.stdout:1/278: dwrite d0/d2/fd [0,4194304] 0 2024-07-03T15:43:50.762 INFO:tasks.workunit.client.1.smithi148.stdout:1/279: creat d0/de/d2f/d40/f57 x:0 0 0 2024-07-03T15:43:50.762 INFO:tasks.workunit.client.1.smithi148.stdout:1/280: write d0/d2/d2e/d39/d41/f43 [3195601,28513] 0 2024-07-03T15:43:50.762 INFO:tasks.workunit.client.1.smithi148.stdout:1/281: chown d0/d2/d6/d14/f15 124 1 2024-07-03T15:43:50.762 INFO:tasks.workunit.client.1.smithi148.stdout:1/282: fdatasync d0/f25 0 2024-07-03T15:43:50.762 INFO:tasks.workunit.client.1.smithi148.stdout:1/283: fdatasync d0/d2/d6/f26 0 2024-07-03T15:43:50.767 INFO:tasks.workunit.client.1.smithi148.stdout:9/588: link d0/d49/d43/d19/d7a/la2 d0/dd/d25/d37/d40/d72/db8/lbe 0 2024-07-03T15:43:50.767 INFO:tasks.workunit.client.1.smithi148.stdout:9/589: fsync d0/d49/f81 0 2024-07-03T15:43:50.767 INFO:tasks.workunit.client.1.smithi148.stdout:9/590: truncate d0/f6 719612 0 2024-07-03T15:43:50.767 INFO:tasks.workunit.client.1.smithi148.stdout:9/591: write d0/d2f/fb5 [1212730,110733] 0 2024-07-03T15:43:50.769 INFO:tasks.workunit.client.1.smithi148.stdout:4/270: mknod d0/d8/d13/d48/c5b 0 2024-07-03T15:43:50.771 INFO:tasks.workunit.client.1.smithi148.stdout:2/427: symlink d5/d5f/d7e/l8d 0 2024-07-03T15:43:50.771 INFO:tasks.workunit.client.1.smithi148.stdout:2/428: creat d5/d7/f8e x:0 0 0 2024-07-03T15:43:50.771 INFO:tasks.workunit.client.1.smithi148.stdout:2/429: write d5/d14/d22/d2b/f1e [1819031,5151] 0 2024-07-03T15:43:50.771 INFO:tasks.workunit.client.1.smithi148.stdout:2/430: write d5/d14/d22/d2b/d13/d41/f4b [9355,105924] 0 2024-07-03T15:43:50.772 INFO:tasks.workunit.client.1.smithi148.stdout:0/327: creat da/d38/f6b x:0 0 0 2024-07-03T15:43:50.774 INFO:tasks.workunit.client.1.smithi148.stdout:4/271: dread d0/d8/f2d [0,4194304] 0 2024-07-03T15:43:50.774 INFO:tasks.workunit.client.1.smithi148.stdout:4/272: chown d0/d8/d22/d2f 578299 1 2024-07-03T15:43:50.774 INFO:tasks.workunit.client.1.smithi148.stdout:4/273: readlink d0/d8/d3a/d36/l37 0 2024-07-03T15:43:50.775 INFO:tasks.workunit.client.1.smithi148.stdout:1/284: creat d0/d2/d6/d14/f58 x:0 0 0 2024-07-03T15:43:50.776 INFO:tasks.workunit.client.1.smithi148.stdout:1/285: readlink d0/d2/d6/d14/d29/l2b 0 2024-07-03T15:43:50.780 INFO:tasks.workunit.client.1.smithi148.stdout:9/592: mknod d0/dd/cbf 0 2024-07-03T15:43:50.780 INFO:tasks.workunit.client.1.smithi148.stdout:9/593: readlink d0/d49/d43/d19/d7a/la4 0 2024-07-03T15:43:50.784 INFO:tasks.workunit.client.1.smithi148.stdout:2/431: mkdir d5/d14/d8f 0 2024-07-03T15:43:50.785 INFO:tasks.workunit.client.1.smithi148.stdout:0/328: rmdir da/d44 39 2024-07-03T15:43:50.785 INFO:tasks.workunit.client.1.smithi148.stdout:0/329: stat da/d44/d13/d14/f61 0 2024-07-03T15:43:50.785 INFO:tasks.workunit.client.1.smithi148.stdout:0/330: readlink da/d44/d13/l56 0 2024-07-03T15:43:50.786 INFO:tasks.workunit.client.1.smithi148.stdout:8/299: getdents d4/da/d14 0 2024-07-03T15:43:50.787 INFO:tasks.workunit.client.1.smithi148.stdout:4/274: mkdir d0/d8/d19/d45/d49/d5c 0 2024-07-03T15:43:50.789 INFO:tasks.workunit.client.1.smithi148.stdout:9/594: creat d0/dd/d11/db4/fc0 x:0 0 0 2024-07-03T15:43:50.789 INFO:tasks.workunit.client.1.smithi148.stdout:9/595: stat d0/dd/d11/db4/lba 0 2024-07-03T15:43:50.800 INFO:tasks.workunit.client.1.smithi148.stdout:2/432: link d5/d14/d22/d2b/d13/d2d/f51 d5/d14/d22/d60/f90 0 2024-07-03T15:43:50.802 INFO:tasks.workunit.client.1.smithi148.stdout:8/300: stat d4/da/d51/c45 0 2024-07-03T15:43:50.802 INFO:tasks.workunit.client.1.smithi148.stdout:8/301: fdatasync d4/da/d51/d18/d1f/f39 0 2024-07-03T15:43:50.803 INFO:tasks.workunit.client.1.smithi148.stdout:8/302: truncate d4/da/d14/f68 591093 0 2024-07-03T15:43:50.803 INFO:tasks.workunit.client.1.smithi148.stdout:8/303: truncate d4/d54/f5b 461037 0 2024-07-03T15:43:50.803 INFO:tasks.workunit.client.1.smithi148.stdout:8/304: chown d4/da/d14/d17/f2a 0 1 2024-07-03T15:43:50.803 INFO:tasks.workunit.client.1.smithi148.stdout:4/275: dwrite d0/d8/d13/f1a [0,4194304] 0 2024-07-03T15:43:50.803 INFO:tasks.workunit.client.1.smithi148.stdout:4/276: stat d0/f30 0 2024-07-03T15:43:50.805 INFO:tasks.workunit.client.1.smithi148.stdout:9/596: rename d0/d18/d24/d47/l6c to d0/dd/lc1 0 2024-07-03T15:43:50.805 INFO:tasks.workunit.client.1.smithi148.stdout:9/597: stat d0/dd/d25/d77/f96 0 2024-07-03T15:43:50.805 INFO:tasks.workunit.client.1.smithi148.stdout:9/598: truncate d0/d18/d24/d47/d4d/fa7 197294 0 2024-07-03T15:43:50.805 INFO:tasks.workunit.client.1.smithi148.stdout:9/599: write d0/d18/d24/f30 [3353417,77410] 0 2024-07-03T15:43:50.807 INFO:tasks.workunit.client.1.smithi148.stdout:8/305: creat d4/da/d51/f69 x:0 0 0 2024-07-03T15:43:50.811 INFO:tasks.workunit.client.1.smithi148.stdout:4/277: mknod d0/d8/d3a/c5d 0 2024-07-03T15:43:50.811 INFO:tasks.workunit.client.1.smithi148.stdout:4/278: write d0/d8/f10 [3748699,95393] 0 2024-07-03T15:43:50.817 INFO:tasks.workunit.client.1.smithi148.stdout:9/600: dread d0/dd/d25/d37/f50 [0,4194304] 0 2024-07-03T15:43:50.818 INFO:tasks.workunit.client.1.smithi148.stdout:0/331: getdents da/d17/d2a/d2f 0 2024-07-03T15:43:50.818 INFO:tasks.workunit.client.1.smithi148.stdout:0/332: dread - da/d38/f4e zero size 2024-07-03T15:43:50.819 INFO:tasks.workunit.client.1.smithi148.stdout:0/333: chown da/d44/d13/d14/f6a 0 1 2024-07-03T15:43:50.819 INFO:tasks.workunit.client.1.smithi148.stdout:9/601: truncate d0/d49/d43/d19/f1e 27904 0 2024-07-03T15:43:50.820 INFO:tasks.workunit.client.1.smithi148.stdout:0/334: link da/f11 da/d36/d5b/f6c 0 2024-07-03T15:43:50.822 INFO:tasks.workunit.client.1.smithi148.stdout:9/602: rename d0/d18/laf to d0/d18/d24/lc2 0 2024-07-03T15:43:50.823 INFO:tasks.workunit.client.1.smithi148.stdout:2/433: dwrite d5/d14/d22/f72 [0,4194304] 0 2024-07-03T15:43:50.823 INFO:tasks.workunit.client.1.smithi148.stdout:8/306: dwrite d4/da/f29 [0,4194304] 0 2024-07-03T15:43:50.825 INFO:tasks.workunit.client.1.smithi148.stdout:0/335: creat da/d44/f6d x:0 0 0 2024-07-03T15:43:50.829 INFO:tasks.workunit.client.1.smithi148.stdout:9/603: mknod d0/d49/d43/d19/d7a/daa/cc3 0 2024-07-03T15:43:50.834 INFO:tasks.workunit.client.1.smithi148.stdout:4/279: dread d0/d8/f10 [0,4194304] 0 2024-07-03T15:43:50.835 INFO:tasks.workunit.client.1.smithi148.stdout:8/307: symlink d4/da/d14/d1a/d32/l6a 0 2024-07-03T15:43:50.839 INFO:tasks.workunit.client.1.smithi148.stdout:9/604: rmdir d0/d49/d43/db0 0 2024-07-03T15:43:50.841 INFO:tasks.workunit.client.1.smithi148.stdout:4/280: rmdir d0/d8/d19/d45/d49/d5c 0 2024-07-03T15:43:50.845 INFO:tasks.workunit.client.1.smithi148.stdout:9/605: unlink d0/dd/l1c 0 2024-07-03T15:43:50.846 INFO:tasks.workunit.client.1.smithi148.stdout:9/606: stat d0/dd/d29 0 2024-07-03T15:43:50.846 INFO:tasks.workunit.client.1.smithi148.stdout:7/414: getdents d5/d16/d1c/d33/d4c/d21/d47 0 2024-07-03T15:43:50.846 INFO:tasks.workunit.client.1.smithi148.stdout:7/415: write d5/d16/d1c/d23/f49 [4182971,118454] 0 2024-07-03T15:43:50.846 INFO:tasks.workunit.client.1.smithi148.stdout:2/434: dwrite d5/d14/d22/d2b/d13/d41/d7b/f3e [0,4194304] 0 2024-07-03T15:43:50.846 INFO:tasks.workunit.client.1.smithi148.stdout:2/435: creat d5/d14/d22/d60/f91 x:0 0 0 2024-07-03T15:43:50.846 INFO:tasks.workunit.client.1.smithi148.stdout:4/281: symlink d0/d28/l5e 0 2024-07-03T15:43:50.849 INFO:tasks.workunit.client.1.smithi148.stdout:8/308: rename d4/da/d51/l23 to d4/da/d47/l6b 0 2024-07-03T15:43:50.850 INFO:tasks.workunit.client.1.smithi148.stdout:9/607: truncate d0/dd/d25/d37/f50 1212674 0 2024-07-03T15:43:50.853 INFO:tasks.workunit.client.1.smithi148.stdout:9/608: read d0/dd/d25/d37/d40/d72/f8d [2771935,117553] 0 2024-07-03T15:43:50.855 INFO:tasks.workunit.client.1.smithi148.stdout:9/609: dread d0/d2f/f41 [0,4194304] 0 2024-07-03T15:43:50.857 INFO:tasks.workunit.client.1.smithi148.stdout:0/336: dwrite da/d17/d2a/f59 [0,4194304] 0 2024-07-03T15:43:50.869 INFO:tasks.workunit.client.1.smithi148.stdout:4/282: mknod d0/d8/d13/c5f 0 2024-07-03T15:43:50.871 INFO:tasks.workunit.client.1.smithi148.stdout:2/436: rename d5/d14/d22/d2b/d13/d41/d7b/f48 to d5/d14/d22/d60/d76/f92 0 2024-07-03T15:43:50.871 INFO:tasks.workunit.client.1.smithi148.stdout:2/437: chown d5/d14/d22/d2b/d13/d2d/c61 600 1 2024-07-03T15:43:50.871 INFO:tasks.workunit.client.1.smithi148.stdout:2/438: read - d5/d14/d22/d2b/f6d zero size 2024-07-03T15:43:50.872 INFO:tasks.workunit.client.1.smithi148.stdout:6/396: rmdir d3 39 2024-07-03T15:43:50.873 INFO:tasks.workunit.client.1.smithi148.stdout:8/309: creat d4/da/d14/f6c x:0 0 0 2024-07-03T15:43:50.874 INFO:tasks.workunit.client.1.smithi148.stdout:7/416: rmdir d5/d16/d1c/d33/d4c/d21/d39 39 2024-07-03T15:43:50.878 INFO:tasks.workunit.client.1.smithi148.stdout:2/439: write d5/d14/d22/d60/f90 [68614,92508] 0 2024-07-03T15:43:50.878 INFO:tasks.workunit.client.1.smithi148.stdout:2/440: chown d5/d14/d22/d2b/d13/d41/l56 4 1 2024-07-03T15:43:50.880 INFO:tasks.workunit.client.1.smithi148.stdout:4/283: symlink d0/d8/d19/d45/l60 0 2024-07-03T15:43:50.881 INFO:tasks.workunit.client.1.smithi148.stdout:4/284: write d0/d8/d19/d45/d49/f56 [287818,100398] 0 2024-07-03T15:43:50.881 INFO:tasks.workunit.client.1.smithi148.stdout:4/285: write d0/d8/d22/d2f/f33 [1039239,17767] 0 2024-07-03T15:43:50.883 INFO:tasks.workunit.client.1.smithi148.stdout:0/337: rename da/d44/d13/d14/d50/f53 to da/d44/d13/d1d/d3e/f6e 0 2024-07-03T15:43:50.885 INFO:tasks.workunit.client.1.smithi148.stdout:6/397: mkdir d3/d88 0 2024-07-03T15:43:50.885 INFO:tasks.workunit.client.1.smithi148.stdout:6/398: truncate d3/d1d/d28/d3e/f58 4481130 0 2024-07-03T15:43:50.887 INFO:tasks.workunit.client.1.smithi148.stdout:8/310: getdents d4/da/d14/d1a/d32/d38/d58 0 2024-07-03T15:43:50.889 INFO:tasks.workunit.client.1.smithi148.stdout:7/417: symlink d5/d16/d1c/d33/l80 0 2024-07-03T15:43:50.895 INFO:tasks.workunit.client.1.smithi148.stdout:0/338: mknod da/d36/c6f 0 2024-07-03T15:43:50.895 INFO:tasks.workunit.client.1.smithi148.stdout:0/339: readlink da/d44/d13/l56 0 2024-07-03T15:43:50.900 INFO:tasks.workunit.client.1.smithi148.stdout:2/441: dwrite d5/d14/d22/d2b/d13/d41/f4b [0,4194304] 0 2024-07-03T15:43:50.900 INFO:tasks.workunit.client.1.smithi148.stdout:6/399: creat d3/d1d/d28/d31/d83/f89 x:0 0 0 2024-07-03T15:43:50.904 INFO:tasks.workunit.client.1.smithi148.stdout:2/442: mkdir d5/d14/d86/d93 0 2024-07-03T15:43:50.905 INFO:tasks.workunit.client.1.smithi148.stdout:5/308: truncate f4 5389149 0 2024-07-03T15:43:50.905 INFO:tasks.workunit.client.1.smithi148.stdout:5/309: write d7/da/d21/f5e [729684,39465] 0 2024-07-03T15:43:50.905 INFO:tasks.workunit.client.1.smithi148.stdout:5/310: write d7/da/df/d16/d1a/f50 [3071881,121759] 0 2024-07-03T15:43:50.906 INFO:tasks.workunit.client.1.smithi148.stdout:8/311: dwrite d4/da/d14/d17/f2a [0,4194304] 0 2024-07-03T15:43:50.907 INFO:tasks.workunit.client.1.smithi148.stdout:6/400: mknod d3/d1d/d28/d31/c8a 0 2024-07-03T15:43:50.907 INFO:tasks.workunit.client.1.smithi148.stdout:6/401: readlink d3/d1d/l3a 0 2024-07-03T15:43:50.909 INFO:tasks.workunit.client.1.smithi148.stdout:0/340: read da/f11 [6340645,2185] 0 2024-07-03T15:43:50.909 INFO:tasks.workunit.client.1.smithi148.stdout:4/286: dwrite d0/d8/f2d [0,4194304] 0 2024-07-03T15:43:50.910 INFO:tasks.workunit.client.1.smithi148.stdout:2/443: symlink d5/d14/d22/d2b/d13/d1d/l94 0 2024-07-03T15:43:50.910 INFO:tasks.workunit.client.1.smithi148.stdout:2/444: chown d5/d5f 39307 1 2024-07-03T15:43:50.913 INFO:tasks.workunit.client.1.smithi148.stdout:5/311: unlink d7/d13/d23/l46 0 2024-07-03T15:43:50.915 INFO:tasks.workunit.client.1.smithi148.stdout:2/445: read d5/d14/d22/d2b/f1e [646901,92836] 0 2024-07-03T15:43:50.915 INFO:tasks.workunit.client.1.smithi148.stdout:2/446: fsync d5/f9 0 2024-07-03T15:43:50.922 INFO:tasks.workunit.client.1.smithi148.stdout:6/402: creat d3/d1d/d28/d3e/f8b x:0 0 0 2024-07-03T15:43:50.922 INFO:tasks.workunit.client.1.smithi148.stdout:6/403: chown d3/d1d/l3a 0 1 2024-07-03T15:43:50.924 INFO:tasks.workunit.client.1.smithi148.stdout:4/287: dwrite d0/f50 [0,4194304] 0 2024-07-03T15:43:50.925 INFO:tasks.workunit.client.1.smithi148.stdout:0/341: symlink da/d44/d13/d1d/d3e/l70 0 2024-07-03T15:43:50.925 INFO:tasks.workunit.client.1.smithi148.stdout:4/288: write d0/d8/d19/d45/d49/f56 [3546,120869] 0 2024-07-03T15:43:50.927 INFO:tasks.workunit.client.1.smithi148.stdout:5/312: symlink d7/da/d21/l67 0 2024-07-03T15:43:50.931 INFO:tasks.workunit.client.1.smithi148.stdout:2/447: unlink d5/d14/d22/d2b/d13/d2d/c61 0 2024-07-03T15:43:50.931 INFO:tasks.workunit.client.1.smithi148.stdout:2/448: write d5/f9 [2397125,79527] 0 2024-07-03T15:43:50.932 INFO:tasks.workunit.client.1.smithi148.stdout:2/449: dread d5/d14/d22/d2b/d13/f29 [0,4194304] 0 2024-07-03T15:43:50.934 INFO:tasks.workunit.client.1.smithi148.stdout:8/312: link d4/da/d51/d18/l24 d4/da/d14/d3a/l6d 0 2024-07-03T15:43:50.935 INFO:tasks.workunit.client.1.smithi148.stdout:8/313: write d4/da/d14/f16 [1660277,16656] 0 2024-07-03T15:43:50.935 INFO:tasks.workunit.client.1.smithi148.stdout:8/314: stat d4/da/d14/d1a/f37 0 2024-07-03T15:43:50.936 INFO:tasks.workunit.client.1.smithi148.stdout:6/404: mkdir d3/d1d/d28/d7f/d85/d8c 0 2024-07-03T15:43:50.936 INFO:tasks.workunit.client.1.smithi148.stdout:6/405: fdatasync d3/f55 0 2024-07-03T15:43:50.939 INFO:tasks.workunit.client.1.smithi148.stdout:2/450: write d5/d14/d22/f72 [341010,88371] 0 2024-07-03T15:43:50.940 INFO:tasks.workunit.client.1.smithi148.stdout:2/451: dread - d5/d14/d22/d60/f91 zero size 2024-07-03T15:43:50.945 INFO:tasks.workunit.client.1.smithi148.stdout:0/342: rename da/d44/f6d to da/d44/d13/f71 0 2024-07-03T15:43:50.949 INFO:tasks.workunit.client.1.smithi148.stdout:5/313: symlink d7/da/d21/l68 0 2024-07-03T15:43:50.951 INFO:tasks.workunit.client.1.smithi148.stdout:8/315: creat d4/da/d14/d1a/f6e x:0 0 0 2024-07-03T15:43:50.951 INFO:tasks.workunit.client.1.smithi148.stdout:8/316: chown d4/da/d14/d1a/d32/d38/d4a/c44 649822 1 2024-07-03T15:43:50.952 INFO:tasks.workunit.client.1.smithi148.stdout:8/317: dread d4/d54/f5b [0,4194304] 0 2024-07-03T15:43:50.952 INFO:tasks.workunit.client.1.smithi148.stdout:6/406: mknod d3/d1d/d28/d7f/c8d 0 2024-07-03T15:43:50.952 INFO:tasks.workunit.client.1.smithi148.stdout:6/407: truncate d3/d1d/d28/d31/d61/f71 1041732 0 2024-07-03T15:43:50.952 INFO:tasks.workunit.client.1.smithi148.stdout:6/408: stat d3/d1d/d28/d53/d6d 0 2024-07-03T15:43:50.954 INFO:tasks.workunit.client.1.smithi148.stdout:2/452: truncate d5/d14/d22/d2b/d13/d2d/f3b 3228380 0 2024-07-03T15:43:50.954 INFO:tasks.workunit.client.1.smithi148.stdout:2/453: readlink d5/d14/d22/d2b/d13/d41/l70 0 2024-07-03T15:43:50.957 INFO:tasks.workunit.client.1.smithi148.stdout:0/343: creat da/d36/d5b/f72 x:0 0 0 2024-07-03T15:43:50.957 INFO:tasks.workunit.client.1.smithi148.stdout:5/314: dread d7/f19 [0,4194304] 0 2024-07-03T15:43:50.961 INFO:tasks.workunit.client.1.smithi148.stdout:8/318: dread d4/da/d51/d18/d1f/f42 [0,4194304] 0 2024-07-03T15:43:50.961 INFO:tasks.workunit.client.1.smithi148.stdout:6/409: unlink d3/c6 0 2024-07-03T15:43:50.964 INFO:tasks.workunit.client.1.smithi148.stdout:2/454: link d5/d14/d22/d2b/d13/d41/c44 d5/d67/c95 0 2024-07-03T15:43:50.964 INFO:tasks.workunit.client.1.smithi148.stdout:2/455: readlink d5/d14/d22/d2b/d13/d1d/l94 0 2024-07-03T15:43:50.964 INFO:tasks.workunit.client.1.smithi148.stdout:2/456: fdatasync d5/d7/f3d 0 2024-07-03T15:43:50.966 INFO:tasks.workunit.client.1.smithi148.stdout:0/344: rename da/d44/d24/f52 to da/f73 0 2024-07-03T15:43:50.969 INFO:tasks.workunit.client.1.smithi148.stdout:5/315: unlink d7/d59/l62 0 2024-07-03T15:43:50.969 INFO:tasks.workunit.client.1.smithi148.stdout:5/316: readlink d7/d13/l48 0 2024-07-03T15:43:50.970 INFO:tasks.workunit.client.1.smithi148.stdout:0/345: dread da/d44/f19 [0,4194304] 0 2024-07-03T15:43:50.970 INFO:tasks.workunit.client.1.smithi148.stdout:0/346: chown da/d36/d5b/c67 31 1 2024-07-03T15:43:50.971 INFO:tasks.workunit.client.1.smithi148.stdout:5/317: dread d7/da/df/d1d/f2e [0,4194304] 0 2024-07-03T15:43:50.976 INFO:tasks.workunit.client.1.smithi148.stdout:6/410: mkdir d3/d50/d77/d8e 0 2024-07-03T15:43:50.980 INFO:tasks.workunit.client.1.smithi148.stdout:8/319: rename d4/da/fb to d4/da/d14/f6f 0 2024-07-03T15:43:50.982 INFO:tasks.workunit.client.1.smithi148.stdout:0/347: symlink da/d36/d5b/l74 0 2024-07-03T15:43:50.985 INFO:tasks.workunit.client.1.smithi148.stdout:5/318: link d7/d13/d54/f4b d7/da/df/d16/d2f/d44/f69 0 2024-07-03T15:43:50.985 INFO:tasks.workunit.client.1.smithi148.stdout:5/319: chown d7/f4f 258 1 2024-07-03T15:43:50.987 INFO:tasks.workunit.client.1.smithi148.stdout:6/411: mkdir d3/d1d/d28/d53/d6d/d8f 0 2024-07-03T15:43:50.989 INFO:tasks.workunit.client.1.smithi148.stdout:0/348: rename da/d44/f19 to da/d44/f75 0 2024-07-03T15:43:50.989 INFO:tasks.workunit.client.1.smithi148.stdout:0/349: chown da/f11 1 1 2024-07-03T15:43:50.991 INFO:tasks.workunit.client.1.smithi148.stdout:6/412: mknod d3/d1d/d28/d3e/c90 0 2024-07-03T15:43:50.991 INFO:tasks.workunit.client.1.smithi148.stdout:6/413: dread - d3/d1d/d28/f5f zero size 2024-07-03T15:43:50.992 INFO:tasks.workunit.client.1.smithi148.stdout:5/320: write d7/da/f2b [2156938,17368] 0 2024-07-03T15:43:50.993 INFO:tasks.workunit.client.1.smithi148.stdout:8/320: symlink d4/da/d51/l70 0 2024-07-03T15:43:50.999 INFO:tasks.workunit.client.1.smithi148.stdout:8/321: rename d4/da/d14/d1a/d32/d38/d4a/d61 to d4/d71 0 2024-07-03T15:43:50.999 INFO:tasks.workunit.client.1.smithi148.stdout:8/322: write d4/d54/f5b [192431,14430] 0 2024-07-03T15:43:51.004 INFO:tasks.workunit.client.1.smithi148.stdout:8/323: rename d4/lf to d4/da/d51/d18/d1f/l72 0 2024-07-03T15:43:51.004 INFO:tasks.workunit.client.1.smithi148.stdout:8/324: fdatasync d4/da/d14/d1a/d32/d38/d4a/f27 0 2024-07-03T15:43:51.009 INFO:tasks.workunit.client.1.smithi148.stdout:5/321: dwrite d7/da/df/d16/d2f/d44/d52/f55 [0,4194304] 0 2024-07-03T15:43:51.010 INFO:tasks.workunit.client.1.smithi148.stdout:6/414: dwrite d3/d1d/d28/d31/d61/f7a [0,4194304] 0 2024-07-03T15:43:51.017 INFO:tasks.workunit.client.1.smithi148.stdout:5/322: mkdir d7/da/df/d16/d6a 0 2024-07-03T15:43:51.017 INFO:tasks.workunit.client.1.smithi148.stdout:5/323: fdatasync d7/da/df/d16/d1a/f64 0 2024-07-03T15:43:51.024 INFO:tasks.workunit.client.1.smithi148.stdout:8/325: dwrite d4/da/d14/f4d [0,4194304] 0 2024-07-03T15:43:51.024 INFO:tasks.workunit.client.1.smithi148.stdout:8/326: creat d4/da/d14/d1a/d32/f73 x:0 0 0 2024-07-03T15:43:51.026 INFO:tasks.workunit.client.1.smithi148.stdout:6/415: rename d3/d1d/d28/f2a to d3/d1d/f91 0 2024-07-03T15:43:51.026 INFO:tasks.workunit.client.1.smithi148.stdout:6/416: fdatasync d3/d1d/d28/f69 0 2024-07-03T15:43:51.026 INFO:tasks.workunit.client.1.smithi148.stdout:6/417: chown d3/d1d/d28/d3e 1313480 1 2024-07-03T15:43:51.026 INFO:tasks.workunit.client.1.smithi148.stdout:6/418: fdatasync d3/d1d/f4e 0 2024-07-03T15:43:51.026 INFO:tasks.workunit.client.1.smithi148.stdout:6/419: truncate d3/d1d/f27 4862066 0 2024-07-03T15:43:51.032 INFO:tasks.workunit.client.1.smithi148.stdout:5/324: dwrite d7/da/df/d16/d2f/d44/d52/f5b [0,4194304] 0 2024-07-03T15:43:51.034 INFO:tasks.workunit.client.1.smithi148.stdout:6/420: mknod d3/d1d/d28/d53/c92 0 2024-07-03T15:43:51.040 INFO:tasks.workunit.client.1.smithi148.stdout:6/421: link d3/d1d/d28/d31/c47 d3/d1d/d28/d7f/d85/c93 0 2024-07-03T15:43:51.043 INFO:tasks.workunit.client.1.smithi148.stdout:6/422: rmdir d3/d1d/d28/d31/d4c 39 2024-07-03T15:43:51.045 INFO:tasks.workunit.client.1.smithi148.stdout:6/423: truncate d3/d1d/f21 1560665 0 2024-07-03T15:43:51.048 INFO:tasks.workunit.client.1.smithi148.stdout:5/325: dwrite d7/da/df/d16/d2f/f38 [0,4194304] 0 2024-07-03T15:43:51.048 INFO:tasks.workunit.client.1.smithi148.stdout:6/424: creat d3/d50/d77/f94 x:0 0 0 2024-07-03T15:43:51.052 INFO:tasks.workunit.client.1.smithi148.stdout:5/326: getdents d7/da 0 2024-07-03T15:43:51.060 INFO:tasks.workunit.client.1.smithi148.stdout:5/327: dread d7/da/df/d16/d1a/f50 [0,4194304] 0 2024-07-03T15:43:51.063 INFO:tasks.workunit.client.1.smithi148.stdout:5/328: mknod d7/d13/d23/d40/c6b 0 2024-07-03T15:43:51.063 INFO:tasks.workunit.client.1.smithi148.stdout:5/329: write d7/f4f [3674480,25859] 0 2024-07-03T15:43:51.063 INFO:tasks.workunit.client.1.smithi148.stdout:6/425: dwrite d3/d1d/d28/d31/d4c/f78 [4194304,4194304] 0 2024-07-03T15:43:51.063 INFO:tasks.workunit.client.1.smithi148.stdout:6/426: read d3/d1d/d28/d31/d61/f71 [988926,106794] 0 2024-07-03T15:43:51.069 INFO:tasks.workunit.client.1.smithi148.stdout:6/427: mkdir d3/d95 0 2024-07-03T15:43:51.069 INFO:tasks.workunit.client.1.smithi148.stdout:6/428: fdatasync d3/d1d/d28/d31/d61/f7a 0 2024-07-03T15:43:51.073 INFO:tasks.workunit.client.1.smithi148.stdout:5/330: write d7/da/df/f33 [3431072,24024] 0 2024-07-03T15:43:51.073 INFO:tasks.workunit.client.1.smithi148.stdout:5/331: creat d7/da/df/d16/d2f/d44/d52/f6c x:0 0 0 2024-07-03T15:43:51.089 INFO:tasks.workunit.client.1.smithi148.stdout:4/289: rmdir d0/d28 39 2024-07-03T15:43:51.089 INFO:tasks.workunit.client.1.smithi148.stdout:4/290: stat d0/d28/c40 0 2024-07-03T15:43:51.092 INFO:tasks.workunit.client.1.smithi148.stdout:6/429: dwrite d3/d1d/d28/d31/d4c/f57 [0,4194304] 0 2024-07-03T15:43:51.092 INFO:tasks.workunit.client.1.smithi148.stdout:6/430: fsync d3/f1e 0 2024-07-03T15:43:51.098 INFO:tasks.workunit.client.1.smithi148.stdout:4/291: read d0/d8/f2d [3661313,44135] 0 2024-07-03T15:43:51.099 INFO:tasks.workunit.client.1.smithi148.stdout:5/332: dwrite d7/da/df/f4d [0,4194304] 0 2024-07-03T15:43:51.101 INFO:tasks.workunit.client.1.smithi148.stdout:4/292: truncate d0/f17 2846203 0 2024-07-03T15:43:51.104 INFO:tasks.workunit.client.1.smithi148.stdout:5/333: mknod d7/c6d 0 2024-07-03T15:43:51.107 INFO:tasks.workunit.client.1.smithi148.stdout:5/334: rename d7/da/d21/c41 to d7/da/df/d16/d1a/c6e 0 2024-07-03T15:43:51.107 INFO:tasks.workunit.client.1.smithi148.stdout:5/335: truncate d7/da/f4c 778957 0 2024-07-03T15:43:51.109 INFO:tasks.workunit.client.1.smithi148.stdout:5/336: rename d7/da/df/d1d/c57 to d7/da/df/d16/d2f/d44/c6f 0 2024-07-03T15:43:51.120 INFO:tasks.workunit.client.1.smithi148.stdout:4/293: dwrite d0/d8/f10 [0,4194304] 0 2024-07-03T15:43:51.120 INFO:tasks.workunit.client.1.smithi148.stdout:5/337: dread d7/da/df/d16/d2f/d44/f69 [0,4194304] 0 2024-07-03T15:43:51.120 INFO:tasks.workunit.client.1.smithi148.stdout:5/338: readlink d7/da/df/d16/d2f/d44/d52/l61 0 2024-07-03T15:43:51.121 INFO:tasks.workunit.client.1.smithi148.stdout:4/294: symlink d0/d8/d22/d2f/d52/l61 0 2024-07-03T15:43:51.121 INFO:tasks.workunit.client.1.smithi148.stdout:4/295: chown d0/d8/d22/d2f 212610 1 2024-07-03T15:43:51.121 INFO:tasks.workunit.client.1.smithi148.stdout:4/296: fdatasync d0/d8/f2d 0 2024-07-03T15:43:51.123 INFO:tasks.workunit.client.1.smithi148.stdout:5/339: creat d7/da/df/d16/d2f/d44/f70 x:0 0 0 2024-07-03T15:43:51.123 INFO:tasks.workunit.client.1.smithi148.stdout:5/340: chown d7/da/d21/l68 77648 1 2024-07-03T15:43:51.123 INFO:tasks.workunit.client.0.smithi028.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-07-03T15:43:51.123 INFO:tasks.workunit.client.0.smithi028.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-07-03T15:43:51.124 INFO:tasks.workunit.client.0.smithi028.stderr:+ make 2024-07-03T15:43:51.124 INFO:tasks.workunit.client.1.smithi148.stdout:4/297: mkdir d0/d8/d22/d62 0 2024-07-03T15:43:51.132 INFO:tasks.workunit.client.1.smithi148.stdout:4/298: read d0/d8/d22/d2f/d41/f4d [530839,80895] 0 2024-07-03T15:43:51.139 INFO:tasks.workunit.client.1.smithi148.stdout:5/341: dwrite d7/d13/d54/f4b [0,4194304] 0 2024-07-03T15:43:51.139 INFO:tasks.workunit.client.1.smithi148.stdout:5/342: creat d7/da/df/f71 x:0 0 0 2024-07-03T15:43:51.142 INFO:tasks.workunit.client.1.smithi148.stdout:5/343: symlink d7/da/df/d16/d2f/d44/l72 0 2024-07-03T15:43:51.144 INFO:tasks.workunit.client.1.smithi148.stdout:5/344: creat d7/da/f73 x:0 0 0 2024-07-03T15:43:51.145 INFO:tasks.workunit.client.1.smithi148.stdout:5/345: creat d7/da/f74 x:0 0 0 2024-07-03T15:43:51.148 INFO:tasks.workunit.client.1.smithi148.stdout:5/346: creat d7/f75 x:0 0 0 2024-07-03T15:43:51.148 INFO:tasks.workunit.client.1.smithi148.stdout:5/347: creat d7/da/df/d16/d2f/d44/d52/f76 x:0 0 0 2024-07-03T15:43:51.156 INFO:tasks.workunit.client.1.smithi148.stdout:5/348: write d7/da/df/d16/d2f/d44/f69 [2289558,21407] 0 2024-07-03T15:43:51.156 INFO:tasks.workunit.client.1.smithi148.stdout:5/349: read d7/da/df/d16/d1a/f50 [410155,122642] 0 2024-07-03T15:43:51.157 INFO:tasks.workunit.client.1.smithi148.stdout:5/350: rename d7/da/df/d16/d1a/f64 to d7/da/f77 0 2024-07-03T15:43:51.157 INFO:tasks.workunit.client.1.smithi148.stdout:5/351: write d7/da/df/f71 [846802,121412] 0 2024-07-03T15:43:51.157 INFO:tasks.workunit.client.1.smithi148.stdout:5/352: readlink d7/da/df/d16/l3c 0 2024-07-03T15:43:51.160 INFO:tasks.workunit.client.1.smithi148.stdout:5/353: dread d7/da/fb [0,4194304] 0 2024-07-03T15:43:51.161 INFO:tasks.workunit.client.1.smithi148.stdout:0/350: fsync da/f73 0 2024-07-03T15:43:51.161 INFO:tasks.workunit.client.1.smithi148.stdout:0/351: fsync da/d17/d39/f45 0 2024-07-03T15:43:51.163 INFO:tasks.workunit.client.1.smithi148.stdout:0/352: link da/d44/d13/d14/d50/f5d da/d17/f76 0 2024-07-03T15:43:51.171 INFO:tasks.workunit.client.1.smithi148.stdout:2/457: dread d5/d14/d22/d2b/d13/d2d/f3b [0,4194304] 0 2024-07-03T15:43:51.172 INFO:tasks.workunit.client.1.smithi148.stdout:2/458: mknod d5/d14/d22/d2b/d68/c96 0 2024-07-03T15:43:51.172 INFO:tasks.workunit.client.1.smithi148.stdout:3/378: sync 2024-07-03T15:43:51.174 INFO:tasks.workunit.client.1.smithi148.stdout:3/379: creat dd/d33/d37/d68/d2e/d40/d45/f78 x:0 0 0 2024-07-03T15:43:51.175 INFO:tasks.workunit.client.1.smithi148.stdout:9/610: sync 2024-07-03T15:43:51.175 INFO:tasks.workunit.client.1.smithi148.stdout:1/286: sync 2024-07-03T15:43:51.176 INFO:tasks.workunit.client.1.smithi148.stdout:9/611: readlink d0/d49/d43/d19/d7a/la4 0 2024-07-03T15:43:51.176 INFO:tasks.workunit.client.1.smithi148.stdout:1/287: creat d0/d2/d11/d1e/f59 x:0 0 0 2024-07-03T15:43:51.177 INFO:tasks.workunit.client.1.smithi148.stdout:7/418: sync 2024-07-03T15:43:51.177 INFO:tasks.workunit.client.1.smithi148.stdout:3/380: rmdir dd/d35/d41/d74 0 2024-07-03T15:43:51.177 INFO:tasks.workunit.client.1.smithi148.stdout:3/381: fdatasync dd/d33/d37/d68/f30 0 2024-07-03T15:43:51.177 INFO:tasks.workunit.client.1.smithi148.stdout:3/382: write dd/d33/d37/d68/f3c [890276,66887] 0 2024-07-03T15:43:51.180 INFO:tasks.workunit.client.1.smithi148.stdout:0/353: dwrite da/d44/d13/d14/d35/f5e [0,4194304] 0 2024-07-03T15:43:51.180 INFO:tasks.workunit.client.1.smithi148.stdout:0/354: readlink da/d44/d13/d14/l62 0 2024-07-03T15:43:51.181 INFO:tasks.workunit.client.1.smithi148.stdout:9/612: unlink d0/d2f/f94 0 2024-07-03T15:43:51.189 INFO:tasks.workunit.client.1.smithi148.stdout:1/288: dread d0/d2/d2e/d39/d42/f49 [0,4194304] 0 2024-07-03T15:43:51.191 INFO:tasks.workunit.client.1.smithi148.stdout:7/419: write d5/d16/d1c/d33/d4c/f26 [3181400,1777] 0 2024-07-03T15:43:51.194 INFO:tasks.workunit.client.1.smithi148.stdout:1/289: creat d0/d2/d6/f5a x:0 0 0 2024-07-03T15:43:51.195 INFO:tasks.workunit.client.1.smithi148.stdout:9/613: read d0/d18/d24/d47/fa5 [301048,16880] 0 2024-07-03T15:43:51.198 INFO:tasks.workunit.client.1.smithi148.stdout:7/420: link d5/d16/d27/c40 d5/d16/d1c/d33/d4c/c81 0 2024-07-03T15:43:51.200 INFO:tasks.workunit.client.1.smithi148.stdout:1/290: creat d0/d2/d11/d3a/f5b x:0 0 0 2024-07-03T15:43:51.200 INFO:tasks.workunit.client.1.smithi148.stdout:1/291: fsync d0/d2/d6/d14/f58 0 2024-07-03T15:43:51.200 INFO:tasks.workunit.client.1.smithi148.stdout:2/459: dwrite d5/d14/d22/d2b/d13/d1d/f7f [0,4194304] 0 2024-07-03T15:43:51.200 INFO:tasks.workunit.client.1.smithi148.stdout:2/460: readlink l2 0 2024-07-03T15:43:51.200 INFO:tasks.workunit.client.1.smithi148.stdout:9/614: creat d0/dd/d46/fc4 x:0 0 0 2024-07-03T15:43:51.201 INFO:tasks.workunit.client.1.smithi148.stdout:2/461: symlink d5/d14/d22/d60/d89/l97 0 2024-07-03T15:43:51.202 INFO:tasks.workunit.client.1.smithi148.stdout:7/421: dread d5/d16/d27/f28 [0,4194304] 0 2024-07-03T15:43:51.202 INFO:tasks.workunit.client.1.smithi148.stdout:9/615: creat d0/fc5 x:0 0 0 2024-07-03T15:43:51.206 INFO:tasks.workunit.client.1.smithi148.stdout:1/292: dread d0/f24 [0,4194304] 0 2024-07-03T15:43:51.206 INFO:tasks.workunit.client.1.smithi148.stdout:0/355: dwrite da/d38/f6b [0,4194304] 0 2024-07-03T15:43:51.206 INFO:tasks.workunit.client.1.smithi148.stdout:7/422: dread d5/f7b [0,4194304] 0 2024-07-03T15:43:51.207 INFO:tasks.workunit.client.1.smithi148.stdout:1/293: stat d0/d2/fd 0 2024-07-03T15:43:51.207 INFO:tasks.workunit.client.1.smithi148.stdout:0/356: truncate da/d36/d5b/f72 685866 0 2024-07-03T15:43:51.207 INFO:tasks.workunit.client.1.smithi148.stdout:7/423: write d5/d16/f37 [1043816,122684] 0 2024-07-03T15:43:51.207 INFO:tasks.workunit.client.1.smithi148.stdout:3/383: dwrite dd/f15 [4194304,4194304] 0 2024-07-03T15:43:51.211 INFO:tasks.workunit.client.1.smithi148.stdout:1/294: mknod d0/d2/d6/d14/d29/c5c 0 2024-07-03T15:43:51.211 INFO:tasks.workunit.client.1.smithi148.stdout:1/295: write d0/f1f [789336,60683] 0 2024-07-03T15:43:51.211 INFO:tasks.workunit.client.1.smithi148.stdout:1/296: chown d0/d2/d6/l53 199230 1 2024-07-03T15:43:51.216 INFO:tasks.workunit.client.1.smithi148.stdout:3/384: mknod dd/d33/d52/c79 0 2024-07-03T15:43:51.216 INFO:tasks.workunit.client.1.smithi148.stdout:7/424: dread d5/d16/d1c/d33/d4c/d21/d31/f45 [0,4194304] 0 2024-07-03T15:43:51.218 INFO:tasks.workunit.client.1.smithi148.stdout:9/616: dwrite d0/d18/d24/f91 [0,4194304] 0 2024-07-03T15:43:51.219 INFO:tasks.workunit.client.1.smithi148.stdout:7/425: mknod d5/d16/d1c/d33/c82 0 2024-07-03T15:43:51.220 INFO:tasks.workunit.client.1.smithi148.stdout:7/426: rmdir d5/d16/d27 39 2024-07-03T15:43:51.221 INFO:tasks.workunit.client.1.smithi148.stdout:7/427: write d5/d16/d1c/d33/d4c/d21/d39/f77 [1040825,2016] 0 2024-07-03T15:43:51.222 INFO:tasks.workunit.client.1.smithi148.stdout:1/297: rename d0/de/d2f/d40 to d0/de/d44/d54/d5d 0 2024-07-03T15:43:51.225 INFO:tasks.workunit.client.1.smithi148.stdout:7/428: link d5/d16/d1c/d33/d4c/d21/d47/f5d d5/d16/d27/f83 0 2024-07-03T15:43:51.227 INFO:tasks.workunit.client.1.smithi148.stdout:1/298: creat d0/de/d44/f5e x:0 0 0 2024-07-03T15:43:51.229 INFO:tasks.workunit.client.1.smithi148.stdout:7/429: mkdir d5/d16/d1c/d33/d4c/d84 0 2024-07-03T15:43:51.232 INFO:tasks.workunit.client.1.smithi148.stdout:1/299: creat d0/d2/d2e/d39/f5f x:0 0 0 2024-07-03T15:43:51.232 INFO:tasks.workunit.client.1.smithi148.stdout:7/430: dread d5/d16/d1c/d23/f68 [0,4194304] 0 2024-07-03T15:43:51.234 INFO:tasks.workunit.client.1.smithi148.stdout:1/300: symlink d0/de/l60 0 2024-07-03T15:43:51.235 INFO:tasks.workunit.client.1.smithi148.stdout:1/301: link d0/l34 d0/de/d44/d54/l61 0 2024-07-03T15:43:51.237 INFO:tasks.workunit.client.1.smithi148.stdout:1/302: rename d0/d2/d6/d14/d29/d4b to d0/d2/d2e/d39/d42/d62 0 2024-07-03T15:43:51.242 INFO:tasks.workunit.client.1.smithi148.stdout:3/385: dwrite dd/f36 [0,4194304] 0 2024-07-03T15:43:51.242 INFO:tasks.workunit.client.1.smithi148.stdout:9/617: dwrite d0/d49/f73 [0,4194304] 0 2024-07-03T15:43:51.246 INFO:tasks.workunit.client.1.smithi148.stdout:7/431: dread d5/d16/d1c/d33/d4c/d21/d39/f3e [0,4194304] 0 2024-07-03T15:43:51.246 INFO:tasks.workunit.client.1.smithi148.stdout:7/432: stat d5/f1f 0 2024-07-03T15:43:51.248 INFO:tasks.workunit.client.1.smithi148.stdout:7/433: symlink d5/d16/d1c/d4a/l85 0 2024-07-03T15:43:51.248 INFO:tasks.workunit.client.1.smithi148.stdout:7/434: fsync d5/d16/d1c/d33/f42 0 2024-07-03T15:43:51.248 INFO:tasks.workunit.client.1.smithi148.stdout:7/435: stat d5/l14 0 2024-07-03T15:43:51.249 INFO:tasks.workunit.client.1.smithi148.stdout:9/618: getdents d0/dd/d25/d37/d40/d72 0 2024-07-03T15:43:51.250 INFO:tasks.workunit.client.1.smithi148.stdout:7/436: symlink d5/d16/d1c/d4a/d52/l86 0 2024-07-03T15:43:51.251 INFO:tasks.workunit.client.1.smithi148.stdout:7/437: fsync d5/d16/d1c/f71 0 2024-07-03T15:43:51.251 INFO:tasks.workunit.client.1.smithi148.stdout:7/438: fdatasync d5/d16/d1c/d4a/d52/f5f 0 2024-07-03T15:43:51.252 INFO:tasks.workunit.client.1.smithi148.stdout:1/303: dwrite d0/d2/d6/fa [0,4194304] 0 2024-07-03T15:43:51.252 INFO:tasks.workunit.client.1.smithi148.stdout:1/304: write d0/d2/f10 [3239501,127938] 0 2024-07-03T15:43:51.253 INFO:tasks.workunit.client.1.smithi148.stdout:3/386: write dd/f36 [3550011,57266] 0 2024-07-03T15:43:51.253 INFO:tasks.workunit.client.1.smithi148.stdout:9/619: truncate d0/d18/d24/d47/fa5 411239 0 2024-07-03T15:43:51.253 INFO:tasks.workunit.client.1.smithi148.stdout:9/620: chown d0/dd/d25/d59 1 1 2024-07-03T15:43:51.254 INFO:tasks.workunit.client.1.smithi148.stdout:3/387: creat dd/d33/d37/d68/d2e/d40/f7a x:0 0 0 2024-07-03T15:43:51.256 INFO:tasks.workunit.client.1.smithi148.stdout:9/621: symlink d0/d49/d43/d19/d8c/lc6 0 2024-07-03T15:43:51.256 INFO:tasks.workunit.client.1.smithi148.stdout:9/622: write d0/d2f/f41 [1585923,97210] 0 2024-07-03T15:43:51.256 INFO:tasks.workunit.client.1.smithi148.stdout:9/623: chown d0/dd/d25/d37/d40/d72/f9d 128174176 1 2024-07-03T15:43:51.256 INFO:tasks.workunit.client.1.smithi148.stdout:9/624: fdatasync d0/dd/d46/fc4 0 2024-07-03T15:43:51.256 INFO:tasks.workunit.client.1.smithi148.stdout:9/625: read d0/dd/f90 [1754168,81753] 0 2024-07-03T15:43:51.256 INFO:tasks.workunit.client.1.smithi148.stdout:9/626: fsync d0/d2f/f7f 0 2024-07-03T15:43:51.267 INFO:tasks.workunit.client.1.smithi148.stdout:1/305: read d0/f25 [2946616,59093] 0 2024-07-03T15:43:51.268 INFO:tasks.workunit.client.1.smithi148.stdout:9/627: write d0/dd/d11/f79 [4162540,72413] 0 2024-07-03T15:43:51.270 INFO:tasks.workunit.client.1.smithi148.stdout:9/628: creat d0/d63/fc7 x:0 0 0 2024-07-03T15:43:51.272 INFO:tasks.workunit.client.1.smithi148.stdout:9/629: rename d0/dd/f16 to d0/d49/d43/d19/fc8 0 2024-07-03T15:43:51.272 INFO:tasks.workunit.client.1.smithi148.stdout:9/630: fsync d0/dd/d25/d37/d40/d72/f8d 0 2024-07-03T15:43:51.273 INFO:tasks.workunit.client.1.smithi148.stdout:9/631: mknod d0/dd/cc9 0 2024-07-03T15:43:51.275 INFO:tasks.workunit.client.1.smithi148.stdout:9/632: mknod d0/d18/d24/d5a/cca 0 2024-07-03T15:43:51.275 INFO:tasks.workunit.client.1.smithi148.stdout:9/633: write d0/fbb [1528717,84717] 0 2024-07-03T15:43:51.277 INFO:tasks.workunit.client.1.smithi148.stdout:9/634: mkdir d0/dd/d25/d59/dcb 0 2024-07-03T15:43:51.277 INFO:tasks.workunit.client.1.smithi148.stdout:7/439: dwrite d5/d16/d1c/d23/f2b [0,4194304] 0 2024-07-03T15:43:51.278 INFO:tasks.workunit.client.1.smithi148.stdout:9/635: stat d0/d18/d24/d47/d4d/dae/lb7 0 2024-07-03T15:43:51.278 INFO:tasks.workunit.client.1.smithi148.stdout:3/388: dwrite dd/f31 [0,4194304] 0 2024-07-03T15:43:51.278 INFO:tasks.workunit.client.1.smithi148.stdout:3/389: chown dd/d33/d37/d68/l5b 5 1 2024-07-03T15:43:51.279 INFO:tasks.workunit.client.1.smithi148.stdout:7/440: mkdir d5/d16/d1c/d4a/d52/d87 0 2024-07-03T15:43:51.280 INFO:tasks.workunit.client.1.smithi148.stdout:7/441: write d5/d16/f37 [588908,43769] 0 2024-07-03T15:43:51.282 INFO:tasks.workunit.client.1.smithi148.stdout:3/390: rename dd/d33/d37/d68/d2e/d40/f7a to dd/d33/d37/d68/d2e/d40/d45/d65/f7b 0 2024-07-03T15:43:51.282 INFO:tasks.workunit.client.1.smithi148.stdout:3/391: chown dd/f21 77293796 1 2024-07-03T15:43:51.286 INFO:tasks.workunit.client.1.smithi148.stdout:1/306: dwrite d0/d2/fd [4194304,4194304] 0 2024-07-03T15:43:51.287 INFO:tasks.workunit.client.1.smithi148.stdout:1/307: creat d0/d2/d2e/d39/d41/f63 x:0 0 0 2024-07-03T15:43:51.288 INFO:tasks.workunit.client.1.smithi148.stdout:7/442: rename d5/d16/d1c/d23/f49 to d5/d16/d1c/d33/d4c/d21/d47/f88 0 2024-07-03T15:43:51.292 INFO:tasks.workunit.client.1.smithi148.stdout:9/636: dread d0/d49/d43/d19/f48 [0,4194304] 0 2024-07-03T15:43:51.293 INFO:tasks.workunit.client.1.smithi148.stdout:7/443: creat d5/d16/d1c/d4a/d52/d87/f89 x:0 0 0 2024-07-03T15:43:51.294 INFO:tasks.workunit.client.1.smithi148.stdout:7/444: link d5/d16/d27/c5a d5/d16/d1c/d33/d4c/d84/c8a 0 2024-07-03T15:43:51.294 INFO:tasks.workunit.client.1.smithi148.stdout:3/392: dread dd/f2b [0,4194304] 0 2024-07-03T15:43:51.296 INFO:tasks.workunit.client.1.smithi148.stdout:3/393: mknod dd/d33/d37/d68/d2e/d40/c7c 0 2024-07-03T15:43:51.298 INFO:tasks.workunit.client.1.smithi148.stdout:3/394: link dd/d33/d37/d68/d2e/c17 dd/d33/d52/d57/c7d 0 2024-07-03T15:43:51.300 INFO:tasks.workunit.client.1.smithi148.stdout:3/395: dread dd/d35/f3f [0,4194304] 0 2024-07-03T15:43:51.301 INFO:tasks.workunit.client.1.smithi148.stdout:3/396: mknod dd/d33/d37/c7e 0 2024-07-03T15:43:51.302 INFO:tasks.workunit.client.1.smithi148.stdout:3/397: fsync dd/f36 0 2024-07-03T15:43:51.302 INFO:tasks.workunit.client.1.smithi148.stdout:3/398: read dd/d33/d37/d68/d2e/f11 [955330,37938] 0 2024-07-03T15:43:51.302 INFO:tasks.workunit.client.1.smithi148.stdout:3/399: write dd/d33/d37/d68/d2e/d40/f4c [433891,69613] 0 2024-07-03T15:43:51.302 INFO:tasks.workunit.client.1.smithi148.stdout:3/400: write dd/f22 [741651,44528] 0 2024-07-03T15:43:51.305 INFO:tasks.workunit.client.1.smithi148.stdout:9/637: write d0/dd/ff [1230538,48825] 0 2024-07-03T15:43:51.306 INFO:tasks.workunit.client.1.smithi148.stdout:7/445: write d5/d16/d1c/d33/d4c/d21/d39/f77 [826135,112679] 0 2024-07-03T15:43:51.313 INFO:tasks.workunit.client.1.smithi148.stdout:3/401: dread dd/f2a [0,4194304] 0 2024-07-03T15:43:51.315 INFO:tasks.workunit.client.1.smithi148.stdout:3/402: symlink dd/d33/d37/d68/d2e/d61/l7f 0 2024-07-03T15:43:51.315 INFO:tasks.workunit.client.1.smithi148.stdout:3/403: write dd/d35/f4f [291283,29529] 0 2024-07-03T15:43:51.316 INFO:tasks.workunit.client.1.smithi148.stdout:3/404: creat dd/d33/d37/d68/d2e/d40/f80 x:0 0 0 2024-07-03T15:43:51.321 INFO:tasks.workunit.client.1.smithi148.stdout:3/405: rmdir dd/d33/d37/d68/d2e/d40/d45 39 2024-07-03T15:43:51.323 INFO:tasks.workunit.client.1.smithi148.stdout:8/327: sync 2024-07-03T15:43:51.327 INFO:tasks.workunit.client.1.smithi148.stdout:6/431: sync 2024-07-03T15:43:51.327 INFO:tasks.workunit.client.1.smithi148.stdout:6/432: stat d3/d1d/f24 0 2024-07-03T15:43:51.327 INFO:tasks.workunit.client.1.smithi148.stdout:6/433: read d3/f1c [1676961,1506] 0 2024-07-03T15:43:51.327 INFO:tasks.workunit.client.1.smithi148.stdout:6/434: chown d3/d50/c5a 86214301 1 2024-07-03T15:43:51.327 INFO:tasks.workunit.client.1.smithi148.stdout:1/308: dwrite d0/d2/d6/f26 [4194304,4194304] 0 2024-07-03T15:43:51.328 INFO:tasks.workunit.client.1.smithi148.stdout:1/309: chown d0/d2/f10 6271 1 2024-07-03T15:43:51.328 INFO:tasks.workunit.client.1.smithi148.stdout:6/435: dread d3/d50/f72 [4194304,4194304] 0 2024-07-03T15:43:51.328 INFO:tasks.workunit.client.1.smithi148.stdout:6/436: fdatasync d3/d1d/d28/d31/f4b 0 2024-07-03T15:43:51.329 INFO:tasks.workunit.client.1.smithi148.stdout:8/328: mknod d4/da/d51/d18/d1f/c74 0 2024-07-03T15:43:51.329 INFO:tasks.workunit.client.1.smithi148.stdout:9/638: dwrite d0/d63/f98 [0,4194304] 0 2024-07-03T15:43:51.329 INFO:tasks.workunit.client.1.smithi148.stdout:5/354: sync 2024-07-03T15:43:51.329 INFO:tasks.workunit.client.1.smithi148.stdout:4/299: sync 2024-07-03T15:43:51.329 INFO:tasks.workunit.client.1.smithi148.stdout:2/462: sync 2024-07-03T15:43:51.332 INFO:tasks.workunit.client.1.smithi148.stdout:7/446: dwrite d5/d16/d1c/d23/f68 [0,4194304] 0 2024-07-03T15:43:51.332 INFO:tasks.workunit.client.1.smithi148.stdout:7/447: write d5/d16/d1c/d33/d4c/d21/f3c [953317,65157] 0 2024-07-03T15:43:51.333 INFO:tasks.workunit.client.0.smithi028.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-07-03T15:43:51.334 INFO:tasks.workunit.client.1.smithi148.stdout:6/437: rmdir d3/d95 0 2024-07-03T15:43:51.335 INFO:tasks.workunit.client.1.smithi148.stdout:9/639: mkdir d0/dd/d25/dcc 0 2024-07-03T15:43:51.337 INFO:tasks.workunit.client.1.smithi148.stdout:5/355: unlink d7/da/df/c37 0 2024-07-03T15:43:51.337 INFO:tasks.workunit.client.1.smithi148.stdout:5/356: read - d7/da/df/d16/d2f/d44/d52/f76 zero size 2024-07-03T15:43:51.337 INFO:tasks.workunit.client.1.smithi148.stdout:5/357: dread - d7/f75 zero size 2024-07-03T15:43:51.337 INFO:tasks.workunit.client.1.smithi148.stdout:4/300: creat d0/d28/f63 x:0 0 0 2024-07-03T15:43:51.338 INFO:tasks.workunit.client.1.smithi148.stdout:2/463: creat d5/d14/d22/d2b/f98 x:0 0 0 2024-07-03T15:43:51.341 INFO:tasks.workunit.client.1.smithi148.stdout:7/448: mkdir d5/d16/d1c/d33/d4c/d21/d47/d8b 0 2024-07-03T15:43:51.341 INFO:tasks.workunit.client.1.smithi148.stdout:7/449: truncate d5/d16/f37 1528248 0 2024-07-03T15:43:51.342 INFO:tasks.workunit.client.1.smithi148.stdout:3/406: rmdir dd/d33/d37 39 2024-07-03T15:43:51.342 INFO:tasks.workunit.client.1.smithi148.stdout:3/407: chown dd/d33/d37/d68/d2e/c54 577253427 1 2024-07-03T15:43:51.342 INFO:tasks.workunit.client.1.smithi148.stdout:6/438: rmdir d3/d5b/d74 39 2024-07-03T15:43:51.344 INFO:tasks.workunit.client.1.smithi148.stdout:2/464: dread d5/d14/d22/d2b/d13/f29 [0,4194304] 0 2024-07-03T15:43:51.345 INFO:tasks.workunit.client.1.smithi148.stdout:4/301: truncate d0/f3d 2592964 0 2024-07-03T15:43:51.346 INFO:tasks.workunit.client.1.smithi148.stdout:7/450: symlink d5/d16/d1c/d23/l8c 0 2024-07-03T15:43:51.346 INFO:tasks.workunit.client.1.smithi148.stdout:7/451: readlink d5/d16/d1c/l72 0 2024-07-03T15:43:51.349 INFO:tasks.workunit.client.1.smithi148.stdout:3/408: mkdir dd/d35/d81 0 2024-07-03T15:43:51.349 INFO:tasks.workunit.client.1.smithi148.stdout:3/409: write dd/f62 [31202,35276] 0 2024-07-03T15:43:51.349 INFO:tasks.workunit.client.1.smithi148.stdout:3/410: chown dd/f31 424032985 1 2024-07-03T15:43:51.350 INFO:tasks.workunit.client.1.smithi148.stdout:0/357: sync 2024-07-03T15:43:51.350 INFO:tasks.workunit.client.1.smithi148.stdout:0/358: dread - da/d36/f63 zero size 2024-07-03T15:43:51.350 INFO:tasks.workunit.client.1.smithi148.stdout:6/439: creat d3/d50/f96 x:0 0 0 2024-07-03T15:43:51.351 INFO:tasks.workunit.client.1.smithi148.stdout:1/310: dwrite d0/de/d2f/f55 [0,4194304] 0 2024-07-03T15:43:51.352 INFO:tasks.workunit.client.1.smithi148.stdout:9/640: dwrite d0/dd/d25/d6a/fb3 [0,4194304] 0 2024-07-03T15:43:51.356 INFO:tasks.workunit.client.1.smithi148.stdout:5/358: rmdir d7/da/df/d16/d2f 39 2024-07-03T15:43:51.356 INFO:tasks.workunit.client.1.smithi148.stdout:5/359: fdatasync d7/da/df/d16/d1a/f50 0 2024-07-03T15:43:51.358 INFO:tasks.workunit.client.1.smithi148.stdout:8/329: dwrite d4/da/d14/f4d [0,4194304] 0 2024-07-03T15:43:51.358 INFO:tasks.workunit.client.1.smithi148.stdout:2/465: mknod d5/d7/c99 0 2024-07-03T15:43:51.358 INFO:tasks.workunit.client.1.smithi148.stdout:2/466: fdatasync d5/d14/d22/d2b/f42 0 2024-07-03T15:43:51.358 INFO:tasks.workunit.client.1.smithi148.stdout:2/467: write d5/d14/d22/d2b/d13/d41/f49 [211061,34671] 0 2024-07-03T15:43:51.358 INFO:tasks.workunit.client.1.smithi148.stdout:2/468: chown d5/d14/d22/d2b/d68/f84 0 1 2024-07-03T15:43:51.361 INFO:tasks.workunit.client.1.smithi148.stdout:4/302: symlink d0/d8/d19/d45/d49/l64 0 2024-07-03T15:43:51.362 INFO:tasks.workunit.client.1.smithi148.stdout:7/452: stat d5/d16/d27/c40 0 2024-07-03T15:43:51.363 INFO:tasks.workunit.client.1.smithi148.stdout:3/411: getdents dd/d35/d41 0 2024-07-03T15:43:51.363 INFO:tasks.workunit.client.1.smithi148.stdout:3/412: chown dd/d33/d37/d68/l5b 6 1 2024-07-03T15:43:51.364 INFO:tasks.workunit.client.1.smithi148.stdout:0/359: symlink da/d44/d24/l77 0 2024-07-03T15:43:51.367 INFO:tasks.workunit.client.1.smithi148.stdout:5/360: creat d7/d13/d23/f78 x:0 0 0 2024-07-03T15:43:51.367 INFO:tasks.workunit.client.1.smithi148.stdout:5/361: fsync d7/da/df/f33 0 2024-07-03T15:43:51.368 INFO:tasks.workunit.client.1.smithi148.stdout:8/330: chown d4/da/d51/l9 15 1 2024-07-03T15:43:51.370 INFO:tasks.workunit.client.1.smithi148.stdout:2/469: link d5/d14/l45 d5/d14/d86/d93/l9a 0 2024-07-03T15:43:51.371 INFO:tasks.workunit.client.1.smithi148.stdout:4/303: unlink d0/d8/f38 0 2024-07-03T15:43:51.373 INFO:tasks.workunit.client.1.smithi148.stdout:6/440: fsync d3/d50/f96 0 2024-07-03T15:43:51.373 INFO:tasks.workunit.client.1.smithi148.stdout:0/360: symlink da/d44/d13/d5a/l78 0 2024-07-03T15:43:51.375 INFO:tasks.workunit.client.1.smithi148.stdout:5/362: mkdir d7/da/d79 0 2024-07-03T15:43:51.378 INFO:tasks.workunit.client.1.smithi148.stdout:2/470: link d5/d14/d22/d60/f91 d5/d14/d22/d2b/d13/f9b 0 2024-07-03T15:43:51.378 INFO:tasks.workunit.client.1.smithi148.stdout:2/471: write d5/d14/d22/f72 [4886107,62033] 0 2024-07-03T15:43:51.380 INFO:tasks.workunit.client.1.smithi148.stdout:4/304: unlink d0/d8/d13/d48/f2e 0 2024-07-03T15:43:51.381 INFO:tasks.workunit.client.1.smithi148.stdout:3/413: rmdir dd/d35 39 2024-07-03T15:43:51.383 INFO:tasks.workunit.client.1.smithi148.stdout:6/441: stat d3/d1d/d28/d53/d6d/c7e 0 2024-07-03T15:43:51.385 INFO:tasks.workunit.client.1.smithi148.stdout:0/361: creat da/d58/f79 x:0 0 0 2024-07-03T15:43:51.386 INFO:tasks.workunit.client.1.smithi148.stdout:8/331: mknod d4/da/d51/c75 0 2024-07-03T15:43:51.386 INFO:tasks.workunit.client.1.smithi148.stdout:6/442: rmdir d3/d1d/d28/d7f/d85 39 2024-07-03T15:43:51.387 INFO:tasks.workunit.client.1.smithi148.stdout:7/453: dwrite d5/f1f [0,4194304] 0 2024-07-03T15:43:51.387 INFO:tasks.workunit.client.1.smithi148.stdout:7/454: chown d5/d16/d1c/d33/c82 22 1 2024-07-03T15:43:51.389 INFO:tasks.workunit.client.1.smithi148.stdout:0/362: unlink da/d38/f57 0 2024-07-03T15:43:51.389 INFO:tasks.workunit.client.1.smithi148.stdout:0/363: write da/d44/f75 [2393670,58780] 0 2024-07-03T15:43:51.389 INFO:tasks.workunit.client.1.smithi148.stdout:0/364: creat da/d17/d2a/f7a x:0 0 0 2024-07-03T15:43:51.391 INFO:tasks.workunit.client.1.smithi148.stdout:8/332: mkdir d4/da/d14/d17/d76 0 2024-07-03T15:43:51.392 INFO:tasks.workunit.client.1.smithi148.stdout:6/443: mknod d3/c97 0 2024-07-03T15:43:51.392 INFO:tasks.workunit.client.1.smithi148.stdout:6/444: truncate d3/d1d/d28/f69 1850978 0 2024-07-03T15:43:51.396 INFO:tasks.workunit.client.1.smithi148.stdout:7/455: read d5/d16/d1c/d33/d4c/d21/d31/f45 [2078105,25130] 0 2024-07-03T15:43:51.396 INFO:tasks.workunit.client.1.smithi148.stdout:7/456: chown d5/d16/d1c/d33/d4c/d21/d39/l61 3614 1 2024-07-03T15:43:51.397 INFO:tasks.workunit.client.1.smithi148.stdout:1/311: dwrite d0/d2/d11/d1e/f3c [4194304,4194304] 0 2024-07-03T15:43:51.397 INFO:tasks.workunit.client.1.smithi148.stdout:8/333: creat d4/da/d50/f77 x:0 0 0 2024-07-03T15:43:51.399 INFO:tasks.workunit.client.1.smithi148.stdout:6/445: creat d3/d1d/d28/d31/f98 x:0 0 0 2024-07-03T15:43:51.401 INFO:tasks.workunit.client.1.smithi148.stdout:7/457: symlink d5/d16/d1c/d33/d4c/d84/l8d 0 2024-07-03T15:43:51.401 INFO:tasks.workunit.client.1.smithi148.stdout:7/458: write d5/d16/d1c/d33/d4c/d21/f32 [615700,6450] 0 2024-07-03T15:43:51.401 INFO:tasks.workunit.client.1.smithi148.stdout:7/459: creat d5/d16/d1c/f8e x:0 0 0 2024-07-03T15:43:51.401 INFO:tasks.workunit.client.1.smithi148.stdout:7/460: fdatasync d5/d16/d1c/d33/d4c/f12 0 2024-07-03T15:43:51.403 INFO:tasks.workunit.client.1.smithi148.stdout:1/312: link d0/d2/d6/d14/d29/c5c d0/de/c64 0 2024-07-03T15:43:51.405 INFO:tasks.workunit.client.1.smithi148.stdout:6/446: truncate d3/f1c 7146883 0 2024-07-03T15:43:51.405 INFO:tasks.workunit.client.1.smithi148.stdout:6/447: write d3/d1d/d28/d3e/f8b [580739,48194] 0 2024-07-03T15:43:51.405 INFO:tasks.workunit.client.1.smithi148.stdout:2/472: dwrite d5/d14/d22/d2b/f3a [0,4194304] 0 2024-07-03T15:43:51.406 INFO:tasks.workunit.client.1.smithi148.stdout:7/461: mknod d5/d16/d1c/d23/c8f 0 2024-07-03T15:43:51.406 INFO:tasks.workunit.client.1.smithi148.stdout:6/448: dread d3/f34 [0,4194304] 0 2024-07-03T15:43:51.407 INFO:tasks.workunit.client.1.smithi148.stdout:6/449: chown d3/d1d/d28/c52 14214 1 2024-07-03T15:43:51.407 INFO:tasks.workunit.client.1.smithi148.stdout:2/473: mkdir d5/d5f/d79/d9c 0 2024-07-03T15:43:51.408 INFO:tasks.workunit.client.1.smithi148.stdout:7/462: symlink d5/d16/d1c/d4a/d52/l90 0 2024-07-03T15:43:51.408 INFO:tasks.workunit.client.1.smithi148.stdout:5/363: dwrite d7/da/df/d16/f20 [4194304,4194304] 0 2024-07-03T15:43:51.408 INFO:tasks.workunit.client.1.smithi148.stdout:5/364: chown d7/d13/d23/f78 723170713 1 2024-07-03T15:43:51.409 INFO:tasks.workunit.client.1.smithi148.stdout:5/365: chown d7/d13/d23/d40 15224966 1 2024-07-03T15:43:51.409 INFO:tasks.workunit.client.1.smithi148.stdout:6/450: truncate d3/d1d/f91 1054977 0 2024-07-03T15:43:51.410 INFO:tasks.workunit.client.1.smithi148.stdout:7/463: mkdir d5/d16/d1c/d33/d4c/d84/d91 0 2024-07-03T15:43:51.414 INFO:tasks.workunit.client.1.smithi148.stdout:0/365: dread da/d44/f75 [0,4194304] 0 2024-07-03T15:43:51.414 INFO:tasks.workunit.client.1.smithi148.stdout:6/451: creat d3/d5b/f99 x:0 0 0 2024-07-03T15:43:51.415 INFO:tasks.workunit.client.1.smithi148.stdout:3/414: dwrite dd/d33/d37/d68/f5c [0,4194304] 0 2024-07-03T15:43:51.417 INFO:tasks.workunit.client.1.smithi148.stdout:6/452: getdents d3/d1d/d28 0 2024-07-03T15:43:51.417 INFO:tasks.workunit.client.1.smithi148.stdout:6/453: fsync d3/d1d/f25 0 2024-07-03T15:43:51.419 INFO:tasks.workunit.client.1.smithi148.stdout:2/474: dread d5/f2f [0,4194304] 0 2024-07-03T15:43:51.419 INFO:tasks.workunit.client.1.smithi148.stdout:8/334: dwrite d4/da/d14/f6c [0,4194304] 0 2024-07-03T15:43:51.419 INFO:tasks.workunit.client.1.smithi148.stdout:8/335: write d4/da/d14/d3a/f41 [5140823,105518] 0 2024-07-03T15:43:51.420 INFO:tasks.workunit.client.1.smithi148.stdout:4/305: dwrite d0/d8/d3a/f47 [0,4194304] 0 2024-07-03T15:43:51.421 INFO:tasks.workunit.client.1.smithi148.stdout:3/415: rename dd/d33/d37/d68/d2e/f2c to dd/d33/d37/f82 0 2024-07-03T15:43:51.421 INFO:tasks.workunit.client.1.smithi148.stdout:8/336: dread d4/d54/f5b [0,4194304] 0 2024-07-03T15:43:51.422 INFO:tasks.workunit.client.1.smithi148.stdout:6/454: rmdir d3/d1d/d28 39 2024-07-03T15:43:51.422 INFO:tasks.workunit.client.1.smithi148.stdout:6/455: fsync d3/f86 0 2024-07-03T15:43:51.422 INFO:tasks.workunit.client.1.smithi148.stdout:1/313: dread d0/d2/d2e/d39/d42/f48 [0,4194304] 0 2024-07-03T15:43:51.422 INFO:tasks.workunit.client.1.smithi148.stdout:1/314: creat d0/d2/d2e/d39/d41/f65 x:0 0 0 2024-07-03T15:43:51.425 INFO:tasks.workunit.client.1.smithi148.stdout:7/464: dread d5/d16/d1c/d33/d4c/d21/d47/f88 [0,4194304] 0 2024-07-03T15:43:51.426 INFO:tasks.workunit.client.1.smithi148.stdout:6/456: write d3/d1d/d28/f69 [64813,104981] 0 2024-07-03T15:43:51.426 INFO:tasks.workunit.client.1.smithi148.stdout:6/457: truncate d3/d1d/d28/d31/d83/f89 99027 0 2024-07-03T15:43:51.428 INFO:tasks.workunit.client.1.smithi148.stdout:3/416: getdents dd 0 2024-07-03T15:43:51.429 INFO:tasks.workunit.client.1.smithi148.stdout:8/337: symlink d4/da/d14/d1a/d32/d38/d58/l78 0 2024-07-03T15:43:51.431 INFO:tasks.workunit.client.1.smithi148.stdout:7/465: symlink d5/d16/l92 0 2024-07-03T15:43:51.433 INFO:tasks.workunit.client.1.smithi148.stdout:8/338: link d4/da/d14/d1a/d32/d38/d58/l78 d4/l79 0 2024-07-03T15:43:51.434 INFO:tasks.workunit.client.1.smithi148.stdout:7/466: mkdir d5/d16/d1c/d4a/d66/d93 0 2024-07-03T15:43:51.435 INFO:tasks.workunit.client.1.smithi148.stdout:3/417: dread dd/d35/f3f [0,4194304] 0 2024-07-03T15:43:51.435 INFO:tasks.workunit.client.1.smithi148.stdout:8/339: dread d4/da/d51/d18/d20/f5a [0,4194304] 0 2024-07-03T15:43:51.435 INFO:tasks.workunit.client.1.smithi148.stdout:2/475: dread d5/d14/d22/f62 [0,4194304] 0 2024-07-03T15:43:51.436 INFO:tasks.workunit.client.1.smithi148.stdout:7/467: creat d5/d16/d1c/d33/f94 x:0 0 0 2024-07-03T15:43:51.436 INFO:tasks.workunit.client.1.smithi148.stdout:7/468: chown d5/f3b 491 1 2024-07-03T15:43:51.437 INFO:tasks.workunit.client.1.smithi148.stdout:6/458: dread d3/d1d/f21 [0,4194304] 0 2024-07-03T15:43:51.437 INFO:tasks.workunit.client.1.smithi148.stdout:3/418: creat dd/d33/d37/f83 x:0 0 0 2024-07-03T15:43:51.438 INFO:tasks.workunit.client.1.smithi148.stdout:8/340: creat d4/da/d14/d1a/d32/d38/f7a x:0 0 0 2024-07-03T15:43:51.440 INFO:tasks.workunit.client.1.smithi148.stdout:8/341: unlink d4/d54/f5b 0 2024-07-03T15:43:51.442 INFO:tasks.workunit.client.1.smithi148.stdout:7/469: dread d5/f9 [0,4194304] 0 2024-07-03T15:43:51.443 INFO:tasks.workunit.client.1.smithi148.stdout:7/470: creat d5/d16/d27/d65/f95 x:0 0 0 2024-07-03T15:43:51.445 INFO:tasks.workunit.client.1.smithi148.stdout:2/476: dread d5/d14/d22/d60/f53 [0,4194304] 0 2024-07-03T15:43:51.447 INFO:tasks.workunit.client.1.smithi148.stdout:2/477: unlink d5/d14/d22/d2b/f3a 0 2024-07-03T15:43:51.448 INFO:tasks.workunit.client.1.smithi148.stdout:2/478: mkdir d5/d14/d22/d2b/d13/d41/d9d 0 2024-07-03T15:43:51.449 INFO:tasks.workunit.client.1.smithi148.stdout:2/479: truncate d5/d7/f83 35864 0 2024-07-03T15:43:51.449 INFO:tasks.workunit.client.1.smithi148.stdout:6/459: dread d3/f1e [0,4194304] 0 2024-07-03T15:43:51.449 INFO:tasks.workunit.client.1.smithi148.stdout:6/460: write d3/f55 [871009,14197] 0 2024-07-03T15:43:51.449 INFO:tasks.workunit.client.1.smithi148.stdout:2/480: rename d5/d14/d22/d60/d89/l97 to d5/d14/d86/d93/l9e 0 2024-07-03T15:43:51.451 INFO:tasks.workunit.client.1.smithi148.stdout:2/481: truncate d5/d14/d22/d2b/d13/f29 487210 0 2024-07-03T15:43:51.451 INFO:tasks.workunit.client.1.smithi148.stdout:2/482: read d5/d14/d22/d2b/d13/d2d/f3b [94946,31857] 0 2024-07-03T15:43:51.453 INFO:tasks.workunit.client.1.smithi148.stdout:7/471: dread d5/d16/d1c/d33/f67 [0,4194304] 0 2024-07-03T15:43:51.453 INFO:tasks.workunit.client.1.smithi148.stdout:7/472: fdatasync d5/d16/d1c/d59/f76 0 2024-07-03T15:43:51.453 INFO:tasks.workunit.client.1.smithi148.stdout:2/483: dread d5/d14/d22/d2b/f42 [0,4194304] 0 2024-07-03T15:43:51.454 INFO:tasks.workunit.client.1.smithi148.stdout:2/484: link d5/d7/d64/l75 d5/d14/l9f 0 2024-07-03T15:43:51.458 INFO:tasks.workunit.client.1.smithi148.stdout:2/485: rmdir d5/d14/d22/d2b/d13 39 2024-07-03T15:43:51.458 INFO:tasks.workunit.client.1.smithi148.stdout:2/486: read d5/d14/d22/d60/f90 [1464997,118248] 0 2024-07-03T15:43:51.458 INFO:tasks.workunit.client.1.smithi148.stdout:2/487: dread - d5/d14/d22/d60/f91 zero size 2024-07-03T15:43:51.458 INFO:tasks.workunit.client.1.smithi148.stdout:2/488: chown d5/d67/c95 0 1 2024-07-03T15:43:51.458 INFO:tasks.workunit.client.1.smithi148.stdout:2/489: fdatasync d5/d14/d22/d2b/d68/f7c 0 2024-07-03T15:43:51.458 INFO:tasks.workunit.client.1.smithi148.stdout:2/490: fsync d5/d7/d64/f71 0 2024-07-03T15:43:51.458 INFO:tasks.workunit.client.1.smithi148.stdout:8/342: write d4/da/f29 [1001027,39057] 0 2024-07-03T15:43:51.458 INFO:tasks.workunit.client.1.smithi148.stdout:2/491: mknod d5/d5f/ca0 0 2024-07-03T15:43:51.460 INFO:tasks.workunit.client.1.smithi148.stdout:8/343: rename d4/da/d47/l6b to d4/da/d14/d1a/d32/d38/d58/l7b 0 2024-07-03T15:43:51.462 INFO:tasks.workunit.client.1.smithi148.stdout:5/366: dwrite d7/d13/d54/f5d [0,4194304] 0 2024-07-03T15:43:51.462 INFO:tasks.workunit.client.1.smithi148.stdout:1/315: dwrite d0/d2/d6/f26 [8388608,4194304] 0 2024-07-03T15:43:51.462 INFO:tasks.workunit.client.1.smithi148.stdout:1/316: creat d0/d2/d11/d1e/f66 x:0 0 0 2024-07-03T15:43:51.462 INFO:tasks.workunit.client.1.smithi148.stdout:5/367: creat d7/da/df/d16/d1a/f7a x:0 0 0 2024-07-03T15:43:51.462 INFO:tasks.workunit.client.1.smithi148.stdout:5/368: chown d7/d59 387871363 1 2024-07-03T15:43:51.462 INFO:tasks.workunit.client.1.smithi148.stdout:5/369: chown l1 406612 1 2024-07-03T15:43:51.462 INFO:tasks.workunit.client.1.smithi148.stdout:1/317: write d0/f24 [2433016,18114] 0 2024-07-03T15:43:51.462 INFO:tasks.workunit.client.1.smithi148.stdout:5/370: write d7/da/df/d16/d1a/f50 [410779,80344] 0 2024-07-03T15:43:51.462 INFO:tasks.workunit.client.1.smithi148.stdout:5/371: chown d7/da/f2b 139625 1 2024-07-03T15:43:51.463 INFO:tasks.workunit.client.1.smithi148.stdout:8/344: symlink d4/l7c 0 2024-07-03T15:43:51.463 INFO:tasks.workunit.client.1.smithi148.stdout:4/306: dwrite d0/f5 [4194304,4194304] 0 2024-07-03T15:43:51.463 INFO:tasks.workunit.client.1.smithi148.stdout:8/345: chown d4/da/d50 124 1 2024-07-03T15:43:51.463 INFO:tasks.workunit.client.1.smithi148.stdout:0/366: dwrite da/d44/d13/d14/d50/f60 [0,4194304] 0 2024-07-03T15:43:51.463 INFO:tasks.workunit.client.1.smithi148.stdout:0/367: dread - da/d17/d2a/f4b zero size 2024-07-03T15:43:51.471 INFO:tasks.workunit.client.1.smithi148.stdout:5/372: rename d7/da/df/f4d to d7/d13/d54/f7b 0 2024-07-03T15:43:51.471 INFO:tasks.workunit.client.1.smithi148.stdout:5/373: fsync d7/da/df/d16/f20 0 2024-07-03T15:43:51.471 INFO:tasks.workunit.client.1.smithi148.stdout:5/374: dread - d7/da/f73 zero size 2024-07-03T15:43:51.475 INFO:tasks.workunit.client.1.smithi148.stdout:4/307: mknod d0/d8/d19/c65 0 2024-07-03T15:43:51.484 INFO:tasks.workunit.client.1.smithi148.stdout:7/473: dwrite d5/d16/d27/f28 [0,4194304] 0 2024-07-03T15:43:51.484 INFO:tasks.workunit.client.1.smithi148.stdout:7/474: chown d5/d16/d1c/d33/l74 6 1 2024-07-03T15:43:51.485 INFO:tasks.workunit.client.1.smithi148.stdout:2/492: dwrite d5/d14/d22/d2b/d13/d1d/f7f [0,4194304] 0 2024-07-03T15:43:51.485 INFO:tasks.workunit.client.1.smithi148.stdout:5/375: dread f4 [0,4194304] 0 2024-07-03T15:43:51.485 INFO:tasks.workunit.client.1.smithi148.stdout:1/318: dwrite d0/de/d44/d54/d5d/f50 [0,4194304] 0 2024-07-03T15:43:51.485 INFO:tasks.workunit.client.1.smithi148.stdout:2/493: stat d5/d14/f7a 0 2024-07-03T15:43:51.485 INFO:tasks.workunit.client.1.smithi148.stdout:5/376: read d7/da/f2b [426574,53358] 0 2024-07-03T15:43:51.485 INFO:tasks.workunit.client.1.smithi148.stdout:5/377: chown d7/da/f2b 24 1 2024-07-03T15:43:51.489 INFO:tasks.workunit.client.1.smithi148.stdout:8/346: rename d4/da/d14/d1a/d32/d38/f7a to d4/da/d14/d17/d76/f7d 0 2024-07-03T15:43:51.489 INFO:tasks.workunit.client.1.smithi148.stdout:8/347: chown d4/da/d51/l9 3681 1 2024-07-03T15:43:51.489 INFO:tasks.workunit.client.1.smithi148.stdout:7/475: dread d5/d16/d1c/d33/d4c/f26 [0,4194304] 0 2024-07-03T15:43:51.491 INFO:tasks.workunit.client.1.smithi148.stdout:2/494: truncate d5/d7/f38 2614744 0 2024-07-03T15:43:51.492 INFO:tasks.workunit.client.1.smithi148.stdout:2/495: chown d5/d5f/d79 7574680 1 2024-07-03T15:43:51.494 INFO:tasks.workunit.client.1.smithi148.stdout:0/368: rename da/d44/d13/d1d/l23 to da/d17/d2a/l7b 0 2024-07-03T15:43:51.494 INFO:tasks.workunit.client.1.smithi148.stdout:0/369: dread - da/d17/d2a/f4b zero size 2024-07-03T15:43:51.494 INFO:tasks.workunit.client.1.smithi148.stdout:0/370: write da/d44/d13/d14/f1f [2563416,118190] 0 2024-07-03T15:43:51.494 INFO:tasks.workunit.client.1.smithi148.stdout:0/371: write da/d44/d13/d14/f54 [1608779,94431] 0 2024-07-03T15:43:51.496 INFO:tasks.workunit.client.1.smithi148.stdout:2/496: mkdir d5/d14/d22/d2b/d13/d1d/da1 0 2024-07-03T15:43:51.497 INFO:tasks.workunit.client.1.smithi148.stdout:4/308: rename d0/f4f to d0/d8/d3a/f66 0 2024-07-03T15:43:51.498 INFO:tasks.workunit.client.1.smithi148.stdout:4/309: readlink d0/d8/d13/l15 0 2024-07-03T15:43:51.500 INFO:tasks.workunit.client.1.smithi148.stdout:2/497: creat d5/d14/d22/d2b/d13/d1d/da1/fa2 x:0 0 0 2024-07-03T15:43:51.500 INFO:tasks.workunit.client.1.smithi148.stdout:2/498: chown d5/d5f/d79/d9c 657 1 2024-07-03T15:43:51.500 INFO:tasks.workunit.client.1.smithi148.stdout:2/499: write d5/d14/d22/d2b/d13/d41/f74 [3500900,127462] 0 2024-07-03T15:43:51.501 INFO:tasks.workunit.client.1.smithi148.stdout:7/476: rename d5/d16/d1c/d33/d4c/ld to d5/d16/d1c/d33/d4c/d21/d31/d7f/l96 0 2024-07-03T15:43:51.501 INFO:tasks.workunit.client.1.smithi148.stdout:7/477: read d5/d16/f37 [132732,41515] 0 2024-07-03T15:43:51.505 INFO:tasks.workunit.client.1.smithi148.stdout:4/310: getdents d0/d28 0 2024-07-03T15:43:51.505 INFO:tasks.workunit.client.1.smithi148.stdout:4/311: fdatasync d0/d28/f63 0 2024-07-03T15:43:51.505 INFO:tasks.workunit.client.1.smithi148.stdout:4/312: dread - d0/d28/f63 zero size 2024-07-03T15:43:51.505 INFO:tasks.workunit.client.1.smithi148.stdout:2/500: write d5/f2f [45214,33601] 0 2024-07-03T15:43:51.506 INFO:tasks.workunit.client.1.smithi148.stdout:7/478: rename d5/d16/d27 to d5/d16/d1c/d33/d4c/d21/d31/d7f/d97 0 2024-07-03T15:43:51.507 INFO:tasks.workunit.client.1.smithi148.stdout:4/313: mkdir d0/d8/d13/d67 0 2024-07-03T15:43:51.508 INFO:tasks.workunit.client.1.smithi148.stdout:4/314: rename d0/d8/d19/d45/l46 to d0/d8/d13/l68 0 2024-07-03T15:43:51.510 INFO:tasks.workunit.client.1.smithi148.stdout:8/348: dwrite d4/d54/f5c [0,4194304] 0 2024-07-03T15:43:51.515 INFO:tasks.workunit.client.1.smithi148.stdout:5/378: dwrite d7/da/df/d1d/f51 [0,4194304] 0 2024-07-03T15:43:51.515 INFO:tasks.workunit.client.1.smithi148.stdout:2/501: dread d5/d14/d22/d2b/d13/d1d/f43 [0,4194304] 0 2024-07-03T15:43:51.515 INFO:tasks.workunit.client.1.smithi148.stdout:5/379: write d7/da/f4c [1253634,62727] 0 2024-07-03T15:43:51.515 INFO:tasks.workunit.client.1.smithi148.stdout:2/502: write d5/d14/f5a [577217,66597] 0 2024-07-03T15:43:51.516 INFO:tasks.workunit.client.1.smithi148.stdout:2/503: fdatasync d5/f9 0 2024-07-03T15:43:51.516 INFO:tasks.workunit.client.1.smithi148.stdout:2/504: fsync d5/d14/d22/d2b/d13/d1d/da1/fa2 0 2024-07-03T15:43:51.516 INFO:tasks.workunit.client.1.smithi148.stdout:2/505: chown d5/d14/d22/d2b/d13/d41/f63 19230 1 2024-07-03T15:43:51.518 INFO:tasks.workunit.client.1.smithi148.stdout:7/479: write d5/d16/d1c/d23/f2b [2720062,48685] 0 2024-07-03T15:43:51.520 INFO:tasks.workunit.client.1.smithi148.stdout:5/380: creat d7/da/df/d16/d2f/d44/f7c x:0 0 0 2024-07-03T15:43:51.520 INFO:tasks.workunit.client.1.smithi148.stdout:5/381: creat d7/d13/d54/f7d x:0 0 0 2024-07-03T15:43:51.522 INFO:tasks.workunit.client.1.smithi148.stdout:2/506: mknod d5/d14/d86/d93/ca3 0 2024-07-03T15:43:51.522 INFO:tasks.workunit.client.1.smithi148.stdout:2/507: chown d5/f1f 66 1 2024-07-03T15:43:51.522 INFO:tasks.workunit.client.1.smithi148.stdout:2/508: dread - d5/d14/d22/d2b/d13/d2d/f4e zero size 2024-07-03T15:43:51.522 INFO:tasks.workunit.client.1.smithi148.stdout:2/509: chown d5/d14/d22/d2b/d13/d41/c87 808677151 1 2024-07-03T15:43:51.523 INFO:tasks.workunit.client.1.smithi148.stdout:7/480: mkdir d5/d16/d1c/d33/d4c/d84/d98 0 2024-07-03T15:43:51.525 INFO:tasks.workunit.client.1.smithi148.stdout:4/315: dwrite d0/f6 [0,4194304] 0 2024-07-03T15:43:51.525 INFO:tasks.workunit.client.1.smithi148.stdout:4/316: write d0/d8/f2d [4988777,7232] 0 2024-07-03T15:43:51.526 INFO:tasks.workunit.client.1.smithi148.stdout:5/382: creat d7/da/df/d16/f7e x:0 0 0 2024-07-03T15:43:51.526 INFO:tasks.workunit.client.1.smithi148.stdout:5/383: dread - d7/da/df/d16/d2f/d44/f7c zero size 2024-07-03T15:43:51.527 INFO:tasks.workunit.client.1.smithi148.stdout:2/510: symlink d5/d14/d22/d2b/d13/d2d/la4 0 2024-07-03T15:43:51.529 INFO:tasks.workunit.client.1.smithi148.stdout:7/481: creat d5/d16/d1c/d33/d4c/f99 x:0 0 0 2024-07-03T15:43:51.532 INFO:tasks.workunit.client.1.smithi148.stdout:2/511: unlink d5/d14/d22/d2b/d13/d41/c44 0 2024-07-03T15:43:51.534 INFO:tasks.workunit.client.1.smithi148.stdout:2/512: symlink d5/d14/d22/d2b/d13/la5 0 2024-07-03T15:43:51.534 INFO:tasks.workunit.client.1.smithi148.stdout:2/513: write d5/d14/d22/d2b/d13/d41/f8c [294187,17933] 0 2024-07-03T15:43:51.535 INFO:tasks.workunit.client.1.smithi148.stdout:5/384: write d7/da/df/d1d/f51 [2519456,8809] 0 2024-07-03T15:43:51.536 INFO:tasks.workunit.client.1.smithi148.stdout:2/514: symlink d5/d5f/d79/la6 0 2024-07-03T15:43:51.537 INFO:tasks.workunit.client.1.smithi148.stdout:5/385: symlink d7/da/df/l7f 0 2024-07-03T15:43:51.544 INFO:tasks.workunit.client.1.smithi148.stdout:2/515: dread d5/d14/d22/d2b/d13/d2d/f51 [0,4194304] 0 2024-07-03T15:43:51.544 INFO:tasks.workunit.client.1.smithi148.stdout:2/516: creat d5/d5f/d7e/fa7 x:0 0 0 2024-07-03T15:43:51.546 INFO:tasks.workunit.client.1.smithi148.stdout:4/317: dwrite d0/d8/d13/d48/f2c [0,4194304] 0 2024-07-03T15:43:51.549 INFO:tasks.workunit.client.1.smithi148.stdout:4/318: unlink d0/d8/d13/l15 0 2024-07-03T15:43:51.550 INFO:tasks.workunit.client.1.smithi148.stdout:4/319: getdents d0/d8/d22/d2f/d41 0 2024-07-03T15:43:51.550 INFO:tasks.workunit.client.1.smithi148.stdout:4/320: write d0/d8/d3a/d36/f3e [354765,28916] 0 2024-07-03T15:43:51.551 INFO:tasks.workunit.client.1.smithi148.stdout:4/321: creat d0/d28/f69 x:0 0 0 2024-07-03T15:43:51.552 INFO:tasks.workunit.client.1.smithi148.stdout:4/322: creat d0/d8/d44/f6a x:0 0 0 2024-07-03T15:43:51.553 INFO:tasks.workunit.client.1.smithi148.stdout:2/517: dread d5/d14/d22/d60/d76/f92 [0,4194304] 0 2024-07-03T15:43:51.553 INFO:tasks.workunit.client.1.smithi148.stdout:4/323: write d0/d8/f2d [4938744,16040] 0 2024-07-03T15:43:51.553 INFO:tasks.workunit.client.1.smithi148.stdout:4/324: write d0/d8/d13/d48/f3f [61056,74848] 0 2024-07-03T15:43:51.554 INFO:tasks.workunit.client.1.smithi148.stdout:5/386: dwrite d7/f19 [4194304,4194304] 0 2024-07-03T15:43:51.554 INFO:tasks.workunit.client.1.smithi148.stdout:2/518: mkdir d5/da8 0 2024-07-03T15:43:51.554 INFO:tasks.workunit.client.1.smithi148.stdout:2/519: readlink d5/d14/d22/d2b/d13/la5 0 2024-07-03T15:43:51.554 INFO:tasks.workunit.client.1.smithi148.stdout:2/520: write d5/d7/d64/f71 [1597146,32946] 0 2024-07-03T15:43:51.555 INFO:tasks.workunit.client.1.smithi148.stdout:2/521: fsync d5/d14/d22/d2b/d13/f1b 0 2024-07-03T15:43:51.557 INFO:tasks.workunit.client.1.smithi148.stdout:4/325: dread d0/f30 [0,4194304] 0 2024-07-03T15:43:51.560 INFO:tasks.workunit.client.1.smithi148.stdout:5/387: symlink d7/da/d21/l80 0 2024-07-03T15:43:51.560 INFO:tasks.workunit.client.1.smithi148.stdout:5/388: dread - d7/d13/d54/f7d zero size 2024-07-03T15:43:51.562 INFO:tasks.workunit.client.1.smithi148.stdout:2/522: rename d5/d14/d22/d2b/l27 to d5/d14/d22/d2b/d13/d1d/la9 0 2024-07-03T15:43:51.562 INFO:tasks.workunit.client.1.smithi148.stdout:2/523: stat d5/d14/d22/d2b/f42 0 2024-07-03T15:43:51.562 INFO:tasks.workunit.client.1.smithi148.stdout:2/524: fdatasync d5/d14/d22/d2b/d13/d41/f4b 0 2024-07-03T15:43:51.562 INFO:tasks.workunit.client.1.smithi148.stdout:2/525: dread - d5/d14/f7a zero size 2024-07-03T15:43:51.562 INFO:tasks.workunit.client.1.smithi148.stdout:4/326: creat d0/d8/d22/d62/f6b x:0 0 0 2024-07-03T15:43:51.567 INFO:tasks.workunit.client.1.smithi148.stdout:2/526: link d5/lc d5/d14/d8f/laa 0 2024-07-03T15:43:51.570 INFO:tasks.workunit.client.1.smithi148.stdout:4/327: rename d0/f16 to d0/d8/d22/d2f/f6c 0 2024-07-03T15:43:51.571 INFO:tasks.workunit.client.1.smithi148.stdout:2/527: mknod d5/cab 0 2024-07-03T15:43:51.573 INFO:tasks.workunit.client.1.smithi148.stdout:4/328: creat d0/f6d x:0 0 0 2024-07-03T15:43:51.573 INFO:tasks.workunit.client.1.smithi148.stdout:9/641: sync 2024-07-03T15:43:51.574 INFO:tasks.workunit.client.1.smithi148.stdout:3/419: sync 2024-07-03T15:43:51.576 INFO:tasks.workunit.client.1.smithi148.stdout:4/329: rename d0/d8/d44/d58 to d0/d28/d6e 0 2024-07-03T15:43:51.583 INFO:tasks.workunit.client.1.smithi148.stdout:5/389: dwrite d7/d13/d23/d40/f56 [4194304,4194304] 0 2024-07-03T15:43:51.583 INFO:tasks.workunit.client.1.smithi148.stdout:5/390: write d7/d13/d23/f78 [1005027,51071] 0 2024-07-03T15:43:51.583 INFO:tasks.workunit.client.1.smithi148.stdout:9/642: rename d0/dd/fe to d0/dd/d25/d39/fcd 0 2024-07-03T15:43:51.583 INFO:tasks.workunit.client.1.smithi148.stdout:9/643: creat d0/fce x:0 0 0 2024-07-03T15:43:51.583 INFO:tasks.workunit.client.1.smithi148.stdout:5/391: symlink d7/d13/d23/d40/l81 0 2024-07-03T15:43:51.583 INFO:tasks.workunit.client.1.smithi148.stdout:5/392: fsync d7/da/df/f33 0 2024-07-03T15:43:51.585 INFO:tasks.workunit.client.1.smithi148.stdout:9/644: dread d0/d2f/fb5 [0,4194304] 0 2024-07-03T15:43:51.586 INFO:tasks.workunit.client.1.smithi148.stdout:9/645: truncate d0/dd/d11/f35 2272647 0 2024-07-03T15:43:51.586 INFO:tasks.workunit.client.1.smithi148.stdout:2/528: dwrite d5/d14/f7a [0,4194304] 0 2024-07-03T15:43:51.586 INFO:tasks.workunit.client.1.smithi148.stdout:2/529: creat d5/d14/d22/d2b/d68/fac x:0 0 0 2024-07-03T15:43:51.588 INFO:tasks.workunit.client.1.smithi148.stdout:4/330: rmdir d0/d8/d44 39 2024-07-03T15:43:51.588 INFO:tasks.workunit.client.1.smithi148.stdout:9/646: unlink d0/d49/f4e 0 2024-07-03T15:43:51.589 INFO:tasks.workunit.client.1.smithi148.stdout:3/420: dread dd/d33/d37/d68/d2e/f13 [4194304,4194304] 0 2024-07-03T15:43:51.589 INFO:tasks.workunit.client.1.smithi148.stdout:3/421: chown dd/d33/f39 1503 1 2024-07-03T15:43:51.589 INFO:tasks.workunit.client.1.smithi148.stdout:3/422: fdatasync dd/d33/d37/f46 0 2024-07-03T15:43:51.589 INFO:tasks.workunit.client.1.smithi148.stdout:3/423: stat dd/d33/d37/d68/d2e/f18 0 2024-07-03T15:43:51.589 INFO:tasks.workunit.client.1.smithi148.stdout:3/424: chown dd/d35/f4e 7615918 1 2024-07-03T15:43:51.589 INFO:tasks.workunit.client.1.smithi148.stdout:3/425: chown dd/d33/d37/d68/c24 4803 1 2024-07-03T15:43:51.589 INFO:tasks.workunit.client.1.smithi148.stdout:3/426: write dd/d33/f4b [1260530,33385] 0 2024-07-03T15:43:51.591 INFO:tasks.workunit.client.1.smithi148.stdout:2/530: dread d5/f9 [4194304,4194304] 0 2024-07-03T15:43:51.595 INFO:tasks.workunit.client.1.smithi148.stdout:3/427: unlink dd/d33/f50 0 2024-07-03T15:43:51.597 INFO:tasks.workunit.client.1.smithi148.stdout:3/428: creat dd/d35/d41/f84 x:0 0 0 2024-07-03T15:43:51.597 INFO:tasks.workunit.client.1.smithi148.stdout:4/331: dread d0/d8/f2d [0,4194304] 0 2024-07-03T15:43:51.598 INFO:tasks.workunit.client.1.smithi148.stdout:4/332: read d0/d8/f2d [2447261,19420] 0 2024-07-03T15:43:51.598 INFO:tasks.workunit.client.1.smithi148.stdout:3/429: creat dd/d35/d49/f85 x:0 0 0 2024-07-03T15:43:51.600 INFO:tasks.workunit.client.1.smithi148.stdout:4/333: mkdir d0/d8/d3a/d36/d6f 0 2024-07-03T15:43:51.601 INFO:tasks.workunit.client.1.smithi148.stdout:9/647: write d0/f6 [2771648,4534] 0 2024-07-03T15:43:51.601 INFO:tasks.workunit.client.1.smithi148.stdout:9/648: write d0/d49/d43/d19/d7a/daa/fb1 [529531,86873] 0 2024-07-03T15:43:51.601 INFO:tasks.workunit.client.1.smithi148.stdout:9/649: chown d0/dd/d29/l45 671736834 1 2024-07-03T15:43:51.603 INFO:tasks.workunit.client.1.smithi148.stdout:3/430: rename dd/d35/d49/l4a to dd/d33/d37/d68/d2e/d40/l86 0 2024-07-03T15:43:51.604 INFO:tasks.workunit.client.1.smithi148.stdout:9/650: mkdir d0/d18/dcf 0 2024-07-03T15:43:51.604 INFO:tasks.workunit.client.1.smithi148.stdout:4/334: mknod d0/d8/d19/d45/d49/c70 0 2024-07-03T15:43:51.606 INFO:tasks.workunit.client.1.smithi148.stdout:4/335: getdents d0/d8/d13 0 2024-07-03T15:43:51.609 INFO:tasks.workunit.client.1.smithi148.stdout:5/393: dwrite d7/d13/d23/d40/f56 [4194304,4194304] 0 2024-07-03T15:43:51.610 INFO:tasks.workunit.client.1.smithi148.stdout:9/651: dread d0/dd/d11/f79 [0,4194304] 0 2024-07-03T15:43:51.612 INFO:tasks.workunit.client.1.smithi148.stdout:2/531: dwrite d5/d14/d22/d2b/d68/fac [0,4194304] 0 2024-07-03T15:43:51.612 INFO:tasks.workunit.client.1.smithi148.stdout:5/394: dread d7/da/fb [0,4194304] 0 2024-07-03T15:43:51.612 INFO:tasks.workunit.client.1.smithi148.stdout:5/395: readlink l0 0 2024-07-03T15:43:51.615 INFO:tasks.workunit.client.1.smithi148.stdout:4/336: write d0/f17 [2140524,50027] 0 2024-07-03T15:43:51.618 INFO:tasks.workunit.client.1.smithi148.stdout:6/461: sync 2024-07-03T15:43:51.618 INFO:tasks.workunit.client.1.smithi148.stdout:1/319: sync 2024-07-03T15:43:51.618 INFO:tasks.workunit.client.1.smithi148.stdout:2/532: symlink d5/d14/d22/d2b/d13/d41/d7b/d6a/d80/lad 0 2024-07-03T15:43:51.620 INFO:tasks.workunit.client.1.smithi148.stdout:2/533: link d5/d14/d22/d2b/d13/d41/d7b/f3e d5/d14/d22/fae 0 2024-07-03T15:43:51.620 INFO:tasks.workunit.client.1.smithi148.stdout:2/534: readlink d5/d14/l45 0 2024-07-03T15:43:51.621 INFO:tasks.workunit.client.1.smithi148.stdout:2/535: truncate d5/d14/d22/d2b/d68/f7c 893059 0 2024-07-03T15:43:51.623 INFO:tasks.workunit.client.1.smithi148.stdout:2/536: rename d5/d14/d22/d2b/d13/d2d/f4e to d5/d7/d64/faf 0 2024-07-03T15:43:51.624 INFO:tasks.workunit.client.1.smithi148.stdout:2/537: creat d5/d14/d22/d60/d76/fb0 x:0 0 0 2024-07-03T15:43:51.626 INFO:tasks.workunit.client.1.smithi148.stdout:0/372: sync 2024-07-03T15:43:51.626 INFO:tasks.workunit.client.1.smithi148.stdout:8/349: sync 2024-07-03T15:43:51.626 INFO:tasks.workunit.client.1.smithi148.stdout:7/482: sync 2024-07-03T15:43:51.626 INFO:tasks.workunit.client.1.smithi148.stdout:8/350: readlink d4/l7c 0 2024-07-03T15:43:51.626 INFO:tasks.workunit.client.1.smithi148.stdout:8/351: dread - d4/d54/f63 zero size 2024-07-03T15:43:51.626 INFO:tasks.workunit.client.1.smithi148.stdout:0/373: creat da/d38/f7c x:0 0 0 2024-07-03T15:43:51.626 INFO:tasks.workunit.client.1.smithi148.stdout:0/374: dread - da/d44/d13/d14/f68 zero size 2024-07-03T15:43:51.627 INFO:tasks.workunit.client.1.smithi148.stdout:9/652: write d0/dd/d25/d6a/fb3 [4030682,44347] 0 2024-07-03T15:43:51.627 INFO:tasks.workunit.client.1.smithi148.stdout:3/431: dwrite dd/d33/f39 [0,4194304] 0 2024-07-03T15:43:51.627 INFO:tasks.workunit.client.1.smithi148.stdout:3/432: creat dd/d33/f87 x:0 0 0 2024-07-03T15:43:51.627 INFO:tasks.workunit.client.1.smithi148.stdout:3/433: read dd/d33/d37/d68/d2e/f18 [4121203,8291] 0 2024-07-03T15:43:51.627 INFO:tasks.workunit.client.1.smithi148.stdout:4/337: dread d0/d8/d22/d2f/f6c [0,4194304] 0 2024-07-03T15:43:51.634 INFO:tasks.workunit.client.1.smithi148.stdout:0/375: mkdir da/d44/d7d 0 2024-07-03T15:43:51.634 INFO:tasks.workunit.client.1.smithi148.stdout:0/376: read - da/d44/d13/d14/f68 zero size 2024-07-03T15:43:51.634 INFO:tasks.workunit.client.1.smithi148.stdout:9/653: symlink d0/d18/d24/d47/d4d/ld0 0 2024-07-03T15:43:51.634 INFO:tasks.workunit.client.1.smithi148.stdout:7/483: link d5/d16/d1c/d33/d4c/d84/l8d d5/d7d/l9a 0 2024-07-03T15:43:51.635 INFO:tasks.workunit.client.1.smithi148.stdout:8/352: dread d4/da/f28 [0,4194304] 0 2024-07-03T15:43:51.639 INFO:tasks.workunit.client.1.smithi148.stdout:0/377: mknod da/d44/d13/d14/c7e 0 2024-07-03T15:43:51.639 INFO:tasks.workunit.client.1.smithi148.stdout:7/484: creat d5/d16/d1c/d4a/d66/f9b x:0 0 0 2024-07-03T15:43:51.639 INFO:tasks.workunit.client.1.smithi148.stdout:7/485: write d5/d16/d1c/d33/d4c/d21/d47/f5d [4794508,38136] 0 2024-07-03T15:43:51.641 INFO:tasks.workunit.client.1.smithi148.stdout:8/353: creat d4/da/d14/d3a/f7e x:0 0 0 2024-07-03T15:43:51.643 INFO:tasks.workunit.client.1.smithi148.stdout:0/378: mknod da/d44/d4a/c7f 0 2024-07-03T15:43:51.645 INFO:tasks.workunit.client.1.smithi148.stdout:5/396: dwrite d7/da/df/d16/d1a/f50 [0,4194304] 0 2024-07-03T15:43:51.645 INFO:tasks.workunit.client.1.smithi148.stdout:5/397: write d7/da/df/d16/d2f/d44/d52/f76 [565595,20703] 0 2024-07-03T15:43:51.647 INFO:tasks.workunit.client.1.smithi148.stdout:7/486: mkdir d5/d16/d1c/d4a/d9c 0 2024-07-03T15:43:51.647 INFO:tasks.workunit.client.1.smithi148.stdout:8/354: creat d4/da/d51/d53/f7f x:0 0 0 2024-07-03T15:43:51.647 INFO:tasks.workunit.client.1.smithi148.stdout:8/355: fdatasync d4/da/d51/d18/d20/f2b 0 2024-07-03T15:43:51.650 INFO:tasks.workunit.client.1.smithi148.stdout:5/398: creat d7/da/df/d16/d2f/d44/d52/f82 x:0 0 0 2024-07-03T15:43:51.652 INFO:tasks.workunit.client.1.smithi148.stdout:7/487: rename d5/d16/d1c/f8e to d5/d16/d1c/d4a/d66/d93/f9d 0 2024-07-03T15:43:51.652 INFO:tasks.workunit.client.1.smithi148.stdout:7/488: fsync d5/d16/f4d 0 2024-07-03T15:43:51.652 INFO:tasks.workunit.client.1.smithi148.stdout:7/489: write d5/d16/d1c/d4a/d66/f9b [343359,33416] 0 2024-07-03T15:43:51.653 INFO:tasks.workunit.client.1.smithi148.stdout:8/356: symlink d4/da/d14/d3a/l80 0 2024-07-03T15:43:51.653 INFO:tasks.workunit.client.1.smithi148.stdout:8/357: fsync d4/da/d51/d18/d1f/f4b 0 2024-07-03T15:43:51.653 INFO:tasks.workunit.client.1.smithi148.stdout:8/358: stat d4/da/d51/d18/d20/f5a 0 2024-07-03T15:43:51.653 INFO:tasks.workunit.client.1.smithi148.stdout:8/359: dread - d4/da/d14/d17/f48 zero size 2024-07-03T15:43:51.654 INFO:tasks.workunit.client.1.smithi148.stdout:5/399: mkdir d7/da/df/d83 0 2024-07-03T15:43:51.654 INFO:tasks.workunit.client.1.smithi148.stdout:5/400: chown d7/da/d21 17 1 2024-07-03T15:43:51.654 INFO:tasks.workunit.client.1.smithi148.stdout:5/401: dread - d7/da/f74 zero size 2024-07-03T15:43:51.655 INFO:tasks.workunit.client.1.smithi148.stdout:5/402: creat d7/da/df/d16/d2f/d44/f84 x:0 0 0 2024-07-03T15:43:51.657 INFO:tasks.workunit.client.1.smithi148.stdout:7/490: mkdir d5/d16/d1c/d33/d4c/d84/d98/d9e 0 2024-07-03T15:43:51.657 INFO:tasks.workunit.client.1.smithi148.stdout:8/360: mknod d4/da/d50/c81 0 2024-07-03T15:43:51.658 INFO:tasks.workunit.client.1.smithi148.stdout:8/361: mkdir d4/da/d51/d18/d20/d82 0 2024-07-03T15:43:51.660 INFO:tasks.workunit.client.1.smithi148.stdout:7/491: creat d5/f9f x:0 0 0 2024-07-03T15:43:51.661 INFO:tasks.workunit.client.1.smithi148.stdout:8/362: creat d4/da/d47/f83 x:0 0 0 2024-07-03T15:43:51.661 INFO:tasks.workunit.client.1.smithi148.stdout:8/363: chown d4/da/d51/d18/d1f/f4b 12 1 2024-07-03T15:43:51.668 INFO:tasks.workunit.client.1.smithi148.stdout:4/338: dwrite d0/d8/d22/d2f/f6c [0,4194304] 0 2024-07-03T15:43:51.683 INFO:tasks.workunit.client.1.smithi148.stdout:8/364: dwrite d4/da/d51/d18/d20/f2b [0,4194304] 0 2024-07-03T15:43:51.693 INFO:tasks.workunit.client.1.smithi148.stdout:1/320: dwrite d0/d2/d2e/d39/d42/f48 [4194304,4194304] 0 2024-07-03T15:43:51.694 INFO:tasks.workunit.client.1.smithi148.stdout:1/321: dread - d0/d2/d2e/d39/d41/f65 zero size 2024-07-03T15:43:51.694 INFO:tasks.workunit.client.1.smithi148.stdout:6/462: dwrite d3/d1d/d28/f69 [0,4194304] 0 2024-07-03T15:43:51.694 INFO:tasks.workunit.client.1.smithi148.stdout:6/463: creat d3/d50/f9a x:0 0 0 2024-07-03T15:43:51.694 INFO:tasks.workunit.client.1.smithi148.stdout:2/538: dwrite d5/d14/d22/d2b/d68/f7c [0,4194304] 0 2024-07-03T15:43:51.694 INFO:tasks.workunit.client.1.smithi148.stdout:3/434: dwrite dd/f44 [0,4194304] 0 2024-07-03T15:43:51.695 INFO:tasks.workunit.client.1.smithi148.stdout:1/322: rename d0/d2/d6/d14/c2c to d0/d2/d2e/c67 0 2024-07-03T15:43:51.697 INFO:tasks.workunit.client.1.smithi148.stdout:0/379: dwrite da/d44/d13/d14/f54 [0,4194304] 0 2024-07-03T15:43:51.697 INFO:tasks.workunit.client.1.smithi148.stdout:9/654: dwrite d0/d49/f81 [0,4194304] 0 2024-07-03T15:43:51.699 INFO:tasks.workunit.client.1.smithi148.stdout:7/492: dwrite d5/d16/d1c/d33/d4c/f12 [0,4194304] 0 2024-07-03T15:43:51.702 INFO:tasks.workunit.client.1.smithi148.stdout:6/464: symlink d3/d1d/d28/d53/l9b 0 2024-07-03T15:43:51.702 INFO:tasks.workunit.client.1.smithi148.stdout:6/465: read d3/d1d/d28/d3e/f64 [559791,42250] 0 2024-07-03T15:43:51.704 INFO:tasks.workunit.client.1.smithi148.stdout:2/539: write d5/f1f [3826523,42874] 0 2024-07-03T15:43:51.704 INFO:tasks.workunit.client.1.smithi148.stdout:2/540: chown d5/d14/d22/d2b/c6f 14840853 1 2024-07-03T15:43:51.704 INFO:tasks.workunit.client.1.smithi148.stdout:1/323: mknod d0/d2/d6/c68 0 2024-07-03T15:43:51.705 INFO:tasks.workunit.client.1.smithi148.stdout:5/403: dwrite d7/d13/d23/f3b [4194304,4194304] 0 2024-07-03T15:43:51.705 INFO:tasks.workunit.client.1.smithi148.stdout:5/404: truncate d7/d13/d54/f7d 23470 0 2024-07-03T15:43:51.705 INFO:tasks.workunit.client.1.smithi148.stdout:5/405: creat d7/d13/d23/f85 x:0 0 0 2024-07-03T15:43:51.708 INFO:tasks.workunit.client.1.smithi148.stdout:7/493: dread d5/d16/d1c/d23/f2b [0,4194304] 0 2024-07-03T15:43:51.708 INFO:tasks.workunit.client.1.smithi148.stdout:0/380: truncate da/d44/d13/f18 270501 0 2024-07-03T15:43:51.709 INFO:tasks.workunit.client.1.smithi148.stdout:9/655: mknod d0/dd/d9f/cd1 0 2024-07-03T15:43:51.709 INFO:tasks.workunit.client.1.smithi148.stdout:9/656: dread - d0/d49/d43/d19/d7a/fb6 zero size 2024-07-03T15:43:51.715 INFO:tasks.workunit.client.1.smithi148.stdout:6/466: mkdir d3/d1d/d28/d7f/d85/d9c 0 2024-07-03T15:43:51.717 INFO:tasks.workunit.client.1.smithi148.stdout:2/541: dread d5/d14/d22/d2b/d13/d41/d7b/d6a/f77 [0,4194304] 0 2024-07-03T15:43:51.718 INFO:tasks.workunit.client.1.smithi148.stdout:1/324: link d0/d2/f10 d0/d2/d2e/d39/d41/f69 0 2024-07-03T15:43:51.718 INFO:tasks.workunit.client.1.smithi148.stdout:1/325: creat d0/d2/d6/d14/f6a x:0 0 0 2024-07-03T15:43:51.723 INFO:tasks.workunit.client.1.smithi148.stdout:4/339: dwrite d0/f17 [0,4194304] 0 2024-07-03T15:43:51.723 INFO:tasks.workunit.client.1.smithi148.stdout:4/340: fsync d0/d8/d3a/f66 0 2024-07-03T15:43:51.723 INFO:tasks.workunit.client.1.smithi148.stdout:4/341: fdatasync d0/d8/d3a/f47 0 2024-07-03T15:43:51.723 INFO:tasks.workunit.client.1.smithi148.stdout:4/342: fdatasync d0/d8/f10 0 2024-07-03T15:43:51.723 INFO:tasks.workunit.client.1.smithi148.stdout:3/435: dwrite dd/d35/f3f [0,4194304] 0 2024-07-03T15:43:51.723 INFO:tasks.workunit.client.1.smithi148.stdout:3/436: stat dd/d35/d81 0 2024-07-03T15:43:51.723 INFO:tasks.workunit.client.1.smithi148.stdout:3/437: creat dd/d33/f88 x:0 0 0 2024-07-03T15:43:51.723 INFO:tasks.workunit.client.1.smithi148.stdout:3/438: fsync dd/d33/d52/f58 0 2024-07-03T15:43:51.724 INFO:tasks.workunit.client.1.smithi148.stdout:5/406: unlink d7/da/df/l7f 0 2024-07-03T15:43:51.724 INFO:tasks.workunit.client.1.smithi148.stdout:5/407: write d7/da/df/d16/f7e [276045,21796] 0 2024-07-03T15:43:51.724 INFO:tasks.workunit.client.1.smithi148.stdout:5/408: readlink d7/da/df/d16/d2f/d44/l72 0 2024-07-03T15:43:51.724 INFO:tasks.workunit.client.1.smithi148.stdout:5/409: chown d7/da/df/d16/d1a/f34 660678 1 2024-07-03T15:43:51.724 INFO:tasks.workunit.client.1.smithi148.stdout:0/381: unlink da/d44/d13/d14/l33 0 2024-07-03T15:43:51.724 INFO:tasks.workunit.client.1.smithi148.stdout:6/467: dread d3/d1d/d28/d31/f4b [0,4194304] 0 2024-07-03T15:43:51.725 INFO:tasks.workunit.client.1.smithi148.stdout:7/494: symlink d5/d7d/la0 0 2024-07-03T15:43:51.726 INFO:tasks.workunit.client.1.smithi148.stdout:9/657: creat d0/dd/d25/d59/fd2 x:0 0 0 2024-07-03T15:43:51.727 INFO:tasks.workunit.client.1.smithi148.stdout:2/542: mknod d5/d14/d22/d2b/d13/cb1 0 2024-07-03T15:43:51.729 INFO:tasks.workunit.client.1.smithi148.stdout:1/326: mknod d0/d2/d11/d1e/c6b 0 2024-07-03T15:43:51.733 INFO:tasks.workunit.client.1.smithi148.stdout:3/439: mkdir dd/d35/d41/d89 0 2024-07-03T15:43:51.735 INFO:tasks.workunit.client.1.smithi148.stdout:5/410: mkdir d7/da/df/d16/d2f/d44/d52/d86 0 2024-07-03T15:43:51.735 INFO:tasks.workunit.client.1.smithi148.stdout:5/411: write d7/d13/d23/d40/f5c [216672,77326] 0 2024-07-03T15:43:51.737 INFO:tasks.workunit.client.1.smithi148.stdout:6/468: mknod d3/d50/c9d 0 2024-07-03T15:43:51.743 INFO:tasks.workunit.client.1.smithi148.stdout:9/658: creat d0/dd/d25/d37/d40/da3/fd3 x:0 0 0 2024-07-03T15:43:51.744 INFO:tasks.workunit.client.1.smithi148.stdout:1/327: symlink d0/d2/d11/l6c 0 2024-07-03T15:43:51.744 INFO:tasks.workunit.client.1.smithi148.stdout:1/328: truncate d0/d2/d11/d1e/f66 624208 0 2024-07-03T15:43:51.744 INFO:tasks.workunit.client.1.smithi148.stdout:1/329: read - d0/de/d44/f5e zero size 2024-07-03T15:43:51.744 INFO:tasks.workunit.client.1.smithi148.stdout:5/412: dread d7/da/df/d16/d1a/f34 [0,4194304] 0 2024-07-03T15:43:51.745 INFO:tasks.workunit.client.1.smithi148.stdout:3/440: symlink dd/d35/d49/l8a 0 2024-07-03T15:43:51.746 INFO:tasks.workunit.client.1.smithi148.stdout:6/469: symlink d3/d1d/d28/d53/d6d/l9e 0 2024-07-03T15:43:51.747 INFO:tasks.workunit.client.1.smithi148.stdout:9/659: symlink d0/ld4 0 2024-07-03T15:43:51.749 INFO:tasks.workunit.client.1.smithi148.stdout:1/330: unlink d0/d2/d11/d1e/f51 0 2024-07-03T15:43:51.749 INFO:tasks.workunit.client.1.smithi148.stdout:1/331: chown d0/d2/d11/l6c 208 1 2024-07-03T15:43:51.749 INFO:tasks.workunit.client.1.smithi148.stdout:1/332: fdatasync d0/d2/d2e/d39/d42/d62/f4f 0 2024-07-03T15:43:51.749 INFO:tasks.workunit.client.1.smithi148.stdout:1/333: dread - d0/d2/d6/f5a zero size 2024-07-03T15:43:51.750 INFO:tasks.workunit.client.1.smithi148.stdout:5/413: truncate d7/da/df/d1d/f51 990072 0 2024-07-03T15:43:51.750 INFO:tasks.workunit.client.1.smithi148.stdout:5/414: chown d7/da/df/d16/d1a/c5a 195677638 1 2024-07-03T15:43:51.752 INFO:tasks.workunit.client.1.smithi148.stdout:3/441: mkdir dd/d35/d49/d8b 0 2024-07-03T15:43:51.753 INFO:tasks.workunit.client.1.smithi148.stdout:1/334: symlink d0/d2/d11/d1e/l6d 0 2024-07-03T15:43:51.759 INFO:tasks.workunit.client.1.smithi148.stdout:6/470: write d3/d1d/d28/d31/d61/f7a [1885290,60550] 0 2024-07-03T15:43:51.760 INFO:tasks.workunit.client.1.smithi148.stdout:7/495: dwrite d5/d16/d1c/d23/f2b [0,4194304] 0 2024-07-03T15:43:51.761 INFO:tasks.workunit.client.1.smithi148.stdout:4/343: dwrite d0/d28/f63 [0,4194304] 0 2024-07-03T15:43:51.761 INFO:tasks.workunit.client.1.smithi148.stdout:0/382: dwrite da/d58/f79 [0,4194304] 0 2024-07-03T15:43:51.761 INFO:tasks.workunit.client.1.smithi148.stdout:0/383: fsync da/d44/d13/d1d/d3e/f6e 0 2024-07-03T15:43:51.762 INFO:tasks.workunit.client.1.smithi148.stdout:2/543: dwrite d5/f40 [0,4194304] 0 2024-07-03T15:43:51.763 INFO:tasks.workunit.client.1.smithi148.stdout:6/471: mknod d3/d1d/d28/d3e/c9f 0 2024-07-03T15:43:51.765 INFO:tasks.workunit.client.1.smithi148.stdout:7/496: mknod d5/d16/d1c/d4a/d9c/ca1 0 2024-07-03T15:43:51.766 INFO:tasks.workunit.client.1.smithi148.stdout:3/442: write dd/d33/d37/f46 [2752475,15439] 0 2024-07-03T15:43:51.766 INFO:tasks.workunit.client.1.smithi148.stdout:3/443: truncate dd/f2b 5022364 0 2024-07-03T15:43:51.768 INFO:tasks.workunit.client.1.smithi148.stdout:4/344: mknod d0/d8/d13/d48/c71 0 2024-07-03T15:43:51.769 INFO:tasks.workunit.client.1.smithi148.stdout:7/497: dread d5/d16/d1c/d33/d4c/d21/f32 [0,4194304] 0 2024-07-03T15:43:51.771 INFO:tasks.workunit.client.1.smithi148.stdout:9/660: dwrite d0/dd/d25/d6a/f82 [0,4194304] 0 2024-07-03T15:43:51.771 INFO:tasks.workunit.client.1.smithi148.stdout:9/661: write d0/dd/d25/d37/d40/d72/f8d [1202091,21615] 0 2024-07-03T15:43:51.772 INFO:tasks.workunit.client.1.smithi148.stdout:0/384: link da/d38/c3f da/d17/d39/c80 0 2024-07-03T15:43:51.772 INFO:tasks.workunit.client.1.smithi148.stdout:0/385: fdatasync da/d44/d13/d14/f61 0 2024-07-03T15:43:51.775 INFO:tasks.workunit.client.1.smithi148.stdout:3/444: mkdir dd/d33/d37/d68/d2e/d40/d8c 0 2024-07-03T15:43:51.775 INFO:tasks.workunit.client.1.smithi148.stdout:3/445: dread - dd/d35/f4e zero size 2024-07-03T15:43:51.776 INFO:tasks.workunit.client.1.smithi148.stdout:5/415: dwrite d7/da/df/d1d/f2e [0,4194304] 0 2024-07-03T15:43:51.777 INFO:tasks.workunit.client.1.smithi148.stdout:7/498: dread d5/f3b [4194304,4194304] 0 2024-07-03T15:43:51.778 INFO:tasks.workunit.client.1.smithi148.stdout:1/335: dwrite d0/de/d2f/f55 [0,4194304] 0 2024-07-03T15:43:51.779 INFO:tasks.workunit.client.1.smithi148.stdout:9/662: mknod d0/dd/cd5 0 2024-07-03T15:43:51.779 INFO:tasks.workunit.client.1.smithi148.stdout:0/386: mknod da/d44/d13/d14/d50/c81 0 2024-07-03T15:43:51.779 INFO:tasks.workunit.client.1.smithi148.stdout:0/387: fsync da/f28 0 2024-07-03T15:43:51.779 INFO:tasks.workunit.client.1.smithi148.stdout:4/345: dread d0/d8/d22/d2f/d41/f4d [0,4194304] 0 2024-07-03T15:43:51.780 INFO:tasks.workunit.client.1.smithi148.stdout:2/544: getdents d5/d14/d22/d2b/d13/d2d 0 2024-07-03T15:43:51.784 INFO:tasks.workunit.client.1.smithi148.stdout:5/416: creat d7/da/df/d1d/f87 x:0 0 0 2024-07-03T15:43:51.786 INFO:tasks.workunit.client.1.smithi148.stdout:1/336: unlink d0/c4a 0 2024-07-03T15:43:51.790 INFO:tasks.workunit.client.1.smithi148.stdout:3/446: read dd/d33/d37/d68/f5c [537107,124730] 0 2024-07-03T15:43:51.790 INFO:tasks.workunit.client.1.smithi148.stdout:0/388: unlink da/d44/d24/f29 0 2024-07-03T15:43:51.790 INFO:tasks.workunit.client.1.smithi148.stdout:0/389: fsync da/d44/d13/d14/d35/f5e 0 2024-07-03T15:43:51.790 INFO:tasks.workunit.client.1.smithi148.stdout:2/545: getdents d5/d14/d22/d2b/d13 0 2024-07-03T15:43:51.790 INFO:tasks.workunit.client.1.smithi148.stdout:2/546: chown d5/d14/d22/d2b/c6f 1495 1 2024-07-03T15:43:51.790 INFO:tasks.workunit.client.1.smithi148.stdout:7/499: dread d5/d16/d1c/d59/f76 [0,4194304] 0 2024-07-03T15:43:51.792 INFO:tasks.workunit.client.1.smithi148.stdout:5/417: symlink d7/da/df/d16/d6a/l88 0 2024-07-03T15:43:51.795 INFO:tasks.workunit.client.1.smithi148.stdout:1/337: getdents d0/d2/d2e/d39/d42/d62 0 2024-07-03T15:43:51.795 INFO:tasks.workunit.client.1.smithi148.stdout:0/390: mknod da/d44/d13/d14/c82 0 2024-07-03T15:43:51.797 INFO:tasks.workunit.client.1.smithi148.stdout:4/346: write d0/d8/f10 [3501131,113124] 0 2024-07-03T15:43:51.797 INFO:tasks.workunit.client.1.smithi148.stdout:5/418: read d7/d13/f2a [138177,4852] 0 2024-07-03T15:43:51.797 INFO:tasks.workunit.client.1.smithi148.stdout:0/391: getdents da/d44/d13 0 2024-07-03T15:43:51.797 INFO:tasks.workunit.client.1.smithi148.stdout:0/392: stat da/d38/f4e 0 2024-07-03T15:43:51.797 INFO:tasks.workunit.client.1.smithi148.stdout:5/419: creat d7/da/df/d16/d2f/d44/d52/f89 x:0 0 0 2024-07-03T15:43:51.797 INFO:tasks.workunit.client.1.smithi148.stdout:5/420: fsync d7/da/df/d16/d2f/d44/f70 0 2024-07-03T15:43:51.797 INFO:tasks.workunit.client.1.smithi148.stdout:5/421: creat d7/da/df/d16/f8a x:0 0 0 2024-07-03T15:43:51.799 INFO:tasks.workunit.client.1.smithi148.stdout:5/422: read d7/da/df/f71 [368257,97075] 0 2024-07-03T15:43:51.808 INFO:tasks.workunit.client.1.smithi148.stdout:0/393: dread da/d17/d39/f45 [0,4194304] 0 2024-07-03T15:43:51.808 INFO:tasks.workunit.client.1.smithi148.stdout:0/394: truncate da/d36/d5b/f72 1493791 0 2024-07-03T15:43:51.808 INFO:tasks.workunit.client.1.smithi148.stdout:0/395: read - da/d36/f63 zero size 2024-07-03T15:43:51.808 INFO:tasks.workunit.client.1.smithi148.stdout:0/396: creat da/d44/d13/d14/d50/f83 x:0 0 0 2024-07-03T15:43:51.808 INFO:tasks.workunit.client.1.smithi148.stdout:5/423: creat d7/da/df/d16/d2f/f8b x:0 0 0 2024-07-03T15:43:51.810 INFO:tasks.workunit.client.1.smithi148.stdout:0/397: truncate da/d17/d2a/f2d 1421266 0 2024-07-03T15:43:51.811 INFO:tasks.workunit.client.1.smithi148.stdout:4/347: dread d0/d8/d22/d2f/f33 [0,4194304] 0 2024-07-03T15:43:51.811 INFO:tasks.workunit.client.1.smithi148.stdout:1/338: dread d0/d2/d11/f47 [0,4194304] 0 2024-07-03T15:43:51.812 INFO:tasks.workunit.client.1.smithi148.stdout:0/398: mkdir da/d44/d24/d84 0 2024-07-03T15:43:51.814 INFO:tasks.workunit.client.1.smithi148.stdout:1/339: creat d0/d2/d2e/d39/d42/d62/f6e x:0 0 0 2024-07-03T15:43:51.817 INFO:tasks.workunit.client.1.smithi148.stdout:0/399: mkdir da/d44/d13/d1d/d3e/d85 0 2024-07-03T15:43:51.817 INFO:tasks.workunit.client.1.smithi148.stdout:0/400: fdatasync da/d44/d13/d14/f32 0 2024-07-03T15:43:51.819 INFO:tasks.workunit.client.1.smithi148.stdout:1/340: unlink d0/d2/fd 0 2024-07-03T15:43:51.820 INFO:tasks.workunit.client.1.smithi148.stdout:0/401: symlink da/d44/d13/d5a/l86 0 2024-07-03T15:43:51.820 INFO:tasks.workunit.client.1.smithi148.stdout:0/402: write da/d17/d2a/f4b [540994,43223] 0 2024-07-03T15:43:51.821 INFO:tasks.workunit.client.1.smithi148.stdout:0/403: rename da/d44/d13/d14/f54 to da/d17/d2a/d2f/f87 0 2024-07-03T15:43:51.822 INFO:tasks.workunit.client.1.smithi148.stdout:0/404: mkdir da/d38/d88 0 2024-07-03T15:43:51.822 INFO:tasks.workunit.client.1.smithi148.stdout:0/405: chown da/d44/d13/d1d 114794667 1 2024-07-03T15:43:51.823 INFO:tasks.workunit.client.1.smithi148.stdout:5/424: write d7/d13/d23/d40/f56 [5861224,23692] 0 2024-07-03T15:43:51.823 INFO:tasks.workunit.client.1.smithi148.stdout:5/425: write d7/da/df/d16/d2f/d44/f84 [264075,12812] 0 2024-07-03T15:43:51.823 INFO:tasks.workunit.client.1.smithi148.stdout:0/406: link da/d36/f63 da/d44/d24/f89 0 2024-07-03T15:43:51.823 INFO:tasks.workunit.client.1.smithi148.stdout:0/407: dread - da/d44/d13/d14/d50/f5d zero size 2024-07-03T15:43:51.824 INFO:tasks.workunit.client.1.smithi148.stdout:5/426: truncate d7/da/df/d16/d2f/d44/f69 3271173 0 2024-07-03T15:43:51.824 INFO:tasks.workunit.client.1.smithi148.stdout:5/427: write d7/da/df/f33 [4928631,23302] 0 2024-07-03T15:43:51.824 INFO:tasks.workunit.client.1.smithi148.stdout:9/663: dwrite d0/d49/d43/d19/f1e [0,4194304] 0 2024-07-03T15:43:51.824 INFO:tasks.workunit.client.1.smithi148.stdout:5/428: read d7/d13/f2a [84483,100170] 0 2024-07-03T15:43:51.825 INFO:tasks.workunit.client.1.smithi148.stdout:5/429: readlink d7/da/l14 0 2024-07-03T15:43:51.825 INFO:tasks.workunit.client.1.smithi148.stdout:0/408: mkdir da/d44/d13/d14/d35/d8a 0 2024-07-03T15:43:51.825 INFO:tasks.workunit.client.1.smithi148.stdout:7/500: dwrite d5/d16/d1c/d33/d4c/d21/d31/d7f/d97/d65/f95 [0,4194304] 0 2024-07-03T15:43:51.825 INFO:tasks.workunit.client.1.smithi148.stdout:6/472: dwrite d3/d1d/d28/d31/d61/f7a [0,4194304] 0 2024-07-03T15:43:51.825 INFO:tasks.workunit.client.1.smithi148.stdout:6/473: chown d3/d1d/d28/d53/f80 11 1 2024-07-03T15:43:51.826 INFO:tasks.workunit.client.1.smithi148.stdout:9/664: rename d0/d2f/f7f to d0/dd/d25/d37/d40/d72/db8/fd6 0 2024-07-03T15:43:51.826 INFO:tasks.workunit.client.1.smithi148.stdout:3/447: dwrite dd/f72 [0,4194304] 0 2024-07-03T15:43:51.826 INFO:tasks.workunit.client.1.smithi148.stdout:5/430: mknod d7/d13/d23/d40/c8c 0 2024-07-03T15:43:51.827 INFO:tasks.workunit.client.1.smithi148.stdout:0/409: creat da/d17/f8b x:0 0 0 2024-07-03T15:43:51.829 INFO:tasks.workunit.client.1.smithi148.stdout:7/501: rename d5/d7d/la0 to d5/d16/d1c/la2 0 2024-07-03T15:43:51.830 INFO:tasks.workunit.client.1.smithi148.stdout:7/502: chown d5/f58 381361 1 2024-07-03T15:43:51.830 INFO:tasks.workunit.client.1.smithi148.stdout:6/474: dread d3/d1d/d28/d31/f4b [0,4194304] 0 2024-07-03T15:43:51.830 INFO:tasks.workunit.client.1.smithi148.stdout:2/547: dwrite d5/d14/d22/d2b/d13/d1d/f43 [4194304,4194304] 0 2024-07-03T15:43:51.831 INFO:tasks.workunit.client.1.smithi148.stdout:3/448: mkdir dd/d33/d37/d8d 0 2024-07-03T15:43:51.831 INFO:tasks.workunit.client.1.smithi148.stdout:5/431: symlink d7/da/d21/l8d 0 2024-07-03T15:43:51.832 INFO:tasks.workunit.client.1.smithi148.stdout:0/410: rename da/d17/d39 to da/d44/d7d/d8c 0 2024-07-03T15:43:51.832 INFO:tasks.workunit.client.1.smithi148.stdout:0/411: chown da/d44/d13/d14/d50 56547 1 2024-07-03T15:43:51.832 INFO:tasks.workunit.client.1.smithi148.stdout:7/503: dread d5/d16/d1c/d33/d4c/d21/d39/f77 [0,4194304] 0 2024-07-03T15:43:51.832 INFO:tasks.workunit.client.1.smithi148.stdout:4/348: dwrite d0/f4 [0,4194304] 0 2024-07-03T15:43:51.832 INFO:tasks.workunit.client.1.smithi148.stdout:7/504: chown d5/d16/d1c/d33/c6f 2055 1 2024-07-03T15:43:51.833 INFO:tasks.workunit.client.1.smithi148.stdout:9/665: dread d0/dd/d25/f3c [0,4194304] 0 2024-07-03T15:43:51.836 INFO:tasks.workunit.client.1.smithi148.stdout:1/341: fsync d0/d2/d2e/d39/d42/d62/f6e 0 2024-07-03T15:43:51.836 INFO:tasks.workunit.client.1.smithi148.stdout:1/342: creat d0/d2/d11/d1e/f6f x:0 0 0 2024-07-03T15:43:51.836 INFO:tasks.workunit.client.1.smithi148.stdout:1/343: creat d0/d2/d11/d3a/f70 x:0 0 0 2024-07-03T15:43:51.836 INFO:tasks.workunit.client.1.smithi148.stdout:1/344: fsync d0/d2/d6/f5a 0 2024-07-03T15:43:51.836 INFO:tasks.workunit.client.1.smithi148.stdout:1/345: creat d0/d2/d2e/d39/d42/d62/f71 x:0 0 0 2024-07-03T15:43:51.838 INFO:tasks.workunit.client.1.smithi148.stdout:1/346: write d0/f45 [400512,84802] 0 2024-07-03T15:43:51.838 INFO:tasks.workunit.client.1.smithi148.stdout:1/347: fsync d0/d2/d6/d14/f58 0 2024-07-03T15:43:51.851 INFO:tasks.workunit.client.1.smithi148.stdout:2/548: mkdir d5/d7/d64/db2 0 2024-07-03T15:43:51.851 INFO:tasks.workunit.client.1.smithi148.stdout:2/549: write d5/d14/f58 [3034967,48591] 0 2024-07-03T15:43:51.851 INFO:tasks.workunit.client.1.smithi148.stdout:2/550: fdatasync d5/d14/d22/d2b/d13/d41/f8c 0 2024-07-03T15:43:51.851 INFO:tasks.workunit.client.1.smithi148.stdout:6/475: symlink d3/d1d/d28/d53/la0 0 2024-07-03T15:43:51.851 INFO:tasks.workunit.client.1.smithi148.stdout:6/476: dread - d3/d1d/d28/d31/f98 zero size 2024-07-03T15:43:51.852 INFO:tasks.workunit.client.1.smithi148.stdout:3/449: symlink dd/d33/d37/d68/d2e/d40/d77/l8e 0 2024-07-03T15:43:51.854 INFO:tasks.workunit.client.1.smithi148.stdout:2/551: dread d5/d14/d22/d60/f6c [0,4194304] 0 2024-07-03T15:43:51.856 INFO:tasks.workunit.client.1.smithi148.stdout:7/505: creat d5/d16/d1c/d33/d4c/d21/d31/d7f/d97/d65/fa3 x:0 0 0 2024-07-03T15:43:51.857 INFO:tasks.workunit.client.1.smithi148.stdout:9/666: mknod d0/d18/d24/d60/cd7 0 2024-07-03T15:43:51.860 INFO:tasks.workunit.client.1.smithi148.stdout:4/349: dwrite d0/d8/d22/d62/f6b [0,4194304] 0 2024-07-03T15:43:51.860 INFO:tasks.workunit.client.1.smithi148.stdout:4/350: dread - d0/f6d zero size 2024-07-03T15:43:51.860 INFO:tasks.workunit.client.1.smithi148.stdout:0/412: dwrite da/d17/f76 [0,4194304] 0 2024-07-03T15:43:51.861 INFO:tasks.workunit.client.1.smithi148.stdout:1/348: write d0/d2/f10 [3776314,99817] 0 2024-07-03T15:43:51.861 INFO:tasks.workunit.client.1.smithi148.stdout:7/506: write d5/d16/d1c/d33/f67 [1890649,76089] 0 2024-07-03T15:43:51.861 INFO:tasks.workunit.client.1.smithi148.stdout:7/507: chown d5/d16/d1c/d33/d4c/d21/d39/f3e 1 1 2024-07-03T15:43:51.861 INFO:tasks.workunit.client.1.smithi148.stdout:6/477: mknod d3/d1d/d28/d7f/ca1 0 2024-07-03T15:43:51.862 INFO:tasks.workunit.client.1.smithi148.stdout:6/478: fdatasync d3/d1d/d28/d31/d61/f7a 0 2024-07-03T15:43:51.862 INFO:tasks.workunit.client.1.smithi148.stdout:3/450: mkdir dd/d33/d37/d68/d2e/d40/d77/d8f 0 2024-07-03T15:43:51.862 INFO:tasks.workunit.client.1.smithi148.stdout:3/451: write dd/d33/d37/f53 [4724047,52472] 0 2024-07-03T15:43:51.863 INFO:tasks.workunit.client.1.smithi148.stdout:3/452: chown dd/d33/d37/d68/f5c 70294 1 2024-07-03T15:43:51.863 INFO:tasks.workunit.client.1.smithi148.stdout:5/432: rmdir d7/da/d21 39 2024-07-03T15:43:51.863 INFO:tasks.workunit.client.1.smithi148.stdout:5/433: write d7/da/df/d16/d2f/d44/d52/f6c [34772,48564] 0 2024-07-03T15:43:51.864 INFO:tasks.workunit.client.1.smithi148.stdout:2/552: mkdir d5/d14/d22/d60/d76/db3 0 2024-07-03T15:43:51.864 INFO:tasks.workunit.client.1.smithi148.stdout:2/553: truncate d5/d14/d22/d2b/f42 1707630 0 2024-07-03T15:43:51.864 INFO:tasks.workunit.client.1.smithi148.stdout:2/554: write d5/d14/d22/d60/f91 [59648,101397] 0 2024-07-03T15:43:51.865 INFO:tasks.workunit.client.1.smithi148.stdout:9/667: truncate d0/d18/d24/f91 3117318 0 2024-07-03T15:43:51.871 INFO:tasks.workunit.client.1.smithi148.stdout:4/351: write d0/f6 [2044070,40164] 0 2024-07-03T15:43:51.871 INFO:tasks.workunit.client.1.smithi148.stdout:1/349: truncate d0/d2/d11/d1e/f3c 8183908 0 2024-07-03T15:43:51.871 INFO:tasks.workunit.client.1.smithi148.stdout:0/413: link da/d44/d13/d5a/l78 da/d44/d7d/d8c/l8d 0 2024-07-03T15:43:51.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:51 smithi028 ceph-mon[29148]: pgmap v238: 65 pgs: 65 active+clean; 652 MiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 49 MiB/s rd, 61 MiB/s wr, 659 op/s 2024-07-03T15:43:51.872 INFO:tasks.workunit.client.1.smithi148.stdout:3/453: dread dd/f34 [0,4194304] 0 2024-07-03T15:43:51.872 INFO:tasks.workunit.client.1.smithi148.stdout:3/454: chown dd/d33/d37/d68/d2e/d61/f76 8179 1 2024-07-03T15:43:51.873 INFO:tasks.workunit.client.1.smithi148.stdout:7/508: mkdir d5/d16/d1c/d33/d4c/d84/da4 0 2024-07-03T15:43:51.873 INFO:tasks.workunit.client.1.smithi148.stdout:7/509: read - d5/f9f zero size 2024-07-03T15:43:51.876 INFO:tasks.workunit.client.1.smithi148.stdout:6/479: rename d3/d50/d77/d8e to d3/d1d/d28/d7f/d85/da2 0 2024-07-03T15:43:51.876 INFO:tasks.workunit.client.1.smithi148.stdout:6/480: stat d3/d1d/d28/d31/d61/f71 0 2024-07-03T15:43:51.876 INFO:tasks.workunit.client.1.smithi148.stdout:6/481: chown d3/d1d/d28/d31/d61/c82 30833 1 2024-07-03T15:43:51.878 INFO:tasks.workunit.client.1.smithi148.stdout:8/365: sync 2024-07-03T15:43:51.878 INFO:tasks.workunit.client.1.smithi148.stdout:5/434: symlink d7/l8e 0 2024-07-03T15:43:51.879 INFO:tasks.workunit.client.1.smithi148.stdout:1/350: mkdir d0/de/d2f/d72 0 2024-07-03T15:43:51.879 INFO:tasks.workunit.client.1.smithi148.stdout:1/351: dread - d0/d2/d2e/d39/d41/f65 zero size 2024-07-03T15:43:51.881 INFO:tasks.workunit.client.1.smithi148.stdout:3/455: truncate dd/f34 3938217 0 2024-07-03T15:43:51.882 INFO:tasks.workunit.client.1.smithi148.stdout:3/456: write dd/f36 [3037744,25509] 0 2024-07-03T15:43:51.882 INFO:tasks.workunit.client.1.smithi148.stdout:7/510: truncate d5/d16/d1c/d23/f68 3085609 0 2024-07-03T15:43:51.885 INFO:tasks.workunit.client.1.smithi148.stdout:6/482: mknod d3/d1d/d28/d31/d83/ca3 0 2024-07-03T15:43:51.886 INFO:tasks.workunit.client.1.smithi148.stdout:8/366: symlink d4/da/d14/d1a/d32/d38/d58/l84 0 2024-07-03T15:43:51.886 INFO:tasks.workunit.client.1.smithi148.stdout:8/367: truncate d4/d54/f63 506855 0 2024-07-03T15:43:51.886 INFO:tasks.workunit.client.1.smithi148.stdout:8/368: stat d4/da/d47/f83 0 2024-07-03T15:43:51.887 INFO:tasks.workunit.client.1.smithi148.stdout:8/369: write d4/da/d14/d1a/f6e [970023,128957] 0 2024-07-03T15:43:51.887 INFO:tasks.workunit.client.1.smithi148.stdout:8/370: truncate d4/da/f3e 2234140 0 2024-07-03T15:43:51.887 INFO:tasks.workunit.client.1.smithi148.stdout:8/371: fdatasync d4/da/d51/f69 0 2024-07-03T15:43:51.888 INFO:tasks.workunit.client.1.smithi148.stdout:9/668: dwrite d0/d2f/fb5 [0,4194304] 0 2024-07-03T15:43:51.888 INFO:tasks.workunit.client.1.smithi148.stdout:9/669: readlink d0/d49/d43/d19/d8c/lc6 0 2024-07-03T15:43:51.888 INFO:tasks.workunit.client.1.smithi148.stdout:9/670: chown d0/dd/l88 3 1 2024-07-03T15:43:51.888 INFO:tasks.workunit.client.1.smithi148.stdout:5/435: mknod d7/d13/d54/c8f 0 2024-07-03T15:43:51.888 INFO:tasks.workunit.client.1.smithi148.stdout:5/436: write d7/da/f2b [823716,34865] 0 2024-07-03T15:43:51.889 INFO:tasks.workunit.client.1.smithi148.stdout:3/457: mknod dd/d33/d52/c90 0 2024-07-03T15:43:51.890 INFO:tasks.workunit.client.1.smithi148.stdout:7/511: creat d5/d16/fa5 x:0 0 0 2024-07-03T15:43:51.891 INFO:tasks.workunit.client.1.smithi148.stdout:0/414: read da/d17/d2a/f59 [45694,36786] 0 2024-07-03T15:43:51.892 INFO:tasks.workunit.client.1.smithi148.stdout:6/483: symlink d3/d1d/d28/d7f/la4 0 2024-07-03T15:43:51.892 INFO:tasks.workunit.client.1.smithi148.stdout:6/484: chown d3/d50/l81 603760503 1 2024-07-03T15:43:51.893 INFO:tasks.workunit.client.1.smithi148.stdout:2/555: dwrite d5/d14/d22/d2b/d13/d41/f4b [0,4194304] 0 2024-07-03T15:43:51.894 INFO:tasks.workunit.client.1.smithi148.stdout:8/372: rename d4/da/d47/l64 to d4/d54/l85 0 2024-07-03T15:43:51.899 INFO:tasks.workunit.client.1.smithi148.stdout:9/671: rmdir d0/d63/da6 0 2024-07-03T15:43:51.900 INFO:tasks.workunit.client.1.smithi148.stdout:1/352: dwrite d0/d2/d2e/d39/d42/d62/f6e [0,4194304] 0 2024-07-03T15:43:51.902 INFO:tasks.workunit.client.1.smithi148.stdout:5/437: truncate d7/da/df/d1d/f2e 1454399 0 2024-07-03T15:43:51.903 INFO:tasks.workunit.client.1.smithi148.stdout:5/438: creat d7/da/df/d16/d2f/d44/d52/f90 x:0 0 0 2024-07-03T15:43:51.904 INFO:tasks.workunit.client.1.smithi148.stdout:0/415: mkdir da/d44/d13/d1d/d3e/d85/d8e 0 2024-07-03T15:43:51.904 INFO:tasks.workunit.client.1.smithi148.stdout:0/416: chown da/d44/d13/l56 407 1 2024-07-03T15:43:51.905 INFO:tasks.workunit.client.1.smithi148.stdout:2/556: dread d5/d14/d22/f62 [0,4194304] 0 2024-07-03T15:43:51.905 INFO:tasks.workunit.client.1.smithi148.stdout:4/352: dwrite d0/d8/d3a/f47 [0,4194304] 0 2024-07-03T15:43:51.905 INFO:tasks.workunit.client.1.smithi148.stdout:2/557: dread - d5/d5f/d7e/fa7 zero size 2024-07-03T15:43:51.905 INFO:tasks.workunit.client.1.smithi148.stdout:2/558: dread d5/d7/f83 [0,4194304] 0 2024-07-03T15:43:51.906 INFO:tasks.workunit.client.1.smithi148.stdout:6/485: mkdir d3/da5 0 2024-07-03T15:43:51.914 INFO:tasks.workunit.client.1.smithi148.stdout:9/672: mkdir d0/d49/d43/dd8 0 2024-07-03T15:43:51.914 INFO:tasks.workunit.client.1.smithi148.stdout:9/673: fsync d0/dd/d25/d37/d40/f62 0 2024-07-03T15:43:51.914 INFO:tasks.workunit.client.1.smithi148.stdout:7/512: dwrite d5/f57 [0,4194304] 0 2024-07-03T15:43:51.914 INFO:tasks.workunit.client.1.smithi148.stdout:7/513: write d5/d16/d1c/d33/f42 [2749674,84567] 0 2024-07-03T15:43:51.914 INFO:tasks.workunit.client.1.smithi148.stdout:7/514: truncate d5/d16/d1c/d33/d4c/f99 514924 0 2024-07-03T15:43:51.917 INFO:tasks.workunit.client.1.smithi148.stdout:1/353: unlink d0/f24 0 2024-07-03T15:43:51.919 INFO:tasks.workunit.client.1.smithi148.stdout:5/439: mknod d7/da/d21/c91 0 2024-07-03T15:43:51.920 INFO:tasks.workunit.client.1.smithi148.stdout:5/440: chown d7/da/df/d16/d2f/d44/c45 2711722 1 2024-07-03T15:43:51.920 INFO:tasks.workunit.client.1.smithi148.stdout:5/441: getdents d7/da/d79 0 2024-07-03T15:43:51.920 INFO:tasks.workunit.client.1.smithi148.stdout:5/442: write d7/da/f74 [359022,54993] 0 2024-07-03T15:43:51.920 INFO:tasks.workunit.client.1.smithi148.stdout:5/443: truncate d7/da/df/f3d 827324 0 2024-07-03T15:43:51.921 INFO:tasks.workunit.client.1.smithi148.stdout:0/417: mknod da/d38/d88/c8f 0 2024-07-03T15:43:51.921 INFO:tasks.workunit.client.1.smithi148.stdout:0/418: fdatasync da/d44/d13/d14/f61 0 2024-07-03T15:43:51.923 INFO:tasks.workunit.client.1.smithi148.stdout:4/353: symlink d0/d8/d3a/l72 0 2024-07-03T15:43:51.924 INFO:tasks.workunit.client.1.smithi148.stdout:6/486: stat d3/d1d/d28/c52 0 2024-07-03T15:43:51.927 INFO:tasks.workunit.client.1.smithi148.stdout:3/458: rename dd/f34 to dd/d35/d81/f91 0 2024-07-03T15:43:51.936 INFO:tasks.workunit.client.1.smithi148.stdout:1/354: symlink d0/d2/d11/l73 0 2024-07-03T15:43:51.936 INFO:tasks.workunit.client.1.smithi148.stdout:1/355: read d0/d2/d6/d14/f15 [6412950,115443] 0 2024-07-03T15:43:51.936 INFO:tasks.workunit.client.1.smithi148.stdout:1/356: stat d0/d2/d6 0 2024-07-03T15:43:51.936 INFO:tasks.workunit.client.0.smithi028.stderr:++ readlink -f fsstress 2024-07-03T15:43:51.937 INFO:tasks.workunit.client.1.smithi148.stdout:5/444: creat d7/da/df/d16/d2f/d44/d52/d60/f92 x:0 0 0 2024-07-03T15:43:51.937 INFO:tasks.workunit.client.1.smithi148.stdout:5/445: creat d7/da/df/d16/d2f/f93 x:0 0 0 2024-07-03T15:43:51.937 INFO:tasks.workunit.client.1.smithi148.stdout:9/674: dwrite d0/d49/d43/d19/f48 [0,4194304] 0 2024-07-03T15:43:51.938 INFO:tasks.workunit.client.0.smithi028.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-07-03T15:43:51.938 INFO:tasks.workunit.client.0.smithi028.stderr:+ popd 2024-07-03T15:43:51.938 INFO:tasks.workunit.client.0.smithi028.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-07-03T15:43:51.938 INFO:tasks.workunit.client.0.smithi028.stderr:+ popd 2024-07-03T15:43:51.939 INFO:tasks.workunit.client.0.smithi028.stdout:~/cephtest/mnt.0/client.0/tmp 2024-07-03T15:43:51.939 INFO:tasks.workunit.client.0.smithi028.stderr:++ mktemp -d -p . 2024-07-03T15:43:51.940 INFO:tasks.workunit.client.1.smithi148.stdout:0/419: mkdir da/d58/d90 0 2024-07-03T15:43:51.940 INFO:tasks.workunit.client.1.smithi148.stdout:4/354: mknod d0/d8/d13/c73 0 2024-07-03T15:43:51.940 INFO:tasks.workunit.client.1.smithi148.stdout:4/355: truncate d0/d8/d3a/d36/f3e 921963 0 2024-07-03T15:43:51.940 INFO:tasks.workunit.client.1.smithi148.stdout:9/675: write d0/dd/d25/d37/d40/f62 [1432843,120935] 0 2024-07-03T15:43:51.940 INFO:tasks.workunit.client.1.smithi148.stdout:9/676: fdatasync d0/dd/d25/d37/d40/d72/db8/fd6 0 2024-07-03T15:43:51.940 INFO:tasks.workunit.client.1.smithi148.stdout:9/677: write d0/dd/d11/f9c [891575,42305] 0 2024-07-03T15:43:51.940 INFO:tasks.workunit.client.1.smithi148.stdout:9/678: write d0/d2f/fb5 [5128605,4937] 0 2024-07-03T15:43:51.940 INFO:tasks.workunit.client.1.smithi148.stdout:6/487: truncate d3/d1d/d28/f51 846304 0 2024-07-03T15:43:51.941 INFO:tasks.workunit.client.0.smithi028.stderr:+ T=./tmp.K6T0EhjVPM 2024-07-03T15:43:51.941 INFO:tasks.workunit.client.0.smithi028.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.K6T0EhjVPM -l 1 -n 1000 -p 10 -v 2024-07-03T15:43:51.942 INFO:tasks.workunit.client.1.smithi148.stdout:8/373: rename d4/d54 to d4/d86 0 2024-07-03T15:43:51.942 INFO:tasks.workunit.client.1.smithi148.stdout:8/374: readlink d4/l5 0 2024-07-03T15:43:51.942 INFO:tasks.workunit.client.1.smithi148.stdout:8/375: dread - d4/da/d51/d18/d1f/f39 zero size 2024-07-03T15:43:51.942 INFO:tasks.workunit.client.1.smithi148.stdout:8/376: write d4/da/d14/d17/f48 [907954,113661] 0 2024-07-03T15:43:51.944 INFO:tasks.workunit.client.1.smithi148.stdout:7/515: rename d5/d16/d1c/d33/l35 to d5/d16/d1c/d4a/d66/la6 0 2024-07-03T15:43:51.945 INFO:tasks.workunit.client.0.smithi028.stdout:seed = 1720390156 2024-07-03T15:43:51.946 INFO:tasks.workunit.client.1.smithi148.stdout:5/446: write d7/da/df/d16/d2f/d44/d52/f55 [211674,10537] 0 2024-07-03T15:43:51.946 INFO:tasks.workunit.client.1.smithi148.stdout:1/357: mknod d0/d2/d6/d14/c74 0 2024-07-03T15:43:51.947 INFO:tasks.workunit.client.1.smithi148.stdout:9/679: write d0/dd/d25/d6a/f82 [653226,50986] 0 2024-07-03T15:43:51.949 INFO:tasks.workunit.client.0.smithi028.stdout:0/0: rename - no filename 2024-07-03T15:43:51.950 INFO:tasks.workunit.client.1.smithi148.stdout:0/420: creat da/d44/d13/d1d/d3e/d85/f91 x:0 0 0 2024-07-03T15:43:51.951 INFO:tasks.workunit.client.1.smithi148.stdout:4/356: symlink d0/d8/d22/d2f/d41/l74 0 2024-07-03T15:43:51.953 INFO:tasks.workunit.client.0.smithi028.stdout:0/1: mknod c0 0 2024-07-03T15:43:51.956 INFO:tasks.workunit.client.1.smithi148.stdout:8/377: mknod d4/da/d14/d17/d76/c87 0 2024-07-03T15:43:51.958 INFO:tasks.workunit.client.0.smithi028.stdout:1/0: creat f0 x:0 0 0 2024-07-03T15:43:51.960 INFO:tasks.workunit.client.0.smithi028.stdout:0/2: rename c0 to c1 0 2024-07-03T15:43:51.960 INFO:tasks.workunit.client.1.smithi148.stdout:4/357: dread d0/d8/d13/d48/f2c [0,4194304] 0 2024-07-03T15:43:51.961 INFO:tasks.workunit.client.0.smithi028.stdout:2/0: mknod c0 0 2024-07-03T15:43:51.962 INFO:tasks.workunit.client.0.smithi028.stdout:3/0: rmdir - no directory 2024-07-03T15:43:51.962 INFO:tasks.workunit.client.1.smithi148.stdout:1/358: creat d0/d2/f75 x:0 0 0 2024-07-03T15:43:51.963 INFO:tasks.workunit.client.0.smithi028.stdout:0/3: mkdir d2 0 2024-07-03T15:43:51.964 INFO:tasks.workunit.client.1.smithi148.stdout:9/680: creat d0/dd/d25/d39/fd9 x:0 0 0 2024-07-03T15:43:51.965 INFO:tasks.workunit.client.0.smithi028.stdout:2/1: symlink l1 0 2024-07-03T15:43:51.966 INFO:tasks.workunit.client.1.smithi148.stdout:0/421: symlink da/d44/d13/d1d/l92 0 2024-07-03T15:43:51.966 INFO:tasks.workunit.client.0.smithi028.stdout:4/0: chown . 1869 1 2024-07-03T15:43:51.967 INFO:tasks.workunit.client.0.smithi028.stdout:4/1: fsync - no filename 2024-07-03T15:43:51.967 INFO:tasks.workunit.client.0.smithi028.stdout:4/2: fsync - no filename 2024-07-03T15:43:51.967 INFO:tasks.workunit.client.0.smithi028.stdout:4/3: dread - no filename 2024-07-03T15:43:51.969 INFO:tasks.workunit.client.0.smithi028.stdout:3/1: creat f0 x:0 0 0 2024-07-03T15:43:51.970 INFO:tasks.workunit.client.0.smithi028.stdout:3/2: read - f0 zero size 2024-07-03T15:43:51.972 INFO:tasks.workunit.client.0.smithi028.stdout:0/4: mkdir d2/d3 0 2024-07-03T15:43:51.974 INFO:tasks.workunit.client.0.smithi028.stdout:2/2: rename c0 to c2 0 2024-07-03T15:43:51.974 INFO:tasks.workunit.client.0.smithi028.stdout:2/3: dread - no filename 2024-07-03T15:43:51.974 INFO:tasks.workunit.client.0.smithi028.stdout:2/4: dwrite - no filename 2024-07-03T15:43:51.974 INFO:tasks.workunit.client.0.smithi028.stdout:2/5: dwrite - no filename 2024-07-03T15:43:51.974 INFO:tasks.workunit.client.0.smithi028.stdout:2/6: dwrite - no filename 2024-07-03T15:43:51.975 INFO:tasks.workunit.client.0.smithi028.stdout:2/7: truncate - no filename 2024-07-03T15:43:51.975 INFO:tasks.workunit.client.1.smithi148.stdout:9/681: dread d0/d49/d43/d19/f1e [0,4194304] 0 2024-07-03T15:43:51.976 INFO:tasks.workunit.client.0.smithi028.stdout:5/0: dread - no filename 2024-07-03T15:43:51.976 INFO:tasks.workunit.client.0.smithi028.stdout:5/1: rename - no filename 2024-07-03T15:43:51.976 INFO:tasks.workunit.client.0.smithi028.stdout:5/2: chown . 118919 1 2024-07-03T15:43:51.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:51 smithi148 ceph-mon[36056]: pgmap v238: 65 pgs: 65 active+clean; 652 MiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 49 MiB/s rd, 61 MiB/s wr, 659 op/s 2024-07-03T15:43:51.985 INFO:tasks.workunit.client.0.smithi028.stdout:0/5: mknod d2/c4 0 2024-07-03T15:43:51.985 INFO:tasks.workunit.client.0.smithi028.stdout:0/6: chown d2/c4 0 1 2024-07-03T15:43:51.986 INFO:tasks.workunit.client.0.smithi028.stdout:2/8: symlink l3 0 2024-07-03T15:43:51.986 INFO:tasks.workunit.client.0.smithi028.stdout:2/9: write - no filename 2024-07-03T15:43:51.987 INFO:tasks.workunit.client.0.smithi028.stdout:2/10: dwrite - no filename 2024-07-03T15:43:51.987 INFO:tasks.workunit.client.0.smithi028.stdout:2/11: write - no filename 2024-07-03T15:43:51.988 INFO:tasks.workunit.client.0.smithi028.stdout:6/0: rmdir - no directory 2024-07-03T15:43:51.989 INFO:tasks.workunit.client.0.smithi028.stdout:5/3: mkdir d0 0 2024-07-03T15:43:51.993 INFO:tasks.workunit.client.0.smithi028.stdout:5/4: rename d0 to d0/d1 22 2024-07-03T15:43:51.994 INFO:tasks.workunit.client.0.smithi028.stdout:5/5: fsync - no filename 2024-07-03T15:43:51.994 INFO:tasks.workunit.client.0.smithi028.stdout:0/7: rename c1 to d2/c5 0 2024-07-03T15:43:51.994 INFO:tasks.workunit.client.0.smithi028.stdout:0/8: write - no filename 2024-07-03T15:43:51.994 INFO:tasks.workunit.client.0.smithi028.stdout:0/9: chown d2 325 1 2024-07-03T15:43:51.994 INFO:tasks.workunit.client.0.smithi028.stdout:0/10: dwrite - no filename 2024-07-03T15:43:51.994 INFO:tasks.workunit.client.0.smithi028.stdout:0/11: dwrite - no filename 2024-07-03T15:43:51.994 INFO:tasks.workunit.client.0.smithi028.stdout:0/12: write - no filename 2024-07-03T15:43:51.994 INFO:tasks.workunit.client.0.smithi028.stdout:0/13: write - no filename 2024-07-03T15:43:51.994 INFO:tasks.workunit.client.0.smithi028.stdout:0/14: fdatasync - no filename 2024-07-03T15:43:51.995 INFO:tasks.workunit.client.0.smithi028.stdout:2/12: creat f4 x:0 0 0 2024-07-03T15:43:51.997 INFO:tasks.workunit.client.0.smithi028.stdout:6/1: creat f0 x:0 0 0 2024-07-03T15:43:51.997 INFO:tasks.workunit.client.0.smithi028.stdout:6/2: chown f0 1446 1 2024-07-03T15:43:51.997 INFO:tasks.workunit.client.0.smithi028.stdout:6/3: dread - f0 zero size 2024-07-03T15:43:52.001 INFO:tasks.workunit.client.0.smithi028.stdout:5/6: creat d0/f2 x:0 0 0 2024-07-03T15:43:52.002 INFO:tasks.workunit.client.1.smithi148.stdout:4/358: creat d0/d8/d19/d45/f75 x:0 0 0 2024-07-03T15:43:52.006 INFO:tasks.workunit.client.0.smithi028.stdout:0/15: mknod d2/c6 0 2024-07-03T15:43:52.007 INFO:tasks.workunit.client.0.smithi028.stdout:2/13: symlink l5 0 2024-07-03T15:43:52.007 INFO:tasks.workunit.client.0.smithi028.stdout:2/14: chown f4 1117883 1 2024-07-03T15:43:52.007 INFO:tasks.workunit.client.0.smithi028.stdout:2/15: write f4 [988258,11679] 0 2024-07-03T15:43:52.009 INFO:tasks.workunit.client.0.smithi028.stdout:7/0: creat f0 x:0 0 0 2024-07-03T15:43:52.009 INFO:tasks.workunit.client.0.smithi028.stdout:7/1: readlink - no filename 2024-07-03T15:43:52.009 INFO:tasks.workunit.client.0.smithi028.stdout:7/2: dread - f0 zero size 2024-07-03T15:43:52.014 INFO:tasks.workunit.client.0.smithi028.stdout:2/16: rename l5 to l6 0 2024-07-03T15:43:52.014 INFO:tasks.workunit.client.0.smithi028.stdout:2/17: creat f7 x:0 0 0 2024-07-03T15:43:52.014 INFO:tasks.workunit.client.0.smithi028.stdout:2/18: write f4 [1350635,66192] 0 2024-07-03T15:43:52.014 INFO:tasks.workunit.client.0.smithi028.stdout:8/0: mkdir d0 0 2024-07-03T15:43:52.014 INFO:tasks.workunit.client.0.smithi028.stdout:8/1: fdatasync - no filename 2024-07-03T15:43:52.014 INFO:tasks.workunit.client.0.smithi028.stdout:8/2: write - no filename 2024-07-03T15:43:52.015 INFO:tasks.workunit.client.0.smithi028.stdout:8/3: chown d0 200523782 1 2024-07-03T15:43:52.015 INFO:tasks.workunit.client.0.smithi028.stdout:8/4: rename d0 to d0/d1 22 2024-07-03T15:43:52.015 INFO:tasks.workunit.client.1.smithi148.stdout:0/422: mkdir da/d17/d2a/d2f/d93 0 2024-07-03T15:43:52.016 INFO:tasks.workunit.client.0.smithi028.stdout:9/0: truncate - no filename 2024-07-03T15:43:52.016 INFO:tasks.workunit.client.0.smithi028.stdout:9/1: write - no filename 2024-07-03T15:43:52.016 INFO:tasks.workunit.client.0.smithi028.stdout:9/2: chown . 2208276 1 2024-07-03T15:43:52.016 INFO:tasks.workunit.client.0.smithi028.stdout:9/3: unlink - no file 2024-07-03T15:43:52.016 INFO:tasks.workunit.client.0.smithi028.stdout:9/4: fsync - no filename 2024-07-03T15:43:52.019 INFO:tasks.workunit.client.0.smithi028.stdout:8/5: mkdir d0/d2 0 2024-07-03T15:43:52.019 INFO:tasks.workunit.client.1.smithi148.stdout:0/423: mkdir da/d44/d94 0 2024-07-03T15:43:52.019 INFO:tasks.workunit.client.1.smithi148.stdout:7/516: rename d5/d16 to d5/d7d/da7 0 2024-07-03T15:43:52.020 INFO:tasks.workunit.client.0.smithi028.stdout:9/5: creat f0 x:0 0 0 2024-07-03T15:43:52.020 INFO:tasks.workunit.client.0.smithi028.stdout:9/6: dread - f0 zero size 2024-07-03T15:43:52.020 INFO:tasks.workunit.client.0.smithi028.stdout:9/7: chown f0 253303459 1 2024-07-03T15:43:52.020 INFO:tasks.workunit.client.0.smithi028.stdout:8/6: mkdir d0/d2/d3 0 2024-07-03T15:43:52.020 INFO:tasks.workunit.client.0.smithi028.stdout:8/7: truncate - no filename 2024-07-03T15:43:52.022 INFO:tasks.workunit.client.1.smithi148.stdout:5/447: rename d7/d13/c65 to d7/d13/d54/c94 0 2024-07-03T15:43:52.023 INFO:tasks.workunit.client.0.smithi028.stdout:9/8: unlink f0 0 2024-07-03T15:43:52.023 INFO:tasks.workunit.client.0.smithi028.stdout:9/9: dwrite - no filename 2024-07-03T15:43:52.024 INFO:tasks.workunit.client.0.smithi028.stdout:9/10: dread - no filename 2024-07-03T15:43:52.024 INFO:tasks.workunit.client.0.smithi028.stdout:9/11: fdatasync - no filename 2024-07-03T15:43:52.024 INFO:tasks.workunit.client.0.smithi028.stdout:8/8: mknod d0/c4 0 2024-07-03T15:43:52.024 INFO:tasks.workunit.client.0.smithi028.stdout:8/9: dread - no filename 2024-07-03T15:43:52.025 INFO:tasks.workunit.client.1.smithi148.stdout:5/448: mknod d7/da/df/d16/d2f/d44/d52/d60/c95 0 2024-07-03T15:43:52.026 INFO:tasks.workunit.client.0.smithi028.stdout:9/12: mkdir d1 0 2024-07-03T15:43:52.031 INFO:tasks.workunit.client.1.smithi148.stdout:5/449: symlink d7/da/df/d83/l96 0 2024-07-03T15:43:52.033 INFO:tasks.workunit.client.1.smithi148.stdout:5/450: mknod d7/da/df/d16/d2f/d44/d52/d60/c97 0 2024-07-03T15:43:52.034 INFO:tasks.workunit.client.1.smithi148.stdout:5/451: chown d7/d13/d23/d40/c8c 26049646 1 2024-07-03T15:43:52.034 INFO:tasks.workunit.client.1.smithi148.stdout:5/452: truncate d7/da/df/d16/d2f/d44/f70 322598 0 2024-07-03T15:43:52.036 INFO:tasks.workunit.client.1.smithi148.stdout:8/378: rename d4/da/d14/d1a/d32/d38/d4a to d4/da/d47/d88 0 2024-07-03T15:43:52.038 INFO:tasks.workunit.client.0.smithi028.stdout:9/13: creat d1/f2 x:0 0 0 2024-07-03T15:43:52.038 INFO:tasks.workunit.client.0.smithi028.stdout:9/14: stat d1 0 2024-07-03T15:43:52.038 INFO:tasks.workunit.client.0.smithi028.stdout:9/15: creat d1/f3 x:0 0 0 2024-07-03T15:43:52.038 INFO:tasks.workunit.client.0.smithi028.stdout:9/16: write d1/f3 [315557,61602] 0 2024-07-03T15:43:52.039 INFO:tasks.workunit.client.0.smithi028.stdout:9/17: chown d1/f3 2948521 1 2024-07-03T15:43:52.039 INFO:tasks.workunit.client.1.smithi148.stdout:5/453: rename d7/da/d21/l67 to d7/da/d79/l98 0 2024-07-03T15:43:52.041 INFO:tasks.workunit.client.0.smithi028.stdout:9/18: symlink d1/l4 0 2024-07-03T15:43:52.042 INFO:tasks.workunit.client.0.smithi028.stdout:9/19: creat d1/f5 x:0 0 0 2024-07-03T15:43:52.044 INFO:tasks.workunit.client.1.smithi148.stdout:5/454: read d7/da/d21/f5e [447697,64817] 0 2024-07-03T15:43:52.076 INFO:tasks.workunit.client.0.smithi028.stdout:1/1: dwrite f0 [0,4194304] 0 2024-07-03T15:43:52.107 INFO:tasks.workunit.client.1.smithi148.stdout:9/682: dwrite d0/dd/d11/f66 [0,4194304] 0 2024-07-03T15:43:52.261 INFO:tasks.workunit.client.1.smithi148.stdout:4/359: dwrite d0/d8/d22/d2f/d41/f4d [0,4194304] 0 2024-07-03T15:43:52.267 INFO:tasks.workunit.client.1.smithi148.stdout:4/360: getdents d0/d8/d22/d2f/d41 0 2024-07-03T15:43:52.267 INFO:tasks.workunit.client.1.smithi148.stdout:4/361: truncate d0/f6d 500964 0 2024-07-03T15:43:52.393 INFO:tasks.workunit.client.1.smithi148.stdout:5/455: dwrite d7/da/df/f71 [0,4194304] 0 2024-07-03T15:43:52.393 INFO:tasks.workunit.client.1.smithi148.stdout:5/456: chown d7/da/df/d16/d1a/l3f 748204 1 2024-07-03T15:43:52.400 INFO:tasks.workunit.client.1.smithi148.stdout:5/457: unlink d7/da/df/d16/l3c 0 2024-07-03T15:43:52.405 INFO:tasks.workunit.client.1.smithi148.stdout:5/458: rename d7/da/df/d16/d2f/d44/d52/f76 to d7/da/df/d16/d2f/d44/f99 0 2024-07-03T15:43:52.406 INFO:tasks.workunit.client.1.smithi148.stdout:5/459: fsync d7/da/df/d16/f8a 0 2024-07-03T15:43:52.406 INFO:tasks.workunit.client.1.smithi148.stdout:5/460: chown d7/da/d21/l8d 107894523 1 2024-07-03T15:43:52.406 INFO:tasks.workunit.client.1.smithi148.stdout:5/461: dread - d7/da/df/d16/d1a/f7a zero size 2024-07-03T15:43:52.407 INFO:tasks.workunit.client.1.smithi148.stdout:5/462: rename d7/f75 to d7/da/df/d16/d2f/d44/f9a 0 2024-07-03T15:43:52.409 INFO:tasks.workunit.client.1.smithi148.stdout:5/463: symlink d7/da/df/d16/d2f/d44/l9b 0 2024-07-03T15:43:52.413 INFO:tasks.workunit.client.1.smithi148.stdout:5/464: creat d7/da/df/d83/f9c x:0 0 0 2024-07-03T15:43:52.413 INFO:tasks.workunit.client.1.smithi148.stdout:5/465: dread - d7/da/df/d16/d2f/f93 zero size 2024-07-03T15:43:52.413 INFO:tasks.workunit.client.1.smithi148.stdout:5/466: fdatasync d7/da/df/d16/d1a/f50 0 2024-07-03T15:43:52.413 INFO:tasks.workunit.client.1.smithi148.stdout:5/467: read - d7/d13/d23/f85 zero size 2024-07-03T15:43:52.413 INFO:tasks.workunit.client.1.smithi148.stdout:5/468: write d7/da/df/d1d/f2e [1497206,67044] 0 2024-07-03T15:43:52.413 INFO:tasks.workunit.client.1.smithi148.stdout:5/469: chown d7/da/df/d16/d2f/d44/d52/d60/c97 297829689 1 2024-07-03T15:43:52.440 INFO:tasks.workunit.client.1.smithi148.stdout:9/683: dread d0/dd/d25/d6a/f82 [0,4194304] 0 2024-07-03T15:43:52.446 INFO:tasks.workunit.client.1.smithi148.stdout:9/684: rmdir d0/dd/d25/d6a 39 2024-07-03T15:43:52.447 INFO:tasks.workunit.client.1.smithi148.stdout:9/685: write d0/dd/d46/fc4 [67792,24191] 0 2024-07-03T15:43:52.449 INFO:tasks.workunit.client.1.smithi148.stdout:9/686: mknod d0/d18/d24/d47/d4d/dae/cda 0 2024-07-03T15:43:52.457 INFO:tasks.workunit.client.1.smithi148.stdout:9/687: rmdir d0/d63 39 2024-07-03T15:43:52.463 INFO:tasks.workunit.client.1.smithi148.stdout:9/688: creat d0/d18/d24/d47/d4d/dae/fdb x:0 0 0 2024-07-03T15:43:52.494 INFO:tasks.workunit.client.1.smithi148.stdout:0/424: dwrite da/d44/d13/d14/d50/f83 [0,4194304] 0 2024-07-03T15:43:52.499 INFO:tasks.workunit.client.1.smithi148.stdout:9/689: dread d0/dd/f14 [8388608,4194304] 0 2024-07-03T15:43:52.499 INFO:tasks.workunit.client.1.smithi148.stdout:9/690: truncate d0/dd/d25/d37/d40/f62 2808269 0 2024-07-03T15:43:52.505 INFO:tasks.workunit.client.0.smithi028.stdout:6/4: dwrite f0 [0,4194304] 0 2024-07-03T15:43:52.514 INFO:tasks.workunit.client.1.smithi148.stdout:0/425: rename da/d44/d7d to da/d44/d13/d14/d50/d95 0 2024-07-03T15:43:52.515 INFO:tasks.workunit.client.1.smithi148.stdout:0/426: write da/d44/d13/d14/f68 [789543,30677] 0 2024-07-03T15:43:52.521 INFO:tasks.workunit.client.1.smithi148.stdout:9/691: truncate d0/d49/d43/f2c 2563320 0 2024-07-03T15:43:52.521 INFO:tasks.workunit.client.1.smithi148.stdout:9/692: creat d0/d18/d24/d47/d4d/fdc x:0 0 0 2024-07-03T15:43:52.534 INFO:tasks.workunit.client.1.smithi148.stdout:5/470: dwrite d7/da/df/d16/d2f/d44/d52/f82 [0,4194304] 0 2024-07-03T15:43:52.541 INFO:tasks.workunit.client.1.smithi148.stdout:5/471: link d7/da/df/d16/d1a/f34 d7/da/df/d16/d2f/f9d 0 2024-07-03T15:43:52.543 INFO:tasks.workunit.client.1.smithi148.stdout:5/472: truncate d7/da/df/d16/d2f/d44/f69 2733437 0 2024-07-03T15:43:52.545 INFO:tasks.workunit.client.1.smithi148.stdout:5/473: mknod d7/d13/c9e 0 2024-07-03T15:43:52.545 INFO:tasks.workunit.client.1.smithi148.stdout:5/474: stat d7/da/df/d16/d2f/d44/d52/d86 0 2024-07-03T15:43:52.561 INFO:tasks.workunit.client.1.smithi148.stdout:8/379: dwrite d4/da/d51/d18/d20/f5a [0,4194304] 0 2024-07-03T15:43:52.578 INFO:tasks.workunit.client.1.smithi148.stdout:8/380: mkdir d4/da/d14/d1a/d32/d89 0 2024-07-03T15:43:52.581 INFO:tasks.workunit.client.1.smithi148.stdout:8/381: mknod d4/da/d14/d1a/d32/c8a 0 2024-07-03T15:43:52.584 INFO:tasks.workunit.client.1.smithi148.stdout:8/382: unlink d4/da/d51/d18/d20/c2e 0 2024-07-03T15:43:52.588 INFO:tasks.workunit.client.1.smithi148.stdout:8/383: link d4/da/d14/f16 d4/da/d14/d1a/d32/d38/d58/f8b 0 2024-07-03T15:43:52.591 INFO:tasks.workunit.client.1.smithi148.stdout:8/384: creat d4/da/d51/d18/f8c x:0 0 0 2024-07-03T15:43:52.591 INFO:tasks.workunit.client.0.smithi028.stdout:5/7: dwrite d0/f2 [0,4194304] 0 2024-07-03T15:43:52.594 INFO:tasks.workunit.client.1.smithi148.stdout:8/385: link d4/da/f29 d4/da/d14/f8d 0 2024-07-03T15:43:52.597 INFO:tasks.workunit.client.1.smithi148.stdout:8/386: rmdir d4/da/d14/d17 39 2024-07-03T15:43:52.597 INFO:tasks.workunit.client.1.smithi148.stdout:8/387: fdatasync d4/da/d47/f83 0 2024-07-03T15:43:52.597 INFO:tasks.workunit.client.1.smithi148.stdout:8/388: fdatasync d4/da/d14/d1a/f37 0 2024-07-03T15:43:52.653 INFO:tasks.workunit.client.1.smithi148.stdout:0/427: dwrite da/d44/d13/d14/f49 [0,4194304] 0 2024-07-03T15:43:52.653 INFO:tasks.workunit.client.1.smithi148.stdout:9/693: dwrite d0/dd/f2b [4194304,4194304] 0 2024-07-03T15:43:52.653 INFO:tasks.workunit.client.1.smithi148.stdout:9/694: dread - d0/dd/d25/d39/fd9 zero size 2024-07-03T15:43:52.653 INFO:tasks.workunit.client.1.smithi148.stdout:9/695: dread - d0/fc5 zero size 2024-07-03T15:43:52.653 INFO:tasks.workunit.client.1.smithi148.stdout:9/696: fdatasync d0/dd/d25/f3c 0 2024-07-03T15:43:52.677 INFO:tasks.workunit.client.1.smithi148.stdout:0/428: dwrite da/d38/f6b [0,4194304] 0 2024-07-03T15:43:52.677 INFO:tasks.workunit.client.1.smithi148.stdout:0/429: creat da/d44/f96 x:0 0 0 2024-07-03T15:43:52.677 INFO:tasks.workunit.client.1.smithi148.stdout:9/697: dread d0/d49/d43/f5b [0,4194304] 0 2024-07-03T15:43:52.677 INFO:tasks.workunit.client.1.smithi148.stdout:9/698: creat d0/dd/d11/fdd x:0 0 0 2024-07-03T15:43:52.683 INFO:tasks.workunit.client.1.smithi148.stdout:9/699: stat d0/dd/d29/l78 0 2024-07-03T15:43:52.684 INFO:tasks.workunit.client.1.smithi148.stdout:9/700: chown d0/dd/d11/lbc 5959 1 2024-07-03T15:43:52.684 INFO:tasks.workunit.client.1.smithi148.stdout:9/701: fdatasync d0/dd/d11/fdd 0 2024-07-03T15:43:52.685 INFO:tasks.workunit.client.1.smithi148.stdout:0/430: dread da/d44/d13/d1d/d3e/f6e [0,4194304] 0 2024-07-03T15:43:52.692 INFO:tasks.workunit.client.1.smithi148.stdout:9/702: rename d0/dd/d25/d77 to d0/dd/d9f/dde 0 2024-07-03T15:43:52.696 INFO:tasks.workunit.client.1.smithi148.stdout:9/703: mkdir d0/d49/d43/dd8/ddf 0 2024-07-03T15:43:52.699 INFO:tasks.workunit.client.1.smithi148.stdout:9/704: unlink d0/dd/ff 0 2024-07-03T15:43:52.700 INFO:tasks.workunit.client.1.smithi148.stdout:9/705: read - d0/dd/d11/db4/fc0 zero size 2024-07-03T15:43:52.700 INFO:tasks.workunit.client.1.smithi148.stdout:9/706: write d0/d18/f21 [7592103,105124] 0 2024-07-03T15:43:52.700 INFO:tasks.workunit.client.0.smithi028.stdout:2/19: dread f4 [0,4194304] 0 2024-07-03T15:43:52.701 INFO:tasks.workunit.client.0.smithi028.stdout:2/20: write f7 [520346,28316] 0 2024-07-03T15:43:52.701 INFO:tasks.workunit.client.0.smithi028.stdout:3/3: getdents . 0 2024-07-03T15:43:52.703 INFO:tasks.workunit.client.1.smithi148.stdout:0/431: write da/d44/d13/d14/d35/f5e [1577238,103400] 0 2024-07-03T15:43:52.705 INFO:tasks.workunit.client.1.smithi148.stdout:9/707: rmdir d0/dd/d25/d37/d40 39 2024-07-03T15:43:52.716 INFO:tasks.workunit.client.1.smithi148.stdout:9/708: dread d0/dd/d29/f5e [0,4194304] 0 2024-07-03T15:43:52.734 INFO:tasks.workunit.client.0.smithi028.stdout:7/3: dwrite f0 [0,4194304] 0 2024-07-03T15:43:52.735 INFO:tasks.workunit.client.0.smithi028.stdout:8/10: getdents d0/d2 0 2024-07-03T15:43:52.748 INFO:tasks.workunit.client.1.smithi148.stdout:9/709: dwrite d0/dd/d25/d37/d40/da3/fd3 [0,4194304] 0 2024-07-03T15:43:52.748 INFO:tasks.workunit.client.1.smithi148.stdout:9/710: creat d0/dd/d25/d37/d40/da3/fe0 x:0 0 0 2024-07-03T15:43:52.763 INFO:tasks.workunit.client.0.smithi028.stdout:9/20: rmdir d1 39 2024-07-03T15:43:52.764 INFO:tasks.workunit.client.0.smithi028.stdout:9/21: truncate d1/f2 418983 0 2024-07-03T15:43:52.764 INFO:tasks.workunit.client.0.smithi028.stdout:9/22: write d1/f5 [214907,117143] 0 2024-07-03T15:43:52.768 INFO:tasks.workunit.client.1.smithi148.stdout:9/711: dwrite d0/dd/d25/d37/d40/d72/f9d [0,4194304] 0 2024-07-03T15:43:52.775 INFO:tasks.workunit.client.1.smithi148.stdout:8/389: dwrite d4/da/d14/d1a/f37 [0,4194304] 0 2024-07-03T15:43:52.778 INFO:tasks.workunit.client.1.smithi148.stdout:8/390: stat d4/da/d51/d18/c3f 0 2024-07-03T15:43:52.781 INFO:tasks.workunit.client.1.smithi148.stdout:8/391: unlink d4/da/d14/f16 0 2024-07-03T15:43:52.783 INFO:tasks.workunit.client.1.smithi148.stdout:8/392: mkdir d4/da/d14/d1a/d32/d38/d58/d8e 0 2024-07-03T15:43:52.785 INFO:tasks.workunit.client.1.smithi148.stdout:8/393: mknod d4/da/d14/d1a/d32/d38/d58/d8e/c8f 0 2024-07-03T15:43:52.787 INFO:tasks.workunit.client.1.smithi148.stdout:8/394: symlink d4/da/d14/d1a/d32/d89/l90 0 2024-07-03T15:43:52.787 INFO:tasks.workunit.client.1.smithi148.stdout:8/395: write d4/da/d14/d17/d76/f7d [591437,78389] 0 2024-07-03T15:43:52.809 INFO:tasks.workunit.client.1.smithi148.stdout:0/432: dwrite da/d44/d24/f89 [0,4194304] 0 2024-07-03T15:43:52.809 INFO:tasks.workunit.client.1.smithi148.stdout:0/433: fdatasync da/f73 0 2024-07-03T15:43:52.812 INFO:tasks.workunit.client.1.smithi148.stdout:0/434: dread da/d17/d2a/f4b [0,4194304] 0 2024-07-03T15:43:52.812 INFO:tasks.workunit.client.1.smithi148.stdout:0/435: chown da/d44/d24 76166 1 2024-07-03T15:43:52.812 INFO:tasks.workunit.client.1.smithi148.stdout:0/436: chown da/d17/d2a/d2f/f37 1339411362 1 2024-07-03T15:43:52.812 INFO:tasks.workunit.client.1.smithi148.stdout:0/437: chown da/d44/d13/d14/d50/d95/d8c/l8d 107912107 1 2024-07-03T15:43:52.819 INFO:tasks.workunit.client.1.smithi148.stdout:0/438: write da/d17/d2a/f2e [2196152,19494] 0 2024-07-03T15:43:52.825 INFO:tasks.workunit.client.1.smithi148.stdout:0/439: link da/d17/d2a/f59 da/d44/d13/d14/d50/d95/f97 0 2024-07-03T15:43:52.828 INFO:tasks.workunit.client.1.smithi148.stdout:0/440: mkdir da/d44/d13/d14/d50/d95/d98 0 2024-07-03T15:43:52.828 INFO:tasks.workunit.client.1.smithi148.stdout:0/441: read da/d44/d13/d1d/d3e/f6e [661578,105061] 0 2024-07-03T15:43:52.828 INFO:tasks.workunit.client.1.smithi148.stdout:0/442: write da/d17/d2a/f2e [3077342,16266] 0 2024-07-03T15:43:52.843 INFO:tasks.workunit.client.1.smithi148.stdout:0/443: dwrite da/d44/d13/d14/d35/f5e [0,4194304] 0 2024-07-03T15:43:53.143 INFO:tasks.workunit.client.1.smithi148.stdout:7/517: sync 2024-07-03T15:43:53.143 INFO:tasks.workunit.client.1.smithi148.stdout:3/459: sync 2024-07-03T15:43:53.143 INFO:tasks.workunit.client.1.smithi148.stdout:1/359: sync 2024-07-03T15:43:53.143 INFO:tasks.workunit.client.1.smithi148.stdout:6/488: sync 2024-07-03T15:43:53.143 INFO:tasks.workunit.client.1.smithi148.stdout:2/559: sync 2024-07-03T15:43:53.143 INFO:tasks.workunit.client.1.smithi148.stdout:1/360: chown d0/d2/d6/c68 0 1 2024-07-03T15:43:53.144 INFO:tasks.workunit.client.1.smithi148.stdout:3/460: read dd/d35/f4f [166782,91629] 0 2024-07-03T15:43:53.144 INFO:tasks.workunit.client.1.smithi148.stdout:3/461: stat dd/d35/c43 0 2024-07-03T15:43:53.146 INFO:tasks.workunit.client.1.smithi148.stdout:6/489: rmdir d3/d1d/d28/d7f/d85/d9c 0 2024-07-03T15:43:53.147 INFO:tasks.workunit.client.1.smithi148.stdout:1/361: write d0/d2/d2e/d39/d42/d62/f4f [223484,69535] 0 2024-07-03T15:43:53.149 INFO:tasks.workunit.client.1.smithi148.stdout:7/518: mkdir d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/da8 0 2024-07-03T15:43:53.152 INFO:tasks.workunit.client.1.smithi148.stdout:3/462: rename dd/f21 to dd/d35/f92 0 2024-07-03T15:43:53.154 INFO:tasks.workunit.client.1.smithi148.stdout:6/490: truncate d3/f1e 2104131 0 2024-07-03T15:43:53.159 INFO:tasks.workunit.client.1.smithi148.stdout:3/463: getdents dd/d33/d37/d68/d2e/d40 0 2024-07-03T15:43:53.162 INFO:tasks.workunit.client.1.smithi148.stdout:7/519: rename d5/d7d/da7/d1c/d4a/d66/la6 to d5/d7d/da7/d1c/d33/d4c/d21/d47/la9 0 2024-07-03T15:43:53.163 INFO:tasks.workunit.client.1.smithi148.stdout:7/520: write d5/d7d/da7/d1c/d33/d4c/d21/f32 [198345,37361] 0 2024-07-03T15:43:53.163 INFO:tasks.workunit.client.1.smithi148.stdout:7/521: write d5/d7d/da7/d1c/d33/d4c/d21/d47/f88 [2795351,127223] 0 2024-07-03T15:43:53.163 INFO:tasks.workunit.client.1.smithi148.stdout:6/491: truncate d3/d1d/f24 3537563 0 2024-07-03T15:43:53.164 INFO:tasks.workunit.client.1.smithi148.stdout:2/560: dwrite d5/d14/d22/d60/f53 [4194304,4194304] 0 2024-07-03T15:43:53.164 INFO:tasks.workunit.client.1.smithi148.stdout:3/464: truncate dd/f2b 2090265 0 2024-07-03T15:43:53.164 INFO:tasks.workunit.client.1.smithi148.stdout:3/465: truncate dd/d35/f4f 896016 0 2024-07-03T15:43:53.165 INFO:tasks.workunit.client.1.smithi148.stdout:1/362: dwrite d0/d2/d2e/d39/d41/f43 [4194304,4194304] 0 2024-07-03T15:43:53.165 INFO:tasks.workunit.client.1.smithi148.stdout:1/363: readlink d0/l3f 0 2024-07-03T15:43:53.165 INFO:tasks.workunit.client.1.smithi148.stdout:1/364: write d0/d2/d2e/d39/d42/f48 [2013254,44223] 0 2024-07-03T15:43:53.168 INFO:tasks.workunit.client.1.smithi148.stdout:7/522: mkdir d5/d7d/da7/d1c/d33/d4c/d84/d91/daa 0 2024-07-03T15:43:53.170 INFO:tasks.workunit.client.1.smithi148.stdout:6/492: creat d3/d1d/d28/fa6 x:0 0 0 2024-07-03T15:43:53.170 INFO:tasks.workunit.client.1.smithi148.stdout:6/493: fdatasync d3/f17 0 2024-07-03T15:43:53.173 INFO:tasks.workunit.client.1.smithi148.stdout:2/561: link d5/d14/l19 d5/d5f/lb4 0 2024-07-03T15:43:53.175 INFO:tasks.workunit.client.1.smithi148.stdout:6/494: dread d3/fa [0,4194304] 0 2024-07-03T15:43:53.175 INFO:tasks.workunit.client.1.smithi148.stdout:6/495: chown d3/d1d/d28/d31/l65 33642677 1 2024-07-03T15:43:53.175 INFO:tasks.workunit.client.1.smithi148.stdout:3/466: symlink dd/l93 0 2024-07-03T15:43:53.179 INFO:tasks.workunit.client.1.smithi148.stdout:1/365: rename d0/d2/d6/d14/d29/c5c to d0/de/d2f/d72/c76 0 2024-07-03T15:43:53.180 INFO:tasks.workunit.client.1.smithi148.stdout:3/467: creat dd/d35/f94 x:0 0 0 2024-07-03T15:43:53.183 INFO:tasks.workunit.client.1.smithi148.stdout:3/468: dread dd/d33/d37/d68/f3c [0,4194304] 0 2024-07-03T15:43:53.183 INFO:tasks.workunit.client.1.smithi148.stdout:3/469: readlink lb 0 2024-07-03T15:43:53.185 INFO:tasks.workunit.client.1.smithi148.stdout:7/523: rename d5/f3b to d5/d7d/da7/fab 0 2024-07-03T15:43:53.185 INFO:tasks.workunit.client.1.smithi148.stdout:7/524: stat d5/d7d/da7/d1c/d33/d4c/d21/d39/f77 0 2024-07-03T15:43:53.186 INFO:tasks.workunit.client.1.smithi148.stdout:1/366: mkdir d0/d2/d2e/d39/d41/d77 0 2024-07-03T15:43:53.186 INFO:tasks.workunit.client.1.smithi148.stdout:1/367: read - d0/de/d2f/f4e zero size 2024-07-03T15:43:53.187 INFO:tasks.workunit.client.1.smithi148.stdout:3/470: symlink dd/d35/d81/l95 0 2024-07-03T15:43:53.190 INFO:tasks.workunit.client.1.smithi148.stdout:6/496: rename d3/d5b/f70 to d3/d50/d77/fa7 0 2024-07-03T15:43:53.190 INFO:tasks.workunit.client.1.smithi148.stdout:2/562: dwrite d5/d14/d22/d2b/f42 [0,4194304] 0 2024-07-03T15:43:53.190 INFO:tasks.workunit.client.1.smithi148.stdout:2/563: write d5/d14/f58 [1194214,18126] 0 2024-07-03T15:43:53.190 INFO:tasks.workunit.client.1.smithi148.stdout:2/564: chown d5/d14/d22/d2b/d13/d41/f63 254640 1 2024-07-03T15:43:53.190 INFO:tasks.workunit.client.1.smithi148.stdout:3/471: symlink dd/l96 0 2024-07-03T15:43:53.192 INFO:tasks.workunit.client.1.smithi148.stdout:7/525: rename d5/d7d/da7/d1c/d4a/d52/f6b to d5/d7d/da7/d1c/d23/fac 0 2024-07-03T15:43:53.192 INFO:tasks.workunit.client.1.smithi148.stdout:7/526: readlink d5/d7d/da7/d1c/d33/l51 0 2024-07-03T15:43:53.192 INFO:tasks.workunit.client.1.smithi148.stdout:7/527: chown d5/d7d/da7/d1c/d33/d4c/f99 33 1 2024-07-03T15:43:53.193 INFO:tasks.workunit.client.1.smithi148.stdout:6/497: truncate d3/d1d/d28/d31/d4c/f78 3191551 0 2024-07-03T15:43:53.195 INFO:tasks.workunit.client.1.smithi148.stdout:6/498: dread d3/d1d/f91 [0,4194304] 0 2024-07-03T15:43:53.197 INFO:tasks.workunit.client.1.smithi148.stdout:2/565: truncate d5/d14/d22/f72 3907984 0 2024-07-03T15:43:53.197 INFO:tasks.workunit.client.1.smithi148.stdout:2/566: write d5/d7/f83 [363143,22455] 0 2024-07-03T15:43:53.197 INFO:tasks.workunit.client.1.smithi148.stdout:7/528: dread d5/d7d/da7/d1c/d4a/d52/f5f [0,4194304] 0 2024-07-03T15:43:53.198 INFO:tasks.workunit.client.1.smithi148.stdout:3/472: creat dd/d35/d81/f97 x:0 0 0 2024-07-03T15:43:53.200 INFO:tasks.workunit.client.1.smithi148.stdout:6/499: creat d3/fa8 x:0 0 0 2024-07-03T15:43:53.200 INFO:tasks.workunit.client.1.smithi148.stdout:6/500: creat d3/fa9 x:0 0 0 2024-07-03T15:43:53.201 INFO:tasks.workunit.client.1.smithi148.stdout:2/567: creat d5/d14/d22/d60/d89/fb5 x:0 0 0 2024-07-03T15:43:53.201 INFO:tasks.workunit.client.1.smithi148.stdout:1/368: dwrite d0/d2/d2e/d39/d41/f43 [8388608,4194304] 0 2024-07-03T15:43:53.204 INFO:tasks.workunit.client.1.smithi148.stdout:7/529: truncate d5/d7d/da7/d1c/d4a/f62 1141557 0 2024-07-03T15:43:53.204 INFO:tasks.workunit.client.1.smithi148.stdout:7/530: write d5/d7d/da7/d1c/d33/d4c/d21/f3c [1544386,29660] 0 2024-07-03T15:43:53.204 INFO:tasks.workunit.client.1.smithi148.stdout:1/369: fdatasync d0/d2/d6/fa 0 2024-07-03T15:43:53.208 INFO:tasks.workunit.client.1.smithi148.stdout:6/501: mkdir d3/daa 0 2024-07-03T15:43:53.208 INFO:tasks.workunit.client.1.smithi148.stdout:3/473: read dd/f44 [104238,16520] 0 2024-07-03T15:43:53.209 INFO:tasks.workunit.client.1.smithi148.stdout:3/474: getdents dd/d35/d49/d8b 0 2024-07-03T15:43:53.211 INFO:tasks.workunit.client.1.smithi148.stdout:7/531: symlink d5/d7d/da7/d1c/d23/lad 0 2024-07-03T15:43:53.214 INFO:tasks.workunit.client.1.smithi148.stdout:1/370: symlink d0/de/l78 0 2024-07-03T15:43:53.220 INFO:tasks.workunit.client.1.smithi148.stdout:3/475: mknod dd/d33/d37/d68/d2e/d40/d77/c98 0 2024-07-03T15:43:53.223 INFO:tasks.workunit.client.1.smithi148.stdout:7/532: creat d5/d7d/da7/d1c/d33/d4c/d21/d31/fae x:0 0 0 2024-07-03T15:43:53.225 INFO:tasks.workunit.client.1.smithi148.stdout:1/371: symlink d0/de/d2f/d72/l79 0 2024-07-03T15:43:53.225 INFO:tasks.workunit.client.1.smithi148.stdout:7/533: truncate d5/d7d/da7/d1c/d33/d4c/d21/d47/f5d 4492973 0 2024-07-03T15:43:53.229 INFO:tasks.workunit.client.1.smithi148.stdout:7/534: rename d5/f6 to d5/d7d/da7/d1c/d4a/d66/d93/faf 0 2024-07-03T15:43:53.231 INFO:tasks.workunit.client.1.smithi148.stdout:7/535: creat d5/d7d/da7/d1c/d33/d4c/d21/d39/fb0 x:0 0 0 2024-07-03T15:43:53.234 INFO:tasks.workunit.client.1.smithi148.stdout:7/536: mkdir d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/db1 0 2024-07-03T15:43:53.236 INFO:tasks.workunit.client.1.smithi148.stdout:7/537: mkdir d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/db2 0 2024-07-03T15:43:53.238 INFO:tasks.workunit.client.1.smithi148.stdout:7/538: dread d5/d7d/da7/d1c/d33/d4c/d21/f29 [0,4194304] 0 2024-07-03T15:43:53.240 INFO:tasks.workunit.client.1.smithi148.stdout:7/539: creat d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/db2/fb3 x:0 0 0 2024-07-03T15:43:53.242 INFO:tasks.workunit.client.1.smithi148.stdout:7/540: mkdir d5/d7d/da7/db4 0 2024-07-03T15:43:53.244 INFO:tasks.workunit.client.1.smithi148.stdout:1/372: dwrite d0/d2/d2e/d39/d41/f69 [4194304,4194304] 0 2024-07-03T15:43:53.244 INFO:tasks.workunit.client.1.smithi148.stdout:1/373: chown d0/d2/d11/d1e/f3c 1088213 1 2024-07-03T15:43:53.245 INFO:tasks.workunit.client.1.smithi148.stdout:7/541: mknod d5/d7d/da7/d1c/d33/d4c/d21/cb5 0 2024-07-03T15:43:53.245 INFO:tasks.workunit.client.1.smithi148.stdout:7/542: truncate d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/db2/fb3 473285 0 2024-07-03T15:43:53.245 INFO:tasks.workunit.client.1.smithi148.stdout:7/543: dread - d5/f9f zero size 2024-07-03T15:43:53.245 INFO:tasks.workunit.client.1.smithi148.stdout:7/544: dread - d5/d7d/da7/d1c/d4a/d52/d87/f89 zero size 2024-07-03T15:43:53.246 INFO:tasks.workunit.client.1.smithi148.stdout:1/374: rmdir d0/d2/d6/d14/d29 39 2024-07-03T15:43:53.246 INFO:tasks.workunit.client.1.smithi148.stdout:1/375: creat d0/d2/d2e/d39/d41/f7a x:0 0 0 2024-07-03T15:43:53.247 INFO:tasks.workunit.client.1.smithi148.stdout:7/545: truncate d5/f1f 539761 0 2024-07-03T15:43:53.253 INFO:tasks.workunit.client.1.smithi148.stdout:0/444: getdents da/d44 0 2024-07-03T15:43:53.254 INFO:tasks.workunit.client.1.smithi148.stdout:1/376: read d0/d2/d11/d1e/f3c [348697,21461] 0 2024-07-03T15:43:53.255 INFO:tasks.workunit.client.1.smithi148.stdout:0/445: unlink da/d17/d2a/f2e 0 2024-07-03T15:43:53.256 INFO:tasks.workunit.client.1.smithi148.stdout:7/546: dread d5/d7d/da7/f34 [0,4194304] 0 2024-07-03T15:43:53.256 INFO:tasks.workunit.client.1.smithi148.stdout:7/547: fsync d5/d7d/da7/d1c/d59/f7c 0 2024-07-03T15:43:53.256 INFO:tasks.workunit.client.1.smithi148.stdout:0/446: write da/d44/d13/d1d/d3e/f6e [171792,93347] 0 2024-07-03T15:43:53.256 INFO:tasks.workunit.client.1.smithi148.stdout:0/447: dread - da/d38/f4e zero size 2024-07-03T15:43:53.257 INFO:tasks.workunit.client.1.smithi148.stdout:1/377: mkdir d0/d2/d2e/d39/d42/d62/d7b 0 2024-07-03T15:43:53.258 INFO:tasks.workunit.client.1.smithi148.stdout:7/548: mkdir d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/db6 0 2024-07-03T15:43:53.259 INFO:tasks.workunit.client.1.smithi148.stdout:9/712: rmdir d0/dd/d9f/dde 39 2024-07-03T15:43:53.262 INFO:tasks.workunit.client.1.smithi148.stdout:0/448: link da/d44/d13/d14/d35/f5e da/d44/d13/d14/f99 0 2024-07-03T15:43:53.262 INFO:tasks.workunit.client.1.smithi148.stdout:0/449: stat da/cb 0 2024-07-03T15:43:53.264 INFO:tasks.workunit.client.1.smithi148.stdout:1/378: link d0/d2/d2e/d39/f5f d0/de/d2f/d72/f7c 0 2024-07-03T15:43:53.266 INFO:tasks.workunit.client.1.smithi148.stdout:7/549: symlink d5/d7d/da7/d1c/d33/d4c/d21/d31/lb7 0 2024-07-03T15:43:53.266 INFO:tasks.workunit.client.1.smithi148.stdout:7/550: chown d5/d7d/da7/d1c/d33/l74 491255080 1 2024-07-03T15:43:53.269 INFO:tasks.workunit.client.1.smithi148.stdout:9/713: rename d0/d49/d43/d19/d8c to d0/d49/d43/d19/de1 0 2024-07-03T15:43:53.269 INFO:tasks.workunit.client.1.smithi148.stdout:9/714: write d0/dd/d25/d6a/f82 [2660533,128682] 0 2024-07-03T15:43:53.269 INFO:tasks.workunit.client.1.smithi148.stdout:9/715: stat d0/d63/f98 0 2024-07-03T15:43:53.270 INFO:tasks.workunit.client.1.smithi148.stdout:0/450: getdents da/d17/d2a/d2f 0 2024-07-03T15:43:53.277 INFO:tasks.workunit.client.1.smithi148.stdout:9/716: rename d0/d49/d43/f87 to d0/d18/dcf/fe2 0 2024-07-03T15:43:53.278 INFO:tasks.workunit.client.0.smithi028.stdout:6/5: symlink l1 0 2024-07-03T15:43:53.280 INFO:tasks.workunit.client.1.smithi148.stdout:0/451: creat da/d44/d24/d84/f9a x:0 0 0 2024-07-03T15:43:53.282 INFO:tasks.workunit.client.1.smithi148.stdout:1/379: mknod d0/d2/c7d 0 2024-07-03T15:43:53.284 INFO:tasks.workunit.client.1.smithi148.stdout:9/717: write d0/f53 [2754249,104235] 0 2024-07-03T15:43:53.285 INFO:tasks.workunit.client.0.smithi028.stdout:6/6: read f0 [701819,51892] 0 2024-07-03T15:43:53.285 INFO:tasks.workunit.client.0.smithi028.stdout:6/7: creat f2 x:0 0 0 2024-07-03T15:43:53.288 INFO:tasks.workunit.client.1.smithi148.stdout:0/452: unlink da/cb 0 2024-07-03T15:43:53.292 INFO:tasks.workunit.client.1.smithi148.stdout:9/718: creat d0/d49/d43/d19/de1/fe3 x:0 0 0 2024-07-03T15:43:53.292 INFO:tasks.workunit.client.1.smithi148.stdout:9/719: read d0/dd/d25/d39/fcd [3691645,81250] 0 2024-07-03T15:43:53.292 INFO:tasks.workunit.client.1.smithi148.stdout:9/720: chown d0/dd/d9f/cd1 1793168 1 2024-07-03T15:43:53.292 INFO:tasks.workunit.client.1.smithi148.stdout:9/721: read d0/d18/f21 [6905546,99895] 0 2024-07-03T15:43:53.294 INFO:tasks.workunit.client.1.smithi148.stdout:0/453: mknod da/d36/d5b/c9b 0 2024-07-03T15:43:53.294 INFO:tasks.workunit.client.1.smithi148.stdout:0/454: chown da/d17/f1e 945 1 2024-07-03T15:43:53.295 INFO:tasks.workunit.client.1.smithi148.stdout:1/380: dwrite d0/d2/d11/d1e/f59 [0,4194304] 0 2024-07-03T15:43:53.303 INFO:tasks.workunit.client.1.smithi148.stdout:1/381: unlink d0/de/c1d 0 2024-07-03T15:43:53.304 INFO:tasks.workunit.client.1.smithi148.stdout:0/455: dread da/d44/d13/d14/f1f [0,4194304] 0 2024-07-03T15:43:53.306 INFO:tasks.workunit.client.1.smithi148.stdout:1/382: mknod d0/d2/d2e/d39/d42/c7e 0 2024-07-03T15:43:53.309 INFO:tasks.workunit.client.1.smithi148.stdout:1/383: mknod d0/d2/d6/d14/c7f 0 2024-07-03T15:43:53.309 INFO:tasks.workunit.client.1.smithi148.stdout:1/384: dread - d0/d2/d2e/d39/d41/f7a zero size 2024-07-03T15:43:53.311 INFO:tasks.workunit.client.1.smithi148.stdout:1/385: mkdir d0/d2/d2e/d39/d41/d77/d80 0 2024-07-03T15:43:53.313 INFO:tasks.workunit.client.1.smithi148.stdout:1/386: mknod d0/de/d2f/d72/c81 0 2024-07-03T15:43:53.313 INFO:tasks.workunit.client.1.smithi148.stdout:0/456: read da/d17/f1e [2251122,98270] 0 2024-07-03T15:43:53.313 INFO:tasks.workunit.client.1.smithi148.stdout:0/457: chown da/d44/d13/d14/l65 37875010 1 2024-07-03T15:43:53.313 INFO:tasks.workunit.client.1.smithi148.stdout:0/458: chown da/d58/f79 177856357 1 2024-07-03T15:43:53.313 INFO:tasks.workunit.client.1.smithi148.stdout:0/459: fsync da/d44/d13/d14/f1f 0 2024-07-03T15:43:53.314 INFO:tasks.workunit.client.1.smithi148.stdout:0/460: dread - da/d38/f7c zero size 2024-07-03T15:43:53.315 INFO:tasks.workunit.client.1.smithi148.stdout:1/387: link d0/d2/d2e/c67 d0/de/d2f/d72/c82 0 2024-07-03T15:43:53.316 INFO:tasks.workunit.client.1.smithi148.stdout:0/461: creat da/d17/d2a/d2f/d93/f9c x:0 0 0 2024-07-03T15:43:53.325 INFO:tasks.workunit.client.1.smithi148.stdout:0/462: write da/d36/f63 [563086,42982] 0 2024-07-03T15:43:53.326 INFO:tasks.workunit.client.1.smithi148.stdout:8/396: rmdir d4/da/d14 39 2024-07-03T15:43:53.329 INFO:tasks.workunit.client.1.smithi148.stdout:0/463: creat da/d44/f9d x:0 0 0 2024-07-03T15:43:53.330 INFO:tasks.workunit.client.1.smithi148.stdout:0/464: dread da/d36/d5b/f72 [0,4194304] 0 2024-07-03T15:43:53.333 INFO:tasks.workunit.client.1.smithi148.stdout:0/465: unlink da/d44/d13/d14/f32 0 2024-07-03T15:43:53.335 INFO:tasks.workunit.client.1.smithi148.stdout:0/466: dread da/d17/d2a/d2f/f37 [0,4194304] 0 2024-07-03T15:43:53.338 INFO:tasks.workunit.client.1.smithi148.stdout:0/467: getdents da/d58 0 2024-07-03T15:43:53.339 INFO:tasks.workunit.client.1.smithi148.stdout:0/468: truncate da/d44/d13/d14/f1f 561451 0 2024-07-03T15:43:53.341 INFO:tasks.workunit.client.1.smithi148.stdout:0/469: creat da/d17/d2a/d2f/f9e x:0 0 0 2024-07-03T15:43:53.343 INFO:tasks.workunit.client.1.smithi148.stdout:0/470: mknod da/c9f 0 2024-07-03T15:43:53.343 INFO:tasks.workunit.client.1.smithi148.stdout:0/471: write da/d38/f7c [532708,80383] 0 2024-07-03T15:43:53.345 INFO:tasks.workunit.client.1.smithi148.stdout:0/472: symlink da/d17/la0 0 2024-07-03T15:43:53.345 INFO:tasks.workunit.client.1.smithi148.stdout:0/473: chown da/f73 90 1 2024-07-03T15:43:53.359 INFO:tasks.workunit.client.1.smithi148.stdout:4/362: sync 2024-07-03T15:43:53.359 INFO:tasks.workunit.client.1.smithi148.stdout:4/363: fsync d0/d8/f2d 0 2024-07-03T15:43:53.359 INFO:tasks.workunit.client.1.smithi148.stdout:5/475: sync 2024-07-03T15:43:53.359 INFO:tasks.workunit.client.1.smithi148.stdout:5/476: chown d7/da/df/d16/d2f/d44/f7c 7 1 2024-07-03T15:43:53.389 INFO:tasks.workunit.client.1.smithi148.stdout:4/364: dwrite d0/d8/d44/f6a [0,4194304] 0 2024-07-03T15:43:53.389 INFO:tasks.workunit.client.1.smithi148.stdout:4/365: fsync d0/d8/d22/d62/f6b 0 2024-07-03T15:43:53.389 INFO:tasks.workunit.client.1.smithi148.stdout:5/477: dwrite d7/da/df/d16/d2f/d44/f70 [0,4194304] 0 2024-07-03T15:43:53.389 INFO:tasks.workunit.client.1.smithi148.stdout:5/478: readlink d7/l8e 0 2024-07-03T15:43:53.395 INFO:tasks.workunit.client.1.smithi148.stdout:5/479: unlink d7/d13/d23/d40/f56 0 2024-07-03T15:43:53.398 INFO:tasks.workunit.client.1.smithi148.stdout:5/480: symlink d7/da/df/l9f 0 2024-07-03T15:43:53.400 INFO:tasks.workunit.client.1.smithi148.stdout:5/481: mkdir d7/da/df/da0 0 2024-07-03T15:43:53.402 INFO:tasks.workunit.client.1.smithi148.stdout:5/482: creat d7/da/df/d16/d6a/fa1 x:0 0 0 2024-07-03T15:43:53.405 INFO:tasks.workunit.client.1.smithi148.stdout:5/483: rename d7/da/df/d16/d1a/c6e to d7/d13/d54/ca2 0 2024-07-03T15:43:53.405 INFO:tasks.workunit.client.1.smithi148.stdout:5/484: fdatasync d7/f19 0 2024-07-03T15:43:53.407 INFO:tasks.workunit.client.1.smithi148.stdout:4/366: dwrite d0/d8/d22/d2f/d41/f4d [0,4194304] 0 2024-07-03T15:43:53.407 INFO:tasks.workunit.client.1.smithi148.stdout:4/367: write d0/d28/f63 [4957875,58628] 0 2024-07-03T15:43:53.407 INFO:tasks.workunit.client.1.smithi148.stdout:4/368: chown d0/d8/d19 2342544 1 2024-07-03T15:43:53.407 INFO:tasks.workunit.client.1.smithi148.stdout:5/485: creat d7/da/df/d16/d2f/d44/d52/d86/fa3 x:0 0 0 2024-07-03T15:43:53.410 INFO:tasks.workunit.client.1.smithi148.stdout:4/369: symlink d0/d8/d22/d2f/d41/l76 0 2024-07-03T15:43:53.413 INFO:tasks.workunit.client.1.smithi148.stdout:4/370: symlink d0/d8/d22/d2f/d41/l77 0 2024-07-03T15:43:53.414 INFO:tasks.workunit.client.1.smithi148.stdout:4/371: mknod d0/d8/d19/c78 0 2024-07-03T15:43:53.416 INFO:tasks.workunit.client.1.smithi148.stdout:4/372: rename d0/d8/d19/c65 to d0/d8/d22/c79 0 2024-07-03T15:43:53.423 INFO:tasks.workunit.client.1.smithi148.stdout:4/373: dread d0/d8/d22/d2f/d41/f4d [0,4194304] 0 2024-07-03T15:43:53.425 INFO:tasks.workunit.client.1.smithi148.stdout:4/374: creat d0/d8/d22/d2f/d41/f7a x:0 0 0 2024-07-03T15:43:53.425 INFO:tasks.workunit.client.1.smithi148.stdout:4/375: fsync d0/d28/f69 0 2024-07-03T15:43:53.426 INFO:tasks.workunit.client.1.smithi148.stdout:4/376: link d0/d8/d13/c12 d0/d8/d13/c7b 0 2024-07-03T15:43:53.428 INFO:tasks.workunit.client.1.smithi148.stdout:4/377: mkdir d0/d8/d3a/d36/d6f/d7c 0 2024-07-03T15:43:53.437 INFO:tasks.workunit.client.1.smithi148.stdout:4/378: dread d0/f4 [0,4194304] 0 2024-07-03T15:43:53.444 INFO:tasks.workunit.client.1.smithi148.stdout:4/379: read d0/f3d [3421099,60014] 0 2024-07-03T15:43:53.578 INFO:tasks.workunit.client.0.smithi028.stdout:2/21: symlink l8 0 2024-07-03T15:43:53.578 INFO:tasks.workunit.client.0.smithi028.stdout:2/22: chown l6 53944 1 2024-07-03T15:43:53.578 INFO:tasks.workunit.client.0.smithi028.stdout:2/23: chown c2 115213459 1 2024-07-03T15:43:53.578 INFO:tasks.workunit.client.0.smithi028.stdout:2/24: creat f9 x:0 0 0 2024-07-03T15:43:53.578 INFO:tasks.workunit.client.0.smithi028.stdout:3/4: symlink l1 0 2024-07-03T15:43:53.595 INFO:tasks.workunit.client.0.smithi028.stdout:7/4: mknod c1 0 2024-07-03T15:43:53.595 INFO:tasks.workunit.client.0.smithi028.stdout:7/5: fdatasync f0 0 2024-07-03T15:43:53.596 INFO:tasks.workunit.client.0.smithi028.stdout:8/11: symlink d0/d2/l5 0 2024-07-03T15:43:53.601 INFO:tasks.workunit.client.0.smithi028.stdout:9/23: unlink d1/f3 0 2024-07-03T15:43:53.604 INFO:tasks.workunit.client.0.smithi028.stdout:7/6: dread f0 [0,4194304] 0 2024-07-03T15:43:53.604 INFO:tasks.workunit.client.0.smithi028.stdout:7/7: fdatasync f0 0 2024-07-03T15:43:53.604 INFO:tasks.workunit.client.0.smithi028.stdout:7/8: chown f0 10 1 2024-07-03T15:43:53.609 INFO:tasks.workunit.client.0.smithi028.stdout:6/8: mknod c3 0 2024-07-03T15:43:53.609 INFO:tasks.workunit.client.0.smithi028.stdout:6/9: chown c3 20546 1 2024-07-03T15:43:53.610 INFO:tasks.workunit.client.0.smithi028.stdout:5/8: rename d0/f2 to d0/f3 0 2024-07-03T15:43:53.613 INFO:tasks.workunit.client.0.smithi028.stdout:2/25: mkdir da 0 2024-07-03T15:43:53.614 INFO:tasks.workunit.client.0.smithi028.stdout:3/5: creat f2 x:0 0 0 2024-07-03T15:43:53.616 INFO:tasks.workunit.client.0.smithi028.stdout:1/2: rename f0 to f1 0 2024-07-03T15:43:53.616 INFO:tasks.workunit.client.0.smithi028.stdout:1/3: chown f1 692 1 2024-07-03T15:43:53.617 INFO:tasks.workunit.client.0.smithi028.stdout:9/24: creat d1/f6 x:0 0 0 2024-07-03T15:43:53.617 INFO:tasks.workunit.client.0.smithi028.stdout:9/25: read d1/f2 [42490,67372] 0 2024-07-03T15:43:53.619 INFO:tasks.workunit.client.0.smithi028.stdout:2/26: rename l6 to da/lb 0 2024-07-03T15:43:53.620 INFO:tasks.workunit.client.0.smithi028.stdout:3/6: creat f3 x:0 0 0 2024-07-03T15:43:53.621 INFO:tasks.workunit.client.0.smithi028.stdout:8/12: symlink d0/d2/d3/l6 0 2024-07-03T15:43:53.621 INFO:tasks.workunit.client.0.smithi028.stdout:8/13: dread - no filename 2024-07-03T15:43:53.621 INFO:tasks.workunit.client.0.smithi028.stdout:8/14: chown d0/d2 64209 1 2024-07-03T15:43:53.621 INFO:tasks.workunit.client.0.smithi028.stdout:8/15: chown d0/d2/d3/l6 292151448 1 2024-07-03T15:43:53.622 INFO:tasks.workunit.client.0.smithi028.stdout:9/26: symlink d1/l7 0 2024-07-03T15:43:53.622 INFO:tasks.workunit.client.0.smithi028.stdout:9/27: chown d1/l4 119 1 2024-07-03T15:43:53.623 INFO:tasks.workunit.client.0.smithi028.stdout:9/28: chown d1/f5 5975 1 2024-07-03T15:43:53.623 INFO:tasks.workunit.client.0.smithi028.stdout:9/29: dread - d1/f6 zero size 2024-07-03T15:43:53.623 INFO:tasks.workunit.client.0.smithi028.stdout:9/30: truncate d1/f6 10697 0 2024-07-03T15:43:53.624 INFO:tasks.workunit.client.0.smithi028.stdout:8/16: creat d0/d2/f7 x:0 0 0 2024-07-03T15:43:53.625 INFO:tasks.workunit.client.0.smithi028.stdout:8/17: symlink d0/l8 0 2024-07-03T15:43:53.626 INFO:tasks.workunit.client.0.smithi028.stdout:2/27: dread f7 [0,4194304] 0 2024-07-03T15:43:53.627 INFO:tasks.workunit.client.0.smithi028.stdout:2/28: chown f7 231037985 1 2024-07-03T15:43:53.630 INFO:tasks.workunit.client.0.smithi028.stdout:2/29: dread f4 [0,4194304] 0 2024-07-03T15:43:53.630 INFO:tasks.workunit.client.0.smithi028.stdout:2/30: stat da 0 2024-07-03T15:43:53.631 INFO:tasks.workunit.client.0.smithi028.stdout:2/31: creat da/fc x:0 0 0 2024-07-03T15:43:53.631 INFO:tasks.workunit.client.0.smithi028.stdout:2/32: chown l8 340 1 2024-07-03T15:43:53.631 INFO:tasks.workunit.client.0.smithi028.stdout:2/33: chown da 6316 1 2024-07-03T15:43:53.631 INFO:tasks.workunit.client.0.smithi028.stdout:2/34: symlink da/ld 0 2024-07-03T15:43:53.633 INFO:tasks.workunit.client.0.smithi028.stdout:2/35: rename da/fc to da/fe 0 2024-07-03T15:43:53.634 INFO:tasks.workunit.client.0.smithi028.stdout:6/10: dwrite f0 [0,4194304] 0 2024-07-03T15:43:53.635 INFO:tasks.workunit.client.0.smithi028.stdout:2/36: dread f4 [0,4194304] 0 2024-07-03T15:43:53.636 INFO:tasks.workunit.client.0.smithi028.stdout:2/37: truncate f7 1339601 0 2024-07-03T15:43:53.638 INFO:tasks.workunit.client.0.smithi028.stdout:2/38: dread f7 [0,4194304] 0 2024-07-03T15:43:53.638 INFO:tasks.workunit.client.0.smithi028.stdout:2/39: chown l1 26900306 1 2024-07-03T15:43:53.638 INFO:tasks.workunit.client.0.smithi028.stdout:2/40: stat da 0 2024-07-03T15:43:53.638 INFO:tasks.workunit.client.0.smithi028.stdout:2/41: chown f4 119 1 2024-07-03T15:43:53.645 INFO:tasks.workunit.client.0.smithi028.stdout:8/18: dwrite d0/d2/f7 [0,4194304] 0 2024-07-03T15:43:53.645 INFO:tasks.workunit.client.0.smithi028.stdout:9/31: dwrite d1/f6 [0,4194304] 0 2024-07-03T15:43:53.651 INFO:tasks.workunit.client.0.smithi028.stdout:8/19: dread d0/d2/f7 [0,4194304] 0 2024-07-03T15:43:53.653 INFO:tasks.workunit.client.0.smithi028.stdout:8/20: rename d0/d2/f7 to d0/d2/d3/f9 0 2024-07-03T15:43:53.676 INFO:tasks.workunit.client.1.smithi148.stdout:7/551: getdents d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97 0 2024-07-03T15:43:53.678 INFO:tasks.workunit.client.1.smithi148.stdout:7/552: dread d5/d7d/da7/d1c/d33/d4c/d21/f3c [0,4194304] 0 2024-07-03T15:43:53.680 INFO:tasks.workunit.client.1.smithi148.stdout:7/553: symlink d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/db6/lb8 0 2024-07-03T15:43:53.681 INFO:tasks.workunit.client.1.smithi148.stdout:7/554: mkdir d5/d7d/da7/d1c/d33/d4c/d84/d91/daa/db9 0 2024-07-03T15:43:53.681 INFO:tasks.workunit.client.1.smithi148.stdout:7/555: creat d5/d7d/da7/d1c/d4a/d9c/fba x:0 0 0 2024-07-03T15:43:53.682 INFO:tasks.workunit.client.1.smithi148.stdout:7/556: creat d5/d7d/da7/d1c/d23/fbb x:0 0 0 2024-07-03T15:43:53.683 INFO:tasks.workunit.client.1.smithi148.stdout:7/557: truncate d5/d7d/da7/d1c/d23/f68 2892312 0 2024-07-03T15:43:53.684 INFO:tasks.workunit.client.1.smithi148.stdout:7/558: mknod d5/d7d/da7/d1c/d33/d4c/d84/cbc 0 2024-07-03T15:43:53.684 INFO:tasks.workunit.client.1.smithi148.stdout:7/559: creat d5/d7d/da7/d1c/d23/fbd x:0 0 0 2024-07-03T15:43:53.686 INFO:tasks.workunit.client.1.smithi148.stdout:7/560: creat d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/db6/fbe x:0 0 0 2024-07-03T15:43:53.694 INFO:tasks.workunit.client.1.smithi148.stdout:1/388: truncate d0/d2/d11/d1e/f59 2152308 0 2024-07-03T15:43:53.694 INFO:tasks.workunit.client.1.smithi148.stdout:1/389: dread - d0/d2/d6/d14/f6a zero size 2024-07-03T15:43:53.696 INFO:tasks.workunit.client.1.smithi148.stdout:1/390: link d0/d2/d2e/d39/d42/f48 d0/de/f83 0 2024-07-03T15:43:53.696 INFO:tasks.workunit.client.1.smithi148.stdout:1/391: chown d0/d2/d2e 978 1 2024-07-03T15:43:53.698 INFO:tasks.workunit.client.1.smithi148.stdout:1/392: getdents d0/d2/d11/d3a 0 2024-07-03T15:43:53.702 INFO:tasks.workunit.client.1.smithi148.stdout:7/561: dwrite d5/d7d/da7/f4b [0,4194304] 0 2024-07-03T15:43:53.703 INFO:tasks.workunit.client.1.smithi148.stdout:7/562: write d5/fa [1105015,8887] 0 2024-07-03T15:43:53.703 INFO:tasks.workunit.client.1.smithi148.stdout:7/563: write d5/d7d/da7/d1c/d33/d4c/f26 [2372607,83429] 0 2024-07-03T15:43:53.707 INFO:tasks.workunit.client.1.smithi148.stdout:7/564: creat d5/d7d/da7/d1c/d33/d4c/d84/fbf x:0 0 0 2024-07-03T15:43:53.712 INFO:tasks.workunit.client.1.smithi148.stdout:7/565: unlink d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/f95 0 2024-07-03T15:43:53.715 INFO:tasks.workunit.client.1.smithi148.stdout:7/566: mknod d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/cc0 0 2024-07-03T15:43:53.716 INFO:tasks.workunit.client.1.smithi148.stdout:1/393: dwrite d0/d2/d2e/d39/d42/f48 [0,4194304] 0 2024-07-03T15:43:53.716 INFO:tasks.workunit.client.1.smithi148.stdout:1/394: truncate d0/d2/d2e/d39/f5f 7513 0 2024-07-03T15:43:53.717 INFO:tasks.workunit.client.1.smithi148.stdout:7/567: link d5/d7d/da7/d1c/f71 d5/d7d/da7/d1c/d23/fc1 0 2024-07-03T15:43:53.717 INFO:tasks.workunit.client.1.smithi148.stdout:7/568: chown d5/d7d/da7/fa5 153 1 2024-07-03T15:43:53.719 INFO:tasks.workunit.client.1.smithi148.stdout:1/395: creat d0/d2/d2e/d39/d41/d77/f84 x:0 0 0 2024-07-03T15:43:53.720 INFO:tasks.workunit.client.1.smithi148.stdout:7/569: unlink d5/d7d/da7/d1c/f2d 0 2024-07-03T15:43:53.720 INFO:tasks.workunit.client.1.smithi148.stdout:7/570: write d5/d7d/da7/d1c/d33/f67 [4166679,32699] 0 2024-07-03T15:43:53.721 INFO:tasks.workunit.client.1.smithi148.stdout:7/571: mkdir d5/d7d/da7/d1c/d33/d4c/d84/d98/dc2 0 2024-07-03T15:43:53.721 INFO:tasks.workunit.client.1.smithi148.stdout:7/572: fdatasync d5/d7d/da7/d1c/d4a/d52/f5f 0 2024-07-03T15:43:53.722 INFO:tasks.workunit.client.1.smithi148.stdout:7/573: symlink d5/d7d/da7/d1c/d33/d4c/d21/d31/lc3 0 2024-07-03T15:43:53.725 INFO:tasks.workunit.client.1.smithi148.stdout:1/396: dread d0/d2/d11/f47 [0,4194304] 0 2024-07-03T15:43:53.726 INFO:tasks.workunit.client.1.smithi148.stdout:1/397: creat d0/d2/d2e/f85 x:0 0 0 2024-07-03T15:43:53.726 INFO:tasks.workunit.client.1.smithi148.stdout:1/398: write d0/d2/d2e/d39/f5f [287739,39941] 0 2024-07-03T15:43:53.727 INFO:tasks.workunit.client.1.smithi148.stdout:1/399: creat d0/de/d2f/d72/f86 x:0 0 0 2024-07-03T15:43:53.729 INFO:tasks.workunit.client.1.smithi148.stdout:1/400: creat d0/d2/d2e/d39/d42/f87 x:0 0 0 2024-07-03T15:43:53.730 INFO:tasks.workunit.client.1.smithi148.stdout:1/401: symlink d0/l88 0 2024-07-03T15:43:53.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:53 smithi148 ceph-mon[36056]: pgmap v239: 65 pgs: 65 active+clean; 954 MiB data, 4.4 GiB used, 532 GiB / 536 GiB avail; 87 MiB/s rd, 112 MiB/s wr, 743 op/s 2024-07-03T15:43:53.738 INFO:tasks.workunit.client.1.smithi148.stdout:1/402: write d0/d2/d6/f26 [8623348,100976] 0 2024-07-03T15:43:53.738 INFO:tasks.workunit.client.1.smithi148.stdout:1/403: write d0/de/d2f/d72/f7c [1311024,26956] 0 2024-07-03T15:43:53.740 INFO:tasks.workunit.client.1.smithi148.stdout:7/574: dwrite d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/f28 [0,4194304] 0 2024-07-03T15:43:53.740 INFO:tasks.workunit.client.1.smithi148.stdout:1/404: mkdir d0/de/d89 0 2024-07-03T15:43:53.741 INFO:tasks.workunit.client.1.smithi148.stdout:7/575: mknod d5/d7d/da7/cc4 0 2024-07-03T15:43:53.742 INFO:tasks.workunit.client.1.smithi148.stdout:7/576: symlink d5/d7d/da7/d1c/d33/d4c/d21/d47/lc5 0 2024-07-03T15:43:53.743 INFO:tasks.workunit.client.1.smithi148.stdout:1/405: unlink d0/d2/c7 0 2024-07-03T15:43:53.747 INFO:tasks.workunit.client.1.smithi148.stdout:7/577: write d5/d7d/da7/d1c/d59/f76 [138108,112935] 0 2024-07-03T15:43:53.757 INFO:tasks.workunit.client.1.smithi148.stdout:4/380: rename d0/d8/d3a/d36 to d0/d28/d7d 0 2024-07-03T15:43:53.758 INFO:tasks.workunit.client.1.smithi148.stdout:4/381: rename d0/d28/d7d/f3e to d0/d8/d22/d2f/d41/f7e 0 2024-07-03T15:43:53.758 INFO:tasks.workunit.client.1.smithi148.stdout:4/382: chown d0/f4 388651783 1 2024-07-03T15:43:53.758 INFO:tasks.workunit.client.1.smithi148.stdout:1/406: dwrite d0/d2/d2e/f85 [0,4194304] 0 2024-07-03T15:43:53.759 INFO:tasks.workunit.client.1.smithi148.stdout:4/383: fsync d0/f50 0 2024-07-03T15:43:53.759 INFO:tasks.workunit.client.1.smithi148.stdout:4/384: chown d0/d8/d13/c5f 41110 1 2024-07-03T15:43:53.759 INFO:tasks.workunit.client.1.smithi148.stdout:1/407: mknod d0/de/d2f/c8a 0 2024-07-03T15:43:53.760 INFO:tasks.workunit.client.1.smithi148.stdout:1/408: read - d0/d2/d6/d14/f6a zero size 2024-07-03T15:43:53.761 INFO:tasks.workunit.client.1.smithi148.stdout:4/385: creat d0/d8/d3a/f7f x:0 0 0 2024-07-03T15:43:53.761 INFO:tasks.workunit.client.1.smithi148.stdout:4/386: dread - d0/d8/d19/d45/f75 zero size 2024-07-03T15:43:53.762 INFO:tasks.workunit.client.1.smithi148.stdout:1/409: creat d0/de/d44/d54/d5d/f8b x:0 0 0 2024-07-03T15:43:53.762 INFO:tasks.workunit.client.1.smithi148.stdout:1/410: creat d0/de/d44/d54/d5d/f8c x:0 0 0 2024-07-03T15:43:53.762 INFO:tasks.workunit.client.1.smithi148.stdout:1/411: read d0/f1f [3459727,19645] 0 2024-07-03T15:43:53.768 INFO:tasks.workunit.client.1.smithi148.stdout:4/387: write d0/f5 [1893658,30956] 0 2024-07-03T15:43:53.770 INFO:tasks.workunit.client.1.smithi148.stdout:4/388: symlink d0/d8/d13/d67/l80 0 2024-07-03T15:43:53.770 INFO:tasks.workunit.client.1.smithi148.stdout:4/389: chown d0/d8/d13/f1a 369898 1 2024-07-03T15:43:53.770 INFO:tasks.workunit.client.1.smithi148.stdout:4/390: readlink d0/d28/l43 0 2024-07-03T15:43:53.770 INFO:tasks.workunit.client.1.smithi148.stdout:4/391: chown d0/l5a 61980 1 2024-07-03T15:43:53.771 INFO:tasks.workunit.client.1.smithi148.stdout:4/392: creat d0/d8/d13/d48/f81 x:0 0 0 2024-07-03T15:43:53.771 INFO:tasks.workunit.client.1.smithi148.stdout:4/393: write d0/d8/d22/d2f/d41/f7e [1846844,32420] 0 2024-07-03T15:43:53.771 INFO:tasks.workunit.client.1.smithi148.stdout:1/412: write d0/d2/d2e/d39/d42/f48 [6699648,29864] 0 2024-07-03T15:43:53.772 INFO:tasks.workunit.client.1.smithi148.stdout:4/394: creat d0/d28/d7d/d6f/f82 x:0 0 0 2024-07-03T15:43:53.802 INFO:tasks.workunit.client.1.smithi148.stdout:4/395: dwrite d0/d8/d13/d48/f2c [0,4194304] 0 2024-07-03T15:43:53.802 INFO:tasks.workunit.client.1.smithi148.stdout:4/396: getdents d0/d28/d6e 0 2024-07-03T15:43:53.803 INFO:tasks.workunit.client.1.smithi148.stdout:1/413: dwrite d0/d2/d2e/d39/d42/f49 [0,4194304] 0 2024-07-03T15:43:53.803 INFO:tasks.workunit.client.1.smithi148.stdout:1/414: fdatasync d0/d2/d2e/d39/d42/f87 0 2024-07-03T15:43:53.804 INFO:tasks.workunit.client.1.smithi148.stdout:4/397: symlink d0/d8/d44/l83 0 2024-07-03T15:43:53.804 INFO:tasks.workunit.client.1.smithi148.stdout:4/398: creat d0/d8/d22/d2f/f84 x:0 0 0 2024-07-03T15:43:53.805 INFO:tasks.workunit.client.1.smithi148.stdout:4/399: symlink d0/d8/d44/l85 0 2024-07-03T15:43:53.806 INFO:tasks.workunit.client.1.smithi148.stdout:4/400: creat d0/d8/d13/d67/f86 x:0 0 0 2024-07-03T15:43:53.807 INFO:tasks.workunit.client.1.smithi148.stdout:4/401: mknod d0/d8/d19/d45/d49/c87 0 2024-07-03T15:43:53.807 INFO:tasks.workunit.client.1.smithi148.stdout:4/402: fdatasync d0/f4 0 2024-07-03T15:43:53.808 INFO:tasks.workunit.client.1.smithi148.stdout:4/403: rename d0/d8/d22/c79 to d0/d8/c88 0 2024-07-03T15:43:53.820 INFO:tasks.workunit.client.0.smithi028.stdout:5/9: rmdir d0 39 2024-07-03T15:43:53.821 INFO:tasks.workunit.client.0.smithi028.stdout:5/10: unlink d0/f3 0 2024-07-03T15:43:53.821 INFO:tasks.workunit.client.0.smithi028.stdout:5/11: truncate - no filename 2024-07-03T15:43:53.821 INFO:tasks.workunit.client.0.smithi028.stdout:5/12: readlink - no filename 2024-07-03T15:43:53.821 INFO:tasks.workunit.client.0.smithi028.stdout:5/13: readlink - no filename 2024-07-03T15:43:53.822 INFO:tasks.workunit.client.1.smithi148.stdout:4/404: dwrite d0/d8/f4b [0,4194304] 0 2024-07-03T15:43:53.822 INFO:tasks.workunit.client.0.smithi028.stdout:5/14: mknod d0/c4 0 2024-07-03T15:43:53.822 INFO:tasks.workunit.client.0.smithi028.stdout:5/15: dread - no filename 2024-07-03T15:43:53.822 INFO:tasks.workunit.client.0.smithi028.stdout:5/16: truncate - no filename 2024-07-03T15:43:53.822 INFO:tasks.workunit.client.0.smithi028.stdout:5/17: truncate - no filename 2024-07-03T15:43:53.822 INFO:tasks.workunit.client.0.smithi028.stdout:5/18: chown d0/c4 52099 1 2024-07-03T15:43:53.823 INFO:tasks.workunit.client.0.smithi028.stdout:1/4: fsync f1 0 2024-07-03T15:43:53.824 INFO:tasks.workunit.client.0.smithi028.stdout:5/19: mknod d0/c5 0 2024-07-03T15:43:53.824 INFO:tasks.workunit.client.0.smithi028.stdout:5/20: dwrite - no filename 2024-07-03T15:43:53.824 INFO:tasks.workunit.client.0.smithi028.stdout:5/21: write - no filename 2024-07-03T15:43:53.824 INFO:tasks.workunit.client.0.smithi028.stdout:5/22: dread - no filename 2024-07-03T15:43:53.824 INFO:tasks.workunit.client.0.smithi028.stdout:5/23: write - no filename 2024-07-03T15:43:53.825 INFO:tasks.workunit.client.0.smithi028.stdout:5/24: dwrite - no filename 2024-07-03T15:43:53.825 INFO:tasks.workunit.client.0.smithi028.stdout:5/25: write - no filename 2024-07-03T15:43:53.825 INFO:tasks.workunit.client.0.smithi028.stdout:5/26: link d0/c4 d0/c6 0 2024-07-03T15:43:53.826 INFO:tasks.workunit.client.0.smithi028.stdout:5/27: creat d0/f7 x:0 0 0 2024-07-03T15:43:53.826 INFO:tasks.workunit.client.0.smithi028.stdout:3/7: getdents . 0 2024-07-03T15:43:53.828 INFO:tasks.workunit.client.0.smithi028.stdout:5/28: rename d0/c6 to d0/c8 0 2024-07-03T15:43:53.828 INFO:tasks.workunit.client.0.smithi028.stdout:5/29: creat d0/f9 x:0 0 0 2024-07-03T15:43:53.829 INFO:tasks.workunit.client.0.smithi028.stdout:3/8: rename f3 to f4 0 2024-07-03T15:43:53.832 INFO:tasks.workunit.client.0.smithi028.stdout:5/30: mknod d0/ca 0 2024-07-03T15:43:53.832 INFO:tasks.workunit.client.0.smithi028.stdout:5/31: truncate d0/f9 891689 0 2024-07-03T15:43:53.832 INFO:tasks.workunit.client.0.smithi028.stdout:5/32: write d0/f7 [458090,123752] 0 2024-07-03T15:43:53.832 INFO:tasks.workunit.client.0.smithi028.stdout:3/9: creat f5 x:0 0 0 2024-07-03T15:43:53.833 INFO:tasks.workunit.client.0.smithi028.stdout:5/33: symlink d0/lb 0 2024-07-03T15:43:53.833 INFO:tasks.workunit.client.0.smithi028.stdout:5/34: chown d0/c4 280944066 1 2024-07-03T15:43:53.834 INFO:tasks.workunit.client.0.smithi028.stdout:1/5: read f1 [3486366,31858] 0 2024-07-03T15:43:53.834 INFO:tasks.workunit.client.0.smithi028.stdout:1/6: creat f2 x:0 0 0 2024-07-03T15:43:53.834 INFO:tasks.workunit.client.0.smithi028.stdout:1/7: rmdir - no directory 2024-07-03T15:43:53.834 INFO:tasks.workunit.client.0.smithi028.stdout:3/10: creat f6 x:0 0 0 2024-07-03T15:43:53.837 INFO:tasks.workunit.client.1.smithi148.stdout:4/405: dwrite d0/d8/d22/d2f/f6c [0,4194304] 0 2024-07-03T15:43:53.838 INFO:tasks.workunit.client.0.smithi028.stdout:5/35: mknod d0/cc 0 2024-07-03T15:43:53.838 INFO:tasks.workunit.client.0.smithi028.stdout:5/36: write d0/f9 [651745,102887] 0 2024-07-03T15:43:53.839 INFO:tasks.workunit.client.0.smithi028.stdout:5/37: chown d0 4895 1 2024-07-03T15:43:53.839 INFO:tasks.workunit.client.0.smithi028.stdout:1/8: mknod c3 0 2024-07-03T15:43:53.839 INFO:tasks.workunit.client.0.smithi028.stdout:1/9: write f2 [592054,119089] 0 2024-07-03T15:43:53.839 INFO:tasks.workunit.client.0.smithi028.stdout:9/32: write d1/f6 [4841782,95792] 0 2024-07-03T15:43:53.839 INFO:tasks.workunit.client.0.smithi028.stdout:3/11: dread - f2 zero size 2024-07-03T15:43:53.839 INFO:tasks.workunit.client.0.smithi028.stdout:3/12: write f6 [650011,80943] 0 2024-07-03T15:43:53.839 INFO:tasks.workunit.client.0.smithi028.stdout:5/38: mkdir d0/dd 0 2024-07-03T15:43:53.839 INFO:tasks.workunit.client.0.smithi028.stdout:8/21: getdents d0/d2 0 2024-07-03T15:43:53.839 INFO:tasks.workunit.client.0.smithi028.stdout:1/10: mkdir d4 0 2024-07-03T15:43:53.842 INFO:tasks.workunit.client.1.smithi148.stdout:4/406: link d0/d8/d22/d2f/f33 d0/d8/d22/d62/f89 0 2024-07-03T15:43:53.842 INFO:tasks.workunit.client.0.smithi028.stdout:9/33: symlink d1/l8 0 2024-07-03T15:43:53.845 INFO:tasks.workunit.client.1.smithi148.stdout:4/407: link d0/d8/d22/d2f/f84 d0/d28/d7d/f8a 0 2024-07-03T15:43:53.846 INFO:tasks.workunit.client.0.smithi028.stdout:5/39: rename d0/cc to d0/dd/ce 0 2024-07-03T15:43:53.846 INFO:tasks.workunit.client.0.smithi028.stdout:5/40: write d0/f7 [1250806,44811] 0 2024-07-03T15:43:53.846 INFO:tasks.workunit.client.0.smithi028.stdout:5/41: truncate d0/f7 2118621 0 2024-07-03T15:43:53.847 INFO:tasks.workunit.client.0.smithi028.stdout:8/22: rename d0/d2/d3/f9 to d0/d2/d3/fa 0 2024-07-03T15:43:53.848 INFO:tasks.workunit.client.0.smithi028.stdout:1/11: creat d4/f5 x:0 0 0 2024-07-03T15:43:53.848 INFO:tasks.workunit.client.0.smithi028.stdout:9/34: dread d1/f6 [4194304,4194304] 0 2024-07-03T15:43:53.848 INFO:tasks.workunit.client.1.smithi148.stdout:4/408: dread d0/d8/d22/d2f/d41/f4d [0,4194304] 0 2024-07-03T15:43:53.849 INFO:tasks.workunit.client.0.smithi028.stdout:3/13: symlink l7 0 2024-07-03T15:43:53.849 INFO:tasks.workunit.client.0.smithi028.stdout:9/35: dread d1/f2 [0,4194304] 0 2024-07-03T15:43:53.849 INFO:tasks.workunit.client.0.smithi028.stdout:9/36: creat d1/f9 x:0 0 0 2024-07-03T15:43:53.852 INFO:tasks.workunit.client.0.smithi028.stdout:5/42: mkdir d0/dd/df 0 2024-07-03T15:43:53.854 INFO:tasks.workunit.client.0.smithi028.stdout:3/14: dread - f0 zero size 2024-07-03T15:43:53.857 INFO:tasks.workunit.client.1.smithi148.stdout:1/415: dread d0/d2/d11/d1e/f59 [0,4194304] 0 2024-07-03T15:43:53.859 INFO:tasks.workunit.client.0.smithi028.stdout:9/37: unlink d1/f6 0 2024-07-03T15:43:53.861 INFO:tasks.workunit.client.0.smithi028.stdout:3/15: readlink l1 0 2024-07-03T15:43:53.863 INFO:tasks.workunit.client.0.smithi028.stdout:9/38: truncate d1/f5 202182 0 2024-07-03T15:43:53.863 INFO:tasks.workunit.client.0.smithi028.stdout:9/39: read - d1/f9 zero size 2024-07-03T15:43:53.864 INFO:tasks.workunit.client.0.smithi028.stdout:9/40: dread d1/f2 [0,4194304] 0 2024-07-03T15:43:53.866 INFO:tasks.workunit.client.0.smithi028.stdout:9/41: dread d1/f2 [0,4194304] 0 2024-07-03T15:43:53.866 INFO:tasks.workunit.client.0.smithi028.stdout:9/42: write d1/f2 [1263167,35600] 0 2024-07-03T15:43:53.869 INFO:tasks.workunit.client.0.smithi028.stdout:9/43: mknod d1/ca 0 2024-07-03T15:43:53.871 INFO:tasks.workunit.client.0.smithi028.stdout:9/44: mknod d1/cb 0 2024-07-03T15:43:53.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:53 smithi028 ceph-mon[29148]: pgmap v239: 65 pgs: 65 active+clean; 954 MiB data, 4.4 GiB used, 532 GiB / 536 GiB avail; 87 MiB/s rd, 112 MiB/s wr, 743 op/s 2024-07-03T15:43:53.873 INFO:tasks.workunit.client.1.smithi148.stdout:7/578: truncate d5/d7d/da7/d1c/d33/f67 3784674 0 2024-07-03T15:43:53.873 INFO:tasks.workunit.client.0.smithi028.stdout:5/43: dwrite d0/f7 [0,4194304] 0 2024-07-03T15:43:53.873 INFO:tasks.workunit.client.0.smithi028.stdout:5/44: fdatasync d0/f7 0 2024-07-03T15:43:53.874 INFO:tasks.workunit.client.0.smithi028.stdout:1/12: dwrite f2 [0,4194304] 0 2024-07-03T15:43:53.876 INFO:tasks.workunit.client.0.smithi028.stdout:5/45: mknod d0/dd/c10 0 2024-07-03T15:43:53.878 INFO:tasks.workunit.client.1.smithi148.stdout:1/416: dwrite d0/de/d44/f5e [0,4194304] 0 2024-07-03T15:43:53.878 INFO:tasks.workunit.client.1.smithi148.stdout:1/417: fsync d0/de/d44/f5e 0 2024-07-03T15:43:53.879 INFO:tasks.workunit.client.1.smithi148.stdout:2/568: sync 2024-07-03T15:43:53.879 INFO:tasks.workunit.client.1.smithi148.stdout:8/397: sync 2024-07-03T15:43:53.879 INFO:tasks.workunit.client.1.smithi148.stdout:6/502: sync 2024-07-03T15:43:53.879 INFO:tasks.workunit.client.1.smithi148.stdout:2/569: write d5/d14/d22/d2b/d13/d41/f63 [340102,105884] 0 2024-07-03T15:43:53.879 INFO:tasks.workunit.client.1.smithi148.stdout:6/503: readlink d3/d1d/d28/d3e/l49 0 2024-07-03T15:43:53.879 INFO:tasks.workunit.client.1.smithi148.stdout:6/504: write d3/d1d/d28/d3e/f64 [355869,18608] 0 2024-07-03T15:43:53.879 INFO:tasks.workunit.client.0.smithi028.stdout:1/13: mknod d4/c6 0 2024-07-03T15:43:53.880 INFO:tasks.workunit.client.1.smithi148.stdout:3/476: sync 2024-07-03T15:43:53.880 INFO:tasks.workunit.client.1.smithi148.stdout:9/722: sync 2024-07-03T15:43:53.881 INFO:tasks.workunit.client.1.smithi148.stdout:5/486: sync 2024-07-03T15:43:53.881 INFO:tasks.workunit.client.1.smithi148.stdout:5/487: dread - d7/da/f5f zero size 2024-07-03T15:43:53.881 INFO:tasks.workunit.client.1.smithi148.stdout:0/474: sync 2024-07-03T15:43:53.881 INFO:tasks.workunit.client.0.smithi028.stdout:1/14: mknod d4/c7 0 2024-07-03T15:43:53.881 INFO:tasks.workunit.client.0.smithi028.stdout:1/15: readlink - no filename 2024-07-03T15:43:53.881 INFO:tasks.workunit.client.1.smithi148.stdout:0/475: write da/d17/d2a/d2f/f37 [393707,121129] 0 2024-07-03T15:43:53.884 INFO:tasks.workunit.client.1.smithi148.stdout:4/409: rmdir d0/d8/d3a 39 2024-07-03T15:43:53.884 INFO:tasks.workunit.client.1.smithi148.stdout:4/410: write d0/d8/d13/d67/f86 [974614,43086] 0 2024-07-03T15:43:53.884 INFO:tasks.workunit.client.1.smithi148.stdout:4/411: fdatasync d0/d28/d7d/d6f/f82 0 2024-07-03T15:43:53.886 INFO:tasks.workunit.client.1.smithi148.stdout:2/570: unlink d5/d14/d22/d2b/d13/d41/f4b 0 2024-07-03T15:43:53.886 INFO:tasks.workunit.client.1.smithi148.stdout:2/571: write d5/d14/f58 [3353946,29827] 0 2024-07-03T15:43:53.886 INFO:tasks.workunit.client.1.smithi148.stdout:9/723: link d0/dd/d25/d6a/f82 d0/dd/d46/fe4 0 2024-07-03T15:43:53.886 INFO:tasks.workunit.client.1.smithi148.stdout:9/724: write d0/d18/dcf/fe2 [379584,2972] 0 2024-07-03T15:43:53.886 INFO:tasks.workunit.client.1.smithi148.stdout:6/505: dread d3/f17 [0,4194304] 0 2024-07-03T15:43:53.887 INFO:tasks.workunit.client.1.smithi148.stdout:8/398: dread d4/da/f28 [0,4194304] 0 2024-07-03T15:43:53.887 INFO:tasks.workunit.client.1.smithi148.stdout:4/412: mkdir d0/d8/d13/d8b 0 2024-07-03T15:43:53.892 INFO:tasks.workunit.client.1.smithi148.stdout:2/572: mknod d5/d14/d22/d60/cb6 0 2024-07-03T15:43:53.896 INFO:tasks.workunit.client.0.smithi028.stdout:1/16: write f2 [1972059,14252] 0 2024-07-03T15:43:53.896 INFO:tasks.workunit.client.1.smithi148.stdout:5/488: link d7/da/df/d16/d1a/c22 d7/da/d21/ca4 0 2024-07-03T15:43:53.896 INFO:tasks.workunit.client.1.smithi148.stdout:6/506: unlink d3/d1d/d28/l4f 0 2024-07-03T15:43:53.897 INFO:tasks.workunit.client.1.smithi148.stdout:6/507: chown d3/d1d/d28/f69 6299 1 2024-07-03T15:43:53.897 INFO:tasks.workunit.client.1.smithi148.stdout:6/508: fsync d3/d50/d77/fa7 0 2024-07-03T15:43:53.897 INFO:tasks.workunit.client.1.smithi148.stdout:6/509: creat d3/d1d/d28/d31/fab x:0 0 0 2024-07-03T15:43:53.897 INFO:tasks.workunit.client.1.smithi148.stdout:6/510: truncate d3/d1d/d28/d53/f59 4353561 0 2024-07-03T15:43:53.897 INFO:tasks.workunit.client.1.smithi148.stdout:6/511: readlink d3/d1d/d28/l3d 0 2024-07-03T15:43:53.897 INFO:tasks.workunit.client.1.smithi148.stdout:6/512: write d3/d1d/d28/d53/f6e [48348,32906] 0 2024-07-03T15:43:53.898 INFO:tasks.workunit.client.1.smithi148.stdout:5/489: write d7/da/f4c [1056669,48936] 0 2024-07-03T15:43:53.898 INFO:tasks.workunit.client.1.smithi148.stdout:5/490: fdatasync d7/da/df/d1d/f2e 0 2024-07-03T15:43:53.899 INFO:tasks.workunit.client.0.smithi028.stdout:1/17: mknod d4/c8 0 2024-07-03T15:43:53.899 INFO:tasks.workunit.client.0.smithi028.stdout:1/18: dread - d4/f5 zero size 2024-07-03T15:43:53.899 INFO:tasks.workunit.client.0.smithi028.stdout:1/19: creat d4/f9 x:0 0 0 2024-07-03T15:43:53.899 INFO:tasks.workunit.client.1.smithi148.stdout:9/725: rmdir d0/dd/d25/d37/d40/da3 39 2024-07-03T15:43:53.903 INFO:tasks.workunit.client.1.smithi148.stdout:8/399: getdents d4/da/d51/d18/d20 0 2024-07-03T15:43:53.906 INFO:tasks.workunit.client.1.smithi148.stdout:9/726: symlink d0/d18/d24/d47/le5 0 2024-07-03T15:43:53.906 INFO:tasks.workunit.client.1.smithi148.stdout:9/727: write d0/d18/d24/f91 [3213046,60933] 0 2024-07-03T15:43:53.906 INFO:tasks.workunit.client.1.smithi148.stdout:9/728: chown d0/dd/cc9 16265435 1 2024-07-03T15:43:53.907 INFO:tasks.workunit.client.1.smithi148.stdout:0/476: read da/d17/d2a/d2f/f87 [933677,124991] 0 2024-07-03T15:43:53.910 INFO:tasks.workunit.client.1.smithi148.stdout:9/729: mkdir d0/dd/d25/d6a/da8/de6 0 2024-07-03T15:43:53.910 INFO:tasks.workunit.client.1.smithi148.stdout:0/477: mkdir da/d44/d13/d1d/da1 0 2024-07-03T15:43:53.910 INFO:tasks.workunit.client.1.smithi148.stdout:0/478: chown da/d17/d2a/f4b 26 1 2024-07-03T15:43:53.914 INFO:tasks.workunit.client.1.smithi148.stdout:6/513: dread d3/d50/f72 [0,4194304] 0 2024-07-03T15:43:53.914 INFO:tasks.workunit.client.1.smithi148.stdout:3/477: dwrite dd/d33/f6c [0,4194304] 0 2024-07-03T15:43:53.914 INFO:tasks.workunit.client.0.smithi028.stdout:3/16: dwrite f6 [0,4194304] 0 2024-07-03T15:43:53.915 INFO:tasks.workunit.client.0.smithi028.stdout:3/17: dread - f2 zero size 2024-07-03T15:43:53.915 INFO:tasks.workunit.client.0.smithi028.stdout:3/18: dread - f2 zero size 2024-07-03T15:43:53.919 INFO:tasks.workunit.client.1.smithi148.stdout:8/400: write d4/da/d14/d17/f2a [2353900,17468] 0 2024-07-03T15:43:53.921 INFO:tasks.workunit.client.1.smithi148.stdout:5/491: dwrite d7/d13/d54/f4b [0,4194304] 0 2024-07-03T15:43:53.921 INFO:tasks.workunit.client.1.smithi148.stdout:5/492: write d7/da/df/d16/d2f/d44/f84 [392377,38040] 0 2024-07-03T15:43:53.923 INFO:tasks.workunit.client.1.smithi148.stdout:0/479: link da/f11 da/d44/d13/d1d/d3e/d85/d8e/fa2 0 2024-07-03T15:43:53.925 INFO:tasks.workunit.client.0.smithi028.stdout:3/19: symlink l8 0 2024-07-03T15:43:53.925 INFO:tasks.workunit.client.0.smithi028.stdout:3/20: write f0 [446830,1767] 0 2024-07-03T15:43:53.925 INFO:tasks.workunit.client.0.smithi028.stdout:3/21: write f2 [407291,1581] 0 2024-07-03T15:43:53.928 INFO:tasks.workunit.client.1.smithi148.stdout:9/730: dwrite d0/dd/d46/fe4 [0,4194304] 0 2024-07-03T15:43:53.929 INFO:tasks.workunit.client.0.smithi028.stdout:3/22: creat f9 x:0 0 0 2024-07-03T15:43:53.929 INFO:tasks.workunit.client.0.smithi028.stdout:3/23: chown f4 534636 1 2024-07-03T15:43:53.931 INFO:tasks.workunit.client.1.smithi148.stdout:6/514: mkdir d3/d1d/d28/d31/d61/dac 0 2024-07-03T15:43:53.931 INFO:tasks.workunit.client.1.smithi148.stdout:6/515: readlink d3/d1d/l60 0 2024-07-03T15:43:53.931 INFO:tasks.workunit.client.0.smithi028.stdout:8/23: fsync d0/d2/d3/fa 0 2024-07-03T15:43:53.932 INFO:tasks.workunit.client.0.smithi028.stdout:8/24: readlink d0/d2/l5 0 2024-07-03T15:43:53.934 INFO:tasks.workunit.client.1.smithi148.stdout:8/401: link d4/da/d14/d1a/f37 d4/da/d14/f91 0 2024-07-03T15:43:53.935 INFO:tasks.workunit.client.0.smithi028.stdout:8/25: rmdir d0/d2 39 2024-07-03T15:43:53.935 INFO:tasks.workunit.client.1.smithi148.stdout:2/573: dwrite d5/d14/d22/d60/f6c [0,4194304] 0 2024-07-03T15:43:53.936 INFO:tasks.workunit.client.1.smithi148.stdout:5/493: mknod d7/da/ca5 0 2024-07-03T15:43:53.936 INFO:tasks.workunit.client.1.smithi148.stdout:5/494: creat d7/da/df/d16/d2f/d44/d52/d60/fa6 x:0 0 0 2024-07-03T15:43:53.937 INFO:tasks.workunit.client.0.smithi028.stdout:8/26: rmdir d0/d2 39 2024-07-03T15:43:53.938 INFO:tasks.workunit.client.1.smithi148.stdout:0/480: mkdir da/d17/d2a/d2f/d93/da3 0 2024-07-03T15:43:53.939 INFO:tasks.workunit.client.1.smithi148.stdout:3/478: dwrite dd/d33/d37/f82 [0,4194304] 0 2024-07-03T15:43:53.939 INFO:tasks.workunit.client.1.smithi148.stdout:3/479: chown dd/d33/d37/d68/d2e/d40/d45/d65/c69 0 1 2024-07-03T15:43:53.940 INFO:tasks.workunit.client.0.smithi028.stdout:1/20: dwrite f1 [0,4194304] 0 2024-07-03T15:43:53.940 INFO:tasks.workunit.client.0.smithi028.stdout:1/21: chown d4/c6 3238329 1 2024-07-03T15:43:53.940 INFO:tasks.workunit.client.1.smithi148.stdout:6/516: symlink d3/d1d/d28/lad 0 2024-07-03T15:43:53.940 INFO:tasks.workunit.client.0.smithi028.stdout:8/27: write d0/d2/d3/fa [3630392,43764] 0 2024-07-03T15:43:53.943 INFO:tasks.workunit.client.1.smithi148.stdout:8/402: rmdir d4/d71 0 2024-07-03T15:43:53.943 INFO:tasks.workunit.client.0.smithi028.stdout:9/45: write d1/f5 [454029,127071] 0 2024-07-03T15:43:53.946 INFO:tasks.workunit.client.0.smithi028.stdout:9/46: fdatasync d1/f5 0 2024-07-03T15:43:53.948 INFO:tasks.workunit.client.0.smithi028.stdout:3/24: dwrite f4 [0,4194304] 0 2024-07-03T15:43:53.948 INFO:tasks.workunit.client.0.smithi028.stdout:3/25: fsync f5 0 2024-07-03T15:43:53.948 INFO:tasks.workunit.client.1.smithi148.stdout:2/574: creat d5/d14/d8f/fb7 x:0 0 0 2024-07-03T15:43:53.949 INFO:tasks.workunit.client.0.smithi028.stdout:9/47: symlink d1/lc 0 2024-07-03T15:43:53.950 INFO:tasks.workunit.client.0.smithi028.stdout:8/28: dread d0/d2/d3/fa [0,4194304] 0 2024-07-03T15:43:53.956 INFO:tasks.workunit.client.1.smithi148.stdout:0/481: mknod da/ca4 0 2024-07-03T15:43:53.957 INFO:tasks.workunit.client.0.smithi028.stdout:8/29: mknod d0/cb 0 2024-07-03T15:43:53.958 INFO:tasks.workunit.client.0.smithi028.stdout:5/46: truncate d0/f9 328856 0 2024-07-03T15:43:53.958 INFO:tasks.workunit.client.0.smithi028.stdout:5/47: chown d0/f9 24985 1 2024-07-03T15:43:53.962 INFO:tasks.workunit.client.0.smithi028.stdout:5/48: symlink d0/l11 0 2024-07-03T15:43:53.963 INFO:tasks.workunit.client.1.smithi148.stdout:6/517: symlink d3/d1d/d28/d7f/lae 0 2024-07-03T15:43:53.967 INFO:tasks.workunit.client.0.smithi028.stdout:5/49: link d0/dd/c10 d0/dd/df/c12 0 2024-07-03T15:43:53.967 INFO:tasks.workunit.client.0.smithi028.stdout:5/50: truncate d0/f7 4195582 0 2024-07-03T15:43:53.968 INFO:tasks.workunit.client.1.smithi148.stdout:2/575: creat d5/d14/d22/d2b/d13/d2d/fb8 x:0 0 0 2024-07-03T15:43:53.968 INFO:tasks.workunit.client.1.smithi148.stdout:2/576: write d5/d14/d22/d2b/f1e [530832,126789] 0 2024-07-03T15:43:53.968 INFO:tasks.workunit.client.1.smithi148.stdout:2/577: read d5/d14/d22/d2b/d13/f29 [475328,115851] 0 2024-07-03T15:43:53.969 INFO:tasks.workunit.client.1.smithi148.stdout:0/482: dread da/d58/f79 [0,4194304] 0 2024-07-03T15:43:53.970 INFO:tasks.workunit.client.0.smithi028.stdout:5/51: mkdir d0/dd/d13 0 2024-07-03T15:43:53.970 INFO:tasks.workunit.client.0.smithi028.stdout:5/52: chown d0/dd/df/c12 344418040 1 2024-07-03T15:43:53.970 INFO:tasks.workunit.client.0.smithi028.stdout:5/53: write d0/f7 [4353404,125167] 0 2024-07-03T15:43:53.972 INFO:tasks.workunit.client.0.smithi028.stdout:3/26: dwrite f0 [0,4194304] 0 2024-07-03T15:43:53.972 INFO:tasks.workunit.client.0.smithi028.stdout:3/27: chown f9 9173396 1 2024-07-03T15:43:53.972 INFO:tasks.workunit.client.0.smithi028.stdout:3/28: read - f5 zero size 2024-07-03T15:43:53.972 INFO:tasks.workunit.client.0.smithi028.stdout:3/29: stat f2 0 2024-07-03T15:43:53.973 INFO:tasks.workunit.client.0.smithi028.stdout:1/22: getdents d4 0 2024-07-03T15:43:53.973 INFO:tasks.workunit.client.0.smithi028.stdout:1/23: write d4/f5 [215528,23224] 0 2024-07-03T15:43:53.975 INFO:tasks.workunit.client.1.smithi148.stdout:6/518: creat d3/d1d/d28/d31/d61/faf x:0 0 0 2024-07-03T15:43:53.979 INFO:tasks.workunit.client.0.smithi028.stdout:1/24: symlink d4/la 0 2024-07-03T15:43:53.979 INFO:tasks.workunit.client.0.smithi028.stdout:1/25: chown c3 1694541514 1 2024-07-03T15:43:53.980 INFO:tasks.workunit.client.0.smithi028.stdout:9/48: dwrite d1/f2 [0,4194304] 0 2024-07-03T15:43:53.983 INFO:tasks.workunit.client.1.smithi148.stdout:5/495: rename d7/d13/d23/d40 to d7/da/df/d16/d1a/d58/da7 0 2024-07-03T15:43:53.984 INFO:tasks.workunit.client.0.smithi028.stdout:3/30: symlink la 0 2024-07-03T15:43:53.984 INFO:tasks.workunit.client.0.smithi028.stdout:3/31: chown l8 3760740 1 2024-07-03T15:43:53.987 INFO:tasks.workunit.client.0.smithi028.stdout:1/26: creat d4/fb x:0 0 0 2024-07-03T15:43:53.989 INFO:tasks.workunit.client.0.smithi028.stdout:8/30: dwrite d0/d2/d3/fa [4194304,4194304] 0 2024-07-03T15:43:53.989 INFO:tasks.workunit.client.1.smithi148.stdout:8/403: rename d4/da/d51/d18/d1f to d4/da/d47/d88/d92 0 2024-07-03T15:43:53.992 INFO:tasks.workunit.client.1.smithi148.stdout:0/483: dwrite da/d17/d2a/d2f/d93/f9c [0,4194304] 0 2024-07-03T15:43:53.992 INFO:tasks.workunit.client.0.smithi028.stdout:9/49: mkdir d1/dd 0 2024-07-03T15:43:53.994 INFO:tasks.workunit.client.1.smithi148.stdout:6/519: rmdir d3/d1d/d28/d7f/d85 39 2024-07-03T15:43:53.994 INFO:tasks.workunit.client.1.smithi148.stdout:6/520: getdents d3/d1d/d28/d53/d6d/d8f 0 2024-07-03T15:43:53.994 INFO:tasks.workunit.client.1.smithi148.stdout:6/521: write d3/f37 [517610,109620] 0 2024-07-03T15:43:53.994 INFO:tasks.workunit.client.1.smithi148.stdout:6/522: creat d3/fb0 x:0 0 0 2024-07-03T15:43:53.994 INFO:tasks.workunit.client.1.smithi148.stdout:0/484: dread da/d44/d13/f18 [0,4194304] 0 2024-07-03T15:43:53.997 INFO:tasks.workunit.client.1.smithi148.stdout:8/404: symlink d4/l93 0 2024-07-03T15:43:53.998 INFO:tasks.workunit.client.0.smithi028.stdout:1/27: rename d4/f9 to d4/fc 0 2024-07-03T15:43:53.998 INFO:tasks.workunit.client.0.smithi028.stdout:1/28: chown f1 71724 1 2024-07-03T15:43:54.001 INFO:tasks.workunit.client.1.smithi148.stdout:5/496: link d7/c6d d7/da/d21/ca8 0 2024-07-03T15:43:54.001 INFO:tasks.workunit.client.1.smithi148.stdout:5/497: write d7/da/df/d16/d6a/fa1 [894947,117623] 0 2024-07-03T15:43:54.003 INFO:tasks.workunit.client.0.smithi028.stdout:3/32: rename f0 to fb 0 2024-07-03T15:43:54.008 INFO:tasks.workunit.client.0.smithi028.stdout:1/29: link f2 d4/fd 0 2024-07-03T15:43:54.008 INFO:tasks.workunit.client.0.smithi028.stdout:9/50: dwrite d1/f5 [0,4194304] 0 2024-07-03T15:43:54.012 INFO:tasks.workunit.client.1.smithi148.stdout:5/498: dread d7/d13/d54/f4b [0,4194304] 0 2024-07-03T15:43:54.015 INFO:tasks.workunit.client.0.smithi028.stdout:1/30: mkdir d4/de 0 2024-07-03T15:43:54.019 INFO:tasks.workunit.client.1.smithi148.stdout:6/523: link d3/f17 d3/d5b/fb1 0 2024-07-03T15:43:54.019 INFO:tasks.workunit.client.1.smithi148.stdout:6/524: fsync d3/f56 0 2024-07-03T15:43:54.019 INFO:tasks.workunit.client.1.smithi148.stdout:6/525: fdatasync d3/d50/f7d 0 2024-07-03T15:43:54.020 INFO:tasks.workunit.client.0.smithi028.stdout:8/31: dwrite d0/d2/d3/fa [4194304,4194304] 0 2024-07-03T15:43:54.020 INFO:tasks.workunit.client.0.smithi028.stdout:9/51: creat d1/dd/fe x:0 0 0 2024-07-03T15:43:54.021 INFO:tasks.workunit.client.0.smithi028.stdout:3/33: rename f5 to fc 0 2024-07-03T15:43:54.026 INFO:tasks.workunit.client.0.smithi028.stdout:8/32: mkdir d0/d2/d3/dc 0 2024-07-03T15:43:54.029 INFO:tasks.workunit.client.0.smithi028.stdout:9/52: mknod d1/dd/cf 0 2024-07-03T15:43:54.030 INFO:tasks.workunit.client.1.smithi148.stdout:0/485: dwrite da/d44/f75 [0,4194304] 0 2024-07-03T15:43:54.030 INFO:tasks.workunit.client.1.smithi148.stdout:0/486: rename da/d44 to da/d44/d13/d1d/da1/da5 22 2024-07-03T15:43:54.033 INFO:tasks.workunit.client.1.smithi148.stdout:9/731: truncate d0/dd/d25/d6a/f82 3838486 0 2024-07-03T15:43:54.035 INFO:tasks.workunit.client.0.smithi028.stdout:8/33: mkdir d0/dd 0 2024-07-03T15:43:54.036 INFO:tasks.workunit.client.1.smithi148.stdout:8/405: dwrite d4/da/d51/d18/d20/f40 [4194304,4194304] 0 2024-07-03T15:43:54.036 INFO:tasks.workunit.client.1.smithi148.stdout:8/406: fdatasync d4/da/d14/f68 0 2024-07-03T15:43:54.036 INFO:tasks.workunit.client.1.smithi148.stdout:8/407: creat d4/da/d47/f94 x:0 0 0 2024-07-03T15:43:54.036 INFO:tasks.workunit.client.1.smithi148.stdout:8/408: chown d4/da/d51/c75 161110807 1 2024-07-03T15:43:54.036 INFO:tasks.workunit.client.1.smithi148.stdout:8/409: read d4/da/f29 [755683,87671] 0 2024-07-03T15:43:54.036 INFO:tasks.workunit.client.1.smithi148.stdout:8/410: creat d4/da/d50/f95 x:0 0 0 2024-07-03T15:43:54.038 INFO:tasks.workunit.client.0.smithi028.stdout:9/53: mkdir d1/d10 0 2024-07-03T15:43:54.038 INFO:tasks.workunit.client.0.smithi028.stdout:9/54: stat d1/ca 0 2024-07-03T15:43:54.041 INFO:tasks.workunit.client.0.smithi028.stdout:8/34: symlink d0/d2/le 0 2024-07-03T15:43:54.043 INFO:tasks.workunit.client.0.smithi028.stdout:3/34: dwrite fc [0,4194304] 0 2024-07-03T15:43:54.043 INFO:tasks.workunit.client.1.smithi148.stdout:0/487: write da/f11 [4118340,62146] 0 2024-07-03T15:43:54.046 INFO:tasks.workunit.client.0.smithi028.stdout:9/55: dread d1/f5 [0,4194304] 0 2024-07-03T15:43:54.048 INFO:tasks.workunit.client.1.smithi148.stdout:6/526: dwrite f1 [0,4194304] 0 2024-07-03T15:43:54.048 INFO:tasks.workunit.client.1.smithi148.stdout:6/527: chown d3/d1d/d28/f73 1727 1 2024-07-03T15:43:54.050 INFO:tasks.workunit.client.1.smithi148.stdout:5/499: rmdir d7/da/df/d16 39 2024-07-03T15:43:54.050 INFO:tasks.workunit.client.1.smithi148.stdout:5/500: fsync d7/da/df/d83/f9c 0 2024-07-03T15:43:54.052 INFO:tasks.workunit.client.1.smithi148.stdout:5/501: dread d7/d13/f2a [0,4194304] 0 2024-07-03T15:43:54.052 INFO:tasks.workunit.client.1.smithi148.stdout:9/732: getdents d0/d18/dcf 0 2024-07-03T15:43:54.056 INFO:tasks.workunit.client.0.smithi028.stdout:8/35: creat d0/d2/ff x:0 0 0 2024-07-03T15:43:54.056 INFO:tasks.workunit.client.0.smithi028.stdout:8/36: write d0/d2/d3/fa [3313801,49850] 0 2024-07-03T15:43:54.056 INFO:tasks.workunit.client.1.smithi148.stdout:5/502: dread d7/d13/d54/f4b [0,4194304] 0 2024-07-03T15:43:54.056 INFO:tasks.workunit.client.1.smithi148.stdout:5/503: write d7/da/f4c [576213,43856] 0 2024-07-03T15:43:54.057 INFO:tasks.workunit.client.1.smithi148.stdout:5/504: write d7/da/df/d16/d2f/d44/d52/f90 [944670,12321] 0 2024-07-03T15:43:54.060 INFO:tasks.workunit.client.1.smithi148.stdout:0/488: stat da/d36/c66 0 2024-07-03T15:43:54.060 INFO:tasks.workunit.client.1.smithi148.stdout:9/733: dread d0/f53 [0,4194304] 0 2024-07-03T15:43:54.061 INFO:tasks.workunit.client.0.smithi028.stdout:5/54: dwrite d0/f9 [0,4194304] 0 2024-07-03T15:43:54.061 INFO:tasks.workunit.client.1.smithi148.stdout:8/411: dwrite d4/da/d14/f6c [0,4194304] 0 2024-07-03T15:43:54.062 INFO:tasks.workunit.client.1.smithi148.stdout:6/528: mkdir d3/d1d/d28/d7f/db2 0 2024-07-03T15:43:54.064 INFO:tasks.workunit.client.0.smithi028.stdout:9/56: link d1/f5 d1/dd/f11 0 2024-07-03T15:43:54.066 INFO:tasks.workunit.client.1.smithi148.stdout:5/505: symlink d7/d13/la9 0 2024-07-03T15:43:54.070 INFO:tasks.workunit.client.1.smithi148.stdout:0/489: write da/d38/f6b [1507361,87866] 0 2024-07-03T15:43:54.070 INFO:tasks.workunit.client.1.smithi148.stdout:0/490: write da/d44/d13/d1d/d3e/f6e [1649334,61880] 0 2024-07-03T15:43:54.071 INFO:tasks.workunit.client.1.smithi148.stdout:9/734: link d0/dd/d25/d37/d40/d72/f8d d0/dd/d25/d37/fe7 0 2024-07-03T15:43:54.071 INFO:tasks.workunit.client.1.smithi148.stdout:9/735: write d0/dd/d11/db4/fc0 [766057,16836] 0 2024-07-03T15:43:54.071 INFO:tasks.workunit.client.1.smithi148.stdout:9/736: chown d0/dd/d25/d37/d40/da3/fe0 1995 1 2024-07-03T15:43:54.073 INFO:tasks.workunit.client.0.smithi028.stdout:9/57: write d1/f2 [1041649,123742] 0 2024-07-03T15:43:54.073 INFO:tasks.workunit.client.0.smithi028.stdout:8/37: dread d0/d2/d3/fa [4194304,4194304] 0 2024-07-03T15:43:54.077 INFO:tasks.workunit.client.1.smithi148.stdout:8/412: symlink d4/da/d14/d17/d76/l96 0 2024-07-03T15:43:54.079 INFO:tasks.workunit.client.0.smithi028.stdout:5/55: mkdir d0/dd/d14 0 2024-07-03T15:43:54.079 INFO:tasks.workunit.client.0.smithi028.stdout:5/56: fsync d0/f9 0 2024-07-03T15:43:54.079 INFO:tasks.workunit.client.0.smithi028.stdout:5/57: readlink d0/lb 0 2024-07-03T15:43:54.080 INFO:tasks.workunit.client.1.smithi148.stdout:6/529: rmdir d3/d1d/d28/d31/d4c 39 2024-07-03T15:43:54.080 INFO:tasks.workunit.client.1.smithi148.stdout:0/491: write da/d44/d13/d14/d50/f83 [3558475,67328] 0 2024-07-03T15:43:54.085 INFO:tasks.workunit.client.0.smithi028.stdout:9/58: symlink d1/l12 0 2024-07-03T15:43:54.085 INFO:tasks.workunit.client.0.smithi028.stdout:9/59: write d1/dd/fe [743664,1868] 0 2024-07-03T15:43:54.088 INFO:tasks.workunit.client.0.smithi028.stdout:8/38: mknod d0/d2/c10 0 2024-07-03T15:43:54.088 INFO:tasks.workunit.client.0.smithi028.stdout:1/31: truncate f2 3117202 0 2024-07-03T15:43:54.093 INFO:tasks.workunit.client.0.smithi028.stdout:8/39: mkdir d0/d2/d3/dc/d11 0 2024-07-03T15:43:54.093 INFO:tasks.workunit.client.0.smithi028.stdout:8/40: write d0/d2/d3/fa [8655265,91610] 0 2024-07-03T15:43:54.094 INFO:tasks.workunit.client.0.smithi028.stdout:8/41: chown d0/l8 1902 1 2024-07-03T15:43:54.095 INFO:tasks.workunit.client.0.smithi028.stdout:1/32: rename f1 to d4/de/ff 0 2024-07-03T15:43:54.097 INFO:tasks.workunit.client.0.smithi028.stdout:8/42: mkdir d0/d2/d3/dc/d11/d12 0 2024-07-03T15:43:54.097 INFO:tasks.workunit.client.0.smithi028.stdout:8/43: fdatasync d0/d2/ff 0 2024-07-03T15:43:54.097 INFO:tasks.workunit.client.0.smithi028.stdout:3/35: truncate fc 671705 0 2024-07-03T15:43:54.100 INFO:tasks.workunit.client.1.smithi148.stdout:5/506: mknod d7/d59/caa 0 2024-07-03T15:43:54.101 INFO:tasks.workunit.client.0.smithi028.stdout:9/60: fsync d1/f5 0 2024-07-03T15:43:54.106 INFO:tasks.workunit.client.0.smithi028.stdout:3/36: rename l7 to ld 0 2024-07-03T15:43:54.106 INFO:tasks.workunit.client.0.smithi028.stdout:5/58: dwrite d0/f9 [0,4194304] 0 2024-07-03T15:43:54.107 INFO:tasks.workunit.client.1.smithi148.stdout:9/737: creat d0/dd/d25/fe8 x:0 0 0 2024-07-03T15:43:54.108 INFO:tasks.workunit.client.0.smithi028.stdout:9/61: write d1/f5 [2610729,42451] 0 2024-07-03T15:43:54.108 INFO:tasks.workunit.client.0.smithi028.stdout:9/62: creat d1/dd/f13 x:0 0 0 2024-07-03T15:43:54.108 INFO:tasks.workunit.client.0.smithi028.stdout:9/63: stat d1/dd/fe 0 2024-07-03T15:43:54.111 INFO:tasks.workunit.client.0.smithi028.stdout:5/59: rename d0/ca to d0/dd/c15 0 2024-07-03T15:43:54.112 INFO:tasks.workunit.client.1.smithi148.stdout:8/413: dread d4/da/d14/d1a/d32/d38/d58/f8b [0,4194304] 0 2024-07-03T15:43:54.112 INFO:tasks.workunit.client.1.smithi148.stdout:6/530: link d3/d1d/d28/d3e/l87 d3/d5b/lb3 0 2024-07-03T15:43:54.114 INFO:tasks.workunit.client.1.smithi148.stdout:0/492: dwrite da/d44/d13/d14/d50/d95/f97 [0,4194304] 0 2024-07-03T15:43:54.114 INFO:tasks.workunit.client.1.smithi148.stdout:0/493: truncate da/d44/d13/f27 3880895 0 2024-07-03T15:43:54.115 INFO:tasks.workunit.client.0.smithi028.stdout:8/44: dwrite d0/d2/ff [0,4194304] 0 2024-07-03T15:43:54.115 INFO:tasks.workunit.client.1.smithi148.stdout:5/507: truncate d7/da/df/f10 3333136 0 2024-07-03T15:43:54.116 INFO:tasks.workunit.client.1.smithi148.stdout:5/508: truncate d7/d13/d23/f85 715672 0 2024-07-03T15:43:54.117 INFO:tasks.workunit.client.1.smithi148.stdout:9/738: rename d0/d49/d43/f2c to d0/d18/d24/d5a/fe9 0 2024-07-03T15:43:54.117 INFO:tasks.workunit.client.1.smithi148.stdout:9/739: write d0/d49/d43/d19/d7a/daa/fb1 [582149,61503] 0 2024-07-03T15:43:54.117 INFO:tasks.workunit.client.1.smithi148.stdout:9/740: write d0/d18/d24/d5a/fe9 [681362,67964] 0 2024-07-03T15:43:54.117 INFO:tasks.workunit.client.1.smithi148.stdout:9/741: readlink d0/d49/d43/d19/de1/lc6 0 2024-07-03T15:43:54.118 INFO:tasks.workunit.client.0.smithi028.stdout:3/37: rename fb to fe 0 2024-07-03T15:43:54.119 INFO:tasks.workunit.client.1.smithi148.stdout:8/414: creat d4/d86/f97 x:0 0 0 2024-07-03T15:43:54.120 INFO:tasks.workunit.client.0.smithi028.stdout:5/60: creat d0/dd/df/f16 x:0 0 0 2024-07-03T15:43:54.121 INFO:tasks.workunit.client.1.smithi148.stdout:6/531: stat d3/d1d/d28/d31/d61/c82 0 2024-07-03T15:43:54.123 INFO:tasks.workunit.client.1.smithi148.stdout:0/494: dread da/d44/d13/d14/d50/f83 [0,4194304] 0 2024-07-03T15:43:54.123 INFO:tasks.workunit.client.1.smithi148.stdout:0/495: truncate da/d38/f7c 1113334 0 2024-07-03T15:43:54.126 INFO:tasks.workunit.client.1.smithi148.stdout:4/413: sync 2024-07-03T15:43:54.126 INFO:tasks.workunit.client.1.smithi148.stdout:7/579: sync 2024-07-03T15:43:54.126 INFO:tasks.workunit.client.1.smithi148.stdout:7/580: creat d5/d7d/da7/d1c/d33/d4c/d21/d31/fc6 x:0 0 0 2024-07-03T15:43:54.130 INFO:tasks.workunit.client.1.smithi148.stdout:1/418: sync 2024-07-03T15:43:54.131 INFO:tasks.workunit.client.0.smithi028.stdout:3/38: creat ff x:0 0 0 2024-07-03T15:43:54.133 INFO:tasks.workunit.client.1.smithi148.stdout:5/509: truncate d7/f4f 1352229 0 2024-07-03T15:43:54.133 INFO:tasks.workunit.client.1.smithi148.stdout:5/510: creat d7/fab x:0 0 0 2024-07-03T15:43:54.134 INFO:tasks.workunit.client.1.smithi148.stdout:5/511: dread - d7/da/df/d16/d2f/f93 zero size 2024-07-03T15:43:54.135 INFO:tasks.workunit.client.1.smithi148.stdout:9/742: dwrite d0/dd/d25/fe8 [0,4194304] 0 2024-07-03T15:43:54.135 INFO:tasks.workunit.client.1.smithi148.stdout:9/743: chown d0/dd/d25/dcc 7720 1 2024-07-03T15:43:54.138 INFO:tasks.workunit.client.1.smithi148.stdout:8/415: creat d4/f98 x:0 0 0 2024-07-03T15:43:54.140 INFO:tasks.workunit.client.0.smithi028.stdout:3/39: symlink l10 0 2024-07-03T15:43:54.142 INFO:tasks.workunit.client.1.smithi148.stdout:6/532: symlink d3/d1d/d28/d7f/lb4 0 2024-07-03T15:43:54.144 INFO:tasks.workunit.client.1.smithi148.stdout:4/414: mkdir d0/d28/d8c 0 2024-07-03T15:43:54.146 INFO:tasks.workunit.client.1.smithi148.stdout:7/581: mkdir d5/d7d/da7/dc7 0 2024-07-03T15:43:54.147 INFO:tasks.workunit.client.1.smithi148.stdout:1/419: rename d0/d2/d11/d1e/c28 to d0/c8d 0 2024-07-03T15:43:54.148 INFO:tasks.workunit.client.1.smithi148.stdout:1/420: chown d0/d2/d2e/d39/d41/d77/f84 1663 1 2024-07-03T15:43:54.148 INFO:tasks.workunit.client.1.smithi148.stdout:1/421: write d0/de/d44/d54/d5d/f57 [74931,73589] 0 2024-07-03T15:43:54.148 INFO:tasks.workunit.client.1.smithi148.stdout:1/422: chown d0/d2/d6/d14/c7f 31 1 2024-07-03T15:43:54.151 INFO:tasks.workunit.client.1.smithi148.stdout:5/512: rmdir d7/da/df/d16/d1a/d58/da7 39 2024-07-03T15:43:54.152 INFO:tasks.workunit.client.0.smithi028.stdout:1/33: truncate d4/fd 3029613 0 2024-07-03T15:43:54.152 INFO:tasks.workunit.client.0.smithi028.stdout:1/34: write d4/f5 [217044,56680] 0 2024-07-03T15:43:54.152 INFO:tasks.workunit.client.0.smithi028.stdout:1/35: truncate d4/f5 1322052 0 2024-07-03T15:43:54.152 INFO:tasks.workunit.client.0.smithi028.stdout:8/45: truncate d0/d2/ff 402173 0 2024-07-03T15:43:54.153 INFO:tasks.workunit.client.0.smithi028.stdout:9/64: truncate d1/f5 2581140 0 2024-07-03T15:43:54.154 INFO:tasks.workunit.client.0.smithi028.stdout:9/65: read - d1/f9 zero size 2024-07-03T15:43:54.154 INFO:tasks.workunit.client.1.smithi148.stdout:4/415: write d0/d28/f63 [4165795,18728] 0 2024-07-03T15:43:54.157 INFO:tasks.workunit.client.0.smithi028.stdout:3/40: dread f6 [0,4194304] 0 2024-07-03T15:43:54.158 INFO:tasks.workunit.client.1.smithi148.stdout:8/416: mknod d4/da/d50/c99 0 2024-07-03T15:43:54.158 INFO:tasks.workunit.client.1.smithi148.stdout:2/578: sync 2024-07-03T15:43:54.159 INFO:tasks.workunit.client.1.smithi148.stdout:3/480: sync 2024-07-03T15:43:54.159 INFO:tasks.workunit.client.1.smithi148.stdout:2/579: fsync d5/d14/d22/d2b/d68/f7c 0 2024-07-03T15:43:54.161 INFO:tasks.workunit.client.1.smithi148.stdout:7/582: link d5/d7d/da7/d1c/d33/l74 d5/d7d/da7/d1c/d59/lc8 0 2024-07-03T15:43:54.164 INFO:tasks.workunit.client.0.smithi028.stdout:9/66: rmdir d1/dd 39 2024-07-03T15:43:54.164 INFO:tasks.workunit.client.1.smithi148.stdout:6/533: rename d3/f37 to d3/d50/d77/fb5 0 2024-07-03T15:43:54.166 INFO:tasks.workunit.client.1.smithi148.stdout:4/416: creat d0/d28/d7d/d6f/d7c/f8d x:0 0 0 2024-07-03T15:43:54.166 INFO:tasks.workunit.client.1.smithi148.stdout:8/417: mkdir d4/d9a 0 2024-07-03T15:43:54.167 INFO:tasks.workunit.client.1.smithi148.stdout:3/481: symlink dd/d33/d52/l99 0 2024-07-03T15:43:54.168 INFO:tasks.workunit.client.1.smithi148.stdout:2/580: dread d5/d14/f5a [0,4194304] 0 2024-07-03T15:43:54.168 INFO:tasks.workunit.client.1.smithi148.stdout:2/581: getdents d5/d14/d22/d2b/d13/d41/d9d 0 2024-07-03T15:43:54.171 INFO:tasks.workunit.client.0.smithi028.stdout:5/61: dwrite d0/f7 [0,4194304] 0 2024-07-03T15:43:54.171 INFO:tasks.workunit.client.1.smithi148.stdout:9/744: dwrite d0/dd/f1f [0,4194304] 0 2024-07-03T15:43:54.171 INFO:tasks.workunit.client.1.smithi148.stdout:9/745: chown d0/dd/l61 661375 1 2024-07-03T15:43:54.172 INFO:tasks.workunit.client.1.smithi148.stdout:6/534: creat d3/d1d/d28/d7f/d85/fb6 x:0 0 0 2024-07-03T15:43:54.172 INFO:tasks.workunit.client.1.smithi148.stdout:7/583: dread d5/d7d/da7/d1c/f41 [0,4194304] 0 2024-07-03T15:43:54.172 INFO:tasks.workunit.client.1.smithi148.stdout:4/417: rmdir d0/d8/d13 39 2024-07-03T15:43:54.173 INFO:tasks.workunit.client.1.smithi148.stdout:6/535: truncate d3/d1d/d28/d7f/d85/fb6 338672 0 2024-07-03T15:43:54.173 INFO:tasks.workunit.client.1.smithi148.stdout:3/482: mkdir dd/d33/d37/d68/d2e/d40/d45/d9a 0 2024-07-03T15:43:54.173 INFO:tasks.workunit.client.1.smithi148.stdout:8/418: creat d4/da/d14/f9b x:0 0 0 2024-07-03T15:43:54.173 INFO:tasks.workunit.client.1.smithi148.stdout:2/582: mkdir d5/d14/d86/db9 0 2024-07-03T15:43:54.173 INFO:tasks.workunit.client.1.smithi148.stdout:8/419: dread - d4/da/d14/f9b zero size 2024-07-03T15:43:54.173 INFO:tasks.workunit.client.1.smithi148.stdout:3/483: write dd/d35/f92 [2636584,83729] 0 2024-07-03T15:43:54.174 INFO:tasks.workunit.client.1.smithi148.stdout:0/496: dwrite da/d44/f75 [0,4194304] 0 2024-07-03T15:43:54.174 INFO:tasks.workunit.client.1.smithi148.stdout:0/497: fsync da/d44/d13/d14/f49 0 2024-07-03T15:43:54.175 INFO:tasks.workunit.client.1.smithi148.stdout:9/746: unlink d0/dd/d11/f9c 0 2024-07-03T15:43:54.178 INFO:tasks.workunit.client.1.smithi148.stdout:7/584: dread d5/d7d/da7/d1c/d4a/d66/d93/faf [0,4194304] 0 2024-07-03T15:43:54.179 INFO:tasks.workunit.client.1.smithi148.stdout:4/418: symlink d0/d28/d7d/d6f/d7c/l8e 0 2024-07-03T15:43:54.179 INFO:tasks.workunit.client.0.smithi028.stdout:5/62: write d0/f9 [2394507,120175] 0 2024-07-03T15:43:54.181 INFO:tasks.workunit.client.1.smithi148.stdout:5/513: dwrite d7/f4f [0,4194304] 0 2024-07-03T15:43:54.181 INFO:tasks.workunit.client.1.smithi148.stdout:5/514: chown d7/da/df/d16/d2f/d44 102 1 2024-07-03T15:43:54.181 INFO:tasks.workunit.client.1.smithi148.stdout:5/515: truncate d7/da/f73 85158 0 2024-07-03T15:43:54.181 INFO:tasks.workunit.client.1.smithi148.stdout:2/583: rmdir d5/d14/d22/d2b/d13 39 2024-07-03T15:43:54.181 INFO:tasks.workunit.client.1.smithi148.stdout:2/584: chown d5/d7/d64/db2 13436914 1 2024-07-03T15:43:54.182 INFO:tasks.workunit.client.0.smithi028.stdout:3/41: dwrite fe [0,4194304] 0 2024-07-03T15:43:54.183 INFO:tasks.workunit.client.0.smithi028.stdout:5/63: rename d0/f7 to d0/dd/d13/f17 0 2024-07-03T15:43:54.186 INFO:tasks.workunit.client.1.smithi148.stdout:8/420: mknod d4/da/d14/d17/d76/c9c 0 2024-07-03T15:43:54.186 INFO:tasks.workunit.client.1.smithi148.stdout:8/421: readlink d4/da/d14/d1a/d32/l62 0 2024-07-03T15:43:54.187 INFO:tasks.workunit.client.1.smithi148.stdout:3/484: dread dd/d33/f39 [0,4194304] 0 2024-07-03T15:43:54.189 INFO:tasks.workunit.client.0.smithi028.stdout:9/67: write d1/f5 [967252,118602] 0 2024-07-03T15:43:54.192 INFO:tasks.workunit.client.0.smithi028.stdout:9/68: read d1/dd/fe [407808,1726] 0 2024-07-03T15:43:54.196 INFO:tasks.workunit.client.0.smithi028.stdout:1/36: truncate d4/de/ff 3575798 0 2024-07-03T15:43:54.197 INFO:tasks.workunit.client.0.smithi028.stdout:9/69: getdents d1/dd 0 2024-07-03T15:43:54.201 INFO:tasks.workunit.client.0.smithi028.stdout:1/37: creat d4/de/f10 x:0 0 0 2024-07-03T15:43:54.201 INFO:tasks.workunit.client.1.smithi148.stdout:3/485: read dd/d33/d37/f82 [1981983,102569] 0 2024-07-03T15:43:54.201 INFO:tasks.workunit.client.1.smithi148.stdout:3/486: chown dd/d35/d41/d89 6 1 2024-07-03T15:43:54.202 INFO:tasks.workunit.client.1.smithi148.stdout:4/419: dwrite d0/d28/f69 [0,4194304] 0 2024-07-03T15:43:54.202 INFO:tasks.workunit.client.1.smithi148.stdout:4/420: chown d0/d8/d19/d45/d49 31 1 2024-07-03T15:43:54.202 INFO:tasks.workunit.client.1.smithi148.stdout:4/421: truncate d0/f50 5066387 0 2024-07-03T15:43:54.203 INFO:tasks.workunit.client.0.smithi028.stdout:1/38: mkdir d4/d11 0 2024-07-03T15:43:54.206 INFO:tasks.workunit.client.0.smithi028.stdout:1/39: unlink d4/f5 0 2024-07-03T15:43:54.206 INFO:tasks.workunit.client.0.smithi028.stdout:9/70: dread d1/f5 [0,4194304] 0 2024-07-03T15:43:54.206 INFO:tasks.workunit.client.0.smithi028.stdout:9/71: truncate d1/f9 146122 0 2024-07-03T15:43:54.207 INFO:tasks.workunit.client.1.smithi148.stdout:5/516: dwrite d7/da/df/f1f [0,4194304] 0 2024-07-03T15:43:54.207 INFO:tasks.workunit.client.1.smithi148.stdout:9/747: symlink d0/d49/d43/d19/de1/lea 0 2024-07-03T15:43:54.208 INFO:tasks.workunit.client.0.smithi028.stdout:3/42: dwrite ff [0,4194304] 0 2024-07-03T15:43:54.209 INFO:tasks.workunit.client.1.smithi148.stdout:6/536: link d3/d1d/f27 d3/fb7 0 2024-07-03T15:43:54.209 INFO:tasks.workunit.client.0.smithi028.stdout:8/46: dwrite d0/d2/ff [0,4194304] 0 2024-07-03T15:43:54.210 INFO:tasks.workunit.client.0.smithi028.stdout:1/40: creat d4/d11/f12 x:0 0 0 2024-07-03T15:43:54.214 INFO:tasks.workunit.client.1.smithi148.stdout:4/422: dread d0/f17 [0,4194304] 0 2024-07-03T15:43:54.214 INFO:tasks.workunit.client.0.smithi028.stdout:9/72: creat d1/dd/f14 x:0 0 0 2024-07-03T15:43:54.214 INFO:tasks.workunit.client.0.smithi028.stdout:9/73: truncate d1/dd/f14 555430 0 2024-07-03T15:43:54.214 INFO:tasks.workunit.client.0.smithi028.stdout:9/74: truncate d1/dd/f14 1469878 0 2024-07-03T15:43:54.219 INFO:tasks.workunit.client.1.smithi148.stdout:2/585: mknod d5/d7/d64/db2/cba 0 2024-07-03T15:43:54.220 INFO:tasks.workunit.client.1.smithi148.stdout:4/423: dread d0/f17 [0,4194304] 0 2024-07-03T15:43:54.220 INFO:tasks.workunit.client.1.smithi148.stdout:4/424: fsync d0/f4 0 2024-07-03T15:43:54.221 INFO:tasks.workunit.client.0.smithi028.stdout:1/41: rename d4/c8 to d4/c13 0 2024-07-03T15:43:54.224 INFO:tasks.workunit.client.0.smithi028.stdout:9/75: symlink d1/l15 0 2024-07-03T15:43:54.227 INFO:tasks.workunit.client.0.smithi028.stdout:5/64: truncate d0/dd/d13/f17 2435823 0 2024-07-03T15:43:54.232 INFO:tasks.workunit.client.1.smithi148.stdout:3/487: link dd/d33/d37/l42 dd/d33/d52/d57/l9b 0 2024-07-03T15:43:54.232 INFO:tasks.workunit.client.1.smithi148.stdout:3/488: write dd/d33/d37/d68/d2e/f13 [1509677,127119] 0 2024-07-03T15:43:54.232 INFO:tasks.workunit.client.1.smithi148.stdout:3/489: fsync dd/d35/f56 0 2024-07-03T15:43:54.232 INFO:tasks.workunit.client.1.smithi148.stdout:0/498: rmdir da/d44/d13/d14/d50 39 2024-07-03T15:43:54.239 INFO:tasks.workunit.client.1.smithi148.stdout:5/517: mkdir d7/da/df/d16/dac 0 2024-07-03T15:43:54.239 INFO:tasks.workunit.client.1.smithi148.stdout:9/748: rmdir d0/d18/d24/d60 39 2024-07-03T15:43:54.239 INFO:tasks.workunit.client.1.smithi148.stdout:5/518: chown d7/da/df/d16/d1a/f34 4 1 2024-07-03T15:43:54.239 INFO:tasks.workunit.client.1.smithi148.stdout:9/749: chown d0/dd/d25/d37/d40/d72/f8d 16757 1 2024-07-03T15:43:54.240 INFO:tasks.workunit.client.1.smithi148.stdout:6/537: symlink d3/d1d/d28/d53/d6d/d8f/lb8 0 2024-07-03T15:43:54.240 INFO:tasks.workunit.client.1.smithi148.stdout:7/585: getdents d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/db2 0 2024-07-03T15:43:54.242 INFO:tasks.workunit.client.1.smithi148.stdout:8/422: rename d4/da/d14/d1a/f6e to d4/da/d51/f9d 0 2024-07-03T15:43:54.242 INFO:tasks.workunit.client.1.smithi148.stdout:8/423: getdents d4/d9a 0 2024-07-03T15:43:54.245 INFO:tasks.workunit.client.1.smithi148.stdout:9/750: creat d0/d18/feb x:0 0 0 2024-07-03T15:43:54.245 INFO:tasks.workunit.client.0.smithi028.stdout:3/43: dwrite fe [0,4194304] 0 2024-07-03T15:43:54.246 INFO:tasks.workunit.client.0.smithi028.stdout:8/47: dwrite d0/d2/ff [0,4194304] 0 2024-07-03T15:43:54.247 INFO:tasks.workunit.client.1.smithi148.stdout:7/586: unlink d5/d7d/da7/d1c/d33/d4c/d21/d31/l64 0 2024-07-03T15:43:54.249 INFO:tasks.workunit.client.1.smithi148.stdout:7/587: creat d5/d7d/da7/d1c/d33/d4c/d84/d98/dc2/fc9 x:0 0 0 2024-07-03T15:43:54.249 INFO:tasks.workunit.client.1.smithi148.stdout:7/588: chown d5/d7d/da7/d1c/d4a/d52/d87/f89 1727939 1 2024-07-03T15:43:54.251 INFO:tasks.workunit.client.0.smithi028.stdout:3/44: rename l1 to l11 0 2024-07-03T15:43:54.251 INFO:tasks.workunit.client.0.smithi028.stdout:3/45: chown l11 6 1 2024-07-03T15:43:54.251 INFO:tasks.workunit.client.0.smithi028.stdout:3/46: stat fc 0 2024-07-03T15:43:54.253 INFO:tasks.workunit.client.0.smithi028.stdout:3/47: symlink l12 0 2024-07-03T15:43:54.254 INFO:tasks.workunit.client.0.smithi028.stdout:3/48: fdatasync f9 0 2024-07-03T15:43:54.254 INFO:tasks.workunit.client.0.smithi028.stdout:3/49: read - f9 zero size 2024-07-03T15:43:54.254 INFO:tasks.workunit.client.1.smithi148.stdout:7/589: rename d5/d7d/da7/d1c/d4a/d9c to d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/db2/dca 0 2024-07-03T15:43:54.256 INFO:tasks.workunit.client.1.smithi148.stdout:7/590: rename d5/d7d/da7/d1c/d33/d4c/d84/c8a to d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/ccb 0 2024-07-03T15:43:54.257 INFO:tasks.workunit.client.1.smithi148.stdout:7/591: dread - d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/fa3 zero size 2024-07-03T15:43:54.257 INFO:tasks.workunit.client.1.smithi148.stdout:8/424: dread d4/d86/f5c [0,4194304] 0 2024-07-03T15:43:54.257 INFO:tasks.workunit.client.1.smithi148.stdout:8/425: write d4/d86/f63 [1286083,35030] 0 2024-07-03T15:43:54.267 INFO:tasks.workunit.client.1.smithi148.stdout:8/426: write d4/da/d51/d18/d20/f2b [1513940,31665] 0 2024-07-03T15:43:54.268 INFO:tasks.workunit.client.1.smithi148.stdout:4/425: dwrite d0/d8/d13/d48/f3f [0,4194304] 0 2024-07-03T15:43:54.269 INFO:tasks.workunit.client.1.smithi148.stdout:2/586: dwrite d5/d7/d64/faf [0,4194304] 0 2024-07-03T15:43:54.269 INFO:tasks.workunit.client.0.smithi028.stdout:1/42: dwrite f2 [0,4194304] 0 2024-07-03T15:43:54.269 INFO:tasks.workunit.client.0.smithi028.stdout:1/43: truncate d4/de/f10 957250 0 2024-07-03T15:43:54.269 INFO:tasks.workunit.client.0.smithi028.stdout:1/44: write d4/fc [3393,3572] 0 2024-07-03T15:43:54.269 INFO:tasks.workunit.client.0.smithi028.stdout:1/45: creat d4/f14 x:0 0 0 2024-07-03T15:43:54.272 INFO:tasks.workunit.client.1.smithi148.stdout:3/490: dwrite dd/d33/d37/d68/d2e/f18 [0,4194304] 0 2024-07-03T15:43:54.272 INFO:tasks.workunit.client.1.smithi148.stdout:3/491: read - dd/d35/f94 zero size 2024-07-03T15:43:54.274 INFO:tasks.workunit.client.0.smithi028.stdout:1/46: mkdir d4/d11/d15 0 2024-07-03T15:43:54.275 INFO:tasks.workunit.client.1.smithi148.stdout:7/592: dwrite d5/f9 [0,4194304] 0 2024-07-03T15:43:54.275 INFO:tasks.workunit.client.1.smithi148.stdout:9/751: dwrite d0/dd/f14 [0,4194304] 0 2024-07-03T15:43:54.276 INFO:tasks.workunit.client.0.smithi028.stdout:8/48: dread d0/d2/d3/fa [4194304,4194304] 0 2024-07-03T15:43:54.278 INFO:tasks.workunit.client.0.smithi028.stdout:1/47: rmdir d4/d11/d15 0 2024-07-03T15:43:54.279 INFO:tasks.workunit.client.0.smithi028.stdout:8/49: mkdir d0/d2/d13 0 2024-07-03T15:43:54.279 INFO:tasks.workunit.client.0.smithi028.stdout:8/50: getdents d0/dd 0 2024-07-03T15:43:54.280 INFO:tasks.workunit.client.1.smithi148.stdout:4/426: dread d0/d8/d13/f1a [0,4194304] 0 2024-07-03T15:43:54.280 INFO:tasks.workunit.client.1.smithi148.stdout:4/427: chown d0/d28/d7d/f8a 0 1 2024-07-03T15:43:54.280 INFO:tasks.workunit.client.1.smithi148.stdout:4/428: chown d0/d8/d44/f6a 5 1 2024-07-03T15:43:54.280 INFO:tasks.workunit.client.1.smithi148.stdout:4/429: dread - d0/d8/d22/d2f/d41/f7a zero size 2024-07-03T15:43:54.280 INFO:tasks.workunit.client.1.smithi148.stdout:4/430: chown d0/d8/d19/c23 15715 1 2024-07-03T15:43:54.281 INFO:tasks.workunit.client.0.smithi028.stdout:1/48: mkdir d4/d11/d16 0 2024-07-03T15:43:54.281 INFO:tasks.workunit.client.0.smithi028.stdout:1/49: chown d4/c13 20091442 1 2024-07-03T15:43:54.284 INFO:tasks.workunit.client.0.smithi028.stdout:8/51: rename d0/d2/d3/l6 to d0/d2/d3/dc/d11/d12/l14 0 2024-07-03T15:43:54.284 INFO:tasks.workunit.client.0.smithi028.stdout:8/52: chown d0/d2/d3/fa 5861243 1 2024-07-03T15:43:54.285 INFO:tasks.workunit.client.0.smithi028.stdout:3/50: truncate fe 2277119 0 2024-07-03T15:43:54.286 INFO:tasks.workunit.client.1.smithi148.stdout:9/752: dread d0/d18/f21 [8388608,4194304] 0 2024-07-03T15:43:54.287 INFO:tasks.workunit.client.0.smithi028.stdout:5/65: dwrite d0/dd/d13/f17 [0,4194304] 0 2024-07-03T15:43:54.287 INFO:tasks.workunit.client.0.smithi028.stdout:5/66: read d0/f9 [2228839,94016] 0 2024-07-03T15:43:54.287 INFO:tasks.workunit.client.0.smithi028.stdout:8/53: creat d0/d2/d3/dc/d11/d12/f15 x:0 0 0 2024-07-03T15:43:54.287 INFO:tasks.workunit.client.1.smithi148.stdout:2/587: truncate d5/d14/d22/d2b/d13/d41/f63 317060 0 2024-07-03T15:43:54.289 INFO:tasks.workunit.client.0.smithi028.stdout:5/67: symlink d0/dd/d14/l18 0 2024-07-03T15:43:54.289 INFO:tasks.workunit.client.0.smithi028.stdout:5/68: truncate d0/dd/df/f16 318555 0 2024-07-03T15:43:54.290 INFO:tasks.workunit.client.1.smithi148.stdout:3/492: unlink dd/d35/f92 0 2024-07-03T15:43:54.292 INFO:tasks.workunit.client.0.smithi028.stdout:8/54: mknod d0/d2/d3/dc/c16 0 2024-07-03T15:43:54.292 INFO:tasks.workunit.client.0.smithi028.stdout:5/69: dread d0/f9 [0,4194304] 0 2024-07-03T15:43:54.293 INFO:tasks.workunit.client.0.smithi028.stdout:5/70: read d0/dd/df/f16 [30392,130980] 0 2024-07-03T15:43:54.295 INFO:tasks.workunit.client.0.smithi028.stdout:1/50: read d4/de/ff [3033201,87083] 0 2024-07-03T15:43:54.295 INFO:tasks.workunit.client.0.smithi028.stdout:1/51: rename d4 to d4/d17 22 2024-07-03T15:43:54.295 INFO:tasks.workunit.client.0.smithi028.stdout:1/52: dread - d4/d11/f12 zero size 2024-07-03T15:43:54.295 INFO:tasks.workunit.client.0.smithi028.stdout:1/53: dread - d4/fb zero size 2024-07-03T15:43:54.295 INFO:tasks.workunit.client.0.smithi028.stdout:5/71: mkdir d0/dd/d13/d19 0 2024-07-03T15:43:54.297 INFO:tasks.workunit.client.1.smithi148.stdout:8/427: rmdir d4/d9a 0 2024-07-03T15:43:54.297 INFO:tasks.workunit.client.1.smithi148.stdout:8/428: chown d4/da/f28 100537 1 2024-07-03T15:43:54.297 INFO:tasks.workunit.client.1.smithi148.stdout:8/429: write d4/da/d47/f94 [886270,22419] 0 2024-07-03T15:43:54.298 INFO:tasks.workunit.client.0.smithi028.stdout:1/54: mkdir d4/d18 0 2024-07-03T15:43:54.299 INFO:tasks.workunit.client.1.smithi148.stdout:4/431: truncate d0/d28/f63 4676542 0 2024-07-03T15:43:54.299 INFO:tasks.workunit.client.1.smithi148.stdout:4/432: creat d0/d8/d13/d48/f8f x:0 0 0 2024-07-03T15:43:54.300 INFO:tasks.workunit.client.1.smithi148.stdout:4/433: truncate d0/d8/d3a/f66 145614 0 2024-07-03T15:43:54.302 INFO:tasks.workunit.client.0.smithi028.stdout:1/55: rename d4/fb to d4/f19 0 2024-07-03T15:43:54.303 INFO:tasks.workunit.client.0.smithi028.stdout:1/56: creat d4/f1a x:0 0 0 2024-07-03T15:43:54.303 INFO:tasks.workunit.client.1.smithi148.stdout:3/493: getdents dd/d33/d52/d57 0 2024-07-03T15:43:54.315 INFO:tasks.workunit.client.1.smithi148.stdout:3/494: creat dd/d33/d37/d68/f9c x:0 0 0 2024-07-03T15:43:54.315 INFO:tasks.workunit.client.1.smithi148.stdout:3/495: truncate dd/d33/f39 4409916 0 2024-07-03T15:43:54.316 INFO:tasks.workunit.client.1.smithi148.stdout:2/588: rename d5/d14/d22/d2b to d5/d14/d86/db9/dbb 0 2024-07-03T15:43:54.318 INFO:tasks.workunit.client.1.smithi148.stdout:3/496: unlink dd/d33/d37/d68/d2e/d40/d77/l8e 0 2024-07-03T15:43:54.320 INFO:tasks.workunit.client.1.smithi148.stdout:8/430: rename d4/da/d14/d17/d76 to d4/da/d14/d9e 0 2024-07-03T15:43:54.321 INFO:tasks.workunit.client.1.smithi148.stdout:8/431: creat d4/da/d51/d18/d20/d82/f9f x:0 0 0 2024-07-03T15:43:54.321 INFO:tasks.workunit.client.1.smithi148.stdout:8/432: fsync d4/da/d14/d17/f48 0 2024-07-03T15:43:54.325 INFO:tasks.workunit.client.1.smithi148.stdout:8/433: dread d4/d86/f5c [0,4194304] 0 2024-07-03T15:43:54.325 INFO:tasks.workunit.client.1.smithi148.stdout:8/434: dread - d4/da/d51/d53/f7f zero size 2024-07-03T15:43:54.326 INFO:tasks.workunit.client.0.smithi028.stdout:8/55: dwrite d0/d2/ff [0,4194304] 0 2024-07-03T15:43:54.326 INFO:tasks.workunit.client.0.smithi028.stdout:8/56: write d0/d2/d3/dc/d11/d12/f15 [810251,57754] 0 2024-07-03T15:43:54.327 INFO:tasks.workunit.client.1.smithi148.stdout:8/435: rename d4/da/d14/d1a/d32/f73 to d4/da/d47/d88/fa0 0 2024-07-03T15:43:54.327 INFO:tasks.workunit.client.1.smithi148.stdout:4/434: dwrite d0/d28/d7d/f8a [0,4194304] 0 2024-07-03T15:43:54.327 INFO:tasks.workunit.client.0.smithi028.stdout:1/57: dwrite d4/f19 [0,4194304] 0 2024-07-03T15:43:54.327 INFO:tasks.workunit.client.0.smithi028.stdout:1/58: write d4/f14 [665701,25171] 0 2024-07-03T15:43:54.328 INFO:tasks.workunit.client.1.smithi148.stdout:8/436: creat d4/da/d14/d1a/d32/d89/fa1 x:0 0 0 2024-07-03T15:43:54.328 INFO:tasks.workunit.client.1.smithi148.stdout:8/437: chown d4/da/d14/d17/f48 52729 1 2024-07-03T15:43:54.328 INFO:tasks.workunit.client.1.smithi148.stdout:3/497: read f5 [5010668,63516] 0 2024-07-03T15:43:54.328 INFO:tasks.workunit.client.1.smithi148.stdout:3/498: truncate dd/f2a 5119148 0 2024-07-03T15:43:54.329 INFO:tasks.workunit.client.1.smithi148.stdout:3/499: stat dd/d33/d37 0 2024-07-03T15:43:54.329 INFO:tasks.workunit.client.1.smithi148.stdout:1/423: sync 2024-07-03T15:43:54.329 INFO:tasks.workunit.client.1.smithi148.stdout:1/424: truncate d0/d2/d2e/d39/d41/f65 143475 0 2024-07-03T15:43:54.339 INFO:tasks.workunit.client.1.smithi148.stdout:3/500: dread dd/d33/d37/f53 [0,4194304] 0 2024-07-03T15:43:54.339 INFO:tasks.workunit.client.1.smithi148.stdout:3/501: dread - dd/d33/d37/d68/d2e/d61/f63 zero size 2024-07-03T15:43:54.339 INFO:tasks.workunit.client.1.smithi148.stdout:3/502: dread - dd/d33/f87 zero size 2024-07-03T15:43:54.339 INFO:tasks.workunit.client.1.smithi148.stdout:3/503: chown dd/d35/d49/l8a 747195 1 2024-07-03T15:43:54.340 INFO:tasks.workunit.client.0.smithi028.stdout:3/51: dread fe [0,4194304] 0 2024-07-03T15:43:54.364 INFO:tasks.workunit.client.0.smithi028.stdout:1/59: dread f2 [0,4194304] 0 2024-07-03T15:43:54.364 INFO:tasks.workunit.client.0.smithi028.stdout:1/60: truncate d4/de/f10 1866350 0 2024-07-03T15:43:54.390 INFO:tasks.workunit.client.0.smithi028.stdout:1/61: write d4/de/f10 [1861726,97768] 0 2024-07-03T15:43:54.390 INFO:tasks.workunit.client.0.smithi028.stdout:1/62: creat d4/d11/f1b x:0 0 0 2024-07-03T15:43:54.392 INFO:tasks.workunit.client.0.smithi028.stdout:5/72: truncate d0/dd/df/f16 314773 0 2024-07-03T15:43:54.392 INFO:tasks.workunit.client.0.smithi028.stdout:5/73: readlink d0/dd/d14/l18 0 2024-07-03T15:43:54.392 INFO:tasks.workunit.client.0.smithi028.stdout:5/74: chown d0/l11 2 1 2024-07-03T15:43:54.392 INFO:tasks.workunit.client.0.smithi028.stdout:5/75: chown d0/l11 14988650 1 2024-07-03T15:43:54.392 INFO:tasks.workunit.client.0.smithi028.stdout:5/76: creat d0/dd/df/f1a x:0 0 0 2024-07-03T15:43:54.392 INFO:tasks.workunit.client.1.smithi148.stdout:4/435: symlink d0/d8/d13/d8b/l90 0 2024-07-03T15:43:54.398 INFO:tasks.workunit.client.0.smithi028.stdout:8/57: dwrite d0/d2/d3/dc/d11/d12/f15 [0,4194304] 0 2024-07-03T15:43:54.399 INFO:tasks.workunit.client.1.smithi148.stdout:8/438: dwrite d4/da/d51/f69 [0,4194304] 0 2024-07-03T15:43:54.399 INFO:tasks.workunit.client.1.smithi148.stdout:1/425: dwrite d0/d2/d2e/d39/d42/f49 [4194304,4194304] 0 2024-07-03T15:43:54.399 INFO:tasks.workunit.client.1.smithi148.stdout:1/426: write d0/d2/d6/fa [2403920,85808] 0 2024-07-03T15:43:54.402 INFO:tasks.workunit.client.0.smithi028.stdout:5/77: creat d0/dd/f1b x:0 0 0 2024-07-03T15:43:54.402 INFO:tasks.workunit.client.0.smithi028.stdout:5/78: truncate d0/dd/f1b 971600 0 2024-07-03T15:43:54.403 INFO:tasks.workunit.client.0.smithi028.stdout:3/52: symlink l13 0 2024-07-03T15:43:54.405 INFO:tasks.workunit.client.1.smithi148.stdout:3/504: creat dd/d33/d37/d68/d2e/d40/d45/d9a/f9d x:0 0 0 2024-07-03T15:43:54.405 INFO:tasks.workunit.client.1.smithi148.stdout:3/505: chown dd/d33/d37/d68/d2e/d40/d45/d9a 31484556 1 2024-07-03T15:43:54.410 INFO:tasks.workunit.client.0.smithi028.stdout:8/58: unlink d0/d2/l5 0 2024-07-03T15:43:54.411 INFO:tasks.workunit.client.1.smithi148.stdout:4/436: dwrite d0/d8/d13/d67/f86 [0,4194304] 0 2024-07-03T15:43:54.413 INFO:tasks.workunit.client.0.smithi028.stdout:5/79: mkdir d0/d1c 0 2024-07-03T15:43:54.414 INFO:tasks.workunit.client.0.smithi028.stdout:5/80: write d0/dd/f1b [681896,81770] 0 2024-07-03T15:43:54.414 INFO:tasks.workunit.client.1.smithi148.stdout:8/439: symlink d4/da/d14/d1a/d32/la2 0 2024-07-03T15:43:54.415 INFO:tasks.workunit.client.0.smithi028.stdout:3/53: creat f14 x:0 0 0 2024-07-03T15:43:54.418 INFO:tasks.workunit.client.0.smithi028.stdout:5/81: symlink d0/l1d 0 2024-07-03T15:43:54.418 INFO:tasks.workunit.client.0.smithi028.stdout:5/82: chown d0/dd/ce 2033 1 2024-07-03T15:43:54.419 INFO:tasks.workunit.client.0.smithi028.stdout:5/83: chown d0/f9 15 1 2024-07-03T15:43:54.422 INFO:tasks.workunit.client.1.smithi148.stdout:1/427: rename d0/d2/d11/d1e/f66 to d0/d2/d6/d14/f8e 0 2024-07-03T15:43:54.422 INFO:tasks.workunit.client.1.smithi148.stdout:1/428: stat d0/d2/d6/d14/f8e 0 2024-07-03T15:43:54.423 INFO:tasks.workunit.client.0.smithi028.stdout:3/54: dread f6 [0,4194304] 0 2024-07-03T15:43:54.423 INFO:tasks.workunit.client.0.smithi028.stdout:5/84: fsync d0/dd/f1b 0 2024-07-03T15:43:54.423 INFO:tasks.workunit.client.0.smithi028.stdout:1/63: truncate d4/de/f10 1181891 0 2024-07-03T15:43:54.424 INFO:tasks.workunit.client.1.smithi148.stdout:4/437: truncate d0/d8/d22/d2f/f6c 12004 0 2024-07-03T15:43:54.425 INFO:tasks.workunit.client.1.smithi148.stdout:3/506: dwrite dd/f2a [4194304,4194304] 0 2024-07-03T15:43:54.425 INFO:tasks.workunit.client.1.smithi148.stdout:3/507: chown dd/d33/d52/c90 0 1 2024-07-03T15:43:54.425 INFO:tasks.workunit.client.1.smithi148.stdout:8/440: creat d4/da/d47/d88/d92/fa3 x:0 0 0 2024-07-03T15:43:54.426 INFO:tasks.workunit.client.1.smithi148.stdout:8/441: dread d4/da/d14/f68 [0,4194304] 0 2024-07-03T15:43:54.427 INFO:tasks.workunit.client.0.smithi028.stdout:5/85: mkdir d0/dd/d1e 0 2024-07-03T15:43:54.428 INFO:tasks.workunit.client.0.smithi028.stdout:1/64: stat d4/c13 0 2024-07-03T15:43:54.429 INFO:tasks.workunit.client.0.smithi028.stdout:5/86: creat d0/dd/d13/d19/f1f x:0 0 0 2024-07-03T15:43:54.431 INFO:tasks.workunit.client.0.smithi028.stdout:1/65: mkdir d4/de/d1c 0 2024-07-03T15:43:54.431 INFO:tasks.workunit.client.0.smithi028.stdout:1/66: fsync d4/f1a 0 2024-07-03T15:43:54.433 INFO:tasks.workunit.client.1.smithi148.stdout:8/442: unlink d4/da/d14/d17/f30 0 2024-07-03T15:43:54.435 INFO:tasks.workunit.client.1.smithi148.stdout:6/538: sync 2024-07-03T15:43:54.435 INFO:tasks.workunit.client.1.smithi148.stdout:6/539: getdents d3/d1d/d28/d7f/d85/da2 0 2024-07-03T15:43:54.435 INFO:tasks.workunit.client.1.smithi148.stdout:5/519: sync 2024-07-03T15:43:54.435 INFO:tasks.workunit.client.1.smithi148.stdout:0/499: sync 2024-07-03T15:43:54.435 INFO:tasks.workunit.client.1.smithi148.stdout:0/500: dread - da/d17/d2a/f7a zero size 2024-07-03T15:43:54.436 INFO:tasks.workunit.client.0.smithi028.stdout:5/87: creat d0/dd/d14/f20 x:0 0 0 2024-07-03T15:43:54.436 INFO:tasks.workunit.client.0.smithi028.stdout:3/55: dread ff [0,4194304] 0 2024-07-03T15:43:54.436 INFO:tasks.workunit.client.0.smithi028.stdout:1/67: symlink d4/d11/d16/l1d 0 2024-07-03T15:43:54.437 INFO:tasks.workunit.client.0.smithi028.stdout:1/68: mkdir d4/de/d1e 0 2024-07-03T15:43:54.438 INFO:tasks.workunit.client.1.smithi148.stdout:5/520: dread d7/da/df/d1d/f51 [0,4194304] 0 2024-07-03T15:43:54.444 INFO:tasks.workunit.client.1.smithi148.stdout:6/540: write d3/f1e [1898856,27400] 0 2024-07-03T15:43:54.444 INFO:tasks.workunit.client.1.smithi148.stdout:6/541: dread - d3/d1d/d28/d31/d4c/f5d zero size 2024-07-03T15:43:54.444 INFO:tasks.workunit.client.0.smithi028.stdout:3/56: dread f4 [0,4194304] 0 2024-07-03T15:43:54.444 INFO:tasks.workunit.client.0.smithi028.stdout:3/57: write f4 [4792495,59707] 0 2024-07-03T15:43:54.444 INFO:tasks.workunit.client.0.smithi028.stdout:3/58: stat f6 0 2024-07-03T15:43:54.445 INFO:tasks.workunit.client.1.smithi148.stdout:6/542: truncate d3/d1d/d28/d53/f80 3119017 0 2024-07-03T15:43:54.445 INFO:tasks.workunit.client.0.smithi028.stdout:3/59: creat f15 x:0 0 0 2024-07-03T15:43:54.446 INFO:tasks.workunit.client.1.smithi148.stdout:6/543: read - d3/d50/d77/fa7 zero size 2024-07-03T15:43:54.446 INFO:tasks.workunit.client.0.smithi028.stdout:3/60: symlink l16 0 2024-07-03T15:43:54.447 INFO:tasks.workunit.client.0.smithi028.stdout:3/61: write fc [837849,127265] 0 2024-07-03T15:43:54.451 INFO:tasks.workunit.client.0.smithi028.stdout:1/69: write d4/fd [2891893,119639] 0 2024-07-03T15:43:54.451 INFO:tasks.workunit.client.0.smithi028.stdout:1/70: write d4/d11/f12 [659455,109711] 0 2024-07-03T15:43:54.452 INFO:tasks.workunit.client.0.smithi028.stdout:1/71: stat d4/de/f10 0 2024-07-03T15:43:54.452 INFO:tasks.workunit.client.0.smithi028.stdout:1/72: creat d4/de/f1f x:0 0 0 2024-07-03T15:43:54.454 INFO:tasks.workunit.client.0.smithi028.stdout:1/73: symlink d4/d18/l20 0 2024-07-03T15:43:54.454 INFO:tasks.workunit.client.0.smithi028.stdout:1/74: chown f2 90797610 1 2024-07-03T15:43:54.456 INFO:tasks.workunit.client.1.smithi148.stdout:4/438: dread d0/d28/d7d/f8a [0,4194304] 0 2024-07-03T15:43:54.459 INFO:tasks.workunit.client.1.smithi148.stdout:5/521: read f4 [3153648,82355] 0 2024-07-03T15:43:54.459 INFO:tasks.workunit.client.1.smithi148.stdout:4/439: mkdir d0/d91 0 2024-07-03T15:43:54.460 INFO:tasks.workunit.client.1.smithi148.stdout:4/440: dread - d0/d8/d19/d45/f75 zero size 2024-07-03T15:43:54.460 INFO:tasks.workunit.client.1.smithi148.stdout:7/593: sync 2024-07-03T15:43:54.461 INFO:tasks.workunit.client.1.smithi148.stdout:4/441: rmdir d0/d28/d8c 0 2024-07-03T15:43:54.461 INFO:tasks.workunit.client.1.smithi148.stdout:7/594: truncate d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/f28 3344884 0 2024-07-03T15:43:54.462 INFO:tasks.workunit.client.1.smithi148.stdout:4/442: symlink d0/d28/d7d/l92 0 2024-07-03T15:43:54.463 INFO:tasks.workunit.client.1.smithi148.stdout:7/595: mkdir d5/d7d/da7/d1c/d33/d4c/d21/d31/dcc 0 2024-07-03T15:43:54.463 INFO:tasks.workunit.client.1.smithi148.stdout:7/596: write d5/d7d/da7/d1c/d4a/d66/d93/f9d [6369,97584] 0 2024-07-03T15:43:54.465 INFO:tasks.workunit.client.1.smithi148.stdout:4/443: symlink d0/d8/d22/d2f/d41/l93 0 2024-07-03T15:43:54.465 INFO:tasks.workunit.client.1.smithi148.stdout:4/444: write d0/d8/d22/d2f/d41/f7a [813110,89736] 0 2024-07-03T15:43:54.466 INFO:tasks.workunit.client.1.smithi148.stdout:4/445: dread - d0/d28/d7d/d6f/f82 zero size 2024-07-03T15:43:54.466 INFO:tasks.workunit.client.1.smithi148.stdout:4/446: write d0/d8/f2d [125347,69690] 0 2024-07-03T15:43:54.466 INFO:tasks.workunit.client.1.smithi148.stdout:4/447: chown d0/d8/d13/c73 124327402 1 2024-07-03T15:43:54.467 INFO:tasks.workunit.client.1.smithi148.stdout:7/597: creat d5/d7d/da7/d1c/d33/d4c/d21/d31/dcc/fcd x:0 0 0 2024-07-03T15:43:54.471 INFO:tasks.workunit.client.1.smithi148.stdout:1/429: dwrite d0/d2/d2e/d39/d41/f7a [0,4194304] 0 2024-07-03T15:43:54.471 INFO:tasks.workunit.client.1.smithi148.stdout:1/430: readlink d0/d2/d11/l73 0 2024-07-03T15:43:54.476 INFO:tasks.workunit.client.1.smithi148.stdout:1/431: mknod d0/d2/d6/d14/d29/c8f 0 2024-07-03T15:43:54.477 INFO:tasks.workunit.client.1.smithi148.stdout:8/443: dwrite d4/da/d47/d88/d92/f4b [0,4194304] 0 2024-07-03T15:43:54.477 INFO:tasks.workunit.client.1.smithi148.stdout:8/444: chown d4/da/d14/d3a/l6d 38 1 2024-07-03T15:43:54.477 INFO:tasks.workunit.client.1.smithi148.stdout:8/445: chown d4/da/d51/d18/c3f 4039 1 2024-07-03T15:43:54.477 INFO:tasks.workunit.client.0.smithi028.stdout:5/88: dwrite d0/dd/df/f1a [0,4194304] 0 2024-07-03T15:43:54.478 INFO:tasks.workunit.client.1.smithi148.stdout:1/432: write d0/d2/d2e/d39/f5f [212377,72883] 0 2024-07-03T15:43:54.478 INFO:tasks.workunit.client.1.smithi148.stdout:7/598: dread d5/d7d/da7/d1c/d23/f2b [0,4194304] 0 2024-07-03T15:43:54.481 INFO:tasks.workunit.client.0.smithi028.stdout:5/89: rename d0/dd/df/f1a to d0/dd/d13/d19/f21 0 2024-07-03T15:43:54.484 INFO:tasks.workunit.client.1.smithi148.stdout:8/446: symlink d4/da/d50/la4 0 2024-07-03T15:43:54.484 INFO:tasks.workunit.client.1.smithi148.stdout:1/433: mknod d0/d2/d2e/d39/d42/c90 0 2024-07-03T15:43:54.485 INFO:tasks.workunit.client.1.smithi148.stdout:7/599: rename d5/d7d/da7/d1c/d23/fbd to d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/fce 0 2024-07-03T15:43:54.486 INFO:tasks.workunit.client.1.smithi148.stdout:8/447: symlink d4/d86/la5 0 2024-07-03T15:43:54.486 INFO:tasks.workunit.client.1.smithi148.stdout:7/600: mkdir d5/d7d/da7/d1c/d4a/dcf 0 2024-07-03T15:43:54.486 INFO:tasks.workunit.client.1.smithi148.stdout:7/601: write d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/fa3 [405910,56860] 0 2024-07-03T15:43:54.487 INFO:tasks.workunit.client.1.smithi148.stdout:7/602: rmdir d5/d7d/da7/d1c/d33/d4c/d21/d39 39 2024-07-03T15:43:54.488 INFO:tasks.workunit.client.1.smithi148.stdout:4/448: dwrite d0/d8/d22/d62/f6b [4194304,4194304] 0 2024-07-03T15:43:54.489 INFO:tasks.workunit.client.1.smithi148.stdout:7/603: link d5/d7d/da7/d1c/l70 d5/d7d/da7/d1c/d59/ld0 0 2024-07-03T15:43:54.489 INFO:tasks.workunit.client.1.smithi148.stdout:7/604: chown d5/d7d/da7/d1c/d4a/d52 0 1 2024-07-03T15:43:54.490 INFO:tasks.workunit.client.1.smithi148.stdout:4/449: unlink d0/d8/d13/c12 0 2024-07-03T15:43:54.491 INFO:tasks.workunit.client.1.smithi148.stdout:7/605: mknod d5/d7d/da7/d1c/d4a/d52/cd1 0 2024-07-03T15:43:54.492 INFO:tasks.workunit.client.1.smithi148.stdout:7/606: mkdir d5/d7d/da7/d1c/d33/d4c/d21/d39/dd2 0 2024-07-03T15:43:54.492 INFO:tasks.workunit.client.1.smithi148.stdout:7/607: readlink d5/d7d/da7/d1c/d23/l8c 0 2024-07-03T15:43:54.492 INFO:tasks.workunit.client.1.smithi148.stdout:7/608: getdents d5/d7d/da7/dc7 0 2024-07-03T15:43:54.492 INFO:tasks.workunit.client.1.smithi148.stdout:7/609: fdatasync d5/d7d/da7/d1c/d23/fc1 0 2024-07-03T15:43:54.493 INFO:tasks.workunit.client.1.smithi148.stdout:7/610: mkdir d5/d7d/da7/d1c/d33/d4c/d21/d47/d8b/dd3 0 2024-07-03T15:43:54.493 INFO:tasks.workunit.client.1.smithi148.stdout:7/611: readlink d5/d7d/da7/d1c/la2 0 2024-07-03T15:43:54.493 INFO:tasks.workunit.client.0.smithi028.stdout:1/75: dwrite d4/f19 [0,4194304] 0 2024-07-03T15:43:54.494 INFO:tasks.workunit.client.1.smithi148.stdout:7/612: mknod d5/d7d/da7/d1c/d33/d4c/d84/da4/cd4 0 2024-07-03T15:43:54.494 INFO:tasks.workunit.client.1.smithi148.stdout:7/613: rmdir d5/d7d/da7/d1c/d33 39 2024-07-03T15:43:54.494 INFO:tasks.workunit.client.1.smithi148.stdout:7/614: read d5/d7d/da7/d1c/d4a/d66/d93/f9d [6366,83026] 0 2024-07-03T15:43:54.494 INFO:tasks.workunit.client.1.smithi148.stdout:7/615: dread - d5/d7d/da7/fa5 zero size 2024-07-03T15:43:54.495 INFO:tasks.workunit.client.1.smithi148.stdout:5/522: dwrite d7/da/df/d16/f8a [0,4194304] 0 2024-07-03T15:43:54.496 INFO:tasks.workunit.client.1.smithi148.stdout:5/523: chown d7/d13/d23/c4a 38823406 1 2024-07-03T15:43:54.496 INFO:tasks.workunit.client.1.smithi148.stdout:6/544: dwrite d3/d50/f96 [0,4194304] 0 2024-07-03T15:43:54.496 INFO:tasks.workunit.client.1.smithi148.stdout:1/434: write d0/d2/d2e/f85 [1433961,124509] 0 2024-07-03T15:43:54.496 INFO:tasks.workunit.client.1.smithi148.stdout:7/616: creat d5/d7d/da7/d1c/d33/d4c/d21/d47/fd5 x:0 0 0 2024-07-03T15:43:54.498 INFO:tasks.workunit.client.1.smithi148.stdout:8/448: dread d4/da/d14/f91 [0,4194304] 0 2024-07-03T15:43:54.498 INFO:tasks.workunit.client.1.smithi148.stdout:8/449: stat d4/da/d50/f95 0 2024-07-03T15:43:54.501 INFO:tasks.workunit.client.1.smithi148.stdout:5/524: creat d7/da/df/d16/d1a/d58/fad x:0 0 0 2024-07-03T15:43:54.501 INFO:tasks.workunit.client.1.smithi148.stdout:5/525: fsync d7/d13/d23/f85 0 2024-07-03T15:43:54.508 INFO:tasks.workunit.client.1.smithi148.stdout:4/450: dwrite d0/d8/d44/f6a [4194304,4194304] 0 2024-07-03T15:43:54.509 INFO:tasks.workunit.client.1.smithi148.stdout:7/617: mkdir d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/db2/dd6 0 2024-07-03T15:43:54.509 INFO:tasks.workunit.client.1.smithi148.stdout:6/545: rename d3/d1d/f25 to d3/da5/fb9 0 2024-07-03T15:43:54.509 INFO:tasks.workunit.client.1.smithi148.stdout:1/435: read d0/d2/d2e/d39/d41/f69 [7344120,50002] 0 2024-07-03T15:43:54.512 INFO:tasks.workunit.client.0.smithi028.stdout:1/76: dwrite d4/de/f1f [0,4194304] 0 2024-07-03T15:43:54.512 INFO:tasks.workunit.client.0.smithi028.stdout:1/77: write d4/f1a [336052,98442] 0 2024-07-03T15:43:54.516 INFO:tasks.workunit.client.1.smithi148.stdout:6/546: creat d3/d1d/d28/d53/d6d/fba x:0 0 0 2024-07-03T15:43:54.517 INFO:tasks.workunit.client.1.smithi148.stdout:9/753: sync 2024-07-03T15:43:54.518 INFO:tasks.workunit.client.1.smithi148.stdout:9/754: creat d0/d18/d24/d47/d4d/fec x:0 0 0 2024-07-03T15:43:54.518 INFO:tasks.workunit.client.1.smithi148.stdout:9/755: chown d0 334024975 1 2024-07-03T15:43:54.518 INFO:tasks.workunit.client.1.smithi148.stdout:8/450: rename d4/da/d14/d17/f48 to d4/da/d14/d1a/d32/d38/d58/fa6 0 2024-07-03T15:43:54.518 INFO:tasks.workunit.client.1.smithi148.stdout:2/589: sync 2024-07-03T15:43:54.524 INFO:tasks.workunit.client.1.smithi148.stdout:1/436: dread d0/d2/d2e/d39/d41/f7a [0,4194304] 0 2024-07-03T15:43:54.524 INFO:tasks.workunit.client.1.smithi148.stdout:6/547: write d3/d1d/d28/d53/f80 [1094761,5782] 0 2024-07-03T15:43:54.525 INFO:tasks.workunit.client.1.smithi148.stdout:5/526: rename d7/da/df/d16/d2f/d44/f70 to d7/da/df/d16/fae 0 2024-07-03T15:43:54.525 INFO:tasks.workunit.client.1.smithi148.stdout:5/527: stat d7/da/df/f3d 0 2024-07-03T15:43:54.525 INFO:tasks.workunit.client.1.smithi148.stdout:6/548: write d3/d1d/d28/d53/f80 [324505,84349] 0 2024-07-03T15:43:54.525 INFO:tasks.workunit.client.1.smithi148.stdout:6/549: write d3/d1d/d28/d31/f98 [391908,11983] 0 2024-07-03T15:43:54.527 INFO:tasks.workunit.client.0.smithi028.stdout:1/78: write d4/de/f1f [280016,31971] 0 2024-07-03T15:43:54.528 INFO:tasks.workunit.client.1.smithi148.stdout:1/437: fdatasync d0/d2/d2e/f85 0 2024-07-03T15:43:54.529 INFO:tasks.workunit.client.1.smithi148.stdout:9/756: dread d0/dd/d25/d6a/f6e [0,4194304] 0 2024-07-03T15:43:54.530 INFO:tasks.workunit.client.0.smithi028.stdout:1/79: mknod d4/de/c21 0 2024-07-03T15:43:54.532 INFO:tasks.workunit.client.1.smithi148.stdout:5/528: mknod d7/d13/d23/caf 0 2024-07-03T15:43:54.532 INFO:tasks.workunit.client.1.smithi148.stdout:4/451: getdents d0/d8/d13 0 2024-07-03T15:43:54.532 INFO:tasks.workunit.client.1.smithi148.stdout:4/452: chown d0/d8/f4b 716209033 1 2024-07-03T15:43:54.533 INFO:tasks.workunit.client.1.smithi148.stdout:8/451: dread d4/da/d51/d18/d20/f2b [0,4194304] 0 2024-07-03T15:43:54.533 INFO:tasks.workunit.client.1.smithi148.stdout:8/452: fdatasync d4/da/d14/d1a/d32/f5e 0 2024-07-03T15:43:54.533 INFO:tasks.workunit.client.1.smithi148.stdout:8/453: creat d4/da/d51/d18/d20/d82/fa7 x:0 0 0 2024-07-03T15:43:54.533 INFO:tasks.workunit.client.1.smithi148.stdout:8/454: fsync d4/da/d14/f8d 0 2024-07-03T15:43:54.533 INFO:tasks.workunit.client.1.smithi148.stdout:6/550: fdatasync d3/d1d/f38 0 2024-07-03T15:43:54.533 INFO:tasks.workunit.client.1.smithi148.stdout:6/551: chown d3/f34 46850200 1 2024-07-03T15:43:54.533 INFO:tasks.workunit.client.1.smithi148.stdout:6/552: chown d3/d1d/d28/c33 502 1 2024-07-03T15:43:54.534 INFO:tasks.workunit.client.0.smithi028.stdout:1/80: mkdir d4/de/d1e/d22 0 2024-07-03T15:43:54.538 INFO:tasks.workunit.client.1.smithi148.stdout:5/529: rename d7/da/df/d1d/f87 to d7/da/df/d1d/fb0 0 2024-07-03T15:43:54.538 INFO:tasks.workunit.client.1.smithi148.stdout:5/530: readlink l1 0 2024-07-03T15:43:54.539 INFO:tasks.workunit.client.0.smithi028.stdout:1/81: symlink d4/de/d1c/l23 0 2024-07-03T15:43:54.539 INFO:tasks.workunit.client.1.smithi148.stdout:2/590: dwrite d5/d14/d22/d60/f91 [0,4194304] 0 2024-07-03T15:43:54.541 INFO:tasks.workunit.client.1.smithi148.stdout:8/455: mkdir d4/da/d14/d3a/da8 0 2024-07-03T15:43:54.542 INFO:tasks.workunit.client.0.smithi028.stdout:1/82: symlink d4/d18/l24 0 2024-07-03T15:43:54.543 INFO:tasks.workunit.client.1.smithi148.stdout:6/553: mknod d3/d1d/d28/d7f/cbb 0 2024-07-03T15:43:54.546 INFO:tasks.workunit.client.1.smithi148.stdout:7/618: rename d5/d7d/da7/d1c/d33/d4c/d21/d31/dcc to d5/d7d/da7/d1c/d4a/d66/dd7 0 2024-07-03T15:43:54.547 INFO:tasks.workunit.client.1.smithi148.stdout:5/531: mknod d7/d13/cb1 0 2024-07-03T15:43:54.548 INFO:tasks.workunit.client.1.smithi148.stdout:3/508: sync 2024-07-03T15:43:54.549 INFO:tasks.workunit.client.1.smithi148.stdout:0/501: sync 2024-07-03T15:43:54.551 INFO:tasks.workunit.client.1.smithi148.stdout:1/438: rename d0/d2/d2e/c52 to d0/c91 0 2024-07-03T15:43:54.552 INFO:tasks.workunit.client.1.smithi148.stdout:1/439: write d0/d2/d11/d3a/f5b [287885,120583] 0 2024-07-03T15:43:54.552 INFO:tasks.workunit.client.1.smithi148.stdout:1/440: write d0/d2/d6/d14/f15 [7711045,129609] 0 2024-07-03T15:43:54.552 INFO:tasks.workunit.client.1.smithi148.stdout:1/441: creat d0/d2/d2e/d39/d41/d77/f92 x:0 0 0 2024-07-03T15:43:54.553 INFO:tasks.workunit.client.1.smithi148.stdout:5/532: mkdir d7/da/df/da0/db2 0 2024-07-03T15:43:54.554 INFO:tasks.workunit.client.1.smithi148.stdout:9/757: dwrite d0/dd/d11/f35 [0,4194304] 0 2024-07-03T15:43:54.557 INFO:tasks.workunit.client.1.smithi148.stdout:1/442: mknod d0/d2/d6/d14/c93 0 2024-07-03T15:43:54.557 INFO:tasks.workunit.client.1.smithi148.stdout:8/456: dread d4/da/d14/f4d [0,4194304] 0 2024-07-03T15:43:54.557 INFO:tasks.workunit.client.1.smithi148.stdout:8/457: fsync d4/da/d14/d17/f26 0 2024-07-03T15:43:54.558 INFO:tasks.workunit.client.1.smithi148.stdout:8/458: write d4/da/d51/f9d [1152817,44810] 0 2024-07-03T15:43:54.558 INFO:tasks.workunit.client.1.smithi148.stdout:8/459: write d4/da/d51/d18/f8c [992172,21032] 0 2024-07-03T15:43:54.558 INFO:tasks.workunit.client.1.smithi148.stdout:4/453: rename d0/d28/d7d/l37 to d0/d8/d13/d48/l94 0 2024-07-03T15:43:54.560 INFO:tasks.workunit.client.1.smithi148.stdout:9/758: mkdir d0/dd/d46/ded 0 2024-07-03T15:43:54.560 INFO:tasks.workunit.client.1.smithi148.stdout:9/759: chown d0/d63/f98 106199 1 2024-07-03T15:43:54.562 INFO:tasks.workunit.client.1.smithi148.stdout:3/509: write dd/d33/d37/d68/f1f [3620870,71038] 0 2024-07-03T15:43:54.562 INFO:tasks.workunit.client.1.smithi148.stdout:3/510: write dd/f72 [5201354,39588] 0 2024-07-03T15:43:54.562 INFO:tasks.workunit.client.1.smithi148.stdout:1/443: mkdir d0/d2/d6/d14/d29/d94 0 2024-07-03T15:43:54.563 INFO:tasks.workunit.client.1.smithi148.stdout:8/460: chown d4/da/d51/d18/c2c 34110 1 2024-07-03T15:43:54.566 INFO:tasks.workunit.client.0.smithi028.stdout:1/83: dread d4/de/f1f [0,4194304] 0 2024-07-03T15:43:54.566 INFO:tasks.workunit.client.0.smithi028.stdout:1/84: readlink d4/d18/l20 0 2024-07-03T15:43:54.567 INFO:tasks.workunit.client.1.smithi148.stdout:2/591: rename d5/d14/d86/db9/dbb/d13/f9b to d5/d14/d86/fbc 0 2024-07-03T15:43:54.567 INFO:tasks.workunit.client.1.smithi148.stdout:6/554: dwrite d3/d1d/d28/f73 [0,4194304] 0 2024-07-03T15:43:54.567 INFO:tasks.workunit.client.1.smithi148.stdout:6/555: read - d3/d1d/d28/d3e/f7b zero size 2024-07-03T15:43:54.567 INFO:tasks.workunit.client.1.smithi148.stdout:6/556: chown d3/da5 100 1 2024-07-03T15:43:54.567 INFO:tasks.workunit.client.1.smithi148.stdout:4/454: symlink d0/d8/d19/l95 0 2024-07-03T15:43:54.568 INFO:tasks.workunit.client.1.smithi148.stdout:2/592: dread d5/d7/f83 [0,4194304] 0 2024-07-03T15:43:54.569 INFO:tasks.workunit.client.0.smithi028.stdout:1/85: link d4/d18/l20 d4/l25 0 2024-07-03T15:43:54.573 INFO:tasks.workunit.client.1.smithi148.stdout:9/760: symlink d0/dd/d25/d59/lee 0 2024-07-03T15:43:54.575 INFO:tasks.workunit.client.1.smithi148.stdout:3/511: mkdir dd/d33/d37/d68/d2e/d40/d77/d9e 0 2024-07-03T15:43:54.576 INFO:tasks.workunit.client.1.smithi148.stdout:3/512: dread - dd/d35/f4e zero size 2024-07-03T15:43:54.581 INFO:tasks.workunit.client.1.smithi148.stdout:6/557: unlink d3/fa8 0 2024-07-03T15:43:54.581 INFO:tasks.workunit.client.1.smithi148.stdout:6/558: creat d3/d1d/d28/d31/fbc x:0 0 0 2024-07-03T15:43:54.582 INFO:tasks.workunit.client.1.smithi148.stdout:6/559: fsync d3/d1d/d28/f69 0 2024-07-03T15:43:54.583 INFO:tasks.workunit.client.1.smithi148.stdout:2/593: unlink d5/d14/d86/db9/dbb/d13/d2d/f2e 0 2024-07-03T15:43:54.583 INFO:tasks.workunit.client.1.smithi148.stdout:2/594: read - d5/d14/d22/d60/d76/fb0 zero size 2024-07-03T15:43:54.583 INFO:tasks.workunit.client.1.smithi148.stdout:2/595: write d5/d5f/d7e/f82 [471863,10230] 0 2024-07-03T15:43:54.584 INFO:tasks.workunit.client.1.smithi148.stdout:5/533: link d7/da/d21/f5e d7/da/d21/fb3 0 2024-07-03T15:43:54.585 INFO:tasks.workunit.client.1.smithi148.stdout:5/534: chown d7/da/df/f3d 224 1 2024-07-03T15:43:54.585 INFO:tasks.workunit.client.1.smithi148.stdout:5/535: chown d7/da/df/d16/d6a 3636662 1 2024-07-03T15:43:54.585 INFO:tasks.workunit.client.1.smithi148.stdout:1/444: dwrite d0/d2/d2e/d39/d41/f43 [4194304,4194304] 0 2024-07-03T15:43:54.586 INFO:tasks.workunit.client.1.smithi148.stdout:3/513: rename dd/d33/f59 to dd/d33/d37/d68/d2e/d40/d45/d73/f9f 0 2024-07-03T15:43:54.587 INFO:tasks.workunit.client.1.smithi148.stdout:8/461: symlink d4/da/d14/d1a/d32/d38/d58/la9 0 2024-07-03T15:43:54.589 INFO:tasks.workunit.client.1.smithi148.stdout:6/560: symlink d3/d1d/d28/d3e/lbd 0 2024-07-03T15:43:54.590 INFO:tasks.workunit.client.1.smithi148.stdout:1/445: rename d0/de/d44/d54/d5d/f57 to d0/de/d89/f95 0 2024-07-03T15:43:54.591 INFO:tasks.workunit.client.1.smithi148.stdout:6/561: mknod d3/d5b/d74/cbe 0 2024-07-03T15:43:54.591 INFO:tasks.workunit.client.1.smithi148.stdout:6/562: fdatasync d3/d50/f9a 0 2024-07-03T15:43:54.595 INFO:tasks.workunit.client.1.smithi148.stdout:8/462: dread d4/da/d47/d88/f27 [0,4194304] 0 2024-07-03T15:43:54.597 INFO:tasks.workunit.client.1.smithi148.stdout:4/455: dwrite d0/f3d [0,4194304] 0 2024-07-03T15:43:54.597 INFO:tasks.workunit.client.1.smithi148.stdout:4/456: chown d0/d8/d22/d2f/f33 500737207 1 2024-07-03T15:43:54.597 INFO:tasks.workunit.client.1.smithi148.stdout:0/502: dwrite da/d38/f6b [0,4194304] 0 2024-07-03T15:43:54.598 INFO:tasks.workunit.client.1.smithi148.stdout:0/503: stat da/d44/d13/d1d/d3e 0 2024-07-03T15:43:54.598 INFO:tasks.workunit.client.1.smithi148.stdout:0/504: stat da/d44/l22 0 2024-07-03T15:43:54.598 INFO:tasks.workunit.client.1.smithi148.stdout:8/463: rmdir d4/da/d14/d1a 39 2024-07-03T15:43:54.598 INFO:tasks.workunit.client.1.smithi148.stdout:9/761: dwrite d0/d49/d43/d19/d7a/daa/fb1 [0,4194304] 0 2024-07-03T15:43:54.598 INFO:tasks.workunit.client.1.smithi148.stdout:9/762: chown d0/d49/f81 277 1 2024-07-03T15:43:54.601 INFO:tasks.workunit.client.1.smithi148.stdout:6/563: write d3/d1d/d28/f2e [1072001,55908] 0 2024-07-03T15:43:54.601 INFO:tasks.workunit.client.1.smithi148.stdout:5/536: dread d7/f4f [0,4194304] 0 2024-07-03T15:43:54.601 INFO:tasks.workunit.client.1.smithi148.stdout:5/537: truncate d7/d13/d23/f78 1875979 0 2024-07-03T15:43:54.601 INFO:tasks.workunit.client.1.smithi148.stdout:5/538: chown d7/da/df/d16/d1a/f7a 18610 1 2024-07-03T15:43:54.602 INFO:tasks.workunit.client.1.smithi148.stdout:7/619: dwrite d5/d7d/da7/f4d [0,4194304] 0 2024-07-03T15:43:54.604 INFO:tasks.workunit.client.1.smithi148.stdout:0/505: symlink da/d44/d4a/la6 0 2024-07-03T15:43:54.604 INFO:tasks.workunit.client.1.smithi148.stdout:0/506: read - da/d44/d13/d14/f61 zero size 2024-07-03T15:43:54.606 INFO:tasks.workunit.client.1.smithi148.stdout:1/446: dread d0/d2/d2e/d39/d42/f48 [0,4194304] 0 2024-07-03T15:43:54.607 INFO:tasks.workunit.client.1.smithi148.stdout:1/447: read d0/d2/d2e/d39/d41/f43 [1328564,45470] 0 2024-07-03T15:43:54.610 INFO:tasks.workunit.client.1.smithi148.stdout:8/464: dread f2 [4194304,4194304] 0 2024-07-03T15:43:54.613 INFO:tasks.workunit.client.1.smithi148.stdout:2/596: dwrite d5/d14/d86/db9/dbb/d13/d1d/f43 [0,4194304] 0 2024-07-03T15:43:54.613 INFO:tasks.workunit.client.1.smithi148.stdout:2/597: write d5/d14/d86/db9/dbb/d13/d2d/f3b [3438980,53291] 0 2024-07-03T15:43:54.614 INFO:tasks.workunit.client.1.smithi148.stdout:0/507: mkdir da/d17/d2a/d2f/d93/da3/da7 0 2024-07-03T15:43:54.618 INFO:tasks.workunit.client.1.smithi148.stdout:9/763: rmdir d0/d2f 39 2024-07-03T15:43:54.618 INFO:tasks.workunit.client.1.smithi148.stdout:7/620: dread d5/f58 [0,4194304] 0 2024-07-03T15:43:54.619 INFO:tasks.workunit.client.1.smithi148.stdout:4/457: dread d0/d8/d22/d2f/f6c [0,4194304] 0 2024-07-03T15:43:54.620 INFO:tasks.workunit.client.1.smithi148.stdout:3/514: dwrite dd/f22 [0,4194304] 0 2024-07-03T15:43:54.621 INFO:tasks.workunit.client.1.smithi148.stdout:1/448: write d0/de/d44/f5e [4111586,75450] 0 2024-07-03T15:43:54.622 INFO:tasks.workunit.client.1.smithi148.stdout:7/621: link d5/d7d/da7/c17 d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/db2/dd6/cd8 0 2024-07-03T15:43:54.623 INFO:tasks.workunit.client.1.smithi148.stdout:5/539: rename d7/da/df/d83 to d7/da/df/d16/d6a/db4 0 2024-07-03T15:43:54.627 INFO:tasks.workunit.client.1.smithi148.stdout:3/515: link dd/d33/d37/f82 dd/d33/d37/d68/d67/fa0 0 2024-07-03T15:43:54.629 INFO:tasks.workunit.client.1.smithi148.stdout:1/449: unlink d0/de/d2f/d72/c76 0 2024-07-03T15:43:54.629 INFO:tasks.workunit.client.1.smithi148.stdout:7/622: symlink d5/d7d/da7/d1c/d33/d4c/d84/d91/daa/db9/ld9 0 2024-07-03T15:43:54.631 INFO:tasks.workunit.client.1.smithi148.stdout:2/598: dread d5/d14/d22/d60/f90 [0,4194304] 0 2024-07-03T15:43:54.631 INFO:tasks.workunit.client.1.smithi148.stdout:2/599: write d5/d14/f58 [2924145,116955] 0 2024-07-03T15:43:54.631 INFO:tasks.workunit.client.1.smithi148.stdout:3/516: mknod dd/d33/d37/d8d/ca1 0 2024-07-03T15:43:54.631 INFO:tasks.workunit.client.1.smithi148.stdout:3/517: stat dd/d33/f4b 0 2024-07-03T15:43:54.631 INFO:tasks.workunit.client.1.smithi148.stdout:3/518: stat dd/d33/d37/d68/d2e/d40 0 2024-07-03T15:43:54.631 INFO:tasks.workunit.client.1.smithi148.stdout:9/764: dread d0/d49/f81 [0,4194304] 0 2024-07-03T15:43:54.632 INFO:tasks.workunit.client.1.smithi148.stdout:1/450: link d0/c1 d0/d2/d2e/d39/d41/d77/d80/c96 0 2024-07-03T15:43:54.632 INFO:tasks.workunit.client.1.smithi148.stdout:2/600: mknod d5/d14/d86/db9/dbb/d13/d41/d9d/cbd 0 2024-07-03T15:43:54.632 INFO:tasks.workunit.client.1.smithi148.stdout:2/601: fsync d5/d7/f83 0 2024-07-03T15:43:54.635 INFO:tasks.workunit.client.1.smithi148.stdout:9/765: mkdir d0/dd/d25/d37/d40/d72/db8/def 0 2024-07-03T15:43:54.635 INFO:tasks.workunit.client.1.smithi148.stdout:9/766: chown d0/d49/d43/d19/de1/fe3 832 1 2024-07-03T15:43:54.635 INFO:tasks.workunit.client.1.smithi148.stdout:9/767: write d0/d18/d24/d5a/fe9 [326485,43648] 0 2024-07-03T15:43:54.635 INFO:tasks.workunit.client.1.smithi148.stdout:9/768: write d0/dd/d25/d37/d40/da3/fd3 [4356555,113779] 0 2024-07-03T15:43:54.637 INFO:tasks.workunit.client.1.smithi148.stdout:6/564: dwrite d3/d1d/d28/d7f/d85/fb6 [0,4194304] 0 2024-07-03T15:43:54.637 INFO:tasks.workunit.client.1.smithi148.stdout:5/540: dread d7/da/df/f71 [0,4194304] 0 2024-07-03T15:43:54.637 INFO:tasks.workunit.client.1.smithi148.stdout:5/541: creat d7/da/fb5 x:0 0 0 2024-07-03T15:43:54.637 INFO:tasks.workunit.client.1.smithi148.stdout:5/542: dread - d7/da/df/d16/d2f/d44/f9a zero size 2024-07-03T15:43:54.643 INFO:tasks.workunit.client.1.smithi148.stdout:6/565: unlink d3/c18 0 2024-07-03T15:43:54.643 INFO:tasks.workunit.client.1.smithi148.stdout:6/566: chown d3/d1d/d28/d7f/c8d 1637 1 2024-07-03T15:43:54.644 INFO:tasks.workunit.client.1.smithi148.stdout:1/451: dread d0/d2/d2e/d39/f5f [0,4194304] 0 2024-07-03T15:43:54.644 INFO:tasks.workunit.client.1.smithi148.stdout:0/508: dwrite da/d17/f1e [0,4194304] 0 2024-07-03T15:43:54.644 INFO:tasks.workunit.client.1.smithi148.stdout:5/543: mknod d7/d59/cb6 0 2024-07-03T15:43:54.646 INFO:tasks.workunit.client.1.smithi148.stdout:4/458: dread d0/d8/d22/d2f/d41/f7a [0,4194304] 0 2024-07-03T15:43:54.648 INFO:tasks.workunit.client.1.smithi148.stdout:7/623: write d5/d7d/da7/f6a [391264,70834] 0 2024-07-03T15:43:54.648 INFO:tasks.workunit.client.1.smithi148.stdout:7/624: write d5/d7d/da7/d1c/d33/d4c/d21/d47/fd5 [873790,37589] 0 2024-07-03T15:43:54.650 INFO:tasks.workunit.client.0.smithi028.stdout:1/86: dwrite d4/f1a [0,4194304] 0 2024-07-03T15:43:54.650 INFO:tasks.workunit.client.0.smithi028.stdout:1/87: chown d4/de/d1e 38175820 1 2024-07-03T15:43:54.651 INFO:tasks.workunit.client.1.smithi148.stdout:2/602: dread d5/d14/d22/d60/d76/f92 [0,4194304] 0 2024-07-03T15:43:54.656 INFO:tasks.workunit.client.1.smithi148.stdout:6/567: mknod d3/d1d/d28/d7f/d85/da2/cbf 0 2024-07-03T15:43:54.657 INFO:tasks.workunit.client.1.smithi148.stdout:1/452: symlink d0/d2/d2e/d39/d42/l97 0 2024-07-03T15:43:54.657 INFO:tasks.workunit.client.1.smithi148.stdout:1/453: creat d0/d2/d6/d14/f98 x:0 0 0 2024-07-03T15:43:54.661 INFO:tasks.workunit.client.1.smithi148.stdout:4/459: symlink d0/d28/d7d/l96 0 2024-07-03T15:43:54.663 INFO:tasks.workunit.client.1.smithi148.stdout:1/454: read d0/de/d2f/d72/f7c [1075867,54226] 0 2024-07-03T15:43:54.664 INFO:tasks.workunit.client.1.smithi148.stdout:5/544: creat d7/da/d79/fb7 x:0 0 0 2024-07-03T15:43:54.664 INFO:tasks.workunit.client.1.smithi148.stdout:7/625: rename d5/d7d/da7/d1c/d33/d4c/d84/da4/cd4 to d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/db1/cda 0 2024-07-03T15:43:54.672 INFO:tasks.workunit.client.1.smithi148.stdout:4/460: unlink d0/d8/d13/d48/l94 0 2024-07-03T15:43:54.673 INFO:tasks.workunit.client.1.smithi148.stdout:5/545: mknod d7/da/d21/cb8 0 2024-07-03T15:43:54.673 INFO:tasks.workunit.client.1.smithi148.stdout:5/546: write d7/da/df/d16/d2f/d44/d52/f89 [765448,79991] 0 2024-07-03T15:43:54.673 INFO:tasks.workunit.client.1.smithi148.stdout:5/547: chown d7/d59 15209409 1 2024-07-03T15:43:54.673 INFO:tasks.workunit.client.1.smithi148.stdout:7/626: rmdir d5/d7d/da7/d1c/d23 39 2024-07-03T15:43:54.674 INFO:tasks.workunit.client.1.smithi148.stdout:7/627: readlink d5/d7d/da7/d1c/d33/l51 0 2024-07-03T15:43:54.675 INFO:tasks.workunit.client.1.smithi148.stdout:4/461: creat d0/d28/d6e/f97 x:0 0 0 2024-07-03T15:43:54.677 INFO:tasks.workunit.client.1.smithi148.stdout:5/548: link d7/da/df/d16/d1a/f34 d7/da/df/d16/d2f/d44/d52/fb9 0 2024-07-03T15:43:54.678 INFO:tasks.workunit.client.1.smithi148.stdout:7/628: symlink d5/d7d/da7/d1c/d33/d4c/ldb 0 2024-07-03T15:43:54.680 INFO:tasks.workunit.client.1.smithi148.stdout:7/629: symlink d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/ldc 0 2024-07-03T15:43:54.682 INFO:tasks.workunit.client.1.smithi148.stdout:7/630: rename d5/d7d/da7/d1c/d4a/dcf to d5/d7d/da7/db4/ddd 0 2024-07-03T15:43:54.682 INFO:tasks.workunit.client.1.smithi148.stdout:7/631: write d5/d7d/da7/d1c/d23/fac [4795721,20824] 0 2024-07-03T15:43:54.682 INFO:tasks.workunit.client.1.smithi148.stdout:7/632: chown d5/d7d/da7/d1c/d4a/d52/cd1 157104346 1 2024-07-03T15:43:54.684 INFO:tasks.workunit.client.1.smithi148.stdout:7/633: creat d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/db2/fde x:0 0 0 2024-07-03T15:43:54.686 INFO:tasks.workunit.client.1.smithi148.stdout:7/634: rmdir d5/d7d/da7/d1c/d33/d4c/d21/d47 39 2024-07-03T15:43:54.687 INFO:tasks.workunit.client.1.smithi148.stdout:7/635: creat d5/d7d/da7/d1c/d33/d4c/d84/d98/d9e/fdf x:0 0 0 2024-07-03T15:43:54.687 INFO:tasks.workunit.client.1.smithi148.stdout:7/636: write d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/db2/fb3 [1113879,34630] 0 2024-07-03T15:43:54.690 INFO:tasks.workunit.client.1.smithi148.stdout:4/462: write d0/f5 [6368178,45880] 0 2024-07-03T15:43:54.690 INFO:tasks.workunit.client.1.smithi148.stdout:4/463: chown d0/d8/c18 186 1 2024-07-03T15:43:54.691 INFO:tasks.workunit.client.1.smithi148.stdout:4/464: symlink d0/l98 0 2024-07-03T15:43:54.691 INFO:tasks.workunit.client.1.smithi148.stdout:5/549: dread d7/da/f4c [0,4194304] 0 2024-07-03T15:43:54.691 INFO:tasks.workunit.client.1.smithi148.stdout:5/550: getdents d7/da/df/da0/db2 0 2024-07-03T15:43:54.692 INFO:tasks.workunit.client.1.smithi148.stdout:4/465: rename d0/d8/c88 to d0/d28/d7d/d6f/c99 0 2024-07-03T15:43:54.693 INFO:tasks.workunit.client.1.smithi148.stdout:5/551: link d7/da/l11 d7/da/lba 0 2024-07-03T15:43:54.693 INFO:tasks.workunit.client.1.smithi148.stdout:5/552: dread - d7/da/df/d16/d6a/db4/f9c zero size 2024-07-03T15:43:54.695 INFO:tasks.workunit.client.1.smithi148.stdout:4/466: getdents d0/d8/d22/d62 0 2024-07-03T15:43:54.695 INFO:tasks.workunit.client.1.smithi148.stdout:5/553: write d7/d13/d23/f78 [515038,10056] 0 2024-07-03T15:43:54.703 INFO:tasks.workunit.client.1.smithi148.stdout:0/509: dwrite da/d44/d13/d14/f6a [0,4194304] 0 2024-07-03T15:43:54.706 INFO:tasks.workunit.client.1.smithi148.stdout:0/510: truncate da/d44/d13/d14/f99 2400166 0 2024-07-03T15:43:54.707 INFO:tasks.workunit.client.1.smithi148.stdout:9/769: dwrite d0/dd/d11/f66 [0,4194304] 0 2024-07-03T15:43:54.708 INFO:tasks.workunit.client.1.smithi148.stdout:9/770: truncate d0/dd/d11/fdd 628625 0 2024-07-03T15:43:54.708 INFO:tasks.workunit.client.1.smithi148.stdout:9/771: fsync d0/dd/d25/d37/f50 0 2024-07-03T15:43:54.708 INFO:tasks.workunit.client.1.smithi148.stdout:0/511: getdents da/d44/d13/d1d/d3e/d85 0 2024-07-03T15:43:54.709 INFO:tasks.workunit.client.1.smithi148.stdout:4/467: dread d0/d8/d13/d48/f3f [0,4194304] 0 2024-07-03T15:43:54.710 INFO:tasks.workunit.client.1.smithi148.stdout:9/772: mkdir d0/dd/d25/d37/d40/da3/df0 0 2024-07-03T15:43:54.710 INFO:tasks.workunit.client.1.smithi148.stdout:9/773: write d0/dd/d25/d59/fd2 [804250,88494] 0 2024-07-03T15:43:54.710 INFO:tasks.workunit.client.1.smithi148.stdout:9/774: fsync d0/d2f/f74 0 2024-07-03T15:43:54.711 INFO:tasks.workunit.client.1.smithi148.stdout:0/512: creat da/d44/d13/d14/d35/d8a/fa8 x:0 0 0 2024-07-03T15:43:54.713 INFO:tasks.workunit.client.1.smithi148.stdout:4/468: unlink d0/d8/d19/d45/d49/f56 0 2024-07-03T15:43:54.714 INFO:tasks.workunit.client.1.smithi148.stdout:4/469: truncate d0/d8/d22/d2f/d41/f7e 562094 0 2024-07-03T15:43:54.714 INFO:tasks.workunit.client.1.smithi148.stdout:4/470: creat d0/d8/d13/d48/f9a x:0 0 0 2024-07-03T15:43:54.717 INFO:tasks.workunit.client.1.smithi148.stdout:1/455: dwrite d0/d2/d6/f26 [8388608,4194304] 0 2024-07-03T15:43:54.718 INFO:tasks.workunit.client.0.smithi028.stdout:1/88: dwrite d4/fd [4194304,4194304] 0 2024-07-03T15:43:54.718 INFO:tasks.workunit.client.1.smithi148.stdout:5/554: dwrite d7/da/df/d16/d1a/f50 [0,4194304] 0 2024-07-03T15:43:54.718 INFO:tasks.workunit.client.1.smithi148.stdout:5/555: fsync d7/da/df/d16/d6a/db4/f9c 0 2024-07-03T15:43:54.719 INFO:tasks.workunit.client.1.smithi148.stdout:5/556: write d7/da/f5f [932338,31180] 0 2024-07-03T15:43:54.719 INFO:tasks.workunit.client.1.smithi148.stdout:7/637: dwrite d5/d7d/da7/f37 [0,4194304] 0 2024-07-03T15:43:54.719 INFO:tasks.workunit.client.1.smithi148.stdout:6/568: dwrite d3/d1d/d28/d31/d61/f71 [0,4194304] 0 2024-07-03T15:43:54.719 INFO:tasks.workunit.client.1.smithi148.stdout:2/603: dwrite d5/d14/d86/db9/dbb/d13/f15 [0,4194304] 0 2024-07-03T15:43:54.730 INFO:tasks.workunit.client.1.smithi148.stdout:5/557: mknod d7/da/df/d16/d1a/d58/da7/cbb 0 2024-07-03T15:43:54.730 INFO:tasks.workunit.client.1.smithi148.stdout:5/558: readlink d7/d13/la9 0 2024-07-03T15:43:54.735 INFO:tasks.workunit.client.1.smithi148.stdout:9/775: dwrite d0/f6 [0,4194304] 0 2024-07-03T15:43:54.735 INFO:tasks.workunit.client.1.smithi148.stdout:9/776: write d0/d2f/f74 [868534,22503] 0 2024-07-03T15:43:54.735 INFO:tasks.workunit.client.1.smithi148.stdout:9/777: chown d0/dd/d25/d59 1443037132 1 2024-07-03T15:43:54.740 INFO:tasks.workunit.client.1.smithi148.stdout:4/471: getdents d0/d28/d6e 0 2024-07-03T15:43:54.748 INFO:tasks.workunit.client.1.smithi148.stdout:1/456: stat d0/de/d2f/d72/c82 0 2024-07-03T15:43:54.748 INFO:tasks.workunit.client.1.smithi148.stdout:1/457: chown d0/c1c 0 1 2024-07-03T15:43:54.749 INFO:tasks.workunit.client.1.smithi148.stdout:1/458: chown d0/d2/d11/d1e/f3c 20263 1 2024-07-03T15:43:54.749 INFO:tasks.workunit.client.1.smithi148.stdout:1/459: stat d0/de/d2f/d72/f86 0 2024-07-03T15:43:54.749 INFO:tasks.workunit.client.1.smithi148.stdout:5/559: rename d7/d13/d54/l18 to d7/da/df/d16/d6a/db4/lbc 0 2024-07-03T15:43:54.751 INFO:tasks.workunit.client.1.smithi148.stdout:7/638: truncate d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/f83 1766474 0 2024-07-03T15:43:54.751 INFO:tasks.workunit.client.1.smithi148.stdout:4/472: write d0/d8/d22/d62/f89 [3268424,123311] 0 2024-07-03T15:43:54.751 INFO:tasks.workunit.client.1.smithi148.stdout:0/513: dwrite da/d17/d2a/f59 [0,4194304] 0 2024-07-03T15:43:54.751 INFO:tasks.workunit.client.1.smithi148.stdout:0/514: creat da/d17/d2a/fa9 x:0 0 0 2024-07-03T15:43:54.751 INFO:tasks.workunit.client.1.smithi148.stdout:0/515: read - da/d17/d2a/fa9 zero size 2024-07-03T15:43:54.751 INFO:tasks.workunit.client.0.smithi028.stdout:1/89: dwrite d4/d11/f12 [0,4194304] 0 2024-07-03T15:43:54.751 INFO:tasks.workunit.client.0.smithi028.stdout:1/90: chown d4/d11 5 1 2024-07-03T15:43:54.752 INFO:tasks.workunit.client.1.smithi148.stdout:2/604: creat d5/d5f/d79/d9c/fbe x:0 0 0 2024-07-03T15:43:54.755 INFO:tasks.workunit.client.1.smithi148.stdout:9/778: rename d0/dd/d25/d59/l68 to d0/dd/d25/d37/d40/da3/df0/lf1 0 2024-07-03T15:43:54.758 INFO:tasks.workunit.client.1.smithi148.stdout:4/473: read d0/d8/d22/d2f/f6c [1209088,46123] 0 2024-07-03T15:43:54.758 INFO:tasks.workunit.client.1.smithi148.stdout:4/474: fsync d0/d8/f2d 0 2024-07-03T15:43:54.758 INFO:tasks.workunit.client.1.smithi148.stdout:5/560: creat d7/da/df/d16/d2f/d44/fbd x:0 0 0 2024-07-03T15:43:54.758 INFO:tasks.workunit.client.1.smithi148.stdout:5/561: read - d7/da/df/d16/d1a/d58/fad zero size 2024-07-03T15:43:54.758 INFO:tasks.workunit.client.1.smithi148.stdout:5/562: chown d7/da/df/d16/d2f/d44/f84 916182677 1 2024-07-03T15:43:54.760 INFO:tasks.workunit.client.1.smithi148.stdout:7/639: link d5/d7d/da7/d1c/d33/l51 d5/d7d/da7/d1c/d33/d4c/d84/d98/d9e/le0 0 2024-07-03T15:43:54.764 INFO:tasks.workunit.client.1.smithi148.stdout:0/516: mkdir da/d58/daa 0 2024-07-03T15:43:54.767 INFO:tasks.workunit.client.0.smithi028.stdout:1/91: dread d4/f19 [0,4194304] 0 2024-07-03T15:43:54.767 INFO:tasks.workunit.client.0.smithi028.stdout:1/92: chown d4/de 88037 1 2024-07-03T15:43:54.771 INFO:tasks.workunit.client.1.smithi148.stdout:0/517: dread da/d44/d13/d1d/d3e/d85/d8e/fa2 [0,4194304] 0 2024-07-03T15:43:54.771 INFO:tasks.workunit.client.1.smithi148.stdout:1/460: creat d0/de/d89/f99 x:0 0 0 2024-07-03T15:43:54.773 INFO:tasks.workunit.client.1.smithi148.stdout:9/779: creat d0/d49/d43/ff2 x:0 0 0 2024-07-03T15:43:54.773 INFO:tasks.workunit.client.1.smithi148.stdout:9/780: stat d0/dd/d11/lbc 0 2024-07-03T15:43:54.776 INFO:tasks.workunit.client.1.smithi148.stdout:5/563: dwrite d7/d13/d54/f7d [0,4194304] 0 2024-07-03T15:43:54.778 INFO:tasks.workunit.client.1.smithi148.stdout:7/640: rmdir d5/d7d/da7/d1c/d4a/d66 39 2024-07-03T15:43:54.779 INFO:tasks.workunit.client.1.smithi148.stdout:2/605: symlink d5/d7/d64/db2/lbf 0 2024-07-03T15:43:54.784 INFO:tasks.workunit.client.1.smithi148.stdout:5/564: mkdir d7/da/dbe 0 2024-07-03T15:43:54.790 INFO:tasks.workunit.client.1.smithi148.stdout:4/475: rmdir d0/d8 39 2024-07-03T15:43:54.790 INFO:tasks.workunit.client.1.smithi148.stdout:8/465: sync 2024-07-03T15:43:54.790 INFO:tasks.workunit.client.1.smithi148.stdout:3/519: sync 2024-07-03T15:43:54.791 INFO:tasks.workunit.client.1.smithi148.stdout:7/641: unlink d5/d7d/da7/d1c/d23/f68 0 2024-07-03T15:43:54.791 INFO:tasks.workunit.client.1.smithi148.stdout:2/606: mkdir d5/d14/d22/dc0 0 2024-07-03T15:43:54.793 INFO:tasks.workunit.client.0.smithi028.stdout:1/93: dwrite d4/fd [4194304,4194304] 0 2024-07-03T15:43:54.793 INFO:tasks.workunit.client.0.smithi028.stdout:1/94: chown d4 2 1 2024-07-03T15:43:54.793 INFO:tasks.workunit.client.1.smithi148.stdout:2/607: dread d5/d14/d86/db9/dbb/d13/f29 [0,4194304] 0 2024-07-03T15:43:54.796 INFO:tasks.workunit.client.1.smithi148.stdout:7/642: mknod d5/d7d/da7/d1c/d33/d4c/ce1 0 2024-07-03T15:43:54.796 INFO:tasks.workunit.client.1.smithi148.stdout:1/461: read d0/de/d44/d54/d5d/f50 [442742,52096] 0 2024-07-03T15:43:54.796 INFO:tasks.workunit.client.1.smithi148.stdout:1/462: read d0/d2/f10 [2081216,95403] 0 2024-07-03T15:43:54.796 INFO:tasks.workunit.client.1.smithi148.stdout:1/463: creat d0/d2/d2e/d39/d42/d62/f9a x:0 0 0 2024-07-03T15:43:54.797 INFO:tasks.workunit.client.1.smithi148.stdout:0/518: rename da/d36/d5b to da/d44/d13/d14/d35/d8a/dab 0 2024-07-03T15:43:54.797 INFO:tasks.workunit.client.1.smithi148.stdout:3/520: dread dd/d33/d37/d68/f1f [0,4194304] 0 2024-07-03T15:43:54.798 INFO:tasks.workunit.client.1.smithi148.stdout:4/476: link d0/d8/d19/l4e d0/d8/d13/d8b/l9b 0 2024-07-03T15:43:54.798 INFO:tasks.workunit.client.1.smithi148.stdout:2/608: rmdir d5/d14/d86/db9/dbb/d13/d41/d7b 39 2024-07-03T15:43:54.798 INFO:tasks.workunit.client.0.smithi028.stdout:1/95: dread d4/f19 [0,4194304] 0 2024-07-03T15:43:54.800 INFO:tasks.workunit.client.1.smithi148.stdout:8/466: rmdir d4/da/d14 39 2024-07-03T15:43:54.801 INFO:tasks.workunit.client.0.smithi028.stdout:1/96: rmdir d4/de/d1e 39 2024-07-03T15:43:54.801 INFO:tasks.workunit.client.0.smithi028.stdout:1/97: fdatasync d4/f14 0 2024-07-03T15:43:54.803 INFO:tasks.workunit.client.0.smithi028.stdout:1/98: dread d4/de/f10 [0,4194304] 0 2024-07-03T15:43:54.803 INFO:tasks.workunit.client.1.smithi148.stdout:7/643: write d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/f28 [611518,1031] 0 2024-07-03T15:43:54.803 INFO:tasks.workunit.client.1.smithi148.stdout:1/464: symlink d0/de/d2f/d72/l9b 0 2024-07-03T15:43:54.805 INFO:tasks.workunit.client.1.smithi148.stdout:9/781: rename d0/dd/d25/d6a to d0/d18/df3 0 2024-07-03T15:43:54.807 INFO:tasks.workunit.client.1.smithi148.stdout:4/477: dread d0/f50 [0,4194304] 0 2024-07-03T15:43:54.807 INFO:tasks.workunit.client.0.smithi028.stdout:1/99: rmdir d4/de/d1e/d22 0 2024-07-03T15:43:54.808 INFO:tasks.workunit.client.0.smithi028.stdout:1/100: creat d4/f26 x:0 0 0 2024-07-03T15:43:54.808 INFO:tasks.workunit.client.1.smithi148.stdout:0/519: truncate da/d44/d13/d14/f68 159330 0 2024-07-03T15:43:54.811 INFO:tasks.workunit.client.1.smithi148.stdout:3/521: rmdir dd/d33/d37/d68/d2e/d40/d77/d9e 0 2024-07-03T15:43:54.812 INFO:tasks.workunit.client.1.smithi148.stdout:2/609: write d5/d14/d86/db9/dbb/d68/fac [2576915,99453] 0 2024-07-03T15:43:54.812 INFO:tasks.workunit.client.1.smithi148.stdout:2/610: chown d5/d14/d86/db9/dbb/d13/l3c 1296378 1 2024-07-03T15:43:54.812 INFO:tasks.workunit.client.1.smithi148.stdout:8/467: fsync d4/da/d51/f57 0 2024-07-03T15:43:54.812 INFO:tasks.workunit.client.1.smithi148.stdout:8/468: dread - d4/da/d14/d1a/d32/d89/fa1 zero size 2024-07-03T15:43:54.814 INFO:tasks.workunit.client.0.smithi028.stdout:1/101: rename d4/fc to d4/d11/d16/f27 0 2024-07-03T15:43:54.817 INFO:tasks.workunit.client.1.smithi148.stdout:1/465: truncate d0/d2/d11/f47 2211574 0 2024-07-03T15:43:54.817 INFO:tasks.workunit.client.1.smithi148.stdout:1/466: stat d0/d2/d2e/c67 0 2024-07-03T15:43:54.817 INFO:tasks.workunit.client.0.smithi028.stdout:1/102: symlink d4/d11/l28 0 2024-07-03T15:43:54.818 INFO:tasks.workunit.client.0.smithi028.stdout:1/103: creat d4/f29 x:0 0 0 2024-07-03T15:43:54.818 INFO:tasks.workunit.client.1.smithi148.stdout:4/478: mknod d0/d8/d19/d45/c9c 0 2024-07-03T15:43:54.820 INFO:tasks.workunit.client.1.smithi148.stdout:0/520: stat da/d17/d2a/l7b 0 2024-07-03T15:43:54.820 INFO:tasks.workunit.client.1.smithi148.stdout:0/521: readlink da/d38/l3d 0 2024-07-03T15:43:54.820 INFO:tasks.workunit.client.1.smithi148.stdout:5/565: rename d7/da/df/d16/d2f/d44/d52/f6c to d7/d13/d54/fbf 0 2024-07-03T15:43:54.822 INFO:tasks.workunit.client.0.smithi028.stdout:1/104: creat d4/d18/f2a x:0 0 0 2024-07-03T15:43:54.823 INFO:tasks.workunit.client.1.smithi148.stdout:2/611: mknod d5/d14/d8b/cc1 0 2024-07-03T15:43:54.824 INFO:tasks.workunit.client.1.smithi148.stdout:2/612: write d5/d14/d86/db9/dbb/d13/d2d/f51 [1758575,99402] 0 2024-07-03T15:43:54.824 INFO:tasks.workunit.client.1.smithi148.stdout:8/469: creat d4/da/d47/d88/faa x:0 0 0 2024-07-03T15:43:54.824 INFO:tasks.workunit.client.1.smithi148.stdout:8/470: fsync d4/f98 0 2024-07-03T15:43:54.825 INFO:tasks.workunit.client.1.smithi148.stdout:4/479: mkdir d0/d8/d3a/d9d 0 2024-07-03T15:43:54.828 INFO:tasks.workunit.client.0.smithi028.stdout:1/105: rename d4/l25 to d4/d11/l2b 0 2024-07-03T15:43:54.828 INFO:tasks.workunit.client.0.smithi028.stdout:1/106: write d4/d11/f1b [137547,57630] 0 2024-07-03T15:43:54.829 INFO:tasks.workunit.client.1.smithi148.stdout:5/566: symlink d7/da/df/d16/d6a/db4/lc0 0 2024-07-03T15:43:54.829 INFO:tasks.workunit.client.1.smithi148.stdout:9/782: dwrite d0/d18/f26 [0,4194304] 0 2024-07-03T15:43:54.830 INFO:tasks.workunit.client.1.smithi148.stdout:9/783: write d0/d18/d24/d5a/fe9 [3090829,37662] 0 2024-07-03T15:43:54.831 INFO:tasks.workunit.client.1.smithi148.stdout:2/613: creat d5/d14/d86/db9/dbb/d68/fc2 x:0 0 0 2024-07-03T15:43:54.832 INFO:tasks.workunit.client.1.smithi148.stdout:8/471: mkdir d4/da/d14/d3a/dab 0 2024-07-03T15:43:54.833 INFO:tasks.workunit.client.1.smithi148.stdout:4/480: rmdir d0/d8/d22/d2f/d52 39 2024-07-03T15:43:54.834 INFO:tasks.workunit.client.1.smithi148.stdout:0/522: dread da/d44/d13/d14/d50/f83 [0,4194304] 0 2024-07-03T15:43:54.836 INFO:tasks.workunit.client.1.smithi148.stdout:4/481: creat d0/d8/d22/d2f/f9e x:0 0 0 2024-07-03T15:43:54.836 INFO:tasks.workunit.client.1.smithi148.stdout:5/567: mkdir d7/da/df/d16/dac/dc1 0 2024-07-03T15:43:54.836 INFO:tasks.workunit.client.1.smithi148.stdout:5/568: write d7/da/df/d16/d1a/f34 [4670430,107802] 0 2024-07-03T15:43:54.836 INFO:tasks.workunit.client.1.smithi148.stdout:5/569: write d7/da/df/d16/f1b [887153,7018] 0 2024-07-03T15:43:54.838 INFO:tasks.workunit.client.1.smithi148.stdout:0/523: mkdir da/d44/d94/dac 0 2024-07-03T15:43:54.841 INFO:tasks.workunit.client.1.smithi148.stdout:4/482: mknod d0/c9f 0 2024-07-03T15:43:54.843 INFO:tasks.workunit.client.1.smithi148.stdout:0/524: mknod da/d58/daa/cad 0 2024-07-03T15:43:54.843 INFO:tasks.workunit.client.1.smithi148.stdout:0/525: chown da/d44/d13/d14/f61 0 1 2024-07-03T15:43:54.843 INFO:tasks.workunit.client.1.smithi148.stdout:0/526: write da/d44/d13/d14/d35/f5e [2449872,29712] 0 2024-07-03T15:43:54.843 INFO:tasks.workunit.client.1.smithi148.stdout:0/527: truncate da/d44/d13/d14/d50/f60 4752567 0 2024-07-03T15:43:54.844 INFO:tasks.workunit.client.1.smithi148.stdout:6/569: sync 2024-07-03T15:43:54.844 INFO:tasks.workunit.client.1.smithi148.stdout:6/570: dread - d3/d1d/d28/fa6 zero size 2024-07-03T15:43:54.844 INFO:tasks.workunit.client.1.smithi148.stdout:6/571: creat d3/fc0 x:0 0 0 2024-07-03T15:43:54.844 INFO:tasks.workunit.client.1.smithi148.stdout:6/572: fdatasync d3/d1d/d28/d31/d61/f7a 0 2024-07-03T15:43:54.844 INFO:tasks.workunit.client.1.smithi148.stdout:2/614: dread d5/d14/d86/db9/dbb/d13/d41/d7b/f3e [0,4194304] 0 2024-07-03T15:43:54.848 INFO:tasks.workunit.client.1.smithi148.stdout:0/528: rename da/d36/c66 to da/d44/d13/d14/d35/d8a/cae 0 2024-07-03T15:43:54.848 INFO:tasks.workunit.client.1.smithi148.stdout:0/529: write da/d44/d13/d14/d50/d95/d8c/f4d [427280,129316] 0 2024-07-03T15:43:54.848 INFO:tasks.workunit.client.1.smithi148.stdout:0/530: chown da/d44/d13/d14/c7e 0 1 2024-07-03T15:43:54.851 INFO:tasks.workunit.client.1.smithi148.stdout:6/573: link d3/d1d/d28/lad d3/d1d/d28/d53/lc1 0 2024-07-03T15:43:54.851 INFO:tasks.workunit.client.1.smithi148.stdout:6/574: write d3/d1d/d28/f51 [1618721,57948] 0 2024-07-03T15:43:54.853 INFO:tasks.workunit.client.1.smithi148.stdout:0/531: symlink da/d44/d13/d1d/d3e/d85/laf 0 2024-07-03T15:43:54.855 INFO:tasks.workunit.client.1.smithi148.stdout:6/575: unlink d3/l39 0 2024-07-03T15:43:54.857 INFO:tasks.workunit.client.1.smithi148.stdout:3/522: dwrite dd/d35/d49/f85 [0,4194304] 0 2024-07-03T15:43:54.858 INFO:tasks.workunit.client.1.smithi148.stdout:0/532: rename da/d44/d24/l77 to da/d44/d24/lb0 0 2024-07-03T15:43:54.858 INFO:tasks.workunit.client.1.smithi148.stdout:0/533: stat da/d44/d13/l56 0 2024-07-03T15:43:54.861 INFO:tasks.workunit.client.1.smithi148.stdout:2/615: mkdir d5/d67/dc3 0 2024-07-03T15:43:54.862 INFO:tasks.workunit.client.1.smithi148.stdout:3/523: mkdir dd/d35/d49/da2 0 2024-07-03T15:43:54.864 INFO:tasks.workunit.client.1.smithi148.stdout:3/524: unlink dd/d33/d52/f58 0 2024-07-03T15:43:54.864 INFO:tasks.workunit.client.1.smithi148.stdout:3/525: write dd/d33/d37/d68/d2e/d40/d45/d65/f7b [555731,45397] 0 2024-07-03T15:43:54.864 INFO:tasks.workunit.client.1.smithi148.stdout:3/526: truncate dd/d33/f3a 607117 0 2024-07-03T15:43:54.864 INFO:tasks.workunit.client.1.smithi148.stdout:3/527: chown dd/f19 14307 1 2024-07-03T15:43:54.865 INFO:tasks.workunit.client.1.smithi148.stdout:3/528: creat dd/d33/d37/d68/d2e/d40/d45/fa3 x:0 0 0 2024-07-03T15:43:54.868 INFO:tasks.workunit.client.1.smithi148.stdout:6/576: write d3/d1d/d28/d31/d61/f7a [2597996,57722] 0 2024-07-03T15:43:54.868 INFO:tasks.workunit.client.1.smithi148.stdout:6/577: readlink d3/d1d/d28/d3e/l41 0 2024-07-03T15:43:54.874 INFO:tasks.workunit.client.1.smithi148.stdout:7/644: dwrite d5/d7d/da7/f4d [0,4194304] 0 2024-07-03T15:43:54.883 INFO:tasks.workunit.client.1.smithi148.stdout:1/467: dwrite d0/d2/d2e/d39/d41/f69 [0,4194304] 0 2024-07-03T15:43:54.883 INFO:tasks.workunit.client.1.smithi148.stdout:1/468: creat d0/d2/d2e/d39/d42/d62/f9c x:0 0 0 2024-07-03T15:43:54.883 INFO:tasks.workunit.client.1.smithi148.stdout:9/784: dwrite d0/dd/f14 [8388608,4194304] 0 2024-07-03T15:43:54.883 INFO:tasks.workunit.client.1.smithi148.stdout:9/785: truncate d0/fc5 935714 0 2024-07-03T15:43:54.885 INFO:tasks.workunit.client.1.smithi148.stdout:1/469: truncate d0/d2/d2e/d39/d41/f7a 1225865 0 2024-07-03T15:43:54.888 INFO:tasks.workunit.client.1.smithi148.stdout:1/470: mkdir d0/d2/d11/d1e/d9d 0 2024-07-03T15:43:54.889 INFO:tasks.workunit.client.1.smithi148.stdout:1/471: creat d0/d2/d6/f9e x:0 0 0 2024-07-03T15:43:54.889 INFO:tasks.workunit.client.1.smithi148.stdout:1/472: dread - d0/d2/d2e/d39/d41/d77/f92 zero size 2024-07-03T15:43:54.890 INFO:tasks.workunit.client.1.smithi148.stdout:0/534: dread da/d44/d13/d14/f68 [0,4194304] 0 2024-07-03T15:43:54.890 INFO:tasks.workunit.client.1.smithi148.stdout:0/535: write da/d44/d24/f89 [2953653,18688] 0 2024-07-03T15:43:54.890 INFO:tasks.workunit.client.1.smithi148.stdout:0/536: readlink da/d44/d13/d5a/l86 0 2024-07-03T15:43:54.892 INFO:tasks.workunit.client.1.smithi148.stdout:0/537: unlink da/d44/d13/d1d/d3e/d85/d8e/fa2 0 2024-07-03T15:43:54.892 INFO:tasks.workunit.client.1.smithi148.stdout:0/538: chown da/d17/d2a/f2d 105232492 1 2024-07-03T15:43:54.893 INFO:tasks.workunit.client.1.smithi148.stdout:6/578: read d3/f1c [5115607,39581] 0 2024-07-03T15:43:54.893 INFO:tasks.workunit.client.1.smithi148.stdout:5/570: dwrite d7/da/f4c [0,4194304] 0 2024-07-03T15:43:54.893 INFO:tasks.workunit.client.1.smithi148.stdout:5/571: chown d7/d59 830928 1 2024-07-03T15:43:54.894 INFO:tasks.workunit.client.1.smithi148.stdout:0/539: mknod da/d17/d2a/d2f/d93/da3/da7/cb1 0 2024-07-03T15:43:54.895 INFO:tasks.workunit.client.1.smithi148.stdout:0/540: truncate da/d44/f96 92665 0 2024-07-03T15:43:54.895 INFO:tasks.workunit.client.1.smithi148.stdout:0/541: chown da/c9f 523067 1 2024-07-03T15:43:54.898 INFO:tasks.workunit.client.1.smithi148.stdout:0/542: mkdir da/d44/d13/d1d/d3e/d85/d8e/db2 0 2024-07-03T15:43:54.909 INFO:tasks.workunit.client.1.smithi148.stdout:6/579: dread d3/d1d/f24 [0,4194304] 0 2024-07-03T15:43:54.911 INFO:tasks.workunit.client.1.smithi148.stdout:0/543: dread da/d17/f1e [0,4194304] 0 2024-07-03T15:43:54.915 INFO:tasks.workunit.client.1.smithi148.stdout:4/483: dwrite d0/d8/d19/d45/f75 [0,4194304] 0 2024-07-03T15:43:54.915 INFO:tasks.workunit.client.1.smithi148.stdout:4/484: write d0/d8/d22/d2f/f9e [975245,71467] 0 2024-07-03T15:43:54.918 INFO:tasks.workunit.client.1.smithi148.stdout:4/485: symlink d0/d8/d19/la0 0 2024-07-03T15:43:54.922 INFO:tasks.workunit.client.1.smithi148.stdout:8/472: dwrite d4/da/d51/d18/d20/f66 [0,4194304] 0 2024-07-03T15:43:54.932 INFO:tasks.workunit.client.1.smithi148.stdout:7/645: dwrite d5/d7d/da7/d1c/d4a/d52/f5f [0,4194304] 0 2024-07-03T15:43:54.938 INFO:tasks.workunit.client.0.smithi028.stdout:1/107: dwrite d4/de/ff [4194304,4194304] 0 2024-07-03T15:43:54.938 INFO:tasks.workunit.client.0.smithi028.stdout:1/108: chown d4/f19 81097167 1 2024-07-03T15:43:54.939 INFO:tasks.workunit.client.1.smithi148.stdout:9/786: dwrite d0/d18/d24/d47/fa5 [0,4194304] 0 2024-07-03T15:43:54.939 INFO:tasks.workunit.client.1.smithi148.stdout:9/787: chown d0/d49/d43/d19/d7a 122 1 2024-07-03T15:43:54.939 INFO:tasks.workunit.client.1.smithi148.stdout:2/616: dwrite d5/d14/d86/fbc [4194304,4194304] 0 2024-07-03T15:43:54.942 INFO:tasks.workunit.client.1.smithi148.stdout:9/788: mkdir d0/dd/d25/d37/d40/da3/df0/df4 0 2024-07-03T15:43:54.942 INFO:tasks.workunit.client.1.smithi148.stdout:9/789: truncate d0/dd/f1f 4833819 0 2024-07-03T15:43:54.943 INFO:tasks.workunit.client.1.smithi148.stdout:8/473: dwrite d4/da/d51/d18/d20/f2b [0,4194304] 0 2024-07-03T15:43:54.945 INFO:tasks.workunit.client.1.smithi148.stdout:5/572: dwrite d7/da/f74 [0,4194304] 0 2024-07-03T15:43:54.950 INFO:tasks.workunit.client.1.smithi148.stdout:2/617: dread d5/d14/d86/db9/dbb/d13/d41/f74 [0,4194304] 0 2024-07-03T15:43:54.950 INFO:tasks.workunit.client.1.smithi148.stdout:2/618: creat d5/d14/d22/d60/d89/fc4 x:0 0 0 2024-07-03T15:43:54.952 INFO:tasks.workunit.client.1.smithi148.stdout:1/473: dwrite d0/d2/d2e/d39/d42/f49 [0,4194304] 0 2024-07-03T15:43:54.952 INFO:tasks.workunit.client.1.smithi148.stdout:1/474: write d0/d2/d6/f9e [500213,36056] 0 2024-07-03T15:43:54.952 INFO:tasks.workunit.client.1.smithi148.stdout:1/475: creat d0/d2/d11/d1e/f9f x:0 0 0 2024-07-03T15:43:54.953 INFO:tasks.workunit.client.1.smithi148.stdout:0/544: dwrite da/d44/d13/d14/d35/d8a/dab/f72 [0,4194304] 0 2024-07-03T15:43:54.955 INFO:tasks.workunit.client.1.smithi148.stdout:7/646: dwrite d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/f83 [0,4194304] 0 2024-07-03T15:43:54.955 INFO:tasks.workunit.client.1.smithi148.stdout:4/486: dread d0/d8/d3a/f47 [0,4194304] 0 2024-07-03T15:43:54.959 INFO:tasks.workunit.client.1.smithi148.stdout:6/580: dwrite d3/d50/d77/fa7 [0,4194304] 0 2024-07-03T15:43:54.959 INFO:tasks.workunit.client.1.smithi148.stdout:6/581: write d3/d5b/f99 [702950,107744] 0 2024-07-03T15:43:54.959 INFO:tasks.workunit.client.1.smithi148.stdout:6/582: write d3/f2c [48615,112170] 0 2024-07-03T15:43:54.959 INFO:tasks.workunit.client.1.smithi148.stdout:6/583: dread - d3/f86 zero size 2024-07-03T15:43:54.959 INFO:tasks.workunit.client.1.smithi148.stdout:6/584: fdatasync d3/fb0 0 2024-07-03T15:43:54.959 INFO:tasks.workunit.client.1.smithi148.stdout:6/585: chown d3/d1d/d28/d31/d61/faf 115848 1 2024-07-03T15:43:54.960 INFO:tasks.workunit.client.1.smithi148.stdout:5/573: unlink d7/da/df/d16/d2f/d44/d52/d86/fa3 0 2024-07-03T15:43:54.963 INFO:tasks.workunit.client.1.smithi148.stdout:2/619: mkdir d5/d14/d86/d93/dc5 0 2024-07-03T15:43:54.966 INFO:tasks.workunit.client.1.smithi148.stdout:1/476: creat d0/d2/d6/d14/d29/d94/fa0 x:0 0 0 2024-07-03T15:43:54.968 INFO:tasks.workunit.client.1.smithi148.stdout:7/647: truncate d5/fa 1016870 0 2024-07-03T15:43:54.970 INFO:tasks.workunit.client.1.smithi148.stdout:4/487: truncate d0/f50 140614 0 2024-07-03T15:43:54.970 INFO:tasks.workunit.client.1.smithi148.stdout:7/648: dread d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/fa3 [0,4194304] 0 2024-07-03T15:43:54.974 INFO:tasks.workunit.client.1.smithi148.stdout:5/574: rmdir d7/da/df/d16/d6a/db4 39 2024-07-03T15:43:54.976 INFO:tasks.workunit.client.1.smithi148.stdout:9/790: dwrite d0/dd/d25/d37/fe7 [4194304,4194304] 0 2024-07-03T15:43:54.977 INFO:tasks.workunit.client.1.smithi148.stdout:4/488: creat d0/d8/d22/d62/fa1 x:0 0 0 2024-07-03T15:43:54.978 INFO:tasks.workunit.client.1.smithi148.stdout:5/575: mkdir d7/da/df/d16/d2f/dc2 0 2024-07-03T15:43:54.989 INFO:tasks.workunit.client.1.smithi148.stdout:2/620: read d5/f40 [179606,50422] 0 2024-07-03T15:43:54.991 INFO:tasks.workunit.client.1.smithi148.stdout:0/545: rename da/d44/d24/lb0 to da/d58/daa/lb3 0 2024-07-03T15:43:54.995 INFO:tasks.workunit.client.1.smithi148.stdout:2/621: truncate d5/d14/d86/db9/dbb/d13/d41/f49 4910416 0 2024-07-03T15:43:54.995 INFO:tasks.workunit.client.1.smithi148.stdout:2/622: readlink d5/d14/d86/db9/dbb/d13/d2d/l34 0 2024-07-03T15:43:54.997 INFO:tasks.workunit.client.1.smithi148.stdout:9/791: write d0/f6 [2842233,89963] 0 2024-07-03T15:43:54.998 INFO:tasks.workunit.client.1.smithi148.stdout:6/586: rename d3/d50/f9a to d3/d1d/d28/d31/d4c/fc2 0 2024-07-03T15:43:54.998 INFO:tasks.workunit.client.1.smithi148.stdout:6/587: readlink d3/d1d/d28/d53/lc1 0 2024-07-03T15:43:54.999 INFO:tasks.workunit.client.1.smithi148.stdout:7/649: dwrite d5/d7d/da7/d1c/d33/f67 [0,4194304] 0 2024-07-03T15:43:55.001 INFO:tasks.workunit.client.1.smithi148.stdout:0/546: mkdir da/d58/d90/db4 0 2024-07-03T15:43:55.003 INFO:tasks.workunit.client.1.smithi148.stdout:6/588: mknod d3/d1d/d28/d3e/cc3 0 2024-07-03T15:43:55.003 INFO:tasks.workunit.client.1.smithi148.stdout:6/589: dread - d3/d1d/d28/d31/fbc zero size 2024-07-03T15:43:55.004 INFO:tasks.workunit.client.1.smithi148.stdout:2/623: read d5/d14/d22/fae [3090569,79163] 0 2024-07-03T15:43:55.007 INFO:tasks.workunit.client.1.smithi148.stdout:1/477: rename d0/d2/d11/d1e/f6f to d0/de/d44/d54/fa1 0 2024-07-03T15:43:55.008 INFO:tasks.workunit.client.1.smithi148.stdout:9/792: dread d0/d49/f73 [0,4194304] 0 2024-07-03T15:43:55.008 INFO:tasks.workunit.client.1.smithi148.stdout:9/793: fdatasync d0/d18/d24/d47/fa5 0 2024-07-03T15:43:55.009 INFO:tasks.workunit.client.1.smithi148.stdout:7/650: mknod d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/ce2 0 2024-07-03T15:43:55.012 INFO:tasks.workunit.client.1.smithi148.stdout:2/624: symlink d5/d14/d86/db9/dbb/d13/d2d/lc6 0 2024-07-03T15:43:55.013 INFO:tasks.workunit.client.1.smithi148.stdout:1/478: mknod d0/d2/d11/d1e/ca2 0 2024-07-03T15:43:55.017 INFO:tasks.workunit.client.1.smithi148.stdout:6/590: dread d3/d1d/f21 [0,4194304] 0 2024-07-03T15:43:55.018 INFO:tasks.workunit.client.1.smithi148.stdout:5/576: dwrite d7/da/f2b [4194304,4194304] 0 2024-07-03T15:43:55.019 INFO:tasks.workunit.client.1.smithi148.stdout:2/625: unlink d5/d7/f83 0 2024-07-03T15:43:55.021 INFO:tasks.workunit.client.1.smithi148.stdout:1/479: mkdir d0/d2/d2e/d39/d41/d77/da3 0 2024-07-03T15:43:55.021 INFO:tasks.workunit.client.1.smithi148.stdout:7/651: write d5/d7d/da7/d1c/d33/d4c/d21/d47/f5d [3569117,121600] 0 2024-07-03T15:43:55.025 INFO:tasks.workunit.client.1.smithi148.stdout:5/577: mknod d7/da/df/cc3 0 2024-07-03T15:43:55.025 INFO:tasks.workunit.client.1.smithi148.stdout:5/578: fdatasync d7/da/df/f1f 0 2024-07-03T15:43:55.027 INFO:tasks.workunit.client.0.smithi028.stdout:1/109: truncate d4/fd 4716384 0 2024-07-03T15:43:55.030 INFO:tasks.workunit.client.1.smithi148.stdout:9/794: dwrite d0/d18/feb [0,4194304] 0 2024-07-03T15:43:55.030 INFO:tasks.workunit.client.1.smithi148.stdout:9/795: creat d0/dd/d46/ff5 x:0 0 0 2024-07-03T15:43:55.031 INFO:tasks.workunit.client.1.smithi148.stdout:1/480: creat d0/d2/d2e/d39/d42/d62/d7b/fa4 x:0 0 0 2024-07-03T15:43:55.031 INFO:tasks.workunit.client.1.smithi148.stdout:7/652: symlink d5/d7d/da7/d1c/d4a/d66/le3 0 2024-07-03T15:43:55.031 INFO:tasks.workunit.client.1.smithi148.stdout:7/653: chown d5/d7d/da7/d1c/d33/d4c/d84/d98/dc2 34566 1 2024-07-03T15:43:55.034 INFO:tasks.workunit.client.1.smithi148.stdout:5/579: mkdir d7/da/df/d16/dac/dc1/dc4 0 2024-07-03T15:43:55.034 INFO:tasks.workunit.client.1.smithi148.stdout:6/591: rename d3/d1d/d28/d7f to d3/d5b/d74/dc4 0 2024-07-03T15:43:55.035 INFO:tasks.workunit.client.1.smithi148.stdout:6/592: fsync d3/d1d/d28/d53/d6d/fba 0 2024-07-03T15:43:55.035 INFO:tasks.workunit.client.1.smithi148.stdout:6/593: chown d3/d1d/d28/l46 55 1 2024-07-03T15:43:55.040 INFO:tasks.workunit.client.1.smithi148.stdout:5/580: mknod d7/da/df/d16/d1a/d58/da7/cc5 0 2024-07-03T15:43:55.043 INFO:tasks.workunit.client.1.smithi148.stdout:6/594: rmdir d3/d5b/d74/dc4/db2 0 2024-07-03T15:43:55.043 INFO:tasks.workunit.client.1.smithi148.stdout:6/595: stat d3/d50/f96 0 2024-07-03T15:43:55.045 INFO:tasks.workunit.client.1.smithi148.stdout:2/626: rename d5/d7/c99 to d5/d14/d8f/cc7 0 2024-07-03T15:43:55.047 INFO:tasks.workunit.client.1.smithi148.stdout:5/581: mkdir d7/da/df/d16/dac/dc6 0 2024-07-03T15:43:55.048 INFO:tasks.workunit.client.1.smithi148.stdout:6/596: truncate d3/d1d/d28/d31/d4c/f57 2764323 0 2024-07-03T15:43:55.049 INFO:tasks.workunit.client.1.smithi148.stdout:4/489: write d0/f50 [762331,123492] 0 2024-07-03T15:43:55.050 INFO:tasks.workunit.client.1.smithi148.stdout:4/490: fdatasync d0/d8/d44/f6a 0 2024-07-03T15:43:55.051 INFO:tasks.workunit.client.1.smithi148.stdout:2/627: creat d5/d14/d86/db9/dbb/d13/d41/d7b/fc8 x:0 0 0 2024-07-03T15:43:55.051 INFO:tasks.workunit.client.1.smithi148.stdout:2/628: chown d5/d14/d22/d60/f65 7 1 2024-07-03T15:43:55.057 INFO:tasks.workunit.client.1.smithi148.stdout:6/597: creat d3/d88/fc5 x:0 0 0 2024-07-03T15:43:55.057 INFO:tasks.workunit.client.1.smithi148.stdout:6/598: readlink d3/d5b/l63 0 2024-07-03T15:43:55.059 INFO:tasks.workunit.client.1.smithi148.stdout:4/491: rename d0/d8/d22/d62/f89 to d0/d91/fa2 0 2024-07-03T15:43:55.059 INFO:tasks.workunit.client.1.smithi148.stdout:4/492: write d0/d28/d7d/d6f/f82 [33899,33769] 0 2024-07-03T15:43:55.061 INFO:tasks.workunit.client.1.smithi148.stdout:5/582: mknod d7/da/d21/cc7 0 2024-07-03T15:43:55.061 INFO:tasks.workunit.client.1.smithi148.stdout:5/583: write d7/da/fb5 [701050,91056] 0 2024-07-03T15:43:55.065 INFO:tasks.workunit.client.1.smithi148.stdout:4/493: write d0/d28/d7d/f8a [2427445,46404] 0 2024-07-03T15:43:55.065 INFO:tasks.workunit.client.1.smithi148.stdout:4/494: fsync d0/d8/d3a/f66 0 2024-07-03T15:43:55.065 INFO:tasks.workunit.client.1.smithi148.stdout:7/654: dwrite d5/d7d/da7/d1c/d59/f7c [0,4194304] 0 2024-07-03T15:43:55.066 INFO:tasks.workunit.client.1.smithi148.stdout:1/481: dwrite d0/d2/d6/d14/d29/d94/fa0 [0,4194304] 0 2024-07-03T15:43:55.066 INFO:tasks.workunit.client.1.smithi148.stdout:9/796: dwrite d0/d49/d43/f22 [0,4194304] 0 2024-07-03T15:43:55.084 INFO:tasks.workunit.client.1.smithi148.stdout:6/599: symlink d3/d1d/d28/d3e/lc6 0 2024-07-03T15:43:55.084 INFO:tasks.workunit.client.1.smithi148.stdout:6/600: write d3/d5b/f99 [1856366,113664] 0 2024-07-03T15:43:55.084 INFO:tasks.workunit.client.1.smithi148.stdout:6/601: chown d3/d1d/d28/d53/d6d/d8f/lb8 14306695 1 2024-07-03T15:43:55.087 INFO:tasks.workunit.client.1.smithi148.stdout:2/629: rename d5/d7/d64/db2/lbf to d5/d14/d22/d60/d76/db3/lc9 0 2024-07-03T15:43:55.088 INFO:tasks.workunit.client.1.smithi148.stdout:5/584: creat d7/da/df/d16/d2f/d44/d52/d86/fc8 x:0 0 0 2024-07-03T15:43:55.090 INFO:tasks.workunit.client.1.smithi148.stdout:1/482: dwrite d0/d2/d11/f47 [0,4194304] 0 2024-07-03T15:43:55.090 INFO:tasks.workunit.client.1.smithi148.stdout:1/483: creat d0/de/d2f/d72/fa5 x:0 0 0 2024-07-03T15:43:55.090 INFO:tasks.workunit.client.1.smithi148.stdout:7/655: creat d5/d7d/da7/d1c/d33/d4c/d21/d39/dd2/fe4 x:0 0 0 2024-07-03T15:43:55.093 INFO:tasks.workunit.client.1.smithi148.stdout:4/495: dwrite d0/f6d [0,4194304] 0 2024-07-03T15:43:55.097 INFO:tasks.workunit.client.1.smithi148.stdout:4/496: dread d0/f30 [0,4194304] 0 2024-07-03T15:43:55.108 INFO:tasks.workunit.client.1.smithi148.stdout:7/656: dwrite d5/d7d/da7/d1c/d33/d4c/d84/fbf [0,4194304] 0 2024-07-03T15:43:55.137 INFO:tasks.workunit.client.1.smithi148.stdout:6/602: creat d3/d5b/d74/dc4/d85/fc7 x:0 0 0 2024-07-03T15:43:55.139 INFO:tasks.workunit.client.1.smithi148.stdout:6/603: write d3/d1d/d28/d3e/f8b [62924,9298] 0 2024-07-03T15:43:55.143 INFO:tasks.workunit.client.1.smithi148.stdout:9/797: rename d0/ld4 to d0/d49/d43/dd8/ddf/lf6 0 2024-07-03T15:43:55.143 INFO:tasks.workunit.client.1.smithi148.stdout:2/630: mkdir d5/d14/d8b/dca 0 2024-07-03T15:43:55.144 INFO:tasks.workunit.client.1.smithi148.stdout:2/631: write d5/d14/f7a [4214393,76146] 0 2024-07-03T15:43:55.146 INFO:tasks.workunit.client.1.smithi148.stdout:5/585: creat d7/da/df/d16/dac/dc1/dc4/fc9 x:0 0 0 2024-07-03T15:43:55.154 INFO:tasks.workunit.client.1.smithi148.stdout:9/798: dread d0/d18/d24/d47/fa5 [0,4194304] 0 2024-07-03T15:43:55.155 INFO:tasks.workunit.client.1.smithi148.stdout:9/799: stat d0/d2f/f74 0 2024-07-03T15:43:55.165 INFO:tasks.workunit.client.1.smithi148.stdout:4/497: mkdir d0/d28/d7d/d6f/d7c/da3 0 2024-07-03T15:43:55.166 INFO:tasks.workunit.client.1.smithi148.stdout:5/586: dwrite d7/da/df/f71 [4194304,4194304] 0 2024-07-03T15:43:55.166 INFO:tasks.workunit.client.1.smithi148.stdout:5/587: fdatasync d7/da/df/d16/d2f/d44/f69 0 2024-07-03T15:43:55.172 INFO:tasks.workunit.client.1.smithi148.stdout:7/657: truncate d5/d7d/da7/d1c/d33/d4c/d21/d47/f5d 1788886 0 2024-07-03T15:43:55.175 INFO:tasks.workunit.client.1.smithi148.stdout:2/632: fdatasync d5/f9 0 2024-07-03T15:43:55.177 INFO:tasks.workunit.client.1.smithi148.stdout:5/588: dread d7/da/df/d16/d2f/d44/f69 [0,4194304] 0 2024-07-03T15:43:55.178 INFO:tasks.workunit.client.1.smithi148.stdout:1/484: rmdir d0/d2/d2e/d39/d42/d62 39 2024-07-03T15:43:55.178 INFO:tasks.workunit.client.1.smithi148.stdout:1/485: truncate d0/d2/d2e/d39/d42/f87 791355 0 2024-07-03T15:43:55.178 INFO:tasks.workunit.client.1.smithi148.stdout:1/486: creat d0/de/d89/fa6 x:0 0 0 2024-07-03T15:43:55.180 INFO:tasks.workunit.client.1.smithi148.stdout:9/800: link d0/d49/d43/d19/d7a/f95 d0/dd/d11/db4/ff7 0 2024-07-03T15:43:55.181 INFO:tasks.workunit.client.1.smithi148.stdout:7/658: dread d5/d7d/da7/f37 [0,4194304] 0 2024-07-03T15:43:55.182 INFO:tasks.workunit.client.1.smithi148.stdout:4/498: rmdir d0/d8/d19/d45/d49 39 2024-07-03T15:43:55.182 INFO:tasks.workunit.client.1.smithi148.stdout:4/499: readlink d0/l5a 0 2024-07-03T15:43:55.187 INFO:tasks.workunit.client.1.smithi148.stdout:6/604: rename d3/d1d/f91 to d3/d1d/d28/d53/d6d/fc8 0 2024-07-03T15:43:55.187 INFO:tasks.workunit.client.1.smithi148.stdout:6/605: dread - d3/d1d/d28/d31/fab zero size 2024-07-03T15:43:55.188 INFO:tasks.workunit.client.1.smithi148.stdout:9/801: dread d0/d18/df3/f82 [0,4194304] 0 2024-07-03T15:43:55.190 INFO:tasks.workunit.client.1.smithi148.stdout:1/487: readlink d0/de/l31 0 2024-07-03T15:43:55.195 INFO:tasks.workunit.client.1.smithi148.stdout:4/500: mknod d0/d28/d7d/d6f/d7c/da3/ca4 0 2024-07-03T15:43:55.198 INFO:tasks.workunit.client.1.smithi148.stdout:9/802: creat d0/dd/d9f/ff8 x:0 0 0 2024-07-03T15:43:55.202 INFO:tasks.workunit.client.1.smithi148.stdout:1/488: unlink d0/de/d89/fa6 0 2024-07-03T15:43:55.202 INFO:tasks.workunit.client.1.smithi148.stdout:4/501: mknod d0/d28/d7d/ca5 0 2024-07-03T15:43:55.203 INFO:tasks.workunit.client.1.smithi148.stdout:2/633: dwrite d5/d14/d86/db9/dbb/d68/f7c [4194304,4194304] 0 2024-07-03T15:43:55.203 INFO:tasks.workunit.client.1.smithi148.stdout:2/634: chown d5/d14/d86/db9/dbb/d13/d2d/la4 7830105 1 2024-07-03T15:43:55.207 INFO:tasks.workunit.client.1.smithi148.stdout:4/502: mkdir d0/d28/d7d/d6f/da6 0 2024-07-03T15:43:55.219 INFO:tasks.workunit.client.1.smithi148.stdout:2/635: dread d5/d14/d86/db9/dbb/d13/d41/f49 [0,4194304] 0 2024-07-03T15:43:55.226 INFO:tasks.workunit.client.1.smithi148.stdout:6/606: dwrite d3/d1d/d28/d31/d61/f71 [0,4194304] 0 2024-07-03T15:43:55.227 INFO:tasks.workunit.client.1.smithi148.stdout:9/803: dwrite d0/dd/d11/db4/fc0 [0,4194304] 0 2024-07-03T15:43:55.227 INFO:tasks.workunit.client.1.smithi148.stdout:9/804: stat d0 0 2024-07-03T15:43:55.227 INFO:tasks.workunit.client.1.smithi148.stdout:9/805: write d0/d18/d24/d47/d4d/dae/fdb [728531,76715] 0 2024-07-03T15:43:55.227 INFO:tasks.workunit.client.1.smithi148.stdout:1/489: dwrite d0/d2/d11/d3a/f70 [0,4194304] 0 2024-07-03T15:43:55.232 INFO:tasks.workunit.client.1.smithi148.stdout:6/607: mknod d3/d1d/d28/d31/d61/cc9 0 2024-07-03T15:43:55.237 INFO:tasks.workunit.client.1.smithi148.stdout:9/806: creat d0/dd/d25/d37/d40/d72/db8/def/ff9 x:0 0 0 2024-07-03T15:43:55.238 INFO:tasks.workunit.client.1.smithi148.stdout:9/807: fdatasync d0/dd/d25/d39/fcd 0 2024-07-03T15:43:55.238 INFO:tasks.workunit.client.1.smithi148.stdout:9/808: stat d0/dd/d46 0 2024-07-03T15:43:55.239 INFO:tasks.workunit.client.1.smithi148.stdout:6/608: dread d3/d1d/d28/d53/f80 [0,4194304] 0 2024-07-03T15:43:55.239 INFO:tasks.workunit.client.1.smithi148.stdout:6/609: readlink d3/d5b/d74/dc4/lb4 0 2024-07-03T15:43:55.239 INFO:tasks.workunit.client.1.smithi148.stdout:6/610: readlink d3/d1d/d28/d53/d6d/d8f/lb8 0 2024-07-03T15:43:55.239 INFO:tasks.workunit.client.1.smithi148.stdout:6/611: stat d3/c23 0 2024-07-03T15:43:55.244 INFO:tasks.workunit.client.1.smithi148.stdout:6/612: dread d3/da5/fb9 [0,4194304] 0 2024-07-03T15:43:55.244 INFO:tasks.workunit.client.1.smithi148.stdout:1/490: dwrite d0/d2/d6/fa [0,4194304] 0 2024-07-03T15:43:55.248 INFO:tasks.workunit.client.1.smithi148.stdout:9/809: creat d0/d2f/ffa x:0 0 0 2024-07-03T15:43:55.248 INFO:tasks.workunit.client.1.smithi148.stdout:9/810: write d0/d18/d24/f30 [691986,111386] 0 2024-07-03T15:43:55.249 INFO:tasks.workunit.client.1.smithi148.stdout:6/613: symlink d3/d5b/d74/dc4/d85/da2/lca 0 2024-07-03T15:43:55.253 INFO:tasks.workunit.client.1.smithi148.stdout:6/614: creat d3/da5/fcb x:0 0 0 2024-07-03T15:43:55.253 INFO:tasks.workunit.client.1.smithi148.stdout:1/491: dread d0/d2/d11/f47 [0,4194304] 0 2024-07-03T15:43:55.260 INFO:tasks.workunit.client.1.smithi148.stdout:9/811: read d0/d49/d43/f22 [2843664,3712] 0 2024-07-03T15:43:55.265 INFO:tasks.workunit.client.1.smithi148.stdout:3/529: sync 2024-07-03T15:43:55.265 INFO:tasks.workunit.client.1.smithi148.stdout:3/530: write dd/d33/d37/d68/d2e/d61/f63 [349417,70462] 0 2024-07-03T15:43:55.266 INFO:tasks.workunit.client.1.smithi148.stdout:3/531: mknod dd/d33/d37/d8d/ca4 0 2024-07-03T15:43:55.270 INFO:tasks.workunit.client.1.smithi148.stdout:3/532: link dd/d33/d37/f83 dd/d33/d37/d68/d2e/d40/d8c/fa5 0 2024-07-03T15:43:55.270 INFO:tasks.workunit.client.1.smithi148.stdout:3/533: rmdir dd/d33/d37/d68/d2e/d40 39 2024-07-03T15:43:55.271 INFO:tasks.workunit.client.1.smithi148.stdout:0/547: sync 2024-07-03T15:43:55.271 INFO:tasks.workunit.client.1.smithi148.stdout:8/474: sync 2024-07-03T15:43:55.272 INFO:tasks.workunit.client.1.smithi148.stdout:0/548: symlink da/d44/d13/d1d/d3e/lb5 0 2024-07-03T15:43:55.272 INFO:tasks.workunit.client.1.smithi148.stdout:0/549: stat da/d36/c6f 0 2024-07-03T15:43:55.274 INFO:tasks.workunit.client.1.smithi148.stdout:0/550: creat da/d44/d13/d14/d50/d95/d8c/fb6 x:0 0 0 2024-07-03T15:43:55.279 INFO:tasks.workunit.client.1.smithi148.stdout:3/534: dread f5 [0,4194304] 0 2024-07-03T15:43:55.281 INFO:tasks.workunit.client.1.smithi148.stdout:7/659: dread d5/d7d/da7/d1c/d33/d4c/d21/d47/f5d [0,4194304] 0 2024-07-03T15:43:55.281 INFO:tasks.workunit.client.1.smithi148.stdout:7/660: chown d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/db1/cda 56 1 2024-07-03T15:43:55.281 INFO:tasks.workunit.client.1.smithi148.stdout:7/661: creat d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/db2/fe5 x:0 0 0 2024-07-03T15:43:55.283 INFO:tasks.workunit.client.1.smithi148.stdout:7/662: dread d5/d7d/da7/f6a [0,4194304] 0 2024-07-03T15:43:55.283 INFO:tasks.workunit.client.1.smithi148.stdout:7/663: truncate d5/d7d/da7/d1c/d33/d4c/d21/d47/fd5 1725446 0 2024-07-03T15:43:55.284 INFO:tasks.workunit.client.1.smithi148.stdout:5/589: rename d7/da/df/cc3 to d7/da/df/d16/d1a/d58/cca 0 2024-07-03T15:43:55.289 INFO:tasks.workunit.client.1.smithi148.stdout:5/590: creat d7/da/df/fcb x:0 0 0 2024-07-03T15:43:55.291 INFO:tasks.workunit.client.1.smithi148.stdout:5/591: getdents d7/da/df/da0 0 2024-07-03T15:43:55.291 INFO:tasks.workunit.client.1.smithi148.stdout:5/592: chown d7/c9 218958 1 2024-07-03T15:43:55.292 INFO:tasks.workunit.client.1.smithi148.stdout:5/593: getdents d7/da/df/d16 0 2024-07-03T15:43:55.294 INFO:tasks.workunit.client.1.smithi148.stdout:5/594: mknod d7/da/dbe/ccc 0 2024-07-03T15:43:55.295 INFO:tasks.workunit.client.1.smithi148.stdout:5/595: mkdir d7/da/df/d16/d2f/d44/d52/d60/dcd 0 2024-07-03T15:43:55.297 INFO:tasks.workunit.client.1.smithi148.stdout:5/596: mknod d7/da/df/d16/dac/dc6/cce 0 2024-07-03T15:43:55.297 INFO:tasks.workunit.client.1.smithi148.stdout:5/597: fsync d7/da/df/d16/d2f/d44/d52/fb9 0 2024-07-03T15:43:55.299 INFO:tasks.workunit.client.1.smithi148.stdout:5/598: dread d7/da/fb5 [0,4194304] 0 2024-07-03T15:43:55.301 INFO:tasks.workunit.client.1.smithi148.stdout:4/503: rename d0/d8/d22 to d0/d28/da7 0 2024-07-03T15:43:55.301 INFO:tasks.workunit.client.1.smithi148.stdout:6/615: rename d3 to d3/d1d/d28/d31/d83/dcc 22 2024-07-03T15:43:55.301 INFO:tasks.workunit.client.1.smithi148.stdout:6/616: chown d3/d50/c9d 393 1 2024-07-03T15:43:55.303 INFO:tasks.workunit.client.1.smithi148.stdout:1/492: link d0/de/f83 d0/d2/d2e/fa7 0 2024-07-03T15:43:55.303 INFO:tasks.workunit.client.1.smithi148.stdout:1/493: write d0/de/d44/f5e [1096404,86464] 0 2024-07-03T15:43:55.303 INFO:tasks.workunit.client.1.smithi148.stdout:1/494: write d0/d2/d2e/d39/d42/d62/f9a [622082,45268] 0 2024-07-03T15:43:55.304 INFO:tasks.workunit.client.1.smithi148.stdout:5/599: truncate d7/da/df/d16/d1a/f34 4460434 0 2024-07-03T15:43:55.307 INFO:tasks.workunit.client.1.smithi148.stdout:8/475: rename d4/d86/la5 to d4/da/d14/d1a/d32/d38/lac 0 2024-07-03T15:43:55.309 INFO:tasks.workunit.client.1.smithi148.stdout:4/504: dread d0/d8/f10 [0,4194304] 0 2024-07-03T15:43:55.309 INFO:tasks.workunit.client.1.smithi148.stdout:8/476: dread d4/da/d47/d88/f27 [0,4194304] 0 2024-07-03T15:43:55.313 INFO:tasks.workunit.client.1.smithi148.stdout:8/477: creat d4/da/d14/d1a/d32/d38/d58/fad x:0 0 0 2024-07-03T15:43:55.314 INFO:tasks.workunit.client.1.smithi148.stdout:1/495: dread d0/d2/d11/d3a/f5b [0,4194304] 0 2024-07-03T15:43:55.317 INFO:tasks.workunit.client.1.smithi148.stdout:8/478: creat d4/da/d14/d1a/d32/fae x:0 0 0 2024-07-03T15:43:55.317 INFO:tasks.workunit.client.1.smithi148.stdout:8/479: creat d4/da/d47/d88/faf x:0 0 0 2024-07-03T15:43:55.317 INFO:tasks.workunit.client.1.smithi148.stdout:6/617: dwrite d3/d1d/d28/d31/d61/faf [0,4194304] 0 2024-07-03T15:43:55.321 INFO:tasks.workunit.client.1.smithi148.stdout:3/535: rename dd/d35/d49 to dd/da6 0 2024-07-03T15:43:55.321 INFO:tasks.workunit.client.1.smithi148.stdout:3/536: write dd/d33/d37/f83 [663852,52361] 0 2024-07-03T15:43:55.322 INFO:tasks.workunit.client.1.smithi148.stdout:4/505: dwrite d0/d28/da7/d2f/d41/f4d [0,4194304] 0 2024-07-03T15:43:55.329 INFO:tasks.workunit.client.1.smithi148.stdout:3/537: mkdir dd/d33/d37/d68/d67/da7 0 2024-07-03T15:43:55.329 INFO:tasks.workunit.client.1.smithi148.stdout:3/538: truncate dd/f2a 8519020 0 2024-07-03T15:43:55.330 INFO:tasks.workunit.client.1.smithi148.stdout:6/618: dwrite d3/fa [0,4194304] 0 2024-07-03T15:43:55.333 INFO:tasks.workunit.client.1.smithi148.stdout:9/812: creat d0/d49/d43/d19/d7a/ffb x:0 0 0 2024-07-03T15:43:55.333 INFO:tasks.workunit.client.1.smithi148.stdout:9/813: creat d0/dd/d46/ffc x:0 0 0 2024-07-03T15:43:55.333 INFO:tasks.workunit.client.1.smithi148.stdout:9/814: write d0/dd/d25/d39/fcd [4304516,10704] 0 2024-07-03T15:43:55.334 INFO:tasks.workunit.client.1.smithi148.stdout:1/496: rmdir d0/de/d2f/d72 39 2024-07-03T15:43:55.336 INFO:tasks.workunit.client.1.smithi148.stdout:8/480: symlink d4/da/d51/d18/lb0 0 2024-07-03T15:43:55.337 INFO:tasks.workunit.client.1.smithi148.stdout:3/539: truncate dd/d33/d37/f46 2328922 0 2024-07-03T15:43:55.341 INFO:tasks.workunit.client.1.smithi148.stdout:7/664: creat d5/d7d/da7/d1c/d33/fe6 x:0 0 0 2024-07-03T15:43:55.341 INFO:tasks.workunit.client.1.smithi148.stdout:7/665: write d5/d7d/da7/f6a [1736151,72788] 0 2024-07-03T15:43:55.341 INFO:tasks.workunit.client.1.smithi148.stdout:7/666: creat d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/db6/fe7 x:0 0 0 2024-07-03T15:43:55.341 INFO:tasks.workunit.client.1.smithi148.stdout:7/667: chown d5/d7d/da7/d1c/d23 1 1 2024-07-03T15:43:55.341 INFO:tasks.workunit.client.1.smithi148.stdout:9/815: dread d0/dd/f14 [8388608,4194304] 0 2024-07-03T15:43:55.343 INFO:tasks.workunit.client.1.smithi148.stdout:4/506: unlink d0/d8/d13/l39 0 2024-07-03T15:43:55.348 INFO:tasks.workunit.client.1.smithi148.stdout:1/497: rename d0/c1c to d0/d2/d6/d14/d29/d94/ca8 0 2024-07-03T15:43:55.350 INFO:tasks.workunit.client.1.smithi148.stdout:8/481: dwrite d4/da/d47/d88/fa0 [0,4194304] 0 2024-07-03T15:43:55.350 INFO:tasks.workunit.client.1.smithi148.stdout:3/540: creat dd/da6/fa8 x:0 0 0 2024-07-03T15:43:55.350 INFO:tasks.workunit.client.1.smithi148.stdout:3/541: write fa [1333835,49264] 0 2024-07-03T15:43:55.352 INFO:tasks.workunit.client.1.smithi148.stdout:6/619: getdents d3/d88 0 2024-07-03T15:43:55.354 INFO:tasks.workunit.client.1.smithi148.stdout:7/668: unlink d5/d7d/da7/d1c/d33/d4c/f26 0 2024-07-03T15:43:55.357 INFO:tasks.workunit.client.1.smithi148.stdout:9/816: mkdir d0/dd/d11/dfd 0 2024-07-03T15:43:55.358 INFO:tasks.workunit.client.1.smithi148.stdout:4/507: mkdir d0/d91/da8 0 2024-07-03T15:43:55.358 INFO:tasks.workunit.client.1.smithi148.stdout:4/508: chown d0/lc 654254 1 2024-07-03T15:43:55.361 INFO:tasks.workunit.client.1.smithi148.stdout:1/498: unlink d0/d2/d6/d14/f98 0 2024-07-03T15:43:55.368 INFO:tasks.workunit.client.1.smithi148.stdout:6/620: dwrite d3/fa [0,4194304] 0 2024-07-03T15:43:55.368 INFO:tasks.workunit.client.1.smithi148.stdout:6/621: dread - d3/d1d/d28/d31/d4c/f54 zero size 2024-07-03T15:43:55.368 INFO:tasks.workunit.client.1.smithi148.stdout:6/622: fdatasync d3/f1c 0 2024-07-03T15:43:55.368 INFO:tasks.workunit.client.1.smithi148.stdout:6/623: chown d3/d5b/d74/dc4/d85/fc7 89709 1 2024-07-03T15:43:55.372 INFO:tasks.workunit.client.1.smithi148.stdout:3/542: mknod dd/d33/d52/ca9 0 2024-07-03T15:43:55.374 INFO:tasks.workunit.client.1.smithi148.stdout:7/669: mkdir d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/de8 0 2024-07-03T15:43:55.374 INFO:tasks.workunit.client.1.smithi148.stdout:7/670: write d5/d7d/da7/d1c/d59/f76 [464193,44570] 0 2024-07-03T15:43:55.376 INFO:tasks.workunit.client.1.smithi148.stdout:9/817: rename d0/dd/l88 to d0/dd/d25/dcc/lfe 0 2024-07-03T15:43:55.377 INFO:tasks.workunit.client.1.smithi148.stdout:4/509: truncate d0/d91/fa2 929281 0 2024-07-03T15:43:55.379 INFO:tasks.workunit.client.1.smithi148.stdout:1/499: symlink d0/d2/d11/la9 0 2024-07-03T15:43:55.382 INFO:tasks.workunit.client.1.smithi148.stdout:6/624: mknod d3/d1d/d28/ccd 0 2024-07-03T15:43:55.384 INFO:tasks.workunit.client.1.smithi148.stdout:8/482: symlink d4/da/d14/d1a/d32/d38/d58/d8e/lb1 0 2024-07-03T15:43:55.385 INFO:tasks.workunit.client.1.smithi148.stdout:3/543: symlink dd/d33/d37/d68/d2e/d40/d45/d9a/laa 0 2024-07-03T15:43:55.385 INFO:tasks.workunit.client.1.smithi148.stdout:3/544: creat dd/d33/d37/d68/d2e/d40/d45/d9a/fab x:0 0 0 2024-07-03T15:43:55.385 INFO:tasks.workunit.client.1.smithi148.stdout:3/545: read - dd/d33/f87 zero size 2024-07-03T15:43:55.386 INFO:tasks.workunit.client.1.smithi148.stdout:3/546: write dd/d33/d37/d68/d67/fa0 [3279941,11266] 0 2024-07-03T15:43:55.386 INFO:tasks.workunit.client.1.smithi148.stdout:3/547: fdatasync dd/d33/d37/d68/d2e/d61/f63 0 2024-07-03T15:43:55.386 INFO:tasks.workunit.client.1.smithi148.stdout:7/671: mknod d5/d7d/da7/d1c/d4a/d52/ce9 0 2024-07-03T15:43:55.387 INFO:tasks.workunit.client.1.smithi148.stdout:7/672: stat d5/d7d/da7/d1c/d4a/d66 0 2024-07-03T15:43:55.387 INFO:tasks.workunit.client.1.smithi148.stdout:7/673: creat d5/d7d/da7/d1c/d33/d4c/d84/d98/d9e/fea x:0 0 0 2024-07-03T15:43:55.387 INFO:tasks.workunit.client.1.smithi148.stdout:7/674: write d5/f58 [682483,24289] 0 2024-07-03T15:43:55.387 INFO:tasks.workunit.client.1.smithi148.stdout:3/548: dread dd/d33/d37/d68/f32 [0,4194304] 0 2024-07-03T15:43:55.388 INFO:tasks.workunit.client.1.smithi148.stdout:9/818: mkdir d0/dd/d29/dff 0 2024-07-03T15:43:55.389 INFO:tasks.workunit.client.1.smithi148.stdout:9/819: dread - d0/dd/d46/ff5 zero size 2024-07-03T15:43:55.390 INFO:tasks.workunit.client.1.smithi148.stdout:6/625: dread d3/d1d/f38 [0,4194304] 0 2024-07-03T15:43:55.390 INFO:tasks.workunit.client.1.smithi148.stdout:4/510: mknod d0/d28/da7/d62/ca9 0 2024-07-03T15:43:55.390 INFO:tasks.workunit.client.1.smithi148.stdout:4/511: fdatasync d0/d28/d7d/d6f/f82 0 2024-07-03T15:43:55.391 INFO:tasks.workunit.client.1.smithi148.stdout:1/500: mknod d0/de/d89/caa 0 2024-07-03T15:43:55.395 INFO:tasks.workunit.client.1.smithi148.stdout:8/483: creat d4/da/d14/d1a/d32/d38/fb2 x:0 0 0 2024-07-03T15:43:55.395 INFO:tasks.workunit.client.1.smithi148.stdout:8/484: creat d4/d86/fb3 x:0 0 0 2024-07-03T15:43:55.403 INFO:tasks.workunit.client.1.smithi148.stdout:7/675: mknod d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/db1/ceb 0 2024-07-03T15:43:55.405 INFO:tasks.workunit.client.1.smithi148.stdout:3/549: mkdir dd/d35/dac 0 2024-07-03T15:43:55.405 INFO:tasks.workunit.client.1.smithi148.stdout:3/550: chown dd/d33/d37/d68/d2e/d61/f64 4 1 2024-07-03T15:43:55.405 INFO:tasks.workunit.client.1.smithi148.stdout:3/551: stat dd/d33/d37/d68/l38 0 2024-07-03T15:43:55.405 INFO:tasks.workunit.client.1.smithi148.stdout:9/820: truncate d0/d49/d43/d19/f1e 2680861 0 2024-07-03T15:43:55.407 INFO:tasks.workunit.client.1.smithi148.stdout:6/626: rmdir d3/d1d/d28/d31/d4c 39 2024-07-03T15:43:55.407 INFO:tasks.workunit.client.1.smithi148.stdout:6/627: write d3/d1d/d28/d3e/f8b [261613,4945] 0 2024-07-03T15:43:55.407 INFO:tasks.workunit.client.1.smithi148.stdout:6/628: write d3/d5b/d74/f84 [348573,116794] 0 2024-07-03T15:43:55.407 INFO:tasks.workunit.client.1.smithi148.stdout:6/629: fdatasync d3/d1d/d28/fa6 0 2024-07-03T15:43:55.407 INFO:tasks.workunit.client.1.smithi148.stdout:6/630: fsync f1 0 2024-07-03T15:43:55.407 INFO:tasks.workunit.client.1.smithi148.stdout:6/631: chown d3/d5b/d74/cbe 129 1 2024-07-03T15:43:55.408 INFO:tasks.workunit.client.1.smithi148.stdout:6/632: fsync d3/d1d/d28/d31/f98 0 2024-07-03T15:43:55.408 INFO:tasks.workunit.client.1.smithi148.stdout:6/633: fsync d3/d1d/d28/fa6 0 2024-07-03T15:43:55.410 INFO:tasks.workunit.client.1.smithi148.stdout:4/512: rename d0/d8/f4b to d0/d8/d19/faa 0 2024-07-03T15:43:55.410 INFO:tasks.workunit.client.1.smithi148.stdout:3/552: rename dd/d33/d37/d68/d2e to dd/d33/d37/d68/d2e/d40/d45/d73/dad 22 2024-07-03T15:43:55.416 INFO:tasks.workunit.client.1.smithi148.stdout:8/485: link d4/da/d14/d3a/l80 d4/da/d14/d1a/lb4 0 2024-07-03T15:43:55.419 INFO:tasks.workunit.client.1.smithi148.stdout:7/676: truncate d5/d7d/da7/f34 2483208 0 2024-07-03T15:43:55.421 INFO:tasks.workunit.client.1.smithi148.stdout:9/821: mkdir d0/dd/d100 0 2024-07-03T15:43:55.421 INFO:tasks.workunit.client.1.smithi148.stdout:5/600: read d7/da/df/d16/d1a/f50 [719336,3554] 0 2024-07-03T15:43:55.421 INFO:tasks.workunit.client.1.smithi148.stdout:5/601: write d7/da/df/d1d/f51 [1031187,73543] 0 2024-07-03T15:43:55.422 INFO:tasks.workunit.client.1.smithi148.stdout:6/634: link d3/da5/fb9 d3/d5b/d74/dc4/fce 0 2024-07-03T15:43:55.425 INFO:tasks.workunit.client.1.smithi148.stdout:3/553: rename dd/d33/d37/d68/f1f to dd/d33/d52/d57/fae 0 2024-07-03T15:43:55.427 INFO:tasks.workunit.client.1.smithi148.stdout:1/501: mknod d0/de/cab 0 2024-07-03T15:43:55.429 INFO:tasks.workunit.client.1.smithi148.stdout:7/677: read d5/d7d/da7/d1c/d33/d4c/f12 [2161500,119384] 0 2024-07-03T15:43:55.431 INFO:tasks.workunit.client.1.smithi148.stdout:9/822: symlink d0/d18/df3/da8/l101 0 2024-07-03T15:43:55.433 INFO:tasks.workunit.client.1.smithi148.stdout:5/602: mknod d7/da/df/d16/d2f/d44/d52/ccf 0 2024-07-03T15:43:55.435 INFO:tasks.workunit.client.1.smithi148.stdout:6/635: rename d3/d50/d77/fa7 to d3/d50/fcf 0 2024-07-03T15:43:55.437 INFO:tasks.workunit.client.1.smithi148.stdout:3/554: rmdir dd 39 2024-07-03T15:43:55.438 INFO:tasks.workunit.client.1.smithi148.stdout:8/486: rmdir d4/da/d51/d18/d20/d82 39 2024-07-03T15:43:55.439 INFO:tasks.workunit.client.1.smithi148.stdout:8/487: chown d4/da/d47/d88/fa0 0 1 2024-07-03T15:43:55.439 INFO:tasks.workunit.client.1.smithi148.stdout:8/488: creat d4/fb5 x:0 0 0 2024-07-03T15:43:55.439 INFO:tasks.workunit.client.1.smithi148.stdout:1/502: read d0/d2/d2e/d39/d42/f49 [1873838,60623] 0 2024-07-03T15:43:55.441 INFO:tasks.workunit.client.1.smithi148.stdout:7/678: unlink d5/d7d/da7/l92 0 2024-07-03T15:43:55.441 INFO:tasks.workunit.client.1.smithi148.stdout:7/679: write d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/f28 [4371499,87015] 0 2024-07-03T15:43:55.441 INFO:tasks.workunit.client.1.smithi148.stdout:7/680: chown d5/d7d/da7/d1c/d33/d4c/ldb 1 1 2024-07-03T15:43:55.443 INFO:tasks.workunit.client.1.smithi148.stdout:5/603: readlink d7/da/d21/l63 0 2024-07-03T15:43:55.444 INFO:tasks.workunit.client.1.smithi148.stdout:5/604: write d7/d13/d23/f78 [1682306,44640] 0 2024-07-03T15:43:55.444 INFO:tasks.workunit.client.1.smithi148.stdout:5/605: fdatasync d7/da/fb 0 2024-07-03T15:43:55.445 INFO:tasks.workunit.client.1.smithi148.stdout:6/636: creat d3/d5b/d74/dc4/d85/d8c/fd0 x:0 0 0 2024-07-03T15:43:55.446 INFO:tasks.workunit.client.1.smithi148.stdout:6/637: truncate d3/d1d/d28/d3e/f8b 1594551 0 2024-07-03T15:43:55.447 INFO:tasks.workunit.client.1.smithi148.stdout:4/513: dwrite d0/d8/d13/d67/f86 [0,4194304] 0 2024-07-03T15:43:55.452 INFO:tasks.workunit.client.1.smithi148.stdout:6/638: dread d3/f17 [0,4194304] 0 2024-07-03T15:43:55.452 INFO:tasks.workunit.client.1.smithi148.stdout:8/489: link d4/da/d50/c81 d4/da/cb6 0 2024-07-03T15:43:55.453 INFO:tasks.workunit.client.1.smithi148.stdout:1/503: dread d0/d2/d2e/d39/d41/f69 [4194304,4194304] 0 2024-07-03T15:43:55.454 INFO:tasks.workunit.client.1.smithi148.stdout:7/681: mkdir d5/d7d/da7/db4/dec 0 2024-07-03T15:43:55.457 INFO:tasks.workunit.client.1.smithi148.stdout:9/823: dwrite d0/d63/fc7 [0,4194304] 0 2024-07-03T15:43:55.458 INFO:tasks.workunit.client.1.smithi148.stdout:5/606: creat d7/da/df/d16/dac/fd0 x:0 0 0 2024-07-03T15:43:55.458 INFO:tasks.workunit.client.1.smithi148.stdout:5/607: dread - d7/da/df/d1d/fb0 zero size 2024-07-03T15:43:55.458 INFO:tasks.workunit.client.1.smithi148.stdout:5/608: chown d7/d13/d23/c43 214095764 1 2024-07-03T15:43:55.463 INFO:tasks.workunit.client.1.smithi148.stdout:4/514: rename d0/d8/d19/d45 to d0/d91/da8/dab 0 2024-07-03T15:43:55.466 INFO:tasks.workunit.client.1.smithi148.stdout:3/555: rmdir dd/d35/d81 39 2024-07-03T15:43:55.468 INFO:tasks.workunit.client.1.smithi148.stdout:6/639: symlink d3/daa/ld1 0 2024-07-03T15:43:55.468 INFO:tasks.workunit.client.1.smithi148.stdout:6/640: read d3/d1d/d28/d31/f4b [2503623,118955] 0 2024-07-03T15:43:55.469 INFO:tasks.workunit.client.1.smithi148.stdout:1/504: creat d0/d2/fac x:0 0 0 2024-07-03T15:43:55.469 INFO:tasks.workunit.client.1.smithi148.stdout:1/505: readlink d0/d2/d11/d1e/l6d 0 2024-07-03T15:43:55.473 INFO:tasks.workunit.client.1.smithi148.stdout:5/609: mkdir d7/da/df/da0/dd1 0 2024-07-03T15:43:55.476 INFO:tasks.workunit.client.1.smithi148.stdout:9/824: dwrite d0/d18/d24/d47/d4d/fec [0,4194304] 0 2024-07-03T15:43:55.476 INFO:tasks.workunit.client.1.smithi148.stdout:9/825: chown d0/dd/ca 4200152 1 2024-07-03T15:43:55.476 INFO:tasks.workunit.client.1.smithi148.stdout:9/826: creat d0/f102 x:0 0 0 2024-07-03T15:43:55.477 INFO:tasks.workunit.client.1.smithi148.stdout:8/490: dwrite d4/da/d50/f77 [0,4194304] 0 2024-07-03T15:43:55.480 INFO:tasks.workunit.client.1.smithi148.stdout:9/827: dread d0/d18/d24/d47/d4d/dae/fdb [0,4194304] 0 2024-07-03T15:43:55.481 INFO:tasks.workunit.client.1.smithi148.stdout:9/828: read d0/d18/d24/d47/d4d/dae/fdb [306724,78385] 0 2024-07-03T15:43:55.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:55 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:43:55.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:55 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:43:55.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:55 smithi148 ceph-mon[36056]: pgmap v240: 65 pgs: 65 active+clean; 1.1 GiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 103 MiB/s rd, 128 MiB/s wr, 662 op/s 2024-07-03T15:43:55.488 INFO:tasks.workunit.client.1.smithi148.stdout:4/515: symlink d0/d91/lac 0 2024-07-03T15:43:55.489 INFO:tasks.workunit.client.1.smithi148.stdout:3/556: mkdir dd/d33/d37/d8d/daf 0 2024-07-03T15:43:55.489 INFO:tasks.workunit.client.1.smithi148.stdout:3/557: write dd/d33/d37/f82 [4729646,81374] 0 2024-07-03T15:43:55.489 INFO:tasks.workunit.client.1.smithi148.stdout:1/506: symlink d0/d2/d2e/lad 0 2024-07-03T15:43:55.492 INFO:tasks.workunit.client.1.smithi148.stdout:5/610: mkdir d7/d13/d54/dd2 0 2024-07-03T15:43:55.492 INFO:tasks.workunit.client.1.smithi148.stdout:5/611: chown d7/da/df/d16/f20 0 1 2024-07-03T15:43:55.492 INFO:tasks.workunit.client.1.smithi148.stdout:5/612: write d7/da/df/d16/d2f/f38 [5171502,44174] 0 2024-07-03T15:43:55.492 INFO:tasks.workunit.client.1.smithi148.stdout:4/516: write d0/d8/d13/d67/f86 [3244985,74417] 0 2024-07-03T15:43:55.496 INFO:tasks.workunit.client.1.smithi148.stdout:5/613: dread d7/da/df/d1d/f2e [0,4194304] 0 2024-07-03T15:43:55.499 INFO:tasks.workunit.client.1.smithi148.stdout:8/491: dwrite d4/da/d51/d18/d20/f21 [0,4194304] 0 2024-07-03T15:43:55.500 INFO:tasks.workunit.client.1.smithi148.stdout:8/492: read - d4/da/d14/f67 zero size 2024-07-03T15:43:55.500 INFO:tasks.workunit.client.1.smithi148.stdout:8/493: write d4/da/f3e [1535069,60402] 0 2024-07-03T15:43:55.500 INFO:tasks.workunit.client.1.smithi148.stdout:8/494: fsync d4/da/d47/d88/d92/fa3 0 2024-07-03T15:43:55.502 INFO:tasks.workunit.client.1.smithi148.stdout:3/558: dwrite dd/d33/f88 [0,4194304] 0 2024-07-03T15:43:55.503 INFO:tasks.workunit.client.1.smithi148.stdout:9/829: symlink d0/d18/d24/d47/d4d/dae/l103 0 2024-07-03T15:43:55.504 INFO:tasks.workunit.client.1.smithi148.stdout:3/559: write dd/d33/f4b [905064,4729] 0 2024-07-03T15:43:55.508 INFO:tasks.workunit.client.1.smithi148.stdout:6/641: getdents d3/d5b/d74/dc4/d85 0 2024-07-03T15:43:55.508 INFO:tasks.workunit.client.1.smithi148.stdout:2/636: sync 2024-07-03T15:43:55.509 INFO:tasks.workunit.client.1.smithi148.stdout:5/614: dwrite d7/da/df/d16/d2f/f93 [0,4194304] 0 2024-07-03T15:43:55.509 INFO:tasks.workunit.client.1.smithi148.stdout:5/615: truncate d7/da/df/d16/d2f/d44/d52/d86/fc8 43825 0 2024-07-03T15:43:55.510 INFO:tasks.workunit.client.1.smithi148.stdout:4/517: mkdir d0/d91/da8/dab/dad 0 2024-07-03T15:43:55.516 INFO:tasks.workunit.client.1.smithi148.stdout:8/495: mkdir d4/da/d14/d1a/d32/d38/db7 0 2024-07-03T15:43:55.521 INFO:tasks.workunit.client.1.smithi148.stdout:9/830: rename d0/dd/d25/d37/fe7 to d0/dd/d25/dcc/f104 0 2024-07-03T15:43:55.523 INFO:tasks.workunit.client.1.smithi148.stdout:3/560: mkdir dd/d35/d41/d89/db0 0 2024-07-03T15:43:55.525 INFO:tasks.workunit.client.1.smithi148.stdout:1/507: mkdir d0/d2/dae 0 2024-07-03T15:43:55.525 INFO:tasks.workunit.client.1.smithi148.stdout:1/508: truncate d0/de/d44/d54/d5d/f8c 464107 0 2024-07-03T15:43:55.525 INFO:tasks.workunit.client.1.smithi148.stdout:1/509: fdatasync d0/d2/d11/d3a/f70 0 2024-07-03T15:43:55.529 INFO:tasks.workunit.client.1.smithi148.stdout:0/551: sync 2024-07-03T15:43:55.529 INFO:tasks.workunit.client.1.smithi148.stdout:0/552: creat da/d44/d13/d14/d50/d95/d8c/fb7 x:0 0 0 2024-07-03T15:43:55.529 INFO:tasks.workunit.client.1.smithi148.stdout:0/553: readlink da/d44/d4a/la6 0 2024-07-03T15:43:55.529 INFO:tasks.workunit.client.1.smithi148.stdout:0/554: truncate da/d17/d2a/d2f/f9e 87391 0 2024-07-03T15:43:55.531 INFO:tasks.workunit.client.1.smithi148.stdout:5/616: mknod d7/da/df/da0/dd1/cd3 0 2024-07-03T15:43:55.534 INFO:tasks.workunit.client.1.smithi148.stdout:4/518: mkdir d0/d28/d7d/d6f/d7c/dae 0 2024-07-03T15:43:55.536 INFO:tasks.workunit.client.1.smithi148.stdout:2/637: dwrite d5/d14/d86/db9/dbb/d13/d2d/fb8 [0,4194304] 0 2024-07-03T15:43:55.536 INFO:tasks.workunit.client.1.smithi148.stdout:2/638: write d5/d14/f58 [4896732,101184] 0 2024-07-03T15:43:55.536 INFO:tasks.workunit.client.1.smithi148.stdout:2/639: write d5/d14/d86/db9/dbb/d68/fc2 [57220,5617] 0 2024-07-03T15:43:55.536 INFO:tasks.workunit.client.1.smithi148.stdout:2/640: write d5/d14/d86/db9/dbb/d13/d2d/f51 [4013221,74088] 0 2024-07-03T15:43:55.537 INFO:tasks.workunit.client.1.smithi148.stdout:2/641: chown d5/d5f/lb4 6986688 1 2024-07-03T15:43:55.537 INFO:tasks.workunit.client.1.smithi148.stdout:2/642: dread - d5/d5f/d79/d9c/fbe zero size 2024-07-03T15:43:55.537 INFO:tasks.workunit.client.1.smithi148.stdout:8/496: mkdir d4/da/d14/d3a/da8/db8 0 2024-07-03T15:43:55.537 INFO:tasks.workunit.client.1.smithi148.stdout:8/497: chown d4/da/d51/c75 3921155 1 2024-07-03T15:43:55.537 INFO:tasks.workunit.client.1.smithi148.stdout:3/561: dread dd/d33/f39 [0,4194304] 0 2024-07-03T15:43:55.537 INFO:tasks.workunit.client.1.smithi148.stdout:3/562: dread - dd/d33/d37/d68/d2e/d40/d45/fa3 zero size 2024-07-03T15:43:55.540 INFO:tasks.workunit.client.1.smithi148.stdout:1/510: mkdir d0/d2/d2e/d39/d42/daf 0 2024-07-03T15:43:55.544 INFO:tasks.workunit.client.1.smithi148.stdout:6/642: link d3/d1d/d28/d31/d61/faf d3/d1d/fd2 0 2024-07-03T15:43:55.544 INFO:tasks.workunit.client.1.smithi148.stdout:6/643: chown d3/lf 24 1 2024-07-03T15:43:55.544 INFO:tasks.workunit.client.1.smithi148.stdout:6/644: write d3/f1e [2462396,110054] 0 2024-07-03T15:43:55.551 INFO:tasks.workunit.client.1.smithi148.stdout:0/555: unlink da/d44/d13/d14/d50/f5d 0 2024-07-03T15:43:55.555 INFO:tasks.workunit.client.1.smithi148.stdout:5/617: creat d7/da/d21/fd4 x:0 0 0 2024-07-03T15:43:55.556 INFO:tasks.workunit.client.1.smithi148.stdout:4/519: mkdir d0/d91/da8/dab/dad/daf 0 2024-07-03T15:43:55.562 INFO:tasks.workunit.client.1.smithi148.stdout:2/643: truncate d5/d14/d86/db9/dbb/d13/f1b 814676 0 2024-07-03T15:43:55.563 INFO:tasks.workunit.client.1.smithi148.stdout:2/644: write d5/d14/d86/db9/dbb/d68/fc2 [546172,122713] 0 2024-07-03T15:43:55.563 INFO:tasks.workunit.client.1.smithi148.stdout:8/498: mkdir d4/da/d14/d1a/d32/d38/d58/d8e/db9 0 2024-07-03T15:43:55.564 INFO:tasks.workunit.client.1.smithi148.stdout:9/831: rmdir d0/dd/d29 39 2024-07-03T15:43:55.566 INFO:tasks.workunit.client.1.smithi148.stdout:5/618: mknod d7/da/df/d16/cd5 0 2024-07-03T15:43:55.567 INFO:tasks.workunit.client.1.smithi148.stdout:4/520: mkdir d0/d28/db0 0 2024-07-03T15:43:55.569 INFO:tasks.workunit.client.1.smithi148.stdout:3/563: dwrite dd/f72 [0,4194304] 0 2024-07-03T15:43:55.570 INFO:tasks.workunit.client.1.smithi148.stdout:6/645: dwrite d3/d50/fcf [0,4194304] 0 2024-07-03T15:43:55.571 INFO:tasks.workunit.client.1.smithi148.stdout:2/645: link d5/d5f/lb4 d5/d14/d86/db9/dbb/d13/d1d/lcb 0 2024-07-03T15:43:55.573 INFO:tasks.workunit.client.1.smithi148.stdout:1/511: dwrite d0/d2/f75 [0,4194304] 0 2024-07-03T15:43:55.573 INFO:tasks.workunit.client.1.smithi148.stdout:1/512: creat d0/d2/d2e/d39/d42/fb0 x:0 0 0 2024-07-03T15:43:55.573 INFO:tasks.workunit.client.1.smithi148.stdout:1/513: fsync d0/f1f 0 2024-07-03T15:43:55.573 INFO:tasks.workunit.client.1.smithi148.stdout:1/514: truncate d0/de/d44/d54/d5d/f8c 643585 0 2024-07-03T15:43:55.573 INFO:tasks.workunit.client.1.smithi148.stdout:1/515: readlink d0/d2/d11/l2a 0 2024-07-03T15:43:55.573 INFO:tasks.workunit.client.1.smithi148.stdout:1/516: write d0/d2/d11/d1e/f3c [4152680,18048] 0 2024-07-03T15:43:55.577 INFO:tasks.workunit.client.1.smithi148.stdout:9/832: creat d0/d49/f105 x:0 0 0 2024-07-03T15:43:55.580 INFO:tasks.workunit.client.1.smithi148.stdout:1/517: write d0/d2/d2e/d39/d42/d62/f4f [935520,29918] 0 2024-07-03T15:43:55.581 INFO:tasks.workunit.client.1.smithi148.stdout:4/521: mkdir d0/d8/d13/d67/db1 0 2024-07-03T15:43:55.581 INFO:tasks.workunit.client.1.smithi148.stdout:4/522: getdents d0/d28/d6e 0 2024-07-03T15:43:55.583 INFO:tasks.workunit.client.1.smithi148.stdout:4/523: write d0/f30 [1024150,120672] 0 2024-07-03T15:43:55.583 INFO:tasks.workunit.client.1.smithi148.stdout:4/524: chown d0/d8/d3a/c5d 2979 1 2024-07-03T15:43:55.592 INFO:tasks.workunit.client.1.smithi148.stdout:3/564: dwrite dd/d33/d37/f53 [0,4194304] 0 2024-07-03T15:43:55.592 INFO:tasks.workunit.client.1.smithi148.stdout:3/565: fsync dd/d33/d37/d68/f9c 0 2024-07-03T15:43:55.592 INFO:tasks.workunit.client.1.smithi148.stdout:3/566: fdatasync dd/da6/f85 0 2024-07-03T15:43:55.601 INFO:tasks.workunit.client.1.smithi148.stdout:9/833: mkdir d0/dd/d100/d106 0 2024-07-03T15:43:55.601 INFO:tasks.workunit.client.1.smithi148.stdout:5/619: rmdir d7/da/df/d16/d1a/d58 39 2024-07-03T15:43:55.602 INFO:tasks.workunit.client.1.smithi148.stdout:5/620: write d7/da/df/fcb [964665,76867] 0 2024-07-03T15:43:55.602 INFO:tasks.workunit.client.1.smithi148.stdout:5/621: creat d7/d13/d54/fd6 x:0 0 0 2024-07-03T15:43:55.602 INFO:tasks.workunit.client.1.smithi148.stdout:9/834: dread d0/dd/d9f/dde/f89 [0,4194304] 0 2024-07-03T15:43:55.606 INFO:tasks.workunit.client.1.smithi148.stdout:1/518: mknod d0/d2/d6/d14/d29/d94/cb1 0 2024-07-03T15:43:55.607 INFO:tasks.workunit.client.1.smithi148.stdout:1/519: read d0/d2/d6/fa [3592882,8729] 0 2024-07-03T15:43:55.607 INFO:tasks.workunit.client.1.smithi148.stdout:4/525: creat d0/d91/da8/dab/d49/fb2 x:0 0 0 2024-07-03T15:43:55.608 INFO:tasks.workunit.client.1.smithi148.stdout:6/646: getdents d3/d1d/d28/d31/d61 0 2024-07-03T15:43:55.609 INFO:tasks.workunit.client.1.smithi148.stdout:3/567: symlink dd/d33/d37/d68/d2e/d61/lb1 0 2024-07-03T15:43:55.611 INFO:tasks.workunit.client.1.smithi148.stdout:8/499: rename d4/da/d50 to d4/da/d14/d1a/d32/d38/dba 0 2024-07-03T15:43:55.614 INFO:tasks.workunit.client.1.smithi148.stdout:9/835: unlink d0/d18/d24/d47/d4d/fdc 0 2024-07-03T15:43:55.615 INFO:tasks.workunit.client.1.smithi148.stdout:1/520: readlink d0/de/l60 0 2024-07-03T15:43:55.617 INFO:tasks.workunit.client.1.smithi148.stdout:4/526: link d0/d91/da8/dab/d49/l4c d0/d28/lb3 0 2024-07-03T15:43:55.617 INFO:tasks.workunit.client.1.smithi148.stdout:4/527: getdents d0/d28/db0 0 2024-07-03T15:43:55.619 INFO:tasks.workunit.client.1.smithi148.stdout:6/647: mkdir d3/d1d/d28/d53/d6d/d8f/dd3 0 2024-07-03T15:43:55.619 INFO:tasks.workunit.client.1.smithi148.stdout:6/648: fdatasync d3/d5b/d74/dc4/d85/fc7 0 2024-07-03T15:43:55.619 INFO:tasks.workunit.client.1.smithi148.stdout:6/649: write d3/d5b/d74/dc4/d85/fc7 [104686,120985] 0 2024-07-03T15:43:55.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:55 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:43:55.625 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:55 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:43:55.625 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:55 smithi028 ceph-mon[29148]: pgmap v240: 65 pgs: 65 active+clean; 1.1 GiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 103 MiB/s rd, 128 MiB/s wr, 662 op/s 2024-07-03T15:43:55.625 INFO:tasks.workunit.client.1.smithi148.stdout:9/836: dread d0/dd/d25/d39/fcd [0,4194304] 0 2024-07-03T15:43:55.625 INFO:tasks.workunit.client.1.smithi148.stdout:3/568: getdents dd/d33/d52/d57 0 2024-07-03T15:43:55.625 INFO:tasks.workunit.client.1.smithi148.stdout:6/650: dread d3/d1d/d28/d31/f98 [0,4194304] 0 2024-07-03T15:43:55.627 INFO:tasks.workunit.client.1.smithi148.stdout:2/646: rename d5/d14/d86/db9/dbb/d13/d41/d7b/l5b to d5/d67/dc3/lcc 0 2024-07-03T15:43:55.629 INFO:tasks.workunit.client.1.smithi148.stdout:8/500: creat d4/da/d14/d1a/d32/d89/fbb x:0 0 0 2024-07-03T15:43:55.629 INFO:tasks.workunit.client.1.smithi148.stdout:8/501: truncate d4/da/f28 1382663 0 2024-07-03T15:43:55.632 INFO:tasks.workunit.client.1.smithi148.stdout:1/521: creat d0/d2/d11/fb2 x:0 0 0 2024-07-03T15:43:55.633 INFO:tasks.workunit.client.1.smithi148.stdout:1/522: dread d0/de/d44/d54/d5d/f8c [0,4194304] 0 2024-07-03T15:43:55.633 INFO:tasks.workunit.client.1.smithi148.stdout:9/837: getdents d0/d18/d24/d47 0 2024-07-03T15:43:55.634 INFO:tasks.workunit.client.1.smithi148.stdout:9/838: truncate d0/d49/d43/d19/f48 4684961 0 2024-07-03T15:43:55.634 INFO:tasks.workunit.client.1.smithi148.stdout:4/528: dwrite d0/d8/f10 [0,4194304] 0 2024-07-03T15:43:55.639 INFO:tasks.workunit.client.1.smithi148.stdout:5/622: rename d7/da/df/d16/d1a/d58/da7/f5c to d7/da/df/d16/d1a/fd7 0 2024-07-03T15:43:55.641 INFO:tasks.workunit.client.1.smithi148.stdout:8/502: rmdir d4/da/d51/d18/d20 39 2024-07-03T15:43:55.644 INFO:tasks.workunit.client.1.smithi148.stdout:1/523: getdents d0/d2/d2e/d39/d42/d62 0 2024-07-03T15:43:55.644 INFO:tasks.workunit.client.1.smithi148.stdout:1/524: creat d0/d2/d11/d1e/fb3 x:0 0 0 2024-07-03T15:43:55.644 INFO:tasks.workunit.client.1.smithi148.stdout:1/525: fdatasync d0/de/d44/d54/d5d/f8c 0 2024-07-03T15:43:55.644 INFO:tasks.workunit.client.1.smithi148.stdout:1/526: stat d0/d2/d6/d14/f6a 0 2024-07-03T15:43:55.644 INFO:tasks.workunit.client.1.smithi148.stdout:1/527: fsync d0/d2/d6/f9e 0 2024-07-03T15:43:55.644 INFO:tasks.workunit.client.1.smithi148.stdout:1/528: truncate d0/d2/d6/f5a 965008 0 2024-07-03T15:43:55.646 INFO:tasks.workunit.client.1.smithi148.stdout:9/839: creat d0/dd/d25/d37/d40/d72/db8/f107 x:0 0 0 2024-07-03T15:43:55.646 INFO:tasks.workunit.client.1.smithi148.stdout:9/840: chown d0/dd/d9f/dde/f89 1360 1 2024-07-03T15:43:55.651 INFO:tasks.workunit.client.1.smithi148.stdout:3/569: dwrite dd/d35/f56 [0,4194304] 0 2024-07-03T15:43:55.651 INFO:tasks.workunit.client.1.smithi148.stdout:3/570: write dd/da6/fa8 [375099,94561] 0 2024-07-03T15:43:55.652 INFO:tasks.workunit.client.1.smithi148.stdout:6/651: dwrite d3/d1d/d28/d3e/f68 [4194304,4194304] 0 2024-07-03T15:43:55.652 INFO:tasks.workunit.client.1.smithi148.stdout:6/652: truncate d3/d1d/d28/d3e/f8b 1864618 0 2024-07-03T15:43:55.656 INFO:tasks.workunit.client.1.smithi148.stdout:1/529: mkdir d0/d2/d2e/d39/d41/d77/db4 0 2024-07-03T15:43:55.656 INFO:tasks.workunit.client.1.smithi148.stdout:1/530: write d0/de/d44/d54/d5d/f8b [554582,63243] 0 2024-07-03T15:43:55.656 INFO:tasks.workunit.client.1.smithi148.stdout:1/531: fdatasync d0/d2/d2e/d39/d41/f65 0 2024-07-03T15:43:55.656 INFO:tasks.workunit.client.1.smithi148.stdout:1/532: chown d0/d2/d2e/f85 14667 1 2024-07-03T15:43:55.656 INFO:tasks.workunit.client.1.smithi148.stdout:1/533: write d0/d2/d2e/d39/d41/d77/f84 [558937,95013] 0 2024-07-03T15:43:55.657 INFO:tasks.workunit.client.1.smithi148.stdout:3/571: write dd/f15 [4926187,3501] 0 2024-07-03T15:43:55.657 INFO:tasks.workunit.client.1.smithi148.stdout:3/572: readlink dd/d33/d52/d57/l9b 0 2024-07-03T15:43:55.657 INFO:tasks.workunit.client.1.smithi148.stdout:3/573: fsync dd/d33/d37/d68/d2e/d61/f76 0 2024-07-03T15:43:55.658 INFO:tasks.workunit.client.1.smithi148.stdout:9/841: link d0/dd/d25/d37/d40/c67 d0/dd/d25/c108 0 2024-07-03T15:43:55.662 INFO:tasks.workunit.client.1.smithi148.stdout:8/503: dwrite d4/da/d47/f94 [0,4194304] 0 2024-07-03T15:43:55.663 INFO:tasks.workunit.client.1.smithi148.stdout:4/529: mkdir d0/d8/d13/db4 0 2024-07-03T15:43:55.671 INFO:tasks.workunit.client.1.smithi148.stdout:8/504: write d4/da/d14/d17/f26 [876757,33794] 0 2024-07-03T15:43:55.672 INFO:tasks.workunit.client.1.smithi148.stdout:8/505: chown d4/da/d51/d18/c3f 195092823 1 2024-07-03T15:43:55.673 INFO:tasks.workunit.client.1.smithi148.stdout:9/842: dwrite d0/dd/d25/d37/d40/f62 [0,4194304] 0 2024-07-03T15:43:55.683 INFO:tasks.workunit.client.1.smithi148.stdout:6/653: getdents d3/daa 0 2024-07-03T15:43:55.683 INFO:tasks.workunit.client.1.smithi148.stdout:6/654: fsync d3/d5b/d74/dc4/fce 0 2024-07-03T15:43:55.686 INFO:tasks.workunit.client.1.smithi148.stdout:9/843: dwrite d0/d2f/fb5 [4194304,4194304] 0 2024-07-03T15:43:55.686 INFO:tasks.workunit.client.1.smithi148.stdout:9/844: chown d0/dd/d25/d37/d40/c80 17424 1 2024-07-03T15:43:55.686 INFO:tasks.workunit.client.1.smithi148.stdout:3/574: unlink dd/da6/fa8 0 2024-07-03T15:43:55.686 INFO:tasks.workunit.client.1.smithi148.stdout:3/575: stat dd/l93 0 2024-07-03T15:43:55.690 INFO:tasks.workunit.client.1.smithi148.stdout:4/530: mkdir d0/d28/d6e/db5 0 2024-07-03T15:43:55.693 INFO:tasks.workunit.client.1.smithi148.stdout:9/845: dread d0/dd/f14 [0,4194304] 0 2024-07-03T15:43:55.693 INFO:tasks.workunit.client.1.smithi148.stdout:9/846: readlink d0/d49/d43/d19/d7a/la4 0 2024-07-03T15:43:55.693 INFO:tasks.workunit.client.1.smithi148.stdout:2/647: rename d5/d7 to d5/d14/d86/db9/dbb/d68/dcd 0 2024-07-03T15:43:55.693 INFO:tasks.workunit.client.1.smithi148.stdout:2/648: chown d5/d14/d86/db9/dbb/d13/d41/f74 1147025 1 2024-07-03T15:43:55.695 INFO:tasks.workunit.client.1.smithi148.stdout:2/649: dread d5/d14/d86/db9/dbb/d68/fc2 [0,4194304] 0 2024-07-03T15:43:55.697 INFO:tasks.workunit.client.1.smithi148.stdout:2/650: write d5/d14/f58 [4463363,96026] 0 2024-07-03T15:43:55.700 INFO:tasks.workunit.client.1.smithi148.stdout:3/576: symlink dd/d33/d37/d68/d2e/d61/lb2 0 2024-07-03T15:43:55.702 INFO:tasks.workunit.client.1.smithi148.stdout:6/655: dwrite d3/d5b/f99 [0,4194304] 0 2024-07-03T15:43:55.703 INFO:tasks.workunit.client.1.smithi148.stdout:4/531: unlink d0/d28/da7/d2f/f9e 0 2024-07-03T15:43:55.703 INFO:tasks.workunit.client.1.smithi148.stdout:4/532: write d0/f50 [1503578,77994] 0 2024-07-03T15:43:55.703 INFO:tasks.workunit.client.1.smithi148.stdout:4/533: chown d0/c9f 7941363 1 2024-07-03T15:43:55.704 INFO:tasks.workunit.client.1.smithi148.stdout:8/506: write d4/da/d51/f7 [3350896,129910] 0 2024-07-03T15:43:55.704 INFO:tasks.workunit.client.1.smithi148.stdout:9/847: truncate d0/d18/f26 777792 0 2024-07-03T15:43:55.706 INFO:tasks.workunit.client.1.smithi148.stdout:1/534: rename d0/d2/fac to d0/d2/d2e/d39/d41/d77/d80/fb5 0 2024-07-03T15:43:55.706 INFO:tasks.workunit.client.1.smithi148.stdout:1/535: chown d0/d2/d2e/d39/d42/d62/d7b/fa4 7 1 2024-07-03T15:43:55.707 INFO:tasks.workunit.client.1.smithi148.stdout:2/651: mkdir d5/d14/d8f/dce 0 2024-07-03T15:43:55.708 INFO:tasks.workunit.client.1.smithi148.stdout:3/577: mkdir dd/da6/db3 0 2024-07-03T15:43:55.713 INFO:tasks.workunit.client.1.smithi148.stdout:6/656: mkdir d3/d1d/d28/d53/dd4 0 2024-07-03T15:43:55.713 INFO:tasks.workunit.client.1.smithi148.stdout:6/657: write d3/d50/f7d [618747,19963] 0 2024-07-03T15:43:55.716 INFO:tasks.workunit.client.1.smithi148.stdout:4/534: mknod d0/d8/cb6 0 2024-07-03T15:43:55.719 INFO:tasks.workunit.client.1.smithi148.stdout:8/507: symlink d4/da/d14/d1a/d32/d38/dba/lbc 0 2024-07-03T15:43:55.719 INFO:tasks.workunit.client.1.smithi148.stdout:8/508: write d4/da/d51/f57 [499776,114606] 0 2024-07-03T15:43:55.721 INFO:tasks.workunit.client.1.smithi148.stdout:9/848: creat d0/dd/d25/d37/f109 x:0 0 0 2024-07-03T15:43:55.721 INFO:tasks.workunit.client.1.smithi148.stdout:9/849: chown d0/d18/d24/d47/c4b 3725 1 2024-07-03T15:43:55.722 INFO:tasks.workunit.client.1.smithi148.stdout:4/535: dread d0/f4 [0,4194304] 0 2024-07-03T15:43:55.722 INFO:tasks.workunit.client.1.smithi148.stdout:7/682: sync 2024-07-03T15:43:55.723 INFO:tasks.workunit.client.1.smithi148.stdout:1/536: creat d0/de/d44/d54/fb6 x:0 0 0 2024-07-03T15:43:55.725 INFO:tasks.workunit.client.1.smithi148.stdout:2/652: mknod d5/d14/ccf 0 2024-07-03T15:43:55.727 INFO:tasks.workunit.client.1.smithi148.stdout:6/658: dwrite d3/d1d/d28/d3e/f58 [0,4194304] 0 2024-07-03T15:43:55.728 INFO:tasks.workunit.client.1.smithi148.stdout:3/578: unlink dd/d33/d37/d68/d2e/l10 0 2024-07-03T15:43:55.728 INFO:tasks.workunit.client.1.smithi148.stdout:3/579: fsync dd/d33/d37/d68/d2e/d61/f64 0 2024-07-03T15:43:55.731 INFO:tasks.workunit.client.1.smithi148.stdout:9/850: unlink d0/dd/d25/d39/fd9 0 2024-07-03T15:43:55.732 INFO:tasks.workunit.client.1.smithi148.stdout:4/536: unlink d0/d8/d19/l4e 0 2024-07-03T15:43:55.734 INFO:tasks.workunit.client.1.smithi148.stdout:7/683: rename d5/c7a to d5/d7d/da7/d1c/d33/d4c/d21/d47/d8b/dd3/ced 0 2024-07-03T15:43:55.736 INFO:tasks.workunit.client.1.smithi148.stdout:1/537: mkdir d0/d2/d2e/d39/d42/db7 0 2024-07-03T15:43:55.738 INFO:tasks.workunit.client.1.smithi148.stdout:3/580: unlink dd/d33/d37/d68/d2e/c54 0 2024-07-03T15:43:55.738 INFO:tasks.workunit.client.1.smithi148.stdout:3/581: fsync dd/d35/f56 0 2024-07-03T15:43:55.740 INFO:tasks.workunit.client.1.smithi148.stdout:8/509: unlink d4/da/d51/d18/c1b 0 2024-07-03T15:43:55.740 INFO:tasks.workunit.client.1.smithi148.stdout:8/510: chown d4/da/d51/d18/d20/f66 573 1 2024-07-03T15:43:55.741 INFO:tasks.workunit.client.1.smithi148.stdout:9/851: mkdir d0/dd/d100/d106/d10a 0 2024-07-03T15:43:55.744 INFO:tasks.workunit.client.1.smithi148.stdout:7/684: dread d5/d7d/da7/d1c/d33/f67 [0,4194304] 0 2024-07-03T15:43:55.744 INFO:tasks.workunit.client.1.smithi148.stdout:7/685: truncate d5/d7d/da7/d1c/d4a/f62 1889332 0 2024-07-03T15:43:55.744 INFO:tasks.workunit.client.1.smithi148.stdout:2/653: rename d5/d5f/d79/f88 to d5/d14/d86/d93/fd0 0 2024-07-03T15:43:55.744 INFO:tasks.workunit.client.1.smithi148.stdout:2/654: dread - d5/d14/d86/db9/dbb/f98 zero size 2024-07-03T15:43:55.746 INFO:tasks.workunit.client.1.smithi148.stdout:1/538: creat d0/d2/d2e/d39/d42/daf/fb8 x:0 0 0 2024-07-03T15:43:55.746 INFO:tasks.workunit.client.1.smithi148.stdout:1/539: stat d0/d2/d6/d14/d29/d94 0 2024-07-03T15:43:55.750 INFO:tasks.workunit.client.1.smithi148.stdout:3/582: creat dd/d35/d41/d89/fb4 x:0 0 0 2024-07-03T15:43:55.752 INFO:tasks.workunit.client.1.smithi148.stdout:8/511: creat d4/da/d51/d18/d20/d82/fbd x:0 0 0 2024-07-03T15:43:55.755 INFO:tasks.workunit.client.1.smithi148.stdout:7/686: creat d5/d7d/da7/d1c/d33/d4c/d21/d47/d8b/dd3/fee x:0 0 0 2024-07-03T15:43:55.757 INFO:tasks.workunit.client.1.smithi148.stdout:6/659: rename d3/d1d/d28/d3e/lbd to d3/daa/ld5 0 2024-07-03T15:43:55.757 INFO:tasks.workunit.client.1.smithi148.stdout:6/660: truncate d3/f55 1830247 0 2024-07-03T15:43:55.757 INFO:tasks.workunit.client.1.smithi148.stdout:6/661: dread - d3/d1d/d28/d31/d4c/f54 zero size 2024-07-03T15:43:55.757 INFO:tasks.workunit.client.1.smithi148.stdout:6/662: dread - d3/f2f zero size 2024-07-03T15:43:55.758 INFO:tasks.workunit.client.1.smithi148.stdout:6/663: write d3/d5b/d74/dc4/fce [775592,66482] 0 2024-07-03T15:43:55.758 INFO:tasks.workunit.client.1.smithi148.stdout:6/664: chown d3/d1d/d28/d53/f59 82 1 2024-07-03T15:43:55.758 INFO:tasks.workunit.client.1.smithi148.stdout:2/655: unlink d5/d14/d86/db9/dbb/d13/d2d/la4 0 2024-07-03T15:43:55.768 INFO:tasks.workunit.client.1.smithi148.stdout:2/656: dread d5/d14/d86/db9/dbb/d68/dcd/f3d [0,4194304] 0 2024-07-03T15:43:55.768 INFO:tasks.workunit.client.1.smithi148.stdout:2/657: read - d5/d14/d86/db9/dbb/d13/d41/d7b/fc8 zero size 2024-07-03T15:43:55.768 INFO:tasks.workunit.client.1.smithi148.stdout:8/512: creat d4/da/d14/fbe x:0 0 0 2024-07-03T15:43:55.769 INFO:tasks.workunit.client.1.smithi148.stdout:7/687: dwrite d5/d7d/da7/d1c/d33/d4c/d84/d98/dc2/fc9 [0,4194304] 0 2024-07-03T15:43:55.778 INFO:tasks.workunit.client.1.smithi148.stdout:4/537: rename d0/d28/lb3 to d0/d8/d13/d67/lb7 0 2024-07-03T15:43:55.778 INFO:tasks.workunit.client.1.smithi148.stdout:4/538: chown d0/d28/d7d/d6f/d7c/f8d 15018218 1 2024-07-03T15:43:55.789 INFO:tasks.workunit.client.1.smithi148.stdout:7/688: link d5/d7d/da7/d1c/l1e d5/d7d/da7/d1c/d33/d4c/lef 0 2024-07-03T15:43:55.789 INFO:tasks.workunit.client.1.smithi148.stdout:7/689: chown d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/fce 1 1 2024-07-03T15:43:55.789 INFO:tasks.workunit.client.1.smithi148.stdout:7/690: chown d5/f9 10766774 1 2024-07-03T15:43:55.789 INFO:tasks.workunit.client.1.smithi148.stdout:7/691: write d5/d7d/da7/d1c/d59/f7c [4876025,47757] 0 2024-07-03T15:43:55.792 INFO:tasks.workunit.client.1.smithi148.stdout:9/852: rename d0/d18/d24/f91 to d0/dd/d25/d37/d40/da3/f10b 0 2024-07-03T15:43:55.795 INFO:tasks.workunit.client.1.smithi148.stdout:4/539: unlink d0/d91/da8/dab/d49/c70 0 2024-07-03T15:43:55.797 INFO:tasks.workunit.client.1.smithi148.stdout:6/665: mkdir d3/d1d/dd6 0 2024-07-03T15:43:55.797 INFO:tasks.workunit.client.1.smithi148.stdout:6/666: fdatasync d3/d1d/d28/d31/d4c/f54 0 2024-07-03T15:43:55.799 INFO:tasks.workunit.client.1.smithi148.stdout:8/513: unlink d4/da/d51/c13 0 2024-07-03T15:43:55.799 INFO:tasks.workunit.client.1.smithi148.stdout:8/514: write d4/da/d51/f57 [604833,115925] 0 2024-07-03T15:43:55.799 INFO:tasks.workunit.client.1.smithi148.stdout:8/515: write d4/f98 [57967,30229] 0 2024-07-03T15:43:55.800 INFO:tasks.workunit.client.1.smithi148.stdout:2/658: dwrite d5/f2f [0,4194304] 0 2024-07-03T15:43:55.802 INFO:tasks.workunit.client.1.smithi148.stdout:0/556: sync 2024-07-03T15:43:55.802 INFO:tasks.workunit.client.1.smithi148.stdout:7/692: rename d5/d7d/da7/db4/dec to d5/d7d/da7/db4/df0 0 2024-07-03T15:43:55.805 INFO:tasks.workunit.client.1.smithi148.stdout:3/583: rename dd/d33/d37/f46 to dd/d33/d37/d68/d2e/d40/d45/d73/fb5 0 2024-07-03T15:43:55.805 INFO:tasks.workunit.client.1.smithi148.stdout:3/584: dread - dd/d33/d37/d68/f9c zero size 2024-07-03T15:43:55.807 INFO:tasks.workunit.client.1.smithi148.stdout:9/853: mkdir d0/dd/d29/dff/d10c 0 2024-07-03T15:43:55.807 INFO:tasks.workunit.client.1.smithi148.stdout:9/854: creat d0/d49/d43/d19/d7a/f10d x:0 0 0 2024-07-03T15:43:55.809 INFO:tasks.workunit.client.1.smithi148.stdout:4/540: dwrite d0/d28/da7/d2f/f6c [0,4194304] 0 2024-07-03T15:43:55.809 INFO:tasks.workunit.client.1.smithi148.stdout:8/516: dread d4/da/d47/f94 [0,4194304] 0 2024-07-03T15:43:55.810 INFO:tasks.workunit.client.1.smithi148.stdout:6/667: creat d3/d1d/d28/d31/d61/fd7 x:0 0 0 2024-07-03T15:43:55.813 INFO:tasks.workunit.client.1.smithi148.stdout:2/659: mknod d5/d14/d86/db9/dbb/d13/d41/d7b/d6a/d80/cd1 0 2024-07-03T15:43:55.816 INFO:tasks.workunit.client.1.smithi148.stdout:0/557: truncate da/f28 1458431 0 2024-07-03T15:43:55.819 INFO:tasks.workunit.client.1.smithi148.stdout:3/585: mknod dd/da6/cb6 0 2024-07-03T15:43:55.819 INFO:tasks.workunit.client.1.smithi148.stdout:3/586: readlink dd/d33/d37/d68/d2e/d61/lb1 0 2024-07-03T15:43:55.819 INFO:tasks.workunit.client.1.smithi148.stdout:3/587: truncate dd/f62 550751 0 2024-07-03T15:43:55.823 INFO:tasks.workunit.client.1.smithi148.stdout:9/855: truncate d0/dd/d25/d37/d40/da3/f10b 1998931 0 2024-07-03T15:43:55.823 INFO:tasks.workunit.client.1.smithi148.stdout:9/856: creat d0/dd/d25/d59/f10e x:0 0 0 2024-07-03T15:43:55.825 INFO:tasks.workunit.client.1.smithi148.stdout:8/517: dwrite d4/da/d14/d1a/d32/fae [0,4194304] 0 2024-07-03T15:43:55.825 INFO:tasks.workunit.client.1.smithi148.stdout:8/518: fsync d4/da/d14/d1a/d32/d89/fa1 0 2024-07-03T15:43:55.825 INFO:tasks.workunit.client.1.smithi148.stdout:7/693: dwrite d5/d7d/da7/d1c/d33/d4c/d21/d39/f77 [0,4194304] 0 2024-07-03T15:43:55.825 INFO:tasks.workunit.client.1.smithi148.stdout:7/694: fdatasync d5/d7d/da7/d1c/d4a/f62 0 2024-07-03T15:43:55.826 INFO:tasks.workunit.client.1.smithi148.stdout:9/857: dread d0/dd/d25/d59/fd2 [0,4194304] 0 2024-07-03T15:43:55.826 INFO:tasks.workunit.client.1.smithi148.stdout:9/858: stat d0/d18/feb 0 2024-07-03T15:43:55.826 INFO:tasks.workunit.client.1.smithi148.stdout:9/859: write d0/d49/d43/d19/d7a/ffb [396953,4976] 0 2024-07-03T15:43:55.828 INFO:tasks.workunit.client.1.smithi148.stdout:4/541: mkdir d0/d8/d3a/db8 0 2024-07-03T15:43:55.828 INFO:tasks.workunit.client.1.smithi148.stdout:4/542: fdatasync d0/d8/d19/faa 0 2024-07-03T15:43:55.830 INFO:tasks.workunit.client.1.smithi148.stdout:6/668: symlink d3/d5b/d74/dc4/d85/ld8 0 2024-07-03T15:43:55.831 INFO:tasks.workunit.client.1.smithi148.stdout:6/669: readlink d3/l14 0 2024-07-03T15:43:55.834 INFO:tasks.workunit.client.1.smithi148.stdout:2/660: rename d5/d14/d86/db9/dbb/f98 to d5/d14/d22/fd2 0 2024-07-03T15:43:55.834 INFO:tasks.workunit.client.1.smithi148.stdout:8/519: read d4/da/d47/d88/d92/f42 [593121,71284] 0 2024-07-03T15:43:55.836 INFO:tasks.workunit.client.1.smithi148.stdout:0/558: write da/d44/d13/d14/d35/d8a/dab/f6c [9382498,64670] 0 2024-07-03T15:43:55.837 INFO:tasks.workunit.client.1.smithi148.stdout:3/588: mknod dd/cb7 0 2024-07-03T15:43:55.846 INFO:tasks.workunit.client.1.smithi148.stdout:4/543: dwrite d0/d8/d13/d48/f8f [0,4194304] 0 2024-07-03T15:43:55.846 INFO:tasks.workunit.client.1.smithi148.stdout:6/670: dwrite d3/d1d/d28/d31/fbc [0,4194304] 0 2024-07-03T15:43:55.857 INFO:tasks.workunit.client.1.smithi148.stdout:2/661: link d5/d14/d86/db9/dbb/d68/dcd/l12 d5/d5f/d7e/ld3 0 2024-07-03T15:43:55.858 INFO:tasks.workunit.client.1.smithi148.stdout:2/662: write d5/d14/f58 [5877588,114485] 0 2024-07-03T15:43:55.862 INFO:tasks.workunit.client.1.smithi148.stdout:0/559: creat da/d44/d13/fb8 x:0 0 0 2024-07-03T15:43:55.863 INFO:tasks.workunit.client.1.smithi148.stdout:3/589: symlink dd/d33/d37/lb8 0 2024-07-03T15:43:55.868 INFO:tasks.workunit.client.1.smithi148.stdout:4/544: mkdir d0/d8/d3a/db9 0 2024-07-03T15:43:55.874 INFO:tasks.workunit.client.1.smithi148.stdout:6/671: link d3/f17 d3/d1d/d28/d31/fd9 0 2024-07-03T15:43:55.876 INFO:tasks.workunit.client.1.smithi148.stdout:7/695: getdents d5/d7d/da7/d1c/d33/d4c/d21/d31 0 2024-07-03T15:43:55.876 INFO:tasks.workunit.client.1.smithi148.stdout:0/560: dwrite da/d44/d13/d14/f61 [0,4194304] 0 2024-07-03T15:43:55.877 INFO:tasks.workunit.client.1.smithi148.stdout:6/672: write d3/d1d/f38 [2224585,69724] 0 2024-07-03T15:43:55.878 INFO:tasks.workunit.client.1.smithi148.stdout:2/663: creat d5/d14/d86/db9/dbb/d68/dcd/d64/fd4 x:0 0 0 2024-07-03T15:43:55.879 INFO:tasks.workunit.client.1.smithi148.stdout:8/520: getdents d4 0 2024-07-03T15:43:55.881 INFO:tasks.workunit.client.1.smithi148.stdout:3/590: creat dd/da6/da2/fb9 x:0 0 0 2024-07-03T15:43:55.881 INFO:tasks.workunit.client.1.smithi148.stdout:3/591: chown dd/d33/d37/d68/d2e/d40/d45/d9a/f9d 1855147 1 2024-07-03T15:43:55.886 INFO:tasks.workunit.client.1.smithi148.stdout:4/545: mkdir d0/d28/d7d/d6f/d7c/dae/dba 0 2024-07-03T15:43:55.886 INFO:tasks.workunit.client.1.smithi148.stdout:7/696: mkdir d5/d7d/da7/d1c/d33/d4c/d84/d91/daa/db9/df1 0 2024-07-03T15:43:55.891 INFO:tasks.workunit.client.1.smithi148.stdout:2/664: mkdir d5/d14/d8b/dd5 0 2024-07-03T15:43:55.894 INFO:tasks.workunit.client.1.smithi148.stdout:8/521: unlink d4/da/d14/d3a/l80 0 2024-07-03T15:43:55.897 INFO:tasks.workunit.client.1.smithi148.stdout:3/592: mknod dd/d33/d37/d68/d2e/d40/d45/d9a/cba 0 2024-07-03T15:43:55.897 INFO:tasks.workunit.client.1.smithi148.stdout:3/593: chown dd/d33/d52/d57 96656 1 2024-07-03T15:43:55.897 INFO:tasks.workunit.client.1.smithi148.stdout:3/594: fdatasync dd/d33/d37/f53 0 2024-07-03T15:43:55.899 INFO:tasks.workunit.client.1.smithi148.stdout:6/673: getdents d3/d5b 0 2024-07-03T15:43:55.901 INFO:tasks.workunit.client.1.smithi148.stdout:2/665: mknod d5/d14/d8b/dd5/cd6 0 2024-07-03T15:43:55.904 INFO:tasks.workunit.client.1.smithi148.stdout:2/666: truncate d5/d14/d86/db9/dbb/d68/dcd/f38 1277895 0 2024-07-03T15:43:55.909 INFO:tasks.workunit.client.1.smithi148.stdout:2/667: link d5/d14/d22/d60/d76/fb0 d5/d14/d86/db9/dbb/d13/d2d/fd7 0 2024-07-03T15:43:55.909 INFO:tasks.workunit.client.1.smithi148.stdout:2/668: truncate d5/d14/d22/d60/f65 681111 0 2024-07-03T15:43:55.914 INFO:tasks.workunit.client.1.smithi148.stdout:8/522: dwrite d4/d86/f63 [0,4194304] 0 2024-07-03T15:43:55.918 INFO:tasks.workunit.client.1.smithi148.stdout:6/674: dread d3/d5b/f99 [0,4194304] 0 2024-07-03T15:43:55.918 INFO:tasks.workunit.client.1.smithi148.stdout:6/675: fsync d3/d50/d77/f94 0 2024-07-03T15:43:55.926 INFO:tasks.workunit.client.1.smithi148.stdout:3/595: dwrite dd/d33/d37/f53 [0,4194304] 0 2024-07-03T15:43:55.926 INFO:tasks.workunit.client.1.smithi148.stdout:3/596: dread - dd/d33/d37/d68/d2e/d40/d45/fa3 zero size 2024-07-03T15:43:55.927 INFO:tasks.workunit.client.1.smithi148.stdout:6/676: creat d3/d1d/d28/d31/d4c/fda x:0 0 0 2024-07-03T15:43:55.931 INFO:tasks.workunit.client.1.smithi148.stdout:3/597: mkdir dd/da6/da2/dbb 0 2024-07-03T15:43:55.931 INFO:tasks.workunit.client.1.smithi148.stdout:3/598: readlink dd/d33/d52/d57/l9b 0 2024-07-03T15:43:55.933 INFO:tasks.workunit.client.1.smithi148.stdout:6/677: mknod d3/d1d/d28/cdb 0 2024-07-03T15:43:55.933 INFO:tasks.workunit.client.1.smithi148.stdout:6/678: write d3/d1d/f27 [5119201,41783] 0 2024-07-03T15:43:55.934 INFO:tasks.workunit.client.1.smithi148.stdout:3/599: mkdir dd/d33/d37/d68/d2e/d40/d8c/dbc 0 2024-07-03T15:43:55.935 INFO:tasks.workunit.client.1.smithi148.stdout:6/679: chown d3/d1d/c26 13 1 2024-07-03T15:43:55.935 INFO:tasks.workunit.client.1.smithi148.stdout:2/669: dwrite d5/d14/d86/db9/dbb/d13/d41/f8c [0,4194304] 0 2024-07-03T15:43:55.937 INFO:tasks.workunit.client.1.smithi148.stdout:8/523: dwrite d4/da/d51/d18/d20/f21 [4194304,4194304] 0 2024-07-03T15:43:55.937 INFO:tasks.workunit.client.1.smithi148.stdout:8/524: fsync d4/da/d14/d1a/d32/d38/dba/f77 0 2024-07-03T15:43:55.937 INFO:tasks.workunit.client.1.smithi148.stdout:3/600: creat dd/d33/d37/d68/d2e/d40/d77/fbd x:0 0 0 2024-07-03T15:43:55.937 INFO:tasks.workunit.client.1.smithi148.stdout:3/601: write dd/f19 [1418677,37591] 0 2024-07-03T15:43:55.937 INFO:tasks.workunit.client.1.smithi148.stdout:3/602: chown dd/d33/d37/d68/d2e/f18 12940 1 2024-07-03T15:43:55.938 INFO:tasks.workunit.client.1.smithi148.stdout:3/603: write dd/f2b [2183314,89492] 0 2024-07-03T15:43:55.938 INFO:tasks.workunit.client.1.smithi148.stdout:3/604: chown dd/d33/d37/d68/c24 103612 1 2024-07-03T15:43:55.943 INFO:tasks.workunit.client.1.smithi148.stdout:2/670: creat d5/d14/d22/dc0/fd8 x:0 0 0 2024-07-03T15:43:55.948 INFO:tasks.workunit.client.1.smithi148.stdout:8/525: symlink d4/da/d14/d9e/lbf 0 2024-07-03T15:43:55.948 INFO:tasks.workunit.client.1.smithi148.stdout:8/526: chown d4/da/d14/d1a/d32/d38/dba/lbc 6 1 2024-07-03T15:43:55.953 INFO:tasks.workunit.client.1.smithi148.stdout:3/605: dwrite dd/f72 [4194304,4194304] 0 2024-07-03T15:43:55.954 INFO:tasks.workunit.client.1.smithi148.stdout:6/680: dwrite d3/d1d/d28/d3e/f7b [0,4194304] 0 2024-07-03T15:43:55.954 INFO:tasks.workunit.client.1.smithi148.stdout:6/681: write d3/d1d/d28/d31/d61/f71 [5135313,48831] 0 2024-07-03T15:43:55.954 INFO:tasks.workunit.client.1.smithi148.stdout:6/682: truncate d3/d1d/d28/d31/f98 796671 0 2024-07-03T15:43:55.954 INFO:tasks.workunit.client.1.smithi148.stdout:6/683: write d3/d1d/d28/f73 [4927349,100512] 0 2024-07-03T15:43:55.955 INFO:tasks.workunit.client.1.smithi148.stdout:6/684: chown d3/d1d/d28/f2e 71 1 2024-07-03T15:43:55.956 INFO:tasks.workunit.client.1.smithi148.stdout:8/527: creat d4/da/d14/d1a/d32/d38/dba/fc0 x:0 0 0 2024-07-03T15:43:55.956 INFO:tasks.workunit.client.1.smithi148.stdout:8/528: write d4/da/d14/d1a/d32/d38/d58/f8b [2188164,74798] 0 2024-07-03T15:43:55.956 INFO:tasks.workunit.client.1.smithi148.stdout:8/529: creat d4/da/d51/d18/d20/d82/fc1 x:0 0 0 2024-07-03T15:43:55.960 INFO:tasks.workunit.client.1.smithi148.stdout:3/606: write dd/d33/f6c [1327697,82428] 0 2024-07-03T15:43:55.963 INFO:tasks.workunit.client.1.smithi148.stdout:8/530: dread d4/da/d51/d18/d20/f66 [0,4194304] 0 2024-07-03T15:43:55.970 INFO:tasks.workunit.client.1.smithi148.stdout:3/607: rename dd/l29 to dd/d33/d52/d57/lbe 0 2024-07-03T15:43:55.974 INFO:tasks.workunit.client.1.smithi148.stdout:8/531: dread d4/da/d51/f7 [8388608,4194304] 0 2024-07-03T15:43:55.977 INFO:tasks.workunit.client.1.smithi148.stdout:6/685: unlink d3/d1d/l60 0 2024-07-03T15:43:55.978 INFO:tasks.workunit.client.1.smithi148.stdout:6/686: write d3/d1d/d28/d31/d4c/f54 [389122,17864] 0 2024-07-03T15:43:55.982 INFO:tasks.workunit.client.1.smithi148.stdout:8/532: creat d4/da/d51/d18/fc2 x:0 0 0 2024-07-03T15:43:55.985 INFO:tasks.workunit.client.1.smithi148.stdout:6/687: link d3/d1d/d28/l30 d3/d1d/d28/ldc 0 2024-07-03T15:43:55.986 INFO:tasks.workunit.client.1.smithi148.stdout:8/533: creat d4/da/d14/d1a/d32/d38/fc3 x:0 0 0 2024-07-03T15:43:55.987 INFO:tasks.workunit.client.1.smithi148.stdout:3/608: dwrite dd/d33/d37/d68/d2e/d40/d45/f78 [0,4194304] 0 2024-07-03T15:43:55.987 INFO:tasks.workunit.client.1.smithi148.stdout:3/609: chown dd/d33/d37/d68/d2e/d40/d77/d8f 12774723 1 2024-07-03T15:43:55.987 INFO:tasks.workunit.client.1.smithi148.stdout:3/610: chown dd/d35/d41 20049 1 2024-07-03T15:43:55.991 INFO:tasks.workunit.client.1.smithi148.stdout:8/534: truncate d4/da/d14/d17/f26 3941143 0 2024-07-03T15:43:55.995 INFO:tasks.workunit.client.1.smithi148.stdout:3/611: symlink dd/d33/d37/d68/d2e/d40/d77/d8f/lbf 0 2024-07-03T15:43:55.995 INFO:tasks.workunit.client.1.smithi148.stdout:5/623: sync 2024-07-03T15:43:55.996 INFO:tasks.workunit.client.1.smithi148.stdout:6/688: rename d3/d5b to d3/d1d/d28/d3e/ddd 0 2024-07-03T15:43:55.998 INFO:tasks.workunit.client.1.smithi148.stdout:3/612: link dd/d35/d81/l95 dd/d33/d37/d68/d2e/d40/d45/lc0 0 2024-07-03T15:43:56.000 INFO:tasks.workunit.client.1.smithi148.stdout:5/624: creat d7/da/df/d16/d1a/d58/fd8 x:0 0 0 2024-07-03T15:43:56.001 INFO:tasks.workunit.client.1.smithi148.stdout:8/535: rename d4/da/d14/d1a/d32/la2 to d4/da/d14/d1a/d32/d89/lc4 0 2024-07-03T15:43:56.002 INFO:tasks.workunit.client.1.smithi148.stdout:8/536: chown d4/da/d14/f6c 13881 1 2024-07-03T15:43:56.004 INFO:tasks.workunit.client.1.smithi148.stdout:3/613: link dd/d33/d37/d68/d2e/d40/d45/d9a/laa dd/d33/d37/d68/d2e/d40/d45/d65/lc1 0 2024-07-03T15:43:56.013 INFO:tasks.workunit.client.1.smithi148.stdout:5/625: rmdir d7/da/df/d16/dac/dc1/dc4 39 2024-07-03T15:43:56.013 INFO:tasks.workunit.client.1.smithi148.stdout:5/626: fsync d7/da/df/d1d/f51 0 2024-07-03T15:43:56.019 INFO:tasks.workunit.client.1.smithi148.stdout:6/689: dwrite d3/f17 [0,4194304] 0 2024-07-03T15:43:56.019 INFO:tasks.workunit.client.1.smithi148.stdout:6/690: write d3/d50/fcf [4856223,130163] 0 2024-07-03T15:43:56.019 INFO:tasks.workunit.client.1.smithi148.stdout:6/691: readlink d3/d1d/d28/d31/l65 0 2024-07-03T15:43:56.023 INFO:tasks.workunit.client.1.smithi148.stdout:8/537: dwrite d4/da/d14/d9e/f7d [0,4194304] 0 2024-07-03T15:43:56.023 INFO:tasks.workunit.client.1.smithi148.stdout:8/538: stat d4/da/d14/d17/c4e 0 2024-07-03T15:43:56.025 INFO:tasks.workunit.client.1.smithi148.stdout:5/627: dread d7/da/df/d16/d2f/f93 [0,4194304] 0 2024-07-03T15:43:56.025 INFO:tasks.workunit.client.1.smithi148.stdout:5/628: chown d7/da/df/d16/f7e 39022304 1 2024-07-03T15:43:56.025 INFO:tasks.workunit.client.1.smithi148.stdout:6/692: symlink d3/lde 0 2024-07-03T15:43:56.025 INFO:tasks.workunit.client.1.smithi148.stdout:6/693: chown d3/d1d/d28/d3e/cc3 85385 1 2024-07-03T15:43:56.025 INFO:tasks.workunit.client.1.smithi148.stdout:3/614: dwrite dd/d33/d37/d68/d2e/f11 [0,4194304] 0 2024-07-03T15:43:56.025 INFO:tasks.workunit.client.1.smithi148.stdout:3/615: write dd/d33/d37/d68/f9c [213207,22075] 0 2024-07-03T15:43:56.030 INFO:tasks.workunit.client.1.smithi148.stdout:8/539: rename d4/da/d47/d88/l49 to d4/da/d51/d53/lc5 0 2024-07-03T15:43:56.030 INFO:tasks.workunit.client.1.smithi148.stdout:8/540: fdatasync d4/da/d14/d1a/d32/d89/fa1 0 2024-07-03T15:43:56.031 INFO:tasks.workunit.client.1.smithi148.stdout:8/541: chown d4/da/d51/d18/c3f 205466818 1 2024-07-03T15:43:56.031 INFO:tasks.workunit.client.1.smithi148.stdout:8/542: truncate d4/da/d14/f9b 33210 0 2024-07-03T15:43:56.032 INFO:tasks.workunit.client.1.smithi148.stdout:3/616: dread dd/d33/d37/d68/d2e/d40/d45/d73/fb5 [0,4194304] 0 2024-07-03T15:43:56.033 INFO:tasks.workunit.client.1.smithi148.stdout:5/629: dread d7/da/df/f71 [4194304,4194304] 0 2024-07-03T15:43:56.036 INFO:tasks.workunit.client.1.smithi148.stdout:3/617: creat dd/d33/d37/d68/d2e/d40/d45/d73/fc2 x:0 0 0 2024-07-03T15:43:56.036 INFO:tasks.workunit.client.1.smithi148.stdout:3/618: fsync dd/d33/d52/f60 0 2024-07-03T15:43:56.036 INFO:tasks.workunit.client.1.smithi148.stdout:3/619: chown dd/f62 10609 1 2024-07-03T15:43:56.036 INFO:tasks.workunit.client.1.smithi148.stdout:3/620: write fa [5149800,112466] 0 2024-07-03T15:43:56.037 INFO:tasks.workunit.client.1.smithi148.stdout:5/630: creat d7/da/df/d16/d6a/db4/fd9 x:0 0 0 2024-07-03T15:43:56.040 INFO:tasks.workunit.client.1.smithi148.stdout:3/621: truncate dd/d33/d52/f60 3055494 0 2024-07-03T15:43:56.040 INFO:tasks.workunit.client.1.smithi148.stdout:3/622: chown dd/d33/d37/d68/d2e/d40/d77/d8f/lbf 715 1 2024-07-03T15:43:56.042 INFO:tasks.workunit.client.1.smithi148.stdout:5/631: creat d7/da/df/d16/d2f/dc2/fda x:0 0 0 2024-07-03T15:43:56.043 INFO:tasks.workunit.client.1.smithi148.stdout:5/632: readlink l1 0 2024-07-03T15:43:56.043 INFO:tasks.workunit.client.1.smithi148.stdout:5/633: write d7/da/fb5 [801557,3010] 0 2024-07-03T15:43:56.046 INFO:tasks.workunit.client.1.smithi148.stdout:5/634: symlink d7/da/df/da0/dd1/ldb 0 2024-07-03T15:43:56.047 INFO:tasks.workunit.client.1.smithi148.stdout:6/694: dwrite d3/d1d/d28/d31/d4c/f78 [0,4194304] 0 2024-07-03T15:43:56.048 INFO:tasks.workunit.client.1.smithi148.stdout:8/543: dwrite f2 [4194304,4194304] 0 2024-07-03T15:43:56.048 INFO:tasks.workunit.client.1.smithi148.stdout:3/623: rmdir dd/d35/d41/d89 39 2024-07-03T15:43:56.049 INFO:tasks.workunit.client.1.smithi148.stdout:3/624: dread dd/d33/d37/d68/d2e/d61/f63 [0,4194304] 0 2024-07-03T15:43:56.049 INFO:tasks.workunit.client.1.smithi148.stdout:5/635: mknod d7/da/d79/cdc 0 2024-07-03T15:43:56.052 INFO:tasks.workunit.client.1.smithi148.stdout:6/695: truncate d3/d1d/d28/d3e/f7b 4081399 0 2024-07-03T15:43:56.054 INFO:tasks.workunit.client.1.smithi148.stdout:8/544: truncate d4/da/d14/d1a/d32/d38/dba/f77 1322300 0 2024-07-03T15:43:56.054 INFO:tasks.workunit.client.1.smithi148.stdout:1/540: sync 2024-07-03T15:43:56.055 INFO:tasks.workunit.client.1.smithi148.stdout:1/541: dread d0/d2/d6/d14/f8e [0,4194304] 0 2024-07-03T15:43:56.056 INFO:tasks.workunit.client.1.smithi148.stdout:3/625: mkdir dd/d33/d37/d68/d2e/dc3 0 2024-07-03T15:43:56.056 INFO:tasks.workunit.client.1.smithi148.stdout:3/626: write dd/d33/d37/d68/d2e/d40/d45/fa3 [234274,113981] 0 2024-07-03T15:43:56.056 INFO:tasks.workunit.client.1.smithi148.stdout:3/627: fdatasync dd/d35/f94 0 2024-07-03T15:43:56.057 INFO:tasks.workunit.client.1.smithi148.stdout:3/628: chown dd/d33/d37/d68/d2e/d40/d8c/dbc 791340 1 2024-07-03T15:43:56.057 INFO:tasks.workunit.client.1.smithi148.stdout:3/629: chown dd/d33/d37/d68/f30 9084616 1 2024-07-03T15:43:56.058 INFO:tasks.workunit.client.1.smithi148.stdout:5/636: rename d7/da/df/d1d/fb0 to d7/da/df/d16/dac/dc1/dc4/fdd 0 2024-07-03T15:43:56.059 INFO:tasks.workunit.client.1.smithi148.stdout:6/696: link d3/d1d/d28/d3e/ddd/d74/dc4/la4 d3/d1d/d28/d3e/ddd/d74/dc4/d85/ldf 0 2024-07-03T15:43:56.075 INFO:tasks.workunit.client.1.smithi148.stdout:8/545: unlink d4/da/d51/l70 0 2024-07-03T15:43:56.075 INFO:tasks.workunit.client.1.smithi148.stdout:8/546: read - d4/da/d14/d1a/d32/d38/dba/f95 zero size 2024-07-03T15:43:56.076 INFO:tasks.workunit.client.1.smithi148.stdout:1/542: symlink d0/d2/d11/d3a/lb9 0 2024-07-03T15:43:56.076 INFO:tasks.workunit.client.1.smithi148.stdout:1/543: stat d0/f45 0 2024-07-03T15:43:56.076 INFO:tasks.workunit.client.1.smithi148.stdout:1/544: write d0/d2/d6/d14/f58 [83844,41596] 0 2024-07-03T15:43:56.077 INFO:tasks.workunit.client.1.smithi148.stdout:3/630: getdents dd/d33/d37/d68/d2e/d40/d45/d65 0 2024-07-03T15:43:56.078 INFO:tasks.workunit.client.1.smithi148.stdout:3/631: chown dd/da6/db3 118 1 2024-07-03T15:43:56.078 INFO:tasks.workunit.client.1.smithi148.stdout:3/632: chown dd/d35/dac 59 1 2024-07-03T15:43:56.079 INFO:tasks.workunit.client.1.smithi148.stdout:5/637: creat d7/da/df/d16/d2f/fde x:0 0 0 2024-07-03T15:43:56.079 INFO:tasks.workunit.client.1.smithi148.stdout:5/638: write d7/da/df/fcb [1722730,18398] 0 2024-07-03T15:43:56.079 INFO:tasks.workunit.client.1.smithi148.stdout:5/639: write d7/da/df/d16/d1a/f50 [4640226,70363] 0 2024-07-03T15:43:56.079 INFO:tasks.workunit.client.1.smithi148.stdout:5/640: stat d7/d13/d23/c4a 0 2024-07-03T15:43:56.082 INFO:tasks.workunit.client.1.smithi148.stdout:8/547: truncate d4/da/d14/f6f 546227 0 2024-07-03T15:43:56.084 INFO:tasks.workunit.client.1.smithi148.stdout:1/545: symlink d0/de/d2f/d72/lba 0 2024-07-03T15:43:56.087 INFO:tasks.workunit.client.1.smithi148.stdout:3/633: truncate dd/d35/d81/f91 1575752 0 2024-07-03T15:43:56.096 INFO:tasks.workunit.client.1.smithi148.stdout:5/641: dwrite d7/f19 [4194304,4194304] 0 2024-07-03T15:43:56.097 INFO:tasks.workunit.client.1.smithi148.stdout:1/546: mkdir d0/d2/d2e/d39/d42/db7/dbb 0 2024-07-03T15:43:56.097 INFO:tasks.workunit.client.1.smithi148.stdout:1/547: write d0/d2/d2e/d39/d41/f7a [2042509,83194] 0 2024-07-03T15:43:56.097 INFO:tasks.workunit.client.1.smithi148.stdout:1/548: write d0/d2/d2e/d39/d42/d62/f71 [970171,44331] 0 2024-07-03T15:43:56.098 INFO:tasks.workunit.client.1.smithi148.stdout:1/549: read d0/d2/d2e/d39/d42/f87 [299098,110134] 0 2024-07-03T15:43:56.099 INFO:tasks.workunit.client.1.smithi148.stdout:3/634: mkdir dd/d33/d37/dc4 0 2024-07-03T15:43:56.102 INFO:tasks.workunit.client.1.smithi148.stdout:8/548: rename d4/da/d47/d88/d92/l72 to d4/da/d14/d1a/lc6 0 2024-07-03T15:43:56.107 INFO:tasks.workunit.client.1.smithi148.stdout:1/550: mknod d0/d2/d11/d3a/cbc 0 2024-07-03T15:43:56.109 INFO:tasks.workunit.client.1.smithi148.stdout:3/635: symlink dd/d33/d37/d8d/lc5 0 2024-07-03T15:43:56.110 INFO:tasks.workunit.client.1.smithi148.stdout:5/642: dwrite d7/da/df/d16/d2f/dc2/fda [0,4194304] 0 2024-07-03T15:43:56.112 INFO:tasks.workunit.client.1.smithi148.stdout:1/551: unlink d0/de/d44/d54/l61 0 2024-07-03T15:43:56.114 INFO:tasks.workunit.client.1.smithi148.stdout:3/636: mknod dd/d35/d41/cc6 0 2024-07-03T15:43:56.117 INFO:tasks.workunit.client.1.smithi148.stdout:3/637: dread fa [4194304,4194304] 0 2024-07-03T15:43:56.118 INFO:tasks.workunit.client.1.smithi148.stdout:8/549: dwrite d4/da/d14/d1a/d32/d38/d58/fad [0,4194304] 0 2024-07-03T15:43:56.118 INFO:tasks.workunit.client.1.smithi148.stdout:8/550: chown d4/da/d14/d1a/d32/c8a 4053550 1 2024-07-03T15:43:56.121 INFO:tasks.workunit.client.1.smithi148.stdout:5/643: symlink d7/da/df/d16/d2f/d44/ldf 0 2024-07-03T15:43:56.122 INFO:tasks.workunit.client.1.smithi148.stdout:9/860: sync 2024-07-03T15:43:56.128 INFO:tasks.workunit.client.1.smithi148.stdout:8/551: symlink d4/da/d51/d18/d20/d82/lc7 0 2024-07-03T15:43:56.130 INFO:tasks.workunit.client.1.smithi148.stdout:3/638: dwrite dd/d33/d37/d68/f30 [0,4194304] 0 2024-07-03T15:43:56.131 INFO:tasks.workunit.client.1.smithi148.stdout:5/644: creat d7/da/df/d16/dac/dc1/fe0 x:0 0 0 2024-07-03T15:43:56.131 INFO:tasks.workunit.client.1.smithi148.stdout:9/861: write d0/dd/f14 [4875571,18963] 0 2024-07-03T15:43:56.132 INFO:tasks.workunit.client.1.smithi148.stdout:1/552: mknod d0/de/cbd 0 2024-07-03T15:43:56.135 INFO:tasks.workunit.client.1.smithi148.stdout:8/552: creat d4/da/d14/d17/fc8 x:0 0 0 2024-07-03T15:43:56.135 INFO:tasks.workunit.client.1.smithi148.stdout:8/553: chown d4/da/d14/f4d 13735 1 2024-07-03T15:43:56.137 INFO:tasks.workunit.client.1.smithi148.stdout:5/645: mknod d7/da/df/d16/d2f/d44/ce1 0 2024-07-03T15:43:56.137 INFO:tasks.workunit.client.1.smithi148.stdout:5/646: fsync d7/da/f74 0 2024-07-03T15:43:56.139 INFO:tasks.workunit.client.1.smithi148.stdout:9/862: creat d0/d49/d43/dd8/ddf/f10f x:0 0 0 2024-07-03T15:43:56.139 INFO:tasks.workunit.client.1.smithi148.stdout:9/863: readlink d0/dd/d25/d37/d40/d72/db8/lbe 0 2024-07-03T15:43:56.139 INFO:tasks.workunit.client.1.smithi148.stdout:9/864: chown d0/fce 3311634 1 2024-07-03T15:43:56.140 INFO:tasks.workunit.client.1.smithi148.stdout:7/697: sync 2024-07-03T15:43:56.140 INFO:tasks.workunit.client.1.smithi148.stdout:2/671: sync 2024-07-03T15:43:56.140 INFO:tasks.workunit.client.1.smithi148.stdout:4/546: sync 2024-07-03T15:43:56.140 INFO:tasks.workunit.client.1.smithi148.stdout:0/561: sync 2024-07-03T15:43:56.144 INFO:tasks.workunit.client.1.smithi148.stdout:1/553: link d0/de/d89/f95 d0/d2/d11/fbe 0 2024-07-03T15:43:56.144 INFO:tasks.workunit.client.1.smithi148.stdout:1/554: chown d0/de/d2f/d72/l79 23811 1 2024-07-03T15:43:56.148 INFO:tasks.workunit.client.1.smithi148.stdout:5/647: rename d7/d13/d54/l29 to d7/da/df/da0/db2/le2 0 2024-07-03T15:43:56.149 INFO:tasks.workunit.client.1.smithi148.stdout:9/865: dread d0/dd/d11/f12 [4194304,4194304] 0 2024-07-03T15:43:56.150 INFO:tasks.workunit.client.1.smithi148.stdout:7/698: mknod d5/d7d/da7/dc7/cf2 0 2024-07-03T15:43:56.153 INFO:tasks.workunit.client.1.smithi148.stdout:1/555: dread d0/de/d44/d54/d5d/f50 [0,4194304] 0 2024-07-03T15:43:56.153 INFO:tasks.workunit.client.1.smithi148.stdout:8/554: dwrite d4/da/d14/f4d [4194304,4194304] 0 2024-07-03T15:43:56.153 INFO:tasks.workunit.client.1.smithi148.stdout:8/555: stat d4/da/d51/d53/f7f 0 2024-07-03T15:43:56.153 INFO:tasks.workunit.client.1.smithi148.stdout:1/556: write d0/de/d89/f99 [593093,41241] 0 2024-07-03T15:43:56.153 INFO:tasks.workunit.client.1.smithi148.stdout:2/672: dread d5/d14/d86/fbc [4194304,4194304] 0 2024-07-03T15:43:56.153 INFO:tasks.workunit.client.1.smithi148.stdout:2/673: truncate d5/d14/d86/db9/dbb/d13/d1d/da1/fa2 383189 0 2024-07-03T15:43:56.155 INFO:tasks.workunit.client.1.smithi148.stdout:5/648: dread d7/da/fb [0,4194304] 0 2024-07-03T15:43:56.160 INFO:tasks.workunit.client.1.smithi148.stdout:4/547: rename d0/d8/d19/l34 to d0/d28/da7/d2f/d41/lbb 0 2024-07-03T15:43:56.162 INFO:tasks.workunit.client.1.smithi148.stdout:7/699: symlink d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/lf3 0 2024-07-03T15:43:56.168 INFO:tasks.workunit.client.1.smithi148.stdout:9/866: dwrite d0/d2f/ffa [0,4194304] 0 2024-07-03T15:43:56.168 INFO:tasks.workunit.client.1.smithi148.stdout:9/867: fsync d0/dd/d11/db4/ff7 0 2024-07-03T15:43:56.170 INFO:tasks.workunit.client.1.smithi148.stdout:1/557: creat d0/de/d2f/fbf x:0 0 0 2024-07-03T15:43:56.173 INFO:tasks.workunit.client.1.smithi148.stdout:4/548: creat d0/d8/d13/db4/fbc x:0 0 0 2024-07-03T15:43:56.176 INFO:tasks.workunit.client.1.smithi148.stdout:7/700: rename d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/db6/lb8 to d5/d7d/da7/d1c/d33/d4c/d84/d91/daa/lf4 0 2024-07-03T15:43:56.176 INFO:tasks.workunit.client.1.smithi148.stdout:7/701: readlink d5/d7d/da7/d1c/d33/d4c/d84/l8d 0 2024-07-03T15:43:56.177 INFO:tasks.workunit.client.1.smithi148.stdout:8/556: mknod d4/da/d51/d18/cc9 0 2024-07-03T15:43:56.177 INFO:tasks.workunit.client.1.smithi148.stdout:8/557: chown d4/da/d47/d88/d92/c5f 185 1 2024-07-03T15:43:56.181 INFO:tasks.workunit.client.1.smithi148.stdout:5/649: dwrite d7/da/df/d16/d2f/d44/d52/f55 [4194304,4194304] 0 2024-07-03T15:43:56.181 INFO:tasks.workunit.client.1.smithi148.stdout:5/650: stat d7/da/df/d16/dac/dc6/cce 0 2024-07-03T15:43:56.182 INFO:tasks.workunit.client.1.smithi148.stdout:2/674: dwrite d5/d14/d86/db9/dbb/d13/d41/f74 [0,4194304] 0 2024-07-03T15:43:56.182 INFO:tasks.workunit.client.1.smithi148.stdout:2/675: write d5/f2f [4621032,92191] 0 2024-07-03T15:43:56.185 INFO:tasks.workunit.client.1.smithi148.stdout:8/558: dread d4/da/d51/d18/d20/f66 [0,4194304] 0 2024-07-03T15:43:56.186 INFO:tasks.workunit.client.1.smithi148.stdout:1/558: mkdir d0/d2/d11/d1e/d9d/dc0 0 2024-07-03T15:43:56.188 INFO:tasks.workunit.client.1.smithi148.stdout:5/651: dread d7/da/df/f10 [0,4194304] 0 2024-07-03T15:43:56.189 INFO:tasks.workunit.client.1.smithi148.stdout:7/702: symlink d5/d7d/da7/d1c/d33/d4c/d21/d31/lf5 0 2024-07-03T15:43:56.189 INFO:tasks.workunit.client.1.smithi148.stdout:7/703: truncate d5/f9 4598190 0 2024-07-03T15:43:56.189 INFO:tasks.workunit.client.1.smithi148.stdout:7/704: readlink d5/d7d/da7/d1c/d33/d4c/d21/d31/lf5 0 2024-07-03T15:43:56.195 INFO:tasks.workunit.client.1.smithi148.stdout:7/705: read d5/d7d/da7/f37 [1113932,108428] 0 2024-07-03T15:43:56.195 INFO:tasks.workunit.client.1.smithi148.stdout:7/706: fdatasync d5/d7d/da7/f37 0 2024-07-03T15:43:56.206 INFO:tasks.workunit.client.1.smithi148.stdout:9/868: rmdir d0/d49/d43 39 2024-07-03T15:43:56.206 INFO:tasks.workunit.client.1.smithi148.stdout:9/869: fdatasync d0/dd/d25/d37/d40/d72/db8/f107 0 2024-07-03T15:43:56.206 INFO:tasks.workunit.client.1.smithi148.stdout:9/870: chown d0/d63/f98 4 1 2024-07-03T15:43:56.206 INFO:tasks.workunit.client.1.smithi148.stdout:9/871: chown d0/dd/cd5 429957749 1 2024-07-03T15:43:56.206 INFO:tasks.workunit.client.1.smithi148.stdout:9/872: fdatasync d0/d2f/f74 0 2024-07-03T15:43:56.211 INFO:tasks.workunit.client.1.smithi148.stdout:1/559: link d0/d2/d11/d1e/f9f d0/de/d44/d54/d5d/fc1 0 2024-07-03T15:43:56.212 INFO:tasks.workunit.client.1.smithi148.stdout:4/549: unlink d0/d8/d13/c26 0 2024-07-03T15:43:56.214 INFO:tasks.workunit.client.1.smithi148.stdout:5/652: unlink d7/da/df/d16/d2f/d44/d52/d86/fc8 0 2024-07-03T15:43:56.216 INFO:tasks.workunit.client.1.smithi148.stdout:7/707: dwrite d5/d7d/da7/d1c/d33/f48 [0,4194304] 0 2024-07-03T15:43:56.216 INFO:tasks.workunit.client.1.smithi148.stdout:9/873: dread d0/d18/f21 [4194304,4194304] 0 2024-07-03T15:43:56.217 INFO:tasks.workunit.client.1.smithi148.stdout:8/559: getdents d4/da/d14/d1a/d32/d38 0 2024-07-03T15:43:56.219 INFO:tasks.workunit.client.1.smithi148.stdout:1/560: mkdir d0/d2/d11/d1e/d9d/dc2 0 2024-07-03T15:43:56.220 INFO:tasks.workunit.client.1.smithi148.stdout:4/550: mkdir d0/d28/da7/dbd 0 2024-07-03T15:43:56.228 INFO:tasks.workunit.client.1.smithi148.stdout:1/561: dread d0/d2/d2e/d39/d42/d62/f6e [0,4194304] 0 2024-07-03T15:43:56.228 INFO:tasks.workunit.client.1.smithi148.stdout:1/562: readlink d0/de/d2f/d72/l79 0 2024-07-03T15:43:56.231 INFO:tasks.workunit.client.1.smithi148.stdout:7/708: truncate d5/d7d/da7/d1c/d33/d4c/d21/d47/f5d 645160 0 2024-07-03T15:43:56.239 INFO:tasks.workunit.client.1.smithi148.stdout:1/563: getdents d0/d2/d2e/d39/d42/daf 0 2024-07-03T15:43:56.240 INFO:tasks.workunit.client.1.smithi148.stdout:1/564: read d0/de/d44/d54/d5d/f8b [171721,29309] 0 2024-07-03T15:43:56.243 INFO:tasks.workunit.client.1.smithi148.stdout:1/565: rename d0/d2/f10 to d0/d2/d2e/d39/d42/daf/fc3 0 2024-07-03T15:43:56.244 INFO:tasks.workunit.client.1.smithi148.stdout:9/874: dwrite d0/d49/d43/dd8/ddf/f10f [0,4194304] 0 2024-07-03T15:43:56.245 INFO:tasks.workunit.client.1.smithi148.stdout:1/566: rename d0/d2/d6/d14/d29/c8f to d0/d2/d2e/d39/d42/db7/cc4 0 2024-07-03T15:43:56.253 INFO:tasks.workunit.client.1.smithi148.stdout:9/875: mkdir d0/dd/d25/d37/d40/da3/df0/d110 0 2024-07-03T15:43:56.253 INFO:tasks.workunit.client.1.smithi148.stdout:9/876: chown d0/d18/d24/c4a 63927672 1 2024-07-03T15:43:56.253 INFO:tasks.workunit.client.1.smithi148.stdout:9/877: readlink d0/d18/d24/l33 0 2024-07-03T15:43:56.256 INFO:tasks.workunit.client.1.smithi148.stdout:9/878: rmdir d0/dd/d29/dff/d10c 0 2024-07-03T15:43:56.258 INFO:tasks.workunit.client.1.smithi148.stdout:1/567: dwrite d0/d2/d2e/d39/d42/f49 [4194304,4194304] 0 2024-07-03T15:43:56.261 INFO:tasks.workunit.client.1.smithi148.stdout:9/879: rename d0/dd/f14 to d0/dd/d25/d59/dcb/f111 0 2024-07-03T15:43:56.261 INFO:tasks.workunit.client.1.smithi148.stdout:9/880: creat d0/dd/d25/d37/f112 x:0 0 0 2024-07-03T15:43:56.263 INFO:tasks.workunit.client.1.smithi148.stdout:9/881: dread d0/d49/d43/d19/d7a/f9b [0,4194304] 0 2024-07-03T15:43:56.265 INFO:tasks.workunit.client.1.smithi148.stdout:1/568: mkdir d0/d2/d11/d3a/dc5 0 2024-07-03T15:43:56.265 INFO:tasks.workunit.client.1.smithi148.stdout:1/569: write d0/d2/d2e/d39/f5f [769059,40900] 0 2024-07-03T15:43:56.266 INFO:tasks.workunit.client.1.smithi148.stdout:6/697: sync 2024-07-03T15:43:56.270 INFO:tasks.workunit.client.1.smithi148.stdout:9/882: mknod d0/dd/d25/d37/d40/d72/c113 0 2024-07-03T15:43:56.273 INFO:tasks.workunit.client.1.smithi148.stdout:6/698: mkdir d3/d88/de0 0 2024-07-03T15:43:56.275 INFO:tasks.workunit.client.1.smithi148.stdout:9/883: creat d0/d49/d43/d19/f114 x:0 0 0 2024-07-03T15:43:56.275 INFO:tasks.workunit.client.1.smithi148.stdout:1/570: dread d0/d2/d2e/d39/d41/f43 [4194304,4194304] 0 2024-07-03T15:43:56.276 INFO:tasks.workunit.client.1.smithi148.stdout:6/699: creat d3/d1d/d28/d3e/ddd/d74/dc4/fe1 x:0 0 0 2024-07-03T15:43:56.278 INFO:tasks.workunit.client.1.smithi148.stdout:9/884: creat d0/dd/d25/d37/d40/da3/f115 x:0 0 0 2024-07-03T15:43:56.285 INFO:tasks.workunit.client.1.smithi148.stdout:2/676: dwrite d5/d14/d22/d60/f65 [0,4194304] 0 2024-07-03T15:43:56.287 INFO:tasks.workunit.client.1.smithi148.stdout:9/885: rmdir d0/d18/d24/d47/d4d/dae 39 2024-07-03T15:43:56.295 INFO:tasks.workunit.client.1.smithi148.stdout:2/677: link d5/d14/d86/db9/dbb/d13/d41/d7b/l33 d5/d5f/ld9 0 2024-07-03T15:43:56.297 INFO:tasks.workunit.client.1.smithi148.stdout:2/678: read d5/d14/d86/db9/dbb/d13/d41/f49 [810920,48132] 0 2024-07-03T15:43:56.298 INFO:tasks.workunit.client.1.smithi148.stdout:3/639: sync 2024-07-03T15:43:56.298 INFO:tasks.workunit.client.1.smithi148.stdout:6/700: dwrite d3/d1d/d28/d31/d61/f6b [0,4194304] 0 2024-07-03T15:43:56.298 INFO:tasks.workunit.client.1.smithi148.stdout:6/701: stat d3/d1d/d28/d3e/ddd/d74/dc4/d85/d8c/fd0 0 2024-07-03T15:43:56.298 INFO:tasks.workunit.client.1.smithi148.stdout:9/886: unlink d0/dd/d29/f5e 0 2024-07-03T15:43:56.300 INFO:tasks.workunit.client.1.smithi148.stdout:4/551: dwrite d0/d8/d3a/f7f [0,4194304] 0 2024-07-03T15:43:56.303 INFO:tasks.workunit.client.1.smithi148.stdout:3/640: creat dd/d33/d37/d68/d67/fc7 x:0 0 0 2024-07-03T15:43:56.305 INFO:tasks.workunit.client.1.smithi148.stdout:6/702: dread d3/d1d/d28/d31/d61/f7a [0,4194304] 0 2024-07-03T15:43:56.305 INFO:tasks.workunit.client.1.smithi148.stdout:7/709: dread d5/d7d/da7/d1c/d59/f7c [0,4194304] 0 2024-07-03T15:43:56.307 INFO:tasks.workunit.client.1.smithi148.stdout:2/679: dread d5/d14/d86/db9/dbb/d13/d1d/f43 [0,4194304] 0 2024-07-03T15:43:56.308 INFO:tasks.workunit.client.1.smithi148.stdout:9/887: write d0/fbb [8103036,22399] 0 2024-07-03T15:43:56.309 INFO:tasks.workunit.client.1.smithi148.stdout:4/552: symlink d0/d28/da7/d2f/d52/lbe 0 2024-07-03T15:43:56.309 INFO:tasks.workunit.client.1.smithi148.stdout:4/553: creat d0/fbf x:0 0 0 2024-07-03T15:43:56.309 INFO:tasks.workunit.client.1.smithi148.stdout:8/560: dwrite d4/da/d14/d1a/d32/d38/dba/f95 [0,4194304] 0 2024-07-03T15:43:56.312 INFO:tasks.workunit.client.1.smithi148.stdout:3/641: rename dd/da6/cb6 to dd/d33/d37/d68/d2e/dc3/cc8 0 2024-07-03T15:43:56.313 INFO:tasks.workunit.client.1.smithi148.stdout:3/642: chown dd/da6/d8b 44 1 2024-07-03T15:43:56.313 INFO:tasks.workunit.client.1.smithi148.stdout:1/571: dwrite d0/d2/d2e/d39/d42/d62/f4f [0,4194304] 0 2024-07-03T15:43:56.317 INFO:tasks.workunit.client.1.smithi148.stdout:6/703: creat d3/d1d/d28/d31/fe2 x:0 0 0 2024-07-03T15:43:56.317 INFO:tasks.workunit.client.1.smithi148.stdout:9/888: write d0/dd/d11/f35 [579873,39855] 0 2024-07-03T15:43:56.317 INFO:tasks.workunit.client.1.smithi148.stdout:9/889: creat d0/d49/d43/d19/f116 x:0 0 0 2024-07-03T15:43:56.318 INFO:tasks.workunit.client.1.smithi148.stdout:2/680: rmdir d5/d14/d8b/dca 0 2024-07-03T15:43:56.321 INFO:tasks.workunit.client.1.smithi148.stdout:4/554: creat d0/d91/da8/dab/fc0 x:0 0 0 2024-07-03T15:43:56.322 INFO:tasks.workunit.client.1.smithi148.stdout:1/572: dread d0/d2/d2e/d39/d42/daf/fc3 [0,4194304] 0 2024-07-03T15:43:56.322 INFO:tasks.workunit.client.1.smithi148.stdout:4/555: dread d0/d8/d3a/f66 [0,4194304] 0 2024-07-03T15:43:56.330 INFO:tasks.workunit.client.1.smithi148.stdout:7/710: rename d5/d7d/da7/d1c/d33/fe6 to d5/d7d/da7/ff6 0 2024-07-03T15:43:56.332 INFO:tasks.workunit.client.1.smithi148.stdout:7/711: dread d5/d7d/da7/d1c/d4a/d66/f9b [0,4194304] 0 2024-07-03T15:43:56.332 INFO:tasks.workunit.client.1.smithi148.stdout:3/643: mkdir dd/d33/d37/d68/d2e/d40/d77/dc9 0 2024-07-03T15:43:56.332 INFO:tasks.workunit.client.1.smithi148.stdout:3/644: fdatasync dd/d33/d37/d68/d2e/d40/f80 0 2024-07-03T15:43:56.334 INFO:tasks.workunit.client.1.smithi148.stdout:4/556: read d0/d8/d13/d48/f8f [422645,85336] 0 2024-07-03T15:43:56.335 INFO:tasks.workunit.client.1.smithi148.stdout:1/573: dwrite d0/d2/d2e/fa7 [8388608,4194304] 0 2024-07-03T15:43:56.343 INFO:tasks.workunit.client.1.smithi148.stdout:6/704: mknod d3/d1d/d28/d31/d61/dac/ce3 0 2024-07-03T15:43:56.344 INFO:tasks.workunit.client.1.smithi148.stdout:6/705: stat d3/d1d/f21 0 2024-07-03T15:43:56.344 INFO:tasks.workunit.client.1.smithi148.stdout:6/706: write d3/d1d/d28/d31/d4c/fda [542430,20561] 0 2024-07-03T15:43:56.347 INFO:tasks.workunit.client.1.smithi148.stdout:4/557: dread d0/d8/d13/d67/f86 [0,4194304] 0 2024-07-03T15:43:56.349 INFO:tasks.workunit.client.1.smithi148.stdout:9/890: symlink d0/d18/d24/d5a/l117 0 2024-07-03T15:43:56.349 INFO:tasks.workunit.client.1.smithi148.stdout:9/891: chown d0/d49/la0 80085481 1 2024-07-03T15:43:56.350 INFO:tasks.workunit.client.1.smithi148.stdout:2/681: getdents d5/d14/d86/db9/dbb/d13/d41/d7b/d6a/d80 0 2024-07-03T15:43:56.351 INFO:tasks.workunit.client.1.smithi148.stdout:2/682: write d5/d14/d86/db9/dbb/d13/d2d/f3b [4486608,76097] 0 2024-07-03T15:43:56.351 INFO:tasks.workunit.client.1.smithi148.stdout:2/683: creat d5/d14/d22/dc0/fda x:0 0 0 2024-07-03T15:43:56.352 INFO:tasks.workunit.client.1.smithi148.stdout:8/561: rmdir d4/da/d47/d88/d92 39 2024-07-03T15:43:56.353 INFO:tasks.workunit.client.1.smithi148.stdout:3/645: dwrite dd/d33/d37/d68/d2e/d40/f5d [0,4194304] 0 2024-07-03T15:43:56.354 INFO:tasks.workunit.client.1.smithi148.stdout:1/574: creat d0/de/d2f/d72/fc6 x:0 0 0 2024-07-03T15:43:56.356 INFO:tasks.workunit.client.1.smithi148.stdout:6/707: symlink d3/d1d/d28/d3e/ddd/d74/dc4/d85/da2/le4 0 2024-07-03T15:43:56.359 INFO:tasks.workunit.client.1.smithi148.stdout:9/892: write d0/d2f/ffa [3474623,39228] 0 2024-07-03T15:43:56.361 INFO:tasks.workunit.client.1.smithi148.stdout:2/684: mkdir d5/d14/d22/d60/d89/ddb 0 2024-07-03T15:43:56.363 INFO:tasks.workunit.client.1.smithi148.stdout:7/712: dwrite d5/d7d/da7/d1c/d33/d4c/d21/d47/d8b/dd3/fee [0,4194304] 0 2024-07-03T15:43:56.363 INFO:tasks.workunit.client.1.smithi148.stdout:4/558: dwrite d0/d8/d13/db4/fbc [0,4194304] 0 2024-07-03T15:43:56.363 INFO:tasks.workunit.client.1.smithi148.stdout:4/559: chown d0/d28 60 1 2024-07-03T15:43:56.369 INFO:tasks.workunit.client.1.smithi148.stdout:6/708: truncate d3/fb7 3466663 0 2024-07-03T15:43:56.369 INFO:tasks.workunit.client.1.smithi148.stdout:6/709: readlink d3/l12 0 2024-07-03T15:43:56.373 INFO:tasks.workunit.client.1.smithi148.stdout:9/893: symlink d0/d18/d24/d47/l118 0 2024-07-03T15:43:56.377 INFO:tasks.workunit.client.1.smithi148.stdout:2/685: dread d5/d14/d22/d60/f90 [0,4194304] 0 2024-07-03T15:43:56.377 INFO:tasks.workunit.client.1.smithi148.stdout:2/686: fdatasync d5/d14/d86/db9/dbb/f1e 0 2024-07-03T15:43:56.377 INFO:tasks.workunit.client.1.smithi148.stdout:2/687: dread - d5/d14/d86/db9/dbb/d13/d2d/fd7 zero size 2024-07-03T15:43:56.378 INFO:tasks.workunit.client.1.smithi148.stdout:7/713: dread d5/d7d/da7/d1c/d59/f76 [0,4194304] 0 2024-07-03T15:43:56.379 INFO:tasks.workunit.client.1.smithi148.stdout:7/714: write d5/f7b [3012010,129299] 0 2024-07-03T15:43:56.379 INFO:tasks.workunit.client.1.smithi148.stdout:7/715: fdatasync d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/db2/fde 0 2024-07-03T15:43:56.379 INFO:tasks.workunit.client.1.smithi148.stdout:7/716: write d5/d7d/da7/d1c/d4a/f62 [315878,91473] 0 2024-07-03T15:43:56.380 INFO:tasks.workunit.client.1.smithi148.stdout:8/562: rmdir d4/da/d14/d1a/d32 39 2024-07-03T15:43:56.382 INFO:tasks.workunit.client.1.smithi148.stdout:3/646: dwrite dd/d33/d37/d68/f9c [0,4194304] 0 2024-07-03T15:43:56.383 INFO:tasks.workunit.client.1.smithi148.stdout:1/575: dwrite d0/d2/d6/f26 [8388608,4194304] 0 2024-07-03T15:43:56.385 INFO:tasks.workunit.client.1.smithi148.stdout:4/560: rename d0/d28/l5e to d0/d8/d13/d67/lc1 0 2024-07-03T15:43:56.386 INFO:tasks.workunit.client.1.smithi148.stdout:6/710: unlink d3/d1d/d28/d53/l9b 0 2024-07-03T15:43:56.389 INFO:tasks.workunit.client.1.smithi148.stdout:2/688: symlink d5/d14/d8b/dd5/ldc 0 2024-07-03T15:43:56.392 INFO:tasks.workunit.client.1.smithi148.stdout:0/562: sync 2024-07-03T15:43:56.392 INFO:tasks.workunit.client.1.smithi148.stdout:7/717: creat d5/d7d/da7/d1c/d59/ff7 x:0 0 0 2024-07-03T15:43:56.392 INFO:tasks.workunit.client.1.smithi148.stdout:7/718: fsync d5/d7d/da7/d1c/d4a/d66/f9b 0 2024-07-03T15:43:56.395 INFO:tasks.workunit.client.1.smithi148.stdout:8/563: mknod d4/da/d51/d18/d20/cca 0 2024-07-03T15:43:56.401 INFO:tasks.workunit.client.1.smithi148.stdout:1/576: mkdir d0/d2/d2e/d39/d41/d77/db4/dc7 0 2024-07-03T15:43:56.401 INFO:tasks.workunit.client.1.smithi148.stdout:1/577: write d0/de/d2f/fbf [166013,59888] 0 2024-07-03T15:43:56.402 INFO:tasks.workunit.client.1.smithi148.stdout:1/578: write d0/de/d2f/fbf [759976,108155] 0 2024-07-03T15:43:56.403 INFO:tasks.workunit.client.1.smithi148.stdout:3/647: rename dd/f2b to dd/d35/d81/fca 0 2024-07-03T15:43:56.403 INFO:tasks.workunit.client.1.smithi148.stdout:3/648: chown dd/f3d 3 1 2024-07-03T15:43:56.405 INFO:tasks.workunit.client.1.smithi148.stdout:4/561: mkdir d0/d8/d44/dc2 0 2024-07-03T15:43:56.405 INFO:tasks.workunit.client.1.smithi148.stdout:4/562: read d0/d91/fa2 [639671,47177] 0 2024-07-03T15:43:56.405 INFO:tasks.workunit.client.1.smithi148.stdout:4/563: write d0/d28/da7/d2f/f33 [1092023,100164] 0 2024-07-03T15:43:56.405 INFO:tasks.workunit.client.1.smithi148.stdout:4/564: truncate d0/d8/d3a/f66 449509 0 2024-07-03T15:43:56.406 INFO:tasks.workunit.client.1.smithi148.stdout:9/894: getdents d0/dd/d25/d37/d40/d72/db8/def 0 2024-07-03T15:43:56.408 INFO:tasks.workunit.client.1.smithi148.stdout:7/719: mknod d5/d7d/da7/db4/ddd/cf8 0 2024-07-03T15:43:56.411 INFO:tasks.workunit.client.1.smithi148.stdout:1/579: unlink d0/d2/d6/d14/c93 0 2024-07-03T15:43:56.413 INFO:tasks.workunit.client.1.smithi148.stdout:3/649: link dd/d33/d37/d68/d67/fa0 dd/d35/d41/d89/db0/fcb 0 2024-07-03T15:43:56.416 INFO:tasks.workunit.client.1.smithi148.stdout:2/689: dwrite d5/d14/d86/db9/dbb/d13/f15 [4194304,4194304] 0 2024-07-03T15:43:56.416 INFO:tasks.workunit.client.1.smithi148.stdout:6/711: dwrite d3/d1d/d28/f2e [0,4194304] 0 2024-07-03T15:43:56.417 INFO:tasks.workunit.client.1.smithi148.stdout:0/563: truncate da/d17/f1e 3578971 0 2024-07-03T15:43:56.417 INFO:tasks.workunit.client.1.smithi148.stdout:0/564: chown da/d44/d13/l56 520635 1 2024-07-03T15:43:56.419 INFO:tasks.workunit.client.1.smithi148.stdout:7/720: fsync d5/d7d/da7/d1c/d4a/d66/d93/faf 0 2024-07-03T15:43:56.419 INFO:tasks.workunit.client.1.smithi148.stdout:8/564: dread d4/da/d14/d1a/d32/d38/dba/f95 [0,4194304] 0 2024-07-03T15:43:56.419 INFO:tasks.workunit.client.1.smithi148.stdout:8/565: fsync d4/da/d51/d18/d20/f5a 0 2024-07-03T15:43:56.419 INFO:tasks.workunit.client.1.smithi148.stdout:8/566: chown d4/da/d51/d18/d20/f5a 5400241 1 2024-07-03T15:43:56.420 INFO:tasks.workunit.client.1.smithi148.stdout:8/567: chown d4/da/d51/d18/d20/f66 3775597 1 2024-07-03T15:43:56.420 INFO:tasks.workunit.client.1.smithi148.stdout:8/568: write d4/da/d51/d53/f7f [963803,1125] 0 2024-07-03T15:43:56.422 INFO:tasks.workunit.client.1.smithi148.stdout:3/650: truncate dd/d33/d52/d57/fae 3317933 0 2024-07-03T15:43:56.424 INFO:tasks.workunit.client.1.smithi148.stdout:4/565: rename d0/d8/d13/d8b/l9b to d0/d28/da7/d2f/d41/lc3 0 2024-07-03T15:43:56.424 INFO:tasks.workunit.client.1.smithi148.stdout:4/566: chown d0/d28/da7/d2f/d41/f7a 103 1 2024-07-03T15:43:56.425 INFO:tasks.workunit.client.1.smithi148.stdout:4/567: read d0/d8/d3a/f66 [81424,104460] 0 2024-07-03T15:43:56.426 INFO:tasks.workunit.client.1.smithi148.stdout:2/690: dread d5/d14/d86/db9/dbb/d68/f7c [0,4194304] 0 2024-07-03T15:43:56.430 INFO:tasks.workunit.client.1.smithi148.stdout:1/580: dwrite d0/de/d44/d54/fa1 [0,4194304] 0 2024-07-03T15:43:56.430 INFO:tasks.workunit.client.1.smithi148.stdout:1/581: truncate d0/de/d44/d54/d5d/f8c 708761 0 2024-07-03T15:43:56.439 INFO:tasks.workunit.client.1.smithi148.stdout:4/568: dwrite d0/d28/da7/d2f/d41/f4d [0,4194304] 0 2024-07-03T15:43:56.444 INFO:tasks.workunit.client.1.smithi148.stdout:6/712: mknod d3/d1d/d28/d53/d6d/ce5 0 2024-07-03T15:43:56.446 INFO:tasks.workunit.client.1.smithi148.stdout:0/565: creat da/d44/d13/d14/fb9 x:0 0 0 2024-07-03T15:43:56.449 INFO:tasks.workunit.client.1.smithi148.stdout:3/651: unlink dd/d33/d37/d68/d2e/d40/d45/d9a/laa 0 2024-07-03T15:43:56.449 INFO:tasks.workunit.client.1.smithi148.stdout:3/652: dread - dd/d33/d37/d68/d2e/d40/f80 zero size 2024-07-03T15:43:56.449 INFO:tasks.workunit.client.1.smithi148.stdout:3/653: fsync dd/d33/d37/d68/d2e/d40/d45/d9a/fab 0 2024-07-03T15:43:56.450 INFO:tasks.workunit.client.1.smithi148.stdout:4/569: dread d0/d28/d7d/f8a [0,4194304] 0 2024-07-03T15:43:56.451 INFO:tasks.workunit.client.1.smithi148.stdout:9/895: rename d0/dd/d25/d59/c7e to d0/dd/d25/d59/dcb/c119 0 2024-07-03T15:43:56.452 INFO:tasks.workunit.client.1.smithi148.stdout:2/691: rmdir d5/d14/d86/db9/dbb 39 2024-07-03T15:43:56.453 INFO:tasks.workunit.client.1.smithi148.stdout:2/692: readlink d5/d14/d86/d93/l9e 0 2024-07-03T15:43:56.458 INFO:tasks.workunit.client.1.smithi148.stdout:1/582: link d0/d2/d11/d3a/cbc d0/d2/d11/d1e/d9d/dc0/cc8 0 2024-07-03T15:43:56.461 INFO:tasks.workunit.client.1.smithi148.stdout:6/713: dwrite d3/d1d/d28/d3e/f68 [4194304,4194304] 0 2024-07-03T15:43:56.466 INFO:tasks.workunit.client.1.smithi148.stdout:8/569: mkdir d4/da/d14/d1a/d32/d38/d58/dcb 0 2024-07-03T15:43:56.466 INFO:tasks.workunit.client.1.smithi148.stdout:8/570: read - d4/da/d14/d1a/d32/d38/fb2 zero size 2024-07-03T15:43:56.468 INFO:tasks.workunit.client.1.smithi148.stdout:4/570: creat d0/d8/d13/d67/db1/fc4 x:0 0 0 2024-07-03T15:43:56.469 INFO:tasks.workunit.client.1.smithi148.stdout:9/896: mkdir d0/dd/d25/d37/d40/da3/df0/d11a 0 2024-07-03T15:43:56.469 INFO:tasks.workunit.client.1.smithi148.stdout:9/897: fsync d0/d49/d43/d19/fc8 0 2024-07-03T15:43:56.471 INFO:tasks.workunit.client.1.smithi148.stdout:2/693: mkdir d5/d67/dc3/ddd 0 2024-07-03T15:43:56.474 INFO:tasks.workunit.client.1.smithi148.stdout:1/583: truncate d0/de/d44/d54/d5d/f8b 313172 0 2024-07-03T15:43:56.474 INFO:tasks.workunit.client.1.smithi148.stdout:1/584: stat d0/d2/d6/c68 0 2024-07-03T15:43:56.474 INFO:tasks.workunit.client.1.smithi148.stdout:1/585: chown d0/d2/d11/d3a/f70 206383804 1 2024-07-03T15:43:56.474 INFO:tasks.workunit.client.1.smithi148.stdout:1/586: fsync d0/d2/d2e/fa7 0 2024-07-03T15:43:56.474 INFO:tasks.workunit.client.1.smithi148.stdout:1/587: chown d0/d2/d11/d3a/f5b 385569295 1 2024-07-03T15:43:56.474 INFO:tasks.workunit.client.1.smithi148.stdout:0/566: write da/d17/f76 [2064492,97022] 0 2024-07-03T15:43:56.474 INFO:tasks.workunit.client.1.smithi148.stdout:6/714: dread d3/d50/fcf [0,4194304] 0 2024-07-03T15:43:56.476 INFO:tasks.workunit.client.1.smithi148.stdout:8/571: rename d4/da/d51/f9d to d4/da/d14/d1a/d32/d89/fcc 0 2024-07-03T15:43:56.477 INFO:tasks.workunit.client.1.smithi148.stdout:9/898: write d0/d18/d24/d47/d4d/fec [1242993,45661] 0 2024-07-03T15:43:56.477 INFO:tasks.workunit.client.1.smithi148.stdout:4/571: symlink d0/d91/da8/lc5 0 2024-07-03T15:43:56.481 INFO:tasks.workunit.client.1.smithi148.stdout:1/588: mknod d0/d2/d2e/d39/d42/d62/cc9 0 2024-07-03T15:43:56.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:56 smithi148 ceph-mon[36056]: Standby manager daemon smithi148.rjzsar restarted 2024-07-03T15:43:56.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:56 smithi148 ceph-mon[36056]: Standby manager daemon smithi148.rjzsar started 2024-07-03T15:43:56.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:56 smithi148 ceph-mon[36056]: from='mgr.? 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.rjzsar/crt"}]: dispatch 2024-07-03T15:43:56.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:56 smithi148 ceph-mon[36056]: from='mgr.? 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-03T15:43:56.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:56 smithi148 ceph-mon[36056]: from='mgr.? 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.rjzsar/key"}]: dispatch 2024-07-03T15:43:56.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:56 smithi148 ceph-mon[36056]: from='mgr.? 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-03T15:43:56.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:56 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:43:56.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:56 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:43:56.483 INFO:tasks.workunit.client.1.smithi148.stdout:0/567: mkdir da/d44/d94/dba 0 2024-07-03T15:43:56.486 INFO:tasks.workunit.client.1.smithi148.stdout:6/715: creat d3/d1d/dd6/fe6 x:0 0 0 2024-07-03T15:43:56.486 INFO:tasks.workunit.client.1.smithi148.stdout:6/716: dread - d3/f2f zero size 2024-07-03T15:43:56.491 INFO:tasks.workunit.client.1.smithi148.stdout:9/899: symlink d0/d63/l11b 0 2024-07-03T15:43:56.493 INFO:tasks.workunit.client.1.smithi148.stdout:2/694: mkdir d5/da8/dde 0 2024-07-03T15:43:56.496 INFO:tasks.workunit.client.1.smithi148.stdout:8/572: dwrite d4/da/d51/f7 [0,4194304] 0 2024-07-03T15:43:56.497 INFO:tasks.workunit.client.1.smithi148.stdout:6/717: mknod d3/d88/ce7 0 2024-07-03T15:43:56.498 INFO:tasks.workunit.client.1.smithi148.stdout:4/572: dwrite d0/d8/d3a/f47 [0,4194304] 0 2024-07-03T15:43:56.501 INFO:tasks.workunit.client.1.smithi148.stdout:9/900: link d0/d49/d43/d19/de1/lc6 d0/d8b/l11c 0 2024-07-03T15:43:56.503 INFO:tasks.workunit.client.1.smithi148.stdout:1/589: rename d0/de/d44/d54/d5d/f50 to d0/d2/fca 0 2024-07-03T15:43:56.506 INFO:tasks.workunit.client.1.smithi148.stdout:8/573: symlink d4/da/d51/d18/d20/lcd 0 2024-07-03T15:43:56.507 INFO:tasks.workunit.client.1.smithi148.stdout:6/718: getdents d3/d1d/d28/d31/d61/dac 0 2024-07-03T15:43:56.508 INFO:tasks.workunit.client.1.smithi148.stdout:5/653: sync 2024-07-03T15:43:56.508 INFO:tasks.workunit.client.1.smithi148.stdout:5/654: creat d7/fe3 x:0 0 0 2024-07-03T15:43:56.508 INFO:tasks.workunit.client.1.smithi148.stdout:5/655: fdatasync d7/da/df/d16/f8a 0 2024-07-03T15:43:56.511 INFO:tasks.workunit.client.1.smithi148.stdout:4/573: mknod d0/d91/da8/cc6 0 2024-07-03T15:43:56.511 INFO:tasks.workunit.client.1.smithi148.stdout:4/574: chown d0/d8/d19/c78 13 1 2024-07-03T15:43:56.513 INFO:tasks.workunit.client.1.smithi148.stdout:9/901: dwrite d0/d49/d43/d19/d7a/f10d [0,4194304] 0 2024-07-03T15:43:56.518 INFO:tasks.workunit.client.1.smithi148.stdout:0/568: rename da/d44/d13/d1d/d3e/d85/laf to da/d58/d90/lbb 0 2024-07-03T15:43:56.520 INFO:tasks.workunit.client.1.smithi148.stdout:1/590: link d0/d2/d2e/d39/d42/d62/f6e d0/fcb 0 2024-07-03T15:43:56.522 INFO:tasks.workunit.client.1.smithi148.stdout:8/574: mkdir d4/da/d47/d88/dce 0 2024-07-03T15:43:56.525 INFO:tasks.workunit.client.1.smithi148.stdout:6/719: symlink d3/le8 0 2024-07-03T15:43:56.529 INFO:tasks.workunit.client.1.smithi148.stdout:4/575: unlink d0/f17 0 2024-07-03T15:43:56.538 INFO:tasks.workunit.client.1.smithi148.stdout:1/591: dwrite d0/d2/d6/d14/f6a [0,4194304] 0 2024-07-03T15:43:56.538 INFO:tasks.workunit.client.1.smithi148.stdout:1/592: chown d0/d2/d2e/d39/d41/d77/f84 155544818 1 2024-07-03T15:43:56.539 INFO:tasks.workunit.client.1.smithi148.stdout:8/575: chown d4/da/d51/c22 921 1 2024-07-03T15:43:56.541 INFO:tasks.workunit.client.1.smithi148.stdout:6/720: creat d3/d1d/d28/d53/d6d/d8f/dd3/fe9 x:0 0 0 2024-07-03T15:43:56.541 INFO:tasks.workunit.client.1.smithi148.stdout:8/576: dread d4/da/d51/d53/f7f [0,4194304] 0 2024-07-03T15:43:56.541 INFO:tasks.workunit.client.1.smithi148.stdout:8/577: chown d4/da/f29 124752538 1 2024-07-03T15:43:56.547 INFO:tasks.workunit.client.1.smithi148.stdout:2/695: rename d5/d5f to d5/d14/d86/db9/dbb/d13/d41/d7b/ddf 0 2024-07-03T15:43:56.548 INFO:tasks.workunit.client.1.smithi148.stdout:8/578: dread d4/da/d14/d1a/d32/fae [0,4194304] 0 2024-07-03T15:43:56.556 INFO:tasks.workunit.client.1.smithi148.stdout:2/696: read d5/d14/d86/db9/dbb/f42 [2320307,123438] 0 2024-07-03T15:43:56.557 INFO:tasks.workunit.client.1.smithi148.stdout:1/593: getdents d0/d2/d2e/d39/d42/d62/d7b 0 2024-07-03T15:43:56.557 INFO:tasks.workunit.client.1.smithi148.stdout:1/594: truncate d0/d2/d6/d14/f58 930084 0 2024-07-03T15:43:56.559 INFO:tasks.workunit.client.1.smithi148.stdout:6/721: mkdir d3/d1d/d28/d3e/ddd/dea 0 2024-07-03T15:43:56.564 INFO:tasks.workunit.client.1.smithi148.stdout:8/579: link d4/da/d51/d18/c3c d4/da/d14/d3a/da8/db8/ccf 0 2024-07-03T15:43:56.564 INFO:tasks.workunit.client.1.smithi148.stdout:8/580: truncate d4/da/d47/d88/d92/fa3 972231 0 2024-07-03T15:43:56.564 INFO:tasks.workunit.client.1.smithi148.stdout:8/581: fdatasync d4/da/d47/d88/d92/f42 0 2024-07-03T15:43:56.564 INFO:tasks.workunit.client.1.smithi148.stdout:8/582: write d4/da/d14/d17/fc8 [487304,124919] 0 2024-07-03T15:43:56.565 INFO:tasks.workunit.client.1.smithi148.stdout:2/697: truncate d5/d14/d86/db9/dbb/d68/f7c 1930303 0 2024-07-03T15:43:56.567 INFO:tasks.workunit.client.1.smithi148.stdout:1/595: truncate d0/d2/d2e/d39/d42/d62/f71 706365 0 2024-07-03T15:43:56.567 INFO:tasks.workunit.client.1.smithi148.stdout:1/596: write d0/d2/d6/f9e [865542,127460] 0 2024-07-03T15:43:56.570 INFO:tasks.workunit.client.1.smithi148.stdout:6/722: unlink d3/d1d/d28/d3e/ddd/d74/cbe 0 2024-07-03T15:43:56.574 INFO:tasks.workunit.client.1.smithi148.stdout:2/698: stat d5/d14/d86/db9/dbb/d68/dcd/c73 0 2024-07-03T15:43:56.576 INFO:tasks.workunit.client.1.smithi148.stdout:6/723: write d3/d1d/d28/d3e/f58 [162110,73962] 0 2024-07-03T15:43:56.576 INFO:tasks.workunit.client.1.smithi148.stdout:6/724: write d3/f34 [569631,83430] 0 2024-07-03T15:43:56.577 INFO:tasks.workunit.client.1.smithi148.stdout:1/597: chown d0/de/c33 3 1 2024-07-03T15:43:56.579 INFO:tasks.workunit.client.1.smithi148.stdout:7/721: sync 2024-07-03T15:43:56.579 INFO:tasks.workunit.client.1.smithi148.stdout:7/722: write d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/db6/fbe [529633,64945] 0 2024-07-03T15:43:56.582 INFO:tasks.workunit.client.1.smithi148.stdout:5/656: rename d7/da/df/d16/d1a/d58 to d7/da/df/d16/de4 0 2024-07-03T15:43:56.582 INFO:tasks.workunit.client.1.smithi148.stdout:5/657: chown d7/da/f74 25 1 2024-07-03T15:43:56.584 INFO:tasks.workunit.client.1.smithi148.stdout:8/583: symlink d4/da/d14/d1a/d32/d38/d58/ld0 0 2024-07-03T15:43:56.586 INFO:tasks.workunit.client.1.smithi148.stdout:1/598: creat d0/d2/d2e/d39/d42/d62/d7b/fcc x:0 0 0 2024-07-03T15:43:56.586 INFO:tasks.workunit.client.1.smithi148.stdout:1/599: write d0/d2/d6/d14/d29/d94/fa0 [4897049,126239] 0 2024-07-03T15:43:56.586 INFO:tasks.workunit.client.1.smithi148.stdout:1/600: chown d0/d2/d2e/d39/d41/d77/d80/fb5 36589 1 2024-07-03T15:43:56.586 INFO:tasks.workunit.client.1.smithi148.stdout:1/601: creat d0/d2/d2e/d39/d41/fcd x:0 0 0 2024-07-03T15:43:56.591 INFO:tasks.workunit.client.1.smithi148.stdout:5/658: mkdir d7/da/df/d16/d1a/de5 0 2024-07-03T15:43:56.591 INFO:tasks.workunit.client.1.smithi148.stdout:5/659: write d7/da/df/d16/f7e [496392,72752] 0 2024-07-03T15:43:56.596 INFO:tasks.workunit.client.1.smithi148.stdout:1/602: dread d0/d2/d11/d1e/f3c [0,4194304] 0 2024-07-03T15:43:56.597 INFO:tasks.workunit.client.1.smithi148.stdout:5/660: readlink d7/da/d21/l68 0 2024-07-03T15:43:56.602 INFO:tasks.workunit.client.1.smithi148.stdout:4/576: rename d0/d8/d19 to d0/d28/d6e/dc7 0 2024-07-03T15:43:56.604 INFO:tasks.workunit.client.1.smithi148.stdout:6/725: rename d3/l42 to d3/d1d/d28/d31/leb 0 2024-07-03T15:43:56.605 INFO:tasks.workunit.client.1.smithi148.stdout:2/699: dwrite d5/d14/d22/d60/f6c [0,4194304] 0 2024-07-03T15:43:56.605 INFO:tasks.workunit.client.1.smithi148.stdout:7/723: dwrite d5/d7d/da7/d1c/d33/d4c/f99 [0,4194304] 0 2024-07-03T15:43:56.608 INFO:tasks.workunit.client.1.smithi148.stdout:8/584: rename d4/da/d14/f4c to d4/da/d14/d1a/d32/d38/fd1 0 2024-07-03T15:43:56.609 INFO:tasks.workunit.client.1.smithi148.stdout:5/661: dread d7/d13/d54/f4b [0,4194304] 0 2024-07-03T15:43:56.612 INFO:tasks.workunit.client.1.smithi148.stdout:2/700: symlink d5/da8/le0 0 2024-07-03T15:43:56.621 INFO:tasks.workunit.client.1.smithi148.stdout:1/603: dwrite d0/d2/d6/d14/d29/d94/fa0 [4194304,4194304] 0 2024-07-03T15:43:56.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:56 smithi028 ceph-mon[29148]: Standby manager daemon smithi148.rjzsar restarted 2024-07-03T15:43:56.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:56 smithi028 ceph-mon[29148]: Standby manager daemon smithi148.rjzsar started 2024-07-03T15:43:56.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:56 smithi028 ceph-mon[29148]: from='mgr.? 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.rjzsar/crt"}]: dispatch 2024-07-03T15:43:56.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:56 smithi028 ceph-mon[29148]: from='mgr.? 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-03T15:43:56.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:56 smithi028 ceph-mon[29148]: from='mgr.? 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.rjzsar/key"}]: dispatch 2024-07-03T15:43:56.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:56 smithi028 ceph-mon[29148]: from='mgr.? 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-03T15:43:56.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:56 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:43:56.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:56 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:43:56.623 INFO:tasks.workunit.client.1.smithi148.stdout:7/724: creat d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/de8/ff9 x:0 0 0 2024-07-03T15:43:56.623 INFO:tasks.workunit.client.1.smithi148.stdout:7/725: truncate d5/d7d/da7/d1c/d33/f94 860885 0 2024-07-03T15:43:56.623 INFO:tasks.workunit.client.1.smithi148.stdout:7/726: write d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/db2/fde [409416,3067] 0 2024-07-03T15:43:56.624 INFO:tasks.workunit.client.1.smithi148.stdout:7/727: read d5/f58 [239643,91010] 0 2024-07-03T15:43:56.625 INFO:tasks.workunit.client.1.smithi148.stdout:2/701: write d5/d14/d86/db9/dbb/d13/d2d/fb8 [3484592,32239] 0 2024-07-03T15:43:56.627 INFO:tasks.workunit.client.1.smithi148.stdout:8/585: mkdir d4/da/d14/dd2 0 2024-07-03T15:43:56.633 INFO:tasks.workunit.client.1.smithi148.stdout:4/577: rename d0/d8/d13/c1e to d0/d28/d7d/cc8 0 2024-07-03T15:43:56.633 INFO:tasks.workunit.client.1.smithi148.stdout:7/728: dread d5/d7d/da7/d1c/d33/d4c/f99 [0,4194304] 0 2024-07-03T15:43:56.634 INFO:tasks.workunit.client.1.smithi148.stdout:7/729: dread d5/d7d/da7/f6a [0,4194304] 0 2024-07-03T15:43:56.635 INFO:tasks.workunit.client.1.smithi148.stdout:2/702: symlink d5/d14/d86/le1 0 2024-07-03T15:43:56.636 INFO:tasks.workunit.client.1.smithi148.stdout:2/703: dread - d5/d14/d86/db9/dbb/d13/d2d/fd7 zero size 2024-07-03T15:43:56.636 INFO:tasks.workunit.client.1.smithi148.stdout:2/704: dread - d5/d14/d86/db9/dbb/d68/dcd/d64/fd4 zero size 2024-07-03T15:43:56.637 INFO:tasks.workunit.client.1.smithi148.stdout:8/586: mkdir d4/da/d14/d3a/da8/dd3 0 2024-07-03T15:43:56.642 INFO:tasks.workunit.client.1.smithi148.stdout:8/587: symlink d4/d86/ld4 0 2024-07-03T15:43:56.642 INFO:tasks.workunit.client.1.smithi148.stdout:8/588: stat d4/da/d47/d88/d92/f42 0 2024-07-03T15:43:56.642 INFO:tasks.workunit.client.1.smithi148.stdout:8/589: chown d4/da/d47/d88 61 1 2024-07-03T15:43:56.645 INFO:tasks.workunit.client.1.smithi148.stdout:5/662: rename d7/da/df/d16/d2f/d44/d52/d86 to d7/da/dbe/de6 0 2024-07-03T15:43:56.648 INFO:tasks.workunit.client.1.smithi148.stdout:1/604: rename d0/de/d2f/f55 to d0/d2/d6/d14/d29/fce 0 2024-07-03T15:43:56.649 INFO:tasks.workunit.client.1.smithi148.stdout:5/663: symlink d7/da/df/d16/d6a/le7 0 2024-07-03T15:43:56.649 INFO:tasks.workunit.client.1.smithi148.stdout:5/664: write d7/da/df/d16/d1a/f7a [864725,123368] 0 2024-07-03T15:43:56.651 INFO:tasks.workunit.client.1.smithi148.stdout:2/705: rename d5/d14/d86/db9/dbb/d13/d41/d7b/ddf/ca0 to d5/d14/d8b/dd5/ce2 0 2024-07-03T15:43:56.652 INFO:tasks.workunit.client.1.smithi148.stdout:1/605: truncate d0/de/d2f/d72/f7c 262538 0 2024-07-03T15:43:56.653 INFO:tasks.workunit.client.1.smithi148.stdout:7/730: dwrite d5/d7d/da7/d1c/d33/f48 [0,4194304] 0 2024-07-03T15:43:56.653 INFO:tasks.workunit.client.1.smithi148.stdout:7/731: chown d5/d7d/da7/db4/df0 2051818 1 2024-07-03T15:43:56.653 INFO:tasks.workunit.client.1.smithi148.stdout:7/732: creat d5/d7d/da7/d1c/d33/d4c/d21/d47/d8b/dd3/ffa x:0 0 0 2024-07-03T15:43:56.654 INFO:tasks.workunit.client.1.smithi148.stdout:5/665: mknod d7/da/df/d16/d1a/ce8 0 2024-07-03T15:43:56.656 INFO:tasks.workunit.client.1.smithi148.stdout:8/590: rename d4/da/d47/f83 to d4/da/d14/d17/fd5 0 2024-07-03T15:43:56.658 INFO:tasks.workunit.client.1.smithi148.stdout:2/706: creat d5/d14/d86/db9/fe3 x:0 0 0 2024-07-03T15:43:56.658 INFO:tasks.workunit.client.1.smithi148.stdout:2/707: chown d5/d14/d86/db9/dbb/d13/d1d/f43 324792032 1 2024-07-03T15:43:56.658 INFO:tasks.workunit.client.1.smithi148.stdout:2/708: write d5/d14/d22/d60/f53 [3030665,66950] 0 2024-07-03T15:43:56.658 INFO:tasks.workunit.client.1.smithi148.stdout:2/709: dread - d5/d14/d86/db9/dbb/f6d zero size 2024-07-03T15:43:56.659 INFO:tasks.workunit.client.1.smithi148.stdout:2/710: fsync d5/d14/d22/fae 0 2024-07-03T15:43:56.660 INFO:tasks.workunit.client.1.smithi148.stdout:1/606: symlink d0/d2/d6/d14/d29/d94/lcf 0 2024-07-03T15:43:56.660 INFO:tasks.workunit.client.1.smithi148.stdout:1/607: fdatasync d0/d2/d11/d3a/f5b 0 2024-07-03T15:43:56.660 INFO:tasks.workunit.client.1.smithi148.stdout:1/608: stat d0/d2/d2e/d39/d41/f43 0 2024-07-03T15:43:56.662 INFO:tasks.workunit.client.1.smithi148.stdout:4/578: getdents d0/d28/d7d 0 2024-07-03T15:43:56.663 INFO:tasks.workunit.client.1.smithi148.stdout:8/591: dread d4/da/d51/d18/d20/f66 [0,4194304] 0 2024-07-03T15:43:56.664 INFO:tasks.workunit.client.1.smithi148.stdout:7/733: creat d5/d7d/da7/db4/ffb x:0 0 0 2024-07-03T15:43:56.666 INFO:tasks.workunit.client.1.smithi148.stdout:2/711: mkdir d5/d14/d86/db9/dbb/d13/d41/d7b/ddf/d79/de4 0 2024-07-03T15:43:56.666 INFO:tasks.workunit.client.1.smithi148.stdout:2/712: fdatasync d5/d14/d86/db9/dbb/d68/dcd/f35 0 2024-07-03T15:43:56.669 INFO:tasks.workunit.client.1.smithi148.stdout:5/666: dwrite d7/da/df/d16/d6a/db4/fd9 [0,4194304] 0 2024-07-03T15:43:56.669 INFO:tasks.workunit.client.1.smithi148.stdout:5/667: read - d7/da/df/d16/d6a/db4/f9c zero size 2024-07-03T15:43:56.669 INFO:tasks.workunit.client.1.smithi148.stdout:4/579: creat d0/d8/d3a/db9/fc9 x:0 0 0 2024-07-03T15:43:56.670 INFO:tasks.workunit.client.1.smithi148.stdout:8/592: truncate d4/da/d14/d1a/f37 4516205 0 2024-07-03T15:43:56.670 INFO:tasks.workunit.client.1.smithi148.stdout:8/593: chown d4/da/d14/dd2 13 1 2024-07-03T15:43:56.670 INFO:tasks.workunit.client.1.smithi148.stdout:8/594: fdatasync d4/da/f28 0 2024-07-03T15:43:56.671 INFO:tasks.workunit.client.1.smithi148.stdout:1/609: mkdir d0/d2/dd0 0 2024-07-03T15:43:56.674 INFO:tasks.workunit.client.1.smithi148.stdout:8/595: write d4/da/d51/d18/d20/f66 [1543849,128461] 0 2024-07-03T15:43:56.674 INFO:tasks.workunit.client.1.smithi148.stdout:8/596: write d4/da/d14/d3a/f7e [283272,90176] 0 2024-07-03T15:43:56.676 INFO:tasks.workunit.client.1.smithi148.stdout:7/734: write d5/d7d/da7/fab [145934,43349] 0 2024-07-03T15:43:56.676 INFO:tasks.workunit.client.1.smithi148.stdout:7/735: truncate d5/d7d/da7/d1c/d33/d4c/d84/d98/d9e/fea 438909 0 2024-07-03T15:43:56.677 INFO:tasks.workunit.client.1.smithi148.stdout:4/580: read d0/d8/d13/db4/fbc [2258827,58564] 0 2024-07-03T15:43:56.683 INFO:tasks.workunit.client.1.smithi148.stdout:2/713: dwrite d5/d14/d86/db9/dbb/f6d [0,4194304] 0 2024-07-03T15:43:56.687 INFO:tasks.workunit.client.1.smithi148.stdout:8/597: symlink d4/da/d47/d88/d92/ld6 0 2024-07-03T15:43:56.687 INFO:tasks.workunit.client.1.smithi148.stdout:8/598: fsync d4/da/d51/d18/d20/d82/fbd 0 2024-07-03T15:43:56.691 INFO:tasks.workunit.client.1.smithi148.stdout:1/610: dwrite d0/de/d89/f99 [0,4194304] 0 2024-07-03T15:43:56.692 INFO:tasks.workunit.client.1.smithi148.stdout:4/581: creat d0/d28/d7d/d6f/d7c/da3/fca x:0 0 0 2024-07-03T15:43:56.694 INFO:tasks.workunit.client.1.smithi148.stdout:2/714: dread d5/d14/d86/db9/dbb/d13/d41/d7b/d6a/f77 [0,4194304] 0 2024-07-03T15:43:56.699 INFO:tasks.workunit.client.1.smithi148.stdout:8/599: creat d4/da/d14/d1a/fd7 x:0 0 0 2024-07-03T15:43:56.699 INFO:tasks.workunit.client.1.smithi148.stdout:8/600: write d4/f98 [1015664,34682] 0 2024-07-03T15:43:56.700 INFO:tasks.workunit.client.1.smithi148.stdout:7/736: rmdir d5/d7d/da7/d1c/d59 39 2024-07-03T15:43:56.705 INFO:tasks.workunit.client.1.smithi148.stdout:1/611: link d0/de/d44/c56 d0/de/d2f/d72/cd1 0 2024-07-03T15:43:56.706 INFO:tasks.workunit.client.1.smithi148.stdout:4/582: creat d0/d28/db0/fcb x:0 0 0 2024-07-03T15:43:56.709 INFO:tasks.workunit.client.1.smithi148.stdout:5/668: rename d7/da/d79 to d7/da/df/d16/d2f/dc2/de9 0 2024-07-03T15:43:56.710 INFO:tasks.workunit.client.1.smithi148.stdout:5/669: dread d7/d13/f2a [0,4194304] 0 2024-07-03T15:43:56.713 INFO:tasks.workunit.client.1.smithi148.stdout:3/654: sync 2024-07-03T15:43:56.713 INFO:tasks.workunit.client.1.smithi148.stdout:9/902: sync 2024-07-03T15:43:56.715 INFO:tasks.workunit.client.1.smithi148.stdout:8/601: mknod d4/cd8 0 2024-07-03T15:43:56.715 INFO:tasks.workunit.client.1.smithi148.stdout:8/602: truncate d4/da/d14/d1a/d32/d38/dba/fc0 711155 0 2024-07-03T15:43:56.716 INFO:tasks.workunit.client.1.smithi148.stdout:7/737: mkdir d5/d7d/da7/d1c/d33/d4c/d21/d31/dfc 0 2024-07-03T15:43:56.717 INFO:tasks.workunit.client.1.smithi148.stdout:4/583: mknod d0/d91/da8/dab/dad/ccc 0 2024-07-03T15:43:56.718 INFO:tasks.workunit.client.1.smithi148.stdout:4/584: creat d0/d8/d13/db4/fcd x:0 0 0 2024-07-03T15:43:56.719 INFO:tasks.workunit.client.1.smithi148.stdout:1/612: rename d0/d2/d2e/d39/d41/f65 to d0/d2/d2e/d39/d41/d77/db4/dc7/fd2 0 2024-07-03T15:43:56.721 INFO:tasks.workunit.client.1.smithi148.stdout:5/670: symlink d7/da/df/d16/de4/da7/lea 0 2024-07-03T15:43:56.721 INFO:tasks.workunit.client.1.smithi148.stdout:5/671: creat d7/d13/d54/feb x:0 0 0 2024-07-03T15:43:56.722 INFO:tasks.workunit.client.1.smithi148.stdout:9/903: creat d0/dd/f11d x:0 0 0 2024-07-03T15:43:56.723 INFO:tasks.workunit.client.1.smithi148.stdout:3/655: creat dd/d33/d37/d8d/fcc x:0 0 0 2024-07-03T15:43:56.728 INFO:tasks.workunit.client.1.smithi148.stdout:7/738: dwrite d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/db2/fe5 [0,4194304] 0 2024-07-03T15:43:56.736 INFO:tasks.workunit.client.1.smithi148.stdout:4/585: mkdir d0/d8/d3a/dce 0 2024-07-03T15:43:56.738 INFO:tasks.workunit.client.1.smithi148.stdout:1/613: symlink d0/d2/d6/d14/d29/ld3 0 2024-07-03T15:43:56.741 INFO:tasks.workunit.client.1.smithi148.stdout:5/672: mknod d7/cec 0 2024-07-03T15:43:56.741 INFO:tasks.workunit.client.1.smithi148.stdout:5/673: fdatasync d7/da/df/d16/d2f/d44/d52/fb9 0 2024-07-03T15:43:56.743 INFO:tasks.workunit.client.1.smithi148.stdout:9/904: rename d0/dd/d25/d37/d40/da3/f10b to d0/dd/d25/d37/d40/da3/f11e 0 2024-07-03T15:43:56.744 INFO:tasks.workunit.client.1.smithi148.stdout:5/674: dread d7/da/df/f71 [4194304,4194304] 0 2024-07-03T15:43:56.745 INFO:tasks.workunit.client.1.smithi148.stdout:5/675: stat d7/da/df/d16/d1a/de5 0 2024-07-03T15:43:56.745 INFO:tasks.workunit.client.1.smithi148.stdout:5/676: write d7/d13/d23/f78 [2558347,53515] 0 2024-07-03T15:43:56.745 INFO:tasks.workunit.client.1.smithi148.stdout:3/656: truncate dd/d33/f39 3665041 0 2024-07-03T15:43:56.747 INFO:tasks.workunit.client.1.smithi148.stdout:8/603: rmdir d4/da/d47 39 2024-07-03T15:43:56.751 INFO:tasks.workunit.client.1.smithi148.stdout:7/739: symlink d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/lfd 0 2024-07-03T15:43:56.753 INFO:tasks.workunit.client.1.smithi148.stdout:4/586: rename d0/d8/f2d to d0/d8/d3a/db9/fcf 0 2024-07-03T15:43:56.753 INFO:tasks.workunit.client.1.smithi148.stdout:4/587: creat d0/d91/da8/dab/d49/fd0 x:0 0 0 2024-07-03T15:43:56.754 INFO:tasks.workunit.client.1.smithi148.stdout:1/614: mknod d0/d2/d2e/d39/cd4 0 2024-07-03T15:43:56.756 INFO:tasks.workunit.client.1.smithi148.stdout:9/905: creat d0/d18/d24/f11f x:0 0 0 2024-07-03T15:43:56.758 INFO:tasks.workunit.client.1.smithi148.stdout:9/906: dread d0/fc5 [0,4194304] 0 2024-07-03T15:43:56.758 INFO:tasks.workunit.client.1.smithi148.stdout:9/907: chown d0/d49/d43/d19/d7a/l92 1 1 2024-07-03T15:43:56.758 INFO:tasks.workunit.client.1.smithi148.stdout:5/677: symlink d7/da/df/d16/dac/dc6/led 0 2024-07-03T15:43:56.760 INFO:tasks.workunit.client.1.smithi148.stdout:3/657: mkdir dd/d33/d37/d68/d2e/d40/d77/dcd 0 2024-07-03T15:43:56.761 INFO:tasks.workunit.client.1.smithi148.stdout:8/604: readlink d4/da/d14/d1a/d32/d38/d58/l65 0 2024-07-03T15:43:56.762 INFO:tasks.workunit.client.1.smithi148.stdout:4/588: dread d0/d28/d7d/f8a [0,4194304] 0 2024-07-03T15:43:56.766 INFO:tasks.workunit.client.1.smithi148.stdout:5/678: rename d7/da/df/d16/de4/fd8 to d7/d13/d23/fee 0 2024-07-03T15:43:56.769 INFO:tasks.workunit.client.1.smithi148.stdout:3/658: creat dd/d35/d41/d89/fce x:0 0 0 2024-07-03T15:43:56.769 INFO:tasks.workunit.client.1.smithi148.stdout:3/659: write dd/d33/d37/d68/d2e/d40/d45/d73/fb5 [554407,123088] 0 2024-07-03T15:43:56.769 INFO:tasks.workunit.client.1.smithi148.stdout:5/679: dread d7/da/df/d1d/f2e [0,4194304] 0 2024-07-03T15:43:56.769 INFO:tasks.workunit.client.1.smithi148.stdout:5/680: stat d7/da/df/d16/d2f/d44/d52/d60/fa6 0 2024-07-03T15:43:56.770 INFO:tasks.workunit.client.1.smithi148.stdout:5/681: read d7/da/df/d16/d2f/d44/d52/f90 [567747,94774] 0 2024-07-03T15:43:56.770 INFO:tasks.workunit.client.1.smithi148.stdout:8/605: chown d4/da/d14/d1a/d32/d38/d58/l7b 421 1 2024-07-03T15:43:56.770 INFO:tasks.workunit.client.1.smithi148.stdout:5/682: chown d7/da/df/d16/d2f/d44/ldf 112 1 2024-07-03T15:43:56.770 INFO:tasks.workunit.client.1.smithi148.stdout:5/683: chown d7/da/df/d16/d2f/d44/d52/d60/c95 134282371 1 2024-07-03T15:43:56.771 INFO:tasks.workunit.client.1.smithi148.stdout:4/589: creat d0/d8/d3a/db8/fd1 x:0 0 0 2024-07-03T15:43:56.773 INFO:tasks.workunit.client.1.smithi148.stdout:3/660: creat dd/fcf x:0 0 0 2024-07-03T15:43:56.773 INFO:tasks.workunit.client.1.smithi148.stdout:3/661: readlink dd/d33/d37/d68/d2e/d61/lb2 0 2024-07-03T15:43:56.774 INFO:tasks.workunit.client.1.smithi148.stdout:8/606: rmdir d4/da/d14/d3a/da8/db8 39 2024-07-03T15:43:56.776 INFO:tasks.workunit.client.1.smithi148.stdout:5/684: truncate d7/f19 1760377 0 2024-07-03T15:43:56.776 INFO:tasks.workunit.client.1.smithi148.stdout:5/685: fdatasync d7/da/df/f3d 0 2024-07-03T15:43:56.781 INFO:tasks.workunit.client.1.smithi148.stdout:3/662: dread dd/f44 [0,4194304] 0 2024-07-03T15:43:56.781 INFO:tasks.workunit.client.1.smithi148.stdout:5/686: mknod d7/da/cef 0 2024-07-03T15:43:56.781 INFO:tasks.workunit.client.1.smithi148.stdout:3/663: truncate dd/d33/d37/d68/d2e/f18 4845500 0 2024-07-03T15:43:56.781 INFO:tasks.workunit.client.1.smithi148.stdout:1/615: dwrite d0/fcb [4194304,4194304] 0 2024-07-03T15:43:56.781 INFO:tasks.workunit.client.1.smithi148.stdout:1/616: dread - d0/d2/d2e/d39/d41/d77/d80/fb5 zero size 2024-07-03T15:43:56.781 INFO:tasks.workunit.client.1.smithi148.stdout:1/617: creat d0/d2/d2e/d39/d41/fd5 x:0 0 0 2024-07-03T15:43:56.781 INFO:tasks.workunit.client.1.smithi148.stdout:7/740: dwrite d5/d7d/da7/f34 [0,4194304] 0 2024-07-03T15:43:56.782 INFO:tasks.workunit.client.1.smithi148.stdout:5/687: dread d7/d13/d23/f85 [0,4194304] 0 2024-07-03T15:43:56.783 INFO:tasks.workunit.client.1.smithi148.stdout:4/590: read d0/f4 [409674,12916] 0 2024-07-03T15:43:56.783 INFO:tasks.workunit.client.1.smithi148.stdout:4/591: stat d0/d28/d7d/d6f/d7c/l8e 0 2024-07-03T15:43:56.783 INFO:tasks.workunit.client.1.smithi148.stdout:4/592: readlink d0/l98 0 2024-07-03T15:43:56.783 INFO:tasks.workunit.client.1.smithi148.stdout:4/593: truncate d0/d28/da7/d2f/d41/f7a 949313 0 2024-07-03T15:43:56.789 INFO:tasks.workunit.client.1.smithi148.stdout:8/607: dwrite d4/da/d14/d1a/d32/d38/fd1 [0,4194304] 0 2024-07-03T15:43:56.790 INFO:tasks.workunit.client.1.smithi148.stdout:3/664: dread dd/d33/d37/d68/d2e/f18 [0,4194304] 0 2024-07-03T15:43:56.793 INFO:tasks.workunit.client.1.smithi148.stdout:7/741: rename d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/fce to d5/d7d/da7/d1c/d4a/d66/dd7/ffe 0 2024-07-03T15:43:56.796 INFO:tasks.workunit.client.1.smithi148.stdout:5/688: creat d7/da/df/d16/de4/da7/ff0 x:0 0 0 2024-07-03T15:43:56.796 INFO:tasks.workunit.client.1.smithi148.stdout:5/689: readlink d7/l8e 0 2024-07-03T15:43:56.797 INFO:tasks.workunit.client.1.smithi148.stdout:3/665: dread dd/d35/d41/d89/db0/fcb [0,4194304] 0 2024-07-03T15:43:56.797 INFO:tasks.workunit.client.1.smithi148.stdout:3/666: truncate dd/da6/f5e 1025737 0 2024-07-03T15:43:56.797 INFO:tasks.workunit.client.1.smithi148.stdout:4/594: symlink d0/d28/da7/dbd/ld2 0 2024-07-03T15:43:56.801 INFO:tasks.workunit.client.1.smithi148.stdout:8/608: mknod d4/da/d14/d3a/dab/cd9 0 2024-07-03T15:43:56.804 INFO:tasks.workunit.client.1.smithi148.stdout:7/742: symlink d5/d7d/da7/d1c/d33/d4c/d21/d47/d8b/dd3/lff 0 2024-07-03T15:43:56.806 INFO:tasks.workunit.client.1.smithi148.stdout:5/690: truncate d7/da/df/d16/d1a/f7a 177434 0 2024-07-03T15:43:56.806 INFO:tasks.workunit.client.1.smithi148.stdout:5/691: creat d7/da/df/d16/de4/da7/ff1 x:0 0 0 2024-07-03T15:43:56.808 INFO:tasks.workunit.client.1.smithi148.stdout:3/667: symlink dd/d33/d37/d8d/ld0 0 2024-07-03T15:43:56.811 INFO:tasks.workunit.client.1.smithi148.stdout:8/609: creat d4/da/d47/d88/d92/fda x:0 0 0 2024-07-03T15:43:56.813 INFO:tasks.workunit.client.1.smithi148.stdout:7/743: symlink d5/d7d/da7/dc7/l100 0 2024-07-03T15:43:56.813 INFO:tasks.workunit.client.1.smithi148.stdout:4/595: dwrite d0/d28/da7/d62/fa1 [0,4194304] 0 2024-07-03T15:43:56.817 INFO:tasks.workunit.client.1.smithi148.stdout:5/692: mkdir d7/da/df/d16/d2f/d44/d52/d60/df2 0 2024-07-03T15:43:56.820 INFO:tasks.workunit.client.1.smithi148.stdout:0/569: sync 2024-07-03T15:43:56.822 INFO:tasks.workunit.client.1.smithi148.stdout:7/744: write d5/f9 [898336,21769] 0 2024-07-03T15:43:56.825 INFO:tasks.workunit.client.1.smithi148.stdout:3/668: rename dd/d33/d37/d68/f3c to dd/d33/d37/d68/d2e/d40/d8c/dbc/fd1 0 2024-07-03T15:43:56.827 INFO:tasks.workunit.client.1.smithi148.stdout:8/610: mknod d4/da/d51/d53/cdb 0 2024-07-03T15:43:56.833 INFO:tasks.workunit.client.1.smithi148.stdout:4/596: mkdir d0/d8/d3a/db8/dd3 0 2024-07-03T15:43:56.836 INFO:tasks.workunit.client.1.smithi148.stdout:5/693: mknod d7/da/df/d16/d2f/d44/d52/cf3 0 2024-07-03T15:43:56.836 INFO:tasks.workunit.client.1.smithi148.stdout:5/694: fdatasync d7/d13/d54/f7d 0 2024-07-03T15:43:56.839 INFO:tasks.workunit.client.1.smithi148.stdout:0/570: creat da/d44/d4a/fbc x:0 0 0 2024-07-03T15:43:56.839 INFO:tasks.workunit.client.1.smithi148.stdout:0/571: write da/d44/d13/d1d/d3e/d85/f91 [306332,119429] 0 2024-07-03T15:43:56.844 INFO:tasks.workunit.client.1.smithi148.stdout:3/669: dwrite dd/d33/d37/d68/d67/fc7 [0,4194304] 0 2024-07-03T15:43:56.844 INFO:tasks.workunit.client.1.smithi148.stdout:3/670: creat dd/d35/d41/d89/fd2 x:0 0 0 2024-07-03T15:43:56.844 INFO:tasks.workunit.client.1.smithi148.stdout:3/671: creat dd/d35/d41/d89/fd3 x:0 0 0 2024-07-03T15:43:56.845 INFO:tasks.workunit.client.1.smithi148.stdout:7/745: rename d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/db2/dd6 to d5/d7d/da7/d1c/d101 0 2024-07-03T15:43:56.847 INFO:tasks.workunit.client.1.smithi148.stdout:8/611: dwrite d4/da/d47/d88/d92/f39 [0,4194304] 0 2024-07-03T15:43:56.855 INFO:tasks.workunit.client.1.smithi148.stdout:0/572: dwrite da/d44/d4a/fbc [0,4194304] 0 2024-07-03T15:43:56.857 INFO:tasks.workunit.client.1.smithi148.stdout:3/672: creat dd/d33/d37/d68/d2e/d40/d8c/fd4 x:0 0 0 2024-07-03T15:43:56.857 INFO:tasks.workunit.client.1.smithi148.stdout:3/673: fsync dd/d33/d37/d68/d2e/d61/f63 0 2024-07-03T15:43:56.857 INFO:tasks.workunit.client.1.smithi148.stdout:3/674: fdatasync dd/d33/d37/d68/d2e/d40/d45/d9a/fab 0 2024-07-03T15:43:56.858 INFO:tasks.workunit.client.1.smithi148.stdout:3/675: dread dd/d33/d37/d68/d2e/d61/f63 [0,4194304] 0 2024-07-03T15:43:56.858 INFO:tasks.workunit.client.1.smithi148.stdout:3/676: chown dd/d33/d37/d8d/ld0 73674 1 2024-07-03T15:43:56.858 INFO:tasks.workunit.client.1.smithi148.stdout:3/677: truncate dd/d33/d52/f60 3354558 0 2024-07-03T15:43:56.859 INFO:tasks.workunit.client.1.smithi148.stdout:4/597: rename d0/d91/da8/dab/dad/daf to d0/d8/dd4 0 2024-07-03T15:43:56.859 INFO:tasks.workunit.client.1.smithi148.stdout:4/598: dread - d0/d91/da8/dab/d49/fd0 zero size 2024-07-03T15:43:56.862 INFO:tasks.workunit.client.1.smithi148.stdout:7/746: mknod d5/d7d/da7/d1c/d4a/d52/c102 0 2024-07-03T15:43:56.875 INFO:tasks.workunit.client.1.smithi148.stdout:8/612: truncate d4/da/d14/d1a/d32/fae 1913685 0 2024-07-03T15:43:56.879 INFO:tasks.workunit.client.1.smithi148.stdout:5/695: rmdir d7/da/df/d16/dac 39 2024-07-03T15:43:56.882 INFO:tasks.workunit.client.1.smithi148.stdout:7/747: dwrite d5/d7d/da7/d1c/d59/f7c [0,4194304] 0 2024-07-03T15:43:56.882 INFO:tasks.workunit.client.1.smithi148.stdout:7/748: write d5/d7d/da7/d1c/d33/d4c/d21/d39/dd2/fe4 [209937,23213] 0 2024-07-03T15:43:56.882 INFO:tasks.workunit.client.1.smithi148.stdout:3/678: dwrite dd/d33/d37/d68/f5c [0,4194304] 0 2024-07-03T15:43:56.891 INFO:tasks.workunit.client.1.smithi148.stdout:8/613: dwrite d4/da/d14/d1a/d32/d38/dba/fc0 [0,4194304] 0 2024-07-03T15:43:56.891 INFO:tasks.workunit.client.1.smithi148.stdout:8/614: creat d4/d86/fdc x:0 0 0 2024-07-03T15:43:56.893 INFO:tasks.workunit.client.1.smithi148.stdout:4/599: mknod d0/cd5 0 2024-07-03T15:43:56.893 INFO:tasks.workunit.client.1.smithi148.stdout:5/696: rmdir d7/da/df/d16/dac/dc1 39 2024-07-03T15:43:56.894 INFO:tasks.workunit.client.1.smithi148.stdout:4/600: write d0/d28/d7d/f8a [1629497,34076] 0 2024-07-03T15:43:56.896 INFO:tasks.workunit.client.1.smithi148.stdout:7/749: symlink d5/d7d/da7/d1c/d33/d4c/d21/d47/l103 0 2024-07-03T15:43:56.896 INFO:tasks.workunit.client.1.smithi148.stdout:7/750: fdatasync d5/f57 0 2024-07-03T15:43:56.899 INFO:tasks.workunit.client.1.smithi148.stdout:3/679: symlink dd/ld5 0 2024-07-03T15:43:56.901 INFO:tasks.workunit.client.1.smithi148.stdout:7/751: write d5/d7d/da7/d1c/d33/f67 [66923,32508] 0 2024-07-03T15:43:56.901 INFO:tasks.workunit.client.1.smithi148.stdout:7/752: chown d5/d7d/da7/d1c/d33/d4c/d21/f3c 1560682886 1 2024-07-03T15:43:56.906 INFO:tasks.workunit.client.1.smithi148.stdout:8/615: dwrite d4/da/d14/d1a/d32/d38/fd1 [4194304,4194304] 0 2024-07-03T15:43:56.906 INFO:tasks.workunit.client.1.smithi148.stdout:2/715: sync 2024-07-03T15:43:56.906 INFO:tasks.workunit.client.1.smithi148.stdout:1/618: sync 2024-07-03T15:43:56.906 INFO:tasks.workunit.client.1.smithi148.stdout:9/908: sync 2024-07-03T15:43:56.906 INFO:tasks.workunit.client.1.smithi148.stdout:6/726: sync 2024-07-03T15:43:56.906 INFO:tasks.workunit.client.1.smithi148.stdout:8/616: write d4/da/d47/d88/faa [421137,73919] 0 2024-07-03T15:43:56.906 INFO:tasks.workunit.client.1.smithi148.stdout:8/617: write d4/d86/f97 [669665,120332] 0 2024-07-03T15:43:56.907 INFO:tasks.workunit.client.1.smithi148.stdout:7/753: dread d5/f58 [0,4194304] 0 2024-07-03T15:43:56.913 INFO:tasks.workunit.client.1.smithi148.stdout:3/680: write dd/d33/d37/d68/d67/fc7 [3849183,26657] 0 2024-07-03T15:43:56.917 INFO:tasks.workunit.client.1.smithi148.stdout:5/697: mknod d7/da/dbe/de6/cf4 0 2024-07-03T15:43:56.917 INFO:tasks.workunit.client.1.smithi148.stdout:4/601: link d0/d28/da7/d2f/d41/lbb d0/d28/d7d/d6f/d7c/dae/dba/ld6 0 2024-07-03T15:43:56.919 INFO:tasks.workunit.client.1.smithi148.stdout:7/754: dread d5/d7d/da7/d1c/d33/d4c/d84/d98/dc2/fc9 [0,4194304] 0 2024-07-03T15:43:56.920 INFO:tasks.workunit.client.1.smithi148.stdout:2/716: dread d5/d14/d22/d60/d76/f92 [0,4194304] 0 2024-07-03T15:43:56.920 INFO:tasks.workunit.client.1.smithi148.stdout:9/909: getdents d0/dd/d25/d37/d40/d72 0 2024-07-03T15:43:56.920 INFO:tasks.workunit.client.1.smithi148.stdout:2/717: dread - d5/d14/d86/db9/dbb/d13/d41/d7b/ddf/d79/d9c/fbe zero size 2024-07-03T15:43:56.920 INFO:tasks.workunit.client.1.smithi148.stdout:9/910: write d0/dd/d25/d59/dcb/f111 [4419611,29785] 0 2024-07-03T15:43:56.920 INFO:tasks.workunit.client.1.smithi148.stdout:9/911: chown d0/d49/la0 54 1 2024-07-03T15:43:56.922 INFO:tasks.workunit.client.1.smithi148.stdout:8/618: link d4/da/f29 d4/da/d14/d1a/d32/d38/d58/d8e/db9/fdd 0 2024-07-03T15:43:56.926 INFO:tasks.workunit.client.1.smithi148.stdout:3/681: symlink dd/d33/d37/d68/d2e/ld6 0 2024-07-03T15:43:56.926 INFO:tasks.workunit.client.1.smithi148.stdout:6/727: dwrite d3/d1d/d28/d3e/ddd/d74/dc4/d85/fc7 [0,4194304] 0 2024-07-03T15:43:56.929 INFO:tasks.workunit.client.1.smithi148.stdout:5/698: dread d7/da/df/f71 [0,4194304] 0 2024-07-03T15:43:56.929 INFO:tasks.workunit.client.1.smithi148.stdout:5/699: write d7/da/df/d16/de4/da7/ff0 [172422,103538] 0 2024-07-03T15:43:56.929 INFO:tasks.workunit.client.1.smithi148.stdout:0/573: rename da/d44 to da/d58/dbd 0 2024-07-03T15:43:56.929 INFO:tasks.workunit.client.1.smithi148.stdout:5/700: read d7/da/df/d16/f7e [424455,97252] 0 2024-07-03T15:43:56.936 INFO:tasks.workunit.client.1.smithi148.stdout:4/602: creat d0/d8/d3a/d9d/fd7 x:0 0 0 2024-07-03T15:43:56.937 INFO:tasks.workunit.client.1.smithi148.stdout:7/755: mknod d5/d7d/da7/d1c/d33/d4c/d21/d47/c104 0 2024-07-03T15:43:56.939 INFO:tasks.workunit.client.1.smithi148.stdout:9/912: symlink d0/dd/d25/dcc/l120 0 2024-07-03T15:43:56.940 INFO:tasks.workunit.client.1.smithi148.stdout:9/913: chown d0/dd/cbf 719 1 2024-07-03T15:43:56.942 INFO:tasks.workunit.client.1.smithi148.stdout:8/619: creat d4/da/d51/d18/d20/fde x:0 0 0 2024-07-03T15:43:56.942 INFO:tasks.workunit.client.1.smithi148.stdout:8/620: write d4/da/d51/d18/d20/fde [169972,44556] 0 2024-07-03T15:43:56.942 INFO:tasks.workunit.client.1.smithi148.stdout:8/621: chown d4/l7c 130590 1 2024-07-03T15:43:56.942 INFO:tasks.workunit.client.1.smithi148.stdout:8/622: chown d4/da/d14/d1a/d32/c8a 1583 1 2024-07-03T15:43:56.944 INFO:tasks.workunit.client.1.smithi148.stdout:3/682: unlink dd/da6/l8a 0 2024-07-03T15:43:56.945 INFO:tasks.workunit.client.1.smithi148.stdout:2/718: dwrite d5/d14/d86/db9/dbb/d13/d1d/f43 [0,4194304] 0 2024-07-03T15:43:56.945 INFO:tasks.workunit.client.1.smithi148.stdout:5/701: dwrite d7/da/df/d16/d2f/d44/f99 [0,4194304] 0 2024-07-03T15:43:56.948 INFO:tasks.workunit.client.1.smithi148.stdout:0/574: mkdir da/d58/dbd/d13/d14/dbe 0 2024-07-03T15:43:56.951 INFO:tasks.workunit.client.1.smithi148.stdout:9/914: write d0/d49/d43/d19/d7a/daa/fb1 [3935987,12523] 0 2024-07-03T15:43:56.956 INFO:tasks.workunit.client.1.smithi148.stdout:7/756: rmdir d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/db2/dca 39 2024-07-03T15:43:56.956 INFO:tasks.workunit.client.1.smithi148.stdout:7/757: creat d5/d7d/da7/db4/f105 x:0 0 0 2024-07-03T15:43:56.956 INFO:tasks.workunit.client.1.smithi148.stdout:2/719: dread d5/d14/f7a [0,4194304] 0 2024-07-03T15:43:56.957 INFO:tasks.workunit.client.1.smithi148.stdout:2/720: read d5/d14/d86/db9/dbb/d13/d2d/fb8 [3448782,75814] 0 2024-07-03T15:43:56.957 INFO:tasks.workunit.client.1.smithi148.stdout:3/683: creat dd/da6/d8b/fd7 x:0 0 0 2024-07-03T15:43:56.957 INFO:tasks.workunit.client.1.smithi148.stdout:3/684: rename dd to dd/d33/d37/d68/d2e/d40/d45/dd8 22 2024-07-03T15:43:56.959 INFO:tasks.workunit.client.1.smithi148.stdout:8/623: dwrite d4/da/f28 [0,4194304] 0 2024-07-03T15:43:56.960 INFO:tasks.workunit.client.1.smithi148.stdout:8/624: stat d4/da/d51/d53/cdb 0 2024-07-03T15:43:56.963 INFO:tasks.workunit.client.1.smithi148.stdout:5/702: dwrite d7/da/df/d16/d2f/d44/fbd [0,4194304] 0 2024-07-03T15:43:56.965 INFO:tasks.workunit.client.1.smithi148.stdout:0/575: mknod da/d58/dbd/d13/d14/d50/d95/d8c/cbf 0 2024-07-03T15:43:56.968 INFO:tasks.workunit.client.1.smithi148.stdout:9/915: mknod d0/dd/d25/d37/c121 0 2024-07-03T15:43:56.973 INFO:tasks.workunit.client.1.smithi148.stdout:7/758: symlink d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/l106 0 2024-07-03T15:43:56.979 INFO:tasks.workunit.client.1.smithi148.stdout:2/721: getdents d5/da8 0 2024-07-03T15:43:56.979 INFO:tasks.workunit.client.1.smithi148.stdout:5/703: dwrite d7/da/df/fcb [0,4194304] 0 2024-07-03T15:43:56.981 INFO:tasks.workunit.client.1.smithi148.stdout:3/685: getdents dd/d33/d37/d68/d2e/dc3 0 2024-07-03T15:43:56.983 INFO:tasks.workunit.client.1.smithi148.stdout:8/625: creat d4/da/fdf x:0 0 0 2024-07-03T15:43:56.986 INFO:tasks.workunit.client.1.smithi148.stdout:0/576: rmdir da/d58/dbd/d13/d5a 39 2024-07-03T15:43:56.987 INFO:tasks.workunit.client.1.smithi148.stdout:0/577: chown da/d58/daa 185318 1 2024-07-03T15:43:56.988 INFO:tasks.workunit.client.1.smithi148.stdout:9/916: link d0/d49/d43/d19/d7a/daa/cc3 d0/d18/dcf/c122 0 2024-07-03T15:43:56.988 INFO:tasks.workunit.client.1.smithi148.stdout:9/917: creat d0/dd/f123 x:0 0 0 2024-07-03T15:43:56.990 INFO:tasks.workunit.client.1.smithi148.stdout:7/759: rename d5/d7d/da7/d1c/d4a/d52/d87 to d5/d7d/da7/d1c/d4a/d52/d107 0 2024-07-03T15:43:56.990 INFO:tasks.workunit.client.1.smithi148.stdout:7/760: read d5/d7d/da7/d1c/d4a/f62 [63615,47920] 0 2024-07-03T15:43:56.991 INFO:tasks.workunit.client.1.smithi148.stdout:7/761: rename d5/d7d/da7 to d5/d7d/da7/d1c/d4a/d52/d107/d108 22 2024-07-03T15:43:56.992 INFO:tasks.workunit.client.1.smithi148.stdout:2/722: creat d5/d14/d86/db9/dbb/d13/d1d/da1/fe5 x:0 0 0 2024-07-03T15:43:56.992 INFO:tasks.workunit.client.1.smithi148.stdout:2/723: chown d5/d14/d86/db9/dbb/d13/d41/f49 249795 1 2024-07-03T15:43:56.992 INFO:tasks.workunit.client.1.smithi148.stdout:5/704: dwrite d7/d13/d23/fee [0,4194304] 0 2024-07-03T15:43:56.999 INFO:tasks.workunit.client.1.smithi148.stdout:3/686: truncate dd/f31 2751903 0 2024-07-03T15:43:56.999 INFO:tasks.workunit.client.1.smithi148.stdout:3/687: stat dd/d33/d37/d68/d2e/d40/l86 0 2024-07-03T15:43:56.999 INFO:tasks.workunit.client.1.smithi148.stdout:3/688: write dd/d35/f3f [4926345,44226] 0 2024-07-03T15:43:57.000 INFO:tasks.workunit.client.1.smithi148.stdout:9/918: dread d0/d18/d24/d5a/fe9 [0,4194304] 0 2024-07-03T15:43:57.001 INFO:tasks.workunit.client.1.smithi148.stdout:8/626: readlink d4/da/d14/d1a/lb4 0 2024-07-03T15:43:57.001 INFO:tasks.workunit.client.1.smithi148.stdout:8/627: read d4/da/d51/d18/d20/f66 [311201,120447] 0 2024-07-03T15:43:57.002 INFO:tasks.workunit.client.1.smithi148.stdout:8/628: truncate d4/da/d47/d88/faa 1139038 0 2024-07-03T15:43:57.002 INFO:tasks.workunit.client.1.smithi148.stdout:8/629: stat d4/da/d51/f57 0 2024-07-03T15:43:57.003 INFO:tasks.workunit.client.1.smithi148.stdout:5/705: dread d7/f4f [4194304,4194304] 0 2024-07-03T15:43:57.004 INFO:tasks.workunit.client.1.smithi148.stdout:5/706: write d7/da/df/d16/de4/da7/ff1 [215725,8483] 0 2024-07-03T15:43:57.004 INFO:tasks.workunit.client.1.smithi148.stdout:5/707: chown d7/d13/d54/c94 1739578 1 2024-07-03T15:43:57.004 INFO:tasks.workunit.client.1.smithi148.stdout:0/578: dwrite da/d17/f1e [0,4194304] 0 2024-07-03T15:43:57.009 INFO:tasks.workunit.client.1.smithi148.stdout:7/762: rename d5/d7d/da7/ff6 to d5/d7d/da7/d1c/d59/f109 0 2024-07-03T15:43:57.014 INFO:tasks.workunit.client.1.smithi148.stdout:5/708: read d7/da/df/d16/d6a/db4/fd9 [2375346,127909] 0 2024-07-03T15:43:57.015 INFO:tasks.workunit.client.1.smithi148.stdout:3/689: dwrite dd/d33/d37/d68/d2e/d61/f76 [0,4194304] 0 2024-07-03T15:43:57.015 INFO:tasks.workunit.client.1.smithi148.stdout:9/919: mkdir d0/d49/d124 0 2024-07-03T15:43:57.019 INFO:tasks.workunit.client.1.smithi148.stdout:8/630: unlink d4/da/d47/d88/d92/f42 0 2024-07-03T15:43:57.019 INFO:tasks.workunit.client.1.smithi148.stdout:8/631: fsync d4/da/d14/d1a/d32/d38/dba/fc0 0 2024-07-03T15:43:57.019 INFO:tasks.workunit.client.1.smithi148.stdout:8/632: chown d4/da/d14/d17/fc8 244264 1 2024-07-03T15:43:57.019 INFO:tasks.workunit.client.1.smithi148.stdout:8/633: truncate d4/da/d14/d1a/d32/d89/fbb 688110 0 2024-07-03T15:43:57.020 INFO:tasks.workunit.client.1.smithi148.stdout:0/579: truncate da/d58/dbd/d13/d14/d50/f60 3468051 0 2024-07-03T15:43:57.020 INFO:tasks.workunit.client.1.smithi148.stdout:0/580: stat da/d58/dbd/d13/d1d/d3e/d85 0 2024-07-03T15:43:57.025 INFO:tasks.workunit.client.1.smithi148.stdout:5/709: symlink d7/da/df/d1d/lf5 0 2024-07-03T15:43:57.026 INFO:tasks.workunit.client.1.smithi148.stdout:9/920: read d0/dd/d25/d37/d40/f62 [3457726,97651] 0 2024-07-03T15:43:57.026 INFO:tasks.workunit.client.1.smithi148.stdout:9/921: creat d0/dd/f125 x:0 0 0 2024-07-03T15:43:57.032 INFO:tasks.workunit.client.1.smithi148.stdout:2/724: rename d5/l31 to d5/d67/dc3/ddd/le6 0 2024-07-03T15:43:57.034 INFO:tasks.workunit.client.1.smithi148.stdout:5/710: creat d7/da/df/d16/dac/dc1/ff6 x:0 0 0 2024-07-03T15:43:57.034 INFO:tasks.workunit.client.1.smithi148.stdout:5/711: write f4 [19104,21338] 0 2024-07-03T15:43:57.045 INFO:tasks.workunit.client.1.smithi148.stdout:2/725: dread d5/d14/d86/db9/dbb/f42 [0,4194304] 0 2024-07-03T15:43:57.045 INFO:tasks.workunit.client.1.smithi148.stdout:2/726: write d5/d14/d22/d60/d76/f92 [4239152,10623] 0 2024-07-03T15:43:57.048 INFO:tasks.workunit.client.1.smithi148.stdout:2/727: rename d5/d14/d86/db9/dbb/d68/dcd/f35 to d5/d14/fe7 0 2024-07-03T15:43:57.050 INFO:tasks.workunit.client.1.smithi148.stdout:7/763: dwrite d5/d7d/da7/d1c/d4a/d66/d93/f9d [0,4194304] 0 2024-07-03T15:43:57.051 INFO:tasks.workunit.client.0.smithi028.stdout:1/110: mkdir d4/d2c 0 2024-07-03T15:43:57.051 INFO:tasks.workunit.client.0.smithi028.stdout:1/111: write d4/de/f1f [521288,115493] 0 2024-07-03T15:43:57.051 INFO:tasks.workunit.client.0.smithi028.stdout:1/112: chown d4/d18/f2a 699718 1 2024-07-03T15:43:57.051 INFO:tasks.workunit.client.0.smithi028.stdout:1/113: creat d4/d18/f2d x:0 0 0 2024-07-03T15:43:57.051 INFO:tasks.workunit.client.0.smithi028.stdout:1/114: read d4/de/f1f [416330,39447] 0 2024-07-03T15:43:57.056 INFO:tasks.workunit.client.1.smithi148.stdout:7/764: truncate d5/d7d/da7/d1c/d33/d4c/d21/d31/f45 665103 0 2024-07-03T15:43:57.057 INFO:tasks.workunit.client.1.smithi148.stdout:3/690: dwrite dd/da6/da2/fb9 [0,4194304] 0 2024-07-03T15:43:57.059 INFO:tasks.workunit.client.1.smithi148.stdout:9/922: dwrite d0/dd/d25/fe8 [0,4194304] 0 2024-07-03T15:43:57.059 INFO:tasks.workunit.client.1.smithi148.stdout:9/923: creat d0/d49/f126 x:0 0 0 2024-07-03T15:43:57.059 INFO:tasks.workunit.client.1.smithi148.stdout:9/924: write d0/d49/d43/d19/d7a/daa/fb1 [2185700,54169] 0 2024-07-03T15:43:57.059 INFO:tasks.workunit.client.1.smithi148.stdout:8/634: dwrite d4/d86/fb3 [0,4194304] 0 2024-07-03T15:43:57.060 INFO:tasks.workunit.client.1.smithi148.stdout:5/712: dwrite d7/d13/d54/f4b [4194304,4194304] 0 2024-07-03T15:43:57.060 INFO:tasks.workunit.client.1.smithi148.stdout:5/713: write d7/da/df/d1d/f2e [1933837,44057] 0 2024-07-03T15:43:57.060 INFO:tasks.workunit.client.1.smithi148.stdout:5/714: fsync d7/da/df/d16/d6a/db4/f9c 0 2024-07-03T15:43:57.060 INFO:tasks.workunit.client.1.smithi148.stdout:0/581: dwrite da/d58/dbd/d13/d14/d50/d95/d8c/f45 [4194304,4194304] 0 2024-07-03T15:43:57.061 INFO:tasks.workunit.client.1.smithi148.stdout:7/765: link d5/d7d/da7/d1c/d33/d4c/d21/d39/f77 d5/d7d/da7/d1c/d4a/d66/f10a 0 2024-07-03T15:43:57.061 INFO:tasks.workunit.client.1.smithi148.stdout:0/582: read da/d58/dbd/d13/d1d/d3e/d85/f91 [147549,96717] 0 2024-07-03T15:43:57.061 INFO:tasks.workunit.client.1.smithi148.stdout:0/583: readlink da/d58/dbd/l20 0 2024-07-03T15:43:57.070 INFO:tasks.workunit.client.1.smithi148.stdout:7/766: dread d5/d7d/da7/f34 [0,4194304] 0 2024-07-03T15:43:57.070 INFO:tasks.workunit.client.1.smithi148.stdout:7/767: stat d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/cc0 0 2024-07-03T15:43:57.071 INFO:tasks.workunit.client.1.smithi148.stdout:7/768: write d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/db2/fde [259739,111218] 0 2024-07-03T15:43:57.071 INFO:tasks.workunit.client.1.smithi148.stdout:8/635: write d4/da/d14/d1a/d32/d38/fd1 [3016108,67104] 0 2024-07-03T15:43:57.077 INFO:tasks.workunit.client.1.smithi148.stdout:9/925: dwrite d0/d49/d43/d19/f116 [0,4194304] 0 2024-07-03T15:43:57.079 INFO:tasks.workunit.client.1.smithi148.stdout:9/926: write d0/d18/f26 [141014,111967] 0 2024-07-03T15:43:57.086 INFO:tasks.workunit.client.1.smithi148.stdout:8/636: dwrite d4/da/d14/d1a/d32/d38/dba/f77 [0,4194304] 0 2024-07-03T15:43:57.089 INFO:tasks.workunit.client.1.smithi148.stdout:3/691: creat dd/d33/d37/d68/d2e/d40/d45/d65/fd9 x:0 0 0 2024-07-03T15:43:57.089 INFO:tasks.workunit.client.1.smithi148.stdout:3/692: creat dd/d33/d52/fda x:0 0 0 2024-07-03T15:43:57.089 INFO:tasks.workunit.client.1.smithi148.stdout:3/693: truncate dd/d35/d41/d89/fd2 518873 0 2024-07-03T15:43:57.099 INFO:tasks.workunit.client.1.smithi148.stdout:3/694: write dd/f2a [4936394,121020] 0 2024-07-03T15:43:57.103 INFO:tasks.workunit.client.1.smithi148.stdout:8/637: dwrite f2 [0,4194304] 0 2024-07-03T15:43:57.108 INFO:tasks.workunit.client.0.smithi028.stdout:4/4: sync 2024-07-03T15:43:57.108 INFO:tasks.workunit.client.0.smithi028.stdout:6/11: sync 2024-07-03T15:43:57.109 INFO:tasks.workunit.client.0.smithi028.stdout:2/42: sync 2024-07-03T15:43:57.109 INFO:tasks.workunit.client.0.smithi028.stdout:4/5: truncate - no filename 2024-07-03T15:43:57.109 INFO:tasks.workunit.client.0.smithi028.stdout:4/6: dwrite - no filename 2024-07-03T15:43:57.109 INFO:tasks.workunit.client.0.smithi028.stdout:4/7: fsync - no filename 2024-07-03T15:43:57.109 INFO:tasks.workunit.client.0.smithi028.stdout:5/90: sync 2024-07-03T15:43:57.109 INFO:tasks.workunit.client.0.smithi028.stdout:0/16: sync 2024-07-03T15:43:57.109 INFO:tasks.workunit.client.0.smithi028.stdout:4/8: chown . 1 1 2024-07-03T15:43:57.109 INFO:tasks.workunit.client.0.smithi028.stdout:6/12: stat f0 0 2024-07-03T15:43:57.109 INFO:tasks.workunit.client.0.smithi028.stdout:4/9: chown . 1271581 1 2024-07-03T15:43:57.109 INFO:tasks.workunit.client.0.smithi028.stdout:4/10: dwrite - no filename 2024-07-03T15:43:57.109 INFO:tasks.workunit.client.0.smithi028.stdout:4/11: dwrite - no filename 2024-07-03T15:43:57.109 INFO:tasks.workunit.client.0.smithi028.stdout:4/12: rename - no filename 2024-07-03T15:43:57.109 INFO:tasks.workunit.client.0.smithi028.stdout:4/13: getdents . 0 2024-07-03T15:43:57.110 INFO:tasks.workunit.client.0.smithi028.stdout:3/62: sync 2024-07-03T15:43:57.110 INFO:tasks.workunit.client.0.smithi028.stdout:3/63: rmdir - no directory 2024-07-03T15:43:57.110 INFO:tasks.workunit.client.0.smithi028.stdout:8/59: sync 2024-07-03T15:43:57.110 INFO:tasks.workunit.client.0.smithi028.stdout:9/76: sync 2024-07-03T15:43:57.110 INFO:tasks.workunit.client.0.smithi028.stdout:7/9: sync 2024-07-03T15:43:57.110 INFO:tasks.workunit.client.1.smithi148.stdout:5/715: unlink d7/da/dbe/ccc 0 2024-07-03T15:43:57.113 INFO:tasks.workunit.client.1.smithi148.stdout:0/584: mknod da/d36/cc0 0 2024-07-03T15:43:57.113 INFO:tasks.workunit.client.1.smithi148.stdout:0/585: chown da/d36/cc0 8 1 2024-07-03T15:43:57.115 INFO:tasks.workunit.client.1.smithi148.stdout:7/769: creat d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/db1/f10b x:0 0 0 2024-07-03T15:43:57.120 INFO:tasks.workunit.client.0.smithi028.stdout:7/10: dread f0 [0,4194304] 0 2024-07-03T15:43:57.121 INFO:tasks.workunit.client.1.smithi148.stdout:9/927: mkdir d0/dd/d25/d39/d127 0 2024-07-03T15:43:57.126 INFO:tasks.workunit.client.1.smithi148.stdout:3/695: unlink dd/d33/d37/d68/d2e/d61/f76 0 2024-07-03T15:43:57.126 INFO:tasks.workunit.client.0.smithi028.stdout:7/11: write f0 [4144963,7468] 0 2024-07-03T15:43:57.127 INFO:tasks.workunit.client.0.smithi028.stdout:1/115: creat d4/d11/d16/f2e x:0 0 0 2024-07-03T15:43:57.129 INFO:tasks.workunit.client.0.smithi028.stdout:5/91: rmdir d0/d1c 0 2024-07-03T15:43:57.131 INFO:tasks.workunit.client.0.smithi028.stdout:2/43: rename da/fe to da/ff 0 2024-07-03T15:43:57.132 INFO:tasks.workunit.client.1.smithi148.stdout:5/716: symlink d7/lf7 0 2024-07-03T15:43:57.132 INFO:tasks.workunit.client.1.smithi148.stdout:5/717: chown d7/d13/d54/dd2 1149649184 1 2024-07-03T15:43:57.134 INFO:tasks.workunit.client.1.smithi148.stdout:6/728: sync 2024-07-03T15:43:57.134 INFO:tasks.workunit.client.1.smithi148.stdout:4/603: sync 2024-07-03T15:43:57.134 INFO:tasks.workunit.client.1.smithi148.stdout:1/619: sync 2024-07-03T15:43:57.134 INFO:tasks.workunit.client.1.smithi148.stdout:7/770: write d5/d7d/da7/d1c/d33/f48 [3137136,81461] 0 2024-07-03T15:43:57.134 INFO:tasks.workunit.client.1.smithi148.stdout:1/620: read - d0/d2/d2e/d39/d42/d62/d7b/fa4 zero size 2024-07-03T15:43:57.134 INFO:tasks.workunit.client.1.smithi148.stdout:6/729: write d3/d1d/f38 [4025824,83390] 0 2024-07-03T15:43:57.134 INFO:tasks.workunit.client.1.smithi148.stdout:4/604: write d0/d28/da7/d62/fa1 [4758767,93901] 0 2024-07-03T15:43:57.134 INFO:tasks.workunit.client.1.smithi148.stdout:6/730: readlink d3/daa/ld1 0 2024-07-03T15:43:57.135 INFO:tasks.workunit.client.1.smithi148.stdout:6/731: fdatasync d3/d1d/d28/d3e/ddd/fb1 0 2024-07-03T15:43:57.135 INFO:tasks.workunit.client.1.smithi148.stdout:4/605: creat d0/d28/d7d/d6f/d7c/da3/fd8 x:0 0 0 2024-07-03T15:43:57.135 INFO:tasks.workunit.client.1.smithi148.stdout:6/732: chown d3/d1d/d28/d3e/ddd/lb3 33 1 2024-07-03T15:43:57.135 INFO:tasks.workunit.client.1.smithi148.stdout:4/606: fsync d0/d28/da7/d62/f6b 0 2024-07-03T15:43:57.135 INFO:tasks.workunit.client.1.smithi148.stdout:6/733: dread - d3/d1d/d28/d53/d6d/fba zero size 2024-07-03T15:43:57.135 INFO:tasks.workunit.client.1.smithi148.stdout:6/734: write d3/da5/fb9 [4313556,71522] 0 2024-07-03T15:43:57.136 INFO:tasks.workunit.client.0.smithi028.stdout:6/13: mkdir d4 0 2024-07-03T15:43:57.136 INFO:tasks.workunit.client.0.smithi028.stdout:6/14: read - f2 zero size 2024-07-03T15:43:57.136 INFO:tasks.workunit.client.0.smithi028.stdout:9/77: dwrite d1/dd/f13 [0,4194304] 0 2024-07-03T15:43:57.136 INFO:tasks.workunit.client.0.smithi028.stdout:9/78: readlink d1/l4 0 2024-07-03T15:43:57.136 INFO:tasks.workunit.client.0.smithi028.stdout:0/17: creat d2/d3/f7 x:0 0 0 2024-07-03T15:43:57.139 INFO:tasks.workunit.client.0.smithi028.stdout:9/79: dread d1/dd/f14 [0,4194304] 0 2024-07-03T15:43:57.141 INFO:tasks.workunit.client.1.smithi148.stdout:0/586: rmdir da/d58/dbd/d13/d14/d35/d8a/dab 39 2024-07-03T15:43:57.141 INFO:tasks.workunit.client.1.smithi148.stdout:0/587: write da/d17/f76 [2132417,10070] 0 2024-07-03T15:43:57.143 INFO:tasks.workunit.client.1.smithi148.stdout:1/621: dread d0/f45 [0,4194304] 0 2024-07-03T15:43:57.143 INFO:tasks.workunit.client.0.smithi028.stdout:4/14: mknod c0 0 2024-07-03T15:43:57.143 INFO:tasks.workunit.client.0.smithi028.stdout:4/15: readlink - no filename 2024-07-03T15:43:57.143 INFO:tasks.workunit.client.0.smithi028.stdout:4/16: rmdir - no directory 2024-07-03T15:43:57.145 INFO:tasks.workunit.client.1.smithi148.stdout:9/928: link d0/dd/d11/f35 d0/f128 0 2024-07-03T15:43:57.148 INFO:tasks.workunit.client.1.smithi148.stdout:3/696: mkdir dd/d33/d37/d68/d2e/ddb 0 2024-07-03T15:43:57.149 INFO:tasks.workunit.client.0.smithi028.stdout:8/60: creat d0/d2/d13/f17 x:0 0 0 2024-07-03T15:43:57.150 INFO:tasks.workunit.client.0.smithi028.stdout:8/61: truncate d0/d2/d13/f17 92643 0 2024-07-03T15:43:57.150 INFO:tasks.workunit.client.0.smithi028.stdout:9/80: dread d1/f2 [0,4194304] 0 2024-07-03T15:43:57.151 INFO:tasks.workunit.client.1.smithi148.stdout:8/638: mkdir d4/da/d14/d1a/d32/d38/d58/de0 0 2024-07-03T15:43:57.156 INFO:tasks.workunit.client.1.smithi148.stdout:8/639: dread d4/da/d51/d53/f7f [0,4194304] 0 2024-07-03T15:43:57.156 INFO:tasks.workunit.client.1.smithi148.stdout:3/697: write dd/d35/d81/fca [326263,36087] 0 2024-07-03T15:43:57.156 INFO:tasks.workunit.client.0.smithi028.stdout:1/116: rename d4/d11/d16/l1d to d4/d18/l2f 0 2024-07-03T15:43:57.158 INFO:tasks.workunit.client.0.smithi028.stdout:6/15: mknod d4/c5 0 2024-07-03T15:43:57.158 INFO:tasks.workunit.client.0.smithi028.stdout:6/16: fdatasync f2 0 2024-07-03T15:43:57.158 INFO:tasks.workunit.client.0.smithi028.stdout:6/17: truncate f2 506073 0 2024-07-03T15:43:57.164 INFO:tasks.workunit.client.1.smithi148.stdout:4/607: creat d0/d8/dd4/fd9 x:0 0 0 2024-07-03T15:43:57.164 INFO:tasks.workunit.client.1.smithi148.stdout:4/608: fsync d0/d91/da8/dab/d49/fd0 0 2024-07-03T15:43:57.165 INFO:tasks.workunit.client.0.smithi028.stdout:4/17: creat f1 x:0 0 0 2024-07-03T15:43:57.165 INFO:tasks.workunit.client.0.smithi028.stdout:4/18: chown f1 10 1 2024-07-03T15:43:57.165 INFO:tasks.workunit.client.0.smithi028.stdout:4/19: dread - f1 zero size 2024-07-03T15:43:57.165 INFO:tasks.workunit.client.0.smithi028.stdout:4/20: truncate f1 929069 0 2024-07-03T15:43:57.167 INFO:tasks.workunit.client.0.smithi028.stdout:3/64: rename fc to f17 0 2024-07-03T15:43:57.169 INFO:tasks.workunit.client.0.smithi028.stdout:3/65: write f2 [363270,113206] 0 2024-07-03T15:43:57.170 INFO:tasks.workunit.client.0.smithi028.stdout:8/62: mknod d0/d2/d3/dc/c18 0 2024-07-03T15:43:57.172 INFO:tasks.workunit.client.0.smithi028.stdout:9/81: mkdir d1/d16 0 2024-07-03T15:43:57.176 INFO:tasks.workunit.client.0.smithi028.stdout:6/18: unlink l1 0 2024-07-03T15:43:57.178 INFO:tasks.workunit.client.0.smithi028.stdout:4/21: mkdir d2 0 2024-07-03T15:43:57.179 INFO:tasks.workunit.client.0.smithi028.stdout:8/63: mknod d0/d2/c19 0 2024-07-03T15:43:57.181 INFO:tasks.workunit.client.0.smithi028.stdout:1/117: rmdir d4/de 39 2024-07-03T15:43:57.181 INFO:tasks.workunit.client.0.smithi028.stdout:1/118: fdatasync d4/f26 0 2024-07-03T15:43:57.183 INFO:tasks.workunit.client.1.smithi148.stdout:5/718: link d7/da/df/d16/dac/dc6/cce d7/da/d21/cf8 0 2024-07-03T15:43:57.183 INFO:tasks.workunit.client.1.smithi148.stdout:5/719: read - d7/da/df/d16/d2f/f8b zero size 2024-07-03T15:43:57.183 INFO:tasks.workunit.client.1.smithi148.stdout:5/720: chown d7/da/df/d16/cd5 111 1 2024-07-03T15:43:57.183 INFO:tasks.workunit.client.1.smithi148.stdout:5/721: fdatasync d7/d13/d54/fd6 0 2024-07-03T15:43:57.183 INFO:tasks.workunit.client.0.smithi028.stdout:6/19: symlink d4/l6 0 2024-07-03T15:43:57.187 INFO:tasks.workunit.client.0.smithi028.stdout:1/119: link d4/de/f10 d4/d2c/f30 0 2024-07-03T15:43:57.187 INFO:tasks.workunit.client.0.smithi028.stdout:1/120: write d4/d18/f2a [219888,63869] 0 2024-07-03T15:43:57.188 INFO:tasks.workunit.client.1.smithi148.stdout:3/698: mkdir dd/d33/d37/d68/d2e/d40/d77/d8f/ddc 0 2024-07-03T15:43:57.192 INFO:tasks.workunit.client.1.smithi148.stdout:0/588: link da/d58/dbd/d13/d14/d50/d95/d8c/f45 da/d17/d2a/d2f/fc1 0 2024-07-03T15:43:57.192 INFO:tasks.workunit.client.1.smithi148.stdout:0/589: chown da/d58/dbd/d13/d1d/d3e/d85 2523 1 2024-07-03T15:43:57.193 INFO:tasks.workunit.client.0.smithi028.stdout:1/121: rename d4/d11/d16/f2e to d4/de/f31 0 2024-07-03T15:43:57.195 INFO:tasks.workunit.client.1.smithi148.stdout:3/699: rmdir dd/d33/d37/d68/d67 39 2024-07-03T15:43:57.195 INFO:tasks.workunit.client.1.smithi148.stdout:3/700: readlink dd/l96 0 2024-07-03T15:43:57.195 INFO:tasks.workunit.client.0.smithi028.stdout:1/122: rename d4/c13 to d4/d2c/c32 0 2024-07-03T15:43:57.196 INFO:tasks.workunit.client.0.smithi028.stdout:1/123: creat d4/d18/f33 x:0 0 0 2024-07-03T15:43:57.197 INFO:tasks.workunit.client.0.smithi028.stdout:1/124: symlink d4/de/d1e/l34 0 2024-07-03T15:43:57.198 INFO:tasks.workunit.client.1.smithi148.stdout:5/722: dread d7/da/df/f10 [0,4194304] 0 2024-07-03T15:43:57.198 INFO:tasks.workunit.client.1.smithi148.stdout:5/723: chown d7/lf7 46169 1 2024-07-03T15:43:57.198 INFO:tasks.workunit.client.1.smithi148.stdout:5/724: chown d7/da/df/d16/d6a/db4/lc0 3271456 1 2024-07-03T15:43:57.198 INFO:tasks.workunit.client.1.smithi148.stdout:3/701: creat dd/d33/d37/d68/d2e/d40/d77/d8f/fdd x:0 0 0 2024-07-03T15:43:57.199 INFO:tasks.workunit.client.0.smithi028.stdout:1/125: mknod d4/c35 0 2024-07-03T15:43:57.199 INFO:tasks.workunit.client.0.smithi028.stdout:1/126: read d4/d18/f2a [247659,6216] 0 2024-07-03T15:43:57.200 INFO:tasks.workunit.client.1.smithi148.stdout:3/702: creat dd/d33/d37/d68/d67/fde x:0 0 0 2024-07-03T15:43:57.200 INFO:tasks.workunit.client.1.smithi148.stdout:3/703: chown dd/d33/f3e 43 1 2024-07-03T15:43:57.202 INFO:tasks.workunit.client.0.smithi028.stdout:3/66: read f6 [3395679,109762] 0 2024-07-03T15:43:57.202 INFO:tasks.workunit.client.0.smithi028.stdout:3/67: readlink la 0 2024-07-03T15:43:57.202 INFO:tasks.workunit.client.0.smithi028.stdout:3/68: fdatasync f17 0 2024-07-03T15:43:57.202 INFO:tasks.workunit.client.0.smithi028.stdout:3/69: chown f17 1151205 1 2024-07-03T15:43:57.202 INFO:tasks.workunit.client.0.smithi028.stdout:3/70: write f6 [178298,32619] 0 2024-07-03T15:43:57.203 INFO:tasks.workunit.client.0.smithi028.stdout:2/44: dwrite f4 [0,4194304] 0 2024-07-03T15:43:57.203 INFO:tasks.workunit.client.1.smithi148.stdout:3/704: link dd/f22 dd/d33/fdf 0 2024-07-03T15:43:57.203 INFO:tasks.workunit.client.1.smithi148.stdout:3/705: getdents dd/da6/da2/dbb 0 2024-07-03T15:43:57.204 INFO:tasks.workunit.client.1.smithi148.stdout:3/706: rename dd/d35/d41/l51 to dd/d33/d37/d68/le0 0 2024-07-03T15:43:57.207 INFO:tasks.workunit.client.1.smithi148.stdout:3/707: mkdir dd/d35/de1 0 2024-07-03T15:43:57.207 INFO:tasks.workunit.client.0.smithi028.stdout:3/71: mkdir d18 0 2024-07-03T15:43:57.208 INFO:tasks.workunit.client.0.smithi028.stdout:3/72: symlink d18/l19 0 2024-07-03T15:43:57.208 INFO:tasks.workunit.client.0.smithi028.stdout:3/73: fdatasync f15 0 2024-07-03T15:43:57.209 INFO:tasks.workunit.client.1.smithi148.stdout:5/725: dread d7/da/df/d16/d2f/d44/d52/f82 [0,4194304] 0 2024-07-03T15:43:57.209 INFO:tasks.workunit.client.1.smithi148.stdout:5/726: chown d7/da/cef 114471 1 2024-07-03T15:43:57.209 INFO:tasks.workunit.client.1.smithi148.stdout:5/727: creat d7/da/df/d1d/ff9 x:0 0 0 2024-07-03T15:43:57.210 INFO:tasks.workunit.client.1.smithi148.stdout:3/708: link dd/f36 dd/da6/fe2 0 2024-07-03T15:43:57.210 INFO:tasks.workunit.client.0.smithi028.stdout:3/74: mkdir d18/d1a 0 2024-07-03T15:43:57.211 INFO:tasks.workunit.client.0.smithi028.stdout:3/75: mknod d18/c1b 0 2024-07-03T15:43:57.212 INFO:tasks.workunit.client.0.smithi028.stdout:3/76: write f9 [1019132,68360] 0 2024-07-03T15:43:57.213 INFO:tasks.workunit.client.1.smithi148.stdout:5/728: mknod d7/da/df/d16/cfa 0 2024-07-03T15:43:57.213 INFO:tasks.workunit.client.1.smithi148.stdout:5/729: write f3 [1700833,64749] 0 2024-07-03T15:43:57.215 INFO:tasks.workunit.client.1.smithi148.stdout:5/730: mkdir d7/da/df/d16/dfb 0 2024-07-03T15:43:57.216 INFO:tasks.workunit.client.1.smithi148.stdout:5/731: mknod d7/da/df/d16/d2f/d44/cfc 0 2024-07-03T15:43:57.217 INFO:tasks.workunit.client.1.smithi148.stdout:5/732: rename d7/da/f77 to d7/da/df/d16/dac/dc1/dc4/ffd 0 2024-07-03T15:43:57.218 INFO:tasks.workunit.client.1.smithi148.stdout:5/733: chown d7/da/df/d16/f1b 22 1 2024-07-03T15:43:57.222 INFO:tasks.workunit.client.1.smithi148.stdout:0/590: dread da/d58/dbd/d13/d14/d35/d8a/dab/f72 [0,4194304] 0 2024-07-03T15:43:57.224 INFO:tasks.workunit.client.1.smithi148.stdout:0/591: link da/d58/dbd/d13/fb8 da/d17/d2a/d2f/fc2 0 2024-07-03T15:43:57.225 INFO:tasks.workunit.client.1.smithi148.stdout:0/592: mkdir da/d58/dbd/d13/d14/dbe/dc3 0 2024-07-03T15:43:57.225 INFO:tasks.workunit.client.1.smithi148.stdout:0/593: read - da/d38/f4e zero size 2024-07-03T15:43:57.226 INFO:tasks.workunit.client.1.smithi148.stdout:0/594: fsync da/d17/f8b 0 2024-07-03T15:43:57.226 INFO:tasks.workunit.client.1.smithi148.stdout:0/595: chown da/d58/daa 164104 1 2024-07-03T15:43:57.227 INFO:tasks.workunit.client.1.smithi148.stdout:0/596: unlink da/d17/d2a/f7a 0 2024-07-03T15:43:57.229 INFO:tasks.workunit.client.1.smithi148.stdout:3/709: write dd/f3d [2229304,57656] 0 2024-07-03T15:43:57.230 INFO:tasks.workunit.client.1.smithi148.stdout:0/597: rename da/d58/dbd/d13/d14/d50/d95/d98 to da/d36/dc4 0 2024-07-03T15:43:57.230 INFO:tasks.workunit.client.1.smithi148.stdout:0/598: chown da/d38 645145995 1 2024-07-03T15:43:57.230 INFO:tasks.workunit.client.1.smithi148.stdout:0/599: stat da/d58/dbd/d94/dac 0 2024-07-03T15:43:57.230 INFO:tasks.workunit.client.1.smithi148.stdout:0/600: fsync da/d17/f76 0 2024-07-03T15:43:57.233 INFO:tasks.workunit.client.1.smithi148.stdout:4/609: dwrite d0/d28/da7/d62/fa1 [4194304,4194304] 0 2024-07-03T15:43:57.233 INFO:tasks.workunit.client.1.smithi148.stdout:4/610: stat d0/d28/d7d/d6f/d7c/da3 0 2024-07-03T15:43:57.233 INFO:tasks.workunit.client.1.smithi148.stdout:4/611: readlink d0/d28/d6e/dc7/la0 0 2024-07-03T15:43:57.233 INFO:tasks.workunit.client.1.smithi148.stdout:4/612: truncate d0/d28/d7d/d6f/d7c/da3/fca 320144 0 2024-07-03T15:43:57.233 INFO:tasks.workunit.client.1.smithi148.stdout:0/601: link da/d58/dbd/d24/c25 da/d58/dbd/d13/d1d/da1/cc5 0 2024-07-03T15:43:57.233 INFO:tasks.workunit.client.1.smithi148.stdout:0/602: write da/d58/dbd/d13/d14/d50/f83 [4310442,17524] 0 2024-07-03T15:43:57.234 INFO:tasks.workunit.client.1.smithi148.stdout:8/640: dwrite d4/da/d47/d88/d92/fa3 [0,4194304] 0 2024-07-03T15:43:57.234 INFO:tasks.workunit.client.1.smithi148.stdout:4/613: symlink d0/d28/da7/d2f/d41/lda 0 2024-07-03T15:43:57.237 INFO:tasks.workunit.client.1.smithi148.stdout:8/641: symlink d4/da/d14/d1a/le1 0 2024-07-03T15:43:57.238 INFO:tasks.workunit.client.1.smithi148.stdout:8/642: creat d4/da/d14/d3a/da8/db8/fe2 x:0 0 0 2024-07-03T15:43:57.238 INFO:tasks.workunit.client.1.smithi148.stdout:8/643: write d4/da/d47/d88/faf [538732,65120] 0 2024-07-03T15:43:57.238 INFO:tasks.workunit.client.1.smithi148.stdout:8/644: fdatasync d4/d86/f5c 0 2024-07-03T15:43:57.239 INFO:tasks.workunit.client.0.smithi028.stdout:0/18: dwrite d2/d3/f7 [0,4194304] 0 2024-07-03T15:43:57.239 INFO:tasks.workunit.client.0.smithi028.stdout:8/64: dwrite d0/d2/d3/dc/d11/d12/f15 [0,4194304] 0 2024-07-03T15:43:57.239 INFO:tasks.workunit.client.0.smithi028.stdout:4/22: dwrite f1 [0,4194304] 0 2024-07-03T15:43:57.240 INFO:tasks.workunit.client.1.smithi148.stdout:8/645: symlink d4/da/d14/d1a/d32/d38/d58/dcb/le3 0 2024-07-03T15:43:57.240 INFO:tasks.workunit.client.1.smithi148.stdout:8/646: dread - d4/da/d47/d88/d92/fda zero size 2024-07-03T15:43:57.240 INFO:tasks.workunit.client.1.smithi148.stdout:0/603: fsync da/d58/dbd/d13/d14/d50/f83 0 2024-07-03T15:43:57.240 INFO:tasks.workunit.client.0.smithi028.stdout:6/20: dwrite f2 [0,4194304] 0 2024-07-03T15:43:57.241 INFO:tasks.workunit.client.0.smithi028.stdout:1/127: dwrite d4/fd [0,4194304] 0 2024-07-03T15:43:57.241 INFO:tasks.workunit.client.0.smithi028.stdout:9/82: dwrite d1/dd/fe [0,4194304] 0 2024-07-03T15:43:57.241 INFO:tasks.workunit.client.1.smithi148.stdout:8/647: rename d4/da/d14/d1a/d32/d38/d58/d8e/lb1 to d4/da/d14/d3a/le4 0 2024-07-03T15:43:57.242 INFO:tasks.workunit.client.1.smithi148.stdout:0/604: creat da/d58/d90/db4/fc6 x:0 0 0 2024-07-03T15:43:57.242 INFO:tasks.workunit.client.1.smithi148.stdout:0/605: write da/d58/dbd/d13/d14/f68 [1196135,53010] 0 2024-07-03T15:43:57.244 INFO:tasks.workunit.client.1.smithi148.stdout:8/648: rename d4/da/d51/c22 to d4/ce5 0 2024-07-03T15:43:57.245 INFO:tasks.workunit.client.1.smithi148.stdout:8/649: creat d4/da/d14/d3a/da8/dd3/fe6 x:0 0 0 2024-07-03T15:43:57.246 INFO:tasks.workunit.client.0.smithi028.stdout:6/21: link c3 d4/c7 0 2024-07-03T15:43:57.247 INFO:tasks.workunit.client.0.smithi028.stdout:1/128: symlink d4/de/d1c/l36 0 2024-07-03T15:43:57.248 INFO:tasks.workunit.client.1.smithi148.stdout:8/650: mknod d4/da/ce7 0 2024-07-03T15:43:57.281 INFO:tasks.workunit.client.1.smithi148.stdout:8/651: dwrite d4/da/d14/d1a/d32/d38/dba/f77 [4194304,4194304] 0 2024-07-03T15:43:57.283 INFO:tasks.workunit.client.1.smithi148.stdout:8/652: creat d4/da/d51/fe8 x:0 0 0 2024-07-03T15:43:57.284 INFO:tasks.workunit.client.1.smithi148.stdout:8/653: link d4/da/d14/d1a/d32/d38/dba/la4 d4/da/d14/d1a/d32/d38/d58/dcb/le9 0 2024-07-03T15:43:57.284 INFO:tasks.workunit.client.1.smithi148.stdout:8/654: write d4/da/d14/d17/fd5 [797860,114469] 0 2024-07-03T15:43:57.284 INFO:tasks.workunit.client.1.smithi148.stdout:8/655: chown d4/da/d47/d88/d92 20274175 1 2024-07-03T15:43:57.285 INFO:tasks.workunit.client.1.smithi148.stdout:8/656: link d4/da/d51/d18/d20/d82/fbd d4/da/d51/fea 0 2024-07-03T15:43:57.286 INFO:tasks.workunit.client.1.smithi148.stdout:8/657: getdents d4/da/d14/d1a/d32/d89 0 2024-07-03T15:43:57.287 INFO:tasks.workunit.client.1.smithi148.stdout:8/658: truncate d4/da/d51/f8 3232700 0 2024-07-03T15:43:57.326 INFO:tasks.workunit.client.0.smithi028.stdout:5/92: dwrite d0/dd/df/f16 [0,4194304] 0 2024-07-03T15:43:57.326 INFO:tasks.workunit.client.0.smithi028.stdout:7/12: dwrite f0 [4194304,4194304] 0 2024-07-03T15:43:57.326 INFO:tasks.workunit.client.0.smithi028.stdout:7/13: creat f2 x:0 0 0 2024-07-03T15:43:57.329 INFO:tasks.workunit.client.1.smithi148.stdout:1/622: dwrite d0/d2/d2e/d39/d42/d62/f71 [0,4194304] 0 2024-07-03T15:43:57.329 INFO:tasks.workunit.client.1.smithi148.stdout:1/623: chown d0/d2/d6/d14/d29 2 1 2024-07-03T15:43:57.329 INFO:tasks.workunit.client.1.smithi148.stdout:1/624: chown d0/d2/d11/d1e/ca2 73446 1 2024-07-03T15:43:57.331 INFO:tasks.workunit.client.0.smithi028.stdout:1/129: dwrite d4/f29 [0,4194304] 0 2024-07-03T15:43:57.331 INFO:tasks.workunit.client.0.smithi028.stdout:1/130: creat d4/f37 x:0 0 0 2024-07-03T15:43:57.337 INFO:tasks.workunit.client.1.smithi148.stdout:5/734: dwrite d7/da/df/d16/d1a/f34 [0,4194304] 0 2024-07-03T15:43:57.337 INFO:tasks.workunit.client.1.smithi148.stdout:5/735: write d7/d13/d23/fee [4887262,129643] 0 2024-07-03T15:43:57.337 INFO:tasks.workunit.client.1.smithi148.stdout:5/736: truncate d7/d13/d54/feb 678407 0 2024-07-03T15:43:57.338 INFO:tasks.workunit.client.1.smithi148.stdout:0/606: dwrite da/d58/dbd/d13/d14/fb9 [0,4194304] 0 2024-07-03T15:43:57.340 INFO:tasks.workunit.client.0.smithi028.stdout:2/45: getdents da 0 2024-07-03T15:43:57.341 INFO:tasks.workunit.client.1.smithi148.stdout:3/710: dwrite dd/d33/d37/d68/d2e/d40/f80 [0,4194304] 0 2024-07-03T15:43:57.343 INFO:tasks.workunit.client.0.smithi028.stdout:6/22: dread f0 [0,4194304] 0 2024-07-03T15:43:57.348 INFO:tasks.workunit.client.0.smithi028.stdout:1/131: mknod d4/d11/d16/c38 0 2024-07-03T15:43:57.352 INFO:tasks.workunit.client.0.smithi028.stdout:4/23: dwrite f1 [0,4194304] 0 2024-07-03T15:43:57.352 INFO:tasks.workunit.client.0.smithi028.stdout:2/46: creat da/f10 x:0 0 0 2024-07-03T15:43:57.353 INFO:tasks.workunit.client.1.smithi148.stdout:4/614: dwrite d0/d91/da8/dab/f75 [0,4194304] 0 2024-07-03T15:43:57.357 INFO:tasks.workunit.client.0.smithi028.stdout:4/24: mknod d2/c3 0 2024-07-03T15:43:57.358 INFO:tasks.workunit.client.0.smithi028.stdout:2/47: truncate da/ff 338464 0 2024-07-03T15:43:57.359 INFO:tasks.workunit.client.0.smithi028.stdout:5/93: dwrite d0/dd/d13/d19/f21 [0,4194304] 0 2024-07-03T15:43:57.359 INFO:tasks.workunit.client.1.smithi148.stdout:1/625: creat d0/d2/d2e/d39/d42/daf/fd6 x:0 0 0 2024-07-03T15:43:57.360 INFO:tasks.workunit.client.1.smithi148.stdout:1/626: chown d0/d2/d11/d1e/d9d/dc0/cc8 9 1 2024-07-03T15:43:57.361 INFO:tasks.workunit.client.0.smithi028.stdout:4/25: creat d2/f4 x:0 0 0 2024-07-03T15:43:57.362 INFO:tasks.workunit.client.0.smithi028.stdout:2/48: creat da/f11 x:0 0 0 2024-07-03T15:43:57.363 INFO:tasks.workunit.client.0.smithi028.stdout:1/132: dread f2 [0,4194304] 0 2024-07-03T15:43:57.364 INFO:tasks.workunit.client.0.smithi028.stdout:8/65: dwrite d0/d2/ff [0,4194304] 0 2024-07-03T15:43:57.366 INFO:tasks.workunit.client.1.smithi148.stdout:5/737: symlink d7/da/df/d16/de4/lfe 0 2024-07-03T15:43:57.366 INFO:tasks.workunit.client.1.smithi148.stdout:5/738: fdatasync d7/da/df/d1d/f51 0 2024-07-03T15:43:57.367 INFO:tasks.workunit.client.0.smithi028.stdout:5/94: dread d0/dd/df/f16 [0,4194304] 0 2024-07-03T15:43:57.367 INFO:tasks.workunit.client.0.smithi028.stdout:5/95: write d0/dd/d13/f17 [4701613,19676] 0 2024-07-03T15:43:57.367 INFO:tasks.workunit.client.0.smithi028.stdout:5/96: chown d0/dd/df/c12 1807902009 1 2024-07-03T15:43:57.368 INFO:tasks.workunit.client.0.smithi028.stdout:2/49: creat da/f12 x:0 0 0 2024-07-03T15:43:57.369 INFO:tasks.workunit.client.0.smithi028.stdout:2/50: write f7 [825054,120053] 0 2024-07-03T15:43:57.370 INFO:tasks.workunit.client.0.smithi028.stdout:2/51: write da/ff [220148,54621] 0 2024-07-03T15:43:57.370 INFO:tasks.workunit.client.0.smithi028.stdout:2/52: write da/f10 [178929,31278] 0 2024-07-03T15:43:57.370 INFO:tasks.workunit.client.0.smithi028.stdout:2/53: readlink l1 0 2024-07-03T15:43:57.370 INFO:tasks.workunit.client.0.smithi028.stdout:1/133: rmdir d4/d2c 39 2024-07-03T15:43:57.371 INFO:tasks.workunit.client.0.smithi028.stdout:1/134: write d4/d18/f2d [510101,78310] 0 2024-07-03T15:43:57.375 INFO:tasks.workunit.client.1.smithi148.stdout:3/711: rename dd/da6/d8b/fd7 to dd/d33/d37/d68/d2e/d40/d45/d9a/fe3 0 2024-07-03T15:43:57.376 INFO:tasks.workunit.client.1.smithi148.stdout:0/607: dwrite da/f11 [4194304,4194304] 0 2024-07-03T15:43:57.376 INFO:tasks.workunit.client.0.smithi028.stdout:9/83: dwrite d1/f2 [0,4194304] 0 2024-07-03T15:43:57.377 INFO:tasks.workunit.client.0.smithi028.stdout:9/84: getdents d1/d10 0 2024-07-03T15:43:57.377 INFO:tasks.workunit.client.0.smithi028.stdout:6/23: dwrite f0 [0,4194304] 0 2024-07-03T15:43:57.377 INFO:tasks.workunit.client.0.smithi028.stdout:6/24: rename d4 to d4/d8 22 2024-07-03T15:43:57.377 INFO:tasks.workunit.client.0.smithi028.stdout:6/25: rename d4 to d4/d9 22 2024-07-03T15:43:57.380 INFO:tasks.workunit.client.1.smithi148.stdout:5/739: dread d7/da/df/d16/d2f/d44/fbd [0,4194304] 0 2024-07-03T15:43:57.381 INFO:tasks.workunit.client.0.smithi028.stdout:2/54: unlink da/f11 0 2024-07-03T15:43:57.388 INFO:tasks.workunit.client.1.smithi148.stdout:4/615: unlink d0/f6d 0 2024-07-03T15:43:57.388 INFO:tasks.workunit.client.1.smithi148.stdout:4/616: write d0/d8/d44/f6a [9163155,9405] 0 2024-07-03T15:43:57.392 INFO:tasks.workunit.client.1.smithi148.stdout:3/712: read dd/f2a [3927394,99478] 0 2024-07-03T15:43:57.393 INFO:tasks.workunit.client.1.smithi148.stdout:1/627: getdents d0/de 0 2024-07-03T15:43:57.396 INFO:tasks.workunit.client.1.smithi148.stdout:0/608: creat da/d17/d2a/d2f/d93/da3/fc7 x:0 0 0 2024-07-03T15:43:57.397 INFO:tasks.workunit.client.1.smithi148.stdout:1/628: dread d0/d2/d2e/d39/f5f [0,4194304] 0 2024-07-03T15:43:57.398 INFO:tasks.workunit.client.1.smithi148.stdout:5/740: mkdir d7/da/df/d16/d2f/dff 0 2024-07-03T15:43:57.400 INFO:tasks.workunit.client.0.smithi028.stdout:9/85: dwrite d1/f5 [0,4194304] 0 2024-07-03T15:43:57.400 INFO:tasks.workunit.client.1.smithi148.stdout:4/617: creat d0/d8/dd4/fdb x:0 0 0 2024-07-03T15:43:57.401 INFO:tasks.workunit.client.1.smithi148.stdout:4/618: chown d0/d28/da7/dbd/ld2 173 1 2024-07-03T15:43:57.401 INFO:tasks.workunit.client.1.smithi148.stdout:3/713: symlink dd/d33/d37/d68/d2e/d40/d8c/dbc/le4 0 2024-07-03T15:43:57.403 INFO:tasks.workunit.client.1.smithi148.stdout:3/714: write dd/d33/f4b [93255,93270] 0 2024-07-03T15:43:57.403 INFO:tasks.workunit.client.1.smithi148.stdout:1/629: mknod d0/de/d44/d54/d5d/cd7 0 2024-07-03T15:43:57.403 INFO:tasks.workunit.client.0.smithi028.stdout:1/135: dwrite d4/f19 [0,4194304] 0 2024-07-03T15:43:57.406 INFO:tasks.workunit.client.1.smithi148.stdout:5/741: rename d7/da/d21/fd4 to d7/da/df/d16/d1a/de5/f100 0 2024-07-03T15:43:57.407 INFO:tasks.workunit.client.1.smithi148.stdout:4/619: rmdir d0/d91 39 2024-07-03T15:43:57.410 INFO:tasks.workunit.client.0.smithi028.stdout:9/86: dread d1/f2 [0,4194304] 0 2024-07-03T15:43:57.411 INFO:tasks.workunit.client.1.smithi148.stdout:3/715: unlink dd/da6/f5e 0 2024-07-03T15:43:57.411 INFO:tasks.workunit.client.1.smithi148.stdout:3/716: chown dd/d33/d52/c90 346098 1 2024-07-03T15:43:57.413 INFO:tasks.workunit.client.0.smithi028.stdout:9/87: mkdir d1/d17 0 2024-07-03T15:43:57.413 INFO:tasks.workunit.client.1.smithi148.stdout:5/742: creat d7/d59/f101 x:0 0 0 2024-07-03T15:43:57.414 INFO:tasks.workunit.client.1.smithi148.stdout:5/743: write d7/da/df/d16/dac/dc1/dc4/ffd [324444,50532] 0 2024-07-03T15:43:57.415 INFO:tasks.workunit.client.1.smithi148.stdout:4/620: creat d0/d28/d7d/d6f/d7c/dae/dba/fdc x:0 0 0 2024-07-03T15:43:57.415 INFO:tasks.workunit.client.1.smithi148.stdout:4/621: read - d0/d8/d3a/d9d/fd7 zero size 2024-07-03T15:43:57.415 INFO:tasks.workunit.client.1.smithi148.stdout:4/622: stat d0/d8/c18 0 2024-07-03T15:43:57.415 INFO:tasks.workunit.client.1.smithi148.stdout:4/623: fsync d0/fbf 0 2024-07-03T15:43:57.420 INFO:tasks.workunit.client.1.smithi148.stdout:4/624: rename d0/d28/da7/d62/fa1 to d0/d28/d7d/fdd 0 2024-07-03T15:43:57.420 INFO:tasks.workunit.client.1.smithi148.stdout:4/625: write d0/d28/db0/fcb [77463,39596] 0 2024-07-03T15:43:57.422 INFO:tasks.workunit.client.1.smithi148.stdout:3/717: dread dd/d33/d37/d68/d2e/d40/f5d [0,4194304] 0 2024-07-03T15:43:57.423 INFO:tasks.workunit.client.0.smithi028.stdout:6/26: dwrite f2 [0,4194304] 0 2024-07-03T15:43:57.426 INFO:tasks.workunit.client.1.smithi148.stdout:5/744: dread d7/d13/d23/f3b [0,4194304] 0 2024-07-03T15:43:57.426 INFO:tasks.workunit.client.1.smithi148.stdout:5/745: creat d7/d13/d54/f102 x:0 0 0 2024-07-03T15:43:57.426 INFO:tasks.workunit.client.0.smithi028.stdout:6/27: unlink c3 0 2024-07-03T15:43:57.430 INFO:tasks.workunit.client.0.smithi028.stdout:6/28: unlink d4/c5 0 2024-07-03T15:43:57.431 INFO:tasks.workunit.client.1.smithi148.stdout:4/626: dread d0/d8/f10 [0,4194304] 0 2024-07-03T15:43:57.432 INFO:tasks.workunit.client.1.smithi148.stdout:3/718: rmdir dd/d33/d37/dc4 0 2024-07-03T15:43:57.435 INFO:tasks.workunit.client.0.smithi028.stdout:6/29: mkdir d4/da 0 2024-07-03T15:43:57.435 INFO:tasks.workunit.client.0.smithi028.stdout:2/55: dwrite f4 [0,4194304] 0 2024-07-03T15:43:57.441 INFO:tasks.workunit.client.1.smithi148.stdout:1/630: dwrite d0/de/d44/d54/d5d/fc1 [0,4194304] 0 2024-07-03T15:43:57.441 INFO:tasks.workunit.client.1.smithi148.stdout:3/719: truncate dd/d33/d37/d68/f5c 3235227 0 2024-07-03T15:43:57.442 INFO:tasks.workunit.client.0.smithi028.stdout:1/136: dwrite d4/de/f10 [0,4194304] 0 2024-07-03T15:43:57.442 INFO:tasks.workunit.client.0.smithi028.stdout:1/137: creat d4/f39 x:0 0 0 2024-07-03T15:43:57.442 INFO:tasks.workunit.client.0.smithi028.stdout:1/138: write d4/d11/f1b [1010890,34] 0 2024-07-03T15:43:57.442 INFO:tasks.workunit.client.1.smithi148.stdout:5/746: rmdir d7/da/df/d16/d2f/dc2 39 2024-07-03T15:43:57.444 INFO:tasks.workunit.client.0.smithi028.stdout:6/30: write f0 [2000205,29447] 0 2024-07-03T15:43:57.444 INFO:tasks.workunit.client.0.smithi028.stdout:9/88: dwrite d1/f2 [0,4194304] 0 2024-07-03T15:43:57.450 INFO:tasks.workunit.client.0.smithi028.stdout:1/139: mkdir d4/d2c/d3a 0 2024-07-03T15:43:57.451 INFO:tasks.workunit.client.0.smithi028.stdout:0/19: truncate d2/d3/f7 1778704 0 2024-07-03T15:43:57.457 INFO:tasks.workunit.client.1.smithi148.stdout:1/631: rename d0/de/d44/d54/d5d/f8c to d0/d2/d11/d1e/d9d/dc2/fd8 0 2024-07-03T15:43:57.458 INFO:tasks.workunit.client.0.smithi028.stdout:6/31: mkdir d4/db 0 2024-07-03T15:43:57.460 INFO:tasks.workunit.client.1.smithi148.stdout:3/720: mknod dd/d33/d37/d68/d2e/d61/ce5 0 2024-07-03T15:43:57.463 INFO:tasks.workunit.client.0.smithi028.stdout:9/89: link d1/f9 d1/d10/f18 0 2024-07-03T15:43:57.469 INFO:tasks.workunit.client.1.smithi148.stdout:5/747: rename d7/da/df/d16/d2f/d44/d52/fb9 to d7/da/df/d16/d2f/d44/d52/d60/f103 0 2024-07-03T15:43:57.471 INFO:tasks.workunit.client.1.smithi148.stdout:1/632: rmdir d0/d2/d2e/d39 39 2024-07-03T15:43:57.471 INFO:tasks.workunit.client.1.smithi148.stdout:1/633: fsync d0/d2/d2e/d39/d41/f43 0 2024-07-03T15:43:57.473 INFO:tasks.workunit.client.0.smithi028.stdout:6/32: mknod d4/da/cc 0 2024-07-03T15:43:57.473 INFO:tasks.workunit.client.0.smithi028.stdout:6/33: read f0 [3059475,119754] 0 2024-07-03T15:43:57.473 INFO:tasks.workunit.client.0.smithi028.stdout:6/34: readlink d4/l6 0 2024-07-03T15:43:57.476 INFO:tasks.workunit.client.1.smithi148.stdout:5/748: symlink d7/da/df/d16/de4/da7/l104 0 2024-07-03T15:43:57.476 INFO:tasks.workunit.client.1.smithi148.stdout:5/749: fdatasync d7/da/df/f10 0 2024-07-03T15:43:57.477 INFO:tasks.workunit.client.0.smithi028.stdout:9/90: creat d1/d16/f19 x:0 0 0 2024-07-03T15:43:57.478 INFO:tasks.workunit.client.0.smithi028.stdout:9/91: write d1/dd/f13 [4359396,84287] 0 2024-07-03T15:43:57.478 INFO:tasks.workunit.client.0.smithi028.stdout:9/92: chown d1/d16 99171502 1 2024-07-03T15:43:57.478 INFO:tasks.workunit.client.0.smithi028.stdout:9/93: fsync d1/dd/fe 0 2024-07-03T15:43:57.479 INFO:tasks.workunit.client.1.smithi148.stdout:1/634: mknod d0/d2/d2e/d39/d41/cd9 0 2024-07-03T15:43:57.479 INFO:tasks.workunit.client.1.smithi148.stdout:3/721: dwrite dd/d33/f3a [0,4194304] 0 2024-07-03T15:43:57.480 INFO:tasks.workunit.client.0.smithi028.stdout:1/140: truncate f2 4392511 0 2024-07-03T15:43:57.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:57 smithi148 ceph-mon[36056]: mgrmap e20: smithi028.vtqrtz(active, since 6m), standbys: smithi148.rjzsar 2024-07-03T15:43:57.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:57 smithi148 ceph-mon[36056]: pgmap v241: 65 pgs: 65 active+clean; 1.4 GiB data, 5.7 GiB used, 531 GiB / 536 GiB avail; 148 MiB/s rd, 178 MiB/s wr, 726 op/s 2024-07-03T15:43:57.485 INFO:tasks.workunit.client.1.smithi148.stdout:1/635: symlink d0/de/d89/lda 0 2024-07-03T15:43:57.486 INFO:tasks.workunit.client.1.smithi148.stdout:1/636: dread d0/d2/d6/d14/f8e [0,4194304] 0 2024-07-03T15:43:57.487 INFO:tasks.workunit.client.0.smithi028.stdout:9/94: mknod d1/c1a 0 2024-07-03T15:43:57.489 INFO:tasks.workunit.client.0.smithi028.stdout:1/141: mknod d4/de/d1c/c3b 0 2024-07-03T15:43:57.492 INFO:tasks.workunit.client.1.smithi148.stdout:3/722: mknod dd/d33/d52/d57/ce6 0 2024-07-03T15:43:57.493 INFO:tasks.workunit.client.0.smithi028.stdout:6/35: dwrite f0 [4194304,4194304] 0 2024-07-03T15:43:57.493 INFO:tasks.workunit.client.0.smithi028.stdout:1/142: mkdir d4/d3c 0 2024-07-03T15:43:57.495 INFO:tasks.workunit.client.0.smithi028.stdout:6/36: mknod d4/da/cd 0 2024-07-03T15:43:57.496 INFO:tasks.workunit.client.1.smithi148.stdout:1/637: creat d0/d2/d11/d1e/d9d/fdb x:0 0 0 2024-07-03T15:43:57.496 INFO:tasks.workunit.client.1.smithi148.stdout:1/638: write d0/d2/d2e/d39/d42/daf/fb8 [210837,99894] 0 2024-07-03T15:43:57.497 INFO:tasks.workunit.client.1.smithi148.stdout:1/639: fsync d0/d2/d11/f47 0 2024-07-03T15:43:57.497 INFO:tasks.workunit.client.0.smithi028.stdout:9/95: write d1/f2 [823299,84105] 0 2024-07-03T15:43:57.498 INFO:tasks.workunit.client.0.smithi028.stdout:9/96: creat d1/dd/f1b x:0 0 0 2024-07-03T15:43:57.498 INFO:tasks.workunit.client.0.smithi028.stdout:1/143: creat d4/de/d1c/f3d x:0 0 0 2024-07-03T15:43:57.501 INFO:tasks.workunit.client.1.smithi148.stdout:3/723: creat dd/d35/dac/fe7 x:0 0 0 2024-07-03T15:43:57.504 INFO:tasks.workunit.client.0.smithi028.stdout:1/144: rename d4/d18/f33 to d4/d11/f3e 0 2024-07-03T15:43:57.504 INFO:tasks.workunit.client.0.smithi028.stdout:1/145: stat d4/de 0 2024-07-03T15:43:57.504 INFO:tasks.workunit.client.0.smithi028.stdout:1/146: dread - d4/d11/f3e zero size 2024-07-03T15:43:57.505 INFO:tasks.workunit.client.1.smithi148.stdout:3/724: mknod dd/da6/db3/ce8 0 2024-07-03T15:43:57.508 INFO:tasks.workunit.client.0.smithi028.stdout:1/147: link d4/d11/l28 d4/d2c/d3a/l3f 0 2024-07-03T15:43:57.510 INFO:tasks.workunit.client.0.smithi028.stdout:1/148: truncate d4/de/ff 1447249 0 2024-07-03T15:43:57.512 INFO:tasks.workunit.client.0.smithi028.stdout:1/149: symlink d4/l40 0 2024-07-03T15:43:57.512 INFO:tasks.workunit.client.0.smithi028.stdout:1/150: dread - d4/f37 zero size 2024-07-03T15:43:57.513 INFO:tasks.workunit.client.0.smithi028.stdout:1/151: mknod d4/d2c/c41 0 2024-07-03T15:43:57.515 INFO:tasks.workunit.client.0.smithi028.stdout:4/26: getdents d2 0 2024-07-03T15:43:57.515 INFO:tasks.workunit.client.0.smithi028.stdout:4/27: stat d2/f4 0 2024-07-03T15:43:57.515 INFO:tasks.workunit.client.0.smithi028.stdout:4/28: chown d2 275 1 2024-07-03T15:43:57.516 INFO:tasks.workunit.client.1.smithi148.stdout:5/750: truncate d7/da/df/d16/d2f/d44/d52/d60/f103 71629 0 2024-07-03T15:43:57.517 INFO:tasks.workunit.client.1.smithi148.stdout:5/751: symlink d7/d13/d54/l105 0 2024-07-03T15:43:57.517 INFO:tasks.workunit.client.1.smithi148.stdout:3/725: dread dd/d33/d37/d68/d2e/f11 [0,4194304] 0 2024-07-03T15:43:57.517 INFO:tasks.workunit.client.1.smithi148.stdout:3/726: creat dd/d35/d41/fe9 x:0 0 0 2024-07-03T15:43:57.518 INFO:tasks.workunit.client.1.smithi148.stdout:5/752: rename d7/da/df/d16/f7e to d7/da/df/d16/d2f/d44/d52/d60/df2/f106 0 2024-07-03T15:43:57.519 INFO:tasks.workunit.client.1.smithi148.stdout:5/753: creat d7/d59/f107 x:0 0 0 2024-07-03T15:43:57.519 INFO:tasks.workunit.client.1.smithi148.stdout:5/754: chown d7/da/dbe/de6/cf4 614586 1 2024-07-03T15:43:57.519 INFO:tasks.workunit.client.0.smithi028.stdout:6/37: dwrite f0 [4194304,4194304] 0 2024-07-03T15:43:57.521 INFO:tasks.workunit.client.0.smithi028.stdout:9/97: dwrite d1/dd/f11 [0,4194304] 0 2024-07-03T15:43:57.521 INFO:tasks.workunit.client.1.smithi148.stdout:3/727: rmdir dd/d33/d37/d68/d2e/d40 39 2024-07-03T15:43:57.521 INFO:tasks.workunit.client.1.smithi148.stdout:1/640: dwrite d0/d2/d11/d1e/fb3 [0,4194304] 0 2024-07-03T15:43:57.526 INFO:tasks.workunit.client.0.smithi028.stdout:6/38: rmdir d4/db 0 2024-07-03T15:43:57.526 INFO:tasks.workunit.client.0.smithi028.stdout:4/29: write f1 [3614876,86640] 0 2024-07-03T15:43:57.526 INFO:tasks.workunit.client.0.smithi028.stdout:4/30: fsync d2/f4 0 2024-07-03T15:43:57.526 INFO:tasks.workunit.client.0.smithi028.stdout:4/31: chown d2/f4 2 1 2024-07-03T15:43:57.526 INFO:tasks.workunit.client.0.smithi028.stdout:9/98: dread d1/dd/f14 [0,4194304] 0 2024-07-03T15:43:57.526 INFO:tasks.workunit.client.0.smithi028.stdout:1/152: dread d4/d11/f1b [0,4194304] 0 2024-07-03T15:43:57.526 INFO:tasks.workunit.client.1.smithi148.stdout:5/755: link d7/d13/d54/l3e d7/da/df/d1d/l108 0 2024-07-03T15:43:57.527 INFO:tasks.workunit.client.0.smithi028.stdout:6/39: dread f0 [0,4194304] 0 2024-07-03T15:43:57.527 INFO:tasks.workunit.client.1.smithi148.stdout:3/728: mknod dd/d33/d37/d68/d2e/d40/d8c/dbc/cea 0 2024-07-03T15:43:57.528 INFO:tasks.workunit.client.0.smithi028.stdout:4/32: mkdir d2/d5 0 2024-07-03T15:43:57.529 INFO:tasks.workunit.client.1.smithi148.stdout:3/729: dread dd/d33/d37/d68/d2e/d40/d45/fa3 [0,4194304] 0 2024-07-03T15:43:57.534 INFO:tasks.workunit.client.0.smithi028.stdout:8/66: truncate d0/d2/ff 993076 0 2024-07-03T15:43:57.534 INFO:tasks.workunit.client.0.smithi028.stdout:8/67: write d0/d2/d13/f17 [408677,70261] 0 2024-07-03T15:43:57.534 INFO:tasks.workunit.client.0.smithi028.stdout:8/68: creat d0/d2/f1a x:0 0 0 2024-07-03T15:43:57.536 INFO:tasks.workunit.client.0.smithi028.stdout:1/153: symlink d4/d11/d16/l42 0 2024-07-03T15:43:57.536 INFO:tasks.workunit.client.0.smithi028.stdout:1/154: readlink d4/d18/l24 0 2024-07-03T15:43:57.536 INFO:tasks.workunit.client.1.smithi148.stdout:1/641: truncate d0/d2/d2e/d39/d42/f49 8249429 0 2024-07-03T15:43:57.537 INFO:tasks.workunit.client.0.smithi028.stdout:6/40: mknod d4/ce 0 2024-07-03T15:43:57.538 INFO:tasks.workunit.client.0.smithi028.stdout:4/33: dread f1 [0,4194304] 0 2024-07-03T15:43:57.539 INFO:tasks.workunit.client.1.smithi148.stdout:3/730: mknod dd/d35/d41/d89/ceb 0 2024-07-03T15:43:57.539 INFO:tasks.workunit.client.1.smithi148.stdout:3/731: getdents dd/d33/d37/d68/d67/da7 0 2024-07-03T15:43:57.539 INFO:tasks.workunit.client.1.smithi148.stdout:3/732: stat dd/d33/d37/d68/d2e/d40/d45/d65 0 2024-07-03T15:43:57.539 INFO:tasks.workunit.client.1.smithi148.stdout:3/733: write dd/f44 [4859450,38857] 0 2024-07-03T15:43:57.541 INFO:tasks.workunit.client.1.smithi148.stdout:3/734: creat dd/da6/fec x:0 0 0 2024-07-03T15:43:57.541 INFO:tasks.workunit.client.1.smithi148.stdout:3/735: write dd/d35/dac/fe7 [281676,39452] 0 2024-07-03T15:43:57.543 INFO:tasks.workunit.client.1.smithi148.stdout:3/736: truncate dd/d33/d37/d68/d67/fa0 4805711 0 2024-07-03T15:43:57.546 INFO:tasks.workunit.client.0.smithi028.stdout:8/69: symlink d0/d2/d3/dc/d11/d12/l1b 0 2024-07-03T15:43:57.549 INFO:tasks.workunit.client.0.smithi028.stdout:1/155: unlink d4/f39 0 2024-07-03T15:43:57.549 INFO:tasks.workunit.client.1.smithi148.stdout:5/756: dwrite d7/da/df/d16/d2f/dc2/fda [0,4194304] 0 2024-07-03T15:43:57.549 INFO:tasks.workunit.client.1.smithi148.stdout:5/757: stat d7/da/df/d16/d6a/db4/lc0 0 2024-07-03T15:43:57.549 INFO:tasks.workunit.client.1.smithi148.stdout:5/758: chown d7/da/df/d1d/f2e 944757 1 2024-07-03T15:43:57.549 INFO:tasks.workunit.client.1.smithi148.stdout:4/627: getdents d0/d28/da7/d62 0 2024-07-03T15:43:57.550 INFO:tasks.workunit.client.0.smithi028.stdout:6/41: mknod d4/da/cf 0 2024-07-03T15:43:57.551 INFO:tasks.workunit.client.1.smithi148.stdout:1/642: dwrite d0/d2/d2e/d39/d42/f87 [0,4194304] 0 2024-07-03T15:43:57.551 INFO:tasks.workunit.client.0.smithi028.stdout:4/34: rmdir d2 39 2024-07-03T15:43:57.553 INFO:tasks.workunit.client.1.smithi148.stdout:3/737: dread dd/f15 [4194304,4194304] 0 2024-07-03T15:43:57.554 INFO:tasks.workunit.client.0.smithi028.stdout:1/156: dread d4/d18/f2a [0,4194304] 0 2024-07-03T15:43:57.554 INFO:tasks.workunit.client.0.smithi028.stdout:1/157: readlink d4/l40 0 2024-07-03T15:43:57.555 INFO:tasks.workunit.client.0.smithi028.stdout:8/70: dread d0/d2/d13/f17 [0,4194304] 0 2024-07-03T15:43:57.555 INFO:tasks.workunit.client.1.smithi148.stdout:4/628: mkdir d0/d28/d7d/d6f/d7c/dae/dba/dde 0 2024-07-03T15:43:57.557 INFO:tasks.workunit.client.1.smithi148.stdout:1/643: dread d0/d2/d2e/d39/d42/f48 [0,4194304] 0 2024-07-03T15:43:57.557 INFO:tasks.workunit.client.0.smithi028.stdout:4/35: symlink d2/d5/l6 0 2024-07-03T15:43:57.561 INFO:tasks.workunit.client.0.smithi028.stdout:1/158: mkdir d4/d2c/d3a/d43 0 2024-07-03T15:43:57.562 INFO:tasks.workunit.client.0.smithi028.stdout:4/36: dread f1 [0,4194304] 0 2024-07-03T15:43:57.563 INFO:tasks.workunit.client.0.smithi028.stdout:4/37: fsync d2/f4 0 2024-07-03T15:43:57.563 INFO:tasks.workunit.client.1.smithi148.stdout:3/738: rmdir dd/d35/d41/d89/db0 39 2024-07-03T15:43:57.564 INFO:tasks.workunit.client.1.smithi148.stdout:5/759: dwrite d7/f19 [0,4194304] 0 2024-07-03T15:43:57.565 INFO:tasks.workunit.client.1.smithi148.stdout:6/735: sync 2024-07-03T15:43:57.565 INFO:tasks.workunit.client.1.smithi148.stdout:7/771: sync 2024-07-03T15:43:57.565 INFO:tasks.workunit.client.1.smithi148.stdout:2/728: sync 2024-07-03T15:43:57.565 INFO:tasks.workunit.client.1.smithi148.stdout:9/929: sync 2024-07-03T15:43:57.565 INFO:tasks.workunit.client.1.smithi148.stdout:2/729: chown d5/d14/d8b/dd5/ce2 690367131 1 2024-07-03T15:43:57.565 INFO:tasks.workunit.client.1.smithi148.stdout:2/730: fsync d5/d14/d22/d60/d76/fb0 0 2024-07-03T15:43:57.566 INFO:tasks.workunit.client.0.smithi028.stdout:1/159: mkdir d4/de/d1c/d44 0 2024-07-03T15:43:57.566 INFO:tasks.workunit.client.0.smithi028.stdout:1/160: chown d4/c35 25 1 2024-07-03T15:43:57.567 INFO:tasks.workunit.client.1.smithi148.stdout:4/629: mkdir d0/d91/ddf 0 2024-07-03T15:43:57.567 INFO:tasks.workunit.client.0.smithi028.stdout:4/38: creat d2/f7 x:0 0 0 2024-07-03T15:43:57.568 INFO:tasks.workunit.client.0.smithi028.stdout:4/39: chown d2/f4 84 1 2024-07-03T15:43:57.568 INFO:tasks.workunit.client.1.smithi148.stdout:1/644: creat d0/d2/d2e/d39/fdc x:0 0 0 2024-07-03T15:43:57.569 INFO:tasks.workunit.client.1.smithi148.stdout:3/739: creat dd/d33/d37/d68/fed x:0 0 0 2024-07-03T15:43:57.569 INFO:tasks.workunit.client.1.smithi148.stdout:3/740: chown dd/d33/f4b 144358997 1 2024-07-03T15:43:57.572 INFO:tasks.workunit.client.0.smithi028.stdout:4/40: creat d2/d5/f8 x:0 0 0 2024-07-03T15:43:57.574 INFO:tasks.workunit.client.1.smithi148.stdout:6/736: link d3/d1d/d28/d3e/f8b d3/d1d/d28/d3e/ddd/fec 0 2024-07-03T15:43:57.576 INFO:tasks.workunit.client.1.smithi148.stdout:9/930: mkdir d0/d2f/d129 0 2024-07-03T15:43:57.579 INFO:tasks.workunit.client.1.smithi148.stdout:7/772: rename d5/d7d/da7/dc7/cf2 to d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/c10c 0 2024-07-03T15:43:57.581 INFO:tasks.workunit.client.1.smithi148.stdout:1/645: symlink d0/de/ldd 0 2024-07-03T15:43:57.581 INFO:tasks.workunit.client.1.smithi148.stdout:0/609: sync 2024-07-03T15:43:57.581 INFO:tasks.workunit.client.1.smithi148.stdout:8/659: sync 2024-07-03T15:43:57.582 INFO:tasks.workunit.client.1.smithi148.stdout:1/646: stat d0/d2/d2e/d39/d41/cd9 0 2024-07-03T15:43:57.582 INFO:tasks.workunit.client.1.smithi148.stdout:0/610: write da/d17/f8b [672767,2018] 0 2024-07-03T15:43:57.582 INFO:tasks.workunit.client.1.smithi148.stdout:1/647: write d0/de/d2f/d72/f86 [585248,104871] 0 2024-07-03T15:43:57.582 INFO:tasks.workunit.client.1.smithi148.stdout:0/611: write da/d17/d2a/d2f/fc2 [246753,59268] 0 2024-07-03T15:43:57.583 INFO:tasks.workunit.client.1.smithi148.stdout:6/737: unlink d3/l12 0 2024-07-03T15:43:57.583 INFO:tasks.workunit.client.1.smithi148.stdout:3/741: write f5 [1167400,74105] 0 2024-07-03T15:43:57.584 INFO:tasks.workunit.client.0.smithi028.stdout:0/20: read d2/d3/f7 [82278,36994] 0 2024-07-03T15:43:57.584 INFO:tasks.workunit.client.0.smithi028.stdout:0/21: chown d2/d3 67 1 2024-07-03T15:43:57.585 INFO:tasks.workunit.client.1.smithi148.stdout:7/773: mkdir d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/db2/d10d 0 2024-07-03T15:43:57.586 INFO:tasks.workunit.client.1.smithi148.stdout:8/660: mkdir d4/da/d14/d17/deb 0 2024-07-03T15:43:57.587 INFO:tasks.workunit.client.1.smithi148.stdout:5/760: dwrite d7/d13/d54/f7d [0,4194304] 0 2024-07-03T15:43:57.589 INFO:tasks.workunit.client.1.smithi148.stdout:1/648: symlink d0/d2/d11/lde 0 2024-07-03T15:43:57.589 INFO:tasks.workunit.client.1.smithi148.stdout:1/649: dread - d0/d2/d11/d1e/d9d/fdb zero size 2024-07-03T15:43:57.589 INFO:tasks.workunit.client.1.smithi148.stdout:1/650: stat d0/de/d89/f99 0 2024-07-03T15:43:57.590 INFO:tasks.workunit.client.1.smithi148.stdout:2/731: dwrite d5/d14/d86/db9/dbb/d13/d41/d7b/ddf/d7e/f82 [0,4194304] 0 2024-07-03T15:43:57.591 INFO:tasks.workunit.client.1.smithi148.stdout:0/612: rename da/d58/dbd/d13/d14/d35 to da/d58/dbd/d24/dc8 0 2024-07-03T15:43:57.594 INFO:tasks.workunit.client.1.smithi148.stdout:7/774: creat d5/d7d/da7/db4/df0/f10e x:0 0 0 2024-07-03T15:43:57.596 INFO:tasks.workunit.client.1.smithi148.stdout:6/738: write d3/d1d/d28/d31/d61/f6b [3355723,12737] 0 2024-07-03T15:43:57.596 INFO:tasks.workunit.client.1.smithi148.stdout:4/630: dwrite d0/d8/d13/d48/f9a [0,4194304] 0 2024-07-03T15:43:57.596 INFO:tasks.workunit.client.1.smithi148.stdout:4/631: chown d0/f30 2 1 2024-07-03T15:43:57.597 INFO:tasks.workunit.client.1.smithi148.stdout:5/761: mkdir d7/da/df/d1d/d109 0 2024-07-03T15:43:57.602 INFO:tasks.workunit.client.1.smithi148.stdout:3/742: dread dd/f44 [4194304,4194304] 0 2024-07-03T15:43:57.602 INFO:tasks.workunit.client.1.smithi148.stdout:3/743: getdents dd/d33/d37/d68/d2e/d40 0 2024-07-03T15:43:57.602 INFO:tasks.workunit.client.1.smithi148.stdout:3/744: dread - dd/d33/d37/d68/d67/fde zero size 2024-07-03T15:43:57.602 INFO:tasks.workunit.client.1.smithi148.stdout:3/745: stat dd/d35/f94 0 2024-07-03T15:43:57.602 INFO:tasks.workunit.client.1.smithi148.stdout:3/746: truncate dd/f36 4205274 0 2024-07-03T15:43:57.602 INFO:tasks.workunit.client.1.smithi148.stdout:3/747: write dd/d35/d41/d89/fce [272506,50173] 0 2024-07-03T15:43:57.604 INFO:tasks.workunit.client.0.smithi028.stdout:4/41: dwrite d2/d5/f8 [0,4194304] 0 2024-07-03T15:43:57.604 INFO:tasks.workunit.client.1.smithi148.stdout:7/775: unlink d5/f58 0 2024-07-03T15:43:57.606 INFO:tasks.workunit.client.1.smithi148.stdout:7/776: dread d5/d7d/da7/d1c/f71 [0,4194304] 0 2024-07-03T15:43:57.608 INFO:tasks.workunit.client.1.smithi148.stdout:9/931: dwrite d0/d18/df3/f6e [0,4194304] 0 2024-07-03T15:43:57.609 INFO:tasks.workunit.client.1.smithi148.stdout:9/932: dread - d0/d18/d24/f11f zero size 2024-07-03T15:43:57.610 INFO:tasks.workunit.client.0.smithi028.stdout:4/42: rename d2/d5/f8 to d2/d5/f9 0 2024-07-03T15:43:57.611 INFO:tasks.workunit.client.1.smithi148.stdout:6/739: link d3/d1d/d28/d31/f4b d3/d88/fed 0 2024-07-03T15:43:57.611 INFO:tasks.workunit.client.1.smithi148.stdout:6/740: chown d3/c22 16378417 1 2024-07-03T15:43:57.612 INFO:tasks.workunit.client.1.smithi148.stdout:4/632: link d0/d28/d7d/ca5 d0/d8/d3a/ce0 0 2024-07-03T15:43:57.614 INFO:tasks.workunit.client.0.smithi028.stdout:4/43: mknod d2/ca 0 2024-07-03T15:43:57.615 INFO:tasks.workunit.client.1.smithi148.stdout:5/762: creat d7/da/df/d16/dac/dc1/dc4/f10a x:0 0 0 2024-07-03T15:43:57.616 INFO:tasks.workunit.client.0.smithi028.stdout:9/99: rmdir d1 39 2024-07-03T15:43:57.619 INFO:tasks.workunit.client.0.smithi028.stdout:4/44: rename d2/ca to d2/cb 0 2024-07-03T15:43:57.619 INFO:tasks.workunit.client.1.smithi148.stdout:9/933: read d0/d18/feb [2261393,37033] 0 2024-07-03T15:43:57.620 INFO:tasks.workunit.client.1.smithi148.stdout:8/661: rename d4/da/d51/l59 to d4/da/d47/d88/lec 0 2024-07-03T15:43:57.620 INFO:tasks.workunit.client.1.smithi148.stdout:8/662: dread - d4/da/d14/f67 zero size 2024-07-03T15:43:57.620 INFO:tasks.workunit.client.1.smithi148.stdout:8/663: chown d4/da/d14/d1a/d32/d38/d58/d8e/db9 40 1 2024-07-03T15:43:57.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:57 smithi028 ceph-mon[29148]: mgrmap e20: smithi028.vtqrtz(active, since 6m), standbys: smithi148.rjzsar 2024-07-03T15:43:57.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:57 smithi028 ceph-mon[29148]: pgmap v241: 65 pgs: 65 active+clean; 1.4 GiB data, 5.7 GiB used, 531 GiB / 536 GiB avail; 148 MiB/s rd, 178 MiB/s wr, 726 op/s 2024-07-03T15:43:57.622 INFO:tasks.workunit.client.1.smithi148.stdout:6/741: write d3/d1d/d28/f73 [3622176,7908] 0 2024-07-03T15:43:57.624 INFO:tasks.workunit.client.1.smithi148.stdout:6/742: write d3/d1d/d28/d31/d4c/fda [101268,97351] 0 2024-07-03T15:43:57.624 INFO:tasks.workunit.client.1.smithi148.stdout:3/748: truncate dd/d33/d37/d68/d2e/f13 3226241 0 2024-07-03T15:43:57.624 INFO:tasks.workunit.client.1.smithi148.stdout:3/749: stat dd/d33/f88 0 2024-07-03T15:43:57.626 INFO:tasks.workunit.client.0.smithi028.stdout:9/100: fdatasync d1/dd/f13 0 2024-07-03T15:43:57.628 INFO:tasks.workunit.client.1.smithi148.stdout:7/777: mkdir d5/d7d/da7/d1c/d4a/d66/d93/d10f 0 2024-07-03T15:43:57.628 INFO:tasks.workunit.client.1.smithi148.stdout:7/778: dread - d5/d7d/da7/d1c/d33/d4c/d21/d31/fae zero size 2024-07-03T15:43:57.628 INFO:tasks.workunit.client.0.smithi028.stdout:0/22: dwrite d2/d3/f7 [0,4194304] 0 2024-07-03T15:43:57.629 INFO:tasks.workunit.client.1.smithi148.stdout:1/651: dwrite d0/de/d44/d54/d5d/f8b [0,4194304] 0 2024-07-03T15:43:57.632 INFO:tasks.workunit.client.0.smithi028.stdout:0/23: link d2/c5 d2/d3/c8 0 2024-07-03T15:43:57.634 INFO:tasks.workunit.client.1.smithi148.stdout:9/934: dread d0/d18/df3/fb3 [0,4194304] 0 2024-07-03T15:43:57.635 INFO:tasks.workunit.client.0.smithi028.stdout:1/161: dread d4/de/ff [0,4194304] 0 2024-07-03T15:43:57.635 INFO:tasks.workunit.client.0.smithi028.stdout:1/162: chown d4/de/d1e 0 1 2024-07-03T15:43:57.636 INFO:tasks.workunit.client.0.smithi028.stdout:0/24: creat d2/f9 x:0 0 0 2024-07-03T15:43:57.636 INFO:tasks.workunit.client.0.smithi028.stdout:0/25: chown d2/f9 106 1 2024-07-03T15:43:57.637 INFO:tasks.workunit.client.1.smithi148.stdout:6/743: dread d3/d1d/d28/d31/fd9 [0,4194304] 0 2024-07-03T15:43:57.637 INFO:tasks.workunit.client.1.smithi148.stdout:6/744: fsync d3/d1d/d28/d3e/ddd/d74/dc4/fce 0 2024-07-03T15:43:57.638 INFO:tasks.workunit.client.1.smithi148.stdout:8/664: rename d4/da/d14/d1a/d32/d38/dba/c81 to d4/da/d14/d9e/ced 0 2024-07-03T15:43:57.640 INFO:tasks.workunit.client.0.smithi028.stdout:1/163: symlink d4/d2c/l45 0 2024-07-03T15:43:57.640 INFO:tasks.workunit.client.0.smithi028.stdout:1/164: creat d4/d18/f46 x:0 0 0 2024-07-03T15:43:57.640 INFO:tasks.workunit.client.0.smithi028.stdout:1/165: chown d4/de/f31 0 1 2024-07-03T15:43:57.640 INFO:tasks.workunit.client.1.smithi148.stdout:3/750: dwrite dd/fcf [0,4194304] 0 2024-07-03T15:43:57.641 INFO:tasks.workunit.client.0.smithi028.stdout:0/26: unlink d2/d3/c8 0 2024-07-03T15:43:57.641 INFO:tasks.workunit.client.0.smithi028.stdout:0/27: creat d2/fa x:0 0 0 2024-07-03T15:43:57.645 INFO:tasks.workunit.client.1.smithi148.stdout:7/779: creat d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/db2/d10d/f110 x:0 0 0 2024-07-03T15:43:57.645 INFO:tasks.workunit.client.1.smithi148.stdout:7/780: readlink d5/d7d/da7/d1c/d33/d4c/d21/d47/l79 0 2024-07-03T15:43:57.646 INFO:tasks.workunit.client.0.smithi028.stdout:1/166: mknod d4/d18/c47 0 2024-07-03T15:43:57.647 INFO:tasks.workunit.client.0.smithi028.stdout:6/42: creat d4/f10 x:0 0 0 2024-07-03T15:43:57.651 INFO:tasks.workunit.client.1.smithi148.stdout:3/751: write dd/d33/f39 [3560052,2570] 0 2024-07-03T15:43:57.652 INFO:tasks.workunit.client.1.smithi148.stdout:9/935: dwrite d0/dd/d25/d37/d40/da3/fe0 [0,4194304] 0 2024-07-03T15:43:57.652 INFO:tasks.workunit.client.0.smithi028.stdout:1/167: symlink d4/d2c/l48 0 2024-07-03T15:43:57.653 INFO:tasks.workunit.client.0.smithi028.stdout:1/168: chown d4/de/d1c 112897401 1 2024-07-03T15:43:57.653 INFO:tasks.workunit.client.0.smithi028.stdout:8/71: getdents d0/d2 0 2024-07-03T15:43:57.656 INFO:tasks.workunit.client.0.smithi028.stdout:0/28: dread d2/d3/f7 [0,4194304] 0 2024-07-03T15:43:57.656 INFO:tasks.workunit.client.0.smithi028.stdout:0/29: stat d2/c4 0 2024-07-03T15:43:57.661 INFO:tasks.workunit.client.1.smithi148.stdout:5/763: rmdir d7/da/df/d16/de4/da7 39 2024-07-03T15:43:57.662 INFO:tasks.workunit.client.1.smithi148.stdout:6/745: rmdir d3/daa 39 2024-07-03T15:43:57.662 INFO:tasks.workunit.client.1.smithi148.stdout:6/746: truncate d3/f2c 862426 0 2024-07-03T15:43:57.662 INFO:tasks.workunit.client.1.smithi148.stdout:6/747: fdatasync d3/d1d/d28/d3e/ddd/fec 0 2024-07-03T15:43:57.663 INFO:tasks.workunit.client.0.smithi028.stdout:8/72: dread d0/d2/d3/fa [0,4194304] 0 2024-07-03T15:43:57.663 INFO:tasks.workunit.client.1.smithi148.stdout:8/665: link d4/da/d14/f4d d4/da/d14/d1a/d32/d89/fee 0 2024-07-03T15:43:57.664 INFO:tasks.workunit.client.0.smithi028.stdout:0/30: dread d2/d3/f7 [0,4194304] 0 2024-07-03T15:43:57.664 INFO:tasks.workunit.client.0.smithi028.stdout:0/31: write d2/f9 [1001826,31387] 0 2024-07-03T15:43:57.665 INFO:tasks.workunit.client.0.smithi028.stdout:8/73: symlink d0/dd/l1c 0 2024-07-03T15:43:57.667 INFO:tasks.workunit.client.0.smithi028.stdout:8/74: dread d0/d2/d13/f17 [0,4194304] 0 2024-07-03T15:43:57.667 INFO:tasks.workunit.client.0.smithi028.stdout:8/75: chown d0/dd/l1c 0 1 2024-07-03T15:43:57.668 INFO:tasks.workunit.client.0.smithi028.stdout:4/45: fsync d2/f7 0 2024-07-03T15:43:57.668 INFO:tasks.workunit.client.0.smithi028.stdout:4/46: creat d2/fc x:0 0 0 2024-07-03T15:43:57.668 INFO:tasks.workunit.client.0.smithi028.stdout:4/47: dread - d2/f7 zero size 2024-07-03T15:43:57.668 INFO:tasks.workunit.client.0.smithi028.stdout:4/48: truncate d2/f4 289427 0 2024-07-03T15:43:57.669 INFO:tasks.workunit.client.1.smithi148.stdout:7/781: mknod d5/d7d/da7/d1c/d33/d4c/d21/c111 0 2024-07-03T15:43:57.669 INFO:tasks.workunit.client.1.smithi148.stdout:9/936: dwrite d0/d49/d43/d19/d7a/f10d [0,4194304] 0 2024-07-03T15:43:57.670 INFO:tasks.workunit.client.0.smithi028.stdout:8/76: link d0/dd/l1c d0/l1d 0 2024-07-03T15:43:57.670 INFO:tasks.workunit.client.0.smithi028.stdout:8/77: creat d0/d2/d13/f1e x:0 0 0 2024-07-03T15:43:57.671 INFO:tasks.workunit.client.1.smithi148.stdout:3/752: mkdir dd/d33/d37/d68/d2e/d40/d77/dc9/dee 0 2024-07-03T15:43:57.678 INFO:tasks.workunit.client.1.smithi148.stdout:1/652: mknod d0/d2/cdf 0 2024-07-03T15:43:57.682 INFO:tasks.workunit.client.1.smithi148.stdout:5/764: mkdir d7/da/df/d16/d2f/d44/d52/d10b 0 2024-07-03T15:43:57.684 INFO:tasks.workunit.client.1.smithi148.stdout:6/748: creat d3/d1d/d28/d3e/ddd/d74/dc4/d85/fee x:0 0 0 2024-07-03T15:43:57.685 INFO:tasks.workunit.client.1.smithi148.stdout:5/765: dread d7/da/f5f [0,4194304] 0 2024-07-03T15:43:57.685 INFO:tasks.workunit.client.1.smithi148.stdout:5/766: dread - d7/da/df/d16/d2f/dc2/de9/fb7 zero size 2024-07-03T15:43:57.685 INFO:tasks.workunit.client.1.smithi148.stdout:8/666: stat d4/da/d14/d3a/le4 0 2024-07-03T15:43:57.685 INFO:tasks.workunit.client.1.smithi148.stdout:7/782: mkdir d5/d7d/da7/d1c/d59/d112 0 2024-07-03T15:43:57.687 INFO:tasks.workunit.client.1.smithi148.stdout:9/937: mkdir d0/d18/df3/da8/d12a 0 2024-07-03T15:43:57.692 INFO:tasks.workunit.client.1.smithi148.stdout:1/653: rename d0/d2/d2e/d39/d41/d77/d80 to d0/d2/d11/d1e/d9d/de0 0 2024-07-03T15:43:57.693 INFO:tasks.workunit.client.1.smithi148.stdout:6/749: chown d3/d1d/f4e 76260 1 2024-07-03T15:43:57.693 INFO:tasks.workunit.client.1.smithi148.stdout:6/750: fdatasync d3/d1d/d28/f73 0 2024-07-03T15:43:57.694 INFO:tasks.workunit.client.1.smithi148.stdout:5/767: mkdir d7/da/df/d16/d2f/dc2/d10c 0 2024-07-03T15:43:57.695 INFO:tasks.workunit.client.1.smithi148.stdout:7/783: write d5/d7d/da7/f4b [3221296,31280] 0 2024-07-03T15:43:57.695 INFO:tasks.workunit.client.1.smithi148.stdout:7/784: write d5/fa [1866649,4763] 0 2024-07-03T15:43:57.695 INFO:tasks.workunit.client.1.smithi148.stdout:7/785: write d5/d7d/da7/f37 [1436720,111094] 0 2024-07-03T15:43:57.695 INFO:tasks.workunit.client.1.smithi148.stdout:9/938: creat d0/d18/df3/da8/d12a/f12b x:0 0 0 2024-07-03T15:43:57.697 INFO:tasks.workunit.client.1.smithi148.stdout:4/633: truncate d0/d8/d13/d48/f9a 906008 0 2024-07-03T15:43:57.697 INFO:tasks.workunit.client.1.smithi148.stdout:7/786: write d5/d7d/da7/d1c/d23/fc1 [161910,55071] 0 2024-07-03T15:43:57.698 INFO:tasks.workunit.client.1.smithi148.stdout:3/753: rmdir dd/d33/d52 39 2024-07-03T15:43:57.698 INFO:tasks.workunit.client.1.smithi148.stdout:3/754: creat dd/d35/d41/d89/fef x:0 0 0 2024-07-03T15:43:57.698 INFO:tasks.workunit.client.1.smithi148.stdout:6/751: write d3/f17 [2830673,22043] 0 2024-07-03T15:43:57.699 INFO:tasks.workunit.client.1.smithi148.stdout:1/654: creat d0/d2/fe1 x:0 0 0 2024-07-03T15:43:57.701 INFO:tasks.workunit.client.1.smithi148.stdout:5/768: mknod d7/da/df/da0/db2/c10d 0 2024-07-03T15:43:57.702 INFO:tasks.workunit.client.1.smithi148.stdout:8/667: mknod d4/da/d14/d1a/cef 0 2024-07-03T15:43:57.703 INFO:tasks.workunit.client.1.smithi148.stdout:9/939: creat d0/d49/d43/d19/de1/f12c x:0 0 0 2024-07-03T15:43:57.703 INFO:tasks.workunit.client.1.smithi148.stdout:9/940: read d0/dd/d9f/dde/f89 [269871,72189] 0 2024-07-03T15:43:57.705 INFO:tasks.workunit.client.1.smithi148.stdout:4/634: creat d0/d8/d13/d67/fe1 x:0 0 0 2024-07-03T15:43:57.706 INFO:tasks.workunit.client.0.smithi028.stdout:4/49: unlink d2/cb 0 2024-07-03T15:43:57.708 INFO:tasks.workunit.client.1.smithi148.stdout:3/755: creat dd/d33/d37/d68/d2e/ff0 x:0 0 0 2024-07-03T15:43:57.710 INFO:tasks.workunit.client.1.smithi148.stdout:6/752: unlink d3/l66 0 2024-07-03T15:43:57.713 INFO:tasks.workunit.client.1.smithi148.stdout:3/756: dread dd/d33/d37/d68/d2e/d40/f5d [0,4194304] 0 2024-07-03T15:43:57.713 INFO:tasks.workunit.client.1.smithi148.stdout:3/757: fdatasync dd/f22 0 2024-07-03T15:43:57.713 INFO:tasks.workunit.client.1.smithi148.stdout:7/787: dwrite d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/de8/ff9 [0,4194304] 0 2024-07-03T15:43:57.713 INFO:tasks.workunit.client.1.smithi148.stdout:5/769: creat d7/da/df/d16/d2f/dc2/d10c/f10e x:0 0 0 2024-07-03T15:43:57.713 INFO:tasks.workunit.client.1.smithi148.stdout:5/770: read - d7/d59/f107 zero size 2024-07-03T15:43:57.715 INFO:tasks.workunit.client.1.smithi148.stdout:8/668: creat d4/da/d14/d1a/d32/d38/d58/d8e/ff0 x:0 0 0 2024-07-03T15:43:57.715 INFO:tasks.workunit.client.1.smithi148.stdout:8/669: creat d4/d86/ff1 x:0 0 0 2024-07-03T15:43:57.716 INFO:tasks.workunit.client.1.smithi148.stdout:9/941: symlink d0/dd/d25/d39/l12d 0 2024-07-03T15:43:57.719 INFO:tasks.workunit.client.1.smithi148.stdout:4/635: dwrite d0/d28/f69 [4194304,4194304] 0 2024-07-03T15:43:57.730 INFO:tasks.workunit.client.1.smithi148.stdout:3/758: symlink dd/d33/d37/d68/lf1 0 2024-07-03T15:43:57.730 INFO:tasks.workunit.client.1.smithi148.stdout:7/788: rename d5/d7d/da7/d1c/d33/d4c/d21/f29 to d5/d7d/da7/d1c/d33/d4c/d21/f113 0 2024-07-03T15:43:57.730 INFO:tasks.workunit.client.1.smithi148.stdout:5/771: mkdir d7/da/df/da0/d10f 0 2024-07-03T15:43:57.731 INFO:tasks.workunit.client.0.smithi028.stdout:0/32: rmdir d2/d3 39 2024-07-03T15:43:57.731 INFO:tasks.workunit.client.0.smithi028.stdout:0/33: creat d2/fb x:0 0 0 2024-07-03T15:43:57.734 INFO:tasks.workunit.client.1.smithi148.stdout:9/942: mkdir d0/dd/d46/ded/d12e 0 2024-07-03T15:43:57.734 INFO:tasks.workunit.client.1.smithi148.stdout:8/670: dwrite d4/da/d14/f67 [0,4194304] 0 2024-07-03T15:43:57.735 INFO:tasks.workunit.client.0.smithi028.stdout:4/50: dwrite d2/fc [0,4194304] 0 2024-07-03T15:43:57.737 INFO:tasks.workunit.client.1.smithi148.stdout:3/759: mkdir dd/d33/df2 0 2024-07-03T15:43:57.737 INFO:tasks.workunit.client.1.smithi148.stdout:3/760: write dd/f15 [139980,100193] 0 2024-07-03T15:43:57.740 INFO:tasks.workunit.client.0.smithi028.stdout:0/34: write d2/d3/f7 [3388096,53775] 0 2024-07-03T15:43:57.746 INFO:tasks.workunit.client.1.smithi148.stdout:7/789: truncate d5/d7d/da7/d1c/d59/f7c 1488074 0 2024-07-03T15:43:57.748 INFO:tasks.workunit.client.1.smithi148.stdout:9/943: mkdir d0/d49/d12f 0 2024-07-03T15:43:57.749 INFO:tasks.workunit.client.0.smithi028.stdout:0/35: mknod d2/d3/cc 0 2024-07-03T15:43:57.749 INFO:tasks.workunit.client.0.smithi028.stdout:0/36: rename d2/d3 to d2/d3/dd 22 2024-07-03T15:43:57.749 INFO:tasks.workunit.client.0.smithi028.stdout:0/37: truncate d2/fb 565877 0 2024-07-03T15:43:57.750 INFO:tasks.workunit.client.1.smithi148.stdout:5/772: dwrite d7/d13/d54/f7d [0,4194304] 0 2024-07-03T15:43:57.753 INFO:tasks.workunit.client.0.smithi028.stdout:4/51: rmdir d2/d5 39 2024-07-03T15:43:57.755 INFO:tasks.workunit.client.1.smithi148.stdout:8/671: truncate d4/da/d51/d18/d20/f21 6197050 0 2024-07-03T15:43:57.755 INFO:tasks.workunit.client.1.smithi148.stdout:8/672: fsync d4/da/d14/f68 0 2024-07-03T15:43:57.755 INFO:tasks.workunit.client.1.smithi148.stdout:8/673: stat d4/da/f28 0 2024-07-03T15:43:57.758 INFO:tasks.workunit.client.0.smithi028.stdout:0/38: creat d2/d3/fe x:0 0 0 2024-07-03T15:43:57.763 INFO:tasks.workunit.client.1.smithi148.stdout:7/790: unlink d5/d7d/da7/db4/ffb 0 2024-07-03T15:43:57.764 INFO:tasks.workunit.client.1.smithi148.stdout:7/791: read - d5/d7d/da7/d1c/d4a/d66/dd7/fcd zero size 2024-07-03T15:43:57.764 INFO:tasks.workunit.client.1.smithi148.stdout:7/792: stat d5/d7d/da7/d1c/d33/d4c/d21/d39/f77 0 2024-07-03T15:43:57.764 INFO:tasks.workunit.client.1.smithi148.stdout:7/793: chown d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/c5e 42619461 1 2024-07-03T15:43:57.764 INFO:tasks.workunit.client.1.smithi148.stdout:7/794: chown d5/d7d/da7/d1c/d33/d4c/d21/d39/c3a 12 1 2024-07-03T15:43:57.766 INFO:tasks.workunit.client.1.smithi148.stdout:8/674: truncate d4/d86/f97 711464 0 2024-07-03T15:43:57.769 INFO:tasks.workunit.client.1.smithi148.stdout:7/795: creat d5/d7d/da7/d1c/d33/d4c/d21/d39/f114 x:0 0 0 2024-07-03T15:43:57.769 INFO:tasks.workunit.client.1.smithi148.stdout:7/796: creat d5/d7d/da7/db4/f115 x:0 0 0 2024-07-03T15:43:57.772 INFO:tasks.workunit.client.1.smithi148.stdout:8/675: truncate d4/da/d14/d1a/f37 3429257 0 2024-07-03T15:43:57.772 INFO:tasks.workunit.client.1.smithi148.stdout:8/676: chown d4/da/d14/d1a/d32/d38/d58/la9 153514773 1 2024-07-03T15:43:57.773 INFO:tasks.workunit.client.1.smithi148.stdout:8/677: truncate d4/da/d47/d88/d92/fa3 2493186 0 2024-07-03T15:43:57.773 INFO:tasks.workunit.client.1.smithi148.stdout:8/678: dread - d4/da/d14/d1a/d32/d38/d58/d8e/ff0 zero size 2024-07-03T15:43:57.776 INFO:tasks.workunit.client.1.smithi148.stdout:1/655: rename d0/d2/d2e/d39/d41 to d0/d2/d11/d1e/d9d/dc2/de2 0 2024-07-03T15:43:57.777 INFO:tasks.workunit.client.1.smithi148.stdout:1/656: write d0/d2/d11/d1e/d9d/dc2/de2/f7a [2952338,80864] 0 2024-07-03T15:43:57.777 INFO:tasks.workunit.client.1.smithi148.stdout:2/732: sync 2024-07-03T15:43:57.779 INFO:tasks.workunit.client.1.smithi148.stdout:6/753: rename d3/d1d/d28/d3e/ddd/d74/dc4/d85/da2/lca to d3/d1d/d28/d31/d61/lef 0 2024-07-03T15:43:57.781 INFO:tasks.workunit.client.1.smithi148.stdout:2/733: mkdir d5/d14/d8b/de8 0 2024-07-03T15:43:57.783 INFO:tasks.workunit.client.1.smithi148.stdout:4/636: rename d0/d8/d44/l85 to d0/d8/d3a/db9/le2 0 2024-07-03T15:43:57.783 INFO:tasks.workunit.client.1.smithi148.stdout:8/679: rename d4 to d4/da/d14/d1a/d32/d38/d58/d8e/db9/df2 22 2024-07-03T15:43:57.787 INFO:tasks.workunit.client.1.smithi148.stdout:2/734: link d5/d14/d86/db9/dbb/d13/d2d/l34 d5/d14/d86/db9/dbb/d13/d41/le9 0 2024-07-03T15:43:57.787 INFO:tasks.workunit.client.1.smithi148.stdout:2/735: creat d5/d14/d22/dc0/fea x:0 0 0 2024-07-03T15:43:57.787 INFO:tasks.workunit.client.1.smithi148.stdout:2/736: readlink d5/d67/dc3/lcc 0 2024-07-03T15:43:57.787 INFO:tasks.workunit.client.1.smithi148.stdout:2/737: chown d5/d14/d86/db9/dbb/d13/d41/d7b/d6a/d80/lad 0 1 2024-07-03T15:43:57.787 INFO:tasks.workunit.client.1.smithi148.stdout:1/657: write d0/d2/d2e/d39/d42/d62/f71 [3607178,64766] 0 2024-07-03T15:43:57.787 INFO:tasks.workunit.client.1.smithi148.stdout:1/658: creat d0/de/d2f/d72/fe3 x:0 0 0 2024-07-03T15:43:57.788 INFO:tasks.workunit.client.1.smithi148.stdout:8/680: rename d4/da/d14/d1a/lc6 to d4/da/d14/d3a/da8/dd3/lf3 0 2024-07-03T15:43:57.790 INFO:tasks.workunit.client.1.smithi148.stdout:4/637: creat d0/d28/d6e/dc7/fe3 x:0 0 0 2024-07-03T15:43:57.792 INFO:tasks.workunit.client.1.smithi148.stdout:2/738: rmdir d5/d14/d22 39 2024-07-03T15:43:57.792 INFO:tasks.workunit.client.1.smithi148.stdout:2/739: write d5/d14/fe7 [4337083,67345] 0 2024-07-03T15:43:57.794 INFO:tasks.workunit.client.1.smithi148.stdout:5/773: dwrite d7/da/df/d16/dac/dc1/dc4/fdd [0,4194304] 0 2024-07-03T15:43:57.795 INFO:tasks.workunit.client.1.smithi148.stdout:7/797: dwrite d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/db2/fe5 [0,4194304] 0 2024-07-03T15:43:57.795 INFO:tasks.workunit.client.1.smithi148.stdout:7/798: truncate d5/d7d/da7/d1c/d33/d4c/d84/fbf 4785362 0 2024-07-03T15:43:57.795 INFO:tasks.workunit.client.1.smithi148.stdout:7/799: readlink d5/d7d/da7/dc7/l100 0 2024-07-03T15:43:57.795 INFO:tasks.workunit.client.1.smithi148.stdout:7/800: stat d5/d7d/da7/d1c/d33/d4c/ce1 0 2024-07-03T15:43:57.796 INFO:tasks.workunit.client.0.smithi028.stdout:4/52: dwrite d2/fc [0,4194304] 0 2024-07-03T15:43:57.796 INFO:tasks.workunit.client.1.smithi148.stdout:1/659: link d0/l88 d0/d2/d11/d1e/d9d/dc2/de2/d77/db4/le4 0 2024-07-03T15:43:57.797 INFO:tasks.workunit.client.1.smithi148.stdout:1/660: truncate d0/d2/d2e/d39/d42/d62/f9a 931624 0 2024-07-03T15:43:57.797 INFO:tasks.workunit.client.1.smithi148.stdout:1/661: creat d0/de/d44/d54/fe5 x:0 0 0 2024-07-03T15:43:57.797 INFO:tasks.workunit.client.1.smithi148.stdout:8/681: truncate d4/da/d47/d88/d92/f39 2657725 0 2024-07-03T15:43:57.797 INFO:tasks.workunit.client.1.smithi148.stdout:8/682: dread - d4/da/d14/d1a/fd7 zero size 2024-07-03T15:43:57.797 INFO:tasks.workunit.client.1.smithi148.stdout:8/683: getdents d4/da/d47/d88/dce 0 2024-07-03T15:43:57.798 INFO:tasks.workunit.client.0.smithi028.stdout:4/53: mknod d2/cd 0 2024-07-03T15:43:57.798 INFO:tasks.workunit.client.0.smithi028.stdout:4/54: truncate d2/f7 854583 0 2024-07-03T15:43:57.799 INFO:tasks.workunit.client.1.smithi148.stdout:4/638: rmdir d0/d91/ddf 0 2024-07-03T15:43:57.800 INFO:tasks.workunit.client.1.smithi148.stdout:2/740: rmdir d5/d67 39 2024-07-03T15:43:57.803 INFO:tasks.workunit.client.1.smithi148.stdout:5/774: creat d7/da/df/d16/d2f/dc2/de9/f110 x:0 0 0 2024-07-03T15:43:57.804 INFO:tasks.workunit.client.1.smithi148.stdout:7/801: mknod d5/d7d/c116 0 2024-07-03T15:43:57.807 INFO:tasks.workunit.client.1.smithi148.stdout:1/662: creat d0/d2/d2e/fe6 x:0 0 0 2024-07-03T15:43:57.808 INFO:tasks.workunit.client.1.smithi148.stdout:8/684: unlink d4/da/d51/d18/d20/d82/lc7 0 2024-07-03T15:43:57.808 INFO:tasks.workunit.client.1.smithi148.stdout:8/685: readlink d4/da/d14/d1a/lb4 0 2024-07-03T15:43:57.809 INFO:tasks.workunit.client.1.smithi148.stdout:4/639: mknod d0/d8/d13/db4/ce4 0 2024-07-03T15:43:57.809 INFO:tasks.workunit.client.1.smithi148.stdout:4/640: chown d0/d28/d6e/dc7/fe3 24020941 1 2024-07-03T15:43:57.809 INFO:tasks.workunit.client.1.smithi148.stdout:8/686: dread d4/da/d47/d88/f27 [0,4194304] 0 2024-07-03T15:43:57.810 INFO:tasks.workunit.client.1.smithi148.stdout:8/687: readlink d4/da/d14/d3a/da8/dd3/lf3 0 2024-07-03T15:43:57.810 INFO:tasks.workunit.client.1.smithi148.stdout:2/741: creat d5/d14/feb x:0 0 0 2024-07-03T15:43:57.810 INFO:tasks.workunit.client.0.smithi028.stdout:0/39: getdents d2 0 2024-07-03T15:43:57.810 INFO:tasks.workunit.client.0.smithi028.stdout:0/40: fdatasync d2/fa 0 2024-07-03T15:43:57.813 INFO:tasks.workunit.client.1.smithi148.stdout:7/802: symlink d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/de8/l117 0 2024-07-03T15:43:57.813 INFO:tasks.workunit.client.1.smithi148.stdout:7/803: truncate d5/d7d/da7/f37 5102436 0 2024-07-03T15:43:57.813 INFO:tasks.workunit.client.1.smithi148.stdout:7/804: write d5/d7d/da7/d1c/d59/f109 [771097,40447] 0 2024-07-03T15:43:57.815 INFO:tasks.workunit.client.1.smithi148.stdout:1/663: mkdir d0/d2/d6/d14/d29/de7 0 2024-07-03T15:43:57.815 INFO:tasks.workunit.client.1.smithi148.stdout:1/664: write d0/d2/d2e/d39/fdc [28935,85672] 0 2024-07-03T15:43:57.815 INFO:tasks.workunit.client.1.smithi148.stdout:7/805: dread d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/db2/fb3 [0,4194304] 0 2024-07-03T15:43:57.816 INFO:tasks.workunit.client.1.smithi148.stdout:1/665: write d0/de/d2f/f4e [656311,65325] 0 2024-07-03T15:43:57.816 INFO:tasks.workunit.client.1.smithi148.stdout:1/666: fdatasync d0/d2/d11/d1e/d9d/dc2/fd8 0 2024-07-03T15:43:57.816 INFO:tasks.workunit.client.1.smithi148.stdout:7/806: stat d5/d7d/da7/d1c/d33/d4c/d84/d91/daa/db9/df1 0 2024-07-03T15:43:57.817 INFO:tasks.workunit.client.1.smithi148.stdout:8/688: dread d4/d86/f63 [0,4194304] 0 2024-07-03T15:43:57.818 INFO:tasks.workunit.client.1.smithi148.stdout:4/641: rename d0/d8/le to d0/d8/le5 0 2024-07-03T15:43:57.818 INFO:tasks.workunit.client.1.smithi148.stdout:4/642: fsync d0/d8/d3a/f47 0 2024-07-03T15:43:57.819 INFO:tasks.workunit.client.1.smithi148.stdout:2/742: write d5/d14/d86/db9/dbb/f6d [2495042,27980] 0 2024-07-03T15:43:57.819 INFO:tasks.workunit.client.1.smithi148.stdout:5/775: link d7/da/d21/c91 d7/d13/c111 0 2024-07-03T15:43:57.819 INFO:tasks.workunit.client.1.smithi148.stdout:5/776: fsync d7/da/df/d16/f1b 0 2024-07-03T15:43:57.821 INFO:tasks.workunit.client.1.smithi148.stdout:1/667: link d0/d2/fe1 d0/d2/d2e/d39/d42/db7/dbb/fe8 0 2024-07-03T15:43:57.823 INFO:tasks.workunit.client.1.smithi148.stdout:7/807: link d5/d7d/da7/d1c/d33/f48 d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/db2/d10d/f118 0 2024-07-03T15:43:57.823 INFO:tasks.workunit.client.1.smithi148.stdout:7/808: write d5/d7d/da7/d1c/d33/d4c/d21/d47/d8b/dd3/ffa [188145,99643] 0 2024-07-03T15:43:57.823 INFO:tasks.workunit.client.1.smithi148.stdout:8/689: getdents d4/da/d14/d3a/da8 0 2024-07-03T15:43:57.829 INFO:tasks.workunit.client.1.smithi148.stdout:2/743: rename d5/d14/d86/db9/dbb/d13/d41/d7b/d6a/d80/cd1 to d5/d14/d22/d60/d89/ddb/cec 0 2024-07-03T15:43:57.830 INFO:tasks.workunit.client.1.smithi148.stdout:5/777: link d7/d13/d54/fd6 d7/d13/d54/dd2/f112 0 2024-07-03T15:43:57.833 INFO:tasks.workunit.client.1.smithi148.stdout:1/668: symlink d0/d2/d11/d1e/d9d/dc0/le9 0 2024-07-03T15:43:57.834 INFO:tasks.workunit.client.1.smithi148.stdout:1/669: write d0/d2/d11/d1e/f59 [2943225,105458] 0 2024-07-03T15:43:57.834 INFO:tasks.workunit.client.1.smithi148.stdout:7/809: write d5/f57 [646667,55070] 0 2024-07-03T15:43:57.838 INFO:tasks.workunit.client.1.smithi148.stdout:7/810: mknod d5/d7d/da7/d1c/d33/c119 0 2024-07-03T15:43:57.838 INFO:tasks.workunit.client.1.smithi148.stdout:7/811: chown d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/l96 0 1 2024-07-03T15:43:57.838 INFO:tasks.workunit.client.1.smithi148.stdout:7/812: dread - d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/db2/dca/fba zero size 2024-07-03T15:43:57.838 INFO:tasks.workunit.client.1.smithi148.stdout:7/813: stat d5/d7d/da7/d1c/d59/ff7 0 2024-07-03T15:43:57.839 INFO:tasks.workunit.client.1.smithi148.stdout:1/670: mknod d0/d2/cea 0 2024-07-03T15:43:57.840 INFO:tasks.workunit.client.1.smithi148.stdout:1/671: dread d0/d2/d11/d1e/d9d/dc2/de2/d77/db4/dc7/fd2 [0,4194304] 0 2024-07-03T15:43:57.840 INFO:tasks.workunit.client.1.smithi148.stdout:0/613: sync 2024-07-03T15:43:57.841 INFO:tasks.workunit.client.1.smithi148.stdout:1/672: creat d0/d2/d2e/d39/d42/db7/feb x:0 0 0 2024-07-03T15:43:57.843 INFO:tasks.workunit.client.1.smithi148.stdout:1/673: mkdir d0/d2/d11/d1e/d9d/dc2/de2/d77/da3/dec 0 2024-07-03T15:43:57.843 INFO:tasks.workunit.client.1.smithi148.stdout:7/814: fsync d5/d7d/da7/d1c/d59/f109 0 2024-07-03T15:43:57.843 INFO:tasks.workunit.client.1.smithi148.stdout:5/778: write d7/da/df/d16/d2f/d44/f99 [3789293,17126] 0 2024-07-03T15:43:57.844 INFO:tasks.workunit.client.1.smithi148.stdout:4/643: dwrite d0/d28/f63 [4194304,4194304] 0 2024-07-03T15:43:57.844 INFO:tasks.workunit.client.1.smithi148.stdout:8/690: dwrite d4/da/d51/f57 [0,4194304] 0 2024-07-03T15:43:57.844 INFO:tasks.workunit.client.1.smithi148.stdout:4/644: creat d0/d8/d13/d67/fe6 x:0 0 0 2024-07-03T15:43:57.844 INFO:tasks.workunit.client.1.smithi148.stdout:4/645: write d0/d28/d7d/d6f/d7c/dae/dba/fdc [120228,63584] 0 2024-07-03T15:43:57.848 INFO:tasks.workunit.client.1.smithi148.stdout:1/674: mknod d0/ced 0 2024-07-03T15:43:57.850 INFO:tasks.workunit.client.1.smithi148.stdout:1/675: dread d0/d2/d11/d1e/d9d/dc2/de2/d77/f84 [0,4194304] 0 2024-07-03T15:43:57.850 INFO:tasks.workunit.client.1.smithi148.stdout:1/676: truncate d0/d2/d2e/d39/d42/daf/fb8 1241830 0 2024-07-03T15:43:57.853 INFO:tasks.workunit.client.1.smithi148.stdout:5/779: rmdir d7/da/df/d16/d1a 39 2024-07-03T15:43:57.856 INFO:tasks.workunit.client.1.smithi148.stdout:8/691: creat d4/da/d14/d3a/ff4 x:0 0 0 2024-07-03T15:43:57.858 INFO:tasks.workunit.client.1.smithi148.stdout:4/646: rename d0/d28/da7/d62 to d0/d8/d3a/db8/dd3/de7 0 2024-07-03T15:43:57.858 INFO:tasks.workunit.client.1.smithi148.stdout:4/647: dread - d0/d8/dd4/fd9 zero size 2024-07-03T15:43:57.859 INFO:tasks.workunit.client.1.smithi148.stdout:7/815: dwrite d5/d7d/da7/f4b [4194304,4194304] 0 2024-07-03T15:43:57.860 INFO:tasks.workunit.client.0.smithi028.stdout:0/41: truncate d2/d3/f7 2235171 0 2024-07-03T15:43:57.861 INFO:tasks.workunit.client.1.smithi148.stdout:8/692: unlink d4/da/d14/d3a/dab/cd9 0 2024-07-03T15:43:57.862 INFO:tasks.workunit.client.1.smithi148.stdout:4/648: mkdir d0/d28/de8 0 2024-07-03T15:43:57.864 INFO:tasks.workunit.client.1.smithi148.stdout:7/816: symlink d5/d7d/da7/d1c/d4a/d66/l11a 0 2024-07-03T15:43:57.866 INFO:tasks.workunit.client.1.smithi148.stdout:7/817: symlink d5/d7d/da7/d1c/d33/d4c/d84/d98/l11b 0 2024-07-03T15:43:57.866 INFO:tasks.workunit.client.1.smithi148.stdout:7/818: write d5/d7d/da7/d1c/d33/d4c/d21/d31/fae [675239,45553] 0 2024-07-03T15:43:57.867 INFO:tasks.workunit.client.1.smithi148.stdout:7/819: readlink d5/d7d/da7/d1c/d4a/d66/le3 0 2024-07-03T15:43:57.868 INFO:tasks.workunit.client.1.smithi148.stdout:4/649: rename d0/d8/d13/l1c to d0/d28/de8/le9 0 2024-07-03T15:43:57.870 INFO:tasks.workunit.client.1.smithi148.stdout:7/820: link d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/ce2 d5/d7d/da7/c11c 0 2024-07-03T15:43:57.872 INFO:tasks.workunit.client.1.smithi148.stdout:4/650: rename d0/d28/c40 to d0/d28/d7d/cea 0 2024-07-03T15:43:57.880 INFO:tasks.workunit.client.1.smithi148.stdout:7/821: dread d5/d7d/da7/d1c/d33/d4c/f99 [0,4194304] 0 2024-07-03T15:43:57.882 INFO:tasks.workunit.client.1.smithi148.stdout:7/822: dread d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/db6/fbe [0,4194304] 0 2024-07-03T15:43:57.884 INFO:tasks.workunit.client.1.smithi148.stdout:4/651: dread d0/d8/d13/d48/f2c [0,4194304] 0 2024-07-03T15:43:57.884 INFO:tasks.workunit.client.1.smithi148.stdout:8/693: dwrite d4/da/d14/f68 [0,4194304] 0 2024-07-03T15:43:57.884 INFO:tasks.workunit.client.1.smithi148.stdout:7/823: truncate d5/d7d/da7/d1c/f41 3891836 0 2024-07-03T15:43:57.896 INFO:tasks.workunit.client.1.smithi148.stdout:4/652: mkdir d0/d8/d3a/db8/dd3/deb 0 2024-07-03T15:43:57.898 INFO:tasks.workunit.client.1.smithi148.stdout:8/694: creat d4/da/d14/d1a/d32/d38/d58/dcb/ff5 x:0 0 0 2024-07-03T15:43:57.898 INFO:tasks.workunit.client.1.smithi148.stdout:8/695: read d4/da/f3e [1548306,80925] 0 2024-07-03T15:43:57.899 INFO:tasks.workunit.client.1.smithi148.stdout:4/653: truncate d0/d8/d44/f6a 6344396 0 2024-07-03T15:43:57.899 INFO:tasks.workunit.client.1.smithi148.stdout:4/654: dread - d0/d8/d13/d67/db1/fc4 zero size 2024-07-03T15:43:57.903 INFO:tasks.workunit.client.1.smithi148.stdout:7/824: dwrite d5/d7d/da7/db4/f115 [0,4194304] 0 2024-07-03T15:43:57.908 INFO:tasks.workunit.client.1.smithi148.stdout:7/825: mknod d5/c11d 0 2024-07-03T15:43:57.914 INFO:tasks.workunit.client.1.smithi148.stdout:7/826: dread d5/d7d/da7/d1c/d33/f94 [0,4194304] 0 2024-07-03T15:43:57.914 INFO:tasks.workunit.client.1.smithi148.stdout:7/827: truncate d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/db2/fb3 2195865 0 2024-07-03T15:43:57.920 INFO:tasks.workunit.client.1.smithi148.stdout:8/696: dwrite d4/d86/f63 [0,4194304] 0 2024-07-03T15:43:57.923 INFO:tasks.workunit.client.1.smithi148.stdout:8/697: getdents d4/da/d14/d1a 0 2024-07-03T15:43:57.924 INFO:tasks.workunit.client.1.smithi148.stdout:4/655: dwrite d0/d28/d7d/d6f/d7c/dae/dba/fdc [0,4194304] 0 2024-07-03T15:43:57.925 INFO:tasks.workunit.client.1.smithi148.stdout:8/698: getdents d4/da/d51/d18/d20 0 2024-07-03T15:43:57.928 INFO:tasks.workunit.client.1.smithi148.stdout:4/656: readlink d0/d8/d13/l68 0 2024-07-03T15:43:57.929 INFO:tasks.workunit.client.1.smithi148.stdout:8/699: symlink d4/da/d14/lf6 0 2024-07-03T15:43:57.934 INFO:tasks.workunit.client.1.smithi148.stdout:4/657: unlink d0/d8/d13/db4/fbc 0 2024-07-03T15:43:57.934 INFO:tasks.workunit.client.1.smithi148.stdout:4/658: truncate d0/d28/d7d/f8a 1875059 0 2024-07-03T15:43:57.934 INFO:tasks.workunit.client.1.smithi148.stdout:4/659: dread - d0/d8/d13/d67/db1/fc4 zero size 2024-07-03T15:43:57.934 INFO:tasks.workunit.client.1.smithi148.stdout:4/660: readlink d0/d8/d13/l68 0 2024-07-03T15:43:57.934 INFO:tasks.workunit.client.1.smithi148.stdout:4/661: fdatasync d0/d28/d7d/fdd 0 2024-07-03T15:43:57.935 INFO:tasks.workunit.client.1.smithi148.stdout:7/828: dwrite d5/f1f [0,4194304] 0 2024-07-03T15:43:57.935 INFO:tasks.workunit.client.1.smithi148.stdout:7/829: stat d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/ldc 0 2024-07-03T15:43:57.935 INFO:tasks.workunit.client.1.smithi148.stdout:4/662: creat d0/d8/d13/d48/fec x:0 0 0 2024-07-03T15:43:57.937 INFO:tasks.workunit.client.1.smithi148.stdout:7/830: symlink d5/d7d/da7/d1c/d33/d4c/d21/d31/dfc/l11e 0 2024-07-03T15:43:57.937 INFO:tasks.workunit.client.1.smithi148.stdout:7/831: stat d5/f7b 0 2024-07-03T15:43:57.938 INFO:tasks.workunit.client.1.smithi148.stdout:7/832: unlink d5/d7d/da7/d1c/d33/f67 0 2024-07-03T15:43:57.939 INFO:tasks.workunit.client.1.smithi148.stdout:7/833: chown d5/d7d/da7/d1c/d33/d4c/d21/d39/l43 766176 1 2024-07-03T15:43:57.944 INFO:tasks.workunit.client.1.smithi148.stdout:4/663: dread d0/d8/d13/d67/f86 [0,4194304] 0 2024-07-03T15:43:57.944 INFO:tasks.workunit.client.1.smithi148.stdout:7/834: dread d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/db2/d10d/f118 [0,4194304] 0 2024-07-03T15:43:57.944 INFO:tasks.workunit.client.1.smithi148.stdout:8/700: dwrite d4/da/fdf [0,4194304] 0 2024-07-03T15:43:57.946 INFO:tasks.workunit.client.1.smithi148.stdout:4/664: creat d0/d8/d3a/db8/dd3/deb/fed x:0 0 0 2024-07-03T15:43:57.946 INFO:tasks.workunit.client.1.smithi148.stdout:4/665: readlink d0/d28/da7/dbd/ld2 0 2024-07-03T15:43:57.946 INFO:tasks.workunit.client.1.smithi148.stdout:4/666: write d0/d8/d13/d48/fec [629727,124680] 0 2024-07-03T15:43:57.946 INFO:tasks.workunit.client.1.smithi148.stdout:4/667: readlink d0/d28/d7d/d6f/d7c/l8e 0 2024-07-03T15:43:57.947 INFO:tasks.workunit.client.1.smithi148.stdout:7/835: symlink d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/db2/d10d/l11f 0 2024-07-03T15:43:57.947 INFO:tasks.workunit.client.1.smithi148.stdout:7/836: write d5/d7d/da7/d1c/f71 [244699,88093] 0 2024-07-03T15:43:57.952 INFO:tasks.workunit.client.1.smithi148.stdout:8/701: mknod d4/da/d51/d53/cf7 0 2024-07-03T15:43:57.953 INFO:tasks.workunit.client.1.smithi148.stdout:8/702: creat d4/da/d14/ff8 x:0 0 0 2024-07-03T15:43:57.953 INFO:tasks.workunit.client.1.smithi148.stdout:4/668: creat d0/d28/de8/fee x:0 0 0 2024-07-03T15:43:57.953 INFO:tasks.workunit.client.1.smithi148.stdout:4/669: chown d0/d8/d3a/db9/fcf 1632 1 2024-07-03T15:43:57.954 INFO:tasks.workunit.client.1.smithi148.stdout:4/670: read - d0/d8/d13/d67/fe1 zero size 2024-07-03T15:43:57.956 INFO:tasks.workunit.client.1.smithi148.stdout:7/837: rename d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/db2/d10d/l11f to d5/d7d/da7/d1c/d33/d4c/d84/d98/l120 0 2024-07-03T15:43:57.956 INFO:tasks.workunit.client.1.smithi148.stdout:7/838: creat d5/d7d/da7/d1c/d33/d4c/d84/d98/d9e/f121 x:0 0 0 2024-07-03T15:43:57.956 INFO:tasks.workunit.client.1.smithi148.stdout:7/839: chown d5/d7d/da7/d1c/d4a/d66/d93/f9d 115983419 1 2024-07-03T15:43:57.957 INFO:tasks.workunit.client.1.smithi148.stdout:8/703: mkdir d4/da/d47/df9 0 2024-07-03T15:43:57.957 INFO:tasks.workunit.client.1.smithi148.stdout:8/704: chown d4/da/d14/d1a/d32/d89 1537 1 2024-07-03T15:43:57.957 INFO:tasks.workunit.client.1.smithi148.stdout:4/671: mknod d0/d8/d3a/db8/cef 0 2024-07-03T15:43:57.958 INFO:tasks.workunit.client.1.smithi148.stdout:7/840: mkdir d5/d7d/da7/d1c/d33/d4c/d21/d39/d122 0 2024-07-03T15:43:57.961 INFO:tasks.workunit.client.1.smithi148.stdout:8/705: link d4/da/d47/d88/d92/f4b d4/da/d14/d17/deb/ffa 0 2024-07-03T15:43:57.961 INFO:tasks.workunit.client.1.smithi148.stdout:8/706: read - d4/da/d51/d18/d20/d82/fc1 zero size 2024-07-03T15:43:57.961 INFO:tasks.workunit.client.1.smithi148.stdout:8/707: dread - d4/da/d51/d18/d20/d82/f9f zero size 2024-07-03T15:43:57.961 INFO:tasks.workunit.client.1.smithi148.stdout:3/761: sync 2024-07-03T15:43:57.961 INFO:tasks.workunit.client.1.smithi148.stdout:3/762: fdatasync dd/d35/d81/f91 0 2024-07-03T15:43:57.961 INFO:tasks.workunit.client.1.smithi148.stdout:9/944: sync 2024-07-03T15:43:57.961 INFO:tasks.workunit.client.1.smithi148.stdout:5/780: sync 2024-07-03T15:43:57.961 INFO:tasks.workunit.client.1.smithi148.stdout:0/614: sync 2024-07-03T15:43:57.961 INFO:tasks.workunit.client.1.smithi148.stdout:2/744: sync 2024-07-03T15:43:57.962 INFO:tasks.workunit.client.1.smithi148.stdout:6/754: sync 2024-07-03T15:43:57.962 INFO:tasks.workunit.client.1.smithi148.stdout:1/677: sync 2024-07-03T15:43:57.962 INFO:tasks.workunit.client.1.smithi148.stdout:5/781: fsync d7/d13/d54/f4b 0 2024-07-03T15:43:57.962 INFO:tasks.workunit.client.1.smithi148.stdout:0/615: chown da/d17/d2a/d2f/fc1 1224223 1 2024-07-03T15:43:57.962 INFO:tasks.workunit.client.1.smithi148.stdout:5/782: stat d7/da/df/d16/d2f/d44 0 2024-07-03T15:43:57.962 INFO:tasks.workunit.client.1.smithi148.stdout:9/945: write d0/dd/d25/d37/d40/da3/f115 [988882,100160] 0 2024-07-03T15:43:57.962 INFO:tasks.workunit.client.1.smithi148.stdout:5/783: write d7/da/df/d16/dac/dc1/ff6 [577733,109423] 0 2024-07-03T15:43:57.962 INFO:tasks.workunit.client.1.smithi148.stdout:5/784: write d7/da/df/d16/d2f/d44/f7c [327448,57174] 0 2024-07-03T15:43:57.962 INFO:tasks.workunit.client.1.smithi148.stdout:4/672: link d0/d28/da7/d2f/d41/f7a d0/d28/d7d/d6f/d7c/ff0 0 2024-07-03T15:43:57.962 INFO:tasks.workunit.client.1.smithi148.stdout:7/841: symlink d5/d7d/l123 0 2024-07-03T15:43:57.963 INFO:tasks.workunit.client.1.smithi148.stdout:8/708: rename d4/da/d14/d1a/le1 to d4/da/d14/d3a/lfb 0 2024-07-03T15:43:57.963 INFO:tasks.workunit.client.1.smithi148.stdout:3/763: creat dd/d33/d37/d68/d2e/d40/d77/ff3 x:0 0 0 2024-07-03T15:43:57.963 INFO:tasks.workunit.client.1.smithi148.stdout:3/764: chown dd/d33/d37/d68/c2d 14545351 1 2024-07-03T15:43:57.963 INFO:tasks.workunit.client.1.smithi148.stdout:3/765: creat dd/d33/d37/d8d/ff4 x:0 0 0 2024-07-03T15:43:57.963 INFO:tasks.workunit.client.1.smithi148.stdout:3/766: creat dd/da6/da2/ff5 x:0 0 0 2024-07-03T15:43:57.963 INFO:tasks.workunit.client.1.smithi148.stdout:2/745: truncate d5/d14/d86/fbc 6022481 0 2024-07-03T15:43:57.964 INFO:tasks.workunit.client.1.smithi148.stdout:1/678: symlink d0/de/d89/lee 0 2024-07-03T15:43:57.966 INFO:tasks.workunit.client.1.smithi148.stdout:0/616: symlink da/d58/dbd/d13/d1d/d3e/d85/d8e/db2/lc9 0 2024-07-03T15:43:57.966 INFO:tasks.workunit.client.1.smithi148.stdout:7/842: mkdir d5/d7d/da7/d1c/d59/d124 0 2024-07-03T15:43:57.967 INFO:tasks.workunit.client.1.smithi148.stdout:9/946: read d0/d49/d43/d19/f1e [343430,53389] 0 2024-07-03T15:43:57.967 INFO:tasks.workunit.client.1.smithi148.stdout:2/746: rmdir d5/da8 39 2024-07-03T15:43:57.967 INFO:tasks.workunit.client.1.smithi148.stdout:2/747: dread - d5/d14/d22/fd2 zero size 2024-07-03T15:43:57.968 INFO:tasks.workunit.client.1.smithi148.stdout:4/673: mknod d0/d8/d13/cf1 0 2024-07-03T15:43:57.969 INFO:tasks.workunit.client.1.smithi148.stdout:7/843: symlink d5/d7d/da7/d1c/d33/d4c/d21/d31/dfc/l125 0 2024-07-03T15:43:57.974 INFO:tasks.workunit.client.1.smithi148.stdout:9/947: creat d0/d49/d43/f130 x:0 0 0 2024-07-03T15:43:57.974 INFO:tasks.workunit.client.1.smithi148.stdout:9/948: creat d0/dd/d25/d37/f131 x:0 0 0 2024-07-03T15:43:57.976 INFO:tasks.workunit.client.1.smithi148.stdout:4/674: mkdir d0/d8/d3a/db8/dd3/df2 0 2024-07-03T15:43:57.976 INFO:tasks.workunit.client.1.smithi148.stdout:4/675: creat d0/d8/d3a/db8/dd3/deb/ff3 x:0 0 0 2024-07-03T15:43:57.976 INFO:tasks.workunit.client.1.smithi148.stdout:4/676: dread - d0/d8/dd4/fd9 zero size 2024-07-03T15:43:57.977 INFO:tasks.workunit.client.1.smithi148.stdout:4/677: dread - d0/d91/da8/dab/d49/fd0 zero size 2024-07-03T15:43:57.977 INFO:tasks.workunit.client.1.smithi148.stdout:0/617: dread da/d38/f7c [0,4194304] 0 2024-07-03T15:43:57.978 INFO:tasks.workunit.client.1.smithi148.stdout:0/618: fsync da/d58/dbd/d13/d14/d50/d95/d8c/f4d 0 2024-07-03T15:43:57.985 INFO:tasks.workunit.client.1.smithi148.stdout:7/844: rmdir d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/db6 39 2024-07-03T15:43:57.987 INFO:tasks.workunit.client.1.smithi148.stdout:8/709: dread d4/da/d14/d1a/d32/d38/dba/f95 [0,4194304] 0 2024-07-03T15:43:57.987 INFO:tasks.workunit.client.1.smithi148.stdout:8/710: creat d4/da/d14/d3a/da8/dd3/ffc x:0 0 0 2024-07-03T15:43:57.992 INFO:tasks.workunit.client.1.smithi148.stdout:3/767: dread f5 [4194304,4194304] 0 2024-07-03T15:43:57.996 INFO:tasks.workunit.client.1.smithi148.stdout:1/679: dwrite d0/d2/d11/f47 [0,4194304] 0 2024-07-03T15:43:57.996 INFO:tasks.workunit.client.1.smithi148.stdout:1/680: readlink d0/d2/d2e/lad 0 2024-07-03T15:43:57.996 INFO:tasks.workunit.client.1.smithi148.stdout:4/678: dread d0/d8/d3a/db9/fcf [0,4194304] 0 2024-07-03T15:43:57.997 INFO:tasks.workunit.client.1.smithi148.stdout:4/679: symlink d0/d28/d6e/dc7/lf4 0 2024-07-03T15:43:57.998 INFO:tasks.workunit.client.1.smithi148.stdout:1/681: rmdir d0/de/d44/d54/d5d 39 2024-07-03T15:43:58.004 INFO:tasks.workunit.client.1.smithi148.stdout:6/755: dwrite d3/d88/fed [0,4194304] 0 2024-07-03T15:43:58.007 INFO:tasks.workunit.client.1.smithi148.stdout:6/756: truncate d3/d1d/d28/d3e/ddd/d74/dc4/d85/fc7 341833 0 2024-07-03T15:43:58.009 INFO:tasks.workunit.client.1.smithi148.stdout:6/757: mknod d3/d1d/d28/d31/cf0 0 2024-07-03T15:43:58.009 INFO:tasks.workunit.client.1.smithi148.stdout:6/758: fsync d3/d1d/d28/d31/d61/f71 0 2024-07-03T15:43:58.012 INFO:tasks.workunit.client.1.smithi148.stdout:6/759: rename d3/d1d/d28/d53/f59 to d3/d1d/d28/d3e/ff1 0 2024-07-03T15:43:58.012 INFO:tasks.workunit.client.1.smithi148.stdout:1/682: dread d0/de/d44/f5e [0,4194304] 0 2024-07-03T15:43:58.013 INFO:tasks.workunit.client.1.smithi148.stdout:6/760: truncate d3/d50/f96 2555882 0 2024-07-03T15:43:58.013 INFO:tasks.workunit.client.1.smithi148.stdout:6/761: chown d3/d1d/d28/d53/dd4 316 1 2024-07-03T15:43:58.013 INFO:tasks.workunit.client.1.smithi148.stdout:6/762: creat d3/d1d/d28/d53/d6d/ff2 x:0 0 0 2024-07-03T15:43:58.013 INFO:tasks.workunit.client.1.smithi148.stdout:1/683: creat d0/d2/d11/d1e/d9d/de0/fef x:0 0 0 2024-07-03T15:43:58.014 INFO:tasks.workunit.client.1.smithi148.stdout:6/763: write d3/f55 [1364397,89818] 0 2024-07-03T15:43:58.014 INFO:tasks.workunit.client.1.smithi148.stdout:6/764: fsync d3/f86 0 2024-07-03T15:43:58.015 INFO:tasks.workunit.client.1.smithi148.stdout:1/684: unlink d0/de/d44/d54/fa1 0 2024-07-03T15:43:58.015 INFO:tasks.workunit.client.1.smithi148.stdout:1/685: dread - d0/d2/d11/fb2 zero size 2024-07-03T15:43:58.015 INFO:tasks.workunit.client.1.smithi148.stdout:1/686: readlink d0/d2/d11/d1e/l27 0 2024-07-03T15:43:58.016 INFO:tasks.workunit.client.1.smithi148.stdout:6/765: rename d3/d1d/d28/d3e/c7c to d3/d1d/d28/d3e/ddd/d74/dc4/cf3 0 2024-07-03T15:43:58.017 INFO:tasks.workunit.client.1.smithi148.stdout:1/687: creat d0/d2/ff0 x:0 0 0 2024-07-03T15:43:58.017 INFO:tasks.workunit.client.1.smithi148.stdout:6/766: mkdir d3/d1d/d28/d3e/df4 0 2024-07-03T15:43:58.018 INFO:tasks.workunit.client.1.smithi148.stdout:6/767: creat d3/d1d/d28/d31/ff5 x:0 0 0 2024-07-03T15:43:58.020 INFO:tasks.workunit.client.1.smithi148.stdout:6/768: mknod d3/d1d/d28/d53/d6d/d8f/dd3/cf6 0 2024-07-03T15:43:58.020 INFO:tasks.workunit.client.1.smithi148.stdout:6/769: write d3/d1d/d28/d3e/f58 [5417504,96976] 0 2024-07-03T15:43:58.020 INFO:tasks.workunit.client.1.smithi148.stdout:6/770: write d3/d1d/d28/d31/d61/fd7 [205608,53189] 0 2024-07-03T15:43:58.020 INFO:tasks.workunit.client.1.smithi148.stdout:6/771: chown d3/d1d/d28/d3e/ddd/d74/dc4/fe1 14368430 1 2024-07-03T15:43:58.020 INFO:tasks.workunit.client.1.smithi148.stdout:6/772: readlink d3/d1d/d28/d3e/ddd/d74/dc4/d85/ldf 0 2024-07-03T15:43:58.020 INFO:tasks.workunit.client.1.smithi148.stdout:6/773: creat d3/d1d/d28/d3e/ff7 x:0 0 0 2024-07-03T15:43:58.025 INFO:tasks.workunit.client.1.smithi148.stdout:0/619: dwrite da/d58/dbd/d13/d14/f99 [0,4194304] 0 2024-07-03T15:43:58.027 INFO:tasks.workunit.client.1.smithi148.stdout:0/620: link da/d58/dbd/d13/d14/l62 da/d58/dbd/d13/d1d/lca 0 2024-07-03T15:43:58.029 INFO:tasks.workunit.client.1.smithi148.stdout:0/621: mknod da/d58/d90/db4/ccb 0 2024-07-03T15:43:58.029 INFO:tasks.workunit.client.1.smithi148.stdout:9/949: dwrite d0/dd/d25/f3c [0,4194304] 0 2024-07-03T15:43:58.029 INFO:tasks.workunit.client.1.smithi148.stdout:5/785: dwrite d7/da/df/d16/d6a/db4/f9c [0,4194304] 0 2024-07-03T15:43:58.029 INFO:tasks.workunit.client.1.smithi148.stdout:5/786: dread - d7/da/df/d16/d2f/dc2/de9/f110 zero size 2024-07-03T15:43:58.030 INFO:tasks.workunit.client.1.smithi148.stdout:7/845: dwrite d5/d7d/da7/d1c/d33/d4c/d84/fbf [4194304,4194304] 0 2024-07-03T15:43:58.031 INFO:tasks.workunit.client.1.smithi148.stdout:2/748: dwrite d5/d14/d86/db9/dbb/d13/d1d/f43 [4194304,4194304] 0 2024-07-03T15:43:58.034 INFO:tasks.workunit.client.1.smithi148.stdout:5/787: mknod d7/c113 0 2024-07-03T15:43:58.034 INFO:tasks.workunit.client.1.smithi148.stdout:5/788: write d7/da/d21/f5e [389034,27567] 0 2024-07-03T15:43:58.036 INFO:tasks.workunit.client.1.smithi148.stdout:5/789: mknod d7/da/df/d1d/c114 0 2024-07-03T15:43:58.036 INFO:tasks.workunit.client.1.smithi148.stdout:5/790: write d7/da/df/d16/d1a/de5/f100 [1048083,74315] 0 2024-07-03T15:43:58.037 INFO:tasks.workunit.client.1.smithi148.stdout:5/791: fdatasync d7/da/fb5 0 2024-07-03T15:43:58.037 INFO:tasks.workunit.client.1.smithi148.stdout:5/792: truncate d7/da/df/d16/de4/da7/ff1 412508 0 2024-07-03T15:43:58.037 INFO:tasks.workunit.client.1.smithi148.stdout:5/793: mknod d7/da/df/d16/dfb/c115 0 2024-07-03T15:43:58.038 INFO:tasks.workunit.client.1.smithi148.stdout:8/711: dwrite d4/da/d51/f57 [0,4194304] 0 2024-07-03T15:43:58.040 INFO:tasks.workunit.client.1.smithi148.stdout:5/794: mkdir d7/da/df/d16/de4/da7/d116 0 2024-07-03T15:43:58.040 INFO:tasks.workunit.client.1.smithi148.stdout:3/768: dwrite dd/d33/d52/f60 [0,4194304] 0 2024-07-03T15:43:58.040 INFO:tasks.workunit.client.1.smithi148.stdout:3/769: creat dd/da6/d8b/ff6 x:0 0 0 2024-07-03T15:43:58.040 INFO:tasks.workunit.client.1.smithi148.stdout:3/770: write dd/d33/f6c [2315155,105711] 0 2024-07-03T15:43:58.042 INFO:tasks.workunit.client.1.smithi148.stdout:8/712: creat d4/da/d47/df9/ffd x:0 0 0 2024-07-03T15:43:58.042 INFO:tasks.workunit.client.1.smithi148.stdout:8/713: creat d4/da/d51/d18/d20/d82/ffe x:0 0 0 2024-07-03T15:43:58.042 INFO:tasks.workunit.client.1.smithi148.stdout:8/714: chown d4/da/d14/d17/c4e 22 1 2024-07-03T15:43:58.042 INFO:tasks.workunit.client.1.smithi148.stdout:8/715: dread - d4/d86/ff1 zero size 2024-07-03T15:43:58.042 INFO:tasks.workunit.client.1.smithi148.stdout:8/716: chown d4/da/d47/d88/c44 12 1 2024-07-03T15:43:58.048 INFO:tasks.workunit.client.1.smithi148.stdout:1/688: dwrite d0/de/d2f/f4e [0,4194304] 0 2024-07-03T15:43:58.052 INFO:tasks.workunit.client.1.smithi148.stdout:1/689: creat d0/d2/d11/d1e/d9d/dc0/ff1 x:0 0 0 2024-07-03T15:43:58.052 INFO:tasks.workunit.client.1.smithi148.stdout:1/690: chown d0/d2/d6/f5a 482632948 1 2024-07-03T15:43:58.054 INFO:tasks.workunit.client.1.smithi148.stdout:1/691: truncate d0/d2/d11/d1e/f59 755169 0 2024-07-03T15:43:58.054 INFO:tasks.workunit.client.1.smithi148.stdout:1/692: readlink d0/l88 0 2024-07-03T15:43:58.054 INFO:tasks.workunit.client.1.smithi148.stdout:8/717: dread d4/da/d14/d1a/f37 [0,4194304] 0 2024-07-03T15:43:58.057 INFO:tasks.workunit.client.1.smithi148.stdout:4/680: truncate d0/d8/d13/d48/fec 704902 0 2024-07-03T15:43:58.057 INFO:tasks.workunit.client.1.smithi148.stdout:6/774: dwrite d3/d1d/d28/d53/d6d/fc8 [0,4194304] 0 2024-07-03T15:43:58.057 INFO:tasks.workunit.client.1.smithi148.stdout:8/718: rmdir d4/da/d51/d53 39 2024-07-03T15:43:58.057 INFO:tasks.workunit.client.1.smithi148.stdout:8/719: creat d4/da/d14/d3a/da8/dd3/fff x:0 0 0 2024-07-03T15:43:58.058 INFO:tasks.workunit.client.1.smithi148.stdout:1/693: read d0/d2/d11/d1e/d9d/dc2/fd8 [132795,115029] 0 2024-07-03T15:43:58.058 INFO:tasks.workunit.client.1.smithi148.stdout:1/694: creat d0/d2/d6/ff2 x:0 0 0 2024-07-03T15:43:58.059 INFO:tasks.workunit.client.1.smithi148.stdout:4/681: truncate d0/d91/da8/dab/f75 557663 0 2024-07-03T15:43:58.060 INFO:tasks.workunit.client.1.smithi148.stdout:2/749: dwrite d5/d14/d86/db9/dbb/d68/f84 [0,4194304] 0 2024-07-03T15:43:58.061 INFO:tasks.workunit.client.1.smithi148.stdout:7/846: dread d5/d7d/da7/d1c/d23/fc1 [0,4194304] 0 2024-07-03T15:43:58.062 INFO:tasks.workunit.client.1.smithi148.stdout:6/775: dread d3/d1d/d28/d3e/f64 [0,4194304] 0 2024-07-03T15:43:58.062 INFO:tasks.workunit.client.1.smithi148.stdout:8/720: rmdir d4/da/d51/d18/d20 39 2024-07-03T15:43:58.062 INFO:tasks.workunit.client.1.smithi148.stdout:8/721: stat d4/da/d14/d1a/d32/d38/d58/dcb/ff5 0 2024-07-03T15:43:58.064 INFO:tasks.workunit.client.1.smithi148.stdout:9/950: dwrite d0/d2f/ffa [0,4194304] 0 2024-07-03T15:43:58.064 INFO:tasks.workunit.client.1.smithi148.stdout:9/951: readlink d0/dd/d11/lab 0 2024-07-03T15:43:58.067 INFO:tasks.workunit.client.1.smithi148.stdout:4/682: creat d0/d28/da7/d2f/d41/ff5 x:0 0 0 2024-07-03T15:43:58.067 INFO:tasks.workunit.client.1.smithi148.stdout:3/771: dwrite dd/d33/d37/d68/f5c [0,4194304] 0 2024-07-03T15:43:58.068 INFO:tasks.workunit.client.1.smithi148.stdout:5/795: dwrite d7/da/df/d16/dac/dc1/fe0 [0,4194304] 0 2024-07-03T15:43:58.068 INFO:tasks.workunit.client.1.smithi148.stdout:5/796: truncate d7/da/df/d16/de4/da7/ff0 640341 0 2024-07-03T15:43:58.070 INFO:tasks.workunit.client.1.smithi148.stdout:2/750: symlink d5/d14/d22/d60/d89/ddb/led 0 2024-07-03T15:43:58.071 INFO:tasks.workunit.client.1.smithi148.stdout:7/847: rmdir d5/d7d/da7/d1c/d33/d4c/d21/d31 39 2024-07-03T15:43:58.072 INFO:tasks.workunit.client.1.smithi148.stdout:6/776: dread d3/d1d/d28/d3e/ddd/d74/dc4/fce [0,4194304] 0 2024-07-03T15:43:58.072 INFO:tasks.workunit.client.1.smithi148.stdout:6/777: stat d3/d1d/d28/d31/d61/faf 0 2024-07-03T15:43:58.072 INFO:tasks.workunit.client.1.smithi148.stdout:6/778: chown d3/d1d/d28/d31/d4c/f54 0 1 2024-07-03T15:43:58.072 INFO:tasks.workunit.client.1.smithi148.stdout:9/952: getdents d0/dd/d25/d37/d40/da3/df0 0 2024-07-03T15:43:58.073 INFO:tasks.workunit.client.1.smithi148.stdout:4/683: creat d0/d8/dd4/ff6 x:0 0 0 2024-07-03T15:43:58.075 INFO:tasks.workunit.client.1.smithi148.stdout:8/722: dread d4/da/d14/d17/f26 [0,4194304] 0 2024-07-03T15:43:58.075 INFO:tasks.workunit.client.1.smithi148.stdout:8/723: truncate d4/da/d14/d1a/d32/d89/fbb 1004490 0 2024-07-03T15:43:58.075 INFO:tasks.workunit.client.1.smithi148.stdout:8/724: dread - d4/da/d14/d3a/da8/dd3/fff zero size 2024-07-03T15:43:58.075 INFO:tasks.workunit.client.1.smithi148.stdout:8/725: read d4/da/d47/d88/f27 [493148,11203] 0 2024-07-03T15:43:58.076 INFO:tasks.workunit.client.1.smithi148.stdout:8/726: creat d4/da/d14/f100 x:0 0 0 2024-07-03T15:43:58.076 INFO:tasks.workunit.client.1.smithi148.stdout:3/772: stat dd/f31 0 2024-07-03T15:43:58.076 INFO:tasks.workunit.client.1.smithi148.stdout:8/727: fdatasync d4/da/d14/d1a/f37 0 2024-07-03T15:43:58.080 INFO:tasks.workunit.client.1.smithi148.stdout:4/684: dread d0/d28/da7/d2f/f84 [0,4194304] 0 2024-07-03T15:43:58.085 INFO:tasks.workunit.client.1.smithi148.stdout:5/797: mkdir d7/da/df/d16/d2f/d44/d52/d60/d117 0 2024-07-03T15:43:58.085 INFO:tasks.workunit.client.1.smithi148.stdout:2/751: symlink d5/d14/d22/d60/d89/ddb/lee 0 2024-07-03T15:43:58.086 INFO:tasks.workunit.client.1.smithi148.stdout:6/779: stat d3/d1d/d28/d31/leb 0 2024-07-03T15:43:58.088 INFO:tasks.workunit.client.1.smithi148.stdout:9/953: mkdir d0/dd/d25/d37/d40/d132 0 2024-07-03T15:43:58.088 INFO:tasks.workunit.client.1.smithi148.stdout:9/954: truncate d0/dd/d25/d37/d40/d72/db8/fd6 1136439 0 2024-07-03T15:43:58.090 INFO:tasks.workunit.client.1.smithi148.stdout:8/728: unlink d4/da/d14/d1a/d32/d38/d58/fad 0 2024-07-03T15:43:58.091 INFO:tasks.workunit.client.1.smithi148.stdout:5/798: dread d7/f4f [4194304,4194304] 0 2024-07-03T15:43:58.091 INFO:tasks.workunit.client.1.smithi148.stdout:5/799: write d7/da/df/d16/de4/da7/ff0 [1661829,102614] 0 2024-07-03T15:43:58.093 INFO:tasks.workunit.client.1.smithi148.stdout:2/752: mkdir d5/d14/d86/db9/dbb/d68/def 0 2024-07-03T15:43:58.093 INFO:tasks.workunit.client.1.smithi148.stdout:2/753: fsync d5/d14/d86/db9/dbb/d68/dcd/f8e 0 2024-07-03T15:43:58.095 INFO:tasks.workunit.client.1.smithi148.stdout:9/955: creat d0/d49/d43/d19/d7a/daa/f133 x:0 0 0 2024-07-03T15:43:58.096 INFO:tasks.workunit.client.1.smithi148.stdout:8/729: readlink d4/da/d14/d1a/d32/d38/d58/l78 0 2024-07-03T15:43:58.097 INFO:tasks.workunit.client.1.smithi148.stdout:2/754: symlink d5/da8/lf0 0 2024-07-03T15:43:58.098 INFO:tasks.workunit.client.1.smithi148.stdout:6/780: dread d3/d1d/d28/d3e/f68 [4194304,4194304] 0 2024-07-03T15:43:58.098 INFO:tasks.workunit.client.1.smithi148.stdout:6/781: getdents d3/d1d/d28/d31/d61/dac 0 2024-07-03T15:43:58.100 INFO:tasks.workunit.client.1.smithi148.stdout:9/956: link d0/d49/c97 d0/dd/d46/c134 0 2024-07-03T15:43:58.100 INFO:tasks.workunit.client.1.smithi148.stdout:8/730: creat d4/da/d14/d1a/d32/f101 x:0 0 0 2024-07-03T15:43:58.100 INFO:tasks.workunit.client.1.smithi148.stdout:8/731: chown d4/da/d14/d1a/d32/d38/fb2 8 1 2024-07-03T15:43:58.100 INFO:tasks.workunit.client.1.smithi148.stdout:8/732: readlink d4/da/d51/d18/l24 0 2024-07-03T15:43:58.100 INFO:tasks.workunit.client.1.smithi148.stdout:8/733: chown d4/da/d47/d88/d92/f39 1 1 2024-07-03T15:43:58.101 INFO:tasks.workunit.client.1.smithi148.stdout:8/734: creat d4/da/d47/d88/d92/f102 x:0 0 0 2024-07-03T15:43:58.101 INFO:tasks.workunit.client.1.smithi148.stdout:9/957: symlink d0/d18/d24/l135 0 2024-07-03T15:43:58.101 INFO:tasks.workunit.client.1.smithi148.stdout:9/958: readlink d0/d18/d24/d47/l118 0 2024-07-03T15:43:58.103 INFO:tasks.workunit.client.1.smithi148.stdout:9/959: mknod d0/d18/d24/d60/c136 0 2024-07-03T15:43:58.111 INFO:tasks.workunit.client.1.smithi148.stdout:6/782: dread d3/d1d/d28/d3e/ddd/d74/dc4/fce [0,4194304] 0 2024-07-03T15:43:58.113 INFO:tasks.workunit.client.1.smithi148.stdout:6/783: unlink d3/da5/fcb 0 2024-07-03T15:43:58.115 INFO:tasks.workunit.client.1.smithi148.stdout:9/960: dread d0/fbb [4194304,4194304] 0 2024-07-03T15:43:58.115 INFO:tasks.workunit.client.1.smithi148.stdout:6/784: rename d3/d1d/d28/d3e/ddd/dea to d3/d88/df8 0 2024-07-03T15:43:58.117 INFO:tasks.workunit.client.1.smithi148.stdout:7/848: dwrite d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/db2/d10d/f118 [0,4194304] 0 2024-07-03T15:43:58.117 INFO:tasks.workunit.client.1.smithi148.stdout:6/785: mknod d3/d50/d77/cf9 0 2024-07-03T15:43:58.119 INFO:tasks.workunit.client.1.smithi148.stdout:7/849: mkdir d5/d7d/da7/d1c/d4a/d52/d107/d126 0 2024-07-03T15:43:58.120 INFO:tasks.workunit.client.1.smithi148.stdout:4/685: dwrite d0/f6 [4194304,4194304] 0 2024-07-03T15:43:58.120 INFO:tasks.workunit.client.1.smithi148.stdout:4/686: chown d0/d28/d7d/f8a 817515 1 2024-07-03T15:43:58.120 INFO:tasks.workunit.client.1.smithi148.stdout:4/687: write d0/d8/d13/d48/fec [1166528,69005] 0 2024-07-03T15:43:58.124 INFO:tasks.workunit.client.1.smithi148.stdout:3/773: dwrite dd/d33/f3e [0,4194304] 0 2024-07-03T15:43:58.124 INFO:tasks.workunit.client.1.smithi148.stdout:3/774: creat dd/d33/d37/d68/ff7 x:0 0 0 2024-07-03T15:43:58.124 INFO:tasks.workunit.client.1.smithi148.stdout:3/775: chown dd/d33/d37/d68/d2e/d61 2738 1 2024-07-03T15:43:58.124 INFO:tasks.workunit.client.1.smithi148.stdout:3/776: dread - dd/d35/f4e zero size 2024-07-03T15:43:58.124 INFO:tasks.workunit.client.1.smithi148.stdout:3/777: chown dd/d33/d37/d68/d2e/d40/f4c 50965394 1 2024-07-03T15:43:58.124 INFO:tasks.workunit.client.1.smithi148.stdout:3/778: read dd/d33/d37/d68/d2e/f13 [1225458,116993] 0 2024-07-03T15:43:58.124 INFO:tasks.workunit.client.1.smithi148.stdout:3/779: fsync dd/d33/d37/d68/d2e/d40/d45/fa3 0 2024-07-03T15:43:58.124 INFO:tasks.workunit.client.1.smithi148.stdout:8/735: dwrite d4/da/d14/d17/f26 [4194304,4194304] 0 2024-07-03T15:43:58.124 INFO:tasks.workunit.client.1.smithi148.stdout:4/688: mkdir d0/d8/d3a/dce/df7 0 2024-07-03T15:43:58.124 INFO:tasks.workunit.client.1.smithi148.stdout:2/755: dwrite d5/d14/d22/f62 [0,4194304] 0 2024-07-03T15:43:58.131 INFO:tasks.workunit.client.1.smithi148.stdout:3/780: creat dd/d33/d37/d68/d2e/d40/d45/ff8 x:0 0 0 2024-07-03T15:43:58.131 INFO:tasks.workunit.client.1.smithi148.stdout:3/781: truncate dd/d33/d37/d68/d2e/d40/d8c/fd4 268945 0 2024-07-03T15:43:58.131 INFO:tasks.workunit.client.1.smithi148.stdout:3/782: readlink dd/d33/d37/d68/d2e/d40/l71 0 2024-07-03T15:43:58.132 INFO:tasks.workunit.client.1.smithi148.stdout:4/689: creat d0/d8/d3a/db8/dd3/df2/ff8 x:0 0 0 2024-07-03T15:43:58.135 INFO:tasks.workunit.client.1.smithi148.stdout:3/783: mknod dd/d35/d41/d89/cf9 0 2024-07-03T15:43:58.135 INFO:tasks.workunit.client.1.smithi148.stdout:9/961: dwrite d0/dd/d11/f93 [0,4194304] 0 2024-07-03T15:43:58.135 INFO:tasks.workunit.client.1.smithi148.stdout:9/962: stat d0/d49/d43/f5b 0 2024-07-03T15:43:58.135 INFO:tasks.workunit.client.1.smithi148.stdout:4/690: symlink d0/d91/da8/dab/d49/lf9 0 2024-07-03T15:43:58.136 INFO:tasks.workunit.client.1.smithi148.stdout:4/691: fsync d0/d8/d13/d48/f2c 0 2024-07-03T15:43:58.136 INFO:tasks.workunit.client.1.smithi148.stdout:4/692: chown d0/d28/f63 64444671 1 2024-07-03T15:43:58.136 INFO:tasks.workunit.client.1.smithi148.stdout:4/693: readlink d0/d28/d6e/dc7/l3b 0 2024-07-03T15:43:58.136 INFO:tasks.workunit.client.1.smithi148.stdout:4/694: stat d0/d28/d6e 0 2024-07-03T15:43:58.136 INFO:tasks.workunit.client.1.smithi148.stdout:3/784: symlink dd/d33/d37/d68/d2e/d40/d77/dc9/dee/lfa 0 2024-07-03T15:43:58.136 INFO:tasks.workunit.client.1.smithi148.stdout:3/785: dread - dd/d33/d37/d68/d2e/d40/d77/d8f/fdd zero size 2024-07-03T15:43:58.137 INFO:tasks.workunit.client.1.smithi148.stdout:8/736: write d4/da/d14/d3a/f41 [6656814,111716] 0 2024-07-03T15:43:58.140 INFO:tasks.workunit.client.1.smithi148.stdout:2/756: dread d5/d14/d86/db9/dbb/d13/d2d/fb8 [0,4194304] 0 2024-07-03T15:43:58.141 INFO:tasks.workunit.client.1.smithi148.stdout:4/695: link d0/d91/da8/dab/d49/c51 d0/d28/d7d/cfa 0 2024-07-03T15:43:58.141 INFO:tasks.workunit.client.1.smithi148.stdout:4/696: stat d0/d28/d7d/d6f/d7c/dae/dba/dde 0 2024-07-03T15:43:58.141 INFO:tasks.workunit.client.1.smithi148.stdout:4/697: dread - d0/d28/da7/d2f/d41/ff5 zero size 2024-07-03T15:43:58.142 INFO:tasks.workunit.client.1.smithi148.stdout:3/786: creat dd/d33/d37/d68/d2e/d61/ffb x:0 0 0 2024-07-03T15:43:58.146 INFO:tasks.workunit.client.1.smithi148.stdout:8/737: symlink d4/da/d47/df9/l103 0 2024-07-03T15:43:58.146 INFO:tasks.workunit.client.1.smithi148.stdout:6/786: dwrite d3/d1d/d28/d3e/f64 [0,4194304] 0 2024-07-03T15:43:58.147 INFO:tasks.workunit.client.1.smithi148.stdout:4/698: dread d0/d8/d13/d48/f2c [0,4194304] 0 2024-07-03T15:43:58.148 INFO:tasks.workunit.client.1.smithi148.stdout:9/963: rename d0/dd/d25/d37/d40/d72 to d0/d2f/d137 0 2024-07-03T15:43:58.148 INFO:tasks.workunit.client.1.smithi148.stdout:9/964: dread - d0/d18/df3/da8/d12a/f12b zero size 2024-07-03T15:43:58.149 INFO:tasks.workunit.client.1.smithi148.stdout:9/965: creat d0/dd/d25/d37/f138 x:0 0 0 2024-07-03T15:43:58.150 INFO:tasks.workunit.client.1.smithi148.stdout:3/787: symlink dd/lfc 0 2024-07-03T15:43:58.159 INFO:tasks.workunit.client.1.smithi148.stdout:9/966: mknod d0/d2f/d137/c139 0 2024-07-03T15:43:58.159 INFO:tasks.workunit.client.1.smithi148.stdout:9/967: write d0/d49/d43/d19/d7a/daa/f133 [970587,55778] 0 2024-07-03T15:43:58.159 INFO:tasks.workunit.client.1.smithi148.stdout:9/968: fsync d0/dd/d11/db4/ff7 0 2024-07-03T15:43:58.159 INFO:tasks.workunit.client.1.smithi148.stdout:9/969: write d0/d49/d43/d19/f1e [2398695,88187] 0 2024-07-03T15:43:58.159 INFO:tasks.workunit.client.1.smithi148.stdout:9/970: write d0/dd/d25/d59/f10e [991882,37718] 0 2024-07-03T15:43:58.159 INFO:tasks.workunit.client.1.smithi148.stdout:9/971: truncate d0/dd/d9f/ff8 622902 0 2024-07-03T15:43:58.159 INFO:tasks.workunit.client.1.smithi148.stdout:3/788: mknod dd/da6/d8b/cfd 0 2024-07-03T15:43:58.161 INFO:tasks.workunit.client.1.smithi148.stdout:2/757: dwrite d5/d14/d22/dc0/fd8 [0,4194304] 0 2024-07-03T15:43:58.162 INFO:tasks.workunit.client.1.smithi148.stdout:8/738: rename d4/da/d51/d18/d20/d82/ffe to d4/da/d14/d1a/d32/d38/d58/f104 0 2024-07-03T15:43:58.162 INFO:tasks.workunit.client.1.smithi148.stdout:8/739: write d4/da/d14/d1a/d32/d38/dba/f95 [4969349,24518] 0 2024-07-03T15:43:58.163 INFO:tasks.workunit.client.1.smithi148.stdout:9/972: symlink d0/d49/d124/l13a 0 2024-07-03T15:43:58.166 INFO:tasks.workunit.client.1.smithi148.stdout:6/787: dwrite d3/d1d/d28/d31/d61/fd7 [0,4194304] 0 2024-07-03T15:43:58.166 INFO:tasks.workunit.client.1.smithi148.stdout:4/699: dwrite d0/d8/d13/d48/f2c [0,4194304] 0 2024-07-03T15:43:58.167 INFO:tasks.workunit.client.1.smithi148.stdout:2/758: creat d5/d14/d86/db9/dbb/d13/d41/d7b/ddf/d79/de4/ff1 x:0 0 0 2024-07-03T15:43:58.168 INFO:tasks.workunit.client.1.smithi148.stdout:9/973: mkdir d0/dd/d46/d13b 0 2024-07-03T15:43:58.168 INFO:tasks.workunit.client.1.smithi148.stdout:9/974: chown d0/d49/d43/d19/d7a/fb6 211953 1 2024-07-03T15:43:58.168 INFO:tasks.workunit.client.1.smithi148.stdout:9/975: dread - d0/dd/f125 zero size 2024-07-03T15:43:58.173 INFO:tasks.workunit.client.1.smithi148.stdout:6/788: rmdir d3/d1d/d28/d3e/ddd 39 2024-07-03T15:43:58.176 INFO:tasks.workunit.client.1.smithi148.stdout:4/700: mknod d0/d8/d44/dc2/cfb 0 2024-07-03T15:43:58.180 INFO:tasks.workunit.client.1.smithi148.stdout:9/976: rename d0/d18/d24/d47/fa5 to d0/d49/d43/d19/de1/f13c 0 2024-07-03T15:43:58.180 INFO:tasks.workunit.client.1.smithi148.stdout:0/622: sync 2024-07-03T15:43:58.180 INFO:tasks.workunit.client.1.smithi148.stdout:8/740: dwrite d4/da/d51/d18/d20/f40 [4194304,4194304] 0 2024-07-03T15:43:58.180 INFO:tasks.workunit.client.1.smithi148.stdout:8/741: readlink d4/da/d14/d3a/le4 0 2024-07-03T15:43:58.183 INFO:tasks.workunit.client.1.smithi148.stdout:4/701: creat d0/d8/d3a/db8/dd3/ffc x:0 0 0 2024-07-03T15:43:58.189 INFO:tasks.workunit.client.1.smithi148.stdout:8/742: dread d4/da/d14/d1a/d32/d89/fee [4194304,4194304] 0 2024-07-03T15:43:58.190 INFO:tasks.workunit.client.1.smithi148.stdout:8/743: write d4/da/d14/d3a/da8/db8/fe2 [706398,101163] 0 2024-07-03T15:43:58.190 INFO:tasks.workunit.client.1.smithi148.stdout:8/744: dread - d4/d86/ff1 zero size 2024-07-03T15:43:58.190 INFO:tasks.workunit.client.1.smithi148.stdout:8/745: chown d4/da/d14/d1a/d32/d38/d58/l65 12554036 1 2024-07-03T15:43:58.192 INFO:tasks.workunit.client.1.smithi148.stdout:2/759: rmdir d5/d14 39 2024-07-03T15:43:58.195 INFO:tasks.workunit.client.1.smithi148.stdout:6/789: mknod d3/d1d/cfa 0 2024-07-03T15:43:58.195 INFO:tasks.workunit.client.1.smithi148.stdout:6/790: read - d3/d1d/d28/fa6 zero size 2024-07-03T15:43:58.195 INFO:tasks.workunit.client.1.smithi148.stdout:6/791: chown d3/f17 242463868 1 2024-07-03T15:43:58.195 INFO:tasks.workunit.client.1.smithi148.stdout:6/792: write d3/d50/f72 [6553032,48258] 0 2024-07-03T15:43:58.195 INFO:tasks.workunit.client.1.smithi148.stdout:6/793: write d3/fc0 [851661,27038] 0 2024-07-03T15:43:58.196 INFO:tasks.workunit.client.1.smithi148.stdout:4/702: creat d0/d28/da7/ffd x:0 0 0 2024-07-03T15:43:58.199 INFO:tasks.workunit.client.1.smithi148.stdout:9/977: dwrite d0/d18/f21 [0,4194304] 0 2024-07-03T15:43:58.200 INFO:tasks.workunit.client.1.smithi148.stdout:4/703: mknod d0/d91/cfe 0 2024-07-03T15:43:58.200 INFO:tasks.workunit.client.1.smithi148.stdout:4/704: truncate d0/f6 9424034 0 2024-07-03T15:43:58.200 INFO:tasks.workunit.client.1.smithi148.stdout:4/705: dread - d0/d8/d3a/db9/fc9 zero size 2024-07-03T15:43:58.201 INFO:tasks.workunit.client.1.smithi148.stdout:4/706: mkdir d0/d8/dff 0 2024-07-03T15:43:58.201 INFO:tasks.workunit.client.1.smithi148.stdout:4/707: readlink d0/d28/da7/d2f/d52/lbe 0 2024-07-03T15:43:58.202 INFO:tasks.workunit.client.1.smithi148.stdout:4/708: truncate d0/d28/da7/d2f/d41/f7e 351638 0 2024-07-03T15:43:58.202 INFO:tasks.workunit.client.1.smithi148.stdout:4/709: stat d0/d28/da7/d2f/f33 0 2024-07-03T15:43:58.202 INFO:tasks.workunit.client.1.smithi148.stdout:4/710: dread - d0/d91/da8/dab/d49/fb2 zero size 2024-07-03T15:43:58.204 INFO:tasks.workunit.client.1.smithi148.stdout:4/711: symlink d0/d28/d7d/d6f/d7c/dae/dba/dde/l100 0 2024-07-03T15:43:58.204 INFO:tasks.workunit.client.1.smithi148.stdout:4/712: chown d0/d28/d7d/d6f/d7c/dae/dba/ld6 12108686 1 2024-07-03T15:43:58.204 INFO:tasks.workunit.client.1.smithi148.stdout:4/713: fdatasync d0/d91/da8/dab/d49/fd0 0 2024-07-03T15:43:58.206 INFO:tasks.workunit.client.1.smithi148.stdout:4/714: unlink d0/d8/c59 0 2024-07-03T15:43:58.210 INFO:tasks.workunit.client.1.smithi148.stdout:8/746: dwrite d4/da/f28 [4194304,4194304] 0 2024-07-03T15:43:58.214 INFO:tasks.workunit.client.1.smithi148.stdout:2/760: dread d5/d14/d86/db9/dbb/d13/d41/f74 [0,4194304] 0 2024-07-03T15:43:58.215 INFO:tasks.workunit.client.1.smithi148.stdout:8/747: mknod d4/da/d47/d88/dce/c105 0 2024-07-03T15:43:58.218 INFO:tasks.workunit.client.1.smithi148.stdout:8/748: dread d4/da/d14/d1a/d32/d89/fee [0,4194304] 0 2024-07-03T15:43:58.220 INFO:tasks.workunit.client.1.smithi148.stdout:6/794: dwrite d3/d50/d77/f94 [0,4194304] 0 2024-07-03T15:43:58.220 INFO:tasks.workunit.client.1.smithi148.stdout:6/795: chown d3/d1d/d28/d53/la0 80421 1 2024-07-03T15:43:58.222 INFO:tasks.workunit.client.1.smithi148.stdout:6/796: symlink d3/d1d/dd6/lfb 0 2024-07-03T15:43:58.222 INFO:tasks.workunit.client.1.smithi148.stdout:6/797: stat d3/d1d/d28/ccd 0 2024-07-03T15:43:58.223 INFO:tasks.workunit.client.1.smithi148.stdout:6/798: chown d3/d1d/d28 2536 1 2024-07-03T15:43:58.224 INFO:tasks.workunit.client.1.smithi148.stdout:6/799: mknod d3/d50/cfc 0 2024-07-03T15:43:58.224 INFO:tasks.workunit.client.1.smithi148.stdout:6/800: truncate d3/d1d/d28/d3e/ff7 531601 0 2024-07-03T15:43:58.225 INFO:tasks.workunit.client.1.smithi148.stdout:6/801: creat d3/d1d/ffd x:0 0 0 2024-07-03T15:43:58.225 INFO:tasks.workunit.client.1.smithi148.stdout:6/802: creat d3/d1d/d28/d31/d61/ffe x:0 0 0 2024-07-03T15:43:58.225 INFO:tasks.workunit.client.1.smithi148.stdout:6/803: write d3/d1d/d28/f73 [1580577,21394] 0 2024-07-03T15:43:58.226 INFO:tasks.workunit.client.1.smithi148.stdout:2/761: write d5/d14/d86/fbc [2112926,18377] 0 2024-07-03T15:43:58.226 INFO:tasks.workunit.client.1.smithi148.stdout:2/762: readlink d5/d14/d86/db9/dbb/d13/d41/l56 0 2024-07-03T15:43:58.226 INFO:tasks.workunit.client.1.smithi148.stdout:8/749: write d4/da/d51/d18/d20/f21 [453653,97140] 0 2024-07-03T15:43:58.227 INFO:tasks.workunit.client.1.smithi148.stdout:6/804: mkdir d3/d50/d77/dff 0 2024-07-03T15:43:58.228 INFO:tasks.workunit.client.1.smithi148.stdout:8/750: stat d4/da/d51/d18/c43 0 2024-07-03T15:43:58.234 INFO:tasks.workunit.client.1.smithi148.stdout:5/800: sync 2024-07-03T15:43:58.234 INFO:tasks.workunit.client.1.smithi148.stdout:1/695: sync 2024-07-03T15:43:58.234 INFO:tasks.workunit.client.1.smithi148.stdout:5/801: creat d7/da/df/d1d/f118 x:0 0 0 2024-07-03T15:43:58.234 INFO:tasks.workunit.client.1.smithi148.stdout:5/802: read d7/da/df/d16/f1b [1642894,6773] 0 2024-07-03T15:43:58.236 INFO:tasks.workunit.client.1.smithi148.stdout:5/803: mkdir d7/d13/d54/d119 0 2024-07-03T15:43:58.238 INFO:tasks.workunit.client.1.smithi148.stdout:6/805: dread d3/d1d/d28/d3e/f7b [0,4194304] 0 2024-07-03T15:43:58.239 INFO:tasks.workunit.client.1.smithi148.stdout:5/804: link d7/d13/d23/f3b d7/da/df/d16/d2f/d44/d52/d60/df2/f11a 0 2024-07-03T15:43:58.240 INFO:tasks.workunit.client.1.smithi148.stdout:9/978: dwrite d0/d18/df3/da8/d12a/f12b [0,4194304] 0 2024-07-03T15:43:58.241 INFO:tasks.workunit.client.1.smithi148.stdout:6/806: mkdir d3/d50/d77/d100 0 2024-07-03T15:43:58.242 INFO:tasks.workunit.client.1.smithi148.stdout:4/715: dwrite d0/d8/d3a/d9d/fd7 [0,4194304] 0 2024-07-03T15:43:58.242 INFO:tasks.workunit.client.1.smithi148.stdout:4/716: dread - d0/d8/dd4/fd9 zero size 2024-07-03T15:43:58.242 INFO:tasks.workunit.client.1.smithi148.stdout:5/805: unlink d7/da/df/d16/d2f/d44/d52/f5b 0 2024-07-03T15:43:58.245 INFO:tasks.workunit.client.1.smithi148.stdout:6/807: mkdir d3/d1d/d28/d3e/ddd/d74/dc4/d85/d8c/d101 0 2024-07-03T15:43:58.247 INFO:tasks.workunit.client.1.smithi148.stdout:5/806: mkdir d7/da/df/d16/d2f/d44/d52/d11b 0 2024-07-03T15:43:58.249 INFO:tasks.workunit.client.1.smithi148.stdout:5/807: link d7/da/df/d16/d2f/dc2/fda d7/da/df/d16/dac/dc1/dc4/f11c 0 2024-07-03T15:43:58.251 INFO:tasks.workunit.client.1.smithi148.stdout:9/979: rename d0/d49/d43 to d0/dd/d25/d13d 0 2024-07-03T15:43:58.251 INFO:tasks.workunit.client.1.smithi148.stdout:2/763: dwrite d5/d14/d86/db9/dbb/d13/d2d/f51 [0,4194304] 0 2024-07-03T15:43:58.253 INFO:tasks.workunit.client.1.smithi148.stdout:2/764: symlink d5/d14/d86/db9/dbb/d13/d2d/lf2 0 2024-07-03T15:43:58.254 INFO:tasks.workunit.client.1.smithi148.stdout:2/765: creat d5/d14/ff3 x:0 0 0 2024-07-03T15:43:58.255 INFO:tasks.workunit.client.1.smithi148.stdout:4/717: read d0/d28/da7/d2f/f33 [2560078,44689] 0 2024-07-03T15:43:58.255 INFO:tasks.workunit.client.1.smithi148.stdout:2/766: symlink d5/d14/d22/d60/d76/db3/lf4 0 2024-07-03T15:43:58.256 INFO:tasks.workunit.client.1.smithi148.stdout:1/696: dwrite d0/d2/d6/d14/f15 [0,4194304] 0 2024-07-03T15:43:58.257 INFO:tasks.workunit.client.1.smithi148.stdout:8/751: dwrite d4/da/d51/d18/d20/d82/fbd [0,4194304] 0 2024-07-03T15:43:58.257 INFO:tasks.workunit.client.1.smithi148.stdout:4/718: creat d0/d28/d7d/d6f/d7c/dae/f101 x:0 0 0 2024-07-03T15:43:58.258 INFO:tasks.workunit.client.1.smithi148.stdout:6/808: write d3/d1d/d28/d3e/f64 [3338564,24134] 0 2024-07-03T15:43:58.258 INFO:tasks.workunit.client.1.smithi148.stdout:2/767: creat d5/d14/d86/db9/dbb/d68/dcd/d64/db2/ff5 x:0 0 0 2024-07-03T15:43:58.258 INFO:tasks.workunit.client.1.smithi148.stdout:1/697: symlink d0/de/d44/d54/d5d/lf3 0 2024-07-03T15:43:58.262 INFO:tasks.workunit.client.1.smithi148.stdout:9/980: dread d0/dd/d25/d37/d40/da3/fe0 [0,4194304] 0 2024-07-03T15:43:58.265 INFO:tasks.workunit.client.1.smithi148.stdout:2/768: link d5/d14/d86/db9/dbb/d13/d1d/da1/fe5 d5/da8/dde/ff6 0 2024-07-03T15:43:58.266 INFO:tasks.workunit.client.1.smithi148.stdout:4/719: dread d0/d28/f69 [0,4194304] 0 2024-07-03T15:43:58.267 INFO:tasks.workunit.client.1.smithi148.stdout:6/809: mkdir d3/d1d/d28/d31/d61/d102 0 2024-07-03T15:43:58.268 INFO:tasks.workunit.client.1.smithi148.stdout:9/981: rename d0/dd/d25/c108 to d0/dd/d25/d37/d40/da3/df0/c13e 0 2024-07-03T15:43:58.269 INFO:tasks.workunit.client.1.smithi148.stdout:2/769: creat d5/d14/d86/db9/dbb/d13/ff7 x:0 0 0 2024-07-03T15:43:58.270 INFO:tasks.workunit.client.1.smithi148.stdout:8/752: write d4/da/d14/d1a/d32/d38/fd1 [4269524,96093] 0 2024-07-03T15:43:58.271 INFO:tasks.workunit.client.1.smithi148.stdout:4/720: mkdir d0/d28/d102 0 2024-07-03T15:43:58.271 INFO:tasks.workunit.client.1.smithi148.stdout:4/721: symlink d0/d28/db0/l103 0 2024-07-03T15:43:58.271 INFO:tasks.workunit.client.1.smithi148.stdout:4/722: chown d0/d8/d3a/dce 138 1 2024-07-03T15:43:58.272 INFO:tasks.workunit.client.1.smithi148.stdout:1/698: write d0/d2/d2e/fa7 [9846322,30263] 0 2024-07-03T15:43:58.274 INFO:tasks.workunit.client.1.smithi148.stdout:1/699: mknod d0/d2/d11/d3a/cf4 0 2024-07-03T15:43:58.276 INFO:tasks.workunit.client.1.smithi148.stdout:1/700: rmdir d0/d2/d2e/d39/d42/d62 39 2024-07-03T15:43:58.276 INFO:tasks.workunit.client.1.smithi148.stdout:1/701: chown d0/d2/d2e/d39/d42/d62/d7b 120596208 1 2024-07-03T15:43:58.276 INFO:tasks.workunit.client.1.smithi148.stdout:1/702: write d0/de/d2f/d72/fa5 [909541,53466] 0 2024-07-03T15:43:58.277 INFO:tasks.workunit.client.1.smithi148.stdout:8/753: dread d4/da/d14/d1a/d32/fae [0,4194304] 0 2024-07-03T15:43:58.277 INFO:tasks.workunit.client.1.smithi148.stdout:1/703: dread d0/d2/d6/d14/f8e [0,4194304] 0 2024-07-03T15:43:58.279 INFO:tasks.workunit.client.1.smithi148.stdout:8/754: mknod d4/da/d14/d1a/d32/d38/db7/c106 0 2024-07-03T15:43:58.279 INFO:tasks.workunit.client.1.smithi148.stdout:8/755: stat d4/da/d47/d88/d92/fda 0 2024-07-03T15:43:58.279 INFO:tasks.workunit.client.1.smithi148.stdout:8/756: dread - d4/da/d14/f100 zero size 2024-07-03T15:43:58.279 INFO:tasks.workunit.client.1.smithi148.stdout:8/757: chown d4/da/d14/d1a/d32/d38/d58/d8e/ff0 152 1 2024-07-03T15:43:58.279 INFO:tasks.workunit.client.1.smithi148.stdout:8/758: creat d4/da/f107 x:0 0 0 2024-07-03T15:43:58.279 INFO:tasks.workunit.client.1.smithi148.stdout:9/982: dread d0/dd/d25/d13d/d19/d7a/daa/fb1 [0,4194304] 0 2024-07-03T15:43:58.281 INFO:tasks.workunit.client.1.smithi148.stdout:6/810: write d3/d50/d77/f94 [282047,19953] 0 2024-07-03T15:43:58.281 INFO:tasks.workunit.client.1.smithi148.stdout:5/808: dwrite d7/da/df/d16/d2f/d44/d52/d60/fa6 [0,4194304] 0 2024-07-03T15:43:58.281 INFO:tasks.workunit.client.1.smithi148.stdout:5/809: fsync d7/da/df/d16/d2f/d44/f7c 0 2024-07-03T15:43:58.284 INFO:tasks.workunit.client.1.smithi148.stdout:8/759: symlink d4/da/d14/d1a/d32/d38/l108 0 2024-07-03T15:43:58.285 INFO:tasks.workunit.client.1.smithi148.stdout:6/811: mkdir d3/d1d/d28/d53/dd4/d103 0 2024-07-03T15:43:58.288 INFO:tasks.workunit.client.1.smithi148.stdout:5/810: stat d7/da/df/da0/db2/le2 0 2024-07-03T15:43:58.290 INFO:tasks.workunit.client.1.smithi148.stdout:5/811: rmdir d7/da/df/d16/d2f/dc2 39 2024-07-03T15:43:58.292 INFO:tasks.workunit.client.1.smithi148.stdout:2/770: dwrite d5/d14/ff3 [0,4194304] 0 2024-07-03T15:43:58.292 INFO:tasks.workunit.client.1.smithi148.stdout:2/771: fsync d5/d14/d22/d60/d76/f92 0 2024-07-03T15:43:58.292 INFO:tasks.workunit.client.1.smithi148.stdout:3/789: sync 2024-07-03T15:43:58.292 INFO:tasks.workunit.client.1.smithi148.stdout:7/850: sync 2024-07-03T15:43:58.293 INFO:tasks.workunit.client.1.smithi148.stdout:2/772: read d5/d14/f5a [546107,53231] 0 2024-07-03T15:43:58.293 INFO:tasks.workunit.client.1.smithi148.stdout:5/812: unlink d7/da/f74 0 2024-07-03T15:43:58.293 INFO:tasks.workunit.client.1.smithi148.stdout:5/813: stat d7/da/df/d16/d2f/d44/d52/d60/f92 0 2024-07-03T15:43:58.293 INFO:tasks.workunit.client.1.smithi148.stdout:5/814: truncate d7/da/df/d16/d6a/db4/fd9 4860102 0 2024-07-03T15:43:58.293 INFO:tasks.workunit.client.1.smithi148.stdout:5/815: fsync d7/da/fb5 0 2024-07-03T15:43:58.295 INFO:tasks.workunit.client.1.smithi148.stdout:7/851: mkdir d5/d7d/da7/d1c/d4a/d52/d127 0 2024-07-03T15:43:58.296 INFO:tasks.workunit.client.1.smithi148.stdout:3/790: creat dd/d33/df2/ffe x:0 0 0 2024-07-03T15:43:58.296 INFO:tasks.workunit.client.1.smithi148.stdout:3/791: write dd/d33/d37/d68/d2e/d40/d8c/fa5 [961386,42221] 0 2024-07-03T15:43:58.296 INFO:tasks.workunit.client.1.smithi148.stdout:8/760: dread d4/da/d14/f91 [0,4194304] 0 2024-07-03T15:43:58.296 INFO:tasks.workunit.client.1.smithi148.stdout:8/761: dread - d4/da/d14/d1a/d32/d38/d58/dcb/ff5 zero size 2024-07-03T15:43:58.296 INFO:tasks.workunit.client.1.smithi148.stdout:8/762: creat d4/da/d14/d1a/d32/d38/d58/d8e/f109 x:0 0 0 2024-07-03T15:43:58.296 INFO:tasks.workunit.client.1.smithi148.stdout:8/763: write d4/da/d51/d18/d20/fde [1022058,130162] 0 2024-07-03T15:43:58.297 INFO:tasks.workunit.client.1.smithi148.stdout:2/773: rename d5/da8/le0 to d5/d14/d86/db9/dbb/d68/def/lf8 0 2024-07-03T15:43:58.297 INFO:tasks.workunit.client.1.smithi148.stdout:5/816: truncate d7/da/df/d16/dac/dc1/dc4/ffd 255495 0 2024-07-03T15:43:58.299 INFO:tasks.workunit.client.1.smithi148.stdout:7/852: link d5/d7d/da7/db4/df0/f10e d5/d7d/da7/d1c/d33/d4c/d84/d91/daa/f128 0 2024-07-03T15:43:58.300 INFO:tasks.workunit.client.1.smithi148.stdout:7/853: dread d5/f9 [4194304,4194304] 0 2024-07-03T15:43:58.300 INFO:tasks.workunit.client.1.smithi148.stdout:3/792: unlink dd/d33/d37/d68/d2e/d40/d45/ff8 0 2024-07-03T15:43:58.303 INFO:tasks.workunit.client.1.smithi148.stdout:5/817: rename d7/da/df/d16/d2f/dc2/de9/fb7 to d7/da/df/d16/d6a/db4/f11d 0 2024-07-03T15:43:58.303 INFO:tasks.workunit.client.1.smithi148.stdout:5/818: truncate d7/da/df/d16/d1a/fd7 977398 0 2024-07-03T15:43:58.303 INFO:tasks.workunit.client.1.smithi148.stdout:5/819: creat d7/da/df/d16/d2f/f11e x:0 0 0 2024-07-03T15:43:58.304 INFO:tasks.workunit.client.1.smithi148.stdout:7/854: symlink d5/d7d/da7/d1c/d33/d4c/d21/l129 0 2024-07-03T15:43:58.307 INFO:tasks.workunit.client.1.smithi148.stdout:5/820: mkdir d7/d13/d54/d11f 0 2024-07-03T15:43:58.307 INFO:tasks.workunit.client.1.smithi148.stdout:5/821: fsync d7/d13/d23/f85 0 2024-07-03T15:43:58.308 INFO:tasks.workunit.client.1.smithi148.stdout:2/774: write d5/d14/d86/db9/dbb/d68/dcd/f3d [988444,37367] 0 2024-07-03T15:43:58.308 INFO:tasks.workunit.client.1.smithi148.stdout:2/775: fdatasync d5/d14/d86/db9/dbb/d13/d41/f49 0 2024-07-03T15:43:58.309 INFO:tasks.workunit.client.1.smithi148.stdout:7/855: rename d5/d7d/da7/d1c/d33/d4c/d21/d39/l53 to d5/d7d/da7/d1c/d4a/d66/dd7/l12a 0 2024-07-03T15:43:58.309 INFO:tasks.workunit.client.1.smithi148.stdout:3/793: rmdir dd/da6/d8b 39 2024-07-03T15:43:58.309 INFO:tasks.workunit.client.1.smithi148.stdout:3/794: truncate dd/d33/d37/d68/f30 4775741 0 2024-07-03T15:43:58.309 INFO:tasks.workunit.client.1.smithi148.stdout:3/795: dread - dd/d33/f87 zero size 2024-07-03T15:43:58.309 INFO:tasks.workunit.client.1.smithi148.stdout:3/796: truncate dd/d33/d37/d68/fed 676788 0 2024-07-03T15:43:58.309 INFO:tasks.workunit.client.1.smithi148.stdout:5/822: truncate d7/da/df/d16/f20 302590 0 2024-07-03T15:43:58.309 INFO:tasks.workunit.client.1.smithi148.stdout:5/823: dread - d7/da/df/d16/d2f/dc2/de9/f110 zero size 2024-07-03T15:43:58.311 INFO:tasks.workunit.client.1.smithi148.stdout:6/812: dwrite d3/fb7 [4194304,4194304] 0 2024-07-03T15:43:58.311 INFO:tasks.workunit.client.1.smithi148.stdout:4/723: dwrite d0/d28/da7/d2f/f33 [0,4194304] 0 2024-07-03T15:43:58.311 INFO:tasks.workunit.client.1.smithi148.stdout:1/704: dwrite d0/d2/f75 [4194304,4194304] 0 2024-07-03T15:43:58.312 INFO:tasks.workunit.client.1.smithi148.stdout:8/764: dwrite d4/da/d14/d1a/d32/d38/fc3 [0,4194304] 0 2024-07-03T15:43:58.312 INFO:tasks.workunit.client.1.smithi148.stdout:8/765: chown d4/da/d14/d1a/fd7 562 1 2024-07-03T15:43:58.318 INFO:tasks.workunit.client.1.smithi148.stdout:3/797: rename dd/d35/d81 to dd/d35/de1/dff 0 2024-07-03T15:43:58.320 INFO:tasks.workunit.client.1.smithi148.stdout:5/824: symlink d7/d13/l120 0 2024-07-03T15:43:58.327 INFO:tasks.workunit.client.1.smithi148.stdout:4/724: getdents d0/d91/da8 0 2024-07-03T15:43:58.327 INFO:tasks.workunit.client.1.smithi148.stdout:2/776: dwrite d5/d14/d22/fd2 [0,4194304] 0 2024-07-03T15:43:58.333 INFO:tasks.workunit.client.1.smithi148.stdout:7/856: dwrite d5/d7d/da7/d1c/d33/d4c/d21/d39/fb0 [0,4194304] 0 2024-07-03T15:43:58.333 INFO:tasks.workunit.client.1.smithi148.stdout:7/857: fdatasync d5/d7d/da7/d1c/d33/d4c/d21/f113 0 2024-07-03T15:43:58.334 INFO:tasks.workunit.client.1.smithi148.stdout:1/705: dwrite d0/d2/d11/d1e/d9d/dc2/de2/f63 [0,4194304] 0 2024-07-03T15:43:58.335 INFO:tasks.workunit.client.1.smithi148.stdout:8/766: rmdir d4/da/d14/d1a/d32/d38/d58/de0 0 2024-07-03T15:43:58.338 INFO:tasks.workunit.client.1.smithi148.stdout:3/798: creat dd/da6/da2/dbb/f100 x:0 0 0 2024-07-03T15:43:58.338 INFO:tasks.workunit.client.1.smithi148.stdout:3/799: chown dd/d33/d37/d68/d2e/d40 1 1 2024-07-03T15:43:58.341 INFO:tasks.workunit.client.1.smithi148.stdout:5/825: rename d7/da/df/d16/dac/dc1/dc4/fc9 to d7/d13/d23/f121 0 2024-07-03T15:43:58.342 INFO:tasks.workunit.client.1.smithi148.stdout:2/777: dwrite d5/d14/d8f/fb7 [0,4194304] 0 2024-07-03T15:43:58.343 INFO:tasks.workunit.client.1.smithi148.stdout:6/813: rmdir d3/d1d/d28/d31 39 2024-07-03T15:43:58.343 INFO:tasks.workunit.client.1.smithi148.stdout:6/814: chown d3/d1d/d28/c36 2374 1 2024-07-03T15:43:58.345 INFO:tasks.workunit.client.1.smithi148.stdout:4/725: creat d0/d8/d3a/dce/df7/f104 x:0 0 0 2024-07-03T15:43:58.351 INFO:tasks.workunit.client.1.smithi148.stdout:2/778: write d5/d14/d8f/fb7 [542385,94811] 0 2024-07-03T15:43:58.351 INFO:tasks.workunit.client.1.smithi148.stdout:2/779: dread - d5/d14/feb zero size 2024-07-03T15:43:58.352 INFO:tasks.workunit.client.1.smithi148.stdout:1/706: symlink d0/d2/d11/d1e/d9d/dc2/de2/lf5 0 2024-07-03T15:43:58.353 INFO:tasks.workunit.client.1.smithi148.stdout:3/800: mknod dd/da6/da2/c101 0 2024-07-03T15:43:58.355 INFO:tasks.workunit.client.1.smithi148.stdout:8/767: rename d4/da/d14/d9e/ced to d4/da/d14/dd2/c10a 0 2024-07-03T15:43:58.355 INFO:tasks.workunit.client.1.smithi148.stdout:8/768: fsync d4/da/d14/f6f 0 2024-07-03T15:43:58.355 INFO:tasks.workunit.client.1.smithi148.stdout:8/769: write d4/da/d14/d3a/da8/dd3/ffc [1002194,50114] 0 2024-07-03T15:43:58.358 INFO:tasks.workunit.client.1.smithi148.stdout:6/815: creat d3/d1d/d28/d3e/df4/f104 x:0 0 0 2024-07-03T15:43:58.358 INFO:tasks.workunit.client.1.smithi148.stdout:6/816: write d3/d1d/d28/d31/fe2 [447031,16469] 0 2024-07-03T15:43:58.359 INFO:tasks.workunit.client.1.smithi148.stdout:5/826: truncate d7/da/df/d16/f20 1259462 0 2024-07-03T15:43:58.360 INFO:tasks.workunit.client.1.smithi148.stdout:3/801: mkdir dd/d33/d37/d68/d2e/d40/d77/dcd/d102 0 2024-07-03T15:43:58.362 INFO:tasks.workunit.client.1.smithi148.stdout:8/770: symlink d4/da/d14/d17/deb/l10b 0 2024-07-03T15:43:58.362 INFO:tasks.workunit.client.1.smithi148.stdout:0/623: sync 2024-07-03T15:43:58.363 INFO:tasks.workunit.client.1.smithi148.stdout:3/802: mknod dd/d33/d37/d68/d67/c103 0 2024-07-03T15:43:58.364 INFO:tasks.workunit.client.1.smithi148.stdout:4/726: rename d0/d8/c18 to d0/d8/d13/c105 0 2024-07-03T15:43:58.364 INFO:tasks.workunit.client.1.smithi148.stdout:4/727: write d0/d8/d13/d67/db1/fc4 [231506,117856] 0 2024-07-03T15:43:58.365 INFO:tasks.workunit.client.1.smithi148.stdout:8/771: mknod d4/da/d14/c10c 0 2024-07-03T15:43:58.368 INFO:tasks.workunit.client.1.smithi148.stdout:0/624: rename da/d58/d90/db4/fc6 to da/d58/d90/db4/fcc 0 2024-07-03T15:43:58.369 INFO:tasks.workunit.client.1.smithi148.stdout:3/803: unlink dd/d33/d37/d68/c2d 0 2024-07-03T15:43:58.369 INFO:tasks.workunit.client.1.smithi148.stdout:3/804: fdatasync dd/d33/d37/d68/d2e/f18 0 2024-07-03T15:43:58.370 INFO:tasks.workunit.client.1.smithi148.stdout:2/780: rename d5/d14/d86/db9/dbb/d13/d41/d7b/ddf/c6b to d5/d14/d86/d93/cf9 0 2024-07-03T15:43:58.372 INFO:tasks.workunit.client.1.smithi148.stdout:0/625: rename da/d17/d2a/d2f/f9e to da/d17/d2a/d2f/fcd 0 2024-07-03T15:43:58.374 INFO:tasks.workunit.client.1.smithi148.stdout:3/805: creat dd/d35/de1/f104 x:0 0 0 2024-07-03T15:43:58.377 INFO:tasks.workunit.client.1.smithi148.stdout:6/817: rename d3/d1d/d28/d31/fab to d3/d1d/d28/d3e/ddd/d74/dc4/d85/f105 0 2024-07-03T15:43:58.377 INFO:tasks.workunit.client.1.smithi148.stdout:6/818: stat d3/le8 0 2024-07-03T15:43:58.378 INFO:tasks.workunit.client.1.smithi148.stdout:6/819: truncate d3/d1d/dd6/fe6 51426 0 2024-07-03T15:43:58.378 INFO:tasks.workunit.client.1.smithi148.stdout:6/820: fsync d3/d1d/d28/d31/d61/faf 0 2024-07-03T15:43:58.378 INFO:tasks.workunit.client.1.smithi148.stdout:6/821: truncate d3/d1d/d28/d53/d6d/fba 69739 0 2024-07-03T15:43:58.378 INFO:tasks.workunit.client.1.smithi148.stdout:6/822: chown d3/d1d/d28/d31/c47 255156 1 2024-07-03T15:43:58.378 INFO:tasks.workunit.client.1.smithi148.stdout:2/781: creat d5/d67/ffa x:0 0 0 2024-07-03T15:43:58.378 INFO:tasks.workunit.client.1.smithi148.stdout:2/782: fsync d5/f2f 0 2024-07-03T15:43:58.378 INFO:tasks.workunit.client.1.smithi148.stdout:2/783: write d5/d14/d86/db9/dbb/d13/f29 [929986,76650] 0 2024-07-03T15:43:58.379 INFO:tasks.workunit.client.1.smithi148.stdout:0/626: creat da/d58/dbd/d13/fce x:0 0 0 2024-07-03T15:43:58.381 INFO:tasks.workunit.client.1.smithi148.stdout:6/823: symlink d3/d1d/d28/d53/l106 0 2024-07-03T15:43:58.384 INFO:tasks.workunit.client.1.smithi148.stdout:0/627: creat da/d58/d90/fcf x:0 0 0 2024-07-03T15:43:58.384 INFO:tasks.workunit.client.1.smithi148.stdout:0/628: write da/d36/f63 [1678221,92656] 0 2024-07-03T15:43:58.388 INFO:tasks.workunit.client.1.smithi148.stdout:0/629: rename da/d17/d2a/f2d to da/d58/dbd/d4a/fd0 0 2024-07-03T15:43:58.388 INFO:tasks.workunit.client.1.smithi148.stdout:0/630: write da/d58/dbd/d24/d84/f9a [795034,122403] 0 2024-07-03T15:43:58.393 INFO:tasks.workunit.client.1.smithi148.stdout:0/631: rename da/d58/daa to da/d58/d90/dd1 0 2024-07-03T15:43:58.395 INFO:tasks.workunit.client.1.smithi148.stdout:3/806: write dd/d33/d37/d68/f30 [4746946,12176] 0 2024-07-03T15:43:58.395 INFO:tasks.workunit.client.1.smithi148.stdout:8/772: dwrite d4/da/d47/d88/d92/fa3 [0,4194304] 0 2024-07-03T15:43:58.395 INFO:tasks.workunit.client.1.smithi148.stdout:3/807: truncate dd/d33/d37/f82 5062264 0 2024-07-03T15:43:58.395 INFO:tasks.workunit.client.1.smithi148.stdout:3/808: readlink dd/d33/d37/d68/d2e/d40/d8c/dbc/le4 0 2024-07-03T15:43:58.395 INFO:tasks.workunit.client.1.smithi148.stdout:3/809: write dd/d33/d37/d68/ff7 [824105,5034] 0 2024-07-03T15:43:58.398 INFO:tasks.workunit.client.1.smithi148.stdout:5/827: dread d7/f4f [0,4194304] 0 2024-07-03T15:43:58.405 INFO:tasks.workunit.client.1.smithi148.stdout:0/632: dread da/d17/d2a/f59 [0,4194304] 0 2024-07-03T15:43:58.411 INFO:tasks.workunit.client.1.smithi148.stdout:8/773: dread d4/d86/f63 [0,4194304] 0 2024-07-03T15:43:58.411 INFO:tasks.workunit.client.1.smithi148.stdout:8/774: creat d4/da/d14/d17/f10d x:0 0 0 2024-07-03T15:43:58.411 INFO:tasks.workunit.client.1.smithi148.stdout:8/775: fdatasync d4/da/d51/d18/d20/d82/f9f 0 2024-07-03T15:43:58.411 INFO:tasks.workunit.client.1.smithi148.stdout:8/776: creat d4/da/d14/d1a/d32/d38/d58/dcb/f10e x:0 0 0 2024-07-03T15:43:58.414 INFO:tasks.workunit.client.1.smithi148.stdout:8/777: mknod d4/da/d14/dd2/c10f 0 2024-07-03T15:43:58.415 INFO:tasks.workunit.client.1.smithi148.stdout:8/778: fdatasync d4/da/f29 0 2024-07-03T15:43:58.415 INFO:tasks.workunit.client.1.smithi148.stdout:8/779: truncate d4/da/d14/f67 3218649 0 2024-07-03T15:43:58.415 INFO:tasks.workunit.client.1.smithi148.stdout:8/780: readlink d4/da/d14/d3a/le4 0 2024-07-03T15:43:58.418 INFO:tasks.workunit.client.1.smithi148.stdout:8/781: dread d4/da/d51/d18/f8c [0,4194304] 0 2024-07-03T15:43:58.419 INFO:tasks.workunit.client.1.smithi148.stdout:8/782: dread - d4/da/d51/fe8 zero size 2024-07-03T15:43:58.420 INFO:tasks.workunit.client.1.smithi148.stdout:8/783: unlink d4/da/d14/d3a/da8/dd3/lf3 0 2024-07-03T15:43:58.420 INFO:tasks.workunit.client.1.smithi148.stdout:8/784: read - d4/da/d14/f100 zero size 2024-07-03T15:43:58.421 INFO:tasks.workunit.client.1.smithi148.stdout:2/784: dwrite d5/d14/f7a [0,4194304] 0 2024-07-03T15:43:58.421 INFO:tasks.workunit.client.1.smithi148.stdout:2/785: stat d5/d14/d8b/dd5/cd6 0 2024-07-03T15:43:58.421 INFO:tasks.workunit.client.1.smithi148.stdout:2/786: read d5/d14/d86/db9/dbb/d13/d2d/fb8 [921625,23869] 0 2024-07-03T15:43:58.421 INFO:tasks.workunit.client.1.smithi148.stdout:8/785: symlink d4/da/d47/df9/l110 0 2024-07-03T15:43:58.421 INFO:tasks.workunit.client.1.smithi148.stdout:8/786: readlink d4/da/d14/d17/deb/l10b 0 2024-07-03T15:43:58.422 INFO:tasks.workunit.client.1.smithi148.stdout:1/707: dwrite d0/d2/d11/f47 [0,4194304] 0 2024-07-03T15:43:58.422 INFO:tasks.workunit.client.1.smithi148.stdout:2/787: getdents d5/d14/d86/db9/dbb/d13/d41/d7b/ddf/d79 0 2024-07-03T15:43:58.422 INFO:tasks.workunit.client.1.smithi148.stdout:2/788: chown d5/d14/d86/db9/dbb/d13/d41/d7b/d6a/d80 24 1 2024-07-03T15:43:58.426 INFO:tasks.workunit.client.1.smithi148.stdout:2/789: read d5/f1f [1879710,50872] 0 2024-07-03T15:43:58.429 INFO:tasks.workunit.client.1.smithi148.stdout:5/828: dwrite d7/d13/d23/fee [4194304,4194304] 0 2024-07-03T15:43:58.429 INFO:tasks.workunit.client.1.smithi148.stdout:3/810: dwrite dd/d33/d37/d68/d2e/d40/d45/d73/fb5 [0,4194304] 0 2024-07-03T15:43:58.429 INFO:tasks.workunit.client.1.smithi148.stdout:3/811: read f5 [6597980,48608] 0 2024-07-03T15:43:58.430 INFO:tasks.workunit.client.1.smithi148.stdout:6/824: dwrite d3/d1d/d28/d53/f6e [0,4194304] 0 2024-07-03T15:43:58.430 INFO:tasks.workunit.client.1.smithi148.stdout:6/825: read - d3/d1d/d28/fa6 zero size 2024-07-03T15:43:58.430 INFO:tasks.workunit.client.1.smithi148.stdout:6/826: chown d3/d1d/d28/d31/ff5 1239656250 1 2024-07-03T15:43:58.430 INFO:tasks.workunit.client.1.smithi148.stdout:6/827: creat d3/da5/f107 x:0 0 0 2024-07-03T15:43:58.430 INFO:tasks.workunit.client.1.smithi148.stdout:1/708: write d0/de/f83 [3130395,12414] 0 2024-07-03T15:43:58.431 INFO:tasks.workunit.client.1.smithi148.stdout:0/633: dwrite da/d58/dbd/d13/d1d/d3e/f6e [0,4194304] 0 2024-07-03T15:43:58.431 INFO:tasks.workunit.client.1.smithi148.stdout:0/634: write da/d58/dbd/d24/f89 [3602473,87210] 0 2024-07-03T15:43:58.431 INFO:tasks.workunit.client.1.smithi148.stdout:5/829: creat d7/da/df/d16/de4/da7/f122 x:0 0 0 2024-07-03T15:43:58.432 INFO:tasks.workunit.client.1.smithi148.stdout:3/812: mkdir dd/d33/d37/d105 0 2024-07-03T15:43:58.433 INFO:tasks.workunit.client.1.smithi148.stdout:3/813: chown dd/d35/d41/d89/cf9 81526 1 2024-07-03T15:43:58.433 INFO:tasks.workunit.client.1.smithi148.stdout:5/830: dread d7/da/df/f3d [0,4194304] 0 2024-07-03T15:43:58.437 INFO:tasks.workunit.client.1.smithi148.stdout:6/828: creat d3/d1d/d28/d53/dd4/f108 x:0 0 0 2024-07-03T15:43:58.442 INFO:tasks.workunit.client.1.smithi148.stdout:0/635: symlink da/d58/dbd/d13/d5a/ld2 0 2024-07-03T15:43:58.444 INFO:tasks.workunit.client.1.smithi148.stdout:3/814: creat dd/d35/f106 x:0 0 0 2024-07-03T15:43:58.444 INFO:tasks.workunit.client.1.smithi148.stdout:3/815: readlink dd/d33/d52/l5a 0 2024-07-03T15:43:58.445 INFO:tasks.workunit.client.1.smithi148.stdout:5/831: unlink d7/da/df/d16/d2f/d44/d52/d60/df2/f11a 0 2024-07-03T15:43:58.446 INFO:tasks.workunit.client.1.smithi148.stdout:6/829: mkdir d3/d50/d77/dff/d109 0 2024-07-03T15:43:58.447 INFO:tasks.workunit.client.1.smithi148.stdout:0/636: mknod da/d58/dbd/d94/cd3 0 2024-07-03T15:43:58.448 INFO:tasks.workunit.client.1.smithi148.stdout:0/637: mkdir da/d58/dbd/d13/d1d/d3e/d85/d8e/dd4 0 2024-07-03T15:43:58.450 INFO:tasks.workunit.client.1.smithi148.stdout:0/638: unlink da/d58/dbd/d13/d14/d50/d95/d8c/fb7 0 2024-07-03T15:43:58.450 INFO:tasks.workunit.client.1.smithi148.stdout:0/639: dread - da/d58/dbd/d13/fce zero size 2024-07-03T15:43:58.453 INFO:tasks.workunit.client.1.smithi148.stdout:1/709: dwrite d0/d2/f75 [4194304,4194304] 0 2024-07-03T15:43:58.453 INFO:tasks.workunit.client.1.smithi148.stdout:1/710: fsync d0/d2/d2e/f85 0 2024-07-03T15:43:58.453 INFO:tasks.workunit.client.1.smithi148.stdout:1/711: write d0/d2/d11/d1e/d9d/dc2/de2/f69 [7213420,39056] 0 2024-07-03T15:43:58.453 INFO:tasks.workunit.client.1.smithi148.stdout:1/712: truncate d0/d2/d2e/d39/d42/d62/d7b/fcc 560235 0 2024-07-03T15:43:58.457 INFO:tasks.workunit.client.1.smithi148.stdout:1/713: unlink d0/de/d2f/f4e 0 2024-07-03T15:43:58.457 INFO:tasks.workunit.client.1.smithi148.stdout:1/714: read d0/d2/d2e/f85 [454063,24147] 0 2024-07-03T15:43:58.466 INFO:tasks.workunit.client.1.smithi148.stdout:9/983: sync 2024-07-03T15:43:58.467 INFO:tasks.workunit.client.1.smithi148.stdout:9/984: fdatasync d0/dd/d25/d37/d40/da3/fd3 0 2024-07-03T15:43:58.467 INFO:tasks.workunit.client.1.smithi148.stdout:9/985: link d0/dd/d46/fe4 d0/dd/d9f/f13f 0 2024-07-03T15:43:58.477 INFO:tasks.workunit.client.1.smithi148.stdout:3/816: dwrite dd/d33/d37/d68/f30 [0,4194304] 0 2024-07-03T15:43:58.478 INFO:tasks.workunit.client.1.smithi148.stdout:0/640: dwrite da/d58/dbd/d13/d14/d50/d95/d8c/fb6 [0,4194304] 0 2024-07-03T15:43:58.478 INFO:tasks.workunit.client.1.smithi148.stdout:3/817: write dd/d33/d37/d68/d2e/d61/f63 [164736,20255] 0 2024-07-03T15:43:58.479 INFO:tasks.workunit.client.1.smithi148.stdout:3/818: chown dd/d33/d37/d68/d2e/d61/lb1 8 1 2024-07-03T15:43:58.479 INFO:tasks.workunit.client.1.smithi148.stdout:6/830: dwrite d3/f1e [0,4194304] 0 2024-07-03T15:43:58.482 INFO:tasks.workunit.client.1.smithi148.stdout:1/715: dwrite d0/de/d2f/d72/f7c [0,4194304] 0 2024-07-03T15:43:58.482 INFO:tasks.workunit.client.1.smithi148.stdout:0/641: symlink da/d17/d2a/ld5 0 2024-07-03T15:43:58.486 INFO:tasks.workunit.client.1.smithi148.stdout:6/831: write d3/d50/f96 [1737154,42645] 0 2024-07-03T15:43:58.487 INFO:tasks.workunit.client.1.smithi148.stdout:0/642: mknod da/d58/d90/dd1/cd6 0 2024-07-03T15:43:58.488 INFO:tasks.workunit.client.1.smithi148.stdout:6/832: mkdir d3/d1d/dd6/d10a 0 2024-07-03T15:43:58.488 INFO:tasks.workunit.client.1.smithi148.stdout:6/833: truncate d3/f2f 880299 0 2024-07-03T15:43:58.489 INFO:tasks.workunit.client.1.smithi148.stdout:0/643: stat da/d38/c40 0 2024-07-03T15:43:58.490 INFO:tasks.workunit.client.1.smithi148.stdout:6/834: unlink d3/d50/f72 0 2024-07-03T15:43:58.491 INFO:tasks.workunit.client.1.smithi148.stdout:6/835: truncate d3/d1d/d28/d31/d61/f6b 2132243 0 2024-07-03T15:43:58.491 INFO:tasks.workunit.client.1.smithi148.stdout:6/836: truncate d3/fb0 175650 0 2024-07-03T15:43:58.492 INFO:tasks.workunit.client.1.smithi148.stdout:9/986: rmdir d0/dd/d9f 39 2024-07-03T15:43:58.492 INFO:tasks.workunit.client.1.smithi148.stdout:9/987: fdatasync d0/d18/df3/f82 0 2024-07-03T15:43:58.494 INFO:tasks.workunit.client.1.smithi148.stdout:1/716: dread d0/d2/d2e/d39/d42/d62/f4f [0,4194304] 0 2024-07-03T15:43:58.494 INFO:tasks.workunit.client.1.smithi148.stdout:0/644: dread da/d17/f8b [0,4194304] 0 2024-07-03T15:43:58.496 INFO:tasks.workunit.client.1.smithi148.stdout:1/717: creat d0/d2/d6/d14/ff6 x:0 0 0 2024-07-03T15:43:58.496 INFO:tasks.workunit.client.1.smithi148.stdout:1/718: chown d0/d2/d11/d3a 20 1 2024-07-03T15:43:58.498 INFO:tasks.workunit.client.1.smithi148.stdout:0/645: rmdir da/d17/d2a/d2f/d93/da3 39 2024-07-03T15:43:58.498 INFO:tasks.workunit.client.1.smithi148.stdout:0/646: write da/d17/d2a/d2f/fcd [1111302,13721] 0 2024-07-03T15:43:58.498 INFO:tasks.workunit.client.1.smithi148.stdout:0/647: fdatasync da/d17/f8b 0 2024-07-03T15:43:58.501 INFO:tasks.workunit.client.1.smithi148.stdout:3/819: dwrite dd/d35/d41/f84 [0,4194304] 0 2024-07-03T15:43:58.504 INFO:tasks.workunit.client.1.smithi148.stdout:6/837: dread d3/f1e [0,4194304] 0 2024-07-03T15:43:58.504 INFO:tasks.workunit.client.1.smithi148.stdout:6/838: fsync d3/d1d/d28/d31/d4c/fc2 0 2024-07-03T15:43:58.506 INFO:tasks.workunit.client.1.smithi148.stdout:1/719: write d0/d2/d2e/d39/d42/d62/f6e [5282936,29866] 0 2024-07-03T15:43:58.506 INFO:tasks.workunit.client.1.smithi148.stdout:1/720: readlink d0/d2/d6/d14/d29/l32 0 2024-07-03T15:43:58.506 INFO:tasks.workunit.client.1.smithi148.stdout:1/721: fsync d0/d2/d11/d1e/d9d/dc2/de2/d77/f92 0 2024-07-03T15:43:58.506 INFO:tasks.workunit.client.1.smithi148.stdout:6/839: symlink d3/d50/d77/dff/l10b 0 2024-07-03T15:43:58.508 INFO:tasks.workunit.client.1.smithi148.stdout:1/722: mknod d0/d2/d6/d14/d29/cf7 0 2024-07-03T15:43:58.508 INFO:tasks.workunit.client.1.smithi148.stdout:9/988: dwrite d0/dd/f2b [4194304,4194304] 0 2024-07-03T15:43:58.508 INFO:tasks.workunit.client.1.smithi148.stdout:9/989: write d0/dd/d25/d37/d40/da3/f115 [1879085,23232] 0 2024-07-03T15:43:58.509 INFO:tasks.workunit.client.1.smithi148.stdout:9/990: chown d0/dd/d25/d37/d40/da3/f11e 34 1 2024-07-03T15:43:58.509 INFO:tasks.workunit.client.1.smithi148.stdout:9/991: creat d0/dd/d25/d37/d40/da3/f140 x:0 0 0 2024-07-03T15:43:58.509 INFO:tasks.workunit.client.1.smithi148.stdout:6/840: stat d3/d1d/d28/l30 0 2024-07-03T15:43:58.509 INFO:tasks.workunit.client.1.smithi148.stdout:6/841: fdatasync d3/da5/f107 0 2024-07-03T15:43:58.512 INFO:tasks.workunit.client.1.smithi148.stdout:9/992: symlink d0/dd/d25/d13d/dd8/l141 0 2024-07-03T15:43:58.514 INFO:tasks.workunit.client.1.smithi148.stdout:6/842: symlink d3/d1d/d28/d3e/ddd/d74/l10c 0 2024-07-03T15:43:58.515 INFO:tasks.workunit.client.1.smithi148.stdout:9/993: mkdir d0/dd/d25/d37/d40/d132/d142 0 2024-07-03T15:43:58.515 INFO:tasks.workunit.client.1.smithi148.stdout:9/994: write d0/d18/d24/f11f [188858,95997] 0 2024-07-03T15:43:58.517 INFO:tasks.workunit.client.1.smithi148.stdout:1/723: dread d0/d2/f75 [4194304,4194304] 0 2024-07-03T15:43:58.517 INFO:tasks.workunit.client.1.smithi148.stdout:1/724: write d0/d2/d2e/d39/d42/d62/f9a [1514618,91268] 0 2024-07-03T15:43:58.524 INFO:tasks.workunit.client.1.smithi148.stdout:0/648: dwrite da/d58/f79 [4194304,4194304] 0 2024-07-03T15:43:58.526 INFO:tasks.workunit.client.1.smithi148.stdout:9/995: dread d0/dd/d25/d13d/d19/de1/f13c [0,4194304] 0 2024-07-03T15:43:58.527 INFO:tasks.workunit.client.1.smithi148.stdout:0/649: creat da/d58/dbd/d94/dba/fd7 x:0 0 0 2024-07-03T15:43:58.529 INFO:tasks.workunit.client.1.smithi148.stdout:9/996: rename d0/dd/d11/db4/fc0 to d0/dd/d25/d13d/d19/d7a/daa/f143 0 2024-07-03T15:43:58.529 INFO:tasks.workunit.client.1.smithi148.stdout:9/997: creat d0/dd/f144 x:0 0 0 2024-07-03T15:43:58.529 INFO:tasks.workunit.client.1.smithi148.stdout:9/998: dread - d0/dd/f125 zero size 2024-07-03T15:43:58.529 INFO:tasks.workunit.client.1.smithi148.stdout:9/999: dread - d0/f102 zero size 2024-07-03T15:43:58.543 INFO:tasks.workunit.client.1.smithi148.stdout:6/843: dwrite d3/d1d/d28/d3e/f58 [0,4194304] 0 2024-07-03T15:43:58.544 INFO:tasks.workunit.client.1.smithi148.stdout:3/820: dwrite dd/d33/d37/d68/d2e/f11 [4194304,4194304] 0 2024-07-03T15:43:58.544 INFO:tasks.workunit.client.1.smithi148.stdout:0/650: dwrite da/d58/dbd/d24/dc8/d8a/dab/f6c [4194304,4194304] 0 2024-07-03T15:43:58.545 INFO:tasks.workunit.client.1.smithi148.stdout:6/844: mknod d3/d50/d77/dff/d109/c10d 0 2024-07-03T15:43:58.545 INFO:tasks.workunit.client.1.smithi148.stdout:6/845: write d3/d1d/d28/d31/d4c/fc2 [7565,71950] 0 2024-07-03T15:43:58.545 INFO:tasks.workunit.client.1.smithi148.stdout:3/821: symlink dd/d33/d37/d8d/daf/l107 0 2024-07-03T15:43:58.545 INFO:tasks.workunit.client.1.smithi148.stdout:3/822: readlink dd/d33/d52/d57/lbe 0 2024-07-03T15:43:58.545 INFO:tasks.workunit.client.1.smithi148.stdout:3/823: fdatasync dd/d33/d37/d68/d2e/d40/d77/fbd 0 2024-07-03T15:43:58.548 INFO:tasks.workunit.client.1.smithi148.stdout:0/651: rename da/d36/dc4 to da/d58/dbd/d13/d1d/d3e/dd8 0 2024-07-03T15:43:58.548 INFO:tasks.workunit.client.1.smithi148.stdout:0/652: creat da/d58/dbd/d13/fd9 x:0 0 0 2024-07-03T15:43:58.548 INFO:tasks.workunit.client.1.smithi148.stdout:3/824: rmdir dd/da6/d8b 39 2024-07-03T15:43:58.548 INFO:tasks.workunit.client.1.smithi148.stdout:3/825: creat dd/d33/d37/d68/d2e/d40/d45/f108 x:0 0 0 2024-07-03T15:43:58.549 INFO:tasks.workunit.client.1.smithi148.stdout:3/826: chown dd/d33/d52/c79 3 1 2024-07-03T15:43:58.549 INFO:tasks.workunit.client.1.smithi148.stdout:6/846: dread d3/d1d/d28/d3e/ddd/d74/dc4/fce [0,4194304] 0 2024-07-03T15:43:58.552 INFO:tasks.workunit.client.1.smithi148.stdout:0/653: unlink da/d58/dbd/d24/d84/f9a 0 2024-07-03T15:43:58.555 INFO:tasks.workunit.client.1.smithi148.stdout:3/827: mkdir dd/d33/d37/d68/d2e/d40/d77/dcd/d109 0 2024-07-03T15:43:58.556 INFO:tasks.workunit.client.1.smithi148.stdout:6/847: mknod d3/d1d/d28/d31/d61/dac/c10e 0 2024-07-03T15:43:58.558 INFO:tasks.workunit.client.1.smithi148.stdout:0/654: mkdir da/d17/d2a/d2f/d93/da3/dda 0 2024-07-03T15:43:58.558 INFO:tasks.workunit.client.1.smithi148.stdout:0/655: write da/d58/dbd/d13/f71 [593489,104346] 0 2024-07-03T15:43:58.560 INFO:tasks.workunit.client.1.smithi148.stdout:6/848: symlink d3/d50/d77/d100/l10f 0 2024-07-03T15:43:58.560 INFO:tasks.workunit.client.1.smithi148.stdout:6/849: chown d3/da5/fb9 9 1 2024-07-03T15:43:58.562 INFO:tasks.workunit.client.1.smithi148.stdout:0/656: link da/d58/d90/fcf da/d17/d2a/d2f/d93/da3/da7/fdb 0 2024-07-03T15:43:58.564 INFO:tasks.workunit.client.1.smithi148.stdout:6/850: rename d3/d1d/d28/d53/d6d/c7e to d3/d1d/d28/c110 0 2024-07-03T15:43:58.564 INFO:tasks.workunit.client.1.smithi148.stdout:6/851: chown d3/d1d/d28/d3e/ddd/d74/dc4/ca1 1 1 2024-07-03T15:43:58.565 INFO:tasks.workunit.client.1.smithi148.stdout:0/657: unlink da/d58/dbd/d13/d14/d50/d95/d8c/f45 0 2024-07-03T15:43:58.565 INFO:tasks.workunit.client.1.smithi148.stdout:0/658: write da/f73 [1288962,32319] 0 2024-07-03T15:43:58.565 INFO:tasks.workunit.client.1.smithi148.stdout:0/659: write da/d17/d2a/d2f/fc2 [536556,1160] 0 2024-07-03T15:43:58.565 INFO:tasks.workunit.client.1.smithi148.stdout:6/852: truncate d3/d1d/d28/d3e/ddd/fb1 3274441 0 2024-07-03T15:43:58.567 INFO:tasks.workunit.client.1.smithi148.stdout:0/660: unlink da/d17/d2a/d2f/f5f 0 2024-07-03T15:43:58.567 INFO:tasks.workunit.client.1.smithi148.stdout:0/661: chown da/d58/dbd/d13/d1d/da1 345494 1 2024-07-03T15:43:58.568 INFO:tasks.workunit.client.1.smithi148.stdout:3/828: dwrite dd/d33/d37/d68/d2e/d40/d77/fbd [0,4194304] 0 2024-07-03T15:43:58.568 INFO:tasks.workunit.client.1.smithi148.stdout:3/829: stat dd/d35/f3f 0 2024-07-03T15:43:58.569 INFO:tasks.workunit.client.1.smithi148.stdout:0/662: mknod da/d58/dbd/d13/d14/d50/d95/d8c/cdc 0 2024-07-03T15:43:58.569 INFO:tasks.workunit.client.1.smithi148.stdout:0/663: stat da/d17/d2a/f4b 0 2024-07-03T15:43:58.573 INFO:tasks.workunit.client.1.smithi148.stdout:3/830: mknod dd/da6/da2/c10a 0 2024-07-03T15:43:58.573 INFO:tasks.workunit.client.1.smithi148.stdout:3/831: chown dd/d33/d37/d68/d2e/d40/d77/d8f/fdd 50311959 1 2024-07-03T15:43:58.574 INFO:tasks.workunit.client.1.smithi148.stdout:3/832: mknod dd/d33/d37/d68/d67/da7/c10b 0 2024-07-03T15:43:58.576 INFO:tasks.workunit.client.1.smithi148.stdout:3/833: link dd/d33/d37/d68/d2e/ld6 dd/d35/d41/d89/l10c 0 2024-07-03T15:43:58.583 INFO:tasks.workunit.client.1.smithi148.stdout:0/664: dwrite da/d36/f63 [0,4194304] 0 2024-07-03T15:43:58.591 INFO:tasks.workunit.client.1.smithi148.stdout:6/853: write d3/d1d/f24 [397157,24410] 0 2024-07-03T15:43:58.592 INFO:tasks.workunit.client.1.smithi148.stdout:6/854: dread - d3/f56 zero size 2024-07-03T15:43:58.594 INFO:tasks.workunit.client.1.smithi148.stdout:0/665: truncate da/d17/d2a/d2f/fcd 251321 0 2024-07-03T15:43:58.598 INFO:tasks.workunit.client.1.smithi148.stdout:6/855: mkdir d3/d1d/d28/d31/d61/dac/d111 0 2024-07-03T15:43:58.599 INFO:tasks.workunit.client.1.smithi148.stdout:3/834: truncate dd/d33/d37/d68/d2e/f11 233045 0 2024-07-03T15:43:58.601 INFO:tasks.workunit.client.1.smithi148.stdout:0/666: rmdir da/d58/dbd/d4a 39 2024-07-03T15:43:58.601 INFO:tasks.workunit.client.1.smithi148.stdout:0/667: fdatasync da/d58/dbd/d13/d14/fb9 0 2024-07-03T15:43:58.601 INFO:tasks.workunit.client.1.smithi148.stdout:0/668: truncate da/d17/f1e 4827451 0 2024-07-03T15:43:58.601 INFO:tasks.workunit.client.1.smithi148.stdout:0/669: chown da/d58/dbd/d13/d14/d50/f83 16366 1 2024-07-03T15:43:58.602 INFO:tasks.workunit.client.1.smithi148.stdout:3/835: symlink dd/d33/df2/l10d 0 2024-07-03T15:43:58.602 INFO:tasks.workunit.client.1.smithi148.stdout:0/670: write da/d17/d2a/d2f/f37 [330072,52895] 0 2024-07-03T15:43:58.603 INFO:tasks.workunit.client.1.smithi148.stdout:3/836: mkdir dd/d33/d37/d68/d2e/d61/d10e 0 2024-07-03T15:43:58.603 INFO:tasks.workunit.client.1.smithi148.stdout:3/837: fdatasync dd/d33/d37/d68/d2e/d40/d8c/dbc/fd1 0 2024-07-03T15:43:58.603 INFO:tasks.workunit.client.1.smithi148.stdout:3/838: chown dd/d35/d41/d89/fef 12 1 2024-07-03T15:43:58.613 INFO:tasks.workunit.client.1.smithi148.stdout:6/856: dwrite d3/d50/d77/fb5 [0,4194304] 0 2024-07-03T15:43:58.613 INFO:tasks.workunit.client.1.smithi148.stdout:6/857: readlink d3/d1d/d28/d3e/lc6 0 2024-07-03T15:43:58.618 INFO:tasks.workunit.client.1.smithi148.stdout:2/790: sync 2024-07-03T15:43:58.618 INFO:tasks.workunit.client.1.smithi148.stdout:2/791: write d5/d14/d22/d60/f90 [4467467,51549] 0 2024-07-03T15:43:58.621 INFO:tasks.workunit.client.1.smithi148.stdout:2/792: rename d5/d14/d86/db9/dbb/d13/d41/d7b/ddf to d5/d14/d22/dc0/dfb 0 2024-07-03T15:43:58.621 INFO:tasks.workunit.client.1.smithi148.stdout:2/793: write d5/d14/d86/db9/dbb/d13/d41/f49 [1385442,117094] 0 2024-07-03T15:43:58.621 INFO:tasks.workunit.client.1.smithi148.stdout:2/794: chown d5/d14/d86/db9/dbb/d13/d41/f74 390339 1 2024-07-03T15:43:58.621 INFO:tasks.workunit.client.1.smithi148.stdout:2/795: truncate d5/d14/d86/db9/dbb/d13/d41/f63 711330 0 2024-07-03T15:43:58.621 INFO:tasks.workunit.client.1.smithi148.stdout:7/858: sync 2024-07-03T15:43:58.621 INFO:tasks.workunit.client.1.smithi148.stdout:4/728: sync 2024-07-03T15:43:58.621 INFO:tasks.workunit.client.1.smithi148.stdout:8/787: sync 2024-07-03T15:43:58.622 INFO:tasks.workunit.client.1.smithi148.stdout:1/725: sync 2024-07-03T15:43:58.622 INFO:tasks.workunit.client.1.smithi148.stdout:5/832: sync 2024-07-03T15:43:58.622 INFO:tasks.workunit.client.1.smithi148.stdout:5/833: readlink d7/d13/la9 0 2024-07-03T15:43:58.623 INFO:tasks.workunit.client.1.smithi148.stdout:2/796: rmdir d5/d14/d86/db9/dbb/d68/def 39 2024-07-03T15:43:58.623 INFO:tasks.workunit.client.1.smithi148.stdout:5/834: rmdir d7/da/df/d16/d1a 39 2024-07-03T15:43:58.624 INFO:tasks.workunit.client.1.smithi148.stdout:7/859: creat d5/d7d/da7/d1c/d33/d4c/d84/d91/daa/f12b x:0 0 0 2024-07-03T15:43:58.624 INFO:tasks.workunit.client.1.smithi148.stdout:7/860: stat d5/d7d/da7/d1c/d59/f7c 0 2024-07-03T15:43:58.624 INFO:tasks.workunit.client.1.smithi148.stdout:2/797: unlink d5/d14/d86/db9/dbb/d13/d41/f8c 0 2024-07-03T15:43:58.625 INFO:tasks.workunit.client.1.smithi148.stdout:5/835: creat d7/da/df/d16/dac/dc6/f123 x:0 0 0 2024-07-03T15:43:58.626 INFO:tasks.workunit.client.1.smithi148.stdout:7/861: mknod d5/d7d/da7/d1c/d4a/d52/c12c 0 2024-07-03T15:43:58.626 INFO:tasks.workunit.client.1.smithi148.stdout:7/862: fdatasync d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/db2/fb3 0 2024-07-03T15:43:58.626 INFO:tasks.workunit.client.1.smithi148.stdout:2/798: creat d5/d14/d22/dc0/dfb/d79/d9c/ffc x:0 0 0 2024-07-03T15:43:58.627 INFO:tasks.workunit.client.1.smithi148.stdout:5/836: truncate d7/da/df/d16/f1b 1445511 0 2024-07-03T15:43:58.627 INFO:tasks.workunit.client.1.smithi148.stdout:5/837: chown d7/da/f5f 0 1 2024-07-03T15:43:58.627 INFO:tasks.workunit.client.1.smithi148.stdout:5/838: write d7/da/f2b [8918735,66215] 0 2024-07-03T15:43:58.628 INFO:tasks.workunit.client.1.smithi148.stdout:7/863: symlink d5/d7d/da7/d1c/d4a/d66/l12d 0 2024-07-03T15:43:58.629 INFO:tasks.workunit.client.1.smithi148.stdout:2/799: mkdir d5/da8/dfd 0 2024-07-03T15:43:58.630 INFO:tasks.workunit.client.1.smithi148.stdout:2/800: write d5/d14/d86/db9/dbb/d13/d41/d7b/f3e [3440735,104440] 0 2024-07-03T15:43:58.630 INFO:tasks.workunit.client.1.smithi148.stdout:5/839: unlink d7/da/df/d16/d2f/d44/d52/d60/fa6 0 2024-07-03T15:43:58.632 INFO:tasks.workunit.client.1.smithi148.stdout:7/864: symlink d5/d7d/da7/d1c/d33/d4c/d21/d31/dfc/l12e 0 2024-07-03T15:43:58.632 INFO:tasks.workunit.client.1.smithi148.stdout:7/865: fsync d5/f1f 0 2024-07-03T15:43:58.632 INFO:tasks.workunit.client.1.smithi148.stdout:7/866: truncate d5/d7d/da7/d1c/d4a/f62 2229960 0 2024-07-03T15:43:58.632 INFO:tasks.workunit.client.1.smithi148.stdout:7/867: dread - d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/db2/d10d/f110 zero size 2024-07-03T15:43:58.632 INFO:tasks.workunit.client.1.smithi148.stdout:8/788: dread d4/da/d51/f69 [0,4194304] 0 2024-07-03T15:43:58.634 INFO:tasks.workunit.client.1.smithi148.stdout:2/801: mkdir d5/d14/d86/db9/dbb/dfe 0 2024-07-03T15:43:58.635 INFO:tasks.workunit.client.1.smithi148.stdout:1/726: read d0/d2/d6/d14/f6a [2176984,84240] 0 2024-07-03T15:43:58.636 INFO:tasks.workunit.client.1.smithi148.stdout:5/840: link d7/d13/d23/c43 d7/d59/c124 0 2024-07-03T15:43:58.636 INFO:tasks.workunit.client.1.smithi148.stdout:8/789: getdents d4/da/d14/d9e 0 2024-07-03T15:43:58.637 INFO:tasks.workunit.client.1.smithi148.stdout:1/727: rename d0/de/cab to d0/d2/d2e/d39/d42/cf8 0 2024-07-03T15:43:58.639 INFO:tasks.workunit.client.1.smithi148.stdout:5/841: rename d7/da/df/d16/dfb/c115 to d7/da/df/da0/db2/c125 0 2024-07-03T15:43:58.640 INFO:tasks.workunit.client.1.smithi148.stdout:5/842: mkdir d7/da/df/d1d/d109/d126 0 2024-07-03T15:43:58.645 INFO:tasks.workunit.client.1.smithi148.stdout:2/802: dread d5/d14/d22/d60/f65 [0,4194304] 0 2024-07-03T15:43:58.645 INFO:tasks.workunit.client.1.smithi148.stdout:2/803: chown d5/d14/d22/dc0/dfb/d79/d9c/fbe 2 1 2024-07-03T15:43:58.646 INFO:tasks.workunit.client.1.smithi148.stdout:8/790: write d4/da/d14/d1a/d32/d38/dba/fc0 [3175182,113306] 0 2024-07-03T15:43:58.647 INFO:tasks.workunit.client.1.smithi148.stdout:0/671: dwrite da/d58/dbd/d24/f89 [0,4194304] 0 2024-07-03T15:43:58.648 INFO:tasks.workunit.client.1.smithi148.stdout:8/791: getdents d4/da/d14/d1a/d32/d38/d58/d8e/db9 0 2024-07-03T15:43:58.649 INFO:tasks.workunit.client.1.smithi148.stdout:3/839: dwrite dd/d35/dac/fe7 [0,4194304] 0 2024-07-03T15:43:58.649 INFO:tasks.workunit.client.1.smithi148.stdout:6/858: dwrite d3/d1d/d28/d53/f6e [0,4194304] 0 2024-07-03T15:43:58.649 INFO:tasks.workunit.client.1.smithi148.stdout:4/729: dwrite d0/d28/de8/fee [0,4194304] 0 2024-07-03T15:43:58.654 INFO:tasks.workunit.client.1.smithi148.stdout:5/843: dread d7/da/df/d16/f8a [0,4194304] 0 2024-07-03T15:43:58.654 INFO:tasks.workunit.client.1.smithi148.stdout:5/844: chown d7/da/df/d16/de4/lfe 283 1 2024-07-03T15:43:58.654 INFO:tasks.workunit.client.1.smithi148.stdout:8/792: write d4/da/d51/d18/d20/f21 [5640560,63420] 0 2024-07-03T15:43:58.658 INFO:tasks.workunit.client.1.smithi148.stdout:1/728: dwrite d0/d2/d2e/d39/f5f [0,4194304] 0 2024-07-03T15:43:58.662 INFO:tasks.workunit.client.1.smithi148.stdout:4/730: mkdir d0/d8/d3a/dce/d106 0 2024-07-03T15:43:58.662 INFO:tasks.workunit.client.1.smithi148.stdout:4/731: chown d0/d8/d3a/db9/fc9 424749210 1 2024-07-03T15:43:58.663 INFO:tasks.workunit.client.1.smithi148.stdout:3/840: getdents dd/d33/d37/d68/d2e/d40/d8c 0 2024-07-03T15:43:58.666 INFO:tasks.workunit.client.1.smithi148.stdout:5/845: creat d7/da/df/d16/d2f/d44/d52/d60/df2/f127 x:0 0 0 2024-07-03T15:43:58.668 INFO:tasks.workunit.client.1.smithi148.stdout:8/793: unlink d4/da/d14/d3a/l6d 0 2024-07-03T15:43:58.669 INFO:tasks.workunit.client.1.smithi148.stdout:4/732: mkdir d0/d8/d44/dc2/d107 0 2024-07-03T15:43:58.671 INFO:tasks.workunit.client.1.smithi148.stdout:5/846: mkdir d7/da/df/d1d/d128 0 2024-07-03T15:43:58.671 INFO:tasks.workunit.client.1.smithi148.stdout:4/733: read d0/d28/d7d/d6f/f82 [29240,38120] 0 2024-07-03T15:43:58.671 INFO:tasks.workunit.client.1.smithi148.stdout:6/859: dwrite d3/f2f [0,4194304] 0 2024-07-03T15:43:58.690 INFO:tasks.workunit.client.1.smithi148.stdout:6/860: dread d3/d1d/f21 [0,4194304] 0 2024-07-03T15:43:58.690 INFO:tasks.workunit.client.1.smithi148.stdout:6/861: chown d3/d50/d77/d100 150 1 2024-07-03T15:43:58.690 INFO:tasks.workunit.client.1.smithi148.stdout:6/862: write d3/d1d/d28/d31/d4c/f78 [5230555,26812] 0 2024-07-03T15:43:58.692 INFO:tasks.workunit.client.1.smithi148.stdout:6/863: rename d3/d1d/d28/d3e/c9f to d3/d1d/d28/d31/d61/d102/c112 0 2024-07-03T15:43:58.693 INFO:tasks.workunit.client.1.smithi148.stdout:6/864: mkdir d3/da5/d113 0 2024-07-03T15:43:58.693 INFO:tasks.workunit.client.1.smithi148.stdout:6/865: write d3/d1d/d28/d3e/ddd/d74/dc4/d85/fc7 [1373937,40392] 0 2024-07-03T15:43:58.694 INFO:tasks.workunit.client.1.smithi148.stdout:6/866: symlink d3/d1d/d28/d53/l114 0 2024-07-03T15:43:58.694 INFO:tasks.workunit.client.1.smithi148.stdout:6/867: write d3/d1d/d28/d3e/ddd/fb1 [2416745,86131] 0 2024-07-03T15:43:58.694 INFO:tasks.workunit.client.1.smithi148.stdout:1/729: dwrite d0/d2/d11/d1e/d9d/fdb [0,4194304] 0 2024-07-03T15:43:58.695 INFO:tasks.workunit.client.1.smithi148.stdout:8/794: dwrite d4/d86/f97 [0,4194304] 0 2024-07-03T15:43:58.695 INFO:tasks.workunit.client.1.smithi148.stdout:3/841: dwrite dd/d33/d37/d68/d2e/d40/d45/d9a/f9d [0,4194304] 0 2024-07-03T15:43:58.695 INFO:tasks.workunit.client.1.smithi148.stdout:3/842: fdatasync dd/da6/fe2 0 2024-07-03T15:43:58.699 INFO:tasks.workunit.client.1.smithi148.stdout:5/847: dwrite d7/da/df/d16/d2f/d44/d52/d60/f103 [0,4194304] 0 2024-07-03T15:43:58.701 INFO:tasks.workunit.client.1.smithi148.stdout:4/734: dwrite d0/d8/d3a/d9d/fd7 [0,4194304] 0 2024-07-03T15:43:58.701 INFO:tasks.workunit.client.1.smithi148.stdout:5/848: mknod d7/da/dbe/c129 0 2024-07-03T15:43:58.701 INFO:tasks.workunit.client.1.smithi148.stdout:5/849: fsync d7/da/df/d1d/f2e 0 2024-07-03T15:43:58.702 INFO:tasks.workunit.client.1.smithi148.stdout:8/795: getdents d4/da/d14/d3a 0 2024-07-03T15:43:58.705 INFO:tasks.workunit.client.1.smithi148.stdout:5/850: rename d7/da/df/d16/de4/da7/l81 to d7/da/df/d16/dac/l12a 0 2024-07-03T15:43:58.706 INFO:tasks.workunit.client.1.smithi148.stdout:4/735: dread d0/d8/f10 [0,4194304] 0 2024-07-03T15:43:58.706 INFO:tasks.workunit.client.1.smithi148.stdout:4/736: write d0/f4 [775339,25588] 0 2024-07-03T15:43:58.708 INFO:tasks.workunit.client.1.smithi148.stdout:1/730: truncate d0/d2/d11/d1e/d9d/fdb 3636028 0 2024-07-03T15:43:58.709 INFO:tasks.workunit.client.1.smithi148.stdout:6/868: write d3/d1d/d28/f69 [255532,83614] 0 2024-07-03T15:43:58.710 INFO:tasks.workunit.client.1.smithi148.stdout:4/737: rename d0/d8/d3a/f7f to d0/d8/d3a/dce/df7/f108 0 2024-07-03T15:43:58.710 INFO:tasks.workunit.client.1.smithi148.stdout:4/738: write d0/d8/d3a/db8/dd3/ffc [798030,83772] 0 2024-07-03T15:43:58.711 INFO:tasks.workunit.client.1.smithi148.stdout:3/843: dread dd/d33/d37/f53 [0,4194304] 0 2024-07-03T15:43:58.711 INFO:tasks.workunit.client.1.smithi148.stdout:3/844: write f5 [6926315,11937] 0 2024-07-03T15:43:58.713 INFO:tasks.workunit.client.1.smithi148.stdout:6/869: link d3/d1d/d28/d3e/f58 d3/d1d/d28/d31/d4c/f115 0 2024-07-03T15:43:58.715 INFO:tasks.workunit.client.1.smithi148.stdout:5/851: getdents d7/da/dbe 0 2024-07-03T15:43:58.715 INFO:tasks.workunit.client.1.smithi148.stdout:8/796: dread d4/da/d51/d18/d20/f2b [0,4194304] 0 2024-07-03T15:43:58.715 INFO:tasks.workunit.client.1.smithi148.stdout:5/852: truncate d7/da/df/d16/d6a/fa1 1809979 0 2024-07-03T15:43:58.725 INFO:tasks.workunit.client.1.smithi148.stdout:8/797: symlink d4/da/d14/l111 0 2024-07-03T15:43:58.726 INFO:tasks.workunit.client.1.smithi148.stdout:1/731: rename d0/d2/d2e/d39 to d0/de/d2f/d72/df9 0 2024-07-03T15:43:58.727 INFO:tasks.workunit.client.1.smithi148.stdout:1/732: truncate d0/d2/d6/d14/f6a 1468952 0 2024-07-03T15:43:58.727 INFO:tasks.workunit.client.1.smithi148.stdout:1/733: chown d0/de/d2f/d72/df9/fdc 514 1 2024-07-03T15:43:58.729 INFO:tasks.workunit.client.1.smithi148.stdout:4/739: dwrite d0/d28/da7/d2f/d41/f7e [0,4194304] 0 2024-07-03T15:43:58.729 INFO:tasks.workunit.client.1.smithi148.stdout:4/740: chown d0/l2 52798872 1 2024-07-03T15:43:58.730 INFO:tasks.workunit.client.1.smithi148.stdout:4/741: chown d0/lc 1531630151 1 2024-07-03T15:43:58.730 INFO:tasks.workunit.client.1.smithi148.stdout:1/734: unlink d0/d2/d6/d14/f15 0 2024-07-03T15:43:58.730 INFO:tasks.workunit.client.1.smithi148.stdout:1/735: write d0/de/d2f/d72/df9/d42/db7/dbb/fe8 [63184,39770] 0 2024-07-03T15:43:58.732 INFO:tasks.workunit.client.1.smithi148.stdout:4/742: link d0/d28/d6e/dc7/la0 d0/d8/d13/d67/db1/l109 0 2024-07-03T15:43:58.736 INFO:tasks.workunit.client.1.smithi148.stdout:8/798: write d4/da/d51/f7 [8918701,39048] 0 2024-07-03T15:43:58.738 INFO:tasks.workunit.client.1.smithi148.stdout:5/853: dread d7/da/df/f33 [0,4194304] 0 2024-07-03T15:43:58.738 INFO:tasks.workunit.client.1.smithi148.stdout:8/799: mknod d4/da/d51/c112 0 2024-07-03T15:43:58.739 INFO:tasks.workunit.client.1.smithi148.stdout:5/854: readlink d7/da/d21/l80 0 2024-07-03T15:43:58.739 INFO:tasks.workunit.client.1.smithi148.stdout:5/855: stat d7/da/df/d16/de4/cca 0 2024-07-03T15:43:58.740 INFO:tasks.workunit.client.1.smithi148.stdout:6/870: dwrite d3/d1d/d28/d53/f6e [0,4194304] 0 2024-07-03T15:43:58.740 INFO:tasks.workunit.client.1.smithi148.stdout:6/871: write d3/d1d/d28/d3e/f64 [2916526,99046] 0 2024-07-03T15:43:58.740 INFO:tasks.workunit.client.1.smithi148.stdout:6/872: stat d3/d1d/d28/d31/c4d 0 2024-07-03T15:43:58.741 INFO:tasks.workunit.client.1.smithi148.stdout:3/845: dwrite dd/d35/de1/dff/fca [0,4194304] 0 2024-07-03T15:43:58.741 INFO:tasks.workunit.client.1.smithi148.stdout:3/846: dread - dd/da6/fec zero size 2024-07-03T15:43:58.742 INFO:tasks.workunit.client.1.smithi148.stdout:3/847: readlink dd/d33/d37/d68/lf1 0 2024-07-03T15:43:58.742 INFO:tasks.workunit.client.1.smithi148.stdout:5/856: dread d7/d13/d54/feb [0,4194304] 0 2024-07-03T15:43:58.742 INFO:tasks.workunit.client.1.smithi148.stdout:3/848: write dd/d33/d37/d68/d2e/d40/d45/d9a/fab [416986,77621] 0 2024-07-03T15:43:58.743 INFO:tasks.workunit.client.1.smithi148.stdout:1/736: write d0/de/d2f/d72/df9/d42/d62/f6e [3562936,82480] 0 2024-07-03T15:43:58.743 INFO:tasks.workunit.client.1.smithi148.stdout:1/737: getdents d0/d2/d11/d1e/d9d/dc2/de2/d77/da3/dec 0 2024-07-03T15:43:58.746 INFO:tasks.workunit.client.1.smithi148.stdout:3/849: mknod dd/d33/d37/d68/d2e/d40/d45/c10f 0 2024-07-03T15:43:58.750 INFO:tasks.workunit.client.1.smithi148.stdout:1/738: mknod d0/d2/d11/cfa 0 2024-07-03T15:43:58.751 INFO:tasks.workunit.client.1.smithi148.stdout:1/739: chown d0/de/d2f/d72/df9/d42/d62/d7b/fa4 293370 1 2024-07-03T15:43:58.751 INFO:tasks.workunit.client.1.smithi148.stdout:5/857: symlink d7/da/d21/l12b 0 2024-07-03T15:43:58.751 INFO:tasks.workunit.client.1.smithi148.stdout:3/850: mknod dd/da6/db3/c110 0 2024-07-03T15:43:58.751 INFO:tasks.workunit.client.1.smithi148.stdout:3/851: truncate dd/d33/d37/d68/d2e/ff0 44163 0 2024-07-03T15:43:58.751 INFO:tasks.workunit.client.1.smithi148.stdout:8/800: getdents d4/da/d51 0 2024-07-03T15:43:58.751 INFO:tasks.workunit.client.1.smithi148.stdout:8/801: readlink d4/da/d14/d1a/d32/d38/d58/ld0 0 2024-07-03T15:43:58.751 INFO:tasks.workunit.client.1.smithi148.stdout:4/743: dwrite d0/d28/d6e/dc7/fe3 [0,4194304] 0 2024-07-03T15:43:58.752 INFO:tasks.workunit.client.1.smithi148.stdout:6/873: dread f1 [0,4194304] 0 2024-07-03T15:43:58.752 INFO:tasks.workunit.client.1.smithi148.stdout:1/740: rename d0/de/d2f/d72/df9/d42/daf/fc3 to d0/d2/d11/d1e/ffb 0 2024-07-03T15:43:58.752 INFO:tasks.workunit.client.1.smithi148.stdout:5/858: mkdir d7/d13/d12c 0 2024-07-03T15:43:58.752 INFO:tasks.workunit.client.1.smithi148.stdout:5/859: creat d7/da/df/d16/de4/da7/f12d x:0 0 0 2024-07-03T15:43:58.753 INFO:tasks.workunit.client.1.smithi148.stdout:3/852: link dd/d33/d37/d68/d2e/d40/d8c/fd4 dd/da6/db3/f111 0 2024-07-03T15:43:58.754 INFO:tasks.workunit.client.1.smithi148.stdout:5/860: dread d7/d13/d54/fbf [0,4194304] 0 2024-07-03T15:43:58.754 INFO:tasks.workunit.client.1.smithi148.stdout:5/861: write d7/da/df/d16/de4/fad [439563,125068] 0 2024-07-03T15:43:58.754 INFO:tasks.workunit.client.1.smithi148.stdout:5/862: stat d7/da/df/d16/d2f/d44/d52/d60/c95 0 2024-07-03T15:43:58.754 INFO:tasks.workunit.client.1.smithi148.stdout:8/802: link d4/d86/fb3 d4/da/d47/d88/f113 0 2024-07-03T15:43:58.757 INFO:tasks.workunit.client.1.smithi148.stdout:6/874: symlink d3/d1d/d28/d3e/l116 0 2024-07-03T15:43:58.758 INFO:tasks.workunit.client.1.smithi148.stdout:1/741: mkdir d0/d2/dae/dfc 0 2024-07-03T15:43:58.758 INFO:tasks.workunit.client.1.smithi148.stdout:1/742: readlink d0/d2/d11/d1e/d9d/dc0/le9 0 2024-07-03T15:43:58.761 INFO:tasks.workunit.client.1.smithi148.stdout:3/853: dread dd/d33/d37/d68/d2e/f13 [0,4194304] 0 2024-07-03T15:43:58.761 INFO:tasks.workunit.client.1.smithi148.stdout:5/863: symlink d7/da/df/da0/d10f/l12e 0 2024-07-03T15:43:58.763 INFO:tasks.workunit.client.1.smithi148.stdout:1/743: getdents d0/de/d2f/d72/df9/d42 0 2024-07-03T15:43:58.765 INFO:tasks.workunit.client.1.smithi148.stdout:3/854: symlink dd/d33/d37/d68/d2e/ddb/l112 0 2024-07-03T15:43:58.765 INFO:tasks.workunit.client.1.smithi148.stdout:6/875: write d3/d50/fcf [180852,116005] 0 2024-07-03T15:43:58.765 INFO:tasks.workunit.client.1.smithi148.stdout:6/876: write d3/d1d/f4e [2288846,125796] 0 2024-07-03T15:43:58.767 INFO:tasks.workunit.client.1.smithi148.stdout:5/864: creat d7/da/df/d16/dfb/f12f x:0 0 0 2024-07-03T15:43:58.767 INFO:tasks.workunit.client.1.smithi148.stdout:5/865: creat d7/d13/d54/f130 x:0 0 0 2024-07-03T15:43:58.777 INFO:tasks.workunit.client.1.smithi148.stdout:1/744: read d0/de/d2f/d72/f7c [2886792,37510] 0 2024-07-03T15:43:58.777 INFO:tasks.workunit.client.1.smithi148.stdout:4/744: dwrite d0/d28/d7d/d6f/f82 [0,4194304] 0 2024-07-03T15:43:58.777 INFO:tasks.workunit.client.1.smithi148.stdout:1/745: write d0/d2/d11/fb2 [701713,92076] 0 2024-07-03T15:43:58.780 INFO:tasks.workunit.client.1.smithi148.stdout:6/877: fsync d3/d50/fcf 0 2024-07-03T15:43:58.780 INFO:tasks.workunit.client.1.smithi148.stdout:5/866: dread f4 [0,4194304] 0 2024-07-03T15:43:58.782 INFO:tasks.workunit.client.1.smithi148.stdout:1/746: mknod d0/de/d2f/d72/df9/cfd 0 2024-07-03T15:43:58.782 INFO:tasks.workunit.client.1.smithi148.stdout:1/747: chown d0/de/d89/lda 461 1 2024-07-03T15:43:58.782 INFO:tasks.workunit.client.1.smithi148.stdout:4/745: symlink d0/d28/da7/d2f/d41/l10a 0 2024-07-03T15:43:58.783 INFO:tasks.workunit.client.1.smithi148.stdout:8/803: dwrite d4/da/d14/d3a/ff4 [0,4194304] 0 2024-07-03T15:43:58.784 INFO:tasks.workunit.client.1.smithi148.stdout:5/867: dread d7/da/df/d16/d2f/d44/d52/d60/df2/f106 [0,4194304] 0 2024-07-03T15:43:58.784 INFO:tasks.workunit.client.1.smithi148.stdout:5/868: dread - d7/da/df/d16/de4/da7/f12d zero size 2024-07-03T15:43:58.784 INFO:tasks.workunit.client.1.smithi148.stdout:6/878: creat d3/d1d/dd6/d10a/f117 x:0 0 0 2024-07-03T15:43:58.784 INFO:tasks.workunit.client.1.smithi148.stdout:6/879: getdents d3/d1d/d28/d31/d61/dac/d111 0 2024-07-03T15:43:58.785 INFO:tasks.workunit.client.1.smithi148.stdout:6/880: write d3/d88/fc5 [1015292,130032] 0 2024-07-03T15:43:58.786 INFO:tasks.workunit.client.1.smithi148.stdout:1/748: creat d0/d2/dd0/ffe x:0 0 0 2024-07-03T15:43:58.788 INFO:tasks.workunit.client.1.smithi148.stdout:3/855: dwrite dd/f44 [0,4194304] 0 2024-07-03T15:43:58.792 INFO:tasks.workunit.client.1.smithi148.stdout:8/804: rename d4/da/d14/d1a/d32/d38/dba/lbc to d4/da/d14/d1a/l114 0 2024-07-03T15:43:58.794 INFO:tasks.workunit.client.1.smithi148.stdout:5/869: link d7/da/df/d16/d2f/d44/f69 d7/d13/d54/f131 0 2024-07-03T15:43:58.794 INFO:tasks.workunit.client.1.smithi148.stdout:5/870: readlink d7/da/df/d16/d2f/dc2/de9/l98 0 2024-07-03T15:43:58.794 INFO:tasks.workunit.client.1.smithi148.stdout:1/749: symlink d0/d2/d6/d14/lff 0 2024-07-03T15:43:58.794 INFO:tasks.workunit.client.1.smithi148.stdout:1/750: fsync d0/d2/d11/d3a/f70 0 2024-07-03T15:43:58.796 INFO:tasks.workunit.client.1.smithi148.stdout:4/746: rmdir d0/d91/da8/dab 39 2024-07-03T15:43:58.797 INFO:tasks.workunit.client.1.smithi148.stdout:3/856: symlink dd/d33/d37/d68/d2e/d40/d77/dcd/l113 0 2024-07-03T15:43:58.798 INFO:tasks.workunit.client.1.smithi148.stdout:3/857: creat dd/d35/f114 x:0 0 0 2024-07-03T15:43:58.798 INFO:tasks.workunit.client.1.smithi148.stdout:3/858: fdatasync dd/d35/d41/d89/db0/fcb 0 2024-07-03T15:43:58.806 INFO:tasks.workunit.client.1.smithi148.stdout:3/859: dread dd/da6/f85 [0,4194304] 0 2024-07-03T15:43:58.808 INFO:tasks.workunit.client.1.smithi148.stdout:5/871: write d7/da/df/d16/d6a/db4/f9c [2360794,129745] 0 2024-07-03T15:43:58.808 INFO:tasks.workunit.client.1.smithi148.stdout:5/872: write d7/da/df/d16/d6a/db4/f9c [1145816,53649] 0 2024-07-03T15:43:58.809 INFO:tasks.workunit.client.1.smithi148.stdout:5/873: getdents d7/d13/d54/d119 0 2024-07-03T15:43:58.809 INFO:tasks.workunit.client.1.smithi148.stdout:5/874: chown d7/da/df 738 1 2024-07-03T15:43:58.809 INFO:tasks.workunit.client.1.smithi148.stdout:5/875: write d7/da/df/d16/d2f/d44/d52/d60/f92 [218863,52726] 0 2024-07-03T15:43:58.811 INFO:tasks.workunit.client.1.smithi148.stdout:8/805: chown d4/da/d14/d1a/d32/d38/d58/l84 19 1 2024-07-03T15:43:58.811 INFO:tasks.workunit.client.1.smithi148.stdout:8/806: write d4/da/d51/d18/d20/d82/fc1 [1031123,100958] 0 2024-07-03T15:43:58.811 INFO:tasks.workunit.client.1.smithi148.stdout:6/881: dwrite d3/d1d/d28/d3e/ddd/d74/dc4/d85/fb6 [0,4194304] 0 2024-07-03T15:43:58.811 INFO:tasks.workunit.client.1.smithi148.stdout:4/747: dread d0/d8/d3a/f47 [0,4194304] 0 2024-07-03T15:43:58.815 INFO:tasks.workunit.client.1.smithi148.stdout:4/748: creat d0/d28/d6e/f10b x:0 0 0 2024-07-03T15:43:58.815 INFO:tasks.workunit.client.1.smithi148.stdout:1/751: dwrite d0/de/d44/f5e [0,4194304] 0 2024-07-03T15:43:58.815 INFO:tasks.workunit.client.1.smithi148.stdout:1/752: fsync d0/de/d2f/d72/df9/d42/db7/dbb/fe8 0 2024-07-03T15:43:58.822 INFO:tasks.workunit.client.1.smithi148.stdout:5/876: rename d7/da/df/d16/d2f/d44/d52 to d7/da/df/da0/db2/d132 0 2024-07-03T15:43:58.824 INFO:tasks.workunit.client.1.smithi148.stdout:4/749: write d0/d28/da7/d2f/f33 [744786,105649] 0 2024-07-03T15:43:58.824 INFO:tasks.workunit.client.1.smithi148.stdout:8/807: write d4/da/d51/fea [4002691,61096] 0 2024-07-03T15:43:58.828 INFO:tasks.workunit.client.1.smithi148.stdout:5/877: write d7/da/df/d16/f8a [2542044,35850] 0 2024-07-03T15:43:58.828 INFO:tasks.workunit.client.1.smithi148.stdout:6/882: dwrite d3/d50/d77/fb5 [0,4194304] 0 2024-07-03T15:43:58.828 INFO:tasks.workunit.client.1.smithi148.stdout:6/883: chown d3/d1d/d28/d31/d61/ffe 13 1 2024-07-03T15:43:58.828 INFO:tasks.workunit.client.1.smithi148.stdout:6/884: write d3/d1d/d28/d3e/ddd/fec [2070492,49812] 0 2024-07-03T15:43:58.828 INFO:tasks.workunit.client.1.smithi148.stdout:4/750: fdatasync d0/d91/fa2 0 2024-07-03T15:43:58.828 INFO:tasks.workunit.client.1.smithi148.stdout:4/751: creat d0/d8/d3a/db8/f10c x:0 0 0 2024-07-03T15:43:58.830 INFO:tasks.workunit.client.1.smithi148.stdout:5/878: mknod d7/da/df/d16/dac/dc1/dc4/c133 0 2024-07-03T15:43:58.831 INFO:tasks.workunit.client.1.smithi148.stdout:6/885: creat d3/da5/d113/f118 x:0 0 0 2024-07-03T15:43:58.832 INFO:tasks.workunit.client.1.smithi148.stdout:5/879: truncate d7/da/df/d16/d6a/db4/fd9 3134278 0 2024-07-03T15:43:58.832 INFO:tasks.workunit.client.1.smithi148.stdout:6/886: mkdir d3/d1d/d28/d3e/ddd/d74/dc4/d85/da2/d119 0 2024-07-03T15:43:58.833 INFO:tasks.workunit.client.1.smithi148.stdout:5/880: rmdir d7/da/df/d16 39 2024-07-03T15:43:58.836 INFO:tasks.workunit.client.1.smithi148.stdout:5/881: symlink d7/da/df/d16/dac/l134 0 2024-07-03T15:43:58.836 INFO:tasks.workunit.client.1.smithi148.stdout:5/882: write d7/da/df/d16/f8a [4852796,1166] 0 2024-07-03T15:43:58.837 INFO:tasks.workunit.client.1.smithi148.stdout:8/808: dread d4/da/d14/d9e/f7d [0,4194304] 0 2024-07-03T15:43:58.837 INFO:tasks.workunit.client.1.smithi148.stdout:8/809: write d4/d86/ff1 [537600,52188] 0 2024-07-03T15:43:58.843 INFO:tasks.workunit.client.1.smithi148.stdout:6/887: dread d3/d1d/d28/d31/d4c/f115 [0,4194304] 0 2024-07-03T15:43:58.845 INFO:tasks.workunit.client.1.smithi148.stdout:6/888: rename d3/fb0 to d3/d50/f11a 0 2024-07-03T15:43:58.861 INFO:tasks.workunit.client.1.smithi148.stdout:8/810: dwrite d4/d86/f5c [0,4194304] 0 2024-07-03T15:43:58.861 INFO:tasks.workunit.client.1.smithi148.stdout:5/883: dwrite d7/da/f2b [4194304,4194304] 0 2024-07-03T15:43:58.862 INFO:tasks.workunit.client.1.smithi148.stdout:5/884: readlink d7/d13/d23/l49 0 2024-07-03T15:43:58.862 INFO:tasks.workunit.client.1.smithi148.stdout:5/885: readlink d7/da/l11 0 2024-07-03T15:43:58.862 INFO:tasks.workunit.client.1.smithi148.stdout:5/886: readlink l0 0 2024-07-03T15:43:58.862 INFO:tasks.workunit.client.1.smithi148.stdout:8/811: creat d4/da/d51/f115 x:0 0 0 2024-07-03T15:43:58.863 INFO:tasks.workunit.client.1.smithi148.stdout:8/812: mknod d4/da/d14/d3a/dab/c116 0 2024-07-03T15:43:58.866 INFO:tasks.workunit.client.1.smithi148.stdout:5/887: dread d7/da/df/d16/d1a/de5/f100 [0,4194304] 0 2024-07-03T15:43:58.866 INFO:tasks.workunit.client.1.smithi148.stdout:5/888: dread - d7/da/df/da0/db2/d132/d60/df2/f127 zero size 2024-07-03T15:43:58.866 INFO:tasks.workunit.client.1.smithi148.stdout:5/889: chown d7/d13/l120 98651 1 2024-07-03T15:43:58.866 INFO:tasks.workunit.client.1.smithi148.stdout:5/890: write d7/da/df/d16/dac/fd0 [281042,45055] 0 2024-07-03T15:43:58.867 INFO:tasks.workunit.client.1.smithi148.stdout:5/891: rename d7/f19 to d7/d13/d54/d119/f135 0 2024-07-03T15:43:58.878 INFO:tasks.workunit.client.1.smithi148.stdout:8/813: dwrite d4/da/d14/d1a/d32/d89/fa1 [0,4194304] 0 2024-07-03T15:43:58.878 INFO:tasks.workunit.client.1.smithi148.stdout:8/814: rename d4 to d4/da/d51/d53/d117 22 2024-07-03T15:43:58.897 INFO:tasks.workunit.client.1.smithi148.stdout:8/815: dwrite d4/da/d14/d1a/d32/d38/fd1 [4194304,4194304] 0 2024-07-03T15:43:58.994 INFO:tasks.workunit.client.1.smithi148.stdout:1/753: dwrite d0/d2/d11/d1e/d9d/fdb [0,4194304] 0 2024-07-03T15:43:58.996 INFO:tasks.workunit.client.1.smithi148.stdout:6/889: rmdir d3/da5 39 2024-07-03T15:43:58.997 INFO:tasks.workunit.client.1.smithi148.stdout:6/890: mknod d3/d50/d77/c11b 0 2024-07-03T15:43:58.998 INFO:tasks.workunit.client.1.smithi148.stdout:1/754: dread d0/d2/d11/d1e/d9d/dc2/de2/f7a [0,4194304] 0 2024-07-03T15:43:58.998 INFO:tasks.workunit.client.1.smithi148.stdout:1/755: dread - d0/de/d44/d54/fe5 zero size 2024-07-03T15:43:58.999 INFO:tasks.workunit.client.1.smithi148.stdout:6/891: creat d3/d1d/d28/d31/d61/dac/f11c x:0 0 0 2024-07-03T15:43:59.003 INFO:tasks.workunit.client.1.smithi148.stdout:4/752: dwrite d0/d28/f69 [0,4194304] 0 2024-07-03T15:43:59.003 INFO:tasks.workunit.client.1.smithi148.stdout:8/816: dwrite d4/da/d14/f68 [0,4194304] 0 2024-07-03T15:43:59.004 INFO:tasks.workunit.client.1.smithi148.stdout:5/892: dwrite d7/da/df/da0/db2/d132/f82 [0,4194304] 0 2024-07-03T15:43:59.008 INFO:tasks.workunit.client.1.smithi148.stdout:5/893: symlink d7/da/dbe/l136 0 2024-07-03T15:43:59.011 INFO:tasks.workunit.client.1.smithi148.stdout:5/894: creat d7/da/dbe/de6/f137 x:0 0 0 2024-07-03T15:43:59.013 INFO:tasks.workunit.client.1.smithi148.stdout:5/895: getdents d7/da/df/d16/d2f/dc2 0 2024-07-03T15:43:59.029 INFO:tasks.workunit.client.1.smithi148.stdout:4/753: dwrite d0/d8/d3a/db9/fcf [0,4194304] 0 2024-07-03T15:43:59.029 INFO:tasks.workunit.client.1.smithi148.stdout:5/896: dwrite d7/d13/d54/f4b [0,4194304] 0 2024-07-03T15:43:59.030 INFO:tasks.workunit.client.1.smithi148.stdout:5/897: chown d7/ce 183260 1 2024-07-03T15:43:59.030 INFO:tasks.workunit.client.1.smithi148.stdout:4/754: symlink d0/d8/d3a/db9/l10d 0 2024-07-03T15:43:59.030 INFO:tasks.workunit.client.1.smithi148.stdout:4/755: dread - d0/d8/d13/d48/f81 zero size 2024-07-03T15:43:59.042 INFO:tasks.workunit.client.1.smithi148.stdout:5/898: dread d7/da/df/d16/d1a/f34 [0,4194304] 0 2024-07-03T15:43:59.042 INFO:tasks.workunit.client.1.smithi148.stdout:8/817: dread d4/da/d14/d17/deb/ffa [0,4194304] 0 2024-07-03T15:43:59.043 INFO:tasks.workunit.client.1.smithi148.stdout:8/818: mknod d4/da/d51/c118 0 2024-07-03T15:43:59.043 INFO:tasks.workunit.client.1.smithi148.stdout:8/819: getdents d4/da/d14/d3a 0 2024-07-03T15:43:59.043 INFO:tasks.workunit.client.1.smithi148.stdout:8/820: dread - d4/da/d14/d17/f10d zero size 2024-07-03T15:43:59.048 INFO:tasks.workunit.client.1.smithi148.stdout:4/756: dwrite d0/d8/d13/d48/f3f [4194304,4194304] 0 2024-07-03T15:43:59.050 INFO:tasks.workunit.client.1.smithi148.stdout:4/757: symlink d0/d28/d6e/dc7/l10e 0 2024-07-03T15:43:59.051 INFO:tasks.workunit.client.1.smithi148.stdout:4/758: readlink d0/d91/da8/dab/l55 0 2024-07-03T15:43:59.051 INFO:tasks.workunit.client.1.smithi148.stdout:4/759: read d0/d28/da7/d2f/f33 [3724536,128524] 0 2024-07-03T15:43:59.051 INFO:tasks.workunit.client.1.smithi148.stdout:4/760: readlink d0/d28/d7d/d6f/d7c/l8e 0 2024-07-03T15:43:59.051 INFO:tasks.workunit.client.1.smithi148.stdout:6/892: dwrite d3/f17 [0,4194304] 0 2024-07-03T15:43:59.051 INFO:tasks.workunit.client.1.smithi148.stdout:5/899: dread d7/f4f [4194304,4194304] 0 2024-07-03T15:43:59.052 INFO:tasks.workunit.client.1.smithi148.stdout:6/893: mknod d3/d1d/d28/d31/d61/c11d 0 2024-07-03T15:43:59.055 INFO:tasks.workunit.client.1.smithi148.stdout:5/900: rmdir d7/da/df/d16/d2f 39 2024-07-03T15:43:59.055 INFO:tasks.workunit.client.1.smithi148.stdout:5/901: chown d7/da/df/d16/dac/fd0 23685 1 2024-07-03T15:43:59.055 INFO:tasks.workunit.client.1.smithi148.stdout:5/902: fsync d7/d13/d54/f131 0 2024-07-03T15:43:59.058 INFO:tasks.workunit.client.1.smithi148.stdout:5/903: read d7/da/fb [897060,84367] 0 2024-07-03T15:43:59.058 INFO:tasks.workunit.client.1.smithi148.stdout:5/904: readlink d7/da/df/d1d/l108 0 2024-07-03T15:43:59.060 INFO:tasks.workunit.client.1.smithi148.stdout:5/905: creat d7/da/df/d16/d1a/de5/f138 x:0 0 0 2024-07-03T15:43:59.060 INFO:tasks.workunit.client.1.smithi148.stdout:5/906: write d7/da/df/d16/d2f/d44/f7c [509877,60410] 0 2024-07-03T15:43:59.061 INFO:tasks.workunit.client.1.smithi148.stdout:5/907: dread d7/da/df/d16/dac/fd0 [0,4194304] 0 2024-07-03T15:43:59.063 INFO:tasks.workunit.client.1.smithi148.stdout:5/908: creat d7/da/df/da0/d10f/f139 x:0 0 0 2024-07-03T15:43:59.065 INFO:tasks.workunit.client.1.smithi148.stdout:5/909: rmdir d7/da/df/da0/db2/d132/d60/d117 0 2024-07-03T15:43:59.066 INFO:tasks.workunit.client.1.smithi148.stdout:6/894: write d3/d1d/d28/d31/d4c/f57 [991042,110125] 0 2024-07-03T15:43:59.067 INFO:tasks.workunit.client.1.smithi148.stdout:6/895: readlink d3/daa/ld1 0 2024-07-03T15:43:59.068 INFO:tasks.workunit.client.1.smithi148.stdout:8/821: dwrite d4/d86/ff1 [0,4194304] 0 2024-07-03T15:43:59.068 INFO:tasks.workunit.client.1.smithi148.stdout:8/822: creat d4/da/f119 x:0 0 0 2024-07-03T15:43:59.068 INFO:tasks.workunit.client.1.smithi148.stdout:8/823: truncate d4/da/d14/f68 4225828 0 2024-07-03T15:43:59.070 INFO:tasks.workunit.client.1.smithi148.stdout:6/896: creat d3/d1d/d28/d53/dd4/d103/f11e x:0 0 0 2024-07-03T15:43:59.070 INFO:tasks.workunit.client.1.smithi148.stdout:4/761: dwrite d0/d8/d44/f6a [0,4194304] 0 2024-07-03T15:43:59.072 INFO:tasks.workunit.client.1.smithi148.stdout:8/824: creat d4/da/d14/d3a/dab/f11a x:0 0 0 2024-07-03T15:43:59.073 INFO:tasks.workunit.client.1.smithi148.stdout:6/897: creat d3/d1d/d28/d3e/ddd/d74/dc4/d85/d8c/f11f x:0 0 0 2024-07-03T15:43:59.073 INFO:tasks.workunit.client.1.smithi148.stdout:6/898: mknod d3/d50/d77/dff/d109/c120 0 2024-07-03T15:43:59.074 INFO:tasks.workunit.client.1.smithi148.stdout:6/899: creat d3/daa/f121 x:0 0 0 2024-07-03T15:43:59.078 INFO:tasks.workunit.client.1.smithi148.stdout:8/825: read d4/da/d47/d88/d92/f39 [390066,41726] 0 2024-07-03T15:43:59.081 INFO:tasks.workunit.client.1.smithi148.stdout:4/762: dread d0/f5 [4194304,4194304] 0 2024-07-03T15:43:59.081 INFO:tasks.workunit.client.1.smithi148.stdout:5/910: dwrite d7/fe3 [0,4194304] 0 2024-07-03T15:43:59.081 INFO:tasks.workunit.client.1.smithi148.stdout:4/763: write d0/f30 [1162741,60417] 0 2024-07-03T15:43:59.081 INFO:tasks.workunit.client.1.smithi148.stdout:5/911: write d7/d13/d23/f85 [538886,94466] 0 2024-07-03T15:43:59.081 INFO:tasks.workunit.client.1.smithi148.stdout:5/912: creat d7/da/df/d16/dfb/f13a x:0 0 0 2024-07-03T15:43:59.085 INFO:tasks.workunit.client.1.smithi148.stdout:5/913: dread d7/da/df/da0/db2/d132/d60/f92 [0,4194304] 0 2024-07-03T15:43:59.086 INFO:tasks.workunit.client.1.smithi148.stdout:5/914: write d7/d13/d23/f85 [1440027,48141] 0 2024-07-03T15:43:59.086 INFO:tasks.workunit.client.1.smithi148.stdout:5/915: write d7/da/fb5 [848354,12722] 0 2024-07-03T15:43:59.086 INFO:tasks.workunit.client.1.smithi148.stdout:5/916: write d7/da/df/d16/dac/dc1/ff6 [1647319,16121] 0 2024-07-03T15:43:59.088 INFO:tasks.workunit.client.1.smithi148.stdout:4/764: dread d0/d8/d44/f6a [0,4194304] 0 2024-07-03T15:43:59.096 INFO:tasks.workunit.client.1.smithi148.stdout:6/900: dwrite d3/d1d/f4e [0,4194304] 0 2024-07-03T15:43:59.097 INFO:tasks.workunit.client.1.smithi148.stdout:6/901: write d3/d50/f7d [331458,21576] 0 2024-07-03T15:43:59.116 INFO:tasks.workunit.client.1.smithi148.stdout:2/804: sync 2024-07-03T15:43:59.116 INFO:tasks.workunit.client.1.smithi148.stdout:7/868: sync 2024-07-03T15:43:59.117 INFO:tasks.workunit.client.1.smithi148.stdout:0/672: sync 2024-07-03T15:43:59.117 INFO:tasks.workunit.client.1.smithi148.stdout:0/673: write da/d17/f1e [4773773,127290] 0 2024-07-03T15:43:59.126 INFO:tasks.workunit.client.0.smithi028.stdout:7/14: sync 2024-07-03T15:43:59.126 INFO:tasks.workunit.client.0.smithi028.stdout:3/77: sync 2024-07-03T15:43:59.126 INFO:tasks.workunit.client.0.smithi028.stdout:3/78: getdents d18/d1a 0 2024-07-03T15:43:59.127 INFO:tasks.workunit.client.1.smithi148.stdout:8/826: mknod d4/da/d51/d18/d20/c11b 0 2024-07-03T15:43:59.129 INFO:tasks.workunit.client.0.smithi028.stdout:7/15: mknod c3 0 2024-07-03T15:43:59.130 INFO:tasks.workunit.client.1.smithi148.stdout:2/805: dwrite d5/d14/d86/db9/dbb/d13/d41/f49 [0,4194304] 0 2024-07-03T15:43:59.130 INFO:tasks.workunit.client.1.smithi148.stdout:2/806: fdatasync d5/d14/d86/db9/dbb/d68/f84 0 2024-07-03T15:43:59.132 INFO:tasks.workunit.client.1.smithi148.stdout:5/917: mknod d7/da/df/d1d/d109/d126/c13b 0 2024-07-03T15:43:59.133 INFO:tasks.workunit.client.1.smithi148.stdout:4/765: symlink d0/d28/d7d/l10f 0 2024-07-03T15:43:59.135 INFO:tasks.workunit.client.1.smithi148.stdout:6/902: link d3/d1d/d28/d31/c8a d3/d1d/d28/d53/d6d/d8f/dd3/c122 0 2024-07-03T15:43:59.136 INFO:tasks.workunit.client.1.smithi148.stdout:7/869: mknod d5/d7d/da7/d1c/d33/d4c/d84/d91/daa/db9/c12f 0 2024-07-03T15:43:59.137 INFO:tasks.workunit.client.1.smithi148.stdout:0/674: truncate da/d58/dbd/d13/f71 32898 0 2024-07-03T15:43:59.139 INFO:tasks.workunit.client.1.smithi148.stdout:8/827: mknod d4/da/d47/c11c 0 2024-07-03T15:43:59.139 INFO:tasks.workunit.client.1.smithi148.stdout:8/828: fdatasync d4/da/d51/d18/d20/d82/fa7 0 2024-07-03T15:43:59.141 INFO:tasks.workunit.client.1.smithi148.stdout:5/918: symlink d7/d13/l13c 0 2024-07-03T15:43:59.141 INFO:tasks.workunit.client.1.smithi148.stdout:5/919: readlink d7/da/df/d16/d6a/db4/lbc 0 2024-07-03T15:43:59.141 INFO:tasks.workunit.client.1.smithi148.stdout:8/829: dread d4/da/d51/d53/f7f [0,4194304] 0 2024-07-03T15:43:59.142 INFO:tasks.workunit.client.1.smithi148.stdout:4/766: mkdir d0/d28/da7/d2f/d110 0 2024-07-03T15:43:59.142 INFO:tasks.workunit.client.1.smithi148.stdout:4/767: read - d0/d28/da7/d2f/d41/ff5 zero size 2024-07-03T15:43:59.142 INFO:tasks.workunit.client.1.smithi148.stdout:6/903: symlink d3/d1d/d28/d31/d83/l123 0 2024-07-03T15:43:59.145 INFO:tasks.workunit.client.1.smithi148.stdout:4/768: rmdir d0/d8/dff 0 2024-07-03T15:43:59.149 INFO:tasks.workunit.client.1.smithi148.stdout:5/920: dread d7/da/df/d16/f8a [0,4194304] 0 2024-07-03T15:43:59.151 INFO:tasks.workunit.client.1.smithi148.stdout:4/769: link d0/d28/da7/d2f/f6c d0/d8/d3a/db8/dd3/f111 0 2024-07-03T15:43:59.151 INFO:tasks.workunit.client.1.smithi148.stdout:0/675: rename da/d38 to da/d58/dbd/d94/dac/ddd 0 2024-07-03T15:43:59.151 INFO:tasks.workunit.client.1.smithi148.stdout:0/676: chown da/d17 1603521 1 2024-07-03T15:43:59.151 INFO:tasks.workunit.client.1.smithi148.stdout:8/830: dread d4/da/d51/f69 [0,4194304] 0 2024-07-03T15:43:59.152 INFO:tasks.workunit.client.1.smithi148.stdout:8/831: read d4/da/d47/d88/f27 [455101,115167] 0 2024-07-03T15:43:59.152 INFO:tasks.workunit.client.1.smithi148.stdout:7/870: dread d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/de8/ff9 [0,4194304] 0 2024-07-03T15:43:59.152 INFO:tasks.workunit.client.1.smithi148.stdout:7/871: creat d5/d7d/da7/d1c/d33/f130 x:0 0 0 2024-07-03T15:43:59.152 INFO:tasks.workunit.client.1.smithi148.stdout:7/872: readlink d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/l106 0 2024-07-03T15:43:59.152 INFO:tasks.workunit.client.0.smithi028.stdout:3/79: dwrite f6 [0,4194304] 0 2024-07-03T15:43:59.152 INFO:tasks.workunit.client.1.smithi148.stdout:5/921: link d7/da/df/l9f d7/da/df/da0/db2/l13d 0 2024-07-03T15:43:59.153 INFO:tasks.workunit.client.1.smithi148.stdout:6/904: rename d3/d1d/d28/d3e/ddd/f99 to d3/d1d/d28/d53/f124 0 2024-07-03T15:43:59.153 INFO:tasks.workunit.client.1.smithi148.stdout:6/905: readlink d3/d1d/d28/d3e/l116 0 2024-07-03T15:43:59.154 INFO:tasks.workunit.client.0.smithi028.stdout:3/80: mknod d18/d1a/c1c 0 2024-07-03T15:43:59.158 INFO:tasks.workunit.client.0.smithi028.stdout:7/16: dwrite f2 [0,4194304] 0 2024-07-03T15:43:59.158 INFO:tasks.workunit.client.0.smithi028.stdout:7/17: creat f4 x:0 0 0 2024-07-03T15:43:59.159 INFO:tasks.workunit.client.0.smithi028.stdout:3/81: creat d18/f1d x:0 0 0 2024-07-03T15:43:59.159 INFO:tasks.workunit.client.1.smithi148.stdout:4/770: truncate d0/d8/f10 2514858 0 2024-07-03T15:43:59.159 INFO:tasks.workunit.client.1.smithi148.stdout:4/771: stat d0/d28/d7d/d6f/d7c/dae/dba 0 2024-07-03T15:43:59.159 INFO:tasks.workunit.client.1.smithi148.stdout:0/677: mkdir da/d58/dbd/d94/dac/ddd/d88/dde 0 2024-07-03T15:43:59.168 INFO:tasks.workunit.client.1.smithi148.stdout:7/873: creat d5/d7d/da7/d1c/d33/d4c/d84/d98/f131 x:0 0 0 2024-07-03T15:43:59.169 INFO:tasks.workunit.client.1.smithi148.stdout:0/678: dread da/d58/dbd/d13/f27 [0,4194304] 0 2024-07-03T15:43:59.169 INFO:tasks.workunit.client.1.smithi148.stdout:0/679: read da/d58/dbd/d13/d14/d50/f83 [4097731,52901] 0 2024-07-03T15:43:59.170 INFO:tasks.workunit.client.0.smithi028.stdout:3/82: mkdir d18/d1a/d1e 0 2024-07-03T15:43:59.170 INFO:tasks.workunit.client.0.smithi028.stdout:3/83: write f14 [47375,2570] 0 2024-07-03T15:43:59.172 INFO:tasks.workunit.client.1.smithi148.stdout:6/906: mkdir d3/d50/d125 0 2024-07-03T15:43:59.172 INFO:tasks.workunit.client.1.smithi148.stdout:6/907: fdatasync d3/fa9 0 2024-07-03T15:43:59.172 INFO:tasks.workunit.client.0.smithi028.stdout:3/84: dread f9 [0,4194304] 0 2024-07-03T15:43:59.174 INFO:tasks.workunit.client.1.smithi148.stdout:4/772: mknod d0/d8/d13/d67/db1/c112 0 2024-07-03T15:43:59.175 INFO:tasks.workunit.client.1.smithi148.stdout:8/832: getdents d4/da/d14/d1a/d32/d38/d58/dcb 0 2024-07-03T15:43:59.178 INFO:tasks.workunit.client.0.smithi028.stdout:3/85: mknod d18/c1f 0 2024-07-03T15:43:59.184 INFO:tasks.workunit.client.0.smithi028.stdout:3/86: creat d18/d1a/f20 x:0 0 0 2024-07-03T15:43:59.184 INFO:tasks.workunit.client.0.smithi028.stdout:3/87: write f14 [670459,93094] 0 2024-07-03T15:43:59.185 INFO:tasks.workunit.client.1.smithi148.stdout:6/908: dread d3/d1d/d28/d53/f6e [0,4194304] 0 2024-07-03T15:43:59.187 INFO:tasks.workunit.client.1.smithi148.stdout:0/680: dwrite da/d58/dbd/d94/dac/ddd/f7c [0,4194304] 0 2024-07-03T15:43:59.187 INFO:tasks.workunit.client.0.smithi028.stdout:7/18: dwrite f0 [4194304,4194304] 0 2024-07-03T15:43:59.188 INFO:tasks.workunit.client.0.smithi028.stdout:7/19: creat f5 x:0 0 0 2024-07-03T15:43:59.188 INFO:tasks.workunit.client.0.smithi028.stdout:7/20: rmdir - no directory 2024-07-03T15:43:59.191 INFO:tasks.workunit.client.1.smithi148.stdout:7/874: dwrite d5/d7d/da7/d1c/d4a/d66/dd7/fcd [0,4194304] 0 2024-07-03T15:43:59.191 INFO:tasks.workunit.client.1.smithi148.stdout:6/909: dread d3/d1d/d28/d31/d61/f6b [0,4194304] 0 2024-07-03T15:43:59.194 INFO:tasks.workunit.client.1.smithi148.stdout:5/922: rename d7/da/df/d16/d1a to d7/da/df/d16/dfb/d13e 0 2024-07-03T15:43:59.194 INFO:tasks.workunit.client.1.smithi148.stdout:5/923: creat d7/da/df/d16/d6a/db4/f13f x:0 0 0 2024-07-03T15:43:59.196 INFO:tasks.workunit.client.1.smithi148.stdout:4/773: creat d0/d28/d7d/d6f/d7c/dae/dba/dde/f113 x:0 0 0 2024-07-03T15:43:59.196 INFO:tasks.workunit.client.1.smithi148.stdout:4/774: fdatasync d0/d91/da8/dab/d49/fb2 0 2024-07-03T15:43:59.200 INFO:tasks.workunit.client.1.smithi148.stdout:0/681: symlink da/d58/dbd/d94/ldf 0 2024-07-03T15:43:59.202 INFO:tasks.workunit.client.1.smithi148.stdout:7/875: link d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/ldc d5/d7d/da7/l132 0 2024-07-03T15:43:59.203 INFO:tasks.workunit.client.1.smithi148.stdout:6/910: mknod d3/d50/d125/c126 0 2024-07-03T15:43:59.203 INFO:tasks.workunit.client.1.smithi148.stdout:6/911: truncate d3/d1d/d28/d31/d4c/f5d 317786 0 2024-07-03T15:43:59.203 INFO:tasks.workunit.client.1.smithi148.stdout:4/775: dread d0/d28/da7/d2f/f6c [0,4194304] 0 2024-07-03T15:43:59.204 INFO:tasks.workunit.client.1.smithi148.stdout:4/776: fdatasync d0/d8/d3a/db8/f10c 0 2024-07-03T15:43:59.204 INFO:tasks.workunit.client.1.smithi148.stdout:4/777: dread - d0/d28/d6e/f97 zero size 2024-07-03T15:43:59.204 INFO:tasks.workunit.client.0.smithi028.stdout:3/88: truncate f6 1628070 0 2024-07-03T15:43:59.205 INFO:tasks.workunit.client.1.smithi148.stdout:5/924: symlink d7/da/df/da0/db2/d132/d60/l140 0 2024-07-03T15:43:59.205 INFO:tasks.workunit.client.1.smithi148.stdout:6/912: dread d3/d1d/d28/d31/d4c/f78 [4194304,4194304] 0 2024-07-03T15:43:59.205 INFO:tasks.workunit.client.1.smithi148.stdout:6/913: chown d3/d1d/d28/d3e/ddd/d74/dc4/d85/ld8 1455334 1 2024-07-03T15:43:59.207 INFO:tasks.workunit.client.1.smithi148.stdout:8/833: link d4/da/d14/d1a/d32/d38/d58/fa6 d4/da/d14/d1a/d32/d38/d58/f11d 0 2024-07-03T15:43:59.207 INFO:tasks.workunit.client.1.smithi148.stdout:8/834: chown d4/da/d47/d88/c3b 55141 1 2024-07-03T15:43:59.208 INFO:tasks.workunit.client.0.smithi028.stdout:7/21: getdents . 0 2024-07-03T15:43:59.209 INFO:tasks.workunit.client.1.smithi148.stdout:7/876: unlink d5/d7d/da7/d1c/d33/d4c/d21/d47/d8b/dd3/ffa 0 2024-07-03T15:43:59.210 INFO:tasks.workunit.client.1.smithi148.stdout:3/860: sync 2024-07-03T15:43:59.212 INFO:tasks.workunit.client.1.smithi148.stdout:4/778: symlink d0/d8/d44/dc2/d107/l114 0 2024-07-03T15:43:59.214 INFO:tasks.workunit.client.1.smithi148.stdout:5/925: mknod d7/da/df/d16/d6a/db4/c141 0 2024-07-03T15:43:59.215 INFO:tasks.workunit.client.0.smithi028.stdout:7/22: rename f5 to f6 0 2024-07-03T15:43:59.217 INFO:tasks.workunit.client.1.smithi148.stdout:6/914: rename d3/d1d/ffd to d3/d1d/d28/d53/d6d/d8f/f127 0 2024-07-03T15:43:59.217 INFO:tasks.workunit.client.1.smithi148.stdout:6/915: truncate d3/d1d/d28/d3e/ddd/d74/dc4/d85/d8c/fd0 836018 0 2024-07-03T15:43:59.219 INFO:tasks.workunit.client.0.smithi028.stdout:7/23: fsync f0 0 2024-07-03T15:43:59.219 INFO:tasks.workunit.client.0.smithi028.stdout:7/24: stat f6 0 2024-07-03T15:43:59.220 INFO:tasks.workunit.client.1.smithi148.stdout:0/682: mknod da/d17/d2a/d2f/d93/da3/ce0 0 2024-07-03T15:43:59.221 INFO:tasks.workunit.client.1.smithi148.stdout:7/877: unlink d5/d7d/da7/d1c/d33/d4c/d21/f3c 0 2024-07-03T15:43:59.222 INFO:tasks.workunit.client.0.smithi028.stdout:7/25: chown f2 25 1 2024-07-03T15:43:59.223 INFO:tasks.workunit.client.1.smithi148.stdout:3/861: truncate dd/d33/d37/d68/d2e/f13 922435 0 2024-07-03T15:43:59.224 INFO:tasks.workunit.client.1.smithi148.stdout:3/862: dread - dd/da6/da2/ff5 zero size 2024-07-03T15:43:59.225 INFO:tasks.workunit.client.0.smithi028.stdout:3/89: dwrite f2 [0,4194304] 0 2024-07-03T15:43:59.227 INFO:tasks.workunit.client.0.smithi028.stdout:3/90: write f4 [4701486,121823] 0 2024-07-03T15:43:59.227 INFO:tasks.workunit.client.0.smithi028.stdout:3/91: truncate d18/d1a/f20 982885 0 2024-07-03T15:43:59.228 INFO:tasks.workunit.client.1.smithi148.stdout:6/916: mknod d3/d1d/dd6/d10a/c128 0 2024-07-03T15:43:59.230 INFO:tasks.workunit.client.1.smithi148.stdout:8/835: symlink d4/da/d51/d18/l11e 0 2024-07-03T15:43:59.233 INFO:tasks.workunit.client.0.smithi028.stdout:3/92: link l12 d18/d1a/l21 0 2024-07-03T15:43:59.234 INFO:tasks.workunit.client.0.smithi028.stdout:3/93: symlink d18/l22 0 2024-07-03T15:43:59.235 INFO:tasks.workunit.client.0.smithi028.stdout:3/94: truncate f6 1504993 0 2024-07-03T15:43:59.235 INFO:tasks.workunit.client.0.smithi028.stdout:3/95: fdatasync ff 0 2024-07-03T15:43:59.235 INFO:tasks.workunit.client.1.smithi148.stdout:5/926: rename d7/da/df/d16/d2f/d44/cfc to d7/da/d21/c142 0 2024-07-03T15:43:59.238 INFO:tasks.workunit.client.1.smithi148.stdout:0/683: rename da/d58/dbd/d94/dac/ddd/d88/dde to da/d58/dbd/d13/d1d/da1/de1 0 2024-07-03T15:43:59.239 INFO:tasks.workunit.client.0.smithi028.stdout:3/96: truncate ff 1232335 0 2024-07-03T15:43:59.244 INFO:tasks.workunit.client.1.smithi148.stdout:6/917: rename d3/d50/c9d to d3/d88/df8/c129 0 2024-07-03T15:43:59.246 INFO:tasks.workunit.client.1.smithi148.stdout:0/684: unlink da/d17/d2a/d2f/fc2 0 2024-07-03T15:43:59.248 INFO:tasks.workunit.client.1.smithi148.stdout:6/918: symlink d3/d1d/d28/d3e/ddd/d74/dc4/d85/l12a 0 2024-07-03T15:43:59.258 INFO:tasks.workunit.client.0.smithi028.stdout:3/97: dread fe [0,4194304] 0 2024-07-03T15:43:59.259 INFO:tasks.workunit.client.0.smithi028.stdout:3/98: mknod d18/d1a/c23 0 2024-07-03T15:43:59.259 INFO:tasks.workunit.client.0.smithi028.stdout:3/99: stat l16 0 2024-07-03T15:43:59.260 INFO:tasks.workunit.client.1.smithi148.stdout:3/863: dwrite dd/d35/d41/d89/fd3 [0,4194304] 0 2024-07-03T15:43:59.262 INFO:tasks.workunit.client.1.smithi148.stdout:3/864: creat dd/da6/d8b/f115 x:0 0 0 2024-07-03T15:43:59.264 INFO:tasks.workunit.client.1.smithi148.stdout:3/865: link dd/d33/d52/d57/l9b dd/d33/df2/l116 0 2024-07-03T15:43:59.264 INFO:tasks.workunit.client.1.smithi148.stdout:4/779: dwrite d0/d8/d13/f1a [4194304,4194304] 0 2024-07-03T15:43:59.265 INFO:tasks.workunit.client.1.smithi148.stdout:4/780: write d0/d28/d7d/d6f/d7c/da3/fca [53631,6291] 0 2024-07-03T15:43:59.265 INFO:tasks.workunit.client.1.smithi148.stdout:3/866: symlink dd/d33/d37/d68/d2e/d40/d45/l117 0 2024-07-03T15:43:59.265 INFO:tasks.workunit.client.1.smithi148.stdout:3/867: chown dd/d33/d37/c7e 13284 1 2024-07-03T15:43:59.266 INFO:tasks.workunit.client.1.smithi148.stdout:4/781: truncate d0/d8/dd4/fd9 936572 0 2024-07-03T15:43:59.267 INFO:tasks.workunit.client.1.smithi148.stdout:3/868: unlink dd/d33/d52/d57/c7d 0 2024-07-03T15:43:59.268 INFO:tasks.workunit.client.1.smithi148.stdout:4/782: symlink d0/d28/d7d/d6f/d7c/dae/dba/dde/l115 0 2024-07-03T15:43:59.268 INFO:tasks.workunit.client.1.smithi148.stdout:4/783: truncate d0/d8/d3a/f47 4488640 0 2024-07-03T15:43:59.269 INFO:tasks.workunit.client.1.smithi148.stdout:4/784: chown d0/d8/d3a/ce0 4722262 1 2024-07-03T15:43:59.269 INFO:tasks.workunit.client.1.smithi148.stdout:4/785: creat d0/d8/dd4/f116 x:0 0 0 2024-07-03T15:43:59.271 INFO:tasks.workunit.client.1.smithi148.stdout:4/786: write d0/f3d [1253957,8358] 0 2024-07-03T15:43:59.280 INFO:tasks.workunit.client.1.smithi148.stdout:3/869: write dd/d35/d41/d89/fd3 [2567428,71855] 0 2024-07-03T15:43:59.280 INFO:tasks.workunit.client.1.smithi148.stdout:3/870: fsync dd/d33/d37/d68/d2e/d40/d45/d9a/fe3 0 2024-07-03T15:43:59.281 INFO:tasks.workunit.client.1.smithi148.stdout:3/871: mknod dd/d33/d37/d68/d2e/d40/d8c/c118 0 2024-07-03T15:43:59.282 INFO:tasks.workunit.client.1.smithi148.stdout:3/872: mkdir dd/d33/d37/d68/d2e/d40/d8c/dbc/d119 0 2024-07-03T15:43:59.285 INFO:tasks.workunit.client.0.smithi028.stdout:7/26: dwrite f2 [0,4194304] 0 2024-07-03T15:43:59.285 INFO:tasks.workunit.client.0.smithi028.stdout:3/100: dwrite f17 [0,4194304] 0 2024-07-03T15:43:59.285 INFO:tasks.workunit.client.0.smithi028.stdout:3/101: write d18/d1a/f20 [1477970,50718] 0 2024-07-03T15:43:59.293 INFO:tasks.workunit.client.1.smithi148.stdout:8/836: dwrite d4/da/d14/d1a/f37 [0,4194304] 0 2024-07-03T15:43:59.293 INFO:tasks.workunit.client.1.smithi148.stdout:8/837: chown d4/da/d14/d3a/dab/f11a 147339543 1 2024-07-03T15:43:59.293 INFO:tasks.workunit.client.1.smithi148.stdout:5/927: dwrite d7/da/df/d16/d6a/db4/fd9 [0,4194304] 0 2024-07-03T15:43:59.293 INFO:tasks.workunit.client.1.smithi148.stdout:6/919: dwrite d3/d1d/d28/d53/d6d/fc8 [0,4194304] 0 2024-07-03T15:43:59.293 INFO:tasks.workunit.client.1.smithi148.stdout:5/928: write d7/d59/f101 [226370,38660] 0 2024-07-03T15:43:59.294 INFO:tasks.workunit.client.1.smithi148.stdout:4/787: dwrite d0/d8/d3a/d9d/fd7 [0,4194304] 0 2024-07-03T15:43:59.294 INFO:tasks.workunit.client.1.smithi148.stdout:4/788: dread - d0/d8/d13/db4/fcd zero size 2024-07-03T15:43:59.294 INFO:tasks.workunit.client.1.smithi148.stdout:4/789: fdatasync d0/d8/d3a/dce/df7/f108 0 2024-07-03T15:43:59.299 INFO:tasks.workunit.client.0.smithi028.stdout:7/27: dread f2 [0,4194304] 0 2024-07-03T15:43:59.300 INFO:tasks.workunit.client.0.smithi028.stdout:3/102: getdents d18/d1a 0 2024-07-03T15:43:59.301 INFO:tasks.workunit.client.0.smithi028.stdout:7/28: chown f4 3557 1 2024-07-03T15:43:59.309 INFO:tasks.workunit.client.0.smithi028.stdout:7/29: read f2 [1761095,93704] 0 2024-07-03T15:43:59.312 INFO:tasks.workunit.client.0.smithi028.stdout:7/30: getdents . 0 2024-07-03T15:43:59.312 INFO:tasks.workunit.client.0.smithi028.stdout:7/31: write f4 [335201,27943] 0 2024-07-03T15:43:59.313 INFO:tasks.workunit.client.0.smithi028.stdout:7/32: rmdir - no directory 2024-07-03T15:43:59.314 INFO:tasks.workunit.client.1.smithi148.stdout:8/838: link d4/da/d14/f9b d4/da/d14/d3a/dab/f11f 0 2024-07-03T15:43:59.315 INFO:tasks.workunit.client.1.smithi148.stdout:3/873: rmdir dd 39 2024-07-03T15:43:59.315 INFO:tasks.workunit.client.1.smithi148.stdout:8/839: write d4/da/d14/d1a/d32/fae [1386557,105086] 0 2024-07-03T15:43:59.316 INFO:tasks.workunit.client.0.smithi028.stdout:7/33: link f4 f7 0 2024-07-03T15:43:59.316 INFO:tasks.workunit.client.1.smithi148.stdout:6/920: symlink d3/d1d/d28/d3e/df4/l12b 0 2024-07-03T15:43:59.317 INFO:tasks.workunit.client.1.smithi148.stdout:5/929: mknod d7/da/df/d16/de4/c143 0 2024-07-03T15:43:59.317 INFO:tasks.workunit.client.1.smithi148.stdout:5/930: fdatasync d7/da/df/d16/d2f/f8b 0 2024-07-03T15:43:59.319 INFO:tasks.workunit.client.0.smithi028.stdout:5/97: sync 2024-07-03T15:43:59.319 INFO:tasks.workunit.client.0.smithi028.stdout:2/56: sync 2024-07-03T15:43:59.319 INFO:tasks.workunit.client.0.smithi028.stdout:9/101: sync 2024-07-03T15:43:59.319 INFO:tasks.workunit.client.0.smithi028.stdout:6/43: sync 2024-07-03T15:43:59.319 INFO:tasks.workunit.client.0.smithi028.stdout:4/55: sync 2024-07-03T15:43:59.319 INFO:tasks.workunit.client.0.smithi028.stdout:4/56: write f1 [2261029,23089] 0 2024-07-03T15:43:59.320 INFO:tasks.workunit.client.0.smithi028.stdout:4/57: chown f1 21 1 2024-07-03T15:43:59.320 INFO:tasks.workunit.client.0.smithi028.stdout:8/78: sync 2024-07-03T15:43:59.320 INFO:tasks.workunit.client.0.smithi028.stdout:8/79: stat d0/d2 0 2024-07-03T15:43:59.320 INFO:tasks.workunit.client.0.smithi028.stdout:0/42: sync 2024-07-03T15:43:59.320 INFO:tasks.workunit.client.0.smithi028.stdout:1/169: sync 2024-07-03T15:43:59.320 INFO:tasks.workunit.client.0.smithi028.stdout:0/43: dread - d2/fa zero size 2024-07-03T15:43:59.320 INFO:tasks.workunit.client.0.smithi028.stdout:0/44: dread - d2/fa zero size 2024-07-03T15:43:59.321 INFO:tasks.workunit.client.0.smithi028.stdout:7/34: symlink l8 0 2024-07-03T15:43:59.321 INFO:tasks.workunit.client.1.smithi148.stdout:4/790: mkdir d0/d28/d6e/d117 0 2024-07-03T15:43:59.321 INFO:tasks.workunit.client.1.smithi148.stdout:4/791: stat d0/d28/d7d/d6f/d7c/dae/f101 0 2024-07-03T15:43:59.321 INFO:tasks.workunit.client.1.smithi148.stdout:4/792: stat d0/d28/de8 0 2024-07-03T15:43:59.321 INFO:tasks.workunit.client.1.smithi148.stdout:0/685: dwrite da/d58/dbd/d13/d14/d50/f60 [4194304,4194304] 0 2024-07-03T15:43:59.321 INFO:tasks.workunit.client.1.smithi148.stdout:3/874: creat dd/d33/d37/d68/d2e/dc3/f11a x:0 0 0 2024-07-03T15:43:59.322 INFO:tasks.workunit.client.0.smithi028.stdout:3/103: dwrite f17 [0,4194304] 0 2024-07-03T15:43:59.322 INFO:tasks.workunit.client.0.smithi028.stdout:6/44: creat d4/f11 x:0 0 0 2024-07-03T15:43:59.322 INFO:tasks.workunit.client.0.smithi028.stdout:6/45: fsync d4/f10 0 2024-07-03T15:43:59.323 INFO:tasks.workunit.client.1.smithi148.stdout:8/840: dread d4/d86/f63 [0,4194304] 0 2024-07-03T15:43:59.323 INFO:tasks.workunit.client.0.smithi028.stdout:9/102: creat d1/d16/f1c x:0 0 0 2024-07-03T15:43:59.324 INFO:tasks.workunit.client.0.smithi028.stdout:9/103: fsync d1/d10/f18 0 2024-07-03T15:43:59.324 INFO:tasks.workunit.client.0.smithi028.stdout:9/104: chown d1/c1a 1 1 2024-07-03T15:43:59.325 INFO:tasks.workunit.client.1.smithi148.stdout:5/931: unlink d7/da/df/d16/d2f/dc2/de9/cdc 0 2024-07-03T15:43:59.326 INFO:tasks.workunit.client.0.smithi028.stdout:4/58: creat d2/d5/fe x:0 0 0 2024-07-03T15:43:59.326 INFO:tasks.workunit.client.0.smithi028.stdout:3/104: dread d18/d1a/f20 [0,4194304] 0 2024-07-03T15:43:59.327 INFO:tasks.workunit.client.1.smithi148.stdout:4/793: dread d0/d8/d3a/db8/dd3/f111 [0,4194304] 0 2024-07-03T15:43:59.329 INFO:tasks.workunit.client.0.smithi028.stdout:8/80: truncate d0/d2/ff 886120 0 2024-07-03T15:43:59.329 INFO:tasks.workunit.client.0.smithi028.stdout:8/81: stat d0/d2/ff 0 2024-07-03T15:43:59.331 INFO:tasks.workunit.client.0.smithi028.stdout:1/170: mknod d4/de/d1e/c49 0 2024-07-03T15:43:59.333 INFO:tasks.workunit.client.0.smithi028.stdout:5/98: write d0/dd/d13/d19/f21 [317378,77563] 0 2024-07-03T15:43:59.334 INFO:tasks.workunit.client.1.smithi148.stdout:0/686: rename da/d36 to da/d58/dbd/d13/d14/d50/d95/de2 0 2024-07-03T15:43:59.334 INFO:tasks.workunit.client.0.smithi028.stdout:0/45: truncate d2/f9 940418 0 2024-07-03T15:43:59.337 INFO:tasks.workunit.client.0.smithi028.stdout:3/105: write ff [588774,110233] 0 2024-07-03T15:43:59.339 INFO:tasks.workunit.client.1.smithi148.stdout:8/841: getdents d4/da/d47/d88 0 2024-07-03T15:43:59.339 INFO:tasks.workunit.client.1.smithi148.stdout:8/842: dread - d4/da/d14/d17/f10d zero size 2024-07-03T15:43:59.339 INFO:tasks.workunit.client.1.smithi148.stdout:8/843: truncate d4/da/d14/d3a/da8/dd3/ffc 1415828 0 2024-07-03T15:43:59.339 INFO:tasks.workunit.client.0.smithi028.stdout:2/57: symlink da/l13 0 2024-07-03T15:43:59.345 INFO:tasks.workunit.client.0.smithi028.stdout:9/105: mknod d1/c1d 0 2024-07-03T15:43:59.346 INFO:tasks.workunit.client.0.smithi028.stdout:8/82: creat d0/d2/d3/dc/f1f x:0 0 0 2024-07-03T15:43:59.350 INFO:tasks.workunit.client.0.smithi028.stdout:4/59: dwrite d2/f4 [0,4194304] 0 2024-07-03T15:43:59.350 INFO:tasks.workunit.client.0.smithi028.stdout:4/60: chown d2/c3 163310452 1 2024-07-03T15:43:59.354 INFO:tasks.workunit.client.0.smithi028.stdout:0/46: creat d2/ff x:0 0 0 2024-07-03T15:43:59.357 INFO:tasks.workunit.client.1.smithi148.stdout:0/687: dwrite da/d58/dbd/d94/dac/ddd/f4e [0,4194304] 0 2024-07-03T15:43:59.358 INFO:tasks.workunit.client.0.smithi028.stdout:7/35: mkdir d9 0 2024-07-03T15:43:59.362 INFO:tasks.workunit.client.0.smithi028.stdout:3/106: dwrite f2 [0,4194304] 0 2024-07-03T15:43:59.362 INFO:tasks.workunit.client.0.smithi028.stdout:2/58: readlink da/lb 0 2024-07-03T15:43:59.362 INFO:tasks.workunit.client.0.smithi028.stdout:2/59: truncate da/ff 592348 0 2024-07-03T15:43:59.363 INFO:tasks.workunit.client.0.smithi028.stdout:2/60: write da/f12 [233904,1478] 0 2024-07-03T15:43:59.363 INFO:tasks.workunit.client.0.smithi028.stdout:6/46: mkdir d4/d12 0 2024-07-03T15:43:59.363 INFO:tasks.workunit.client.0.smithi028.stdout:6/47: fdatasync f2 0 2024-07-03T15:43:59.363 INFO:tasks.workunit.client.0.smithi028.stdout:6/48: fdatasync d4/f10 0 2024-07-03T15:43:59.364 INFO:tasks.workunit.client.0.smithi028.stdout:9/106: mknod d1/d17/c1e 0 2024-07-03T15:43:59.365 INFO:tasks.workunit.client.0.smithi028.stdout:9/107: dread d1/d10/f18 [0,4194304] 0 2024-07-03T15:43:59.365 INFO:tasks.workunit.client.0.smithi028.stdout:8/83: creat d0/f20 x:0 0 0 2024-07-03T15:43:59.368 INFO:tasks.workunit.client.1.smithi148.stdout:3/875: rename dd/d33/df2/l116 to dd/d33/d37/d68/d2e/l11b 0 2024-07-03T15:43:59.370 INFO:tasks.workunit.client.1.smithi148.stdout:6/921: rmdir d3/d1d/d28/d31/d61/dac/d111 0 2024-07-03T15:43:59.372 INFO:tasks.workunit.client.1.smithi148.stdout:8/844: symlink d4/da/d14/d9e/l120 0 2024-07-03T15:43:59.372 INFO:tasks.workunit.client.1.smithi148.stdout:8/845: fsync d4/da/d51/d18/fc2 0 2024-07-03T15:43:59.372 INFO:tasks.workunit.client.1.smithi148.stdout:8/846: creat d4/da/d14/d1a/d32/d38/d58/d8e/f121 x:0 0 0 2024-07-03T15:43:59.373 INFO:tasks.workunit.client.0.smithi028.stdout:4/61: symlink d2/lf 0 2024-07-03T15:43:59.373 INFO:tasks.workunit.client.0.smithi028.stdout:8/84: write d0/d2/d3/dc/d11/d12/f15 [1072098,39409] 0 2024-07-03T15:43:59.375 INFO:tasks.workunit.client.0.smithi028.stdout:5/99: getdents d0 0 2024-07-03T15:43:59.375 INFO:tasks.workunit.client.0.smithi028.stdout:5/100: readlink d0/lb 0 2024-07-03T15:43:59.375 INFO:tasks.workunit.client.1.smithi148.stdout:5/932: rmdir d7/da/df/d16/dac/dc1 39 2024-07-03T15:43:59.376 INFO:tasks.workunit.client.0.smithi028.stdout:7/36: fdatasync f4 0 2024-07-03T15:43:59.376 INFO:tasks.workunit.client.1.smithi148.stdout:5/933: dread d7/d13/d54/fbf [0,4194304] 0 2024-07-03T15:43:59.380 INFO:tasks.workunit.client.0.smithi028.stdout:3/107: creat d18/d1a/f24 x:0 0 0 2024-07-03T15:43:59.381 INFO:tasks.workunit.client.0.smithi028.stdout:6/49: truncate f0 6106632 0 2024-07-03T15:43:59.382 INFO:tasks.workunit.client.1.smithi148.stdout:2/807: sync 2024-07-03T15:43:59.382 INFO:tasks.workunit.client.1.smithi148.stdout:1/756: sync 2024-07-03T15:43:59.382 INFO:tasks.workunit.client.1.smithi148.stdout:1/757: fsync d0/de/d2f/d72/df9/d42/db7/feb 0 2024-07-03T15:43:59.382 INFO:tasks.workunit.client.1.smithi148.stdout:1/758: dread - d0/de/d2f/d72/df9/d42/d62/d7b/fa4 zero size 2024-07-03T15:43:59.383 INFO:tasks.workunit.client.1.smithi148.stdout:4/794: rename d0/d28/d7d/d6f/c99 to d0/d28/d7d/d6f/d7c/dae/c118 0 2024-07-03T15:43:59.384 INFO:tasks.workunit.client.0.smithi028.stdout:9/108: getdents d1 0 2024-07-03T15:43:59.385 INFO:tasks.workunit.client.0.smithi028.stdout:9/109: dread d1/d10/f18 [0,4194304] 0 2024-07-03T15:43:59.385 INFO:tasks.workunit.client.1.smithi148.stdout:3/876: truncate dd/d33/d52/d57/fae 23010 0 2024-07-03T15:43:59.386 INFO:tasks.workunit.client.1.smithi148.stdout:3/877: chown dd/d33/d37/d68/f32 283 1 2024-07-03T15:43:59.387 INFO:tasks.workunit.client.0.smithi028.stdout:8/85: unlink d0/d2/ff 0 2024-07-03T15:43:59.387 INFO:tasks.workunit.client.0.smithi028.stdout:8/86: chown d0/d2/d3 550 1 2024-07-03T15:43:59.387 INFO:tasks.workunit.client.0.smithi028.stdout:8/87: write d0/d2/d3/dc/d11/d12/f15 [1768132,127237] 0 2024-07-03T15:43:59.387 INFO:tasks.workunit.client.0.smithi028.stdout:8/88: creat d0/d2/d3/dc/f21 x:0 0 0 2024-07-03T15:43:59.388 INFO:tasks.workunit.client.0.smithi028.stdout:8/89: write d0/f20 [783995,96031] 0 2024-07-03T15:43:59.389 INFO:tasks.workunit.client.1.smithi148.stdout:6/922: mknod d3/d50/d77/dff/c12c 0 2024-07-03T15:43:59.391 INFO:tasks.workunit.client.0.smithi028.stdout:6/50: dread f2 [0,4194304] 0 2024-07-03T15:43:59.391 INFO:tasks.workunit.client.0.smithi028.stdout:6/51: dread - d4/f11 zero size 2024-07-03T15:43:59.391 INFO:tasks.workunit.client.0.smithi028.stdout:4/62: dwrite d2/fc [4194304,4194304] 0 2024-07-03T15:43:59.392 INFO:tasks.workunit.client.0.smithi028.stdout:4/63: creat d2/d5/f10 x:0 0 0 2024-07-03T15:43:59.393 INFO:tasks.workunit.client.1.smithi148.stdout:8/847: mknod d4/da/d14/d17/c122 0 2024-07-03T15:43:59.393 INFO:tasks.workunit.client.0.smithi028.stdout:6/52: fsync f0 0 2024-07-03T15:43:59.395 INFO:tasks.workunit.client.0.smithi028.stdout:3/108: symlink d18/l25 0 2024-07-03T15:43:59.399 INFO:tasks.workunit.client.1.smithi148.stdout:2/808: creat d5/d14/d86/d93/dc5/fff x:0 0 0 2024-07-03T15:43:59.401 INFO:tasks.workunit.client.0.smithi028.stdout:2/61: rename da/l13 to da/l14 0 2024-07-03T15:43:59.403 INFO:tasks.workunit.client.1.smithi148.stdout:1/759: getdents d0/de/d2f/d72/df9/d42/daf 0 2024-07-03T15:43:59.404 INFO:tasks.workunit.client.0.smithi028.stdout:9/110: creat d1/dd/f1f x:0 0 0 2024-07-03T15:43:59.405 INFO:tasks.workunit.client.0.smithi028.stdout:9/111: creat d1/dd/f20 x:0 0 0 2024-07-03T15:43:59.406 INFO:tasks.workunit.client.1.smithi148.stdout:5/934: rename d7/da/d21/l68 to d7/da/df/l144 0 2024-07-03T15:43:59.407 INFO:tasks.workunit.client.1.smithi148.stdout:4/795: symlink d0/d28/d7d/l119 0 2024-07-03T15:43:59.408 INFO:tasks.workunit.client.1.smithi148.stdout:4/796: fdatasync d0/d28/d7d/d6f/d7c/dae/f101 0 2024-07-03T15:43:59.408 INFO:tasks.workunit.client.1.smithi148.stdout:3/878: dwrite dd/d33/f88 [0,4194304] 0 2024-07-03T15:43:59.409 INFO:tasks.workunit.client.1.smithi148.stdout:6/923: link d3/d50/fcf d3/d1d/d28/d53/dd4/d103/f12d 0 2024-07-03T15:43:59.410 INFO:tasks.workunit.client.0.smithi028.stdout:5/101: getdents d0/dd/d13/d19 0 2024-07-03T15:43:59.410 INFO:tasks.workunit.client.0.smithi028.stdout:5/102: stat d0/dd/d13/d19/f21 0 2024-07-03T15:43:59.411 INFO:tasks.workunit.client.0.smithi028.stdout:5/103: creat d0/dd/f22 x:0 0 0 2024-07-03T15:43:59.411 INFO:tasks.workunit.client.1.smithi148.stdout:2/809: readlink d5/lc 0 2024-07-03T15:43:59.411 INFO:tasks.workunit.client.1.smithi148.stdout:2/810: write d5/d14/d22/d60/d76/f92 [360759,55946] 0 2024-07-03T15:43:59.411 INFO:tasks.workunit.client.1.smithi148.stdout:2/811: dread - d5/d14/d22/dc0/fea zero size 2024-07-03T15:43:59.415 INFO:tasks.workunit.client.1.smithi148.stdout:5/935: write d7/da/df/fcb [3444468,85219] 0 2024-07-03T15:43:59.416 INFO:tasks.workunit.client.1.smithi148.stdout:8/848: rename d4/da/d14/d1a/d32/d38/d58/d8e/c8f to d4/da/d14/d3a/c123 0 2024-07-03T15:43:59.418 INFO:tasks.workunit.client.1.smithi148.stdout:5/936: dread d7/da/df/d16/f8a [0,4194304] 0 2024-07-03T15:43:59.418 INFO:tasks.workunit.client.1.smithi148.stdout:5/937: truncate d7/da/df/d16/dac/dc1/ff6 2609461 0 2024-07-03T15:43:59.419 INFO:tasks.workunit.client.0.smithi028.stdout:3/109: truncate f4 3750575 0 2024-07-03T15:43:59.420 INFO:tasks.workunit.client.1.smithi148.stdout:3/879: mknod dd/d33/d37/d68/d2e/c11c 0 2024-07-03T15:43:59.421 INFO:tasks.workunit.client.1.smithi148.stdout:3/880: fdatasync dd/d33/d37/d68/d67/fde 0 2024-07-03T15:43:59.424 INFO:tasks.workunit.client.1.smithi148.stdout:6/924: creat d3/d1d/d28/d31/d61/f12e x:0 0 0 2024-07-03T15:43:59.424 INFO:tasks.workunit.client.0.smithi028.stdout:6/53: dwrite d4/f10 [0,4194304] 0 2024-07-03T15:43:59.426 INFO:tasks.workunit.client.0.smithi028.stdout:2/62: unlink l3 0 2024-07-03T15:43:59.428 INFO:tasks.workunit.client.0.smithi028.stdout:7/37: dwrite f0 [4194304,4194304] 0 2024-07-03T15:43:59.429 INFO:tasks.workunit.client.0.smithi028.stdout:7/38: read f4 [203880,86942] 0 2024-07-03T15:43:59.430 INFO:tasks.workunit.client.1.smithi148.stdout:5/938: read d7/da/df/d16/d2f/dc2/fda [1649706,115736] 0 2024-07-03T15:43:59.430 INFO:tasks.workunit.client.1.smithi148.stdout:5/939: readlink d7/da/df/d16/de4/da7/lea 0 2024-07-03T15:43:59.430 INFO:tasks.workunit.client.1.smithi148.stdout:5/940: chown d7/d13/d54/f130 4 1 2024-07-03T15:43:59.433 INFO:tasks.workunit.client.1.smithi148.stdout:2/812: unlink d5/d14/d86/db9/dbb/d13/d1d/da1/fe5 0 2024-07-03T15:43:59.433 INFO:tasks.workunit.client.1.smithi148.stdout:3/881: dread dd/d35/de1/dff/fca [0,4194304] 0 2024-07-03T15:43:59.435 INFO:tasks.workunit.client.1.smithi148.stdout:1/760: getdents d0/d2/d2e 0 2024-07-03T15:43:59.435 INFO:tasks.workunit.client.1.smithi148.stdout:1/761: creat d0/de/d2f/f100 x:0 0 0 2024-07-03T15:43:59.437 INFO:tasks.workunit.client.0.smithi028.stdout:9/112: creat d1/f21 x:0 0 0 2024-07-03T15:43:59.437 INFO:tasks.workunit.client.0.smithi028.stdout:9/113: chown d1/f2 4554902 1 2024-07-03T15:43:59.439 INFO:tasks.workunit.client.1.smithi148.stdout:8/849: dwrite d4/da/d51/d18/d20/d82/f9f [0,4194304] 0 2024-07-03T15:43:59.440 INFO:tasks.workunit.client.1.smithi148.stdout:8/850: readlink d4/da/d14/d17/deb/l10b 0 2024-07-03T15:43:59.440 INFO:tasks.workunit.client.1.smithi148.stdout:6/925: mkdir d3/d1d/d28/d3e/ddd/d74/d12f 0 2024-07-03T15:43:59.442 INFO:tasks.workunit.client.1.smithi148.stdout:5/941: rename d7/da/df/da0/db2/d132/d60/df2/f106 to d7/da/df/d16/f145 0 2024-07-03T15:43:59.442 INFO:tasks.workunit.client.1.smithi148.stdout:5/942: chown d7/da/df/d16/d2f/dc2 846950159 1 2024-07-03T15:43:59.444 INFO:tasks.workunit.client.0.smithi028.stdout:5/104: symlink d0/dd/df/l23 0 2024-07-03T15:43:59.445 INFO:tasks.workunit.client.0.smithi028.stdout:5/105: write d0/dd/d13/d19/f21 [4284499,62548] 0 2024-07-03T15:43:59.446 INFO:tasks.workunit.client.0.smithi028.stdout:3/110: dwrite d18/d1a/f24 [0,4194304] 0 2024-07-03T15:43:59.446 INFO:tasks.workunit.client.0.smithi028.stdout:3/111: chown d18/d1a/f24 3972153 1 2024-07-03T15:43:59.446 INFO:tasks.workunit.client.1.smithi148.stdout:1/762: dread d0/fcb [0,4194304] 0 2024-07-03T15:43:59.447 INFO:tasks.workunit.client.1.smithi148.stdout:1/763: fdatasync d0/d2/f75 0 2024-07-03T15:43:59.447 INFO:tasks.workunit.client.0.smithi028.stdout:6/54: dwrite d4/f11 [0,4194304] 0 2024-07-03T15:43:59.447 INFO:tasks.workunit.client.0.smithi028.stdout:6/55: write f0 [1981258,87611] 0 2024-07-03T15:43:59.448 INFO:tasks.workunit.client.1.smithi148.stdout:8/851: creat d4/da/d14/d1a/d32/d38/d58/dcb/f124 x:0 0 0 2024-07-03T15:43:59.448 INFO:tasks.workunit.client.1.smithi148.stdout:2/813: dread d5/d14/ff3 [0,4194304] 0 2024-07-03T15:43:59.448 INFO:tasks.workunit.client.1.smithi148.stdout:2/814: write d5/d14/d86/db9/dbb/d68/fac [2871532,86592] 0 2024-07-03T15:43:59.448 INFO:tasks.workunit.client.1.smithi148.stdout:2/815: fsync d5/d14/d86/db9/dbb/d13/f15 0 2024-07-03T15:43:59.448 INFO:tasks.workunit.client.1.smithi148.stdout:6/926: creat d3/d1d/d28/d31/f130 x:0 0 0 2024-07-03T15:43:59.450 INFO:tasks.workunit.client.0.smithi028.stdout:2/63: creat da/f15 x:0 0 0 2024-07-03T15:43:59.453 INFO:tasks.workunit.client.0.smithi028.stdout:7/39: creat d9/fa x:0 0 0 2024-07-03T15:43:59.457 INFO:tasks.workunit.client.1.smithi148.stdout:3/882: rename dd/d35/f56 to dd/d35/d41/d89/db0/f11d 0 2024-07-03T15:43:59.458 INFO:tasks.workunit.client.0.smithi028.stdout:3/112: write f2 [57377,32293] 0 2024-07-03T15:43:59.458 INFO:tasks.workunit.client.1.smithi148.stdout:5/943: dwrite d7/da/d21/f5e [0,4194304] 0 2024-07-03T15:43:59.459 INFO:tasks.workunit.client.1.smithi148.stdout:5/944: fdatasync d7/da/df/f3d 0 2024-07-03T15:43:59.459 INFO:tasks.workunit.client.1.smithi148.stdout:1/764: rmdir d0/d2/d11/d1e/d9d/dc2 39 2024-07-03T15:43:59.463 INFO:tasks.workunit.client.1.smithi148.stdout:2/816: mknod d5/da8/c100 0 2024-07-03T15:43:59.463 INFO:tasks.workunit.client.1.smithi148.stdout:2/817: chown d5/d14/d22/d60 2154 1 2024-07-03T15:43:59.466 INFO:tasks.workunit.client.1.smithi148.stdout:6/927: mknod d3/d1d/d28/d3e/ddd/d74/dc4/d85/da2/d119/c131 0 2024-07-03T15:43:59.466 INFO:tasks.workunit.client.1.smithi148.stdout:6/928: creat d3/d1d/d28/d3e/ddd/d74/dc4/f132 x:0 0 0 2024-07-03T15:43:59.466 INFO:tasks.workunit.client.1.smithi148.stdout:5/945: write d7/da/df/d16/d2f/d44/fbd [2010472,2036] 0 2024-07-03T15:43:59.466 INFO:tasks.workunit.client.1.smithi148.stdout:5/946: creat d7/da/df/f146 x:0 0 0 2024-07-03T15:43:59.467 INFO:tasks.workunit.client.1.smithi148.stdout:5/947: fdatasync d7/da/fb5 0 2024-07-03T15:43:59.467 INFO:tasks.workunit.client.1.smithi148.stdout:5/948: dread - d7/da/df/d16/d2f/dc2/d10c/f10e zero size 2024-07-03T15:43:59.467 INFO:tasks.workunit.client.0.smithi028.stdout:8/90: rename d0/d2/d13 to d0/dd/d22 0 2024-07-03T15:43:59.468 INFO:tasks.workunit.client.0.smithi028.stdout:8/91: readlink d0/l8 0 2024-07-03T15:43:59.468 INFO:tasks.workunit.client.0.smithi028.stdout:8/92: creat d0/d2/d3/dc/f23 x:0 0 0 2024-07-03T15:43:59.468 INFO:tasks.workunit.client.1.smithi148.stdout:3/883: mkdir dd/d33/d37/d68/d2e/d40/d77/dcd/d109/d11e 0 2024-07-03T15:43:59.471 INFO:tasks.workunit.client.0.smithi028.stdout:5/106: symlink d0/dd/l24 0 2024-07-03T15:43:59.474 INFO:tasks.workunit.client.1.smithi148.stdout:1/765: symlink d0/d2/d11/d1e/d9d/dc2/de2/d77/da3/l101 0 2024-07-03T15:43:59.475 INFO:tasks.workunit.client.1.smithi148.stdout:1/766: stat d0/d2/d11/cfa 0 2024-07-03T15:43:59.475 INFO:tasks.workunit.client.1.smithi148.stdout:1/767: write d0/fcb [6212738,121424] 0 2024-07-03T15:43:59.475 INFO:tasks.workunit.client.1.smithi148.stdout:1/768: creat d0/d2/d11/d1e/f102 x:0 0 0 2024-07-03T15:43:59.475 INFO:tasks.workunit.client.1.smithi148.stdout:1/769: write d0/de/d2f/d72/fc6 [793468,32361] 0 2024-07-03T15:43:59.479 INFO:tasks.workunit.client.0.smithi028.stdout:5/107: write d0/dd/df/f16 [3347560,90475] 0 2024-07-03T15:43:59.482 INFO:tasks.workunit.client.1.smithi148.stdout:8/852: rmdir d4/da/d14/d3a/dab 39 2024-07-03T15:43:59.484 INFO:tasks.workunit.client.0.smithi028.stdout:2/64: creat da/f16 x:0 0 0 2024-07-03T15:43:59.487 INFO:tasks.workunit.client.0.smithi028.stdout:3/113: symlink d18/d1a/d1e/l26 0 2024-07-03T15:43:59.487 INFO:tasks.workunit.client.0.smithi028.stdout:3/114: chown d18/d1a/l21 7390348 1 2024-07-03T15:43:59.489 INFO:tasks.workunit.client.1.smithi148.stdout:2/818: mkdir d5/d14/d22/dc0/d101 0 2024-07-03T15:43:59.490 INFO:tasks.workunit.client.1.smithi148.stdout:8/853: write d4/d86/fb3 [3409078,49357] 0 2024-07-03T15:43:59.491 INFO:tasks.workunit.client.1.smithi148.stdout:8/854: write d4/da/d51/d18/d20/f66 [332436,125145] 0 2024-07-03T15:43:59.491 INFO:tasks.workunit.client.0.smithi028.stdout:9/114: rename d1/dd/cf to d1/c22 0 2024-07-03T15:43:59.491 INFO:tasks.workunit.client.0.smithi028.stdout:8/93: rename d0/d2/d3/dc/d11 to d0/d2/d3/dc/d11/d24 22 2024-07-03T15:43:59.493 INFO:tasks.workunit.client.0.smithi028.stdout:7/40: dread f2 [0,4194304] 0 2024-07-03T15:43:59.496 INFO:tasks.workunit.client.0.smithi028.stdout:7/41: dread f4 [0,4194304] 0 2024-07-03T15:43:59.498 INFO:tasks.workunit.client.1.smithi148.stdout:6/929: mkdir d3/d50/d133 0 2024-07-03T15:43:59.500 INFO:tasks.workunit.client.0.smithi028.stdout:6/56: mkdir d4/da/d13 0 2024-07-03T15:43:59.500 INFO:tasks.workunit.client.0.smithi028.stdout:6/57: chown d4/da/cd 5 1 2024-07-03T15:43:59.503 INFO:tasks.workunit.client.0.smithi028.stdout:2/65: creat da/f17 x:0 0 0 2024-07-03T15:43:59.503 INFO:tasks.workunit.client.0.smithi028.stdout:5/108: dwrite d0/dd/f1b [0,4194304] 0 2024-07-03T15:43:59.505 INFO:tasks.workunit.client.1.smithi148.stdout:5/949: link d7/da/cc d7/d13/d54/d119/c147 0 2024-07-03T15:43:59.505 INFO:tasks.workunit.client.1.smithi148.stdout:7/878: sync 2024-07-03T15:43:59.505 INFO:tasks.workunit.client.1.smithi148.stdout:5/950: dread - d7/da/df/d1d/f118 zero size 2024-07-03T15:43:59.505 INFO:tasks.workunit.client.1.smithi148.stdout:7/879: chown d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65 2062 1 2024-07-03T15:43:59.505 INFO:tasks.workunit.client.1.smithi148.stdout:5/951: write d7/d13/f2a [911385,21235] 0 2024-07-03T15:43:59.505 INFO:tasks.workunit.client.1.smithi148.stdout:0/688: sync 2024-07-03T15:43:59.506 INFO:tasks.workunit.client.0.smithi028.stdout:8/94: creat d0/dd/d22/f25 x:0 0 0 2024-07-03T15:43:59.508 INFO:tasks.workunit.client.0.smithi028.stdout:7/42: mknod d9/cb 0 2024-07-03T15:43:59.510 INFO:tasks.workunit.client.0.smithi028.stdout:2/66: creat da/f18 x:0 0 0 2024-07-03T15:43:59.516 INFO:tasks.workunit.client.0.smithi028.stdout:8/95: unlink d0/c4 0 2024-07-03T15:43:59.517 INFO:tasks.workunit.client.1.smithi148.stdout:2/819: symlink d5/d14/d22/d60/d89/ddb/l102 0 2024-07-03T15:43:59.517 INFO:tasks.workunit.client.1.smithi148.stdout:2/820: write d5/d14/d86/db9/dbb/d13/d41/f63 [1023401,110617] 0 2024-07-03T15:43:59.517 INFO:tasks.workunit.client.0.smithi028.stdout:7/43: mknod d9/cc 0 2024-07-03T15:43:59.518 INFO:tasks.workunit.client.0.smithi028.stdout:3/115: dwrite f17 [0,4194304] 0 2024-07-03T15:43:59.519 INFO:tasks.workunit.client.1.smithi148.stdout:8/855: creat d4/da/d51/d18/f125 x:0 0 0 2024-07-03T15:43:59.519 INFO:tasks.workunit.client.0.smithi028.stdout:5/109: rmdir d0/dd/d14 39 2024-07-03T15:43:59.520 INFO:tasks.workunit.client.1.smithi148.stdout:6/930: truncate d3/d50/d77/f94 1434090 0 2024-07-03T15:43:59.521 INFO:tasks.workunit.client.0.smithi028.stdout:9/115: dwrite d1/d16/f19 [0,4194304] 0 2024-07-03T15:43:59.521 INFO:tasks.workunit.client.0.smithi028.stdout:6/58: dwrite d4/f11 [4194304,4194304] 0 2024-07-03T15:43:59.522 INFO:tasks.workunit.client.0.smithi028.stdout:8/96: rename d0/l1d to d0/l26 0 2024-07-03T15:43:59.522 INFO:tasks.workunit.client.0.smithi028.stdout:8/97: chown d0 1 1 2024-07-03T15:43:59.523 INFO:tasks.workunit.client.1.smithi148.stdout:7/880: getdents d5/d7d/da7/d1c/d4a/d66 0 2024-07-03T15:43:59.523 INFO:tasks.workunit.client.1.smithi148.stdout:7/881: dread - d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/db1/f10b zero size 2024-07-03T15:43:59.524 INFO:tasks.workunit.client.0.smithi028.stdout:7/44: symlink d9/ld 0 2024-07-03T15:43:59.524 INFO:tasks.workunit.client.0.smithi028.stdout:7/45: creat d9/fe x:0 0 0 2024-07-03T15:43:59.525 INFO:tasks.workunit.client.1.smithi148.stdout:8/856: dread d4/da/d14/d9e/f7d [0,4194304] 0 2024-07-03T15:43:59.525 INFO:tasks.workunit.client.1.smithi148.stdout:5/952: mknod d7/d13/d23/c148 0 2024-07-03T15:43:59.526 INFO:tasks.workunit.client.0.smithi028.stdout:3/116: symlink d18/d1a/d1e/l27 0 2024-07-03T15:43:59.528 INFO:tasks.workunit.client.1.smithi148.stdout:0/689: creat da/d17/d2a/d2f/d93/da3/fe3 x:0 0 0 2024-07-03T15:43:59.529 INFO:tasks.workunit.client.0.smithi028.stdout:2/67: mknod da/c19 0 2024-07-03T15:43:59.529 INFO:tasks.workunit.client.0.smithi028.stdout:2/68: truncate da/f12 846401 0 2024-07-03T15:43:59.531 INFO:tasks.workunit.client.1.smithi148.stdout:3/884: rename dd/d33/d52/c79 to dd/d33/d37/d68/d2e/d40/d45/d65/c11f 0 2024-07-03T15:43:59.534 INFO:tasks.workunit.client.0.smithi028.stdout:5/110: rename d0/dd/d14/f20 to d0/dd/d13/f25 0 2024-07-03T15:43:59.534 INFO:tasks.workunit.client.0.smithi028.stdout:5/111: dread - d0/dd/d13/d19/f1f zero size 2024-07-03T15:43:59.535 INFO:tasks.workunit.client.1.smithi148.stdout:2/821: creat d5/d14/d22/d60/d89/ddb/f103 x:0 0 0 2024-07-03T15:43:59.536 INFO:tasks.workunit.client.1.smithi148.stdout:4/797: sync 2024-07-03T15:43:59.539 INFO:tasks.workunit.client.1.smithi148.stdout:6/931: dwrite d3/d1d/d28/d53/d6d/fc8 [0,4194304] 0 2024-07-03T15:43:59.539 INFO:tasks.workunit.client.1.smithi148.stdout:6/932: write d3/d1d/f24 [528043,127445] 0 2024-07-03T15:43:59.539 INFO:tasks.workunit.client.1.smithi148.stdout:6/933: readlink d3/daa/ld1 0 2024-07-03T15:43:59.542 INFO:tasks.workunit.client.1.smithi148.stdout:8/857: rmdir d4/da/d14/d1a/d32/d38/db7 39 2024-07-03T15:43:59.543 INFO:tasks.workunit.client.0.smithi028.stdout:6/59: link d4/da/cd d4/da/c14 0 2024-07-03T15:43:59.545 INFO:tasks.workunit.client.0.smithi028.stdout:2/69: dwrite da/f10 [0,4194304] 0 2024-07-03T15:43:59.546 INFO:tasks.workunit.client.1.smithi148.stdout:5/953: creat d7/d13/d54/d119/f149 x:0 0 0 2024-07-03T15:43:59.548 INFO:tasks.workunit.client.0.smithi028.stdout:8/98: creat d0/d2/f27 x:0 0 0 2024-07-03T15:43:59.548 INFO:tasks.workunit.client.0.smithi028.stdout:6/60: write f2 [870559,89453] 0 2024-07-03T15:43:59.550 INFO:tasks.workunit.client.1.smithi148.stdout:0/690: mkdir da/d58/dbd/d13/d14/d50/de4 0 2024-07-03T15:43:59.553 INFO:tasks.workunit.client.1.smithi148.stdout:7/882: rename d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/l106 to d5/d7d/da7/d1c/d4a/d52/d107/d126/l133 0 2024-07-03T15:43:59.554 INFO:tasks.workunit.client.0.smithi028.stdout:7/46: mknod d9/cf 0 2024-07-03T15:43:59.555 INFO:tasks.workunit.client.1.smithi148.stdout:4/798: mknod d0/d28/d7d/c11a 0 2024-07-03T15:43:59.556 INFO:tasks.workunit.client.1.smithi148.stdout:6/934: dwrite d3/d1d/d28/d3e/f7b [0,4194304] 0 2024-07-03T15:43:59.556 INFO:tasks.workunit.client.1.smithi148.stdout:6/935: fsync d3/d1d/d28/d3e/ff7 0 2024-07-03T15:43:59.557 INFO:tasks.workunit.client.0.smithi028.stdout:3/117: unlink d18/c1f 0 2024-07-03T15:43:59.559 INFO:tasks.workunit.client.0.smithi028.stdout:3/118: write d18/d1a/f20 [1099011,46872] 0 2024-07-03T15:43:59.560 INFO:tasks.workunit.client.1.smithi148.stdout:8/858: creat d4/da/d14/d1a/d32/d38/d58/f126 x:0 0 0 2024-07-03T15:43:59.561 INFO:tasks.workunit.client.1.smithi148.stdout:5/954: dwrite d7/da/df/f71 [4194304,4194304] 0 2024-07-03T15:43:59.561 INFO:tasks.workunit.client.1.smithi148.stdout:6/936: dread d3/d1d/d28/d3e/f64 [0,4194304] 0 2024-07-03T15:43:59.562 INFO:tasks.workunit.client.1.smithi148.stdout:1/770: sync 2024-07-03T15:43:59.562 INFO:tasks.workunit.client.1.smithi148.stdout:1/771: fsync d0/d2/d11/d1e/d9d/dc2/de2/f7a 0 2024-07-03T15:43:59.568 INFO:tasks.workunit.client.0.smithi028.stdout:3/119: write f6 [1183310,43280] 0 2024-07-03T15:43:59.568 INFO:tasks.workunit.client.1.smithi148.stdout:2/822: rename d5/d14/d86/db9/dbb/d68/fac to d5/d14/d22/d60/d76/db3/f104 0 2024-07-03T15:43:59.569 INFO:tasks.workunit.client.0.smithi028.stdout:5/112: rmdir d0/dd/d13/d19 39 2024-07-03T15:43:59.571 INFO:tasks.workunit.client.1.smithi148.stdout:7/883: symlink d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/db6/l134 0 2024-07-03T15:43:59.574 INFO:tasks.workunit.client.1.smithi148.stdout:4/799: unlink d0/d28/d7d/l96 0 2024-07-03T15:43:59.578 INFO:tasks.workunit.client.1.smithi148.stdout:8/859: creat d4/da/d14/d17/deb/f127 x:0 0 0 2024-07-03T15:43:59.580 INFO:tasks.workunit.client.1.smithi148.stdout:5/955: getdents d7/da/df/da0/dd1 0 2024-07-03T15:43:59.580 INFO:tasks.workunit.client.0.smithi028.stdout:8/99: mknod d0/dd/d22/c28 0 2024-07-03T15:43:59.582 INFO:tasks.workunit.client.1.smithi148.stdout:6/937: symlink d3/d1d/dd6/l134 0 2024-07-03T15:43:59.582 INFO:tasks.workunit.client.0.smithi028.stdout:5/113: write d0/dd/f1b [779323,62236] 0 2024-07-03T15:43:59.585 INFO:tasks.workunit.client.1.smithi148.stdout:0/691: unlink da/d17/la0 0 2024-07-03T15:43:59.585 INFO:tasks.workunit.client.1.smithi148.stdout:0/692: truncate da/d17/d2a/d2f/d93/da3/da7/fdb 493758 0 2024-07-03T15:43:59.585 INFO:tasks.workunit.client.1.smithi148.stdout:5/956: write d7/da/df/d1d/f2e [1316279,74665] 0 2024-07-03T15:43:59.585 INFO:tasks.workunit.client.1.smithi148.stdout:1/772: dwrite d0/d2/ff0 [0,4194304] 0 2024-07-03T15:43:59.586 INFO:tasks.workunit.client.0.smithi028.stdout:7/47: link f4 d9/f10 0 2024-07-03T15:43:59.588 INFO:tasks.workunit.client.1.smithi148.stdout:2/823: link d5/d14/l21 d5/d14/d86/db9/dbb/d13/l105 0 2024-07-03T15:43:59.590 INFO:tasks.workunit.client.1.smithi148.stdout:7/884: rmdir d5/d7d/da7/d1c/d33/d4c/d21/d31/dfc 39 2024-07-03T15:43:59.593 INFO:tasks.workunit.client.1.smithi148.stdout:6/938: read d3/d1d/d28/d31/f4b [1889384,38030] 0 2024-07-03T15:43:59.595 INFO:tasks.workunit.client.0.smithi028.stdout:3/120: unlink d18/l22 0 2024-07-03T15:43:59.595 INFO:tasks.workunit.client.0.smithi028.stdout:3/121: creat d18/d1a/f28 x:0 0 0 2024-07-03T15:43:59.597 INFO:tasks.workunit.client.1.smithi148.stdout:1/773: symlink d0/d2/d11/d1e/d9d/dc2/de2/d77/db4/l103 0 2024-07-03T15:43:59.599 INFO:tasks.workunit.client.1.smithi148.stdout:2/824: creat d5/d14/d86/db9/dbb/f106 x:0 0 0 2024-07-03T15:43:59.599 INFO:tasks.workunit.client.1.smithi148.stdout:2/825: write d5/d14/d86/db9/fe3 [380580,9576] 0 2024-07-03T15:43:59.599 INFO:tasks.workunit.client.0.smithi028.stdout:2/70: mknod da/c1a 0 2024-07-03T15:43:59.600 INFO:tasks.workunit.client.0.smithi028.stdout:2/71: chown f7 0 1 2024-07-03T15:43:59.600 INFO:tasks.workunit.client.0.smithi028.stdout:2/72: truncate da/f15 653810 0 2024-07-03T15:43:59.600 INFO:tasks.workunit.client.0.smithi028.stdout:2/73: readlink da/l14 0 2024-07-03T15:43:59.600 INFO:tasks.workunit.client.1.smithi148.stdout:7/885: mknod d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/c135 0 2024-07-03T15:43:59.601 INFO:tasks.workunit.client.0.smithi028.stdout:8/100: mknod d0/d2/d3/dc/d11/d12/c29 0 2024-07-03T15:43:59.602 INFO:tasks.workunit.client.0.smithi028.stdout:8/101: creat d0/f2a x:0 0 0 2024-07-03T15:43:59.602 INFO:tasks.workunit.client.0.smithi028.stdout:2/74: dread da/f12 [0,4194304] 0 2024-07-03T15:43:59.603 INFO:tasks.workunit.client.1.smithi148.stdout:2/826: dread d5/f9 [0,4194304] 0 2024-07-03T15:43:59.603 INFO:tasks.workunit.client.1.smithi148.stdout:2/827: read d5/d14/d8f/fb7 [283485,90306] 0 2024-07-03T15:43:59.603 INFO:tasks.workunit.client.1.smithi148.stdout:2/828: chown d5/f2f 66 1 2024-07-03T15:43:59.603 INFO:tasks.workunit.client.1.smithi148.stdout:4/800: dwrite d0/d8/d13/d48/f9a [0,4194304] 0 2024-07-03T15:43:59.603 INFO:tasks.workunit.client.1.smithi148.stdout:8/860: getdents d4/da/d51/d18/d20/d82 0 2024-07-03T15:43:59.603 INFO:tasks.workunit.client.1.smithi148.stdout:8/861: fdatasync d4/d86/f97 0 2024-07-03T15:43:59.603 INFO:tasks.workunit.client.0.smithi028.stdout:6/61: mkdir d4/d15 0 2024-07-03T15:43:59.609 INFO:tasks.workunit.client.0.smithi028.stdout:5/114: mkdir d0/dd/d13/d19/d26 0 2024-07-03T15:43:59.609 INFO:tasks.workunit.client.0.smithi028.stdout:5/115: write d0/dd/f22 [917661,44805] 0 2024-07-03T15:43:59.611 INFO:tasks.workunit.client.1.smithi148.stdout:5/957: dwrite d7/da/df/d16/fae [4194304,4194304] 0 2024-07-03T15:43:59.612 INFO:tasks.workunit.client.1.smithi148.stdout:5/958: write d7/d13/d54/f130 [763990,16120] 0 2024-07-03T15:43:59.612 INFO:tasks.workunit.client.1.smithi148.stdout:5/959: truncate d7/d13/d23/f121 764063 0 2024-07-03T15:43:59.612 INFO:tasks.workunit.client.1.smithi148.stdout:7/886: rename d5/d7d/da7/d1c/d23/c78 to d5/d7d/da7/d1c/d4a/d66/d93/c136 0 2024-07-03T15:43:59.615 INFO:tasks.workunit.client.1.smithi148.stdout:0/693: dwrite da/d58/dbd/d13/fce [0,4194304] 0 2024-07-03T15:43:59.615 INFO:tasks.workunit.client.1.smithi148.stdout:0/694: chown da/d58/d90/dd1/cd6 34291165 1 2024-07-03T15:43:59.615 INFO:tasks.workunit.client.1.smithi148.stdout:0/695: stat da/d58/dbd/d13/d1d/d3e/f6e 0 2024-07-03T15:43:59.615 INFO:tasks.workunit.client.1.smithi148.stdout:4/801: symlink d0/d91/da8/l11b 0 2024-07-03T15:43:59.618 INFO:tasks.workunit.client.1.smithi148.stdout:4/802: dread d0/d8/d13/d48/fec [0,4194304] 0 2024-07-03T15:43:59.621 INFO:tasks.workunit.client.1.smithi148.stdout:6/939: rmdir d3/d1d/d28/d53/dd4 39 2024-07-03T15:43:59.621 INFO:tasks.workunit.client.1.smithi148.stdout:6/940: chown d3/d1d/d28/d31/c40 403 1 2024-07-03T15:43:59.621 INFO:tasks.workunit.client.0.smithi028.stdout:1/171: sync 2024-07-03T15:43:59.621 INFO:tasks.workunit.client.0.smithi028.stdout:1/172: write d4/de/d1c/f3d [159974,81672] 0 2024-07-03T15:43:59.621 INFO:tasks.workunit.client.0.smithi028.stdout:1/173: stat d4/de/d1c/l36 0 2024-07-03T15:43:59.622 INFO:tasks.workunit.client.0.smithi028.stdout:7/48: symlink d9/l11 0 2024-07-03T15:43:59.622 INFO:tasks.workunit.client.0.smithi028.stdout:7/49: stat l8 0 2024-07-03T15:43:59.623 INFO:tasks.workunit.client.1.smithi148.stdout:1/774: link d0/de/c64 d0/d2/c104 0 2024-07-03T15:43:59.623 INFO:tasks.workunit.client.1.smithi148.stdout:1/775: truncate d0/d2/d11/d1e/d9d/dc0/ff1 194950 0 2024-07-03T15:43:59.625 INFO:tasks.workunit.client.0.smithi028.stdout:3/122: mknod d18/c29 0 2024-07-03T15:43:59.626 INFO:tasks.workunit.client.1.smithi148.stdout:8/862: dwrite d4/da/d47/d88/d92/fda [0,4194304] 0 2024-07-03T15:43:59.626 INFO:tasks.workunit.client.0.smithi028.stdout:9/116: getdents d1 0 2024-07-03T15:43:59.626 INFO:tasks.workunit.client.1.smithi148.stdout:5/960: rename d7/da/d21/l80 to d7/d13/l14a 0 2024-07-03T15:43:59.627 INFO:tasks.workunit.client.0.smithi028.stdout:1/174: dread d4/de/f1f [0,4194304] 0 2024-07-03T15:43:59.627 INFO:tasks.workunit.client.0.smithi028.stdout:1/175: readlink d4/d2c/d3a/l3f 0 2024-07-03T15:43:59.627 INFO:tasks.workunit.client.0.smithi028.stdout:1/176: stat d4/la 0 2024-07-03T15:43:59.632 INFO:tasks.workunit.client.1.smithi148.stdout:0/696: symlink da/d58/dbd/d24/le5 0 2024-07-03T15:43:59.632 INFO:tasks.workunit.client.1.smithi148.stdout:2/829: dwrite d5/f9 [0,4194304] 0 2024-07-03T15:43:59.632 INFO:tasks.workunit.client.1.smithi148.stdout:2/830: chown d5/d14/d22/dc0/dfb/lb4 87483 1 2024-07-03T15:43:59.632 INFO:tasks.workunit.client.1.smithi148.stdout:2/831: chown d5/d14/d22/f72 229293750 1 2024-07-03T15:43:59.632 INFO:tasks.workunit.client.0.smithi028.stdout:8/102: rename d0/dd/d22/f1e to d0/d2/d3/dc/f2b 0 2024-07-03T15:43:59.632 INFO:tasks.workunit.client.0.smithi028.stdout:8/103: truncate d0/d2/f27 710171 0 2024-07-03T15:43:59.634 INFO:tasks.workunit.client.1.smithi148.stdout:4/803: symlink d0/d8/d3a/db8/l11c 0 2024-07-03T15:43:59.634 INFO:tasks.workunit.client.1.smithi148.stdout:4/804: truncate d0/f5 9218138 0 2024-07-03T15:43:59.635 INFO:tasks.workunit.client.0.smithi028.stdout:6/62: symlink d4/d15/l16 0 2024-07-03T15:43:59.636 INFO:tasks.workunit.client.1.smithi148.stdout:4/805: write d0/d28/da7/d2f/d41/f7a [730517,89341] 0 2024-07-03T15:43:59.637 INFO:tasks.workunit.client.1.smithi148.stdout:6/941: truncate d3/d1d/d28/d31/d4c/f78 4562026 0 2024-07-03T15:43:59.638 INFO:tasks.workunit.client.0.smithi028.stdout:5/116: mknod d0/dd/d13/d19/c27 0 2024-07-03T15:43:59.641 INFO:tasks.workunit.client.0.smithi028.stdout:7/50: write f6 [303972,25417] 0 2024-07-03T15:43:59.641 INFO:tasks.workunit.client.0.smithi028.stdout:7/51: stat d9/cc 0 2024-07-03T15:43:59.645 INFO:tasks.workunit.client.0.smithi028.stdout:3/123: link d18/f1d d18/d1a/f2a 0 2024-07-03T15:43:59.645 INFO:tasks.workunit.client.0.smithi028.stdout:3/124: creat d18/f2b x:0 0 0 2024-07-03T15:43:59.646 INFO:tasks.workunit.client.0.smithi028.stdout:3/125: truncate fe 3296073 0 2024-07-03T15:43:59.646 INFO:tasks.workunit.client.0.smithi028.stdout:3/126: readlink d18/l19 0 2024-07-03T15:43:59.648 INFO:tasks.workunit.client.1.smithi148.stdout:8/863: mknod d4/da/d14/d1a/d32/d38/db7/c128 0 2024-07-03T15:43:59.648 INFO:tasks.workunit.client.1.smithi148.stdout:8/864: getdents d4/da/d14/d1a/d32/d38/d58/d8e/db9 0 2024-07-03T15:43:59.648 INFO:tasks.workunit.client.1.smithi148.stdout:8/865: write d4/da/d47/d88/d92/f102 [412362,1266] 0 2024-07-03T15:43:59.649 INFO:tasks.workunit.client.1.smithi148.stdout:8/866: dread d4/da/d14/d3a/dab/f11f [0,4194304] 0 2024-07-03T15:43:59.650 INFO:tasks.workunit.client.0.smithi028.stdout:1/177: symlink d4/de/d1e/l4a 0 2024-07-03T15:43:59.652 INFO:tasks.workunit.client.0.smithi028.stdout:0/47: sync 2024-07-03T15:43:59.652 INFO:tasks.workunit.client.0.smithi028.stdout:4/64: sync 2024-07-03T15:43:59.652 INFO:tasks.workunit.client.0.smithi028.stdout:4/65: readlink d2/lf 0 2024-07-03T15:43:59.652 INFO:tasks.workunit.client.0.smithi028.stdout:4/66: write d2/d5/fe [762163,56709] 0 2024-07-03T15:43:59.653 INFO:tasks.workunit.client.0.smithi028.stdout:4/67: stat d2 0 2024-07-03T15:43:59.653 INFO:tasks.workunit.client.0.smithi028.stdout:4/68: write d2/f4 [5176668,104756] 0 2024-07-03T15:43:59.653 INFO:tasks.workunit.client.0.smithi028.stdout:4/69: creat d2/d5/f11 x:0 0 0 2024-07-03T15:43:59.653 INFO:tasks.workunit.client.1.smithi148.stdout:7/887: rename d5/d7d/da7/d1c/d33/d4c/f99 to d5/d7d/da7/d1c/d33/f137 0 2024-07-03T15:43:59.654 INFO:tasks.workunit.client.1.smithi148.stdout:2/832: dwrite d5/d14/d86/db9/dbb/d13/d41/d7b/d6a/f77 [0,4194304] 0 2024-07-03T15:43:59.654 INFO:tasks.workunit.client.1.smithi148.stdout:2/833: read - d5/d67/ffa zero size 2024-07-03T15:43:59.654 INFO:tasks.workunit.client.1.smithi148.stdout:2/834: fsync d5/d14/d86/db9/dbb/f1e 0 2024-07-03T15:43:59.657 INFO:tasks.workunit.client.0.smithi028.stdout:2/75: rename da/l14 to da/l1b 0 2024-07-03T15:43:59.657 INFO:tasks.workunit.client.1.smithi148.stdout:8/867: dread d4/da/d14/d17/fd5 [0,4194304] 0 2024-07-03T15:43:59.659 INFO:tasks.workunit.client.0.smithi028.stdout:2/76: write da/f12 [510880,114776] 0 2024-07-03T15:43:59.660 INFO:tasks.workunit.client.1.smithi148.stdout:0/697: symlink da/d58/dbd/d13/d1d/d3e/le6 0 2024-07-03T15:43:59.663 INFO:tasks.workunit.client.0.smithi028.stdout:6/63: unlink d4/da/cc 0 2024-07-03T15:43:59.665 INFO:tasks.workunit.client.0.smithi028.stdout:8/104: dwrite d0/d2/d3/fa [4194304,4194304] 0 2024-07-03T15:43:59.667 INFO:tasks.workunit.client.0.smithi028.stdout:7/52: mknod d9/c12 0 2024-07-03T15:43:59.667 INFO:tasks.workunit.client.0.smithi028.stdout:7/53: chown f0 159 1 2024-07-03T15:43:59.670 INFO:tasks.workunit.client.0.smithi028.stdout:8/105: write d0/f20 [80514,30410] 0 2024-07-03T15:43:59.670 INFO:tasks.workunit.client.0.smithi028.stdout:8/106: write d0/f20 [1160022,36583] 0 2024-07-03T15:43:59.671 INFO:tasks.workunit.client.0.smithi028.stdout:7/54: dread f7 [0,4194304] 0 2024-07-03T15:43:59.671 INFO:tasks.workunit.client.0.smithi028.stdout:7/55: creat d9/f13 x:0 0 0 2024-07-03T15:43:59.671 INFO:tasks.workunit.client.0.smithi028.stdout:7/56: dread - d9/f13 zero size 2024-07-03T15:43:59.675 INFO:tasks.workunit.client.0.smithi028.stdout:0/48: mknod d2/d3/c10 0 2024-07-03T15:43:59.675 INFO:tasks.workunit.client.0.smithi028.stdout:0/49: stat d2/f9 0 2024-07-03T15:43:59.675 INFO:tasks.workunit.client.0.smithi028.stdout:0/50: dread - d2/fa zero size 2024-07-03T15:43:59.675 INFO:tasks.workunit.client.0.smithi028.stdout:0/51: chown d2/d3 72885226 1 2024-07-03T15:43:59.675 INFO:tasks.workunit.client.0.smithi028.stdout:0/52: creat d2/d3/f11 x:0 0 0 2024-07-03T15:43:59.679 INFO:tasks.workunit.client.0.smithi028.stdout:5/117: dwrite d0/dd/d13/f25 [0,4194304] 0 2024-07-03T15:43:59.683 INFO:tasks.workunit.client.1.smithi148.stdout:4/806: mknod d0/d8/d13/db4/c11d 0 2024-07-03T15:43:59.684 INFO:tasks.workunit.client.0.smithi028.stdout:7/57: symlink d9/l14 0 2024-07-03T15:43:59.685 INFO:tasks.workunit.client.1.smithi148.stdout:6/942: mknod d3/d1d/d28/d53/dd4/d103/c135 0 2024-07-03T15:43:59.685 INFO:tasks.workunit.client.0.smithi028.stdout:4/70: dread d2/f4 [0,4194304] 0 2024-07-03T15:43:59.686 INFO:tasks.workunit.client.0.smithi028.stdout:3/127: dwrite f15 [0,4194304] 0 2024-07-03T15:43:59.686 INFO:tasks.workunit.client.0.smithi028.stdout:3/128: write d18/d1a/f28 [214801,79247] 0 2024-07-03T15:43:59.686 INFO:tasks.workunit.client.0.smithi028.stdout:3/129: chown f4 2343787 1 2024-07-03T15:43:59.687 INFO:tasks.workunit.client.1.smithi148.stdout:1/776: rename d0/d2/d11/d1e/d9d/dc2/de2/d77/db4 to d0/de/d44/d54/d105 0 2024-07-03T15:43:59.687 INFO:tasks.workunit.client.1.smithi148.stdout:1/777: truncate d0/de/d2f/d72/df9/d42/daf/fd6 391029 0 2024-07-03T15:43:59.688 INFO:tasks.workunit.client.0.smithi028.stdout:0/53: unlink d2/d3/cc 0 2024-07-03T15:43:59.688 INFO:tasks.workunit.client.0.smithi028.stdout:0/54: dread - d2/fa zero size 2024-07-03T15:43:59.692 INFO:tasks.workunit.client.1.smithi148.stdout:7/888: rmdir d5/d7d/da7/d1c/d4a/d52/d127 0 2024-07-03T15:43:59.695 INFO:tasks.workunit.client.1.smithi148.stdout:2/835: link d5/d14/d86/db9/dbb/d13/d41/l6e d5/d14/d86/d93/dc5/l107 0 2024-07-03T15:43:59.696 INFO:tasks.workunit.client.0.smithi028.stdout:1/178: dwrite d4/de/f10 [0,4194304] 0 2024-07-03T15:43:59.698 INFO:tasks.workunit.client.1.smithi148.stdout:8/868: dwrite d4/da/d47/d88/fa0 [0,4194304] 0 2024-07-03T15:43:59.700 INFO:tasks.workunit.client.1.smithi148.stdout:4/807: creat d0/d91/da8/dab/dad/f11e x:0 0 0 2024-07-03T15:43:59.701 INFO:tasks.workunit.client.0.smithi028.stdout:5/118: mknod d0/dd/d14/c28 0 2024-07-03T15:43:59.701 INFO:tasks.workunit.client.0.smithi028.stdout:5/119: chown d0/dd/d13/f25 12823416 1 2024-07-03T15:43:59.702 INFO:tasks.workunit.client.0.smithi028.stdout:6/64: dwrite d4/f10 [0,4194304] 0 2024-07-03T15:43:59.702 INFO:tasks.workunit.client.1.smithi148.stdout:1/778: truncate d0/de/d44/d54/d5d/fc1 3712398 0 2024-07-03T15:43:59.704 INFO:tasks.workunit.client.1.smithi148.stdout:1/779: dread d0/de/d2f/d72/df9/d42/d62/d7b/fcc [0,4194304] 0 2024-07-03T15:43:59.704 INFO:tasks.workunit.client.0.smithi028.stdout:2/77: rename da/f18 to da/f1c 0 2024-07-03T15:43:59.705 INFO:tasks.workunit.client.0.smithi028.stdout:2/78: stat da/ff 0 2024-07-03T15:43:59.705 INFO:tasks.workunit.client.1.smithi148.stdout:0/698: dwrite da/d58/dbd/d4a/fbc [0,4194304] 0 2024-07-03T15:43:59.708 INFO:tasks.workunit.client.0.smithi028.stdout:7/58: symlink d9/l15 0 2024-07-03T15:43:59.708 INFO:tasks.workunit.client.1.smithi148.stdout:6/943: dwrite d3/d1d/d28/d53/dd4/d103/f12d [4194304,4194304] 0 2024-07-03T15:43:59.708 INFO:tasks.workunit.client.1.smithi148.stdout:6/944: fsync d3/d1d/d28/d31/d61/dac/f11c 0 2024-07-03T15:43:59.711 INFO:tasks.workunit.client.1.smithi148.stdout:8/869: symlink d4/da/d14/d3a/da8/l129 0 2024-07-03T15:43:59.714 INFO:tasks.workunit.client.0.smithi028.stdout:5/120: write d0/dd/d13/f17 [4027757,65494] 0 2024-07-03T15:43:59.714 INFO:tasks.workunit.client.0.smithi028.stdout:5/121: fsync d0/dd/f22 0 2024-07-03T15:43:59.716 INFO:tasks.workunit.client.0.smithi028.stdout:4/71: mkdir d2/d12 0 2024-07-03T15:43:59.716 INFO:tasks.workunit.client.0.smithi028.stdout:4/72: fdatasync d2/f7 0 2024-07-03T15:43:59.718 INFO:tasks.workunit.client.1.smithi148.stdout:4/808: mknod d0/d28/c11f 0 2024-07-03T15:43:59.718 INFO:tasks.workunit.client.1.smithi148.stdout:6/945: dread d3/d1d/d28/d3e/f64 [0,4194304] 0 2024-07-03T15:43:59.718 INFO:tasks.workunit.client.1.smithi148.stdout:6/946: chown d3/d1d/d28/d53/l114 10989130 1 2024-07-03T15:43:59.720 INFO:tasks.workunit.client.1.smithi148.stdout:1/780: mkdir d0/d2/dae/d106 0 2024-07-03T15:43:59.720 INFO:tasks.workunit.client.1.smithi148.stdout:1/781: fdatasync d0/de/d2f/d72/df9/d42/d62/f71 0 2024-07-03T15:43:59.722 INFO:tasks.workunit.client.0.smithi028.stdout:3/130: rename d18/d1a/c23 to d18/d1a/d1e/c2c 0 2024-07-03T15:43:59.722 INFO:tasks.workunit.client.0.smithi028.stdout:3/131: creat d18/f2d x:0 0 0 2024-07-03T15:43:59.723 INFO:tasks.workunit.client.0.smithi028.stdout:3/132: creat d18/d1a/f2e x:0 0 0 2024-07-03T15:43:59.725 INFO:tasks.workunit.client.0.smithi028.stdout:1/179: symlink d4/de/d1c/l4b 0 2024-07-03T15:43:59.725 INFO:tasks.workunit.client.0.smithi028.stdout:1/180: chown d4/f29 95713 1 2024-07-03T15:43:59.725 INFO:tasks.workunit.client.0.smithi028.stdout:0/55: dwrite d2/d3/f11 [0,4194304] 0 2024-07-03T15:43:59.730 INFO:tasks.workunit.client.1.smithi148.stdout:6/947: dread d3/fb7 [4194304,4194304] 0 2024-07-03T15:43:59.731 INFO:tasks.workunit.client.0.smithi028.stdout:3/133: dread f17 [0,4194304] 0 2024-07-03T15:43:59.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:43:59 smithi148 ceph-mon[36056]: pgmap v242: 65 pgs: 65 active+clean; 1.5 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 144 MiB/s rd, 175 MiB/s wr, 538 op/s 2024-07-03T15:43:59.735 INFO:tasks.workunit.client.1.smithi148.stdout:0/699: creat da/d58/dbd/d24/d84/fe7 x:0 0 0 2024-07-03T15:43:59.736 INFO:tasks.workunit.client.0.smithi028.stdout:6/65: creat d4/da/f17 x:0 0 0 2024-07-03T15:43:59.741 INFO:tasks.workunit.client.0.smithi028.stdout:5/122: mkdir d0/dd/d13/d29 0 2024-07-03T15:43:59.741 INFO:tasks.workunit.client.0.smithi028.stdout:5/123: stat d0/dd/f1b 0 2024-07-03T15:43:59.742 INFO:tasks.workunit.client.1.smithi148.stdout:7/889: dwrite d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/db6/fe7 [0,4194304] 0 2024-07-03T15:43:59.743 INFO:tasks.workunit.client.1.smithi148.stdout:2/836: dwrite d5/d14/d86/db9/dbb/d68/fc2 [0,4194304] 0 2024-07-03T15:43:59.743 INFO:tasks.workunit.client.1.smithi148.stdout:8/870: rename d4/da/d14/d1a/d32/d89/l90 to d4/da/l12a 0 2024-07-03T15:43:59.743 INFO:tasks.workunit.client.1.smithi148.stdout:8/871: truncate d4/da/d51/f115 1007053 0 2024-07-03T15:43:59.743 INFO:tasks.workunit.client.1.smithi148.stdout:8/872: dread - d4/da/d14/d1a/d32/f5e zero size 2024-07-03T15:43:59.745 INFO:tasks.workunit.client.1.smithi148.stdout:4/809: stat d0/d91/da8/dab/d49/l4c 0 2024-07-03T15:43:59.746 INFO:tasks.workunit.client.1.smithi148.stdout:4/810: dread d0/d28/d7d/d6f/d7c/da3/fca [0,4194304] 0 2024-07-03T15:43:59.746 INFO:tasks.workunit.client.1.smithi148.stdout:4/811: fdatasync d0/d28/da7/ffd 0 2024-07-03T15:43:59.747 INFO:tasks.workunit.client.1.smithi148.stdout:1/782: rmdir d0/de 39 2024-07-03T15:43:59.748 INFO:tasks.workunit.client.1.smithi148.stdout:6/948: dwrite d3/fa9 [0,4194304] 0 2024-07-03T15:43:59.748 INFO:tasks.workunit.client.1.smithi148.stdout:6/949: fdatasync d3/d1d/d28/d53/d6d/d8f/f127 0 2024-07-03T15:43:59.748 INFO:tasks.workunit.client.1.smithi148.stdout:6/950: chown d3/daa/ld5 42487577 1 2024-07-03T15:43:59.749 INFO:tasks.workunit.client.1.smithi148.stdout:1/783: dread d0/de/d2f/fbf [0,4194304] 0 2024-07-03T15:43:59.750 INFO:tasks.workunit.client.0.smithi028.stdout:1/181: rename d4/d2c/d3a/l3f to d4/de/d1c/d44/l4c 0 2024-07-03T15:43:59.751 INFO:tasks.workunit.client.1.smithi148.stdout:0/700: creat da/d17/fe8 x:0 0 0 2024-07-03T15:43:59.756 INFO:tasks.workunit.client.0.smithi028.stdout:0/56: link d2/d3/c10 d2/d3/c12 0 2024-07-03T15:43:59.756 INFO:tasks.workunit.client.0.smithi028.stdout:0/57: readlink - no filename 2024-07-03T15:43:59.773 INFO:tasks.workunit.client.1.smithi148.stdout:7/890: mknod d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/db1/c138 0 2024-07-03T15:43:59.773 INFO:tasks.workunit.client.1.smithi148.stdout:7/891: truncate d5/d7d/da7/d1c/d59/ff7 298100 0 2024-07-03T15:43:59.776 INFO:tasks.workunit.client.0.smithi028.stdout:6/66: stat d4/l6 0 2024-07-03T15:43:59.777 INFO:tasks.workunit.client.0.smithi028.stdout:1/182: dwrite d4/f29 [4194304,4194304] 0 2024-07-03T15:43:59.778 INFO:tasks.workunit.client.1.smithi148.stdout:0/701: dread da/d58/dbd/d13/d1d/d3e/f5c [0,4194304] 0 2024-07-03T15:43:59.778 INFO:tasks.workunit.client.1.smithi148.stdout:0/702: dread - da/d58/dbd/d13/fd9 zero size 2024-07-03T15:43:59.778 INFO:tasks.workunit.client.1.smithi148.stdout:0/703: creat da/d17/d2a/d2f/d93/fe9 x:0 0 0 2024-07-03T15:43:59.778 INFO:tasks.workunit.client.1.smithi148.stdout:0/704: fsync da/d58/dbd/d13/d14/d50/d95/d8c/f4d 0 2024-07-03T15:43:59.778 INFO:tasks.workunit.client.1.smithi148.stdout:0/705: creat da/d58/dbd/d13/fea x:0 0 0 2024-07-03T15:43:59.778 INFO:tasks.workunit.client.1.smithi148.stdout:0/706: chown da/d58/dbd/d13/d1d/da1 34988 1 2024-07-03T15:43:59.779 INFO:tasks.workunit.client.0.smithi028.stdout:2/79: unlink da/ld 0 2024-07-03T15:43:59.779 INFO:tasks.workunit.client.1.smithi148.stdout:2/837: creat d5/d67/f108 x:0 0 0 2024-07-03T15:43:59.782 INFO:tasks.workunit.client.0.smithi028.stdout:5/124: rename d0/dd/c10 to d0/dd/d1e/c2a 0 2024-07-03T15:43:59.791 INFO:tasks.workunit.client.1.smithi148.stdout:6/951: creat d3/d1d/d28/d53/dd4/d103/f136 x:0 0 0 2024-07-03T15:43:59.791 INFO:tasks.workunit.client.1.smithi148.stdout:6/952: write d3/d1d/d28/d3e/ddd/d74/dc4/fce [2170713,98742] 0 2024-07-03T15:43:59.794 INFO:tasks.workunit.client.1.smithi148.stdout:1/784: unlink d0/de/d2f/d72/df9/d42/f87 0 2024-07-03T15:43:59.796 INFO:tasks.workunit.client.1.smithi148.stdout:7/892: rmdir d5/d7d/da7/d1c 39 2024-07-03T15:43:59.796 INFO:tasks.workunit.client.1.smithi148.stdout:7/893: write d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/db6/fbe [1501619,44154] 0 2024-07-03T15:43:59.800 INFO:tasks.workunit.client.0.smithi028.stdout:0/58: symlink d2/d3/l13 0 2024-07-03T15:43:59.802 INFO:tasks.workunit.client.1.smithi148.stdout:2/838: dwrite d5/d14/d86/db9/dbb/d13/ff7 [0,4194304] 0 2024-07-03T15:43:59.802 INFO:tasks.workunit.client.1.smithi148.stdout:2/839: chown d5/d14/l9f 32646 1 2024-07-03T15:43:59.802 INFO:tasks.workunit.client.1.smithi148.stdout:2/840: chown d5/f9 76 1 2024-07-03T15:43:59.802 INFO:tasks.workunit.client.1.smithi148.stdout:2/841: readlink d5/d14/d86/db9/dbb/d13/d41/le9 0 2024-07-03T15:43:59.803 INFO:tasks.workunit.client.0.smithi028.stdout:6/67: unlink d4/l6 0 2024-07-03T15:43:59.803 INFO:tasks.workunit.client.0.smithi028.stdout:6/68: dread - d4/da/f17 zero size 2024-07-03T15:43:59.804 INFO:tasks.workunit.client.1.smithi148.stdout:4/812: getdents d0/d28/da7/d2f/d41 0 2024-07-03T15:43:59.804 INFO:tasks.workunit.client.1.smithi148.stdout:4/813: chown d0/d8/d13/d48/f3f 524 1 2024-07-03T15:43:59.804 INFO:tasks.workunit.client.1.smithi148.stdout:4/814: creat d0/d8/d3a/dce/df7/f120 x:0 0 0 2024-07-03T15:43:59.806 INFO:tasks.workunit.client.1.smithi148.stdout:1/785: stat d0/de/c64 0 2024-07-03T15:43:59.808 INFO:tasks.workunit.client.0.smithi028.stdout:1/183: rmdir d4/d2c/d3a 39 2024-07-03T15:43:59.808 INFO:tasks.workunit.client.0.smithi028.stdout:1/184: write d4/de/ff [354346,73393] 0 2024-07-03T15:43:59.809 INFO:tasks.workunit.client.1.smithi148.stdout:7/894: truncate d5/d7d/da7/d1c/d4a/d52/f5f 3853756 0 2024-07-03T15:43:59.812 INFO:tasks.workunit.client.0.smithi028.stdout:5/125: stat d0/dd/ce 0 2024-07-03T15:43:59.815 INFO:tasks.workunit.client.1.smithi148.stdout:2/842: symlink d5/d14/d86/db9/dbb/d13/d1d/da1/l109 0 2024-07-03T15:43:59.815 INFO:tasks.workunit.client.1.smithi148.stdout:1/786: write d0/d2/d6/d14/d29/d94/fa0 [3252600,11902] 0 2024-07-03T15:43:59.816 INFO:tasks.workunit.client.0.smithi028.stdout:6/69: creat d4/f18 x:0 0 0 2024-07-03T15:43:59.816 INFO:tasks.workunit.client.0.smithi028.stdout:6/70: chown d4/da/cf 0 1 2024-07-03T15:43:59.816 INFO:tasks.workunit.client.0.smithi028.stdout:5/126: dread d0/dd/d13/f17 [0,4194304] 0 2024-07-03T15:43:59.819 INFO:tasks.workunit.client.0.smithi028.stdout:1/185: mknod d4/d11/c4d 0 2024-07-03T15:43:59.820 INFO:tasks.workunit.client.1.smithi148.stdout:4/815: creat d0/d28/f121 x:0 0 0 2024-07-03T15:43:59.820 INFO:tasks.workunit.client.1.smithi148.stdout:4/816: write d0/d8/dd4/f116 [313144,125236] 0 2024-07-03T15:43:59.822 INFO:tasks.workunit.client.0.smithi028.stdout:1/186: mkdir d4/de/d1e/d4e 0 2024-07-03T15:43:59.824 INFO:tasks.workunit.client.1.smithi148.stdout:7/895: unlink d5/d7d/da7/d1c/d33/d4c/d84/d98/d9e/fdf 0 2024-07-03T15:43:59.824 INFO:tasks.workunit.client.1.smithi148.stdout:2/843: write d5/d14/f58 [7625038,84087] 0 2024-07-03T15:43:59.824 INFO:tasks.workunit.client.0.smithi028.stdout:5/127: dread d0/dd/d13/f25 [0,4194304] 0 2024-07-03T15:43:59.824 INFO:tasks.workunit.client.0.smithi028.stdout:5/128: fdatasync d0/dd/d13/f25 0 2024-07-03T15:43:59.824 INFO:tasks.workunit.client.0.smithi028.stdout:5/129: mknod d0/dd/d14/c2b 0 2024-07-03T15:43:59.826 INFO:tasks.workunit.client.1.smithi148.stdout:2/844: link d5/d14/f58 d5/d14/d22/d60/d89/f10a 0 2024-07-03T15:43:59.829 INFO:tasks.workunit.client.1.smithi148.stdout:2/845: creat d5/d14/d8b/dd5/f10b x:0 0 0 2024-07-03T15:43:59.829 INFO:tasks.workunit.client.1.smithi148.stdout:2/846: truncate d5/d14/d22/d60/d89/ddb/f103 113536 0 2024-07-03T15:43:59.829 INFO:tasks.workunit.client.1.smithi148.stdout:2/847: creat d5/d14/f10c x:0 0 0 2024-07-03T15:43:59.829 INFO:tasks.workunit.client.1.smithi148.stdout:2/848: fsync d5/d14/d86/db9/dbb/d68/fc2 0 2024-07-03T15:43:59.829 INFO:tasks.workunit.client.1.smithi148.stdout:2/849: truncate d5/d14/d86/db9/dbb/d13/d41/d7b/d6a/f77 4361588 0 2024-07-03T15:43:59.829 INFO:tasks.workunit.client.1.smithi148.stdout:2/850: chown d5/d14/d86/db9/dbb/d13/d41/d7b/fc8 475273152 1 2024-07-03T15:43:59.832 INFO:tasks.workunit.client.1.smithi148.stdout:2/851: creat d5/d14/d86/db9/dbb/d68/dcd/f10d x:0 0 0 2024-07-03T15:43:59.833 INFO:tasks.workunit.client.1.smithi148.stdout:2/852: mknod d5/d14/d22/dc0/dfb/c10e 0 2024-07-03T15:43:59.837 INFO:tasks.workunit.client.1.smithi148.stdout:3/885: sync 2024-07-03T15:43:59.837 INFO:tasks.workunit.client.1.smithi148.stdout:5/961: sync 2024-07-03T15:43:59.837 INFO:tasks.workunit.client.1.smithi148.stdout:5/962: chown d7/da/df/da0/db2/c125 0 1 2024-07-03T15:43:59.839 INFO:tasks.workunit.client.1.smithi148.stdout:3/886: rename dd/d33/d37/d68/d2e/d61/d10e to dd/d35/dac/d120 0 2024-07-03T15:43:59.839 INFO:tasks.workunit.client.1.smithi148.stdout:3/887: stat dd/d33/d37/d68 0 2024-07-03T15:43:59.840 INFO:tasks.workunit.client.0.smithi028.stdout:1/187: dread d4/d11/d16/f27 [0,4194304] 0 2024-07-03T15:43:59.841 INFO:tasks.workunit.client.0.smithi028.stdout:1/188: symlink d4/d2c/d3a/d43/l4f 0 2024-07-03T15:43:59.842 INFO:tasks.workunit.client.1.smithi148.stdout:3/888: rename dd/da6 to dd/d33/d52/d57/d121 0 2024-07-03T15:43:59.842 INFO:tasks.workunit.client.0.smithi028.stdout:1/189: mknod d4/d11/d16/c50 0 2024-07-03T15:43:59.845 INFO:tasks.workunit.client.1.smithi148.stdout:3/889: unlink dd/d33/d52/d57/fae 0 2024-07-03T15:43:59.845 INFO:tasks.workunit.client.1.smithi148.stdout:5/963: rename d7/d13/d23/f3b to d7/da/df/d16/dfb/f14b 0 2024-07-03T15:43:59.846 INFO:tasks.workunit.client.0.smithi028.stdout:1/190: mkdir d4/de/d1e/d4e/d51 0 2024-07-03T15:43:59.847 INFO:tasks.workunit.client.0.smithi028.stdout:1/191: creat d4/de/f52 x:0 0 0 2024-07-03T15:43:59.847 INFO:tasks.workunit.client.0.smithi028.stdout:1/192: getdents d4/d3c 0 2024-07-03T15:43:59.847 INFO:tasks.workunit.client.1.smithi148.stdout:1/787: dread d0/d2/d2e/f85 [0,4194304] 0 2024-07-03T15:43:59.849 INFO:tasks.workunit.client.0.smithi028.stdout:5/130: dread d0/dd/f1b [0,4194304] 0 2024-07-03T15:43:59.850 INFO:tasks.workunit.client.0.smithi028.stdout:5/131: chown d0/dd/d14/c28 10556826 1 2024-07-03T15:43:59.851 INFO:tasks.workunit.client.0.smithi028.stdout:5/132: truncate d0/dd/df/f16 982474 0 2024-07-03T15:43:59.853 INFO:tasks.workunit.client.1.smithi148.stdout:6/953: read d3/d1d/d28/d31/d61/fd7 [2325689,113748] 0 2024-07-03T15:43:59.854 INFO:tasks.workunit.client.1.smithi148.stdout:5/964: read d7/da/df/d16/dfb/d13e/f50 [2511019,130836] 0 2024-07-03T15:43:59.855 INFO:tasks.workunit.client.1.smithi148.stdout:1/788: rmdir d0/d2/d11/d1e/d9d/dc2/de2/d77/da3/dec 0 2024-07-03T15:43:59.855 INFO:tasks.workunit.client.1.smithi148.stdout:1/789: stat d0/de/d2f/d72/df9/d42/db7/dbb 0 2024-07-03T15:43:59.856 INFO:tasks.workunit.client.1.smithi148.stdout:6/954: symlink d3/d1d/d28/d3e/ddd/d74/d12f/l137 0 2024-07-03T15:43:59.856 INFO:tasks.workunit.client.1.smithi148.stdout:6/955: write d3/f55 [2584906,4917] 0 2024-07-03T15:43:59.858 INFO:tasks.workunit.client.1.smithi148.stdout:1/790: rmdir d0/de/d2f/d72/df9/d42/db7 39 2024-07-03T15:43:59.858 INFO:tasks.workunit.client.1.smithi148.stdout:1/791: chown d0/d2/d6/c68 246861 1 2024-07-03T15:43:59.859 INFO:tasks.workunit.client.1.smithi148.stdout:6/956: symlink d3/d50/d133/l138 0 2024-07-03T15:43:59.859 INFO:tasks.workunit.client.1.smithi148.stdout:6/957: chown d3/d1d/d28/d53/la0 3115 1 2024-07-03T15:43:59.859 INFO:tasks.workunit.client.1.smithi148.stdout:6/958: write d3/d1d/d28/d3e/ddd/fec [3056531,64166] 0 2024-07-03T15:43:59.861 INFO:tasks.workunit.client.1.smithi148.stdout:1/792: symlink d0/d2/dae/dfc/l107 0 2024-07-03T15:43:59.861 INFO:tasks.workunit.client.1.smithi148.stdout:1/793: chown d0/d2/d2e/fa7 3 1 2024-07-03T15:43:59.868 INFO:tasks.workunit.client.0.smithi028.stdout:6/71: dread d4/f11 [4194304,4194304] 0 2024-07-03T15:43:59.868 INFO:tasks.workunit.client.0.smithi028.stdout:6/72: chown d4/da/cd 74688 1 2024-07-03T15:43:59.872 INFO:tasks.workunit.client.1.smithi148.stdout:0/707: dwrite da/d58/dbd/d94/dba/fd7 [0,4194304] 0 2024-07-03T15:43:59.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:43:59 smithi028 ceph-mon[29148]: pgmap v242: 65 pgs: 65 active+clean; 1.5 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 144 MiB/s rd, 175 MiB/s wr, 538 op/s 2024-07-03T15:43:59.876 INFO:tasks.workunit.client.0.smithi028.stdout:2/80: dwrite f4 [4194304,4194304] 0 2024-07-03T15:43:59.877 INFO:tasks.workunit.client.1.smithi148.stdout:4/817: dwrite d0/d28/db0/fcb [0,4194304] 0 2024-07-03T15:43:59.879 INFO:tasks.workunit.client.0.smithi028.stdout:1/193: fsync d4/de/f52 0 2024-07-03T15:43:59.886 INFO:tasks.workunit.client.1.smithi148.stdout:6/959: rmdir d3/d1d 39 2024-07-03T15:43:59.886 INFO:tasks.workunit.client.1.smithi148.stdout:6/960: creat d3/d50/f139 x:0 0 0 2024-07-03T15:43:59.891 INFO:tasks.workunit.client.0.smithi028.stdout:0/59: dwrite d2/f9 [0,4194304] 0 2024-07-03T15:43:59.892 INFO:tasks.workunit.client.1.smithi148.stdout:7/896: dwrite d5/f9 [0,4194304] 0 2024-07-03T15:43:59.893 INFO:tasks.workunit.client.1.smithi148.stdout:5/965: dwrite d7/da/df/d1d/ff9 [0,4194304] 0 2024-07-03T15:43:59.895 INFO:tasks.workunit.client.0.smithi028.stdout:1/194: write d4/d11/f12 [103997,12673] 0 2024-07-03T15:43:59.895 INFO:tasks.workunit.client.0.smithi028.stdout:1/195: getdents d4/d3c 0 2024-07-03T15:43:59.900 INFO:tasks.workunit.client.0.smithi028.stdout:1/196: mknod d4/d2c/c53 0 2024-07-03T15:43:59.904 INFO:tasks.workunit.client.0.smithi028.stdout:0/60: rmdir d2/d3 39 2024-07-03T15:43:59.905 INFO:tasks.workunit.client.0.smithi028.stdout:0/61: write d2/fb [790545,129290] 0 2024-07-03T15:43:59.905 INFO:tasks.workunit.client.0.smithi028.stdout:0/62: readlink d2/d3/l13 0 2024-07-03T15:43:59.905 INFO:tasks.workunit.client.0.smithi028.stdout:5/133: dwrite d0/dd/d13/f17 [0,4194304] 0 2024-07-03T15:43:59.906 INFO:tasks.workunit.client.1.smithi148.stdout:1/794: dwrite d0/d2/d11/d1e/f59 [0,4194304] 0 2024-07-03T15:43:59.908 INFO:tasks.workunit.client.0.smithi028.stdout:0/63: link d2/d3/c12 d2/c14 0 2024-07-03T15:43:59.913 INFO:tasks.workunit.client.1.smithi148.stdout:6/961: rmdir d3/d1d/d28/d31/d4c 39 2024-07-03T15:43:59.913 INFO:tasks.workunit.client.1.smithi148.stdout:6/962: fsync d3/d1d/d28/d53/dd4/d103/f11e 0 2024-07-03T15:43:59.913 INFO:tasks.workunit.client.1.smithi148.stdout:6/963: chown d3/d1d/d28/l45 257865809 1 2024-07-03T15:43:59.915 INFO:tasks.workunit.client.0.smithi028.stdout:6/73: dwrite d4/f10 [0,4194304] 0 2024-07-03T15:43:59.916 INFO:tasks.workunit.client.1.smithi148.stdout:7/897: unlink d5/f9f 0 2024-07-03T15:43:59.916 INFO:tasks.workunit.client.0.smithi028.stdout:2/81: dwrite f7 [0,4194304] 0 2024-07-03T15:43:59.919 INFO:tasks.workunit.client.1.smithi148.stdout:7/898: write d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/db6/fbe [692067,85846] 0 2024-07-03T15:43:59.919 INFO:tasks.workunit.client.1.smithi148.stdout:7/899: write d5/d7d/da7/d1c/d4a/d66/dd7/ffe [702259,83147] 0 2024-07-03T15:43:59.919 INFO:tasks.workunit.client.1.smithi148.stdout:7/900: readlink d5/d7d/da7/d1c/l1e 0 2024-07-03T15:43:59.920 INFO:tasks.workunit.client.1.smithi148.stdout:5/966: creat d7/da/df/da0/db2/d132/d60/f14c x:0 0 0 2024-07-03T15:43:59.921 INFO:tasks.workunit.client.1.smithi148.stdout:5/967: stat d7/da/df/d16/de4/da7/c6b 0 2024-07-03T15:43:59.923 INFO:tasks.workunit.client.1.smithi148.stdout:7/901: truncate d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/f83 357198 0 2024-07-03T15:43:59.924 INFO:tasks.workunit.client.1.smithi148.stdout:5/968: rmdir d7/da/df/d1d/d109/d126 39 2024-07-03T15:43:59.927 INFO:tasks.workunit.client.0.smithi028.stdout:0/64: rmdir d2 39 2024-07-03T15:43:59.929 INFO:tasks.workunit.client.1.smithi148.stdout:7/902: dread d5/d7d/da7/d1c/d33/d4c/d21/f113 [0,4194304] 0 2024-07-03T15:43:59.929 INFO:tasks.workunit.client.1.smithi148.stdout:7/903: chown d5/d7d/da7/d1c/d33/d4c/d21/c111 13896 1 2024-07-03T15:43:59.929 INFO:tasks.workunit.client.1.smithi148.stdout:7/904: chown d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/db1/cda 110876 1 2024-07-03T15:43:59.930 INFO:tasks.workunit.client.1.smithi148.stdout:4/818: dwrite d0/d28/da7/d2f/d41/f7e [0,4194304] 0 2024-07-03T15:43:59.932 INFO:tasks.workunit.client.1.smithi148.stdout:0/708: dwrite da/d58/dbd/d94/dac/ddd/f6b [0,4194304] 0 2024-07-03T15:43:59.946 INFO:tasks.workunit.client.0.smithi028.stdout:1/197: dwrite d4/d11/f1b [0,4194304] 0 2024-07-03T15:43:59.946 INFO:tasks.workunit.client.0.smithi028.stdout:6/74: dread d4/f10 [0,4194304] 0 2024-07-03T15:43:59.948 INFO:tasks.workunit.client.1.smithi148.stdout:0/709: dread da/d58/dbd/d94/dac/ddd/f7c [0,4194304] 0 2024-07-03T15:43:59.948 INFO:tasks.workunit.client.1.smithi148.stdout:1/795: dwrite d0/de/d89/f95 [0,4194304] 0 2024-07-03T15:43:59.948 INFO:tasks.workunit.client.1.smithi148.stdout:0/710: write da/d58/dbd/d24/dc8/d8a/fa8 [58475,45676] 0 2024-07-03T15:43:59.948 INFO:tasks.workunit.client.1.smithi148.stdout:0/711: fsync da/d17/d2a/d2f/d93/fe9 0 2024-07-03T15:43:59.948 INFO:tasks.workunit.client.1.smithi148.stdout:0/712: chown da/d58/dbd/d24/d84/fe7 8138339 1 2024-07-03T15:43:59.949 INFO:tasks.workunit.client.1.smithi148.stdout:1/796: dread d0/d2/d6/f5a [0,4194304] 0 2024-07-03T15:43:59.950 INFO:tasks.workunit.client.1.smithi148.stdout:1/797: dread d0/de/d2f/d72/df9/d42/d62/d7b/fcc [0,4194304] 0 2024-07-03T15:43:59.953 INFO:tasks.workunit.client.1.smithi148.stdout:1/798: mknod d0/de/d44/d54/d105/c108 0 2024-07-03T15:43:59.953 INFO:tasks.workunit.client.1.smithi148.stdout:1/799: creat d0/d2/d6/d14/f109 x:0 0 0 2024-07-03T15:43:59.954 INFO:tasks.workunit.client.1.smithi148.stdout:6/964: dwrite d3/d1d/d28/d3e/ff1 [0,4194304] 0 2024-07-03T15:43:59.955 INFO:tasks.workunit.client.0.smithi028.stdout:1/198: mkdir d4/d3c/d54 0 2024-07-03T15:43:59.956 INFO:tasks.workunit.client.0.smithi028.stdout:5/134: dwrite d0/dd/d13/f17 [0,4194304] 0 2024-07-03T15:43:59.957 INFO:tasks.workunit.client.0.smithi028.stdout:5/135: write d0/dd/f1b [1631649,17268] 0 2024-07-03T15:43:59.957 INFO:tasks.workunit.client.0.smithi028.stdout:5/136: chown d0/c8 115 1 2024-07-03T15:43:59.957 INFO:tasks.workunit.client.0.smithi028.stdout:5/137: chown d0/f9 56580 1 2024-07-03T15:43:59.958 INFO:tasks.workunit.client.1.smithi148.stdout:1/800: symlink d0/d2/dd0/l10a 0 2024-07-03T15:43:59.958 INFO:tasks.workunit.client.0.smithi028.stdout:1/199: mkdir d4/de/d1c/d55 0 2024-07-03T15:43:59.961 INFO:tasks.workunit.client.0.smithi028.stdout:5/138: link d0/f9 d0/f2c 0 2024-07-03T15:43:59.962 INFO:tasks.workunit.client.0.smithi028.stdout:2/82: dwrite f9 [0,4194304] 0 2024-07-03T15:43:59.962 INFO:tasks.workunit.client.0.smithi028.stdout:0/65: dwrite d2/d3/f11 [0,4194304] 0 2024-07-03T15:43:59.963 INFO:tasks.workunit.client.1.smithi148.stdout:7/905: dwrite d5/d7d/da7/d1c/d33/f94 [0,4194304] 0 2024-07-03T15:43:59.963 INFO:tasks.workunit.client.0.smithi028.stdout:1/200: mknod d4/d18/c56 0 2024-07-03T15:43:59.966 INFO:tasks.workunit.client.1.smithi148.stdout:5/969: dwrite d7/da/df/d16/d6a/db4/f11d [0,4194304] 0 2024-07-03T15:43:59.966 INFO:tasks.workunit.client.1.smithi148.stdout:5/970: write d7/da/f4c [4979802,123216] 0 2024-07-03T15:43:59.967 INFO:tasks.workunit.client.0.smithi028.stdout:5/139: symlink d0/dd/d13/d29/l2d 0 2024-07-03T15:43:59.970 INFO:tasks.workunit.client.1.smithi148.stdout:5/971: dread d7/da/df/d16/de4/da7/ff1 [0,4194304] 0 2024-07-03T15:43:59.971 INFO:tasks.workunit.client.1.smithi148.stdout:5/972: write d7/da/df/d16/d2f/f11e [15827,68926] 0 2024-07-03T15:43:59.971 INFO:tasks.workunit.client.1.smithi148.stdout:6/965: creat d3/d1d/d28/d53/f13a x:0 0 0 2024-07-03T15:43:59.973 INFO:tasks.workunit.client.1.smithi148.stdout:0/713: dwrite da/d17/d2a/d2f/fc1 [4194304,4194304] 0 2024-07-03T15:43:59.976 INFO:tasks.workunit.client.0.smithi028.stdout:0/66: creat d2/f15 x:0 0 0 2024-07-03T15:43:59.976 INFO:tasks.workunit.client.0.smithi028.stdout:0/67: chown d2/d3/fe 59639 1 2024-07-03T15:43:59.978 INFO:tasks.workunit.client.0.smithi028.stdout:2/83: dread f4 [4194304,4194304] 0 2024-07-03T15:43:59.978 INFO:tasks.workunit.client.0.smithi028.stdout:5/140: creat d0/dd/d13/d19/d26/f2e x:0 0 0 2024-07-03T15:43:59.983 INFO:tasks.workunit.client.0.smithi028.stdout:5/141: unlink d0/c5 0 2024-07-03T15:43:59.983 INFO:tasks.workunit.client.0.smithi028.stdout:1/201: dwrite d4/f1a [0,4194304] 0 2024-07-03T15:43:59.985 INFO:tasks.workunit.client.0.smithi028.stdout:2/84: rename da/f1c to da/f1d 0 2024-07-03T15:43:59.987 INFO:tasks.workunit.client.0.smithi028.stdout:5/142: link d0/c4 d0/dd/df/c2f 0 2024-07-03T15:43:59.989 INFO:tasks.workunit.client.0.smithi028.stdout:1/202: truncate d4/f29 8197022 0 2024-07-03T15:43:59.991 INFO:tasks.workunit.client.1.smithi148.stdout:5/973: symlink d7/da/df/d16/de4/da7/d116/l14d 0 2024-07-03T15:43:59.991 INFO:tasks.workunit.client.1.smithi148.stdout:5/974: creat d7/da/df/d16/dac/dc6/f14e x:0 0 0 2024-07-03T15:43:59.991 INFO:tasks.workunit.client.1.smithi148.stdout:5/975: creat d7/da/dbe/de6/f14f x:0 0 0 2024-07-03T15:43:59.992 INFO:tasks.workunit.client.0.smithi028.stdout:5/143: mkdir d0/dd/d13/d19/d30 0 2024-07-03T15:43:59.992 INFO:tasks.workunit.client.1.smithi148.stdout:6/966: link d3/d1d/d28/d31/d4c/fda d3/d88/f13b 0 2024-07-03T15:43:59.993 INFO:tasks.workunit.client.0.smithi028.stdout:1/203: mkdir d4/d11/d16/d57 0 2024-07-03T15:43:59.994 INFO:tasks.workunit.client.0.smithi028.stdout:1/204: write d4/de/f1f [1503405,79149] 0 2024-07-03T15:43:59.994 INFO:tasks.workunit.client.0.smithi028.stdout:1/205: dread - d4/de/f31 zero size 2024-07-03T15:43:59.995 INFO:tasks.workunit.client.1.smithi148.stdout:1/801: mkdir d0/d2/d10b 0 2024-07-03T15:43:59.995 INFO:tasks.workunit.client.1.smithi148.stdout:1/802: chown d0/d2/d6/d14/f58 1439 1 2024-07-03T15:43:59.995 INFO:tasks.workunit.client.1.smithi148.stdout:1/803: readlink d0/d2/d11/d1e/l27 0 2024-07-03T15:43:59.995 INFO:tasks.workunit.client.0.smithi028.stdout:5/144: mknod d0/dd/d13/d19/c31 0 2024-07-03T15:43:59.997 INFO:tasks.workunit.client.0.smithi028.stdout:1/206: unlink d4/d11/d16/c38 0 2024-07-03T15:43:59.998 INFO:tasks.workunit.client.0.smithi028.stdout:1/207: creat d4/de/d1c/f58 x:0 0 0 2024-07-03T15:43:59.998 INFO:tasks.workunit.client.0.smithi028.stdout:1/208: truncate d4/d11/f12 4249247 0 2024-07-03T15:43:59.999 INFO:tasks.workunit.client.0.smithi028.stdout:5/145: mknod d0/dd/d13/d19/d30/c32 0 2024-07-03T15:44:00.002 INFO:tasks.workunit.client.1.smithi148.stdout:0/714: creat da/d58/dbd/d4a/feb x:0 0 0 2024-07-03T15:44:00.003 INFO:tasks.workunit.client.0.smithi028.stdout:2/85: dread f7 [0,4194304] 0 2024-07-03T15:44:00.004 INFO:tasks.workunit.client.1.smithi148.stdout:7/906: getdents d5/d7d/da7/d1c/d33/d4c/d84/d98/dc2 0 2024-07-03T15:44:00.005 INFO:tasks.workunit.client.0.smithi028.stdout:5/146: symlink d0/dd/d13/d29/l33 0 2024-07-03T15:44:00.007 INFO:tasks.workunit.client.1.smithi148.stdout:5/976: dwrite d7/d13/d23/f85 [0,4194304] 0 2024-07-03T15:44:00.012 INFO:tasks.workunit.client.1.smithi148.stdout:6/967: link d3/d1d/d28/l30 d3/d1d/d28/d3e/ddd/d74/dc4/d85/d8c/d101/l13c 0 2024-07-03T15:44:00.012 INFO:tasks.workunit.client.1.smithi148.stdout:6/968: truncate d3/d1d/d28/d3e/f8b 3824375 0 2024-07-03T15:44:00.014 INFO:tasks.workunit.client.1.smithi148.stdout:1/804: unlink d0/d2/d11/d1e/d9d/dc2/de2/cd9 0 2024-07-03T15:44:00.014 INFO:tasks.workunit.client.0.smithi028.stdout:2/86: mkdir da/d1e 0 2024-07-03T15:44:00.017 INFO:tasks.workunit.client.1.smithi148.stdout:0/715: rename da/d58/dbd/d13/d14/d50/d95/d8c/f4d to da/d17/d2a/d2f/d93/da3/fec 0 2024-07-03T15:44:00.018 INFO:tasks.workunit.client.0.smithi028.stdout:1/209: rename d4/d18 to d4/de/d1e/d59 0 2024-07-03T15:44:00.020 INFO:tasks.workunit.client.1.smithi148.stdout:7/907: unlink d5/d7d/da7/f4d 0 2024-07-03T15:44:00.021 INFO:tasks.workunit.client.0.smithi028.stdout:2/87: mknod da/d1e/c1f 0 2024-07-03T15:44:00.021 INFO:tasks.workunit.client.0.smithi028.stdout:2/88: chown l8 2020 1 2024-07-03T15:44:00.024 INFO:tasks.workunit.client.1.smithi148.stdout:0/716: stat da/d58/dbd/d13/d1d/lca 0 2024-07-03T15:44:00.026 INFO:tasks.workunit.client.1.smithi148.stdout:7/908: creat d5/d7d/da7/d1c/d59/d112/f139 x:0 0 0 2024-07-03T15:44:00.028 INFO:tasks.workunit.client.1.smithi148.stdout:0/717: creat da/d58/fed x:0 0 0 2024-07-03T15:44:00.028 INFO:tasks.workunit.client.1.smithi148.stdout:0/718: read - da/d58/dbd/f9d zero size 2024-07-03T15:44:00.028 INFO:tasks.workunit.client.0.smithi028.stdout:5/147: dwrite d0/f2c [4194304,4194304] 0 2024-07-03T15:44:00.028 INFO:tasks.workunit.client.1.smithi148.stdout:2/853: sync 2024-07-03T15:44:00.029 INFO:tasks.workunit.client.1.smithi148.stdout:8/873: sync 2024-07-03T15:44:00.033 INFO:tasks.workunit.client.1.smithi148.stdout:7/909: mkdir d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/db2/d13a 0 2024-07-03T15:44:00.034 INFO:tasks.workunit.client.1.smithi148.stdout:0/719: creat da/d58/dbd/d13/d14/d50/fee x:0 0 0 2024-07-03T15:44:00.034 INFO:tasks.workunit.client.0.smithi028.stdout:1/210: write d4/fd [3283713,72199] 0 2024-07-03T15:44:00.035 INFO:tasks.workunit.client.0.smithi028.stdout:1/211: readlink d4/de/d1e/d59/l2f 0 2024-07-03T15:44:00.035 INFO:tasks.workunit.client.0.smithi028.stdout:1/212: creat d4/de/f5a x:0 0 0 2024-07-03T15:44:00.035 INFO:tasks.workunit.client.1.smithi148.stdout:2/854: creat d5/d67/dc3/ddd/f10f x:0 0 0 2024-07-03T15:44:00.036 INFO:tasks.workunit.client.1.smithi148.stdout:5/977: rename d7/d13/d54 to d7/da/df/da0/db2/d132/d10b/d150 0 2024-07-03T15:44:00.037 INFO:tasks.workunit.client.1.smithi148.stdout:5/978: chown d7/da/df/d16/d2f/f8b 36 1 2024-07-03T15:44:00.037 INFO:tasks.workunit.client.1.smithi148.stdout:5/979: truncate d7/da/df/d16/d2f/d44/f84 461394 0 2024-07-03T15:44:00.039 INFO:tasks.workunit.client.1.smithi148.stdout:0/720: symlink da/d58/dbd/d24/dc8/d8a/dab/lef 0 2024-07-03T15:44:00.041 INFO:tasks.workunit.client.1.smithi148.stdout:5/980: rename d7/da/df/d16/d2f/d44/f9a to d7/da/df/da0/db2/d132/d10b/f151 0 2024-07-03T15:44:00.043 INFO:tasks.workunit.client.1.smithi148.stdout:8/874: write d4/da/fdf [4168573,26035] 0 2024-07-03T15:44:00.044 INFO:tasks.workunit.client.0.smithi028.stdout:1/213: write d4/de/f10 [1937200,50098] 0 2024-07-03T15:44:00.045 INFO:tasks.workunit.client.1.smithi148.stdout:0/721: rename da/d58/dbd/d13/d1d/d3e/f5c to da/d58/dbd/d13/d14/dbe/ff0 0 2024-07-03T15:44:00.046 INFO:tasks.workunit.client.0.smithi028.stdout:1/214: mkdir d4/d2c/d3a/d43/d5b 0 2024-07-03T15:44:00.046 INFO:tasks.workunit.client.0.smithi028.stdout:1/215: readlink d4/de/d1e/d59/l24 0 2024-07-03T15:44:00.048 INFO:tasks.workunit.client.1.smithi148.stdout:8/875: mknod d4/da/d47/c12b 0 2024-07-03T15:44:00.049 INFO:tasks.workunit.client.1.smithi148.stdout:6/969: dwrite d3/d1d/d28/f73 [0,4194304] 0 2024-07-03T15:44:00.050 INFO:tasks.workunit.client.1.smithi148.stdout:0/722: truncate da/d17/d2a/f59 3856547 0 2024-07-03T15:44:00.050 INFO:tasks.workunit.client.1.smithi148.stdout:0/723: write da/d58/dbd/d24/d84/fe7 [588962,106480] 0 2024-07-03T15:44:00.051 INFO:tasks.workunit.client.0.smithi028.stdout:2/89: dwrite da/ff [0,4194304] 0 2024-07-03T15:44:00.052 INFO:tasks.workunit.client.0.smithi028.stdout:1/216: rename d4/d2c/d3a/d43/d5b to d4/d2c/d3a/d5c 0 2024-07-03T15:44:00.053 INFO:tasks.workunit.client.1.smithi148.stdout:6/970: symlink d3/d1d/d28/d53/dd4/d103/l13d 0 2024-07-03T15:44:00.053 INFO:tasks.workunit.client.1.smithi148.stdout:6/971: write d3/d1d/d28/d53/d6d/d8f/f127 [457349,133] 0 2024-07-03T15:44:00.056 INFO:tasks.workunit.client.0.smithi028.stdout:2/90: dread f7 [0,4194304] 0 2024-07-03T15:44:00.056 INFO:tasks.workunit.client.0.smithi028.stdout:2/91: chown da/l1b 280 1 2024-07-03T15:44:00.056 INFO:tasks.workunit.client.0.smithi028.stdout:2/92: readlink da/l1b 0 2024-07-03T15:44:00.056 INFO:tasks.workunit.client.0.smithi028.stdout:1/217: creat d4/de/d1e/f5d x:0 0 0 2024-07-03T15:44:00.056 INFO:tasks.workunit.client.0.smithi028.stdout:1/218: write d4/de/f52 [349543,95033] 0 2024-07-03T15:44:00.058 INFO:tasks.workunit.client.1.smithi148.stdout:2/855: dwrite d5/d14/d86/db9/dbb/d68/dcd/d64/db2/ff5 [0,4194304] 0 2024-07-03T15:44:00.058 INFO:tasks.workunit.client.1.smithi148.stdout:2/856: write d5/f1f [2796480,129639] 0 2024-07-03T15:44:00.059 INFO:tasks.workunit.client.1.smithi148.stdout:0/724: write da/d17/d2a/d2f/f87 [3735249,52309] 0 2024-07-03T15:44:00.059 INFO:tasks.workunit.client.1.smithi148.stdout:0/725: write da/d58/dbd/d4a/feb [75795,106875] 0 2024-07-03T15:44:00.059 INFO:tasks.workunit.client.1.smithi148.stdout:0/726: fdatasync da/d58/dbd/d24/dc8/d8a/dab/f72 0 2024-07-03T15:44:00.059 INFO:tasks.workunit.client.1.smithi148.stdout:0/727: read da/d17/f1e [4835901,95088] 0 2024-07-03T15:44:00.061 INFO:tasks.workunit.client.1.smithi148.stdout:6/972: rename d3/l13 to d3/d1d/d28/d31/d61/l13e 0 2024-07-03T15:44:00.061 INFO:tasks.workunit.client.1.smithi148.stdout:6/973: write d3/d1d/d28/d31/d4c/f54 [1231266,23836] 0 2024-07-03T15:44:00.061 INFO:tasks.workunit.client.1.smithi148.stdout:6/974: creat d3/d1d/d28/d53/f13f x:0 0 0 2024-07-03T15:44:00.061 INFO:tasks.workunit.client.1.smithi148.stdout:6/975: fdatasync d3/f56 0 2024-07-03T15:44:00.061 INFO:tasks.workunit.client.1.smithi148.stdout:6/976: write d3/d1d/d28/d3e/f64 [3469461,127071] 0 2024-07-03T15:44:00.062 INFO:tasks.workunit.client.0.smithi028.stdout:2/93: symlink da/l20 0 2024-07-03T15:44:00.062 INFO:tasks.workunit.client.1.smithi148.stdout:8/876: dread d4/da/d51/f8 [0,4194304] 0 2024-07-03T15:44:00.066 INFO:tasks.workunit.client.1.smithi148.stdout:5/981: dwrite d7/da/df/d16/dfb/f14b [4194304,4194304] 0 2024-07-03T15:44:00.066 INFO:tasks.workunit.client.1.smithi148.stdout:5/982: dread - d7/da/df/da0/db2/d132/d10b/d150/d119/f149 zero size 2024-07-03T15:44:00.067 INFO:tasks.workunit.client.1.smithi148.stdout:7/910: dwrite d5/d7d/da7/d1c/d4a/d66/d93/faf [0,4194304] 0 2024-07-03T15:44:00.077 INFO:tasks.workunit.client.1.smithi148.stdout:8/877: write d4/d86/f97 [1567931,93906] 0 2024-07-03T15:44:00.080 INFO:tasks.workunit.client.1.smithi148.stdout:7/911: dread d5/d7d/da7/d1c/d4a/d66/f10a [0,4194304] 0 2024-07-03T15:44:00.085 INFO:tasks.workunit.client.1.smithi148.stdout:5/983: write d7/da/d21/f5e [1767527,10507] 0 2024-07-03T15:44:00.085 INFO:tasks.workunit.client.1.smithi148.stdout:5/984: dread - d7/da/df/d16/de4/da7/f122 zero size 2024-07-03T15:44:00.086 INFO:tasks.workunit.client.1.smithi148.stdout:8/878: dread d4/da/d14/d17/deb/ffa [0,4194304] 0 2024-07-03T15:44:00.087 INFO:tasks.workunit.client.1.smithi148.stdout:5/985: dread d7/da/df/d16/dac/dc1/dc4/ffd [0,4194304] 0 2024-07-03T15:44:00.087 INFO:tasks.workunit.client.0.smithi028.stdout:2/94: dwrite da/f10 [0,4194304] 0 2024-07-03T15:44:00.087 INFO:tasks.workunit.client.1.smithi148.stdout:7/912: dread d5/d7d/da7/d1c/d4a/f62 [0,4194304] 0 2024-07-03T15:44:00.089 INFO:tasks.workunit.client.1.smithi148.stdout:2/857: dwrite d5/d14/d86/db9/dbb/d68/f84 [0,4194304] 0 2024-07-03T15:44:00.120 INFO:tasks.workunit.client.1.smithi148.stdout:0/728: getdents da/d58/dbd 0 2024-07-03T15:44:00.120 INFO:tasks.workunit.client.1.smithi148.stdout:0/729: read - da/d58/dbd/d13/fd9 zero size 2024-07-03T15:44:00.133 INFO:tasks.workunit.client.1.smithi148.stdout:4/819: sync 2024-07-03T15:44:00.133 INFO:tasks.workunit.client.1.smithi148.stdout:3/890: sync 2024-07-03T15:44:00.134 INFO:tasks.workunit.client.1.smithi148.stdout:3/891: readlink dd/d35/de1/dff/l95 0 2024-07-03T15:44:00.134 INFO:tasks.workunit.client.1.smithi148.stdout:3/892: chown dd/d33/d37/d68/c24 144381 1 2024-07-03T15:44:00.149 INFO:tasks.workunit.client.1.smithi148.stdout:3/893: dwrite dd/d33/f6c [0,4194304] 0 2024-07-03T15:44:00.150 INFO:tasks.workunit.client.1.smithi148.stdout:3/894: write dd/d33/d37/d68/d2e/d40/d45/f108 [405925,68113] 0 2024-07-03T15:44:00.188 INFO:tasks.workunit.client.1.smithi148.stdout:7/913: unlink d5/d7d/da7/d1c/d23/fbb 0 2024-07-03T15:44:00.189 INFO:tasks.workunit.client.0.smithi028.stdout:2/95: getdents da 0 2024-07-03T15:44:00.195 INFO:tasks.workunit.client.1.smithi148.stdout:2/858: creat d5/d14/d86/db9/f110 x:0 0 0 2024-07-03T15:44:00.195 INFO:tasks.workunit.client.1.smithi148.stdout:2/859: read - d5/d14/d22/dc0/dfb/d79/d9c/ffc zero size 2024-07-03T15:44:00.227 INFO:tasks.workunit.client.1.smithi148.stdout:4/820: creat d0/d91/da8/dab/d49/f122 x:0 0 0 2024-07-03T15:44:00.229 INFO:tasks.workunit.client.1.smithi148.stdout:3/895: link dd/d33/d37/d68/l2f dd/d35/de1/l122 0 2024-07-03T15:44:00.232 INFO:tasks.workunit.client.1.smithi148.stdout:5/986: mkdir d7/da/df/d16/dac/d152 0 2024-07-03T15:44:00.241 INFO:tasks.workunit.client.1.smithi148.stdout:7/914: unlink d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/db6/fe7 0 2024-07-03T15:44:00.243 INFO:tasks.workunit.client.1.smithi148.stdout:2/860: mkdir d5/d14/d22/dc0/dfb/d79/d111 0 2024-07-03T15:44:00.243 INFO:tasks.workunit.client.1.smithi148.stdout:2/861: read d5/d14/d86/db9/dbb/d68/dcd/f3d [3471609,37548] 0 2024-07-03T15:44:00.245 INFO:tasks.workunit.client.1.smithi148.stdout:0/730: unlink da/d17/f47 0 2024-07-03T15:44:00.246 INFO:tasks.workunit.client.1.smithi148.stdout:4/821: symlink d0/d8/d3a/db8/dd3/l123 0 2024-07-03T15:44:00.249 INFO:tasks.workunit.client.1.smithi148.stdout:3/896: creat dd/d33/d37/d68/d2e/d40/d77/d8f/ddc/f123 x:0 0 0 2024-07-03T15:44:00.249 INFO:tasks.workunit.client.1.smithi148.stdout:3/897: readlink dd/d33/d37/d8d/daf/l107 0 2024-07-03T15:44:00.249 INFO:tasks.workunit.client.1.smithi148.stdout:3/898: dread - dd/d33/d37/d68/d2e/d61/ffb zero size 2024-07-03T15:44:00.251 INFO:tasks.workunit.client.1.smithi148.stdout:7/915: symlink d5/d7d/da7/d1c/d33/d4c/d21/d31/dfc/l13b 0 2024-07-03T15:44:00.254 INFO:tasks.workunit.client.1.smithi148.stdout:3/899: symlink dd/d35/d41/l124 0 2024-07-03T15:44:00.257 INFO:tasks.workunit.client.1.smithi148.stdout:7/916: symlink d5/d7d/da7/d1c/d4a/l13c 0 2024-07-03T15:44:00.259 INFO:tasks.workunit.client.1.smithi148.stdout:0/731: dread da/d58/f79 [4194304,4194304] 0 2024-07-03T15:44:00.259 INFO:tasks.workunit.client.1.smithi148.stdout:3/900: truncate dd/d33/d37/d68/d2e/d40/d45/d65/f7b 173579 0 2024-07-03T15:44:00.261 INFO:tasks.workunit.client.1.smithi148.stdout:4/822: dwrite d0/d8/d13/d48/f9a [4194304,4194304] 0 2024-07-03T15:44:00.261 INFO:tasks.workunit.client.1.smithi148.stdout:0/732: creat da/d58/dbd/d94/ff1 x:0 0 0 2024-07-03T15:44:00.261 INFO:tasks.workunit.client.1.smithi148.stdout:0/733: truncate da/d17/f8b 1327850 0 2024-07-03T15:44:00.264 INFO:tasks.workunit.client.1.smithi148.stdout:2/862: dwrite d5/d14/d22/dc0/fda [0,4194304] 0 2024-07-03T15:44:00.264 INFO:tasks.workunit.client.1.smithi148.stdout:2/863: stat d5/d14/d86/db9/dbb/d68/dcd/f3d 0 2024-07-03T15:44:00.264 INFO:tasks.workunit.client.1.smithi148.stdout:2/864: truncate d5/d14/d86/db9/dbb/d68/dcd/d64/fd4 523048 0 2024-07-03T15:44:00.265 INFO:tasks.workunit.client.1.smithi148.stdout:3/901: creat dd/d33/d52/d57/d121/da2/f125 x:0 0 0 2024-07-03T15:44:00.270 INFO:tasks.workunit.client.0.smithi028.stdout:8/107: sync 2024-07-03T15:44:00.270 INFO:tasks.workunit.client.0.smithi028.stdout:8/108: fsync d0/f2a 0 2024-07-03T15:44:00.271 INFO:tasks.workunit.client.0.smithi028.stdout:8/109: write d0/d2/f1a [735350,95573] 0 2024-07-03T15:44:00.271 INFO:tasks.workunit.client.0.smithi028.stdout:8/110: write d0/d2/f1a [847757,25824] 0 2024-07-03T15:44:00.271 INFO:tasks.workunit.client.0.smithi028.stdout:8/111: dread - d0/d2/d3/dc/f21 zero size 2024-07-03T15:44:00.271 INFO:tasks.workunit.client.0.smithi028.stdout:9/117: sync 2024-07-03T15:44:00.271 INFO:tasks.workunit.client.1.smithi148.stdout:0/734: write da/d58/dbd/d13/d14/d50/d95/f97 [2486579,101423] 0 2024-07-03T15:44:00.272 INFO:tasks.workunit.client.1.smithi148.stdout:3/902: creat dd/d33/d52/d57/f126 x:0 0 0 2024-07-03T15:44:00.274 INFO:tasks.workunit.client.0.smithi028.stdout:8/112: link d0/l26 d0/d2/d3/dc/d11/l2c 0 2024-07-03T15:44:00.275 INFO:tasks.workunit.client.0.smithi028.stdout:9/118: symlink d1/dd/l23 0 2024-07-03T15:44:00.277 INFO:tasks.workunit.client.1.smithi148.stdout:6/977: rename d3/d1d/d28/d3e/ddd/d74 to d3/d88/d140 0 2024-07-03T15:44:00.280 INFO:tasks.workunit.client.0.smithi028.stdout:8/113: mknod d0/d2/d3/dc/d11/c2d 0 2024-07-03T15:44:00.280 INFO:tasks.workunit.client.1.smithi148.stdout:2/865: dwrite d5/d14/d86/db9/dbb/d13/d2d/f51 [0,4194304] 0 2024-07-03T15:44:00.281 INFO:tasks.workunit.client.1.smithi148.stdout:2/866: fdatasync d5/d14/d86/db9/dbb/d13/f29 0 2024-07-03T15:44:00.281 INFO:tasks.workunit.client.1.smithi148.stdout:3/903: truncate dd/d33/d37/f53 2841623 0 2024-07-03T15:44:00.283 INFO:tasks.workunit.client.1.smithi148.stdout:8/879: rename d4/da/d51/d18/c43 to d4/da/d14/c12c 0 2024-07-03T15:44:00.283 INFO:tasks.workunit.client.1.smithi148.stdout:8/880: creat d4/da/d14/d1a/d32/d38/d58/dcb/f12d x:0 0 0 2024-07-03T15:44:00.284 INFO:tasks.workunit.client.1.smithi148.stdout:0/735: getdents da/d58/dbd/d24/d84 0 2024-07-03T15:44:00.287 INFO:tasks.workunit.client.1.smithi148.stdout:2/867: creat d5/d14/d22/dc0/d101/f112 x:0 0 0 2024-07-03T15:44:00.291 INFO:tasks.workunit.client.1.smithi148.stdout:3/904: unlink dd/d33/d37/d68/d2e/d40/l71 0 2024-07-03T15:44:00.291 INFO:tasks.workunit.client.1.smithi148.stdout:3/905: chown dd/d35/de1 11 1 2024-07-03T15:44:00.293 INFO:tasks.workunit.client.1.smithi148.stdout:5/987: rename d7/da/df/d16/de4/da7/c6b to d7/da/df/d16/dac/dc6/c153 0 2024-07-03T15:44:00.295 INFO:tasks.workunit.client.1.smithi148.stdout:6/978: dwrite d3/d1d/d28/d31/d4c/f115 [0,4194304] 0 2024-07-03T15:44:00.297 INFO:tasks.workunit.client.1.smithi148.stdout:5/988: dread d7/da/df/d16/d6a/fa1 [0,4194304] 0 2024-07-03T15:44:00.297 INFO:tasks.workunit.client.1.smithi148.stdout:5/989: write d7/da/df/da0/db2/d132/d10b/d150/dd2/f112 [772945,32729] 0 2024-07-03T15:44:00.297 INFO:tasks.workunit.client.1.smithi148.stdout:8/881: dread d4/da/d14/f8d [0,4194304] 0 2024-07-03T15:44:00.298 INFO:tasks.workunit.client.1.smithi148.stdout:8/882: write d4/da/f29 [1013988,23901] 0 2024-07-03T15:44:00.298 INFO:tasks.workunit.client.1.smithi148.stdout:8/883: readlink d4/da/d14/d1a/d32/d38/d58/l7b 0 2024-07-03T15:44:00.300 INFO:tasks.workunit.client.1.smithi148.stdout:5/990: creat d7/da/df/da0/f154 x:0 0 0 2024-07-03T15:44:00.301 INFO:tasks.workunit.client.0.smithi028.stdout:8/114: dwrite d0/dd/d22/f25 [0,4194304] 0 2024-07-03T15:44:00.301 INFO:tasks.workunit.client.0.smithi028.stdout:8/115: chown d0/d2/d3/dc/d11/c2d 55191266 1 2024-07-03T15:44:00.301 INFO:tasks.workunit.client.0.smithi028.stdout:8/116: readlink d0/d2/d3/dc/d11/l2c 0 2024-07-03T15:44:00.307 INFO:tasks.workunit.client.1.smithi148.stdout:5/991: unlink f3 0 2024-07-03T15:44:00.307 INFO:tasks.workunit.client.1.smithi148.stdout:5/992: creat d7/da/df/d16/dfb/f155 x:0 0 0 2024-07-03T15:44:00.307 INFO:tasks.workunit.client.1.smithi148.stdout:5/993: getdents d7/da/df/da0/dd1 0 2024-07-03T15:44:00.307 INFO:tasks.workunit.client.1.smithi148.stdout:5/994: fdatasync d7/da/df/d16/d6a/db4/f13f 0 2024-07-03T15:44:00.317 INFO:tasks.workunit.client.1.smithi148.stdout:6/979: dwrite d3/d1d/d28/d31/d61/faf [0,4194304] 0 2024-07-03T15:44:00.321 INFO:tasks.workunit.client.1.smithi148.stdout:6/980: mkdir d3/d1d/d28/d53/dd4/d141 0 2024-07-03T15:44:00.322 INFO:tasks.workunit.client.1.smithi148.stdout:6/981: chown d3/d50/cfc 357 1 2024-07-03T15:44:00.322 INFO:tasks.workunit.client.1.smithi148.stdout:6/982: chown d3/d1d/d28/d31/d61/c11d 2745702 1 2024-07-03T15:44:00.322 INFO:tasks.workunit.client.1.smithi148.stdout:6/983: symlink d3/d1d/d28/d31/d61/d102/l142 0 2024-07-03T15:44:00.324 INFO:tasks.workunit.client.1.smithi148.stdout:6/984: rmdir d3/d1d/dd6/d10a 39 2024-07-03T15:44:00.324 INFO:tasks.workunit.client.1.smithi148.stdout:6/985: write d3/d1d/d28/d53/dd4/d103/f11e [149217,84045] 0 2024-07-03T15:44:00.324 INFO:tasks.workunit.client.1.smithi148.stdout:6/986: readlink d3/d88/d140/d12f/l137 0 2024-07-03T15:44:00.324 INFO:tasks.workunit.client.1.smithi148.stdout:6/987: readlink d3/d1d/d28/d3e/l87 0 2024-07-03T15:44:00.324 INFO:tasks.workunit.client.1.smithi148.stdout:6/988: symlink d3/d1d/d28/d31/d4c/l143 0 2024-07-03T15:44:00.327 INFO:tasks.workunit.client.1.smithi148.stdout:6/989: link d3/d88/d140/dc4/d85/fee d3/d88/d140/dc4/d85/d8c/d101/f144 0 2024-07-03T15:44:00.335 INFO:tasks.workunit.client.1.smithi148.stdout:5/995: dwrite d7/da/df/da0/db2/d132/d10b/d150/fd6 [0,4194304] 0 2024-07-03T15:44:00.336 INFO:tasks.workunit.client.1.smithi148.stdout:5/996: chown d7/d59 488844462 1 2024-07-03T15:44:00.339 INFO:tasks.workunit.client.1.smithi148.stdout:5/997: unlink d7/da/df/d1d/d109/d126/c13b 0 2024-07-03T15:44:00.339 INFO:tasks.workunit.client.1.smithi148.stdout:5/998: read - d7/da/df/d16/dfb/f12f zero size 2024-07-03T15:44:00.339 INFO:tasks.workunit.client.1.smithi148.stdout:5/999: chown d7/da/df/da0/db2/d132/f82 6933 1 2024-07-03T15:44:00.376 INFO:tasks.workunit.client.1.smithi148.stdout:3/906: rmdir dd/d33/d52/d57 39 2024-07-03T15:44:00.379 INFO:tasks.workunit.client.1.smithi148.stdout:1/805: sync 2024-07-03T15:44:00.379 INFO:tasks.workunit.client.1.smithi148.stdout:1/806: write d0/d2/d11/d1e/f3c [2035781,65047] 0 2024-07-03T15:44:00.379 INFO:tasks.workunit.client.1.smithi148.stdout:1/807: write d0/d2/d11/d1e/d9d/dc2/de2/fcd [358169,36779] 0 2024-07-03T15:44:00.386 INFO:tasks.workunit.client.1.smithi148.stdout:8/884: rmdir d4/da 39 2024-07-03T15:44:00.391 INFO:tasks.workunit.client.0.smithi028.stdout:8/117: mkdir d0/d2e 0 2024-07-03T15:44:00.393 INFO:tasks.workunit.client.0.smithi028.stdout:8/118: dread d0/d2/f27 [0,4194304] 0 2024-07-03T15:44:00.394 INFO:tasks.workunit.client.1.smithi148.stdout:7/917: rename d5/d7d/da7/d1c/d33/d4c/d21/l129 to d5/l13d 0 2024-07-03T15:44:00.394 INFO:tasks.workunit.client.1.smithi148.stdout:7/918: write d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/db1/f10b [180348,5026] 0 2024-07-03T15:44:00.394 INFO:tasks.workunit.client.1.smithi148.stdout:7/919: chown d5/d7d 2194 1 2024-07-03T15:44:00.396 INFO:tasks.workunit.client.1.smithi148.stdout:3/907: dwrite dd/f22 [0,4194304] 0 2024-07-03T15:44:00.396 INFO:tasks.workunit.client.1.smithi148.stdout:3/908: stat dd/d33/d37/d68/d2e/c11c 0 2024-07-03T15:44:00.396 INFO:tasks.workunit.client.1.smithi148.stdout:3/909: readlink dd/l96 0 2024-07-03T15:44:00.396 INFO:tasks.workunit.client.1.smithi148.stdout:1/808: dwrite d0/d2/d6/fa [4194304,4194304] 0 2024-07-03T15:44:00.397 INFO:tasks.workunit.client.0.smithi028.stdout:2/96: unlink f7 0 2024-07-03T15:44:00.398 INFO:tasks.workunit.client.1.smithi148.stdout:4/823: rename d0/d8/dd4/ff6 to d0/d28/da7/d2f/d41/f124 0 2024-07-03T15:44:00.398 INFO:tasks.workunit.client.1.smithi148.stdout:4/824: write d0/d28/f121 [513203,113554] 0 2024-07-03T15:44:00.398 INFO:tasks.workunit.client.1.smithi148.stdout:4/825: write d0/fbf [413408,98998] 0 2024-07-03T15:44:00.398 INFO:tasks.workunit.client.1.smithi148.stdout:1/809: dread d0/d2/d6/f5a [0,4194304] 0 2024-07-03T15:44:00.399 INFO:tasks.workunit.client.0.smithi028.stdout:8/119: dread d0/d2/f1a [0,4194304] 0 2024-07-03T15:44:00.399 INFO:tasks.workunit.client.0.smithi028.stdout:8/120: fdatasync d0/d2/f1a 0 2024-07-03T15:44:00.400 INFO:tasks.workunit.client.1.smithi148.stdout:8/885: write d4/da/d14/f91 [1465557,98878] 0 2024-07-03T15:44:00.400 INFO:tasks.workunit.client.1.smithi148.stdout:8/886: dread - d4/da/d14/d1a/d32/f101 zero size 2024-07-03T15:44:00.400 INFO:tasks.workunit.client.0.smithi028.stdout:2/97: creat da/d1e/f21 x:0 0 0 2024-07-03T15:44:00.401 INFO:tasks.workunit.client.1.smithi148.stdout:3/910: creat dd/d33/d37/d68/d2e/d40/d77/dcd/d102/f127 x:0 0 0 2024-07-03T15:44:00.403 INFO:tasks.workunit.client.0.smithi028.stdout:8/121: rename d0/d2/d3/fa to d0/d2/d3/dc/d11/d12/f2f 0 2024-07-03T15:44:00.405 INFO:tasks.workunit.client.0.smithi028.stdout:2/98: rename da/f16 to da/f22 0 2024-07-03T15:44:00.407 INFO:tasks.workunit.client.1.smithi148.stdout:0/736: rename da/d58/dbd/d13/d1d/d3e/le6 to da/d58/dbd/d13/d14/d50/de4/lf2 0 2024-07-03T15:44:00.408 INFO:tasks.workunit.client.0.smithi028.stdout:8/122: creat d0/d2/d3/dc/d11/f30 x:0 0 0 2024-07-03T15:44:00.408 INFO:tasks.workunit.client.1.smithi148.stdout:7/920: mkdir d5/d7d/da7/d13e 0 2024-07-03T15:44:00.409 INFO:tasks.workunit.client.1.smithi148.stdout:7/921: dread - d5/d7d/da7/d1c/d33/d4c/d84/d91/daa/f128 zero size 2024-07-03T15:44:00.409 INFO:tasks.workunit.client.1.smithi148.stdout:4/826: mknod d0/d28/d6e/c125 0 2024-07-03T15:44:00.409 INFO:tasks.workunit.client.1.smithi148.stdout:4/827: fdatasync d0/d28/d7d/f8a 0 2024-07-03T15:44:00.410 INFO:tasks.workunit.client.1.smithi148.stdout:1/810: truncate d0/d2/d11/d1e/d9d/dc2/de2/f7a 2265687 0 2024-07-03T15:44:00.412 INFO:tasks.workunit.client.0.smithi028.stdout:8/123: mknod d0/d2/d3/dc/d11/d12/c31 0 2024-07-03T15:44:00.415 INFO:tasks.workunit.client.0.smithi028.stdout:8/124: getdents d0/dd 0 2024-07-03T15:44:00.415 INFO:tasks.workunit.client.0.smithi028.stdout:2/99: read da/ff [2685312,121858] 0 2024-07-03T15:44:00.417 INFO:tasks.workunit.client.0.smithi028.stdout:8/125: creat d0/d2e/f32 x:0 0 0 2024-07-03T15:44:00.418 INFO:tasks.workunit.client.1.smithi148.stdout:7/922: dread d5/d7d/da7/db4/f115 [0,4194304] 0 2024-07-03T15:44:00.419 INFO:tasks.workunit.client.0.smithi028.stdout:2/100: mkdir da/d23 0 2024-07-03T15:44:00.420 INFO:tasks.workunit.client.1.smithi148.stdout:3/911: creat dd/d33/d52/d57/d121/da2/dbb/f128 x:0 0 0 2024-07-03T15:44:00.422 INFO:tasks.workunit.client.1.smithi148.stdout:2/868: mkdir d5/d14/d86/db9/dbb/d113 0 2024-07-03T15:44:00.424 INFO:tasks.workunit.client.1.smithi148.stdout:7/923: write d5/d7d/da7/d1c/d33/d4c/d21/d39/f77 [1038162,66910] 0 2024-07-03T15:44:00.427 INFO:tasks.workunit.client.1.smithi148.stdout:0/737: creat da/d58/dbd/d24/d84/ff3 x:0 0 0 2024-07-03T15:44:00.428 INFO:tasks.workunit.client.1.smithi148.stdout:1/811: dwrite d0/de/d2f/d72/df9/d42/f48 [8388608,4194304] 0 2024-07-03T15:44:00.428 INFO:tasks.workunit.client.1.smithi148.stdout:1/812: dread - d0/d2/d6/d14/f109 zero size 2024-07-03T15:44:00.429 INFO:tasks.workunit.client.1.smithi148.stdout:1/813: dread - d0/d2/dd0/ffe zero size 2024-07-03T15:44:00.431 INFO:tasks.workunit.client.1.smithi148.stdout:2/869: mknod d5/d14/d86/db9/dbb/d13/d1d/da1/c114 0 2024-07-03T15:44:00.435 INFO:tasks.workunit.client.1.smithi148.stdout:7/924: truncate d5/d7d/da7/f34 326476 0 2024-07-03T15:44:00.435 INFO:tasks.workunit.client.0.smithi028.stdout:2/101: dwrite f4 [8388608,4194304] 0 2024-07-03T15:44:00.436 INFO:tasks.workunit.client.1.smithi148.stdout:1/814: creat d0/f10c x:0 0 0 2024-07-03T15:44:00.436 INFO:tasks.workunit.client.1.smithi148.stdout:1/815: dread - d0/de/d2f/d72/df9/d42/d62/f9c zero size 2024-07-03T15:44:00.437 INFO:tasks.workunit.client.1.smithi148.stdout:4/828: getdents d0/d8/d3a/db8/dd3/df2 0 2024-07-03T15:44:00.438 INFO:tasks.workunit.client.0.smithi028.stdout:2/102: mkdir da/d24 0 2024-07-03T15:44:00.439 INFO:tasks.workunit.client.1.smithi148.stdout:0/738: dread da/d58/dbd/d13/d14/f61 [0,4194304] 0 2024-07-03T15:44:00.439 INFO:tasks.workunit.client.1.smithi148.stdout:0/739: dread - da/d58/fed zero size 2024-07-03T15:44:00.439 INFO:tasks.workunit.client.1.smithi148.stdout:0/740: truncate da/d58/dbd/d24/dc8/d8a/dab/f72 4871846 0 2024-07-03T15:44:00.440 INFO:tasks.workunit.client.1.smithi148.stdout:3/912: link dd/d33/d37/f83 dd/d33/d37/d68/d2e/d40/d45/d65/f129 0 2024-07-03T15:44:00.441 INFO:tasks.workunit.client.1.smithi148.stdout:1/816: mkdir d0/de/d44/d54/d5d/d10d 0 2024-07-03T15:44:00.445 INFO:tasks.workunit.client.0.smithi028.stdout:2/103: rename da/l20 to da/l25 0 2024-07-03T15:44:00.446 INFO:tasks.workunit.client.0.smithi028.stdout:2/104: write da/f15 [83617,5235] 0 2024-07-03T15:44:00.446 INFO:tasks.workunit.client.0.smithi028.stdout:2/105: readlink da/lb 0 2024-07-03T15:44:00.446 INFO:tasks.workunit.client.0.smithi028.stdout:2/106: chown da/f1d 47665530 1 2024-07-03T15:44:00.447 INFO:tasks.workunit.client.1.smithi148.stdout:4/829: link d0/f4 d0/d8/d13/d48/f126 0 2024-07-03T15:44:00.448 INFO:tasks.workunit.client.1.smithi148.stdout:3/913: symlink dd/d33/d52/d57/l12a 0 2024-07-03T15:44:00.448 INFO:tasks.workunit.client.1.smithi148.stdout:3/914: readlink dd/d33/d37/d68/le0 0 2024-07-03T15:44:00.449 INFO:tasks.workunit.client.1.smithi148.stdout:0/741: dread da/d58/dbd/f26 [0,4194304] 0 2024-07-03T15:44:00.449 INFO:tasks.workunit.client.0.smithi028.stdout:2/107: link da/f17 da/f26 0 2024-07-03T15:44:00.451 INFO:tasks.workunit.client.1.smithi148.stdout:0/742: symlink da/d58/dbd/d13/d1d/d3e/d85/d8e/lf4 0 2024-07-03T15:44:00.452 INFO:tasks.workunit.client.0.smithi028.stdout:2/108: symlink da/d23/l27 0 2024-07-03T15:44:00.452 INFO:tasks.workunit.client.0.smithi028.stdout:2/109: read - da/f26 zero size 2024-07-03T15:44:00.452 INFO:tasks.workunit.client.0.smithi028.stdout:2/110: dread - da/f22 zero size 2024-07-03T15:44:00.452 INFO:tasks.workunit.client.0.smithi028.stdout:2/111: read - da/f26 zero size 2024-07-03T15:44:00.452 INFO:tasks.workunit.client.0.smithi028.stdout:2/112: chown da/ff 11082 1 2024-07-03T15:44:00.453 INFO:tasks.workunit.client.0.smithi028.stdout:2/113: mkdir da/d23/d28 0 2024-07-03T15:44:00.453 INFO:tasks.workunit.client.0.smithi028.stdout:2/114: write da/ff [2047857,31037] 0 2024-07-03T15:44:00.454 INFO:tasks.workunit.client.0.smithi028.stdout:2/115: write da/d1e/f21 [345760,114185] 0 2024-07-03T15:44:00.454 INFO:tasks.workunit.client.0.smithi028.stdout:2/116: chown da/f12 1154840004 1 2024-07-03T15:44:00.458 INFO:tasks.workunit.client.1.smithi148.stdout:7/925: dwrite d5/d7d/da7/d1c/d33/f137 [0,4194304] 0 2024-07-03T15:44:00.458 INFO:tasks.workunit.client.0.smithi028.stdout:2/117: mknod da/d23/d28/c29 0 2024-07-03T15:44:00.458 INFO:tasks.workunit.client.0.smithi028.stdout:2/118: chown da/f12 79649233 1 2024-07-03T15:44:00.463 INFO:tasks.workunit.client.0.smithi028.stdout:3/134: sync 2024-07-03T15:44:00.463 INFO:tasks.workunit.client.0.smithi028.stdout:4/73: sync 2024-07-03T15:44:00.463 INFO:tasks.workunit.client.0.smithi028.stdout:0/68: sync 2024-07-03T15:44:00.463 INFO:tasks.workunit.client.0.smithi028.stdout:0/69: dread - d2/f15 zero size 2024-07-03T15:44:00.463 INFO:tasks.workunit.client.0.smithi028.stdout:7/59: sync 2024-07-03T15:44:00.463 INFO:tasks.workunit.client.0.smithi028.stdout:0/70: readlink d2/d3/l13 0 2024-07-03T15:44:00.463 INFO:tasks.workunit.client.0.smithi028.stdout:7/60: write d9/f13 [450985,101415] 0 2024-07-03T15:44:00.463 INFO:tasks.workunit.client.0.smithi028.stdout:7/61: stat d9/ld 0 2024-07-03T15:44:00.463 INFO:tasks.workunit.client.0.smithi028.stdout:6/75: sync 2024-07-03T15:44:00.468 INFO:tasks.workunit.client.1.smithi148.stdout:7/926: symlink d5/d7d/da7/d1c/d33/d4c/d21/d39/l13f 0 2024-07-03T15:44:00.470 INFO:tasks.workunit.client.0.smithi028.stdout:2/119: symlink da/l2a 0 2024-07-03T15:44:00.472 INFO:tasks.workunit.client.0.smithi028.stdout:4/74: rename d2/d5/f11 to d2/f13 0 2024-07-03T15:44:00.472 INFO:tasks.workunit.client.0.smithi028.stdout:4/75: creat d2/f14 x:0 0 0 2024-07-03T15:44:00.473 INFO:tasks.workunit.client.1.smithi148.stdout:1/817: dwrite d0/d2/d11/d1e/d9d/fdb [0,4194304] 0 2024-07-03T15:44:00.473 INFO:tasks.workunit.client.1.smithi148.stdout:3/915: dwrite dd/d33/d37/d68/d2e/f11 [0,4194304] 0 2024-07-03T15:44:00.475 INFO:tasks.workunit.client.0.smithi028.stdout:3/135: mknod d18/d1a/c2f 0 2024-07-03T15:44:00.475 INFO:tasks.workunit.client.1.smithi148.stdout:4/830: read d0/d8/d3a/d9d/fd7 [3474792,63503] 0 2024-07-03T15:44:00.478 INFO:tasks.workunit.client.0.smithi028.stdout:8/126: rmdir d0/d2/d3/dc/d11/d12 39 2024-07-03T15:44:00.478 INFO:tasks.workunit.client.0.smithi028.stdout:8/127: readlink d0/l8 0 2024-07-03T15:44:00.478 INFO:tasks.workunit.client.0.smithi028.stdout:8/128: chown d0/l26 3 1 2024-07-03T15:44:00.478 INFO:tasks.workunit.client.0.smithi028.stdout:8/129: readlink d0/d2/d3/dc/d11/d12/l1b 0 2024-07-03T15:44:00.478 INFO:tasks.workunit.client.0.smithi028.stdout:8/130: chown d0/d2/d3 1793308406 1 2024-07-03T15:44:00.479 INFO:tasks.workunit.client.0.smithi028.stdout:8/131: fdatasync d0/d2/d3/dc/f21 0 2024-07-03T15:44:00.479 INFO:tasks.workunit.client.1.smithi148.stdout:3/916: unlink dd/d33/d37/d68/d2e/l27 0 2024-07-03T15:44:00.479 INFO:tasks.workunit.client.1.smithi148.stdout:3/917: chown dd/d33/d37/d68/d2e/d40/d45/d73/f9f 147455 1 2024-07-03T15:44:00.479 INFO:tasks.workunit.client.0.smithi028.stdout:0/71: mknod d2/d3/c16 0 2024-07-03T15:44:00.479 INFO:tasks.workunit.client.0.smithi028.stdout:0/72: readlink d2/d3/l13 0 2024-07-03T15:44:00.482 INFO:tasks.workunit.client.0.smithi028.stdout:3/136: write f17 [779889,20044] 0 2024-07-03T15:44:00.482 INFO:tasks.workunit.client.0.smithi028.stdout:7/62: symlink d9/l16 0 2024-07-03T15:44:00.486 INFO:tasks.workunit.client.1.smithi148.stdout:7/927: rmdir d5/d7d/da7/d1c/d33/d4c/d84/d98/d9e 39 2024-07-03T15:44:00.488 INFO:tasks.workunit.client.0.smithi028.stdout:4/76: mknod d2/c15 0 2024-07-03T15:44:00.489 INFO:tasks.workunit.client.0.smithi028.stdout:2/120: dwrite da/d1e/f21 [0,4194304] 0 2024-07-03T15:44:00.489 INFO:tasks.workunit.client.0.smithi028.stdout:2/121: fsync f4 0 2024-07-03T15:44:00.489 INFO:tasks.workunit.client.0.smithi028.stdout:2/122: dread - da/f1d zero size 2024-07-03T15:44:00.491 INFO:tasks.workunit.client.1.smithi148.stdout:4/831: rename d0/d8/d13/d48/c71 to d0/d8/d3a/dce/c127 0 2024-07-03T15:44:00.491 INFO:tasks.workunit.client.0.smithi028.stdout:8/132: mkdir d0/d2/d3/dc/d33 0 2024-07-03T15:44:00.492 INFO:tasks.workunit.client.1.smithi148.stdout:1/818: getdents d0/de/d89 0 2024-07-03T15:44:00.493 INFO:tasks.workunit.client.0.smithi028.stdout:0/73: mknod d2/d3/c17 0 2024-07-03T15:44:00.495 INFO:tasks.workunit.client.1.smithi148.stdout:7/928: rename d5/d7d/da7/d1c/d59/f109 to d5/d7d/da7/d1c/d33/d4c/d21/f140 0 2024-07-03T15:44:00.495 INFO:tasks.workunit.client.1.smithi148.stdout:7/929: write d5/d7d/da7/db4/f105 [145879,84001] 0 2024-07-03T15:44:00.496 INFO:tasks.workunit.client.0.smithi028.stdout:4/77: unlink d2/c15 0 2024-07-03T15:44:00.497 INFO:tasks.workunit.client.1.smithi148.stdout:1/819: symlink d0/d2/d11/l10e 0 2024-07-03T15:44:00.500 INFO:tasks.workunit.client.1.smithi148.stdout:7/930: mkdir d5/d7d/da7/d1c/d33/d4c/d84/d98/d141 0 2024-07-03T15:44:00.502 INFO:tasks.workunit.client.0.smithi028.stdout:2/123: unlink da/d1e/c1f 0 2024-07-03T15:44:00.503 INFO:tasks.workunit.client.1.smithi148.stdout:4/832: write d0/d28/da7/d2f/d41/f4d [642240,98124] 0 2024-07-03T15:44:00.507 INFO:tasks.workunit.client.0.smithi028.stdout:8/133: rename d0/d2e/f32 to d0/d2/d3/dc/f34 0 2024-07-03T15:44:00.507 INFO:tasks.workunit.client.0.smithi028.stdout:8/134: dread - d0/f2a zero size 2024-07-03T15:44:00.508 INFO:tasks.workunit.client.0.smithi028.stdout:6/76: dwrite d4/f18 [0,4194304] 0 2024-07-03T15:44:00.508 INFO:tasks.workunit.client.1.smithi148.stdout:7/931: rmdir d5/d7d/da7/d1c/d4a/d66/dd7 39 2024-07-03T15:44:00.509 INFO:tasks.workunit.client.1.smithi148.stdout:7/932: creat d5/d7d/da7/d1c/d33/d4c/d21/d39/dd2/f142 x:0 0 0 2024-07-03T15:44:00.509 INFO:tasks.workunit.client.1.smithi148.stdout:7/933: chown d5/d7d/da7/d1c/d101 1616 1 2024-07-03T15:44:00.510 INFO:tasks.workunit.client.0.smithi028.stdout:0/74: mknod d2/d3/c18 0 2024-07-03T15:44:00.511 INFO:tasks.workunit.client.1.smithi148.stdout:0/743: dwrite da/d58/dbd/d13/d14/d50/d95/f97 [0,4194304] 0 2024-07-03T15:44:00.513 INFO:tasks.workunit.client.0.smithi028.stdout:4/78: symlink d2/l16 0 2024-07-03T15:44:00.514 INFO:tasks.workunit.client.1.smithi148.stdout:3/918: dwrite dd/f15 [0,4194304] 0 2024-07-03T15:44:00.514 INFO:tasks.workunit.client.0.smithi028.stdout:3/137: dwrite d18/d1a/f24 [4194304,4194304] 0 2024-07-03T15:44:00.515 INFO:tasks.workunit.client.1.smithi148.stdout:4/833: creat d0/d28/d7d/d6f/da6/f128 x:0 0 0 2024-07-03T15:44:00.515 INFO:tasks.workunit.client.1.smithi148.stdout:4/834: truncate d0/d28/da7/ffd 1023911 0 2024-07-03T15:44:00.517 INFO:tasks.workunit.client.0.smithi028.stdout:6/77: write d4/f11 [1763230,4656] 0 2024-07-03T15:44:00.518 INFO:tasks.workunit.client.1.smithi148.stdout:1/820: rmdir d0/d2/d6/d14/d29/d94 39 2024-07-03T15:44:00.522 INFO:tasks.workunit.client.0.smithi028.stdout:2/124: dwrite da/f10 [0,4194304] 0 2024-07-03T15:44:00.522 INFO:tasks.workunit.client.0.smithi028.stdout:4/79: dread d2/fc [0,4194304] 0 2024-07-03T15:44:00.524 INFO:tasks.workunit.client.1.smithi148.stdout:7/934: mkdir d5/d7d/da7/d1c/d33/d4c/d84/d98/d143 0 2024-07-03T15:44:00.524 INFO:tasks.workunit.client.1.smithi148.stdout:7/935: fdatasync d5/d7d/da7/d1c/d4a/d52/d107/f89 0 2024-07-03T15:44:00.524 INFO:tasks.workunit.client.1.smithi148.stdout:7/936: chown d5/d7d/da7/c11c 1016920045 1 2024-07-03T15:44:00.524 INFO:tasks.workunit.client.1.smithi148.stdout:7/937: stat d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/c135 0 2024-07-03T15:44:00.525 INFO:tasks.workunit.client.0.smithi028.stdout:2/125: dread da/f12 [0,4194304] 0 2024-07-03T15:44:00.525 INFO:tasks.workunit.client.0.smithi028.stdout:0/75: rmdir d2/d3 39 2024-07-03T15:44:00.525 INFO:tasks.workunit.client.1.smithi148.stdout:4/835: write d0/d28/da7/d2f/d41/f7e [4046054,80936] 0 2024-07-03T15:44:00.527 INFO:tasks.workunit.client.0.smithi028.stdout:5/148: sync 2024-07-03T15:44:00.527 INFO:tasks.workunit.client.0.smithi028.stdout:1/219: sync 2024-07-03T15:44:00.527 INFO:tasks.workunit.client.0.smithi028.stdout:1/220: chown d4/d11/f12 1676 1 2024-07-03T15:44:00.534 INFO:tasks.workunit.client.1.smithi148.stdout:4/836: write d0/f6 [8201458,16413] 0 2024-07-03T15:44:00.536 INFO:tasks.workunit.client.0.smithi028.stdout:3/138: truncate f2 2603436 0 2024-07-03T15:44:00.536 INFO:tasks.workunit.client.0.smithi028.stdout:3/139: truncate d18/f1d 755523 0 2024-07-03T15:44:00.539 INFO:tasks.workunit.client.0.smithi028.stdout:6/78: creat d4/da/d13/f19 x:0 0 0 2024-07-03T15:44:00.539 INFO:tasks.workunit.client.0.smithi028.stdout:6/79: chown d4/f18 1 1 2024-07-03T15:44:00.542 INFO:tasks.workunit.client.0.smithi028.stdout:0/76: dread d2/d3/f11 [0,4194304] 0 2024-07-03T15:44:00.544 INFO:tasks.workunit.client.1.smithi148.stdout:1/821: rename d0/d2/d6/d14/d29/l2b to d0/de/d2f/d72/df9/d42/l10f 0 2024-07-03T15:44:00.544 INFO:tasks.workunit.client.1.smithi148.stdout:1/822: chown d0/de/l31 0 1 2024-07-03T15:44:00.544 INFO:tasks.workunit.client.0.smithi028.stdout:4/80: creat d2/d12/f17 x:0 0 0 2024-07-03T15:44:00.548 INFO:tasks.workunit.client.1.smithi148.stdout:7/938: mknod d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/c144 0 2024-07-03T15:44:00.548 INFO:tasks.workunit.client.0.smithi028.stdout:1/221: dread d4/f19 [0,4194304] 0 2024-07-03T15:44:00.549 INFO:tasks.workunit.client.1.smithi148.stdout:7/939: write d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/db6/fbe [971068,9218] 0 2024-07-03T15:44:00.549 INFO:tasks.workunit.client.0.smithi028.stdout:3/140: mknod d18/c30 0 2024-07-03T15:44:00.550 INFO:tasks.workunit.client.1.smithi148.stdout:4/837: getdents d0/d8/d44/dc2/d107 0 2024-07-03T15:44:00.550 INFO:tasks.workunit.client.1.smithi148.stdout:4/838: dread - d0/d8/d13/d48/f81 zero size 2024-07-03T15:44:00.552 INFO:tasks.workunit.client.0.smithi028.stdout:0/77: getdents d2/d3 0 2024-07-03T15:44:00.553 INFO:tasks.workunit.client.0.smithi028.stdout:6/80: dread d4/f18 [0,4194304] 0 2024-07-03T15:44:00.555 INFO:tasks.workunit.client.0.smithi028.stdout:4/81: rename d2/lf to d2/l18 0 2024-07-03T15:44:00.556 INFO:tasks.workunit.client.1.smithi148.stdout:1/823: truncate d0/d2/d11/d1e/d9d/dc2/de2/f69 11471842 0 2024-07-03T15:44:00.558 INFO:tasks.workunit.client.0.smithi028.stdout:3/141: mkdir d18/d31 0 2024-07-03T15:44:00.558 INFO:tasks.workunit.client.0.smithi028.stdout:3/142: truncate d18/f1d 917438 0 2024-07-03T15:44:00.560 INFO:tasks.workunit.client.0.smithi028.stdout:8/135: rename d0/d2e to d0/d2/d3/d35 0 2024-07-03T15:44:00.560 INFO:tasks.workunit.client.0.smithi028.stdout:8/136: truncate d0/d2/d3/dc/d11/d12/f15 4814222 0 2024-07-03T15:44:00.560 INFO:tasks.workunit.client.0.smithi028.stdout:8/137: truncate d0/d2/d3/dc/d11/f30 38936 0 2024-07-03T15:44:00.560 INFO:tasks.workunit.client.0.smithi028.stdout:8/138: fsync d0/d2/d3/dc/f1f 0 2024-07-03T15:44:00.560 INFO:tasks.workunit.client.0.smithi028.stdout:0/78: creat d2/d3/f19 x:0 0 0 2024-07-03T15:44:00.561 INFO:tasks.workunit.client.0.smithi028.stdout:6/81: getdents d4/d15 0 2024-07-03T15:44:00.561 INFO:tasks.workunit.client.0.smithi028.stdout:6/82: write f2 [515568,23283] 0 2024-07-03T15:44:00.563 INFO:tasks.workunit.client.0.smithi028.stdout:3/143: stat l12 0 2024-07-03T15:44:00.563 INFO:tasks.workunit.client.0.smithi028.stdout:3/144: truncate d18/d1a/f2e 690349 0 2024-07-03T15:44:00.567 INFO:tasks.workunit.client.0.smithi028.stdout:2/126: rename da/d23/l27 to da/d23/d28/l2b 0 2024-07-03T15:44:00.569 INFO:tasks.workunit.client.0.smithi028.stdout:1/222: dread f2 [0,4194304] 0 2024-07-03T15:44:00.569 INFO:tasks.workunit.client.0.smithi028.stdout:1/223: fdatasync d4/de/ff 0 2024-07-03T15:44:00.570 INFO:tasks.workunit.client.0.smithi028.stdout:8/139: unlink d0/dd/d22/f17 0 2024-07-03T15:44:00.570 INFO:tasks.workunit.client.0.smithi028.stdout:8/140: write d0/d2/d3/dc/f1f [906491,88485] 0 2024-07-03T15:44:00.570 INFO:tasks.workunit.client.0.smithi028.stdout:8/141: fsync d0/d2/f27 0 2024-07-03T15:44:00.572 INFO:tasks.workunit.client.0.smithi028.stdout:6/83: dread d4/f10 [0,4194304] 0 2024-07-03T15:44:00.572 INFO:tasks.workunit.client.0.smithi028.stdout:6/84: truncate d4/da/f17 447789 0 2024-07-03T15:44:00.572 INFO:tasks.workunit.client.0.smithi028.stdout:4/82: getdents d2/d5 0 2024-07-03T15:44:00.573 INFO:tasks.workunit.client.0.smithi028.stdout:4/83: dread - d2/f14 zero size 2024-07-03T15:44:00.574 INFO:tasks.workunit.client.0.smithi028.stdout:3/145: mknod d18/d1a/d1e/c32 0 2024-07-03T15:44:00.575 INFO:tasks.workunit.client.1.smithi148.stdout:2/870: sync 2024-07-03T15:44:00.575 INFO:tasks.workunit.client.1.smithi148.stdout:8/887: sync 2024-07-03T15:44:00.575 INFO:tasks.workunit.client.1.smithi148.stdout:6/990: sync 2024-07-03T15:44:00.578 INFO:tasks.workunit.client.0.smithi028.stdout:1/224: link d4/de/f31 d4/de/d1e/d4e/d51/f5e 0 2024-07-03T15:44:00.578 INFO:tasks.workunit.client.0.smithi028.stdout:3/146: read f9 [448054,53829] 0 2024-07-03T15:44:00.578 INFO:tasks.workunit.client.1.smithi148.stdout:8/888: mkdir d4/da/d47/d88/dce/d12e 0 2024-07-03T15:44:00.578 INFO:tasks.workunit.client.1.smithi148.stdout:8/889: chown d4/da/d51/d18/d20/f2b 191043 1 2024-07-03T15:44:00.578 INFO:tasks.workunit.client.1.smithi148.stdout:8/890: chown d4/da/d14/d1a/d32/d38/dba/c99 36 1 2024-07-03T15:44:00.580 INFO:tasks.workunit.client.1.smithi148.stdout:6/991: symlink d3/d1d/d28/d31/d61/d102/l145 0 2024-07-03T15:44:00.581 INFO:tasks.workunit.client.0.smithi028.stdout:6/85: link d4/da/cd d4/da/c1a 0 2024-07-03T15:44:00.583 INFO:tasks.workunit.client.1.smithi148.stdout:6/992: creat d3/d1d/d28/d53/d6d/d8f/dd3/f146 x:0 0 0 2024-07-03T15:44:00.583 INFO:tasks.workunit.client.1.smithi148.stdout:6/993: dread - d3/da5/d113/f118 zero size 2024-07-03T15:44:00.583 INFO:tasks.workunit.client.1.smithi148.stdout:4/839: dwrite d0/d8/dd4/fdb [0,4194304] 0 2024-07-03T15:44:00.584 INFO:tasks.workunit.client.1.smithi148.stdout:1/824: dwrite d0/de/d2f/d72/f86 [0,4194304] 0 2024-07-03T15:44:00.584 INFO:tasks.workunit.client.1.smithi148.stdout:7/940: dwrite d5/f57 [0,4194304] 0 2024-07-03T15:44:00.589 INFO:tasks.workunit.client.0.smithi028.stdout:1/225: truncate d4/d11/f12 196303 0 2024-07-03T15:44:00.590 INFO:tasks.workunit.client.0.smithi028.stdout:1/226: fdatasync d4/de/f5a 0 2024-07-03T15:44:00.590 INFO:tasks.workunit.client.0.smithi028.stdout:1/227: stat d4/de/d1e/d4e 0 2024-07-03T15:44:00.590 INFO:tasks.workunit.client.0.smithi028.stdout:6/86: stat d4/c7 0 2024-07-03T15:44:00.591 INFO:tasks.workunit.client.0.smithi028.stdout:6/87: fsync d4/f11 0 2024-07-03T15:44:00.591 INFO:tasks.workunit.client.0.smithi028.stdout:6/88: write d4/f11 [2549754,71241] 0 2024-07-03T15:44:00.596 INFO:tasks.workunit.client.1.smithi148.stdout:7/941: read d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/db2/fe5 [959398,61223] 0 2024-07-03T15:44:00.596 INFO:tasks.workunit.client.1.smithi148.stdout:7/942: chown d5/d7d/da7 0 1 2024-07-03T15:44:00.598 INFO:tasks.workunit.client.1.smithi148.stdout:6/994: dread d3/d1d/d28/d3e/f64 [0,4194304] 0 2024-07-03T15:44:00.598 INFO:tasks.workunit.client.1.smithi148.stdout:2/871: read d5/d14/d86/db9/dbb/d68/fc2 [787696,27346] 0 2024-07-03T15:44:00.603 INFO:tasks.workunit.client.0.smithi028.stdout:9/119: sync 2024-07-03T15:44:00.605 INFO:tasks.workunit.client.0.smithi028.stdout:0/79: rmdir d2/d3 39 2024-07-03T15:44:00.606 INFO:tasks.workunit.client.0.smithi028.stdout:2/127: rename da/d23/d28 to da/d24/d2c 0 2024-07-03T15:44:00.606 INFO:tasks.workunit.client.0.smithi028.stdout:2/128: chown l1 574 1 2024-07-03T15:44:00.608 INFO:tasks.workunit.client.1.smithi148.stdout:3/919: write dd/f31 [1297483,103392] 0 2024-07-03T15:44:00.608 INFO:tasks.workunit.client.1.smithi148.stdout:3/920: chown dd/d35/d41/d89 497609 1 2024-07-03T15:44:00.608 INFO:tasks.workunit.client.1.smithi148.stdout:4/840: write d0/d28/d7d/fdd [2902560,55468] 0 2024-07-03T15:44:00.608 INFO:tasks.workunit.client.1.smithi148.stdout:8/891: dwrite d4/da/d14/d3a/da8/dd3/ffc [0,4194304] 0 2024-07-03T15:44:00.608 INFO:tasks.workunit.client.1.smithi148.stdout:4/841: fsync d0/d8/d13/d67/fe1 0 2024-07-03T15:44:00.608 INFO:tasks.workunit.client.1.smithi148.stdout:4/842: fsync d0/d8/d3a/dce/df7/f108 0 2024-07-03T15:44:00.613 INFO:tasks.workunit.client.1.smithi148.stdout:2/872: rmdir d5/d67 39 2024-07-03T15:44:00.617 INFO:tasks.workunit.client.0.smithi028.stdout:1/228: creat d4/de/d1e/d4e/d51/f5f x:0 0 0 2024-07-03T15:44:00.618 INFO:tasks.workunit.client.0.smithi028.stdout:2/129: write da/f12 [839302,68516] 0 2024-07-03T15:44:00.618 INFO:tasks.workunit.client.1.smithi148.stdout:6/995: dread d3/da5/fb9 [0,4194304] 0 2024-07-03T15:44:00.620 INFO:tasks.workunit.client.1.smithi148.stdout:3/921: mknod dd/d33/d37/d68/d2e/d40/d77/c12b 0 2024-07-03T15:44:00.620 INFO:tasks.workunit.client.1.smithi148.stdout:1/825: dwrite d0/d2/fe1 [0,4194304] 0 2024-07-03T15:44:00.621 INFO:tasks.workunit.client.0.smithi028.stdout:8/142: rename d0/d2/d3/dc/d11/c2d to d0/d2/c36 0 2024-07-03T15:44:00.624 INFO:tasks.workunit.client.0.smithi028.stdout:1/229: symlink d4/d2c/d3a/d43/l60 0 2024-07-03T15:44:00.626 INFO:tasks.workunit.client.0.smithi028.stdout:2/130: mknod da/d23/c2d 0 2024-07-03T15:44:00.627 INFO:tasks.workunit.client.1.smithi148.stdout:6/996: dread d3/f55 [0,4194304] 0 2024-07-03T15:44:00.627 INFO:tasks.workunit.client.0.smithi028.stdout:6/89: dwrite d4/da/f17 [0,4194304] 0 2024-07-03T15:44:00.627 INFO:tasks.workunit.client.1.smithi148.stdout:4/843: unlink d0/d91/da8/dab/d49/lf9 0 2024-07-03T15:44:00.629 INFO:tasks.workunit.client.0.smithi028.stdout:4/84: rename d2/d5 to d2/d12/d19 0 2024-07-03T15:44:00.629 INFO:tasks.workunit.client.0.smithi028.stdout:4/85: dread - d2/d12/d19/f10 zero size 2024-07-03T15:44:00.631 INFO:tasks.workunit.client.1.smithi148.stdout:1/826: dread d0/d2/d11/d1e/d9d/fdb [0,4194304] 0 2024-07-03T15:44:00.631 INFO:tasks.workunit.client.1.smithi148.stdout:1/827: fdatasync d0/de/d44/f5e 0 2024-07-03T15:44:00.631 INFO:tasks.workunit.client.0.smithi028.stdout:8/143: mkdir d0/dd/d37 0 2024-07-03T15:44:00.632 INFO:tasks.workunit.client.1.smithi148.stdout:8/892: rename d4/da/f107 to d4/da/d47/d88/d92/f12f 0 2024-07-03T15:44:00.632 INFO:tasks.workunit.client.1.smithi148.stdout:3/922: read dd/d35/dac/fe7 [3989591,98096] 0 2024-07-03T15:44:00.632 INFO:tasks.workunit.client.1.smithi148.stdout:8/893: chown d4/da/d47/d88/f27 29644 1 2024-07-03T15:44:00.632 INFO:tasks.workunit.client.1.smithi148.stdout:3/923: write dd/f2a [7496368,130910] 0 2024-07-03T15:44:00.632 INFO:tasks.workunit.client.0.smithi028.stdout:2/131: link da/d23/c2d da/d1e/c2e 0 2024-07-03T15:44:00.633 INFO:tasks.workunit.client.1.smithi148.stdout:7/943: dwrite d5/d7d/da7/d1c/d4a/d66/d93/faf [0,4194304] 0 2024-07-03T15:44:00.634 INFO:tasks.workunit.client.1.smithi148.stdout:7/944: write d5/d7d/da7/d1c/d33/d4c/d21/d39/f3e [5677591,33323] 0 2024-07-03T15:44:00.634 INFO:tasks.workunit.client.1.smithi148.stdout:7/945: write d5/d7d/da7/d1c/d33/d4c/d21/f32 [456357,123990] 0 2024-07-03T15:44:00.642 INFO:tasks.workunit.client.0.smithi028.stdout:6/90: truncate d4/f10 1294971 0 2024-07-03T15:44:00.643 INFO:tasks.workunit.client.1.smithi148.stdout:1/828: dread d0/de/d44/d54/d5d/fc1 [0,4194304] 0 2024-07-03T15:44:00.643 INFO:tasks.workunit.client.1.smithi148.stdout:1/829: fdatasync d0/d2/d6/d14/d29/d94/fa0 0 2024-07-03T15:44:00.643 INFO:tasks.workunit.client.1.smithi148.stdout:1/830: fsync d0/de/d2f/d72/df9/d42/db7/dbb/fe8 0 2024-07-03T15:44:00.643 INFO:tasks.workunit.client.1.smithi148.stdout:6/997: getdents d3/d88/d140/dc4/d85/da2/d119 0 2024-07-03T15:44:00.643 INFO:tasks.workunit.client.1.smithi148.stdout:1/831: creat d0/d2/d11/d1e/d9d/dc0/f110 x:0 0 0 2024-07-03T15:44:00.644 INFO:tasks.workunit.client.0.smithi028.stdout:0/80: rename d2 to d2/d1a 22 2024-07-03T15:44:00.644 INFO:tasks.workunit.client.0.smithi028.stdout:3/147: rename d18/f2b to d18/d1a/f33 0 2024-07-03T15:44:00.645 INFO:tasks.workunit.client.0.smithi028.stdout:4/86: truncate d2/fc 4539915 0 2024-07-03T15:44:00.647 INFO:tasks.workunit.client.0.smithi028.stdout:9/120: dwrite d1/dd/f11 [4194304,4194304] 0 2024-07-03T15:44:00.647 INFO:tasks.workunit.client.0.smithi028.stdout:1/230: dread d4/f1a [0,4194304] 0 2024-07-03T15:44:00.647 INFO:tasks.workunit.client.0.smithi028.stdout:3/148: dread d18/d1a/f2e [0,4194304] 0 2024-07-03T15:44:00.647 INFO:tasks.workunit.client.0.smithi028.stdout:2/132: unlink da/d23/c2d 0 2024-07-03T15:44:00.647 INFO:tasks.workunit.client.0.smithi028.stdout:2/133: fsync da/f1d 0 2024-07-03T15:44:00.648 INFO:tasks.workunit.client.1.smithi148.stdout:7/946: mknod d5/d7d/c145 0 2024-07-03T15:44:00.648 INFO:tasks.workunit.client.1.smithi148.stdout:7/947: write d5/d7d/da7/f34 [939747,119153] 0 2024-07-03T15:44:00.649 INFO:tasks.workunit.client.0.smithi028.stdout:0/81: truncate d2/fb 780519 0 2024-07-03T15:44:00.649 INFO:tasks.workunit.client.1.smithi148.stdout:6/998: mknod d3/d88/c147 0 2024-07-03T15:44:00.649 INFO:tasks.workunit.client.1.smithi148.stdout:6/999: truncate d3/d1d/d28/d53/f13f 577209 0 2024-07-03T15:44:00.651 INFO:tasks.workunit.client.0.smithi028.stdout:4/87: creat d2/d12/d19/f1a x:0 0 0 2024-07-03T15:44:00.651 INFO:tasks.workunit.client.1.smithi148.stdout:1/832: mknod d0/de/d2f/d72/df9/d42/db7/c111 0 2024-07-03T15:44:00.654 INFO:tasks.workunit.client.1.smithi148.stdout:7/948: rename d5/d7d/da7/d1c/d33/d4c/d21/d31/f45 to d5/d7d/da7/d1c/d33/d4c/d84/d98/d143/f146 0 2024-07-03T15:44:00.654 INFO:tasks.workunit.client.1.smithi148.stdout:7/949: chown d5/d7d 2 1 2024-07-03T15:44:00.655 INFO:tasks.workunit.client.0.smithi028.stdout:1/231: mknod d4/d3c/d54/c61 0 2024-07-03T15:44:00.655 INFO:tasks.workunit.client.0.smithi028.stdout:9/121: creat d1/d10/f24 x:0 0 0 2024-07-03T15:44:00.655 INFO:tasks.workunit.client.0.smithi028.stdout:9/122: write d1/d16/f1c [116694,67277] 0 2024-07-03T15:44:00.656 INFO:tasks.workunit.client.1.smithi148.stdout:7/950: unlink d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/c5a 0 2024-07-03T15:44:00.657 INFO:tasks.workunit.client.0.smithi028.stdout:2/134: rename da/c1a to da/d1e/c2f 0 2024-07-03T15:44:00.658 INFO:tasks.workunit.client.0.smithi028.stdout:6/91: symlink d4/l1b 0 2024-07-03T15:44:00.659 INFO:tasks.workunit.client.0.smithi028.stdout:4/88: truncate d2/f7 48601 0 2024-07-03T15:44:00.659 INFO:tasks.workunit.client.0.smithi028.stdout:4/89: creat d2/f1b x:0 0 0 2024-07-03T15:44:00.659 INFO:tasks.workunit.client.0.smithi028.stdout:4/90: dread - d2/f14 zero size 2024-07-03T15:44:00.660 INFO:tasks.workunit.client.0.smithi028.stdout:1/232: rmdir d4/de/d1e/d59 39 2024-07-03T15:44:00.661 INFO:tasks.workunit.client.1.smithi148.stdout:7/951: mkdir d5/d7d/da7/d1c/d147 0 2024-07-03T15:44:00.661 INFO:tasks.workunit.client.0.smithi028.stdout:9/123: creat d1/d17/f25 x:0 0 0 2024-07-03T15:44:00.661 INFO:tasks.workunit.client.0.smithi028.stdout:2/135: mkdir da/d23/d30 0 2024-07-03T15:44:00.662 INFO:tasks.workunit.client.0.smithi028.stdout:6/92: mkdir d4/da/d1c 0 2024-07-03T15:44:00.662 INFO:tasks.workunit.client.0.smithi028.stdout:6/93: read f0 [3693352,109037] 0 2024-07-03T15:44:00.663 INFO:tasks.workunit.client.0.smithi028.stdout:4/91: mknod d2/d12/d19/c1c 0 2024-07-03T15:44:00.663 INFO:tasks.workunit.client.0.smithi028.stdout:1/233: symlink d4/d2c/l62 0 2024-07-03T15:44:00.663 INFO:tasks.workunit.client.0.smithi028.stdout:1/234: stat d4/c6 0 2024-07-03T15:44:00.665 INFO:tasks.workunit.client.0.smithi028.stdout:9/124: rename d1/f5 to d1/dd/f26 0 2024-07-03T15:44:00.665 INFO:tasks.workunit.client.0.smithi028.stdout:9/125: write d1/f9 [238435,74501] 0 2024-07-03T15:44:00.665 INFO:tasks.workunit.client.0.smithi028.stdout:9/126: read - d1/d17/f25 zero size 2024-07-03T15:44:00.666 INFO:tasks.workunit.client.0.smithi028.stdout:2/136: mknod da/d23/d30/c31 0 2024-07-03T15:44:00.666 INFO:tasks.workunit.client.0.smithi028.stdout:2/137: write da/f1d [674403,17616] 0 2024-07-03T15:44:00.666 INFO:tasks.workunit.client.0.smithi028.stdout:4/92: symlink d2/l1d 0 2024-07-03T15:44:00.667 INFO:tasks.workunit.client.0.smithi028.stdout:1/235: mknod d4/de/d1c/c63 0 2024-07-03T15:44:00.667 INFO:tasks.workunit.client.0.smithi028.stdout:1/236: fsync d4/de/d1e/d59/f2d 0 2024-07-03T15:44:00.668 INFO:tasks.workunit.client.0.smithi028.stdout:2/138: creat da/d24/d2c/f32 x:0 0 0 2024-07-03T15:44:00.669 INFO:tasks.workunit.client.0.smithi028.stdout:4/93: creat d2/d12/d19/f1e x:0 0 0 2024-07-03T15:44:00.669 INFO:tasks.workunit.client.0.smithi028.stdout:4/94: stat d2/d12/d19/f9 0 2024-07-03T15:44:00.671 INFO:tasks.workunit.client.0.smithi028.stdout:1/237: symlink d4/d2c/l64 0 2024-07-03T15:44:00.671 INFO:tasks.workunit.client.0.smithi028.stdout:1/238: link d4/la d4/d11/d16/d57/l65 0 2024-07-03T15:44:00.671 INFO:tasks.workunit.client.0.smithi028.stdout:1/239: write d4/de/d1c/f58 [884235,15587] 0 2024-07-03T15:44:00.673 INFO:tasks.workunit.client.0.smithi028.stdout:1/240: mkdir d4/d11/d16/d57/d66 0 2024-07-03T15:44:00.674 INFO:tasks.workunit.client.0.smithi028.stdout:1/241: read d4/de/f1f [4041996,120443] 0 2024-07-03T15:44:00.674 INFO:tasks.workunit.client.0.smithi028.stdout:1/242: rename d4/f1a to d4/d11/d16/d57/d66/f67 0 2024-07-03T15:44:00.674 INFO:tasks.workunit.client.1.smithi148.stdout:7/952: dread d5/d7d/da7/d1c/d33/f94 [0,4194304] 0 2024-07-03T15:44:00.675 INFO:tasks.workunit.client.1.smithi148.stdout:1/833: dwrite d0/de/d2f/d72/df9/d42/f49 [4194304,4194304] 0 2024-07-03T15:44:00.675 INFO:tasks.workunit.client.0.smithi028.stdout:1/243: creat d4/d2c/d3a/f68 x:0 0 0 2024-07-03T15:44:00.675 INFO:tasks.workunit.client.0.smithi028.stdout:0/82: dwrite d2/f15 [0,4194304] 0 2024-07-03T15:44:00.676 INFO:tasks.workunit.client.1.smithi148.stdout:7/953: rename d5/d7d/da7/d1c/d33/d4c/d21/d47/fd5 to d5/d7d/da7/d1c/d147/f148 0 2024-07-03T15:44:00.676 INFO:tasks.workunit.client.1.smithi148.stdout:7/954: read d5/d7d/da7/d1c/d33/d4c/f12 [1551911,117898] 0 2024-07-03T15:44:00.677 INFO:tasks.workunit.client.1.smithi148.stdout:7/955: dread - d5/d7d/da7/fa5 zero size 2024-07-03T15:44:00.678 INFO:tasks.workunit.client.1.smithi148.stdout:1/834: creat d0/de/d2f/d72/df9/d42/d62/d7b/f112 x:0 0 0 2024-07-03T15:44:00.679 INFO:tasks.workunit.client.0.smithi028.stdout:0/83: chown d2/d3/c16 21 1 2024-07-03T15:44:00.679 INFO:tasks.workunit.client.0.smithi028.stdout:0/84: dread - d2/fa zero size 2024-07-03T15:44:00.680 INFO:tasks.workunit.client.1.smithi148.stdout:7/956: symlink d5/d7d/da7/d1c/d147/l149 0 2024-07-03T15:44:00.680 INFO:tasks.workunit.client.1.smithi148.stdout:7/957: write d5/d7d/da7/d1c/d4a/d52/d107/f89 [521897,77358] 0 2024-07-03T15:44:00.681 INFO:tasks.workunit.client.0.smithi028.stdout:0/85: symlink d2/l1b 0 2024-07-03T15:44:00.682 INFO:tasks.workunit.client.1.smithi148.stdout:1/835: unlink d0/l34 0 2024-07-03T15:44:00.682 INFO:tasks.workunit.client.1.smithi148.stdout:7/958: mknod d5/d7d/da7/d1c/d59/d124/c14a 0 2024-07-03T15:44:00.683 INFO:tasks.workunit.client.1.smithi148.stdout:1/836: symlink d0/d2/dae/l113 0 2024-07-03T15:44:00.683 INFO:tasks.workunit.client.1.smithi148.stdout:7/959: rmdir d5/d7d/da7/d1c/d33/d4c/d21/d31 39 2024-07-03T15:44:00.684 INFO:tasks.workunit.client.1.smithi148.stdout:1/837: mkdir d0/d2/dae/d114 0 2024-07-03T15:44:00.685 INFO:tasks.workunit.client.1.smithi148.stdout:1/838: creat d0/d2/d11/d1e/d9d/dc2/de2/d77/f115 x:0 0 0 2024-07-03T15:44:00.685 INFO:tasks.workunit.client.1.smithi148.stdout:1/839: fdatasync d0/de/d44/d54/fb6 0 2024-07-03T15:44:00.686 INFO:tasks.workunit.client.1.smithi148.stdout:7/960: write d5/d7d/da7/d1c/d4a/f62 [1151480,100405] 0 2024-07-03T15:44:00.687 INFO:tasks.workunit.client.1.smithi148.stdout:7/961: creat d5/d7d/da7/d1c/d33/d4c/d84/f14b x:0 0 0 2024-07-03T15:44:00.687 INFO:tasks.workunit.client.1.smithi148.stdout:7/962: stat d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f 0 2024-07-03T15:44:00.687 INFO:tasks.workunit.client.1.smithi148.stdout:7/963: stat d5/d7d/da7/d1c/d33/d4c/d84/d91/daa/f128 0 2024-07-03T15:44:00.698 INFO:tasks.workunit.client.1.smithi148.stdout:1/840: dread d0/f25 [0,4194304] 0 2024-07-03T15:44:00.698 INFO:tasks.workunit.client.1.smithi148.stdout:1/841: write d0/d2/d11/d1e/d9d/dc2/de2/fcd [587852,1086] 0 2024-07-03T15:44:00.698 INFO:tasks.workunit.client.1.smithi148.stdout:1/842: chown d0/d2/d11/d1e/c6b 10224409 1 2024-07-03T15:44:00.698 INFO:tasks.workunit.client.1.smithi148.stdout:1/843: chown d0/de/d89/f99 10072871 1 2024-07-03T15:44:00.710 INFO:tasks.workunit.client.1.smithi148.stdout:1/844: read d0/de/d89/f95 [4165680,80373] 0 2024-07-03T15:44:00.710 INFO:tasks.workunit.client.1.smithi148.stdout:1/845: write d0/d2/d11/d1e/d9d/dc2/de2/d77/f92 [182130,84959] 0 2024-07-03T15:44:00.710 INFO:tasks.workunit.client.1.smithi148.stdout:1/846: read d0/d2/d11/fbe [2658122,76911] 0 2024-07-03T15:44:00.722 INFO:tasks.workunit.client.1.smithi148.stdout:1/847: dread d0/de/d2f/d72/df9/d42/d62/f4f [4194304,4194304] 0 2024-07-03T15:44:00.798 INFO:tasks.workunit.client.1.smithi148.stdout:4/844: dwrite d0/d28/f63 [0,4194304] 0 2024-07-03T15:44:00.799 INFO:tasks.workunit.client.1.smithi148.stdout:4/845: read - d0/d91/da8/dab/fc0 zero size 2024-07-03T15:44:00.801 INFO:tasks.workunit.client.0.smithi028.stdout:6/94: dwrite f2 [0,4194304] 0 2024-07-03T15:44:00.802 INFO:tasks.workunit.client.0.smithi028.stdout:6/95: symlink d4/d15/l1d 0 2024-07-03T15:44:00.803 INFO:tasks.workunit.client.0.smithi028.stdout:6/96: creat d4/da/d1c/f1e x:0 0 0 2024-07-03T15:44:00.803 INFO:tasks.workunit.client.0.smithi028.stdout:6/97: write d4/f18 [4947953,48666] 0 2024-07-03T15:44:00.804 INFO:tasks.workunit.client.1.smithi148.stdout:3/924: dwrite dd/d33/d37/d68/d2e/ff0 [0,4194304] 0 2024-07-03T15:44:00.804 INFO:tasks.workunit.client.1.smithi148.stdout:3/925: truncate dd/d33/d52/fda 750915 0 2024-07-03T15:44:00.804 INFO:tasks.workunit.client.1.smithi148.stdout:3/926: readlink dd/d33/d37/d68/l38 0 2024-07-03T15:44:00.811 INFO:tasks.workunit.client.0.smithi028.stdout:6/98: symlink d4/l1f 0 2024-07-03T15:44:00.813 INFO:tasks.workunit.client.0.smithi028.stdout:2/139: dwrite da/f26 [0,4194304] 0 2024-07-03T15:44:00.813 INFO:tasks.workunit.client.0.smithi028.stdout:2/140: fsync da/f22 0 2024-07-03T15:44:00.813 INFO:tasks.workunit.client.1.smithi148.stdout:1/848: dwrite d0/de/d2f/d72/df9/d42/d62/d7b/fa4 [0,4194304] 0 2024-07-03T15:44:00.813 INFO:tasks.workunit.client.1.smithi148.stdout:8/894: dwrite d4/da/d14/f67 [0,4194304] 0 2024-07-03T15:44:00.813 INFO:tasks.workunit.client.1.smithi148.stdout:8/895: stat d4/l79 0 2024-07-03T15:44:00.814 INFO:tasks.workunit.client.1.smithi148.stdout:8/896: write d4/da/d14/d1a/d32/d38/fd1 [1132815,125794] 0 2024-07-03T15:44:00.814 INFO:tasks.workunit.client.0.smithi028.stdout:1/244: dwrite d4/d11/f1b [0,4194304] 0 2024-07-03T15:44:00.817 INFO:tasks.workunit.client.0.smithi028.stdout:6/99: rename d4/da/f17 to d4/d15/f20 0 2024-07-03T15:44:00.818 INFO:tasks.workunit.client.1.smithi148.stdout:8/897: dread d4/da/d14/d1a/d32/d89/fee [0,4194304] 0 2024-07-03T15:44:00.818 INFO:tasks.workunit.client.1.smithi148.stdout:8/898: chown d4/da/d47/d88/d92/f4b 0 1 2024-07-03T15:44:00.818 INFO:tasks.workunit.client.1.smithi148.stdout:8/899: truncate d4/da/d14/d1a/d32/d38/d58/dcb/ff5 197364 0 2024-07-03T15:44:00.819 INFO:tasks.workunit.client.1.smithi148.stdout:3/927: dread dd/d33/d52/d57/d121/da2/fb9 [0,4194304] 0 2024-07-03T15:44:00.820 INFO:tasks.workunit.client.0.smithi028.stdout:3/149: dwrite d18/d1a/f20 [0,4194304] 0 2024-07-03T15:44:00.825 INFO:tasks.workunit.client.0.smithi028.stdout:2/141: symlink da/d23/l33 0 2024-07-03T15:44:00.825 INFO:tasks.workunit.client.1.smithi148.stdout:4/846: dwrite d0/d28/da7/d2f/d41/f124 [0,4194304] 0 2024-07-03T15:44:00.828 INFO:tasks.workunit.client.0.smithi028.stdout:2/142: dread da/f15 [0,4194304] 0 2024-07-03T15:44:00.828 INFO:tasks.workunit.client.0.smithi028.stdout:2/143: fsync da/f22 0 2024-07-03T15:44:00.829 INFO:tasks.workunit.client.0.smithi028.stdout:2/144: chown da/f12 1638 1 2024-07-03T15:44:00.829 INFO:tasks.workunit.client.0.smithi028.stdout:2/145: truncate da/f22 305798 0 2024-07-03T15:44:00.832 INFO:tasks.workunit.client.0.smithi028.stdout:1/245: creat d4/d2c/d3a/d43/f69 x:0 0 0 2024-07-03T15:44:00.834 INFO:tasks.workunit.client.1.smithi148.stdout:4/847: dread d0/d8/dd4/fdb [0,4194304] 0 2024-07-03T15:44:00.836 INFO:tasks.workunit.client.0.smithi028.stdout:6/100: creat d4/d12/f21 x:0 0 0 2024-07-03T15:44:00.846 INFO:tasks.workunit.client.0.smithi028.stdout:3/150: truncate f9 772647 0 2024-07-03T15:44:00.851 INFO:tasks.workunit.client.0.smithi028.stdout:4/95: link d2/fc d2/d12/d19/f1f 0 2024-07-03T15:44:00.855 INFO:tasks.workunit.client.0.smithi028.stdout:1/246: dwrite d4/de/d1e/f5d [0,4194304] 0 2024-07-03T15:44:00.855 INFO:tasks.workunit.client.1.smithi148.stdout:4/848: dwrite d0/d8/f10 [0,4194304] 0 2024-07-03T15:44:00.856 INFO:tasks.workunit.client.0.smithi028.stdout:2/146: creat da/d23/f34 x:0 0 0 2024-07-03T15:44:00.864 INFO:tasks.workunit.client.0.smithi028.stdout:6/101: symlink d4/d12/l22 0 2024-07-03T15:44:00.865 INFO:tasks.workunit.client.0.smithi028.stdout:2/147: write da/f26 [2414561,75164] 0 2024-07-03T15:44:00.866 INFO:tasks.workunit.client.0.smithi028.stdout:2/148: stat da/f17 0 2024-07-03T15:44:00.867 INFO:tasks.workunit.client.1.smithi148.stdout:8/900: unlink d4/da/d14/d1a/d32/d89/fcc 0 2024-07-03T15:44:00.871 INFO:tasks.workunit.client.1.smithi148.stdout:3/928: unlink dd/d33/d52/l99 0 2024-07-03T15:44:00.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:00 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:44:00.872 INFO:tasks.workunit.client.0.smithi028.stdout:4/96: unlink d2/fc 0 2024-07-03T15:44:00.874 INFO:tasks.workunit.client.0.smithi028.stdout:4/97: dread d2/f7 [0,4194304] 0 2024-07-03T15:44:00.888 INFO:tasks.workunit.client.1.smithi148.stdout:1/849: rename d0/d2/d6 to d0/d2/d11/d3a/dc5/d116 0 2024-07-03T15:44:00.891 INFO:tasks.workunit.client.0.smithi028.stdout:1/247: creat d4/de/d1e/d59/f6a x:0 0 0 2024-07-03T15:44:00.893 INFO:tasks.workunit.client.1.smithi148.stdout:8/901: dwrite d4/da/d14/d1a/d32/d38/fb2 [0,4194304] 0 2024-07-03T15:44:00.893 INFO:tasks.workunit.client.0.smithi028.stdout:6/102: dwrite d4/d15/f20 [0,4194304] 0 2024-07-03T15:44:00.894 INFO:tasks.workunit.client.1.smithi148.stdout:1/850: dread d0/d2/d11/d1e/d9d/dc2/de2/fcd [0,4194304] 0 2024-07-03T15:44:00.899 INFO:tasks.workunit.client.0.smithi028.stdout:2/149: unlink c2 0 2024-07-03T15:44:00.899 INFO:tasks.workunit.client.0.smithi028.stdout:2/150: dread - da/d23/f34 zero size 2024-07-03T15:44:00.902 INFO:tasks.workunit.client.1.smithi148.stdout:3/929: creat dd/d33/d37/d68/f12c x:0 0 0 2024-07-03T15:44:00.904 INFO:tasks.workunit.client.1.smithi148.stdout:4/849: rename d0/c9f to d0/d8/d3a/d9d/c129 0 2024-07-03T15:44:00.904 INFO:tasks.workunit.client.1.smithi148.stdout:4/850: truncate d0/d8/d13/d67/fe1 993132 0 2024-07-03T15:44:00.912 INFO:tasks.workunit.client.1.smithi148.stdout:8/902: mkdir d4/da/d14/d3a/d130 0 2024-07-03T15:44:00.912 INFO:tasks.workunit.client.1.smithi148.stdout:8/903: creat d4/d86/f131 x:0 0 0 2024-07-03T15:44:00.913 INFO:tasks.workunit.client.0.smithi028.stdout:6/103: creat d4/f23 x:0 0 0 2024-07-03T15:44:00.913 INFO:tasks.workunit.client.0.smithi028.stdout:6/104: write d4/da/d1c/f1e [489302,109579] 0 2024-07-03T15:44:00.913 INFO:tasks.workunit.client.0.smithi028.stdout:6/105: chown d4/da/c1a 1844 1 2024-07-03T15:44:00.914 INFO:tasks.workunit.client.1.smithi148.stdout:4/851: dread d0/d8/d3a/db8/dd3/de7/f6b [0,4194304] 0 2024-07-03T15:44:00.914 INFO:tasks.workunit.client.1.smithi148.stdout:1/851: creat d0/d2/d11/d3a/f117 x:0 0 0 2024-07-03T15:44:00.916 INFO:tasks.workunit.client.1.smithi148.stdout:3/930: symlink dd/l12d 0 2024-07-03T15:44:00.920 INFO:tasks.workunit.client.1.smithi148.stdout:8/904: creat d4/da/d51/d18/d20/f132 x:0 0 0 2024-07-03T15:44:00.921 INFO:tasks.workunit.client.0.smithi028.stdout:4/98: rename d2/d12/d19/f1e to d2/f20 0 2024-07-03T15:44:00.924 INFO:tasks.workunit.client.1.smithi148.stdout:4/852: rename d0/d91/da8/dab/d49/c51 to d0/d91/c12a 0 2024-07-03T15:44:00.924 INFO:tasks.workunit.client.1.smithi148.stdout:4/853: fdatasync d0/d8/d3a/db8/fd1 0 2024-07-03T15:44:00.929 INFO:tasks.workunit.client.0.smithi028.stdout:1/248: rename d4/de/d1c/l23 to d4/de/d1e/d4e/d51/l6b 0 2024-07-03T15:44:00.929 INFO:tasks.workunit.client.0.smithi028.stdout:1/249: fsync d4/de/d1e/f5d 0 2024-07-03T15:44:00.929 INFO:tasks.workunit.client.0.smithi028.stdout:1/250: chown d4/de/d1e/d4e/d51 309946171 1 2024-07-03T15:44:00.932 INFO:tasks.workunit.client.1.smithi148.stdout:8/905: symlink d4/da/d47/df9/l133 0 2024-07-03T15:44:00.934 INFO:tasks.workunit.client.0.smithi028.stdout:4/99: mknod d2/d12/c21 0 2024-07-03T15:44:00.935 INFO:tasks.workunit.client.1.smithi148.stdout:4/854: truncate d0/d91/fa2 2855774 0 2024-07-03T15:44:00.936 INFO:tasks.workunit.client.0.smithi028.stdout:6/106: symlink d4/l24 0 2024-07-03T15:44:00.941 INFO:tasks.workunit.client.1.smithi148.stdout:1/852: rmdir d0/d2/d11/d1e/d9d 39 2024-07-03T15:44:00.941 INFO:tasks.workunit.client.0.smithi028.stdout:2/151: dwrite da/d1e/f21 [0,4194304] 0 2024-07-03T15:44:00.942 INFO:tasks.workunit.client.1.smithi148.stdout:8/906: mkdir d4/da/d14/d9e/d134 0 2024-07-03T15:44:00.944 INFO:tasks.workunit.client.1.smithi148.stdout:4/855: link d0/d8/d3a/db8/dd3/deb/fed d0/d28/da7/d2f/d52/f12b 0 2024-07-03T15:44:00.946 INFO:tasks.workunit.client.1.smithi148.stdout:3/931: dwrite dd/f22 [0,4194304] 0 2024-07-03T15:44:00.946 INFO:tasks.workunit.client.1.smithi148.stdout:3/932: readlink l7 0 2024-07-03T15:44:00.946 INFO:tasks.workunit.client.1.smithi148.stdout:3/933: chown dd/d33/d52/d57/d121/da2/f125 714393 1 2024-07-03T15:44:00.947 INFO:tasks.workunit.client.0.smithi028.stdout:2/152: link l8 da/l35 0 2024-07-03T15:44:00.949 INFO:tasks.workunit.client.1.smithi148.stdout:1/853: symlink d0/de/d2f/d72/df9/d42/d62/d7b/l118 0 2024-07-03T15:44:00.950 INFO:tasks.workunit.client.0.smithi028.stdout:1/251: dwrite d4/de/f52 [0,4194304] 0 2024-07-03T15:44:00.951 INFO:tasks.workunit.client.1.smithi148.stdout:8/907: link d4/da/d51/d18/d20/fde d4/da/d14/d9e/f135 0 2024-07-03T15:44:00.952 INFO:tasks.workunit.client.1.smithi148.stdout:8/908: stat d4/da/d47/d88/c44 0 2024-07-03T15:44:00.955 INFO:tasks.workunit.client.1.smithi148.stdout:4/856: rmdir d0/d28/d6e/db5 0 2024-07-03T15:44:00.955 INFO:tasks.workunit.client.0.smithi028.stdout:4/100: write d2/d12/d19/f9 [3589467,13249] 0 2024-07-03T15:44:00.963 INFO:tasks.workunit.client.1.smithi148.stdout:3/934: mkdir dd/d33/d37/d68/d2e/d40/d8c/d12e 0 2024-07-03T15:44:00.964 INFO:tasks.workunit.client.0.smithi028.stdout:2/153: write da/f10 [117616,81155] 0 2024-07-03T15:44:00.968 INFO:tasks.workunit.client.1.smithi148.stdout:8/909: creat d4/da/f136 x:0 0 0 2024-07-03T15:44:00.968 INFO:tasks.workunit.client.1.smithi148.stdout:2/873: sync 2024-07-03T15:44:00.973 INFO:tasks.workunit.client.0.smithi028.stdout:6/107: dwrite d4/da/d13/f19 [0,4194304] 0 2024-07-03T15:44:00.973 INFO:tasks.workunit.client.0.smithi028.stdout:6/108: fsync d4/f11 0 2024-07-03T15:44:00.974 INFO:tasks.workunit.client.1.smithi148.stdout:4/857: link d0/d28/f121 d0/d28/da7/d2f/d52/f12c 0 2024-07-03T15:44:00.974 INFO:tasks.workunit.client.0.smithi028.stdout:1/252: dwrite d4/d11/f3e [0,4194304] 0 2024-07-03T15:44:00.974 INFO:tasks.workunit.client.0.smithi028.stdout:1/253: read - d4/de/f5a zero size 2024-07-03T15:44:00.974 INFO:tasks.workunit.client.0.smithi028.stdout:1/254: stat d4/c6 0 2024-07-03T15:44:00.977 INFO:tasks.workunit.client.0.smithi028.stdout:4/101: dwrite d2/f14 [0,4194304] 0 2024-07-03T15:44:00.977 INFO:tasks.workunit.client.0.smithi028.stdout:4/102: creat d2/f22 x:0 0 0 2024-07-03T15:44:00.979 INFO:tasks.workunit.client.1.smithi148.stdout:8/910: write d4/da/d47/d88/d92/fa3 [892299,42704] 0 2024-07-03T15:44:00.980 INFO:tasks.workunit.client.0.smithi028.stdout:2/154: truncate da/ff 1625356 0 2024-07-03T15:44:00.980 INFO:tasks.workunit.client.1.smithi148.stdout:3/935: creat dd/d33/d37/d68/d2e/d40/d45/d9a/f12f x:0 0 0 2024-07-03T15:44:00.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:00 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:44:00.987 INFO:tasks.workunit.client.0.smithi028.stdout:6/109: symlink d4/d15/l25 0 2024-07-03T15:44:00.987 INFO:tasks.workunit.client.0.smithi028.stdout:6/110: fdatasync f2 0 2024-07-03T15:44:00.988 INFO:tasks.workunit.client.1.smithi148.stdout:4/858: dread d0/d8/d13/d67/f86 [0,4194304] 0 2024-07-03T15:44:00.995 INFO:tasks.workunit.client.1.smithi148.stdout:2/874: dwrite d5/d14/d86/db9/dbb/d13/d1d/f43 [4194304,4194304] 0 2024-07-03T15:44:00.996 INFO:tasks.workunit.client.0.smithi028.stdout:1/255: mkdir d4/d2c/d3a/d5c/d6c 0 2024-07-03T15:44:00.997 INFO:tasks.workunit.client.1.smithi148.stdout:3/936: getdents dd/d33/d37/d8d/daf 0 2024-07-03T15:44:00.997 INFO:tasks.workunit.client.1.smithi148.stdout:3/937: write dd/d33/d37/d68/d2e/d40/d45/d73/fc2 [436792,118536] 0 2024-07-03T15:44:00.998 INFO:tasks.workunit.client.1.smithi148.stdout:4/859: getdents d0/d8/dd4 0 2024-07-03T15:44:01.002 INFO:tasks.workunit.client.0.smithi028.stdout:6/111: dread f0 [0,4194304] 0 2024-07-03T15:44:01.002 INFO:tasks.workunit.client.0.smithi028.stdout:6/112: chown d4/da/cd 21795 1 2024-07-03T15:44:01.003 INFO:tasks.workunit.client.1.smithi148.stdout:3/938: rmdir dd/d33/d37/d68/d2e/d40/d45/d73 39 2024-07-03T15:44:01.003 INFO:tasks.workunit.client.1.smithi148.stdout:3/939: chown dd/l12d 56472725 1 2024-07-03T15:44:01.003 INFO:tasks.workunit.client.1.smithi148.stdout:3/940: chown dd/d33/d37/d68/d2e/ddb 215 1 2024-07-03T15:44:01.003 INFO:tasks.workunit.client.1.smithi148.stdout:3/941: write dd/d33/d37/d68/d2e/d40/d77/d8f/fdd [418663,27872] 0 2024-07-03T15:44:01.013 INFO:tasks.workunit.client.1.smithi148.stdout:4/860: write d0/d28/f63 [7895568,106184] 0 2024-07-03T15:44:01.014 INFO:tasks.workunit.client.0.smithi028.stdout:1/256: write d4/d11/f1b [1433234,27930] 0 2024-07-03T15:44:01.020 INFO:tasks.workunit.client.0.smithi028.stdout:1/257: rename d4/de/d1e/d59/l24 to d4/d11/l6d 0 2024-07-03T15:44:01.021 INFO:tasks.workunit.client.0.smithi028.stdout:1/258: dread - d4/de/d1e/d59/f46 zero size 2024-07-03T15:44:01.023 INFO:tasks.workunit.client.1.smithi148.stdout:0/744: sync 2024-07-03T15:44:01.023 INFO:tasks.workunit.client.1.smithi148.stdout:0/745: chown da/d58/dbd/d13/c42 20757839 1 2024-07-03T15:44:01.026 INFO:tasks.workunit.client.1.smithi148.stdout:0/746: creat da/d17/d2a/d2f/ff5 x:0 0 0 2024-07-03T15:44:01.029 INFO:tasks.workunit.client.1.smithi148.stdout:7/964: sync 2024-07-03T15:44:01.030 INFO:tasks.workunit.client.1.smithi148.stdout:0/747: rmdir da/d58/dbd/d13/d14/dbe/dc3 0 2024-07-03T15:44:01.030 INFO:tasks.workunit.client.1.smithi148.stdout:0/748: write da/f16 [1616842,49355] 0 2024-07-03T15:44:01.030 INFO:tasks.workunit.client.1.smithi148.stdout:3/942: dwrite dd/d33/f87 [0,4194304] 0 2024-07-03T15:44:01.030 INFO:tasks.workunit.client.1.smithi148.stdout:3/943: write dd/d33/d52/d57/f126 [200876,127939] 0 2024-07-03T15:44:01.031 INFO:tasks.workunit.client.0.smithi028.stdout:2/155: dwrite da/f10 [4194304,4194304] 0 2024-07-03T15:44:01.032 INFO:tasks.workunit.client.1.smithi148.stdout:7/965: dread d5/d7d/da7/d1c/d4a/f62 [0,4194304] 0 2024-07-03T15:44:01.033 INFO:tasks.workunit.client.1.smithi148.stdout:2/875: dwrite d5/d14/d22/d60/d89/f10a [4194304,4194304] 0 2024-07-03T15:44:01.035 INFO:tasks.workunit.client.1.smithi148.stdout:0/749: mkdir da/d58/dbd/d4a/df6 0 2024-07-03T15:44:01.036 INFO:tasks.workunit.client.0.smithi028.stdout:6/113: dwrite d4/d15/f20 [0,4194304] 0 2024-07-03T15:44:01.037 INFO:tasks.workunit.client.1.smithi148.stdout:3/944: creat dd/d33/d37/d8d/f130 x:0 0 0 2024-07-03T15:44:01.038 INFO:tasks.workunit.client.1.smithi148.stdout:7/966: rename d5/d7d/da7/d1c/d33/d4c/d84/d91/daa/db9 to d5/d7d/da7/d1c/d33/d4c/d84/d91/daa/d14c 0 2024-07-03T15:44:01.042 INFO:tasks.workunit.client.1.smithi148.stdout:2/876: mkdir d5/d14/d86/db9/dbb/dfe/d115 0 2024-07-03T15:44:01.043 INFO:tasks.workunit.client.1.smithi148.stdout:3/945: symlink dd/d35/d41/d89/l131 0 2024-07-03T15:44:01.043 INFO:tasks.workunit.client.1.smithi148.stdout:3/946: write dd/d33/d37/d68/d2e/d40/d45/fa3 [461936,102239] 0 2024-07-03T15:44:01.044 INFO:tasks.workunit.client.1.smithi148.stdout:2/877: creat d5/d14/d22/d60/f116 x:0 0 0 2024-07-03T15:44:01.045 INFO:tasks.workunit.client.1.smithi148.stdout:0/750: dread da/d58/f79 [0,4194304] 0 2024-07-03T15:44:01.045 INFO:tasks.workunit.client.1.smithi148.stdout:4/861: dwrite d0/d28/da7/d2f/d41/f7a [0,4194304] 0 2024-07-03T15:44:01.045 INFO:tasks.workunit.client.1.smithi148.stdout:0/751: fdatasync da/d58/dbd/f26 0 2024-07-03T15:44:01.047 INFO:tasks.workunit.client.1.smithi148.stdout:0/752: dread da/d17/d2a/d2f/d93/da3/fec [0,4194304] 0 2024-07-03T15:44:01.048 INFO:tasks.workunit.client.1.smithi148.stdout:3/947: creat dd/d35/dac/f132 x:0 0 0 2024-07-03T15:44:01.049 INFO:tasks.workunit.client.0.smithi028.stdout:6/114: dread d4/d15/f20 [0,4194304] 0 2024-07-03T15:44:01.050 INFO:tasks.workunit.client.1.smithi148.stdout:2/878: mkdir d5/d14/d22/dc0/dfb/d7e/d117 0 2024-07-03T15:44:01.054 INFO:tasks.workunit.client.1.smithi148.stdout:4/862: mknod d0/d28/de8/c12d 0 2024-07-03T15:44:01.054 INFO:tasks.workunit.client.1.smithi148.stdout:4/863: write d0/d8/d3a/db8/dd3/deb/ff3 [486714,65314] 0 2024-07-03T15:44:01.055 INFO:tasks.workunit.client.1.smithi148.stdout:0/753: unlink da/d58/dbd/d13/d14/fb9 0 2024-07-03T15:44:01.055 INFO:tasks.workunit.client.1.smithi148.stdout:0/754: creat da/d58/dbd/d24/d84/ff7 x:0 0 0 2024-07-03T15:44:01.056 INFO:tasks.workunit.client.1.smithi148.stdout:3/948: getdents dd/d33/d37/d68/d67 0 2024-07-03T15:44:01.058 INFO:tasks.workunit.client.1.smithi148.stdout:4/864: symlink d0/d8/d3a/db9/l12e 0 2024-07-03T15:44:01.059 INFO:tasks.workunit.client.1.smithi148.stdout:3/949: rmdir dd/d35/d41 39 2024-07-03T15:44:01.065 INFO:tasks.workunit.client.1.smithi148.stdout:3/950: unlink dd/d33/d37/f82 0 2024-07-03T15:44:01.065 INFO:tasks.workunit.client.1.smithi148.stdout:3/951: dread - dd/d35/de1/dff/f97 zero size 2024-07-03T15:44:01.066 INFO:tasks.workunit.client.1.smithi148.stdout:0/755: write da/d58/dbd/d13/d14/d50/d95/f97 [2344682,38115] 0 2024-07-03T15:44:01.066 INFO:tasks.workunit.client.1.smithi148.stdout:0/756: write da/d17/d2a/d2f/d93/da3/fec [1432862,74188] 0 2024-07-03T15:44:01.066 INFO:tasks.workunit.client.1.smithi148.stdout:0/757: write da/d58/d90/fcf [1195984,24552] 0 2024-07-03T15:44:01.066 INFO:tasks.workunit.client.1.smithi148.stdout:0/758: write da/d17/f8b [837818,86403] 0 2024-07-03T15:44:01.072 INFO:tasks.workunit.client.0.smithi028.stdout:6/115: dwrite d4/f18 [0,4194304] 0 2024-07-03T15:44:01.072 INFO:tasks.workunit.client.0.smithi028.stdout:6/116: stat d4/d15/l25 0 2024-07-03T15:44:01.074 INFO:tasks.workunit.client.1.smithi148.stdout:4/865: dwrite d0/d8/d13/d67/fe6 [0,4194304] 0 2024-07-03T15:44:01.074 INFO:tasks.workunit.client.1.smithi148.stdout:4/866: fdatasync d0/d28/da7/d2f/d41/f7e 0 2024-07-03T15:44:01.074 INFO:tasks.workunit.client.1.smithi148.stdout:4/867: chown d0/d91 29314 1 2024-07-03T15:44:01.074 INFO:tasks.workunit.client.1.smithi148.stdout:4/868: chown d0/d28/d7d/d6f/d7c/dae/dba/fdc 9003 1 2024-07-03T15:44:01.075 INFO:tasks.workunit.client.1.smithi148.stdout:1/854: rename d0/de to d0/d2/d11/d3a/dc5/d119 0 2024-07-03T15:44:01.076 INFO:tasks.workunit.client.1.smithi148.stdout:0/759: write da/d17/d2a/d2f/fc1 [4348673,6448] 0 2024-07-03T15:44:01.076 INFO:tasks.workunit.client.0.smithi028.stdout:6/117: rename d4/d12/l22 to d4/da/l26 0 2024-07-03T15:44:01.080 INFO:tasks.workunit.client.0.smithi028.stdout:6/118: symlink d4/da/d1c/l27 0 2024-07-03T15:44:01.083 INFO:tasks.workunit.client.1.smithi148.stdout:8/911: rename d4/da/d14/d3a/da8/db8/ccf to d4/da/d14/d3a/c137 0 2024-07-03T15:44:01.086 INFO:tasks.workunit.client.1.smithi148.stdout:0/760: creat da/ff8 x:0 0 0 2024-07-03T15:44:01.086 INFO:tasks.workunit.client.1.smithi148.stdout:0/761: getdents da/d58/dbd/d4a/df6 0 2024-07-03T15:44:01.087 INFO:tasks.workunit.client.0.smithi028.stdout:4/103: truncate d2/d12/d19/f9 1067012 0 2024-07-03T15:44:01.088 INFO:tasks.workunit.client.1.smithi148.stdout:1/855: dread d0/d2/d11/d3a/dc5/d116/f26 [4194304,4194304] 0 2024-07-03T15:44:01.089 INFO:tasks.workunit.client.1.smithi148.stdout:8/912: rename d4/da/d14/d1a/d32/d38/dba/f95 to d4/da/f138 0 2024-07-03T15:44:01.090 INFO:tasks.workunit.client.0.smithi028.stdout:4/104: chown d2/d12/d19/fe 220 1 2024-07-03T15:44:01.096 INFO:tasks.workunit.client.1.smithi148.stdout:4/869: dwrite d0/d8/d3a/dce/df7/f120 [0,4194304] 0 2024-07-03T15:44:01.096 INFO:tasks.workunit.client.1.smithi148.stdout:4/870: creat d0/d8/d3a/db8/dd3/f12f x:0 0 0 2024-07-03T15:44:01.098 INFO:tasks.workunit.client.0.smithi028.stdout:2/156: dread da/ff [0,4194304] 0 2024-07-03T15:44:01.098 INFO:tasks.workunit.client.0.smithi028.stdout:2/157: write da/f15 [702831,93174] 0 2024-07-03T15:44:01.098 INFO:tasks.workunit.client.1.smithi148.stdout:0/762: rmdir da/d58/dbd/d13/d1d/da1/de1 0 2024-07-03T15:44:01.100 INFO:tasks.workunit.client.0.smithi028.stdout:2/158: mknod da/d24/d2c/c36 0 2024-07-03T15:44:01.103 INFO:tasks.workunit.client.1.smithi148.stdout:1/856: mknod d0/d2/d11/d3a/dc5/d119/d89/c11a 0 2024-07-03T15:44:01.103 INFO:tasks.workunit.client.0.smithi028.stdout:2/159: creat da/d24/f37 x:0 0 0 2024-07-03T15:44:01.103 INFO:tasks.workunit.client.0.smithi028.stdout:2/160: write da/f15 [1465373,18282] 0 2024-07-03T15:44:01.103 INFO:tasks.workunit.client.0.smithi028.stdout:2/161: fsync da/f26 0 2024-07-03T15:44:01.107 INFO:tasks.workunit.client.0.smithi028.stdout:3/151: truncate f9 418763 0 2024-07-03T15:44:01.109 INFO:tasks.workunit.client.0.smithi028.stdout:2/162: mknod da/d1e/c38 0 2024-07-03T15:44:01.110 INFO:tasks.workunit.client.1.smithi148.stdout:4/871: creat d0/d28/da7/d2f/f130 x:0 0 0 2024-07-03T15:44:01.111 INFO:tasks.workunit.client.1.smithi148.stdout:0/763: rmdir da/d58/dbd/d13/d1d/d3e/d85/d8e/db2 39 2024-07-03T15:44:01.112 INFO:tasks.workunit.client.1.smithi148.stdout:1/857: mknod d0/d2/d11/d1e/d9d/de0/c11b 0 2024-07-03T15:44:01.113 INFO:tasks.workunit.client.1.smithi148.stdout:4/872: unlink d0/d28/d7d/d6f/d7c/dae/c118 0 2024-07-03T15:44:01.116 INFO:tasks.workunit.client.1.smithi148.stdout:0/764: rename da/d58/dbd/d13/d14/f1f to da/d58/dbd/d13/d1d/d3e/d85/d8e/db2/ff9 0 2024-07-03T15:44:01.116 INFO:tasks.workunit.client.1.smithi148.stdout:0/765: fdatasync da/f73 0 2024-07-03T15:44:01.116 INFO:tasks.workunit.client.1.smithi148.stdout:8/913: dwrite d4/da/d14/d9e/f135 [0,4194304] 0 2024-07-03T15:44:01.116 INFO:tasks.workunit.client.1.smithi148.stdout:8/914: write d4/da/d14/f9b [562113,43563] 0 2024-07-03T15:44:01.117 INFO:tasks.workunit.client.1.smithi148.stdout:1/858: symlink d0/d2/d11/d3a/dc5/d119/d2f/l11c 0 2024-07-03T15:44:01.117 INFO:tasks.workunit.client.1.smithi148.stdout:1/859: chown d0/d2/d11/d3a/dc5/d116/d14/d29/d94 3 1 2024-07-03T15:44:01.117 INFO:tasks.workunit.client.0.smithi028.stdout:2/163: dread da/f12 [0,4194304] 0 2024-07-03T15:44:01.117 INFO:tasks.workunit.client.0.smithi028.stdout:2/164: creat da/d24/f39 x:0 0 0 2024-07-03T15:44:01.117 INFO:tasks.workunit.client.0.smithi028.stdout:2/165: fdatasync da/d24/d2c/f32 0 2024-07-03T15:44:01.119 INFO:tasks.workunit.client.1.smithi148.stdout:1/860: dread d0/d2/d11/d3a/dc5/d119/d2f/d72/df9/d42/daf/fb8 [0,4194304] 0 2024-07-03T15:44:01.119 INFO:tasks.workunit.client.1.smithi148.stdout:4/873: rename d0/d8/d3a/db8/dd3/de7 to d0/d28/de8/d131 0 2024-07-03T15:44:01.120 INFO:tasks.workunit.client.0.smithi028.stdout:2/166: rename da/l35 to da/d23/d30/l3a 0 2024-07-03T15:44:01.121 INFO:tasks.workunit.client.1.smithi148.stdout:0/766: symlink da/lfa 0 2024-07-03T15:44:01.121 INFO:tasks.workunit.client.0.smithi028.stdout:3/152: write f15 [2431786,80084] 0 2024-07-03T15:44:01.122 INFO:tasks.workunit.client.1.smithi148.stdout:1/861: dread d0/d2/d2e/f85 [0,4194304] 0 2024-07-03T15:44:01.122 INFO:tasks.workunit.client.1.smithi148.stdout:1/862: fsync d0/d2/d11/d1e/d9d/dc2/de2/f7a 0 2024-07-03T15:44:01.123 INFO:tasks.workunit.client.0.smithi028.stdout:2/167: link da/f22 da/d23/f3b 0 2024-07-03T15:44:01.125 INFO:tasks.workunit.client.1.smithi148.stdout:3/952: getdents dd/d35/dac 0 2024-07-03T15:44:01.126 INFO:tasks.workunit.client.0.smithi028.stdout:3/153: link l10 d18/l34 0 2024-07-03T15:44:01.128 INFO:tasks.workunit.client.1.smithi148.stdout:8/915: creat d4/da/d47/df9/f139 x:0 0 0 2024-07-03T15:44:01.128 INFO:tasks.workunit.client.1.smithi148.stdout:8/916: chown d4/da/d51/f115 9709 1 2024-07-03T15:44:01.129 INFO:tasks.workunit.client.0.smithi028.stdout:2/168: dread da/f26 [0,4194304] 0 2024-07-03T15:44:01.130 INFO:tasks.workunit.client.0.smithi028.stdout:3/154: mknod d18/c35 0 2024-07-03T15:44:01.131 INFO:tasks.workunit.client.1.smithi148.stdout:3/953: symlink dd/d33/d37/d68/d2e/d40/d8c/l133 0 2024-07-03T15:44:01.132 INFO:tasks.workunit.client.1.smithi148.stdout:3/954: dread - dd/d35/de1/f104 zero size 2024-07-03T15:44:01.133 INFO:tasks.workunit.client.0.smithi028.stdout:2/169: symlink da/d24/d2c/l3c 0 2024-07-03T15:44:01.133 INFO:tasks.workunit.client.1.smithi148.stdout:8/917: symlink d4/d86/l13a 0 2024-07-03T15:44:01.134 INFO:tasks.workunit.client.1.smithi148.stdout:8/918: chown d4/da/d14/d1a/d32/d38/d58/d8e/f109 6461 1 2024-07-03T15:44:01.135 INFO:tasks.workunit.client.0.smithi028.stdout:3/155: mknod d18/d1a/d1e/c36 0 2024-07-03T15:44:01.137 INFO:tasks.workunit.client.0.smithi028.stdout:2/170: rename da/d23/f34 to da/d24/d2c/f3d 0 2024-07-03T15:44:01.139 INFO:tasks.workunit.client.0.smithi028.stdout:4/105: mknod d2/d12/d19/c23 0 2024-07-03T15:44:01.144 INFO:tasks.workunit.client.1.smithi148.stdout:3/955: symlink dd/d33/d52/d57/d121/d8b/l134 0 2024-07-03T15:44:01.146 INFO:tasks.workunit.client.0.smithi028.stdout:4/106: write d2/d12/d19/f1a [831861,86922] 0 2024-07-03T15:44:01.151 INFO:tasks.workunit.client.1.smithi148.stdout:4/874: rename d0/d91/da8/dab/d49 to d0/d28/de8/d132 0 2024-07-03T15:44:01.155 INFO:tasks.workunit.client.0.smithi028.stdout:2/171: dwrite da/f10 [4194304,4194304] 0 2024-07-03T15:44:01.155 INFO:tasks.workunit.client.0.smithi028.stdout:2/172: write da/d24/d2c/f32 [747606,130717] 0 2024-07-03T15:44:01.157 INFO:tasks.workunit.client.1.smithi148.stdout:2/879: dwrite d5/d14/d86/db9/dbb/d68/dcd/f38 [0,4194304] 0 2024-07-03T15:44:01.157 INFO:tasks.workunit.client.1.smithi148.stdout:2/880: stat d5/d14/d86/db9/dbb/d68/dcd/f10d 0 2024-07-03T15:44:01.158 INFO:tasks.workunit.client.1.smithi148.stdout:2/881: fsync d5/d14/d86/db9/dbb/d13/d41/d7b/fc8 0 2024-07-03T15:44:01.158 INFO:tasks.workunit.client.1.smithi148.stdout:2/882: readlink d5/d14/d22/dc0/dfb/d79/la6 0 2024-07-03T15:44:01.158 INFO:tasks.workunit.client.1.smithi148.stdout:2/883: dread - d5/d14/d86/d93/dc5/fff zero size 2024-07-03T15:44:01.158 INFO:tasks.workunit.client.1.smithi148.stdout:0/767: rename da/d17/d2a/d2f/d93/da3/fec to da/d58/dbd/d13/d14/d50/d95/ffb 0 2024-07-03T15:44:01.160 INFO:tasks.workunit.client.0.smithi028.stdout:2/173: creat da/d23/d30/f3e x:0 0 0 2024-07-03T15:44:01.162 INFO:tasks.workunit.client.1.smithi148.stdout:4/875: creat d0/d8/d44/dc2/f133 x:0 0 0 2024-07-03T15:44:01.162 INFO:tasks.workunit.client.1.smithi148.stdout:4/876: read d0/d28/da7/d2f/d41/f7e [3834940,111289] 0 2024-07-03T15:44:01.162 INFO:tasks.workunit.client.1.smithi148.stdout:4/877: chown d0/d8/d13/d67 80040 1 2024-07-03T15:44:01.162 INFO:tasks.workunit.client.1.smithi148.stdout:4/878: truncate d0/f5 9342471 0 2024-07-03T15:44:01.163 INFO:tasks.workunit.client.1.smithi148.stdout:4/879: readlink d0/d28/d7d/d6f/d7c/l8e 0 2024-07-03T15:44:01.163 INFO:tasks.workunit.client.1.smithi148.stdout:4/880: readlink d0/d91/lac 0 2024-07-03T15:44:01.165 INFO:tasks.workunit.client.1.smithi148.stdout:3/956: dwrite dd/d33/d37/d68/d2e/f18 [0,4194304] 0 2024-07-03T15:44:01.165 INFO:tasks.workunit.client.1.smithi148.stdout:3/957: chown dd/d33/d37/d68/d2e/d40/d45/d73/fb5 938172 1 2024-07-03T15:44:01.166 INFO:tasks.workunit.client.0.smithi028.stdout:2/174: rename da/c19 to da/d23/c3f 0 2024-07-03T15:44:01.169 INFO:tasks.workunit.client.1.smithi148.stdout:1/863: rename d0/d2/d11/d3a/dc5/d119/d2f/d72/df9/d42/db7/dbb/fe8 to d0/d2/d11/d3a/dc5/f11d 0 2024-07-03T15:44:01.169 INFO:tasks.workunit.client.1.smithi148.stdout:1/864: write d0/d2/d11/d3a/dc5/d119/f83 [2982282,81613] 0 2024-07-03T15:44:01.171 INFO:tasks.workunit.client.1.smithi148.stdout:4/881: unlink d0/d8/dd4/f116 0 2024-07-03T15:44:01.171 INFO:tasks.workunit.client.1.smithi148.stdout:4/882: chown d0/d28/da7/d2f/d52/f12b 28 1 2024-07-03T15:44:01.177 INFO:tasks.workunit.client.0.smithi028.stdout:5/149: sync 2024-07-03T15:44:01.177 INFO:tasks.workunit.client.0.smithi028.stdout:9/127: sync 2024-07-03T15:44:01.177 INFO:tasks.workunit.client.0.smithi028.stdout:9/128: dread - d1/dd/f1b zero size 2024-07-03T15:44:01.178 INFO:tasks.workunit.client.0.smithi028.stdout:9/129: fdatasync d1/dd/f26 0 2024-07-03T15:44:01.178 INFO:tasks.workunit.client.0.smithi028.stdout:9/130: truncate d1/d17/f25 19858 0 2024-07-03T15:44:01.178 INFO:tasks.workunit.client.0.smithi028.stdout:9/131: write d1/dd/f20 [982868,46316] 0 2024-07-03T15:44:01.178 INFO:tasks.workunit.client.0.smithi028.stdout:9/132: readlink d1/lc 0 2024-07-03T15:44:01.178 INFO:tasks.workunit.client.0.smithi028.stdout:0/86: sync 2024-07-03T15:44:01.178 INFO:tasks.workunit.client.0.smithi028.stdout:7/63: sync 2024-07-03T15:44:01.178 INFO:tasks.workunit.client.0.smithi028.stdout:7/64: creat d9/f17 x:0 0 0 2024-07-03T15:44:01.178 INFO:tasks.workunit.client.0.smithi028.stdout:7/65: write d9/f10 [1050532,21491] 0 2024-07-03T15:44:01.178 INFO:tasks.workunit.client.0.smithi028.stdout:8/144: sync 2024-07-03T15:44:01.179 INFO:tasks.workunit.client.1.smithi148.stdout:2/884: dwrite d5/d14/d22/d60/f65 [0,4194304] 0 2024-07-03T15:44:01.179 INFO:tasks.workunit.client.0.smithi028.stdout:5/150: mknod d0/dd/d13/d29/c34 0 2024-07-03T15:44:01.180 INFO:tasks.workunit.client.0.smithi028.stdout:9/133: creat d1/f27 x:0 0 0 2024-07-03T15:44:01.180 INFO:tasks.workunit.client.0.smithi028.stdout:9/134: write d1/dd/f1b [970580,80706] 0 2024-07-03T15:44:01.180 INFO:tasks.workunit.client.0.smithi028.stdout:9/135: dread - d1/f21 zero size 2024-07-03T15:44:01.184 INFO:tasks.workunit.client.1.smithi148.stdout:2/885: creat d5/d14/d22/dc0/dfb/d79/f118 x:0 0 0 2024-07-03T15:44:01.184 INFO:tasks.workunit.client.1.smithi148.stdout:2/886: write d5/d14/d22/d60/f90 [4524009,9527] 0 2024-07-03T15:44:01.187 INFO:tasks.workunit.client.1.smithi148.stdout:2/887: symlink d5/d14/d86/d93/dc5/l119 0 2024-07-03T15:44:01.187 INFO:tasks.workunit.client.1.smithi148.stdout:2/888: readlink l2 0 2024-07-03T15:44:01.188 INFO:tasks.workunit.client.1.smithi148.stdout:2/889: write d5/d14/d86/db9/dbb/d13/d1d/da1/fa2 [870519,56161] 0 2024-07-03T15:44:01.188 INFO:tasks.workunit.client.1.smithi148.stdout:2/890: creat d5/d14/d8b/dd5/f11a x:0 0 0 2024-07-03T15:44:01.189 INFO:tasks.workunit.client.1.smithi148.stdout:4/883: dwrite d0/d8/d3a/db9/fcf [4194304,4194304] 0 2024-07-03T15:44:01.192 INFO:tasks.workunit.client.0.smithi028.stdout:9/136: mkdir d1/d10/d28 0 2024-07-03T15:44:01.192 INFO:tasks.workunit.client.0.smithi028.stdout:9/137: truncate d1/dd/f1b 1067861 0 2024-07-03T15:44:01.193 INFO:tasks.workunit.client.0.smithi028.stdout:9/138: read d1/d17/f25 [17723,55210] 0 2024-07-03T15:44:01.193 INFO:tasks.workunit.client.0.smithi028.stdout:9/139: creat d1/d17/f29 x:0 0 0 2024-07-03T15:44:01.193 INFO:tasks.workunit.client.0.smithi028.stdout:9/140: stat d1/l15 0 2024-07-03T15:44:01.193 INFO:tasks.workunit.client.0.smithi028.stdout:9/141: chown d1/l4 1338 1 2024-07-03T15:44:01.193 INFO:tasks.workunit.client.0.smithi028.stdout:5/151: write d0/f9 [4198107,50989] 0 2024-07-03T15:44:01.194 INFO:tasks.workunit.client.1.smithi148.stdout:4/884: creat d0/d91/f134 x:0 0 0 2024-07-03T15:44:01.194 INFO:tasks.workunit.client.1.smithi148.stdout:4/885: dread - d0/d8/d3a/db8/f10c zero size 2024-07-03T15:44:01.194 INFO:tasks.workunit.client.1.smithi148.stdout:4/886: creat d0/d8/d44/dc2/f135 x:0 0 0 2024-07-03T15:44:01.196 INFO:tasks.workunit.client.1.smithi148.stdout:2/891: mkdir d5/da8/dde/d11b 0 2024-07-03T15:44:01.199 INFO:tasks.workunit.client.0.smithi028.stdout:0/87: dwrite d2/f15 [0,4194304] 0 2024-07-03T15:44:01.200 INFO:tasks.workunit.client.1.smithi148.stdout:2/892: mkdir d5/d67/dc3/ddd/d11c 0 2024-07-03T15:44:01.203 INFO:tasks.workunit.client.0.smithi028.stdout:3/156: rmdir d18/d1a 39 2024-07-03T15:44:01.203 INFO:tasks.workunit.client.0.smithi028.stdout:3/157: write d18/d1a/f2e [759033,85306] 0 2024-07-03T15:44:01.203 INFO:tasks.workunit.client.0.smithi028.stdout:9/142: mkdir d1/d2a 0 2024-07-03T15:44:01.204 INFO:tasks.workunit.client.1.smithi148.stdout:2/893: getdents d5/d14/d22/dc0 0 2024-07-03T15:44:01.204 INFO:tasks.workunit.client.1.smithi148.stdout:2/894: chown d5/d14/d22/dc0/dfb/d79/d9c 474 1 2024-07-03T15:44:01.205 INFO:tasks.workunit.client.1.smithi148.stdout:2/895: write d5/d14/d86/db9/dbb/d13/f29 [881609,68704] 0 2024-07-03T15:44:01.206 INFO:tasks.workunit.client.0.smithi028.stdout:5/152: rename d0/dd/f22 to d0/dd/d13/d29/f35 0 2024-07-03T15:44:01.207 INFO:tasks.workunit.client.0.smithi028.stdout:0/88: dread d2/f15 [0,4194304] 0 2024-07-03T15:44:01.210 INFO:tasks.workunit.client.0.smithi028.stdout:3/158: mknod d18/d31/c37 0 2024-07-03T15:44:01.212 INFO:tasks.workunit.client.0.smithi028.stdout:5/153: creat d0/dd/d1e/f36 x:0 0 0 2024-07-03T15:44:01.215 INFO:tasks.workunit.client.0.smithi028.stdout:0/89: mkdir d2/d1c 0 2024-07-03T15:44:01.221 INFO:tasks.workunit.client.1.smithi148.stdout:3/958: rename dd/d33/d52 to dd/d33/d37/d68/d2e/d40/d77/dc9/dee/d135 0 2024-07-03T15:44:01.222 INFO:tasks.workunit.client.0.smithi028.stdout:9/143: dwrite d1/f21 [0,4194304] 0 2024-07-03T15:44:01.222 INFO:tasks.workunit.client.0.smithi028.stdout:9/144: creat d1/f2b x:0 0 0 2024-07-03T15:44:01.224 INFO:tasks.workunit.client.1.smithi148.stdout:1/865: rename d0/d2/d11/d3a/dc5/d119/ldd to d0/d2/d11/d3a/dc5/d119/d2f/d72/df9/l11e 0 2024-07-03T15:44:01.225 INFO:tasks.workunit.client.1.smithi148.stdout:1/866: write d0/d2/d11/d1e/ffb [11562859,24234] 0 2024-07-03T15:44:01.226 INFO:tasks.workunit.client.0.smithi028.stdout:5/154: link d0/dd/df/l23 d0/l37 0 2024-07-03T15:44:01.226 INFO:tasks.workunit.client.1.smithi148.stdout:3/959: symlink dd/d35/d41/d89/l136 0 2024-07-03T15:44:01.230 INFO:tasks.workunit.client.0.smithi028.stdout:4/107: rename d2/d12/d19/fe to d2/d12/d19/f24 0 2024-07-03T15:44:01.234 INFO:tasks.workunit.client.1.smithi148.stdout:4/887: rename d0/d8/d13/d48/f8f to d0/d8/d3a/dce/d106/f136 0 2024-07-03T15:44:01.234 INFO:tasks.workunit.client.1.smithi148.stdout:4/888: write d0/d28/da7/d2f/f84 [2384791,125558] 0 2024-07-03T15:44:01.237 INFO:tasks.workunit.client.1.smithi148.stdout:1/867: symlink d0/d2/d11/d3a/dc5/d119/d44/d54/l11f 0 2024-07-03T15:44:01.237 INFO:tasks.workunit.client.1.smithi148.stdout:1/868: stat d0/d2/d11/d3a/f70 0 2024-07-03T15:44:01.237 INFO:tasks.workunit.client.1.smithi148.stdout:1/869: fdatasync d0/d2/d11/d3a/dc5/d116/f5a 0 2024-07-03T15:44:01.240 INFO:tasks.workunit.client.0.smithi028.stdout:7/66: getdents d9 0 2024-07-03T15:44:01.244 INFO:tasks.workunit.client.1.smithi148.stdout:4/889: write d0/d28/db0/fcb [3436630,31885] 0 2024-07-03T15:44:01.244 INFO:tasks.workunit.client.0.smithi028.stdout:5/155: stat d0/c8 0 2024-07-03T15:44:01.246 INFO:tasks.workunit.client.1.smithi148.stdout:1/870: link d0/d2/d11/d3a/dc5/d119/l60 d0/d2/d11/d1e/d9d/dc0/l120 0 2024-07-03T15:44:01.249 INFO:tasks.workunit.client.0.smithi028.stdout:7/67: mknod d9/c18 0 2024-07-03T15:44:01.251 INFO:tasks.workunit.client.1.smithi148.stdout:3/960: creat dd/d33/d37/d68/d67/da7/f137 x:0 0 0 2024-07-03T15:44:01.251 INFO:tasks.workunit.client.1.smithi148.stdout:3/961: chown dd/d33/d37/d68/d2e/d40/d77/dc9/dee/d135/d57/l12a 434 1 2024-07-03T15:44:01.252 INFO:tasks.workunit.client.0.smithi028.stdout:5/156: truncate d0/dd/d13/d19/f21 1908566 0 2024-07-03T15:44:01.252 INFO:tasks.workunit.client.0.smithi028.stdout:5/157: dread - d0/dd/d13/d19/d26/f2e zero size 2024-07-03T15:44:01.252 INFO:tasks.workunit.client.0.smithi028.stdout:5/158: chown d0/dd/d13/d19/c31 16139 1 2024-07-03T15:44:01.256 INFO:tasks.workunit.client.1.smithi148.stdout:4/890: symlink d0/d8/d13/d67/l137 0 2024-07-03T15:44:01.257 INFO:tasks.workunit.client.0.smithi028.stdout:5/159: symlink d0/dd/d14/l38 0 2024-07-03T15:44:01.258 INFO:tasks.workunit.client.0.smithi028.stdout:3/159: dwrite fe [0,4194304] 0 2024-07-03T15:44:01.259 INFO:tasks.workunit.client.0.smithi028.stdout:3/160: write f4 [3973898,19190] 0 2024-07-03T15:44:01.259 INFO:tasks.workunit.client.0.smithi028.stdout:9/145: dwrite d1/dd/f14 [0,4194304] 0 2024-07-03T15:44:01.259 INFO:tasks.workunit.client.0.smithi028.stdout:9/146: readlink d1/l15 0 2024-07-03T15:44:01.259 INFO:tasks.workunit.client.0.smithi028.stdout:0/90: dwrite d2/d3/f19 [0,4194304] 0 2024-07-03T15:44:01.260 INFO:tasks.workunit.client.1.smithi148.stdout:3/962: dread dd/d33/d37/f53 [0,4194304] 0 2024-07-03T15:44:01.260 INFO:tasks.workunit.client.1.smithi148.stdout:2/896: dwrite d5/d14/d86/db9/dbb/d13/d2d/fb8 [0,4194304] 0 2024-07-03T15:44:01.261 INFO:tasks.workunit.client.0.smithi028.stdout:3/161: truncate f17 935508 0 2024-07-03T15:44:01.261 INFO:tasks.workunit.client.0.smithi028.stdout:3/162: truncate f6 2346162 0 2024-07-03T15:44:01.262 INFO:tasks.workunit.client.1.smithi148.stdout:4/891: mknod d0/d8/d13/c138 0 2024-07-03T15:44:01.262 INFO:tasks.workunit.client.1.smithi148.stdout:4/892: creat d0/d28/da7/d2f/f139 x:0 0 0 2024-07-03T15:44:01.263 INFO:tasks.workunit.client.0.smithi028.stdout:3/163: dread f9 [0,4194304] 0 2024-07-03T15:44:01.272 INFO:tasks.workunit.client.0.smithi028.stdout:4/108: rmdir d2/d12 39 2024-07-03T15:44:01.273 INFO:tasks.workunit.client.0.smithi028.stdout:3/164: dread d18/d1a/f2a [0,4194304] 0 2024-07-03T15:44:01.273 INFO:tasks.workunit.client.0.smithi028.stdout:3/165: stat d18/c35 0 2024-07-03T15:44:01.273 INFO:tasks.workunit.client.1.smithi148.stdout:2/897: stat d5/d14/d86/db9/dbb/d13/l105 0 2024-07-03T15:44:01.275 INFO:tasks.workunit.client.1.smithi148.stdout:3/963: symlink dd/d35/de1/dff/l138 0 2024-07-03T15:44:01.275 INFO:tasks.workunit.client.1.smithi148.stdout:4/893: dread d0/d28/da7/d2f/d41/f4d [0,4194304] 0 2024-07-03T15:44:01.275 INFO:tasks.workunit.client.1.smithi148.stdout:4/894: creat d0/d8/d3a/db9/f13a x:0 0 0 2024-07-03T15:44:01.275 INFO:tasks.workunit.client.1.smithi148.stdout:4/895: fsync d0/d8/d3a/db9/fc9 0 2024-07-03T15:44:01.278 INFO:tasks.workunit.client.1.smithi148.stdout:3/964: link dd/d33/d37/d68/d2e/d40/d77/dc9/dee/d135/ca9 dd/d33/d37/d68/d2e/d40/d45/d73/c139 0 2024-07-03T15:44:01.278 INFO:tasks.workunit.client.1.smithi148.stdout:3/965: chown dd/d33/d37/d68/d2e/d40/d77/dc9/dee/d135/d57/lbe 0 1 2024-07-03T15:44:01.279 INFO:tasks.workunit.client.1.smithi148.stdout:3/966: creat dd/d33/d37/d68/d2e/d40/d77/dcd/d102/f13a x:0 0 0 2024-07-03T15:44:01.279 INFO:tasks.workunit.client.1.smithi148.stdout:3/967: chown dd/d33/d37/d68/d2e/d40/d77/dc9/dee/d135/ca9 4713096 1 2024-07-03T15:44:01.279 INFO:tasks.workunit.client.1.smithi148.stdout:3/968: creat dd/d33/d37/d68/d2e/d40/d77/dcd/d102/f13b x:0 0 0 2024-07-03T15:44:01.283 INFO:tasks.workunit.client.0.smithi028.stdout:7/68: dwrite d9/f10 [0,4194304] 0 2024-07-03T15:44:01.283 INFO:tasks.workunit.client.0.smithi028.stdout:9/147: dwrite d1/f21 [0,4194304] 0 2024-07-03T15:44:01.286 INFO:tasks.workunit.client.1.smithi148.stdout:2/898: dread d5/d14/d86/db9/dbb/d13/d41/d7b/d6a/f77 [0,4194304] 0 2024-07-03T15:44:01.287 INFO:tasks.workunit.client.1.smithi148.stdout:3/969: link dd/d33/fdf dd/d33/d37/d68/d2e/d40/d45/f13c 0 2024-07-03T15:44:01.289 INFO:tasks.workunit.client.1.smithi148.stdout:4/896: write d0/d28/de8/fee [2187104,90055] 0 2024-07-03T15:44:01.290 INFO:tasks.workunit.client.0.smithi028.stdout:4/109: rename d2/l18 to d2/d12/l25 0 2024-07-03T15:44:01.290 INFO:tasks.workunit.client.0.smithi028.stdout:5/160: dread d0/dd/d13/d19/f21 [0,4194304] 0 2024-07-03T15:44:01.291 INFO:tasks.workunit.client.1.smithi148.stdout:2/899: mkdir d5/d14/d11d 0 2024-07-03T15:44:01.292 INFO:tasks.workunit.client.1.smithi148.stdout:2/900: readlink d5/d14/d86/db9/dbb/d13/d41/d7b/l33 0 2024-07-03T15:44:01.292 INFO:tasks.workunit.client.1.smithi148.stdout:2/901: chown d5/d14/d86/db9/dbb/f1e 598 1 2024-07-03T15:44:01.292 INFO:tasks.workunit.client.1.smithi148.stdout:2/902: chown d5/d14/d86/db9/dbb/d68/dcd/c73 26046 1 2024-07-03T15:44:01.292 INFO:tasks.workunit.client.0.smithi028.stdout:3/166: creat d18/d1a/d1e/f38 x:0 0 0 2024-07-03T15:44:01.294 INFO:tasks.workunit.client.1.smithi148.stdout:4/897: unlink d0/d28/d6e/dc7/l95 0 2024-07-03T15:44:01.298 INFO:tasks.workunit.client.0.smithi028.stdout:9/148: symlink d1/l2c 0 2024-07-03T15:44:01.300 INFO:tasks.workunit.client.0.smithi028.stdout:4/110: link d2/f4 d2/d12/f26 0 2024-07-03T15:44:01.300 INFO:tasks.workunit.client.0.smithi028.stdout:4/111: creat d2/f27 x:0 0 0 2024-07-03T15:44:01.301 INFO:tasks.workunit.client.0.smithi028.stdout:4/112: fsync d2/d12/f17 0 2024-07-03T15:44:01.302 INFO:tasks.workunit.client.0.smithi028.stdout:3/167: truncate ff 880900 0 2024-07-03T15:44:01.303 INFO:tasks.workunit.client.0.smithi028.stdout:9/149: truncate d1/dd/f14 111747 0 2024-07-03T15:44:01.303 INFO:tasks.workunit.client.0.smithi028.stdout:9/150: creat d1/f2d x:0 0 0 2024-07-03T15:44:01.303 INFO:tasks.workunit.client.0.smithi028.stdout:9/151: creat d1/d16/f2e x:0 0 0 2024-07-03T15:44:01.303 INFO:tasks.workunit.client.0.smithi028.stdout:9/152: creat d1/d16/f2f x:0 0 0 2024-07-03T15:44:01.303 INFO:tasks.workunit.client.0.smithi028.stdout:9/153: write d1/dd/f1f [974104,65866] 0 2024-07-03T15:44:01.304 INFO:tasks.workunit.client.1.smithi148.stdout:2/903: write d5/d14/d86/db9/dbb/d13/d1d/f43 [5944837,29328] 0 2024-07-03T15:44:01.304 INFO:tasks.workunit.client.1.smithi148.stdout:2/904: dread - d5/d14/d22/dc0/dfb/d79/d9c/ffc zero size 2024-07-03T15:44:01.305 INFO:tasks.workunit.client.1.smithi148.stdout:4/898: write d0/d28/d7d/fdd [8029227,108228] 0 2024-07-03T15:44:01.307 INFO:tasks.workunit.client.1.smithi148.stdout:2/905: creat d5/d14/d8f/dce/f11e x:0 0 0 2024-07-03T15:44:01.307 INFO:tasks.workunit.client.1.smithi148.stdout:2/906: chown d5/d14/d86/db9/dbb/d68/f84 431929778 1 2024-07-03T15:44:01.308 INFO:tasks.workunit.client.0.smithi028.stdout:5/161: getdents d0/dd/d13/d19/d26 0 2024-07-03T15:44:01.308 INFO:tasks.workunit.client.0.smithi028.stdout:5/162: write d0/dd/d13/f25 [4720893,1326] 0 2024-07-03T15:44:01.308 INFO:tasks.workunit.client.0.smithi028.stdout:5/163: readlink d0/lb 0 2024-07-03T15:44:01.309 INFO:tasks.workunit.client.1.smithi148.stdout:4/899: symlink d0/d28/l13b 0 2024-07-03T15:44:01.312 INFO:tasks.workunit.client.0.smithi028.stdout:3/168: creat d18/d31/f39 x:0 0 0 2024-07-03T15:44:01.312 INFO:tasks.workunit.client.0.smithi028.stdout:3/169: dread - d18/f2d zero size 2024-07-03T15:44:01.313 INFO:tasks.workunit.client.1.smithi148.stdout:2/907: getdents d5/d14/d22/dc0/dfb/d7e 0 2024-07-03T15:44:01.315 INFO:tasks.workunit.client.1.smithi148.stdout:3/970: dwrite dd/d33/d37/d68/f9c [0,4194304] 0 2024-07-03T15:44:01.315 INFO:tasks.workunit.client.1.smithi148.stdout:3/971: fdatasync f5 0 2024-07-03T15:44:01.315 INFO:tasks.workunit.client.1.smithi148.stdout:3/972: write dd/d33/d37/d8d/fcc [830461,6685] 0 2024-07-03T15:44:01.316 INFO:tasks.workunit.client.1.smithi148.stdout:4/900: mknod d0/d8/d3a/d9d/c13c 0 2024-07-03T15:44:01.316 INFO:tasks.workunit.client.1.smithi148.stdout:4/901: fsync d0/d8/d13/d48/f81 0 2024-07-03T15:44:01.316 INFO:tasks.workunit.client.1.smithi148.stdout:4/902: creat d0/d28/d7d/d6f/d7c/dae/dba/f13d x:0 0 0 2024-07-03T15:44:01.318 INFO:tasks.workunit.client.1.smithi148.stdout:2/908: truncate d5/d14/d22/d60/f65 2533899 0 2024-07-03T15:44:01.318 INFO:tasks.workunit.client.1.smithi148.stdout:2/909: write d5/d14/d22/d60/d89/fc4 [135424,49707] 0 2024-07-03T15:44:01.319 INFO:tasks.workunit.client.0.smithi028.stdout:0/91: dwrite d2/d3/f7 [0,4194304] 0 2024-07-03T15:44:01.322 INFO:tasks.workunit.client.0.smithi028.stdout:9/154: mkdir d1/d17/d30 0 2024-07-03T15:44:01.322 INFO:tasks.workunit.client.0.smithi028.stdout:9/155: creat d1/f31 x:0 0 0 2024-07-03T15:44:01.322 INFO:tasks.workunit.client.0.smithi028.stdout:9/156: write d1/d17/f29 [459561,79041] 0 2024-07-03T15:44:01.323 INFO:tasks.workunit.client.1.smithi148.stdout:3/973: getdents dd/d33 0 2024-07-03T15:44:01.324 INFO:tasks.workunit.client.0.smithi028.stdout:9/157: read d1/d17/f29 [219580,6593] 0 2024-07-03T15:44:01.328 INFO:tasks.workunit.client.1.smithi148.stdout:2/910: mknod d5/d14/d86/db9/dbb/d13/d41/d7b/d6a/c11f 0 2024-07-03T15:44:01.331 INFO:tasks.workunit.client.1.smithi148.stdout:3/974: link dd/d33/d37/d68/d2e/d40/d77/dc9/dee/d135/fda dd/d33/d37/d68/d2e/d40/d77/d8f/f13d 0 2024-07-03T15:44:01.333 INFO:tasks.workunit.client.0.smithi028.stdout:9/158: symlink d1/d10/l32 0 2024-07-03T15:44:01.333 INFO:tasks.workunit.client.0.smithi028.stdout:0/92: write d2/d3/f7 [3935619,25476] 0 2024-07-03T15:44:01.334 INFO:tasks.workunit.client.0.smithi028.stdout:5/164: dwrite d0/dd/d13/f17 [0,4194304] 0 2024-07-03T15:44:01.336 INFO:tasks.workunit.client.0.smithi028.stdout:4/113: dwrite d2/d12/d19/f9 [0,4194304] 0 2024-07-03T15:44:01.336 INFO:tasks.workunit.client.0.smithi028.stdout:4/114: read - d2/d12/f17 zero size 2024-07-03T15:44:01.337 INFO:tasks.workunit.client.1.smithi148.stdout:3/975: mkdir dd/d33/d37/d68/d2e/d40/d77/dc9/dee/d135/d57/d121/d8b/d13e 0 2024-07-03T15:44:01.337 INFO:tasks.workunit.client.1.smithi148.stdout:3/976: write dd/d35/de1/f104 [244144,129272] 0 2024-07-03T15:44:01.337 INFO:tasks.workunit.client.1.smithi148.stdout:4/903: dwrite d0/d8/d13/db4/fcd [0,4194304] 0 2024-07-03T15:44:01.337 INFO:tasks.workunit.client.1.smithi148.stdout:2/911: read d5/d14/d86/db9/dbb/d68/f84 [2886069,17424] 0 2024-07-03T15:44:01.338 INFO:tasks.workunit.client.0.smithi028.stdout:9/159: symlink d1/d10/d28/l33 0 2024-07-03T15:44:01.338 INFO:tasks.workunit.client.0.smithi028.stdout:9/160: readlink d1/l15 0 2024-07-03T15:44:01.338 INFO:tasks.workunit.client.0.smithi028.stdout:0/93: unlink d2/c6 0 2024-07-03T15:44:01.339 INFO:tasks.workunit.client.1.smithi148.stdout:3/977: mkdir dd/d35/dac/d120/d13f 0 2024-07-03T15:44:01.340 INFO:tasks.workunit.client.0.smithi028.stdout:5/165: creat d0/dd/d13/f39 x:0 0 0 2024-07-03T15:44:01.342 INFO:tasks.workunit.client.0.smithi028.stdout:4/115: rmdir d2/d12/d19 39 2024-07-03T15:44:01.342 INFO:tasks.workunit.client.0.smithi028.stdout:4/116: chown d2/f7 3423 1 2024-07-03T15:44:01.344 INFO:tasks.workunit.client.0.smithi028.stdout:9/161: rmdir d1/d10/d28 39 2024-07-03T15:44:01.346 INFO:tasks.workunit.client.1.smithi148.stdout:4/904: rename d0/d28/de8/d132/l64 to d0/d8/d3a/dce/l13e 0 2024-07-03T15:44:01.349 INFO:tasks.workunit.client.0.smithi028.stdout:5/166: symlink d0/dd/d13/d19/l3a 0 2024-07-03T15:44:01.350 INFO:tasks.workunit.client.1.smithi148.stdout:2/912: dwrite d5/d14/d8b/dd5/f11a [0,4194304] 0 2024-07-03T15:44:01.350 INFO:tasks.workunit.client.1.smithi148.stdout:2/913: chown d5/d14/d86/db9/dbb/d13/d1d/f43 2282022 1 2024-07-03T15:44:01.350 INFO:tasks.workunit.client.1.smithi148.stdout:2/914: stat d5/d14/d22/dc0/dfb/d79/d9c/fbe 0 2024-07-03T15:44:01.351 INFO:tasks.workunit.client.1.smithi148.stdout:3/978: rename dd/d33/d37/f83 to dd/d33/f140 0 2024-07-03T15:44:01.351 INFO:tasks.workunit.client.1.smithi148.stdout:3/979: write dd/d35/de1/dff/f97 [921455,79510] 0 2024-07-03T15:44:01.351 INFO:tasks.workunit.client.1.smithi148.stdout:3/980: truncate dd/d33/df2/ffe 305562 0 2024-07-03T15:44:01.352 INFO:tasks.workunit.client.0.smithi028.stdout:3/170: rename d18/d31 to d18/d3a 0 2024-07-03T15:44:01.356 INFO:tasks.workunit.client.0.smithi028.stdout:9/162: symlink d1/d10/l34 0 2024-07-03T15:44:01.356 INFO:tasks.workunit.client.0.smithi028.stdout:9/163: fdatasync d1/d17/f25 0 2024-07-03T15:44:01.357 INFO:tasks.workunit.client.0.smithi028.stdout:3/171: dread ff [0,4194304] 0 2024-07-03T15:44:01.362 INFO:tasks.workunit.client.1.smithi148.stdout:3/981: dread dd/f3d [0,4194304] 0 2024-07-03T15:44:01.363 INFO:tasks.workunit.client.0.smithi028.stdout:4/117: rename d2/d12/d19/f10 to d2/d12/f28 0 2024-07-03T15:44:01.366 INFO:tasks.workunit.client.0.smithi028.stdout:3/172: symlink d18/l3b 0 2024-07-03T15:44:01.366 INFO:tasks.workunit.client.0.smithi028.stdout:3/173: fdatasync d18/f1d 0 2024-07-03T15:44:01.370 INFO:tasks.workunit.client.0.smithi028.stdout:3/174: write d18/f1d [15630,50094] 0 2024-07-03T15:44:01.375 INFO:tasks.workunit.client.0.smithi028.stdout:0/94: dwrite d2/d3/f11 [0,4194304] 0 2024-07-03T15:44:01.376 INFO:tasks.workunit.client.0.smithi028.stdout:3/175: dread f14 [0,4194304] 0 2024-07-03T15:44:01.381 INFO:tasks.workunit.client.1.smithi148.stdout:2/915: mknod d5/d14/d8b/de8/c120 0 2024-07-03T15:44:01.381 INFO:tasks.workunit.client.1.smithi148.stdout:2/916: chown d5/d14/d22/dc0/dfb/d7e/fa7 2322981 1 2024-07-03T15:44:01.381 INFO:tasks.workunit.client.1.smithi148.stdout:2/917: fsync d5/d14/d86/db9/dbb/d68/f84 0 2024-07-03T15:44:01.382 INFO:tasks.workunit.client.1.smithi148.stdout:2/918: chown d5/d14/d22/d60/d89 1 1 2024-07-03T15:44:01.382 INFO:tasks.workunit.client.1.smithi148.stdout:3/982: dwrite dd/d33/d37/d68/f5c [0,4194304] 0 2024-07-03T15:44:01.382 INFO:tasks.workunit.client.1.smithi148.stdout:3/983: readlink dd/d33/d37/d68/d2e/d40/d77/dc9/dee/d135/d57/l12a 0 2024-07-03T15:44:01.382 INFO:tasks.workunit.client.0.smithi028.stdout:0/95: readlink d2/d3/l13 0 2024-07-03T15:44:01.383 INFO:tasks.workunit.client.0.smithi028.stdout:3/176: mknod d18/d3a/c3c 0 2024-07-03T15:44:01.383 INFO:tasks.workunit.client.0.smithi028.stdout:3/177: write d18/f1d [1821726,30175] 0 2024-07-03T15:44:01.385 INFO:tasks.workunit.client.0.smithi028.stdout:4/118: dwrite d2/f4 [0,4194304] 0 2024-07-03T15:44:01.386 INFO:tasks.workunit.client.0.smithi028.stdout:0/96: creat d2/d3/f1d x:0 0 0 2024-07-03T15:44:01.388 INFO:tasks.workunit.client.1.smithi148.stdout:2/919: mknod d5/d14/d86/d93/dc5/c121 0 2024-07-03T15:44:01.388 INFO:tasks.workunit.client.0.smithi028.stdout:3/178: rename f2 to d18/d3a/f3d 0 2024-07-03T15:44:01.388 INFO:tasks.workunit.client.0.smithi028.stdout:3/179: write d18/d1a/f2a [776382,41923] 0 2024-07-03T15:44:01.389 INFO:tasks.workunit.client.0.smithi028.stdout:3/180: creat d18/d1a/d1e/f3e x:0 0 0 2024-07-03T15:44:01.389 INFO:tasks.workunit.client.0.smithi028.stdout:3/181: chown d18/d1a/f2e 15022 1 2024-07-03T15:44:01.393 INFO:tasks.workunit.client.1.smithi148.stdout:3/984: rename dd/d35/d41/c48 to dd/d33/d37/d68/d2e/ddb/c141 0 2024-07-03T15:44:01.395 INFO:tasks.workunit.client.1.smithi148.stdout:2/920: mknod d5/d14/c122 0 2024-07-03T15:44:01.395 INFO:tasks.workunit.client.1.smithi148.stdout:2/921: write d5/d14/d86/db9/dbb/d13/d1d/f43 [6077525,25768] 0 2024-07-03T15:44:01.395 INFO:tasks.workunit.client.1.smithi148.stdout:2/922: chown d5/d14/d8b/dd5 1 1 2024-07-03T15:44:01.396 INFO:tasks.workunit.client.0.smithi028.stdout:3/182: creat d18/d3a/f3f x:0 0 0 2024-07-03T15:44:01.396 INFO:tasks.workunit.client.0.smithi028.stdout:3/183: readlink d18/d1a/d1e/l27 0 2024-07-03T15:44:01.398 INFO:tasks.workunit.client.1.smithi148.stdout:2/923: creat d5/d67/dc3/f123 x:0 0 0 2024-07-03T15:44:01.398 INFO:tasks.workunit.client.0.smithi028.stdout:3/184: mkdir d18/d40 0 2024-07-03T15:44:01.401 INFO:tasks.workunit.client.1.smithi148.stdout:7/967: sync 2024-07-03T15:44:01.401 INFO:tasks.workunit.client.1.smithi148.stdout:8/919: sync 2024-07-03T15:44:01.401 INFO:tasks.workunit.client.0.smithi028.stdout:4/119: dwrite d2/d12/d19/f1a [0,4194304] 0 2024-07-03T15:44:01.405 INFO:tasks.workunit.client.1.smithi148.stdout:8/920: symlink d4/da/d14/d9e/l13b 0 2024-07-03T15:44:01.408 INFO:tasks.workunit.client.1.smithi148.stdout:7/968: creat d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/da8/f14d x:0 0 0 2024-07-03T15:44:01.408 INFO:tasks.workunit.client.1.smithi148.stdout:2/924: dread d5/d14/d22/fd2 [0,4194304] 0 2024-07-03T15:44:01.409 INFO:tasks.workunit.client.1.smithi148.stdout:8/921: creat d4/da/d14/d9e/d134/f13c x:0 0 0 2024-07-03T15:44:01.413 INFO:tasks.workunit.client.0.smithi028.stdout:4/120: rename d2/d12/d19/l6 to d2/d12/d19/l29 0 2024-07-03T15:44:01.413 INFO:tasks.workunit.client.1.smithi148.stdout:7/969: rename d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/db1/c138 to d5/d7d/da7/db4/c14e 0 2024-07-03T15:44:01.415 INFO:tasks.workunit.client.1.smithi148.stdout:7/970: rmdir d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f 39 2024-07-03T15:44:01.416 INFO:tasks.workunit.client.0.smithi028.stdout:3/185: getdents d18/d3a 0 2024-07-03T15:44:01.416 INFO:tasks.workunit.client.0.smithi028.stdout:3/186: stat d18/d3a/f3d 0 2024-07-03T15:44:01.419 INFO:tasks.workunit.client.1.smithi148.stdout:2/925: dread d5/d14/d86/db9/dbb/d68/dcd/f3d [0,4194304] 0 2024-07-03T15:44:01.419 INFO:tasks.workunit.client.1.smithi148.stdout:2/926: dread - d5/d67/ffa zero size 2024-07-03T15:44:01.420 INFO:tasks.workunit.client.1.smithi148.stdout:2/927: rename d5/d14/d22/d60/cb6 to d5/d14/d22/d60/c124 0 2024-07-03T15:44:01.424 INFO:tasks.workunit.client.0.smithi028.stdout:3/187: dread f6 [0,4194304] 0 2024-07-03T15:44:01.424 INFO:tasks.workunit.client.0.smithi028.stdout:3/188: dread - d18/d3a/f39 zero size 2024-07-03T15:44:01.424 INFO:tasks.workunit.client.0.smithi028.stdout:3/189: chown d18/d1a/d1e/l26 1153782517 1 2024-07-03T15:44:01.424 INFO:tasks.workunit.client.0.smithi028.stdout:3/190: dread - d18/d1a/d1e/f38 zero size 2024-07-03T15:44:01.425 INFO:tasks.workunit.client.0.smithi028.stdout:3/191: truncate d18/d1a/f24 4396473 0 2024-07-03T15:44:01.426 INFO:tasks.workunit.client.0.smithi028.stdout:3/192: unlink f4 0 2024-07-03T15:44:01.427 INFO:tasks.workunit.client.1.smithi148.stdout:3/985: dwrite dd/d35/f106 [0,4194304] 0 2024-07-03T15:44:01.427 INFO:tasks.workunit.client.1.smithi148.stdout:7/971: dread d5/d7d/da7/d1c/d4a/d66/d93/f9d [0,4194304] 0 2024-07-03T15:44:01.427 INFO:tasks.workunit.client.1.smithi148.stdout:3/986: chown dd/d33/d37/d68/d2e/d40/d77/dc9/dee/d135/l5a 525835 1 2024-07-03T15:44:01.428 INFO:tasks.workunit.client.1.smithi148.stdout:8/922: dwrite d4/da/d14/f6f [0,4194304] 0 2024-07-03T15:44:01.428 INFO:tasks.workunit.client.0.smithi028.stdout:0/97: dwrite d2/d3/f7 [0,4194304] 0 2024-07-03T15:44:01.428 INFO:tasks.workunit.client.0.smithi028.stdout:0/98: chown d2/d3/c16 0 1 2024-07-03T15:44:01.428 INFO:tasks.workunit.client.0.smithi028.stdout:0/99: chown d2/fa 2 1 2024-07-03T15:44:01.428 INFO:tasks.workunit.client.0.smithi028.stdout:3/193: rename d18/d1a/f2e to d18/d1a/d1e/f41 0 2024-07-03T15:44:01.428 INFO:tasks.workunit.client.0.smithi028.stdout:3/194: chown ld 129917715 1 2024-07-03T15:44:01.431 INFO:tasks.workunit.client.1.smithi148.stdout:2/928: dread d5/d14/d86/db9/dbb/d68/fc2 [0,4194304] 0 2024-07-03T15:44:01.432 INFO:tasks.workunit.client.0.smithi028.stdout:3/195: rename d18/d1a/f33 to d18/f42 0 2024-07-03T15:44:01.434 INFO:tasks.workunit.client.0.smithi028.stdout:0/100: getdents d2/d3 0 2024-07-03T15:44:01.435 INFO:tasks.workunit.client.0.smithi028.stdout:0/101: fdatasync d2/d3/f19 0 2024-07-03T15:44:01.435 INFO:tasks.workunit.client.0.smithi028.stdout:0/102: readlink d2/d3/l13 0 2024-07-03T15:44:01.441 INFO:tasks.workunit.client.0.smithi028.stdout:0/103: creat d2/d1c/f1e x:0 0 0 2024-07-03T15:44:01.442 INFO:tasks.workunit.client.1.smithi148.stdout:8/923: mknod d4/da/d14/c13d 0 2024-07-03T15:44:01.446 INFO:tasks.workunit.client.1.smithi148.stdout:2/929: link d5/d14/d86/db9/dbb/d68/dcd/f8e d5/d14/d86/db9/dbb/d68/dcd/f125 0 2024-07-03T15:44:01.448 INFO:tasks.workunit.client.0.smithi028.stdout:0/104: link d2/c4 d2/c1f 0 2024-07-03T15:44:01.448 INFO:tasks.workunit.client.0.smithi028.stdout:0/105: creat d2/d1c/f20 x:0 0 0 2024-07-03T15:44:01.448 INFO:tasks.workunit.client.0.smithi028.stdout:0/106: creat d2/d1c/f21 x:0 0 0 2024-07-03T15:44:01.450 INFO:tasks.workunit.client.1.smithi148.stdout:8/924: creat d4/da/d51/d53/f13e x:0 0 0 2024-07-03T15:44:01.450 INFO:tasks.workunit.client.1.smithi148.stdout:8/925: readlink d4/da/d47/df9/l110 0 2024-07-03T15:44:01.453 INFO:tasks.workunit.client.1.smithi148.stdout:2/930: symlink d5/d14/d86/db9/dbb/d68/l126 0 2024-07-03T15:44:01.453 INFO:tasks.workunit.client.1.smithi148.stdout:2/931: write d5/d14/f10c [759054,96512] 0 2024-07-03T15:44:01.457 INFO:tasks.workunit.client.1.smithi148.stdout:2/932: symlink d5/da8/l127 0 2024-07-03T15:44:01.466 INFO:tasks.workunit.client.0.smithi028.stdout:3/196: dwrite d18/d1a/f28 [0,4194304] 0 2024-07-03T15:44:01.466 INFO:tasks.workunit.client.0.smithi028.stdout:3/197: stat d18/d1a/f20 0 2024-07-03T15:44:01.466 INFO:tasks.workunit.client.0.smithi028.stdout:3/198: fdatasync f9 0 2024-07-03T15:44:01.467 INFO:tasks.workunit.client.1.smithi148.stdout:2/933: dread d5/d14/d86/db9/dbb/d13/d41/f74 [0,4194304] 0 2024-07-03T15:44:01.467 INFO:tasks.workunit.client.1.smithi148.stdout:3/987: dwrite dd/d33/d37/d68/d2e/d40/d77/dc9/dee/d135/f60 [0,4194304] 0 2024-07-03T15:44:01.468 INFO:tasks.workunit.client.1.smithi148.stdout:2/934: write d5/d14/d86/db9/dbb/d13/d2d/fd7 [579133,40684] 0 2024-07-03T15:44:01.468 INFO:tasks.workunit.client.1.smithi148.stdout:2/935: fsync d5/d14/d86/db9/dbb/d13/d2d/fb8 0 2024-07-03T15:44:01.470 INFO:tasks.workunit.client.1.smithi148.stdout:2/936: rename d5/d14/d22/d60/f90 to d5/da8/dde/f128 0 2024-07-03T15:44:01.472 INFO:tasks.workunit.client.1.smithi148.stdout:2/937: unlink d5/d14/d22/dc0/dfb/d7e/l8d 0 2024-07-03T15:44:01.473 INFO:tasks.workunit.client.1.smithi148.stdout:2/938: unlink d5/d14/d86/db9/dbb/d13/c5c 0 2024-07-03T15:44:01.475 INFO:tasks.workunit.client.0.smithi028.stdout:0/107: dwrite d2/d3/f19 [0,4194304] 0 2024-07-03T15:44:01.476 INFO:tasks.workunit.client.1.smithi148.stdout:7/972: dwrite d5/d7d/da7/d1c/d4a/d66/f9b [0,4194304] 0 2024-07-03T15:44:01.476 INFO:tasks.workunit.client.1.smithi148.stdout:7/973: write d5/d7d/da7/d1c/d33/d4c/d21/f113 [976917,29238] 0 2024-07-03T15:44:01.476 INFO:tasks.workunit.client.1.smithi148.stdout:7/974: truncate d5/d7d/da7/d1c/d33/d4c/d84/f14b 641883 0 2024-07-03T15:44:01.476 INFO:tasks.workunit.client.1.smithi148.stdout:8/926: dwrite d4/da/d51/d18/fc2 [0,4194304] 0 2024-07-03T15:44:01.479 INFO:tasks.workunit.client.0.smithi028.stdout:0/108: symlink d2/d3/l22 0 2024-07-03T15:44:01.482 INFO:tasks.workunit.client.0.smithi028.stdout:0/109: mknod d2/c23 0 2024-07-03T15:44:01.483 INFO:tasks.workunit.client.0.smithi028.stdout:3/199: dread d18/d1a/f24 [0,4194304] 0 2024-07-03T15:44:01.483 INFO:tasks.workunit.client.1.smithi148.stdout:7/975: write d5/d7d/da7/d1c/d4a/d66/d93/f9d [2891476,16000] 0 2024-07-03T15:44:01.485 INFO:tasks.workunit.client.1.smithi148.stdout:8/927: mkdir d4/da/d47/d88/d92/d13f 0 2024-07-03T15:44:01.485 INFO:tasks.workunit.client.1.smithi148.stdout:2/939: dread d5/f2f [0,4194304] 0 2024-07-03T15:44:01.486 INFO:tasks.workunit.client.1.smithi148.stdout:7/976: mknod d5/d7d/da7/d1c/d33/d4c/d84/d98/c14f 0 2024-07-03T15:44:01.487 INFO:tasks.workunit.client.1.smithi148.stdout:3/988: dwrite dd/d33/d37/d68/d2e/d61/f64 [0,4194304] 0 2024-07-03T15:44:01.488 INFO:tasks.workunit.client.1.smithi148.stdout:8/928: symlink d4/da/d51/l140 0 2024-07-03T15:44:01.489 INFO:tasks.workunit.client.1.smithi148.stdout:2/940: creat d5/d14/d86/db9/dbb/d68/def/f129 x:0 0 0 2024-07-03T15:44:01.489 INFO:tasks.workunit.client.1.smithi148.stdout:2/941: chown d5/d14/d86/db9/dbb/d68/dcd/l12 27349 1 2024-07-03T15:44:01.502 INFO:tasks.workunit.client.1.smithi148.stdout:8/929: dread d4/da/d14/d1a/d32/d38/d58/d8e/db9/fdd [0,4194304] 0 2024-07-03T15:44:01.503 INFO:tasks.workunit.client.1.smithi148.stdout:8/930: write d4/da/d51/d53/f13e [389114,60184] 0 2024-07-03T15:44:01.503 INFO:tasks.workunit.client.1.smithi148.stdout:8/931: chown d4/da/d14/d17/c122 1712 1 2024-07-03T15:44:01.503 INFO:tasks.workunit.client.1.smithi148.stdout:8/932: stat d4/da/d14/f68 0 2024-07-03T15:44:01.505 INFO:tasks.workunit.client.1.smithi148.stdout:8/933: mknod d4/da/d14/c141 0 2024-07-03T15:44:01.505 INFO:tasks.workunit.client.1.smithi148.stdout:8/934: truncate d4/da/d51/f57 4717573 0 2024-07-03T15:44:01.506 INFO:tasks.workunit.client.1.smithi148.stdout:8/935: creat d4/da/d14/d3a/da8/f142 x:0 0 0 2024-07-03T15:44:01.507 INFO:tasks.workunit.client.1.smithi148.stdout:8/936: mkdir d4/da/d14/d143 0 2024-07-03T15:44:01.508 INFO:tasks.workunit.client.1.smithi148.stdout:8/937: truncate d4/da/d14/d3a/f7e 178038 0 2024-07-03T15:44:01.508 INFO:tasks.workunit.client.1.smithi148.stdout:8/938: chown d4/da/d14/f100 36656 1 2024-07-03T15:44:01.510 INFO:tasks.workunit.client.1.smithi148.stdout:8/939: mknod d4/da/d14/d1a/d32/c144 0 2024-07-03T15:44:01.510 INFO:tasks.workunit.client.1.smithi148.stdout:8/940: write d4/da/d51/d18/d20/f40 [9291909,123736] 0 2024-07-03T15:44:01.522 INFO:tasks.workunit.client.1.smithi148.stdout:2/942: dwrite d5/d14/d86/db9/dbb/f1e [0,4194304] 0 2024-07-03T15:44:01.522 INFO:tasks.workunit.client.0.smithi028.stdout:0/110: dwrite d2/d1c/f21 [0,4194304] 0 2024-07-03T15:44:01.524 INFO:tasks.workunit.client.1.smithi148.stdout:2/943: mkdir d5/d14/d8b/de8/d12a 0 2024-07-03T15:44:01.526 INFO:tasks.workunit.client.1.smithi148.stdout:2/944: getdents d5/d14/d86/db9 0 2024-07-03T15:44:01.527 INFO:tasks.workunit.client.1.smithi148.stdout:2/945: rename d5/d14/d86/db9/dbb/d13/d41/f63 to d5/d14/d22/dc0/dfb/d79/de4/f12b 0 2024-07-03T15:44:01.530 INFO:tasks.workunit.client.0.smithi028.stdout:0/111: dread d2/d1c/f21 [0,4194304] 0 2024-07-03T15:44:01.531 INFO:tasks.workunit.client.0.smithi028.stdout:0/112: symlink d2/d1c/l24 0 2024-07-03T15:44:01.531 INFO:tasks.workunit.client.0.smithi028.stdout:0/113: chown d2/d1c 920257 1 2024-07-03T15:44:01.531 INFO:tasks.workunit.client.0.smithi028.stdout:2/175: sync 2024-07-03T15:44:01.531 INFO:tasks.workunit.client.0.smithi028.stdout:6/119: sync 2024-07-03T15:44:01.532 INFO:tasks.workunit.client.0.smithi028.stdout:1/259: sync 2024-07-03T15:44:01.532 INFO:tasks.workunit.client.0.smithi028.stdout:6/120: fdatasync d4/f18 0 2024-07-03T15:44:01.532 INFO:tasks.workunit.client.0.smithi028.stdout:1/260: fdatasync d4/d2c/d3a/f68 0 2024-07-03T15:44:01.532 INFO:tasks.workunit.client.0.smithi028.stdout:6/121: creat d4/da/d1c/f28 x:0 0 0 2024-07-03T15:44:01.532 INFO:tasks.workunit.client.0.smithi028.stdout:1/261: creat d4/de/d1e/f6e x:0 0 0 2024-07-03T15:44:01.532 INFO:tasks.workunit.client.0.smithi028.stdout:1/262: fdatasync d4/f26 0 2024-07-03T15:44:01.532 INFO:tasks.workunit.client.0.smithi028.stdout:1/263: chown d4/d2c/l48 1164724 1 2024-07-03T15:44:01.533 INFO:tasks.workunit.client.0.smithi028.stdout:1/264: dread d4/de/d1e/d59/f2d [0,4194304] 0 2024-07-03T15:44:01.534 INFO:tasks.workunit.client.0.smithi028.stdout:6/122: creat d4/f29 x:0 0 0 2024-07-03T15:44:01.535 INFO:tasks.workunit.client.0.smithi028.stdout:2/176: rmdir da/d23 39 2024-07-03T15:44:01.537 INFO:tasks.workunit.client.0.smithi028.stdout:1/265: link d4/de/d1e/d59/f6a d4/de/f6f 0 2024-07-03T15:44:01.537 INFO:tasks.workunit.client.1.smithi148.stdout:2/946: dread d5/d14/d86/db9/dbb/d68/dcd/f38 [0,4194304] 0 2024-07-03T15:44:01.538 INFO:tasks.workunit.client.1.smithi148.stdout:3/989: dwrite dd/d33/d37/d68/d2e/d40/d77/dcd/d102/f127 [0,4194304] 0 2024-07-03T15:44:01.538 INFO:tasks.workunit.client.1.smithi148.stdout:3/990: dread - dd/d33/d37/d68/d2e/d40/d77/dc9/dee/d135/d57/d121/fec zero size 2024-07-03T15:44:01.538 INFO:tasks.workunit.client.0.smithi028.stdout:3/200: dwrite d18/f2d [0,4194304] 0 2024-07-03T15:44:01.538 INFO:tasks.workunit.client.0.smithi028.stdout:1/266: mkdir d4/d2c/d3a/d43/d70 0 2024-07-03T15:44:01.538 INFO:tasks.workunit.client.0.smithi028.stdout:1/267: truncate d4/d2c/d3a/f68 809734 0 2024-07-03T15:44:01.538 INFO:tasks.workunit.client.0.smithi028.stdout:1/268: creat d4/de/d1e/d59/f71 x:0 0 0 2024-07-03T15:44:01.539 INFO:tasks.workunit.client.1.smithi148.stdout:8/941: dwrite d4/da/d14/d1a/d32/d38/d58/f104 [0,4194304] 0 2024-07-03T15:44:01.539 INFO:tasks.workunit.client.1.smithi148.stdout:7/977: dwrite d5/d7d/da7/d1c/d4a/d66/d93/f9d [0,4194304] 0 2024-07-03T15:44:01.544 INFO:tasks.workunit.client.0.smithi028.stdout:2/177: dread da/f17 [0,4194304] 0 2024-07-03T15:44:01.545 INFO:tasks.workunit.client.0.smithi028.stdout:6/123: getdents d4/da/d1c 0 2024-07-03T15:44:01.546 INFO:tasks.workunit.client.0.smithi028.stdout:0/114: read d2/d3/f7 [1256165,126149] 0 2024-07-03T15:44:01.546 INFO:tasks.workunit.client.0.smithi028.stdout:0/115: chown d2/d1c/f1e 10 1 2024-07-03T15:44:01.546 INFO:tasks.workunit.client.1.smithi148.stdout:3/991: mkdir dd/d33/d37/d68/d2e/ddb/d142 0 2024-07-03T15:44:01.548 INFO:tasks.workunit.client.1.smithi148.stdout:8/942: read d4/da/d47/d88/fa0 [1408859,130227] 0 2024-07-03T15:44:01.548 INFO:tasks.workunit.client.1.smithi148.stdout:8/943: fdatasync d4/da/d47/d88/d92/fa3 0 2024-07-03T15:44:01.548 INFO:tasks.workunit.client.1.smithi148.stdout:8/944: write f2 [10227333,69476] 0 2024-07-03T15:44:01.548 INFO:tasks.workunit.client.1.smithi148.stdout:8/945: readlink d4/da/d47/df9/l110 0 2024-07-03T15:44:01.551 INFO:tasks.workunit.client.0.smithi028.stdout:1/269: mknod d4/d2c/d3a/d5c/c72 0 2024-07-03T15:44:01.551 INFO:tasks.workunit.client.0.smithi028.stdout:1/270: write d4/de/f1f [3854371,20549] 0 2024-07-03T15:44:01.552 INFO:tasks.workunit.client.0.smithi028.stdout:3/201: write d18/f2d [5127083,49299] 0 2024-07-03T15:44:01.554 INFO:tasks.workunit.client.0.smithi028.stdout:2/178: symlink da/d23/d30/l40 0 2024-07-03T15:44:01.556 INFO:tasks.workunit.client.1.smithi148.stdout:7/978: truncate d5/d7d/da7/d1c/d4a/d52/f5f 3118009 0 2024-07-03T15:44:01.556 INFO:tasks.workunit.client.0.smithi028.stdout:3/202: dread f6 [0,4194304] 0 2024-07-03T15:44:01.557 INFO:tasks.workunit.client.0.smithi028.stdout:6/124: rename d4/da/d1c/f28 to d4/d15/f2a 0 2024-07-03T15:44:01.558 INFO:tasks.workunit.client.1.smithi148.stdout:2/947: mknod d5/c12c 0 2024-07-03T15:44:01.558 INFO:tasks.workunit.client.1.smithi148.stdout:2/948: creat d5/d14/d86/db9/dbb/d68/dcd/d64/db2/f12d x:0 0 0 2024-07-03T15:44:01.560 INFO:tasks.workunit.client.0.smithi028.stdout:2/179: link da/d23/d30/l40 da/l41 0 2024-07-03T15:44:01.560 INFO:tasks.workunit.client.0.smithi028.stdout:2/180: chown da/d23/c3f 381068722 1 2024-07-03T15:44:01.561 INFO:tasks.workunit.client.0.smithi028.stdout:6/125: mknod d4/d12/c2b 0 2024-07-03T15:44:01.562 INFO:tasks.workunit.client.1.smithi148.stdout:7/979: mknod d5/d7d/da7/c150 0 2024-07-03T15:44:01.564 INFO:tasks.workunit.client.1.smithi148.stdout:2/949: mknod d5/d14/d22/d60/d76/db3/c12e 0 2024-07-03T15:44:01.565 INFO:tasks.workunit.client.1.smithi148.stdout:7/980: link d5/d7d/da7/d1c/d33/d4c/d84/d98/d9e/f121 d5/d7d/da7/d1c/d33/d4c/d21/d47/d8b/dd3/f151 0 2024-07-03T15:44:01.565 INFO:tasks.workunit.client.1.smithi148.stdout:7/981: read d5/d7d/da7/d1c/d33/d4c/f12 [668044,120233] 0 2024-07-03T15:44:01.565 INFO:tasks.workunit.client.1.smithi148.stdout:7/982: read d5/d7d/da7/fab [3359693,97167] 0 2024-07-03T15:44:01.566 INFO:tasks.workunit.client.1.smithi148.stdout:2/950: symlink d5/d14/d86/db9/l12f 0 2024-07-03T15:44:01.566 INFO:tasks.workunit.client.1.smithi148.stdout:2/951: chown d5/d14/d86/db9/dbb/d13/d1d/l94 779 1 2024-07-03T15:44:01.566 INFO:tasks.workunit.client.1.smithi148.stdout:2/952: read - d5/d14/d86/db9/dbb/d68/def/f129 zero size 2024-07-03T15:44:01.566 INFO:tasks.workunit.client.1.smithi148.stdout:2/953: chown d5/d14/d8f 15674503 1 2024-07-03T15:44:01.567 INFO:tasks.workunit.client.1.smithi148.stdout:7/983: link d5/d7d/da7/d1c/d33/d4c/d21/d39/f77 d5/d7d/da7/d1c/d59/f152 0 2024-07-03T15:44:01.567 INFO:tasks.workunit.client.1.smithi148.stdout:2/954: rename d5/d14/d86/db9/dbb/d13/d41/d7b/d6a/f77 to d5/d14/d86/db9/dbb/d13/d41/d7b/d6a/d80/f130 0 2024-07-03T15:44:01.569 INFO:tasks.workunit.client.1.smithi148.stdout:2/955: mkdir d5/d14/d22/dc0/dfb/d79/d111/d131 0 2024-07-03T15:44:01.569 INFO:tasks.workunit.client.1.smithi148.stdout:2/956: getdents d5/d14/d22/dc0/dfb/d7e/d117 0 2024-07-03T15:44:01.569 INFO:tasks.workunit.client.1.smithi148.stdout:2/957: fdatasync d5/d14/d86/db9/dbb/d68/dcd/d64/f71 0 2024-07-03T15:44:01.569 INFO:tasks.workunit.client.1.smithi148.stdout:2/958: read d5/d14/d86/db9/dbb/f42 [1729644,79404] 0 2024-07-03T15:44:01.569 INFO:tasks.workunit.client.1.smithi148.stdout:2/959: readlink d5/d14/d22/d60/d89/ddb/lee 0 2024-07-03T15:44:01.569 INFO:tasks.workunit.client.1.smithi148.stdout:2/960: truncate d5/d67/ffa 458347 0 2024-07-03T15:44:01.569 INFO:tasks.workunit.client.1.smithi148.stdout:2/961: write d5/da8/dde/ff6 [1018257,67847] 0 2024-07-03T15:44:01.582 INFO:tasks.workunit.client.1.smithi148.stdout:2/962: dread d5/d14/d22/d60/f65 [0,4194304] 0 2024-07-03T15:44:01.583 INFO:tasks.workunit.client.1.smithi148.stdout:2/963: creat d5/d14/d22/f132 x:0 0 0 2024-07-03T15:44:01.583 INFO:tasks.workunit.client.1.smithi148.stdout:2/964: fdatasync d5/d14/d86/db9/dbb/d13/f29 0 2024-07-03T15:44:01.585 INFO:tasks.workunit.client.1.smithi148.stdout:0/768: sync 2024-07-03T15:44:01.585 INFO:tasks.workunit.client.1.smithi148.stdout:4/905: sync 2024-07-03T15:44:01.586 INFO:tasks.workunit.client.1.smithi148.stdout:0/769: chown da/d58/dbd/d13/l51 1 1 2024-07-03T15:44:01.586 INFO:tasks.workunit.client.1.smithi148.stdout:1/871: sync 2024-07-03T15:44:01.587 INFO:tasks.workunit.client.1.smithi148.stdout:7/984: dread d5/d7d/da7/d1c/d33/d4c/d84/d98/dc2/fc9 [0,4194304] 0 2024-07-03T15:44:01.587 INFO:tasks.workunit.client.1.smithi148.stdout:3/992: dwrite dd/fcf [0,4194304] 0 2024-07-03T15:44:01.588 INFO:tasks.workunit.client.1.smithi148.stdout:4/906: mkdir d0/d8/d44/dc2/d13f 0 2024-07-03T15:44:01.588 INFO:tasks.workunit.client.1.smithi148.stdout:0/770: symlink da/d58/dbd/d13/d14/d50/d95/de2/lfc 0 2024-07-03T15:44:01.588 INFO:tasks.workunit.client.1.smithi148.stdout:8/946: dwrite d4/da/d14/f6c [0,4194304] 0 2024-07-03T15:44:01.589 INFO:tasks.workunit.client.1.smithi148.stdout:3/993: truncate dd/f2a 7418717 0 2024-07-03T15:44:01.590 INFO:tasks.workunit.client.1.smithi148.stdout:7/985: rmdir d5/d7d/da7/d1c/d33/d4c/d84/d98/d143 39 2024-07-03T15:44:01.591 INFO:tasks.workunit.client.0.smithi028.stdout:0/116: dwrite d2/fa [0,4194304] 0 2024-07-03T15:44:01.592 INFO:tasks.workunit.client.0.smithi028.stdout:1/271: dwrite d4/de/d1e/d59/f46 [0,4194304] 0 2024-07-03T15:44:01.592 INFO:tasks.workunit.client.1.smithi148.stdout:4/907: link d0/d91/da8/dab/dad/f11e d0/d8/dd4/f140 0 2024-07-03T15:44:01.594 INFO:tasks.workunit.client.1.smithi148.stdout:0/771: truncate da/d17/d2a/f4b 188494 0 2024-07-03T15:44:01.595 INFO:tasks.workunit.client.0.smithi028.stdout:1/272: write d4/de/d1c/f58 [262852,91342] 0 2024-07-03T15:44:01.598 INFO:tasks.workunit.client.1.smithi148.stdout:3/994: mkdir dd/d35/de1/d143 0 2024-07-03T15:44:01.599 INFO:tasks.workunit.client.1.smithi148.stdout:1/872: write d0/d2/d11/d3a/dc5/d116/f26 [6187403,46869] 0 2024-07-03T15:44:01.600 INFO:tasks.workunit.client.0.smithi028.stdout:0/117: truncate d2/f15 692258 0 2024-07-03T15:44:01.600 INFO:tasks.workunit.client.0.smithi028.stdout:0/118: dread - d2/d1c/f20 zero size 2024-07-03T15:44:01.601 INFO:tasks.workunit.client.1.smithi148.stdout:7/986: symlink d5/d7d/da7/l153 0 2024-07-03T15:44:01.602 INFO:tasks.workunit.client.1.smithi148.stdout:7/987: write d5/d7d/da7/d1c/d33/d4c/d84/d98/f131 [947840,100184] 0 2024-07-03T15:44:01.602 INFO:tasks.workunit.client.1.smithi148.stdout:7/988: readlink d5/d7d/da7/d1c/d33/d4c/d21/d31/lc3 0 2024-07-03T15:44:01.602 INFO:tasks.workunit.client.1.smithi148.stdout:7/989: creat d5/d7d/da7/db4/f154 x:0 0 0 2024-07-03T15:44:01.602 INFO:tasks.workunit.client.1.smithi148.stdout:7/990: write d5/d7d/da7/d1c/d4a/d66/dd7/ffe [446755,24122] 0 2024-07-03T15:44:01.608 INFO:tasks.workunit.client.0.smithi028.stdout:1/273: truncate d4/f19 3875874 0 2024-07-03T15:44:01.608 INFO:tasks.workunit.client.0.smithi028.stdout:1/274: chown d4/de/d1c/d44/l4c 19 1 2024-07-03T15:44:01.608 INFO:tasks.workunit.client.1.smithi148.stdout:0/772: write da/d58/dbd/d13/d14/d50/d95/d8c/fb6 [3106469,43761] 0 2024-07-03T15:44:01.610 INFO:tasks.workunit.client.1.smithi148.stdout:3/995: read dd/d33/f3a [2891018,9975] 0 2024-07-03T15:44:01.610 INFO:tasks.workunit.client.1.smithi148.stdout:3/996: chown dd/d33/d37/d68/d2e/d40/d77/fbd 7677 1 2024-07-03T15:44:01.611 INFO:tasks.workunit.client.1.smithi148.stdout:0/773: dread da/f73 [0,4194304] 0 2024-07-03T15:44:01.611 INFO:tasks.workunit.client.1.smithi148.stdout:0/774: stat da/d58/dbd/d13/d14/d50/d95/ffb 0 2024-07-03T15:44:01.614 INFO:tasks.workunit.client.0.smithi028.stdout:1/275: mkdir d4/d2c/d3a/d5c/d6c/d73 0 2024-07-03T15:44:01.614 INFO:tasks.workunit.client.0.smithi028.stdout:1/276: rename d4/d2c/d3a/d5c/d6c to d4/d2c/d3a/d5c/d6c/d73/d74 22 2024-07-03T15:44:01.614 INFO:tasks.workunit.client.0.smithi028.stdout:1/277: chown d4/d2c/f30 127 1 2024-07-03T15:44:01.615 INFO:tasks.workunit.client.0.smithi028.stdout:0/119: rmdir d2/d1c 39 2024-07-03T15:44:01.616 INFO:tasks.workunit.client.1.smithi148.stdout:8/947: dwrite d4/da/d14/d1a/d32/d38/d58/dcb/ff5 [0,4194304] 0 2024-07-03T15:44:01.617 INFO:tasks.workunit.client.0.smithi028.stdout:1/278: rename d4/f14 to d4/d2c/f75 0 2024-07-03T15:44:01.621 INFO:tasks.workunit.client.1.smithi148.stdout:4/908: dwrite d0/d8/d3a/dce/d106/f136 [0,4194304] 0 2024-07-03T15:44:01.621 INFO:tasks.workunit.client.1.smithi148.stdout:4/909: chown d0/d28/d6e 1 1 2024-07-03T15:44:01.621 INFO:tasks.workunit.client.1.smithi148.stdout:3/997: dread fa [0,4194304] 0 2024-07-03T15:44:01.621 INFO:tasks.workunit.client.1.smithi148.stdout:3/998: write dd/d33/d37/d68/d2e/d40/d8c/dbc/fd1 [24830,72872] 0 2024-07-03T15:44:01.622 INFO:tasks.workunit.client.1.smithi148.stdout:3/999: fdatasync dd/d33/d37/d68/d2e/d40/d77/dcd/d102/f13b 0 2024-07-03T15:44:01.622 INFO:tasks.workunit.client.0.smithi028.stdout:0/120: rmdir d2/d1c 39 2024-07-03T15:44:01.622 INFO:tasks.workunit.client.0.smithi028.stdout:0/121: readlink d2/d3/l22 0 2024-07-03T15:44:01.622 INFO:tasks.workunit.client.1.smithi148.stdout:1/873: dwrite d0/d2/dd0/ffe [0,4194304] 0 2024-07-03T15:44:01.624 INFO:tasks.workunit.client.1.smithi148.stdout:8/948: dread d4/da/d14/d1a/d32/fae [0,4194304] 0 2024-07-03T15:44:01.625 INFO:tasks.workunit.client.1.smithi148.stdout:8/949: chown d4/d86/f97 10669 1 2024-07-03T15:44:01.625 INFO:tasks.workunit.client.1.smithi148.stdout:8/950: write d4/da/d14/d3a/da8/db8/fe2 [1807717,91242] 0 2024-07-03T15:44:01.625 INFO:tasks.workunit.client.1.smithi148.stdout:7/991: creat d5/d7d/da7/d1c/d33/d4c/d21/d47/d8b/f155 x:0 0 0 2024-07-03T15:44:01.625 INFO:tasks.workunit.client.0.smithi028.stdout:1/279: getdents d4/de 0 2024-07-03T15:44:01.627 INFO:tasks.workunit.client.1.smithi148.stdout:0/775: rename da/d58/f79 to da/d58/dbd/d94/dba/ffd 0 2024-07-03T15:44:01.627 INFO:tasks.workunit.client.1.smithi148.stdout:0/776: chown da/d58/dbd/d24/dc8/f5e 9029144 1 2024-07-03T15:44:01.629 INFO:tasks.workunit.client.0.smithi028.stdout:1/280: truncate d4/d2c/d3a/f68 449991 0 2024-07-03T15:44:01.629 INFO:tasks.workunit.client.1.smithi148.stdout:4/910: read d0/d8/d3a/dce/d106/f136 [3791506,153] 0 2024-07-03T15:44:01.629 INFO:tasks.workunit.client.1.smithi148.stdout:4/911: creat d0/d8/d13/d48/f141 x:0 0 0 2024-07-03T15:44:01.629 INFO:tasks.workunit.client.1.smithi148.stdout:4/912: truncate d0/d28/d7d/d6f/d7c/f8d 58072 0 2024-07-03T15:44:01.629 INFO:tasks.workunit.client.1.smithi148.stdout:4/913: dread - d0/d91/da8/dab/dad/f11e zero size 2024-07-03T15:44:01.633 INFO:tasks.workunit.client.0.smithi028.stdout:1/281: mknod d4/d2c/d3a/c76 0 2024-07-03T15:44:01.633 INFO:tasks.workunit.client.0.smithi028.stdout:1/282: creat d4/d2c/d3a/d43/f77 x:0 0 0 2024-07-03T15:44:01.636 INFO:tasks.workunit.client.1.smithi148.stdout:1/874: link d0/d2/dd0/ffe d0/d2/d11/d3a/dc5/d119/d2f/d72/df9/d42/db7/f121 0 2024-07-03T15:44:01.639 INFO:tasks.workunit.client.1.smithi148.stdout:4/914: write d0/d28/f69 [2554076,11051] 0 2024-07-03T15:44:01.639 INFO:tasks.workunit.client.1.smithi148.stdout:4/915: chown d0/d28/da7/d2f/d41/l42 30424679 1 2024-07-03T15:44:01.639 INFO:tasks.workunit.client.0.smithi028.stdout:1/283: creat d4/d2c/f78 x:0 0 0 2024-07-03T15:44:01.640 INFO:tasks.workunit.client.1.smithi148.stdout:8/951: creat d4/da/d47/d88/d92/d13f/f145 x:0 0 0 2024-07-03T15:44:01.641 INFO:tasks.workunit.client.1.smithi148.stdout:8/952: truncate d4/da/d14/d1a/d32/d38/d58/d8e/f109 555561 0 2024-07-03T15:44:01.641 INFO:tasks.workunit.client.1.smithi148.stdout:8/953: chown d4/da/d47/f94 6088704 1 2024-07-03T15:44:01.641 INFO:tasks.workunit.client.1.smithi148.stdout:7/992: rename d5/d7d/da7/d1c/d33/d4c/d21/d47/d8b/dd3/lff to d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/db6/l156 0 2024-07-03T15:44:01.642 INFO:tasks.workunit.client.0.smithi028.stdout:0/122: dwrite d2/d3/f7 [0,4194304] 0 2024-07-03T15:44:01.642 INFO:tasks.workunit.client.1.smithi148.stdout:0/777: creat da/d58/dbd/d94/dac/ffe x:0 0 0 2024-07-03T15:44:01.644 INFO:tasks.workunit.client.0.smithi028.stdout:0/123: creat d2/d3/f25 x:0 0 0 2024-07-03T15:44:01.646 INFO:tasks.workunit.client.1.smithi148.stdout:0/778: dread da/d58/dbd/d13/d14/f68 [0,4194304] 0 2024-07-03T15:44:01.646 INFO:tasks.workunit.client.1.smithi148.stdout:1/875: symlink d0/d2/d11/d3a/dc5/d116/d14/d29/de7/l122 0 2024-07-03T15:44:01.646 INFO:tasks.workunit.client.1.smithi148.stdout:1/876: getdents d0/d2/d10b 0 2024-07-03T15:44:01.647 INFO:tasks.workunit.client.1.smithi148.stdout:4/916: mknod d0/d8/d13/d48/c142 0 2024-07-03T15:44:01.647 INFO:tasks.workunit.client.1.smithi148.stdout:4/917: chown d0/d8/d3a/f47 39260 1 2024-07-03T15:44:01.650 INFO:tasks.workunit.client.1.smithi148.stdout:8/954: creat d4/da/d14/d9e/f146 x:0 0 0 2024-07-03T15:44:01.652 INFO:tasks.workunit.client.1.smithi148.stdout:0/779: symlink da/d58/dbd/d13/d14/d50/de4/lff 0 2024-07-03T15:44:01.653 INFO:tasks.workunit.client.1.smithi148.stdout:1/877: creat d0/d2/d10b/f123 x:0 0 0 2024-07-03T15:44:01.653 INFO:tasks.workunit.client.1.smithi148.stdout:1/878: write d0/d2/d11/d3a/dc5/d119/d2f/f100 [561929,37936] 0 2024-07-03T15:44:01.653 INFO:tasks.workunit.client.0.smithi028.stdout:0/124: write d2/d1c/f21 [759911,48244] 0 2024-07-03T15:44:01.654 INFO:tasks.workunit.client.1.smithi148.stdout:8/955: symlink d4/da/d47/d88/d92/l147 0 2024-07-03T15:44:01.655 INFO:tasks.workunit.client.1.smithi148.stdout:0/780: symlink da/d58/dbd/d94/dac/l100 0 2024-07-03T15:44:01.656 INFO:tasks.workunit.client.1.smithi148.stdout:7/993: dwrite d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/f83 [0,4194304] 0 2024-07-03T15:44:01.657 INFO:tasks.workunit.client.1.smithi148.stdout:1/879: unlink d0/d2/d11/d1e/d9d/dc2/de2/d77/f92 0 2024-07-03T15:44:01.657 INFO:tasks.workunit.client.1.smithi148.stdout:1/880: truncate d0/d2/d11/d3a/dc5/d119/d2f/d72/df9/d42/d62/f4f 8821350 0 2024-07-03T15:44:01.659 INFO:tasks.workunit.client.0.smithi028.stdout:0/125: mknod d2/d3/c26 0 2024-07-03T15:44:01.659 INFO:tasks.workunit.client.0.smithi028.stdout:0/126: stat d2/d3/l13 0 2024-07-03T15:44:01.660 INFO:tasks.workunit.client.1.smithi148.stdout:8/956: mknod d4/da/d14/d17/c148 0 2024-07-03T15:44:01.661 INFO:tasks.workunit.client.0.smithi028.stdout:0/127: creat d2/d3/f27 x:0 0 0 2024-07-03T15:44:01.669 INFO:tasks.workunit.client.1.smithi148.stdout:7/994: creat d5/d7d/da7/d1c/d4a/d66/d93/f157 x:0 0 0 2024-07-03T15:44:01.670 INFO:tasks.workunit.client.1.smithi148.stdout:8/957: dread d4/da/d51/fea [0,4194304] 0 2024-07-03T15:44:01.672 INFO:tasks.workunit.client.1.smithi148.stdout:8/958: unlink d4/da/d51/d18/lb0 0 2024-07-03T15:44:01.673 INFO:tasks.workunit.client.1.smithi148.stdout:8/959: mkdir d4/da/d14/d3a/da8/d149 0 2024-07-03T15:44:01.676 INFO:tasks.workunit.client.1.smithi148.stdout:0/781: getdents da/d58/dbd/d94/dac 0 2024-07-03T15:44:01.676 INFO:tasks.workunit.client.1.smithi148.stdout:0/782: chown da/d58/dbd/d24/d84/fe7 150373 1 2024-07-03T15:44:01.678 INFO:tasks.workunit.client.1.smithi148.stdout:0/783: symlink da/d58/d90/db4/l101 0 2024-07-03T15:44:01.679 INFO:tasks.workunit.client.1.smithi148.stdout:7/995: dread d5/d7d/da7/d1c/d33/f137 [0,4194304] 0 2024-07-03T15:44:01.679 INFO:tasks.workunit.client.1.smithi148.stdout:0/784: link da/d58/dbd/d4a/l69 da/d58/dbd/d13/d14/d50/l102 0 2024-07-03T15:44:01.680 INFO:tasks.workunit.client.1.smithi148.stdout:0/785: rename da/d58/dbd/d94/ff1 to da/d58/dbd/d4a/f103 0 2024-07-03T15:44:01.684 INFO:tasks.workunit.client.0.smithi028.stdout:1/284: dwrite d4/d2c/d3a/f68 [0,4194304] 0 2024-07-03T15:44:01.684 INFO:tasks.workunit.client.1.smithi148.stdout:1/881: dwrite d0/d2/d11/d3a/dc5/d119/d44/d54/d5d/fc1 [0,4194304] 0 2024-07-03T15:44:01.689 INFO:tasks.workunit.client.1.smithi148.stdout:1/882: symlink d0/d2/d11/d3a/dc5/d119/d44/d54/d5d/l124 0 2024-07-03T15:44:01.689 INFO:tasks.workunit.client.1.smithi148.stdout:1/883: write d0/d2/d11/d3a/dc5/d116/ff2 [750981,74002] 0 2024-07-03T15:44:01.692 INFO:tasks.workunit.client.0.smithi028.stdout:8/145: sync 2024-07-03T15:44:01.693 INFO:tasks.workunit.client.0.smithi028.stdout:1/285: rename d4/d11 to d4/d3c/d54/d79 0 2024-07-03T15:44:01.693 INFO:tasks.workunit.client.0.smithi028.stdout:1/286: readlink d4/la 0 2024-07-03T15:44:01.696 INFO:tasks.workunit.client.0.smithi028.stdout:8/146: getdents d0/dd 0 2024-07-03T15:44:01.697 INFO:tasks.workunit.client.0.smithi028.stdout:1/287: symlink d4/d3c/d54/d79/l7a 0 2024-07-03T15:44:01.698 INFO:tasks.workunit.client.1.smithi148.stdout:7/996: dwrite d5/d7d/da7/db4/f154 [0,4194304] 0 2024-07-03T15:44:01.699 INFO:tasks.workunit.client.0.smithi028.stdout:8/147: getdents d0/d2/d3/dc 0 2024-07-03T15:44:01.699 INFO:tasks.workunit.client.0.smithi028.stdout:8/148: fdatasync d0/d2/d3/dc/f23 0 2024-07-03T15:44:01.699 INFO:tasks.workunit.client.0.smithi028.stdout:8/149: dread - d0/d2/d3/dc/f23 zero size 2024-07-03T15:44:01.699 INFO:tasks.workunit.client.0.smithi028.stdout:8/150: chown d0/d2/c19 14320741 1 2024-07-03T15:44:01.699 INFO:tasks.workunit.client.0.smithi028.stdout:8/151: chown d0/d2/d3/dc/c18 0 1 2024-07-03T15:44:01.699 INFO:tasks.workunit.client.0.smithi028.stdout:8/152: stat d0/f2a 0 2024-07-03T15:44:01.700 INFO:tasks.workunit.client.0.smithi028.stdout:1/288: creat d4/d3c/f7b x:0 0 0 2024-07-03T15:44:01.700 INFO:tasks.workunit.client.1.smithi148.stdout:7/997: write d5/d7d/da7/d1c/d59/f7c [1372124,34308] 0 2024-07-03T15:44:01.702 INFO:tasks.workunit.client.0.smithi028.stdout:8/153: rmdir d0/d2/d3 39 2024-07-03T15:44:01.702 INFO:tasks.workunit.client.0.smithi028.stdout:8/154: getdents d0/d2/d3/dc 0 2024-07-03T15:44:01.702 INFO:tasks.workunit.client.0.smithi028.stdout:8/155: readlink d0/d2/d3/dc/d11/d12/l1b 0 2024-07-03T15:44:01.702 INFO:tasks.workunit.client.0.smithi028.stdout:8/156: truncate d0/f2a 288246 0 2024-07-03T15:44:01.702 INFO:tasks.workunit.client.0.smithi028.stdout:8/157: dread - d0/d2/d3/dc/f21 zero size 2024-07-03T15:44:01.703 INFO:tasks.workunit.client.0.smithi028.stdout:1/289: mkdir d4/d3c/d54/d79/d16/d7c 0 2024-07-03T15:44:01.703 INFO:tasks.workunit.client.0.smithi028.stdout:1/290: getdents d4/d3c/d54/d79/d16/d7c 0 2024-07-03T15:44:01.703 INFO:tasks.workunit.client.0.smithi028.stdout:1/291: write d4/d3c/d54/d79/d16/f27 [522702,84190] 0 2024-07-03T15:44:01.703 INFO:tasks.workunit.client.0.smithi028.stdout:1/292: write d4/d3c/d54/d79/f12 [257449,8689] 0 2024-07-03T15:44:01.703 INFO:tasks.workunit.client.0.smithi028.stdout:1/293: fdatasync d4/fd 0 2024-07-03T15:44:01.704 INFO:tasks.workunit.client.0.smithi028.stdout:1/294: chown d4/de/d1e/l34 12 1 2024-07-03T15:44:01.705 INFO:tasks.workunit.client.0.smithi028.stdout:1/295: rmdir d4/d2c/d3a/d43/d70 0 2024-07-03T15:44:01.706 INFO:tasks.workunit.client.1.smithi148.stdout:1/884: mknod d0/d2/d11/d3a/dc5/d119/d2f/d72/df9/c125 0 2024-07-03T15:44:01.710 INFO:tasks.workunit.client.0.smithi028.stdout:1/296: getdents d4/de/d1e/d59 0 2024-07-03T15:44:01.710 INFO:tasks.workunit.client.0.smithi028.stdout:1/297: truncate d4/de/f31 750495 0 2024-07-03T15:44:01.711 INFO:tasks.workunit.client.0.smithi028.stdout:1/298: write d4/d3c/d54/d79/d16/f27 [33869,96417] 0 2024-07-03T15:44:01.721 INFO:tasks.workunit.client.1.smithi148.stdout:1/885: dread d0/d2/ff0 [0,4194304] 0 2024-07-03T15:44:01.728 INFO:tasks.workunit.client.0.smithi028.stdout:8/158: dwrite d0/d2/d3/dc/d11/d12/f15 [4194304,4194304] 0 2024-07-03T15:44:01.729 INFO:tasks.workunit.client.0.smithi028.stdout:8/159: mkdir d0/d2/d3/d35/d38 0 2024-07-03T15:44:01.730 INFO:tasks.workunit.client.0.smithi028.stdout:8/160: rename d0/d2/f1a to d0/d2/d3/dc/d11/d12/f39 0 2024-07-03T15:44:01.731 INFO:tasks.workunit.client.0.smithi028.stdout:8/161: chown d0/d2/d3/dc/d11/d12/f39 676959411 1 2024-07-03T15:44:01.731 INFO:tasks.workunit.client.1.smithi148.stdout:7/998: dwrite d5/d7d/da7/d1c/d33/d4c/d21/d31/d7f/d97/d65/db2/fde [0,4194304] 0 2024-07-03T15:44:01.732 INFO:tasks.workunit.client.0.smithi028.stdout:1/299: dwrite d4/f37 [0,4194304] 0 2024-07-03T15:44:01.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:01 smithi148 ceph-mon[36056]: pgmap v243: 65 pgs: 65 active+clean; 1.8 GiB data, 6.9 GiB used, 530 GiB / 536 GiB avail; 175 MiB/s rd, 221 MiB/s wr, 583 op/s 2024-07-03T15:44:01.732 INFO:tasks.workunit.client.1.smithi148.stdout:1/886: dread d0/d2/d11/d3a/dc5/d116/f26 [0,4194304] 0 2024-07-03T15:44:01.735 INFO:tasks.workunit.client.1.smithi148.stdout:7/999: symlink d5/d7d/da7/d1c/d33/d4c/d84/d98/l158 0 2024-07-03T15:44:01.739 INFO:tasks.workunit.client.1.smithi148.stdout:1/887: dread d0/d2/dd0/ffe [0,4194304] 0 2024-07-03T15:44:01.740 INFO:tasks.workunit.client.0.smithi028.stdout:1/300: dread d4/f37 [0,4194304] 0 2024-07-03T15:44:01.741 INFO:tasks.workunit.client.0.smithi028.stdout:1/301: truncate d4/f29 6484331 0 2024-07-03T15:44:01.744 INFO:tasks.workunit.client.0.smithi028.stdout:1/302: truncate d4/f37 3569618 0 2024-07-03T15:44:01.746 INFO:tasks.workunit.client.0.smithi028.stdout:8/162: dwrite d0/d2/d3/dc/d11/d12/f15 [4194304,4194304] 0 2024-07-03T15:44:01.749 INFO:tasks.workunit.client.0.smithi028.stdout:8/163: dread d0/d2/d3/dc/d11/d12/f39 [0,4194304] 0 2024-07-03T15:44:01.749 INFO:tasks.workunit.client.0.smithi028.stdout:8/164: write d0/f20 [1747956,53377] 0 2024-07-03T15:44:01.750 INFO:tasks.workunit.client.0.smithi028.stdout:1/303: rename d4/de/d1e to d4/d2c/d3a/d5c/d6c/d7d 0 2024-07-03T15:44:01.752 INFO:tasks.workunit.client.0.smithi028.stdout:8/165: rename d0/d2/d3/dc/d11/f30 to d0/d2/d3/f3a 0 2024-07-03T15:44:01.753 INFO:tasks.workunit.client.0.smithi028.stdout:8/166: symlink d0/dd/l3b 0 2024-07-03T15:44:01.755 INFO:tasks.workunit.client.1.smithi148.stdout:1/888: dwrite d0/d2/d11/d3a/dc5/d119/d2f/d72/df9/d42/d62/f4f [4194304,4194304] 0 2024-07-03T15:44:01.760 INFO:tasks.workunit.client.0.smithi028.stdout:8/167: getdents d0/d2/d3/dc/d11 0 2024-07-03T15:44:01.761 INFO:tasks.workunit.client.0.smithi028.stdout:8/168: write d0/f2a [763403,116063] 0 2024-07-03T15:44:01.761 INFO:tasks.workunit.client.0.smithi028.stdout:8/169: chown d0/d2/f27 1 1 2024-07-03T15:44:01.761 INFO:tasks.workunit.client.1.smithi148.stdout:1/889: write d0/d2/d11/d1e/d9d/fdb [1101952,112698] 0 2024-07-03T15:44:01.765 INFO:tasks.workunit.client.0.smithi028.stdout:1/304: dwrite d4/d2c/d3a/d5c/d6c/d7d/f6e [0,4194304] 0 2024-07-03T15:44:01.765 INFO:tasks.workunit.client.0.smithi028.stdout:1/305: write d4/d2c/d3a/d5c/d6c/d7d/d59/f2a [1050863,105174] 0 2024-07-03T15:44:01.767 INFO:tasks.workunit.client.1.smithi148.stdout:4/918: sync 2024-07-03T15:44:01.767 INFO:tasks.workunit.client.1.smithi148.stdout:2/965: sync 2024-07-03T15:44:01.767 INFO:tasks.workunit.client.1.smithi148.stdout:2/966: truncate d5/f40 4958555 0 2024-07-03T15:44:01.767 INFO:tasks.workunit.client.1.smithi148.stdout:4/919: read d0/f3d [2193053,6234] 0 2024-07-03T15:44:01.768 INFO:tasks.workunit.client.0.smithi028.stdout:8/170: truncate d0/d2/d3/dc/d11/d12/f15 7723836 0 2024-07-03T15:44:01.770 INFO:tasks.workunit.client.0.smithi028.stdout:1/306: mknod d4/de/d1c/d55/c7e 0 2024-07-03T15:44:01.773 INFO:tasks.workunit.client.0.smithi028.stdout:1/307: rmdir d4/d2c/d3a/d5c/d6c/d73 0 2024-07-03T15:44:01.779 INFO:tasks.workunit.client.1.smithi148.stdout:1/890: dwrite d0/d2/d11/d3a/dc5/d119/d2f/d72/df9/f5f [0,4194304] 0 2024-07-03T15:44:01.781 INFO:tasks.workunit.client.1.smithi148.stdout:1/891: symlink d0/d2/d11/d3a/dc5/d119/d2f/l126 0 2024-07-03T15:44:01.781 INFO:tasks.workunit.client.1.smithi148.stdout:1/892: chown d0/d2/d11/d3a/dc5/d119/d2f/d72/cd1 106 1 2024-07-03T15:44:01.782 INFO:tasks.workunit.client.1.smithi148.stdout:4/920: dread d0/d28/d7d/d6f/d7c/dae/dba/fdc [0,4194304] 0 2024-07-03T15:44:01.783 INFO:tasks.workunit.client.1.smithi148.stdout:1/893: unlink d0/f1f 0 2024-07-03T15:44:01.783 INFO:tasks.workunit.client.1.smithi148.stdout:1/894: fsync d0/d2/d11/d1e/d9d/dc2/de2/f69 0 2024-07-03T15:44:01.783 INFO:tasks.workunit.client.1.smithi148.stdout:4/921: rmdir d0/d8/d44 39 2024-07-03T15:44:01.784 INFO:tasks.workunit.client.1.smithi148.stdout:1/895: symlink d0/d2/d11/d3a/dc5/d116/l127 0 2024-07-03T15:44:01.785 INFO:tasks.workunit.client.1.smithi148.stdout:4/922: mknod d0/d8/d44/dc2/c143 0 2024-07-03T15:44:01.785 INFO:tasks.workunit.client.1.smithi148.stdout:4/923: write d0/d8/d3a/db8/dd3/deb/fed [94705,91506] 0 2024-07-03T15:44:01.787 INFO:tasks.workunit.client.0.smithi028.stdout:1/308: dread d4/d3c/d54/d79/d16/d57/d66/f67 [0,4194304] 0 2024-07-03T15:44:01.797 INFO:tasks.workunit.client.1.smithi148.stdout:1/896: write d0/d2/d11/d3a/dc5/d119/f83 [10134344,3226] 0 2024-07-03T15:44:01.797 INFO:tasks.workunit.client.1.smithi148.stdout:1/897: read d0/d2/fca [2407926,43462] 0 2024-07-03T15:44:01.797 INFO:tasks.workunit.client.1.smithi148.stdout:1/898: chown d0/d2/dae/dfc 500491317 1 2024-07-03T15:44:01.803 INFO:tasks.workunit.client.1.smithi148.stdout:2/967: dwrite d5/d14/d22/dc0/dfb/d79/de4/f12b [0,4194304] 0 2024-07-03T15:44:01.805 INFO:tasks.workunit.client.0.smithi028.stdout:8/171: dwrite d0/f20 [0,4194304] 0 2024-07-03T15:44:01.807 INFO:tasks.workunit.client.0.smithi028.stdout:8/172: getdents d0/d2/d3 0 2024-07-03T15:44:01.808 INFO:tasks.workunit.client.0.smithi028.stdout:8/173: creat d0/d2/d3/f3c x:0 0 0 2024-07-03T15:44:01.809 INFO:tasks.workunit.client.0.smithi028.stdout:8/174: fdatasync d0/d2/d3/f3a 0 2024-07-03T15:44:01.809 INFO:tasks.workunit.client.0.smithi028.stdout:8/175: getdents d0/d2/d3/dc/d33 0 2024-07-03T15:44:01.809 INFO:tasks.workunit.client.0.smithi028.stdout:8/176: dread - d0/d2/d3/f3c zero size 2024-07-03T15:44:01.811 INFO:tasks.workunit.client.1.smithi148.stdout:1/899: dread d0/d2/d11/d3a/dc5/d119/d2f/d72/df9/d42/f48 [0,4194304] 0 2024-07-03T15:44:01.811 INFO:tasks.workunit.client.1.smithi148.stdout:1/900: fdatasync d0/d2/d11/d3a/dc5/d119/d2f/d72/f7c 0 2024-07-03T15:44:01.811 INFO:tasks.workunit.client.1.smithi148.stdout:1/901: readlink d0/d2/d11/d3a/dc5/d119/d2f/d72/lba 0 2024-07-03T15:44:01.811 INFO:tasks.workunit.client.1.smithi148.stdout:1/902: readlink d0/d2/d11/l6c 0 2024-07-03T15:44:01.811 INFO:tasks.workunit.client.1.smithi148.stdout:1/903: mknod d0/d2/dae/c128 0 2024-07-03T15:44:01.812 INFO:tasks.workunit.client.0.smithi028.stdout:8/177: symlink d0/d2/d3/l3d 0 2024-07-03T15:44:01.812 INFO:tasks.workunit.client.0.smithi028.stdout:8/178: fsync d0/d2/d3/dc/f21 0 2024-07-03T15:44:01.812 INFO:tasks.workunit.client.0.smithi028.stdout:8/179: rename d0/cb to d0/d2/d3/dc/d11/d12/c3e 0 2024-07-03T15:44:01.814 INFO:tasks.workunit.client.0.smithi028.stdout:8/180: rename d0/dd/d37 to d0/d2/d3/dc/d3f 0 2024-07-03T15:44:01.816 INFO:tasks.workunit.client.0.smithi028.stdout:8/181: rmdir d0/dd/d22 39 2024-07-03T15:44:01.824 INFO:tasks.workunit.client.1.smithi148.stdout:4/924: dwrite d0/d28/da7/d2f/f130 [0,4194304] 0 2024-07-03T15:44:01.825 INFO:tasks.workunit.client.0.smithi028.stdout:1/309: dwrite d4/de/f31 [0,4194304] 0 2024-07-03T15:44:01.825 INFO:tasks.workunit.client.1.smithi148.stdout:2/968: dwrite d5/d67/ffa [0,4194304] 0 2024-07-03T15:44:01.830 INFO:tasks.workunit.client.0.smithi028.stdout:8/182: dread d0/d2/d3/dc/d11/d12/f2f [4194304,4194304] 0 2024-07-03T15:44:01.830 INFO:tasks.workunit.client.0.smithi028.stdout:8/183: truncate d0/d2/d3/dc/f23 515120 0 2024-07-03T15:44:01.830 INFO:tasks.workunit.client.0.smithi028.stdout:8/184: chown d0/dd/l1c 233518 1 2024-07-03T15:44:01.830 INFO:tasks.workunit.client.0.smithi028.stdout:8/185: fsync d0/d2/d3/dc/d11/d12/f15 0 2024-07-03T15:44:01.831 INFO:tasks.workunit.client.0.smithi028.stdout:8/186: dread d0/d2/d3/f3a [0,4194304] 0 2024-07-03T15:44:01.833 INFO:tasks.workunit.client.0.smithi028.stdout:8/187: unlink d0/l26 0 2024-07-03T15:44:01.834 INFO:tasks.workunit.client.0.smithi028.stdout:1/310: write d4/f19 [3751290,16960] 0 2024-07-03T15:44:01.835 INFO:tasks.workunit.client.1.smithi148.stdout:4/925: mkdir d0/d28/de8/d131/d144 0 2024-07-03T15:44:01.835 INFO:tasks.workunit.client.1.smithi148.stdout:4/926: write d0/d8/d3a/dce/df7/f104 [835538,43353] 0 2024-07-03T15:44:01.835 INFO:tasks.workunit.client.1.smithi148.stdout:4/927: stat d0/d8/dd4/f140 0 2024-07-03T15:44:01.836 INFO:tasks.workunit.client.0.smithi028.stdout:8/188: mkdir d0/dd/d22/d40 0 2024-07-03T15:44:01.838 INFO:tasks.workunit.client.0.smithi028.stdout:1/311: mkdir d4/d2c/d3a/d5c/d6c/d7d/d59/d7f 0 2024-07-03T15:44:01.839 INFO:tasks.workunit.client.1.smithi148.stdout:2/969: creat d5/d14/d86/db9/dbb/d13/d1d/da1/f133 x:0 0 0 2024-07-03T15:44:01.841 INFO:tasks.workunit.client.1.smithi148.stdout:4/928: creat d0/d28/d7d/f145 x:0 0 0 2024-07-03T15:44:01.841 INFO:tasks.workunit.client.1.smithi148.stdout:4/929: chown d0/d8/d13/d48/f81 1796750 1 2024-07-03T15:44:01.841 INFO:tasks.workunit.client.1.smithi148.stdout:4/930: creat d0/d28/d7d/d6f/d7c/dae/f146 x:0 0 0 2024-07-03T15:44:01.842 INFO:tasks.workunit.client.0.smithi028.stdout:8/189: rename d0/d2/d3/d35/d38 to d0/d41 0 2024-07-03T15:44:01.843 INFO:tasks.workunit.client.1.smithi148.stdout:2/970: creat d5/da8/f134 x:0 0 0 2024-07-03T15:44:01.846 INFO:tasks.workunit.client.1.smithi148.stdout:2/971: unlink d5/d14/d86/db9/dbb/d68/dcd/d64/f71 0 2024-07-03T15:44:01.850 INFO:tasks.workunit.client.0.smithi028.stdout:1/312: dread d4/f29 [0,4194304] 0 2024-07-03T15:44:01.855 INFO:tasks.workunit.client.1.smithi148.stdout:2/972: write d5/d14/d22/dc0/dfb/d7e/f82 [1269305,4401] 0 2024-07-03T15:44:01.855 INFO:tasks.workunit.client.1.smithi148.stdout:2/973: creat d5/d14/d22/d60/d89/ddb/f135 x:0 0 0 2024-07-03T15:44:01.856 INFO:tasks.workunit.client.0.smithi028.stdout:8/190: dread d0/f20 [0,4194304] 0 2024-07-03T15:44:01.856 INFO:tasks.workunit.client.1.smithi148.stdout:2/974: mknod d5/d14/d22/dc0/dfb/d7e/c136 0 2024-07-03T15:44:01.857 INFO:tasks.workunit.client.0.smithi028.stdout:8/191: creat d0/d2/d3/dc/d11/f42 x:0 0 0 2024-07-03T15:44:01.858 INFO:tasks.workunit.client.1.smithi148.stdout:2/975: mkdir d5/d14/d22/dc0/dfb/d79/d9c/d137 0 2024-07-03T15:44:01.859 INFO:tasks.workunit.client.0.smithi028.stdout:8/192: symlink d0/d2/d3/dc/d33/l43 0 2024-07-03T15:44:01.860 INFO:tasks.workunit.client.0.smithi028.stdout:1/313: dread d4/fd [0,4194304] 0 2024-07-03T15:44:01.860 INFO:tasks.workunit.client.0.smithi028.stdout:8/193: link d0/d2/d3/dc/f2b d0/f44 0 2024-07-03T15:44:01.861 INFO:tasks.workunit.client.0.smithi028.stdout:1/314: mknod d4/d2c/d3a/d5c/d6c/d7d/d59/c80 0 2024-07-03T15:44:01.862 INFO:tasks.workunit.client.0.smithi028.stdout:8/194: mkdir d0/dd/d22/d40/d45 0 2024-07-03T15:44:01.862 INFO:tasks.workunit.client.0.smithi028.stdout:1/315: mkdir d4/de/d1c/d81 0 2024-07-03T15:44:01.863 INFO:tasks.workunit.client.1.smithi148.stdout:0/786: sync 2024-07-03T15:44:01.863 INFO:tasks.workunit.client.1.smithi148.stdout:0/787: write da/d58/dbd/d24/dc8/d8a/fa8 [634098,85699] 0 2024-07-03T15:44:01.863 INFO:tasks.workunit.client.1.smithi148.stdout:8/960: sync 2024-07-03T15:44:01.863 INFO:tasks.workunit.client.0.smithi028.stdout:8/195: mkdir d0/d2/d3/d35/d46 0 2024-07-03T15:44:01.869 INFO:tasks.workunit.client.1.smithi148.stdout:4/931: dwrite d0/d8/d13/d67/fe1 [0,4194304] 0 2024-07-03T15:44:01.870 INFO:tasks.workunit.client.0.smithi028.stdout:1/316: dread d4/d3c/d54/d79/f3e [0,4194304] 0 2024-07-03T15:44:01.871 INFO:tasks.workunit.client.1.smithi148.stdout:2/976: dread d5/da8/dde/f128 [0,4194304] 0 2024-07-03T15:44:01.871 INFO:tasks.workunit.client.1.smithi148.stdout:2/977: truncate d5/d14/d22/dc0/dfb/d79/de4/ff1 479268 0 2024-07-03T15:44:01.871 INFO:tasks.workunit.client.1.smithi148.stdout:2/978: chown d5/d67/dc3/ddd/le6 445750638 1 2024-07-03T15:44:01.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:01 smithi028 ceph-mon[29148]: pgmap v243: 65 pgs: 65 active+clean; 1.8 GiB data, 6.9 GiB used, 530 GiB / 536 GiB avail; 175 MiB/s rd, 221 MiB/s wr, 583 op/s 2024-07-03T15:44:01.874 INFO:tasks.workunit.client.1.smithi148.stdout:2/979: getdents d5/d14 0 2024-07-03T15:44:01.876 INFO:tasks.workunit.client.1.smithi148.stdout:0/788: read da/d58/dbd/d13/d14/f61 [2435454,99467] 0 2024-07-03T15:44:01.877 INFO:tasks.workunit.client.1.smithi148.stdout:0/789: chown da/d17/fe8 8737563 1 2024-07-03T15:44:01.877 INFO:tasks.workunit.client.1.smithi148.stdout:2/980: dread d5/d14/f10c [0,4194304] 0 2024-07-03T15:44:01.877 INFO:tasks.workunit.client.1.smithi148.stdout:2/981: fsync d5/d14/d22/d60/d76/f92 0 2024-07-03T15:44:01.878 INFO:tasks.workunit.client.0.smithi028.stdout:9/164: sync 2024-07-03T15:44:01.878 INFO:tasks.workunit.client.0.smithi028.stdout:5/167: sync 2024-07-03T15:44:01.878 INFO:tasks.workunit.client.0.smithi028.stdout:5/168: readlink d0/dd/l24 0 2024-07-03T15:44:01.878 INFO:tasks.workunit.client.0.smithi028.stdout:9/165: fdatasync d1/f31 0 2024-07-03T15:44:01.879 INFO:tasks.workunit.client.0.smithi028.stdout:7/69: sync 2024-07-03T15:44:01.879 INFO:tasks.workunit.client.0.smithi028.stdout:9/166: dread - d1/d16/f2f zero size 2024-07-03T15:44:01.879 INFO:tasks.workunit.client.0.smithi028.stdout:5/169: truncate d0/dd/d13/d19/d26/f2e 750269 0 2024-07-03T15:44:01.879 INFO:tasks.workunit.client.0.smithi028.stdout:9/167: chown d1/lc 0 1 2024-07-03T15:44:01.879 INFO:tasks.workunit.client.0.smithi028.stdout:5/170: dread - d0/dd/d13/d19/f1f zero size 2024-07-03T15:44:01.879 INFO:tasks.workunit.client.0.smithi028.stdout:7/70: truncate f4 4966987 0 2024-07-03T15:44:01.879 INFO:tasks.workunit.client.0.smithi028.stdout:5/171: write d0/dd/d13/f39 [383197,94019] 0 2024-07-03T15:44:01.879 INFO:tasks.workunit.client.0.smithi028.stdout:5/172: chown d0/dd/d13/d29/c34 24475711 1 2024-07-03T15:44:01.882 INFO:tasks.workunit.client.0.smithi028.stdout:9/168: unlink d1/dd/fe 0 2024-07-03T15:44:01.882 INFO:tasks.workunit.client.0.smithi028.stdout:7/71: rmdir d9 39 2024-07-03T15:44:01.882 INFO:tasks.workunit.client.0.smithi028.stdout:7/72: fdatasync f2 0 2024-07-03T15:44:01.882 INFO:tasks.workunit.client.0.smithi028.stdout:7/73: truncate d9/fe 489092 0 2024-07-03T15:44:01.883 INFO:tasks.workunit.client.0.smithi028.stdout:5/173: fdatasync d0/dd/d13/f39 0 2024-07-03T15:44:01.883 INFO:tasks.workunit.client.0.smithi028.stdout:5/174: chown d0 12168741 1 2024-07-03T15:44:01.885 INFO:tasks.workunit.client.0.smithi028.stdout:9/169: link d1/d10/f24 d1/dd/f35 0 2024-07-03T15:44:01.885 INFO:tasks.workunit.client.0.smithi028.stdout:9/170: read - d1/d10/f24 zero size 2024-07-03T15:44:01.885 INFO:tasks.workunit.client.0.smithi028.stdout:9/171: dread - d1/d16/f2f zero size 2024-07-03T15:44:01.886 INFO:tasks.workunit.client.0.smithi028.stdout:8/196: dwrite d0/d2/d3/dc/f21 [0,4194304] 0 2024-07-03T15:44:01.886 INFO:tasks.workunit.client.0.smithi028.stdout:8/197: dread - d0/f44 zero size 2024-07-03T15:44:01.890 INFO:tasks.workunit.client.0.smithi028.stdout:5/175: getdents d0/dd/d13/d19/d30 0 2024-07-03T15:44:01.892 INFO:tasks.workunit.client.0.smithi028.stdout:5/176: getdents d0/dd/df 0 2024-07-03T15:44:01.894 INFO:tasks.workunit.client.0.smithi028.stdout:5/177: unlink d0/dd/d14/c2b 0 2024-07-03T15:44:01.894 INFO:tasks.workunit.client.0.smithi028.stdout:5/178: stat d0/dd/d13/d19/d26/f2e 0 2024-07-03T15:44:01.894 INFO:tasks.workunit.client.0.smithi028.stdout:5/179: write d0/dd/f1b [3448576,56500] 0 2024-07-03T15:44:01.895 INFO:tasks.workunit.client.0.smithi028.stdout:5/180: mkdir d0/dd/d1e/d3b 0 2024-07-03T15:44:01.896 INFO:tasks.workunit.client.0.smithi028.stdout:1/317: dwrite d4/de/f31 [4194304,4194304] 0 2024-07-03T15:44:01.896 INFO:tasks.workunit.client.0.smithi028.stdout:8/198: dread d0/d2/f27 [0,4194304] 0 2024-07-03T15:44:01.896 INFO:tasks.workunit.client.0.smithi028.stdout:8/199: dread - d0/d2/d3/dc/f34 zero size 2024-07-03T15:44:01.896 INFO:tasks.workunit.client.0.smithi028.stdout:5/181: rename d0/dd/d13/d29/l2d to d0/dd/d13/d19/l3c 0 2024-07-03T15:44:01.898 INFO:tasks.workunit.client.0.smithi028.stdout:8/200: rmdir d0/d2/d3/dc 39 2024-07-03T15:44:01.900 INFO:tasks.workunit.client.0.smithi028.stdout:5/182: unlink d0/dd/d13/d19/c31 0 2024-07-03T15:44:01.901 INFO:tasks.workunit.client.0.smithi028.stdout:5/183: mknod d0/dd/d1e/d3b/c3d 0 2024-07-03T15:44:01.902 INFO:tasks.workunit.client.0.smithi028.stdout:5/184: rename d0/dd/d13/f25 to d0/dd/df/f3e 0 2024-07-03T15:44:01.903 INFO:tasks.workunit.client.0.smithi028.stdout:5/185: chown d0/dd/d13 11182 1 2024-07-03T15:44:01.903 INFO:tasks.workunit.client.0.smithi028.stdout:5/186: write d0/dd/d13/f39 [1214924,23566] 0 2024-07-03T15:44:01.908 INFO:tasks.workunit.client.0.smithi028.stdout:1/318: read d4/d3c/d54/d79/d16/d57/d66/f67 [3492674,106978] 0 2024-07-03T15:44:01.927 INFO:tasks.workunit.client.1.smithi148.stdout:0/790: dwrite da/d17/d2a/f59 [0,4194304] 0 2024-07-03T15:44:01.928 INFO:tasks.workunit.client.1.smithi148.stdout:0/791: link da/d17/fe8 da/d17/d2a/d2f/d93/f104 0 2024-07-03T15:44:01.929 INFO:tasks.workunit.client.0.smithi028.stdout:5/187: fsync d0/dd/d13/f39 0 2024-07-03T15:44:01.929 INFO:tasks.workunit.client.0.smithi028.stdout:9/172: dwrite d1/d10/f24 [0,4194304] 0 2024-07-03T15:44:01.929 INFO:tasks.workunit.client.0.smithi028.stdout:8/201: dwrite d0/f44 [0,4194304] 0 2024-07-03T15:44:01.929 INFO:tasks.workunit.client.0.smithi028.stdout:8/202: chown d0/d2/d3/dc/d11/d12/c3e 458 1 2024-07-03T15:44:01.929 INFO:tasks.workunit.client.0.smithi028.stdout:8/203: chown d0/d2/d3/dc/d11/d12/f2f 753925 1 2024-07-03T15:44:01.930 INFO:tasks.workunit.client.1.smithi148.stdout:2/982: dwrite d5/f40 [4194304,4194304] 0 2024-07-03T15:44:01.930 INFO:tasks.workunit.client.1.smithi148.stdout:2/983: write d5/d14/d22/f72 [4938240,104640] 0 2024-07-03T15:44:01.930 INFO:tasks.workunit.client.1.smithi148.stdout:0/792: creat da/d58/dbd/d94/dac/ddd/d88/f105 x:0 0 0 2024-07-03T15:44:01.936 INFO:tasks.workunit.client.1.smithi148.stdout:2/984: creat d5/d14/d8f/f138 x:0 0 0 2024-07-03T15:44:01.938 INFO:tasks.workunit.client.0.smithi028.stdout:9/173: mknod d1/d2a/c36 0 2024-07-03T15:44:01.940 INFO:tasks.workunit.client.1.smithi148.stdout:0/793: creat da/d58/dbd/f106 x:0 0 0 2024-07-03T15:44:01.940 INFO:tasks.workunit.client.1.smithi148.stdout:0/794: chown da/d17/d2a/d2f/d93/da3/da7/fdb 282753068 1 2024-07-03T15:44:01.942 INFO:tasks.workunit.client.0.smithi028.stdout:9/174: unlink d1/f27 0 2024-07-03T15:44:01.942 INFO:tasks.workunit.client.0.smithi028.stdout:9/175: dread - d1/f31 zero size 2024-07-03T15:44:01.945 INFO:tasks.workunit.client.1.smithi148.stdout:0/795: creat da/d58/dbd/d13/d1d/d3e/d85/d8e/dd4/f107 x:0 0 0 2024-07-03T15:44:01.945 INFO:tasks.workunit.client.1.smithi148.stdout:0/796: creat da/d58/dbd/d4a/f108 x:0 0 0 2024-07-03T15:44:01.945 INFO:tasks.workunit.client.1.smithi148.stdout:0/797: chown da/d58/dbd/d24/dc8 0 1 2024-07-03T15:44:01.952 INFO:tasks.workunit.client.1.smithi148.stdout:1/904: sync 2024-07-03T15:44:01.953 INFO:tasks.workunit.client.1.smithi148.stdout:2/985: dwrite d5/d14/d22/d60/f65 [0,4194304] 0 2024-07-03T15:44:01.953 INFO:tasks.workunit.client.0.smithi028.stdout:5/188: dwrite d0/dd/d13/d19/d26/f2e [0,4194304] 0 2024-07-03T15:44:01.954 INFO:tasks.workunit.client.1.smithi148.stdout:1/905: creat d0/d2/d11/d1e/d9d/dc2/f129 x:0 0 0 2024-07-03T15:44:01.954 INFO:tasks.workunit.client.1.smithi148.stdout:1/906: chown d0/d2/d11/d3a/dc5/d119/d2f/d72/df9/d42/d62/f71 1 1 2024-07-03T15:44:01.955 INFO:tasks.workunit.client.1.smithi148.stdout:1/907: read d0/d2/d11/d3a/dc5/d119/d89/f95 [941496,52234] 0 2024-07-03T15:44:01.956 INFO:tasks.workunit.client.1.smithi148.stdout:2/986: rename d5/d14/d22/d60/f91 to d5/d14/d86/db9/dbb/d13/d41/d7b/d6a/f139 0 2024-07-03T15:44:01.958 INFO:tasks.workunit.client.0.smithi028.stdout:5/189: mkdir d0/dd/d1e/d3b/d3f 0 2024-07-03T15:44:01.959 INFO:tasks.workunit.client.0.smithi028.stdout:5/190: mknod d0/dd/d1e/d3b/c40 0 2024-07-03T15:44:01.961 INFO:tasks.workunit.client.0.smithi028.stdout:5/191: dread d0/dd/d13/f39 [0,4194304] 0 2024-07-03T15:44:01.962 INFO:tasks.workunit.client.0.smithi028.stdout:5/192: mkdir d0/dd/d13/d19/d26/d41 0 2024-07-03T15:44:01.963 INFO:tasks.workunit.client.0.smithi028.stdout:5/193: getdents d0/dd/d1e 0 2024-07-03T15:44:01.966 INFO:tasks.workunit.client.1.smithi148.stdout:1/908: dread d0/d2/f75 [4194304,4194304] 0 2024-07-03T15:44:01.966 INFO:tasks.workunit.client.1.smithi148.stdout:1/909: fsync d0/d2/d11/d3a/dc5/d119/d2f/d72/fc6 0 2024-07-03T15:44:01.967 INFO:tasks.workunit.client.1.smithi148.stdout:1/910: symlink d0/d2/d11/d3a/dc5/d119/d2f/d72/df9/d42/l12a 0 2024-07-03T15:44:01.967 INFO:tasks.workunit.client.1.smithi148.stdout:1/911: write d0/d2/d11/d1e/d9d/dc2/de2/f63 [4548748,2664] 0 2024-07-03T15:44:01.969 INFO:tasks.workunit.client.1.smithi148.stdout:1/912: unlink d0/d2/d11/d3a/dc5/d116/d14/c37 0 2024-07-03T15:44:01.972 INFO:tasks.workunit.client.1.smithi148.stdout:0/798: dwrite da/d58/dbd/d94/dba/fd7 [4194304,4194304] 0 2024-07-03T15:44:01.972 INFO:tasks.workunit.client.1.smithi148.stdout:2/987: dread d5/d67/ffa [0,4194304] 0 2024-07-03T15:44:01.972 INFO:tasks.workunit.client.1.smithi148.stdout:0/799: fdatasync da/d58/dbd/d13/f27 0 2024-07-03T15:44:01.972 INFO:tasks.workunit.client.1.smithi148.stdout:0/800: fsync da/d58/dbd/d13/fce 0 2024-07-03T15:44:01.972 INFO:tasks.workunit.client.1.smithi148.stdout:2/988: write d5/d14/d86/d93/fd0 [224456,104560] 0 2024-07-03T15:44:01.972 INFO:tasks.workunit.client.1.smithi148.stdout:2/989: stat d5/d14/d86/d93/fd0 0 2024-07-03T15:44:01.974 INFO:tasks.workunit.client.1.smithi148.stdout:0/801: mkdir da/d58/d90/dd1/d109 0 2024-07-03T15:44:01.975 INFO:tasks.workunit.client.1.smithi148.stdout:0/802: creat da/d58/dbd/d13/d1d/d3e/d85/d8e/f10a x:0 0 0 2024-07-03T15:44:01.977 INFO:tasks.workunit.client.1.smithi148.stdout:2/990: write d5/d14/d86/db9/dbb/d68/dcd/f38 [904710,108415] 0 2024-07-03T15:44:01.979 INFO:tasks.workunit.client.1.smithi148.stdout:2/991: dread d5/d14/d86/db9/dbb/d13/d1d/da1/fa2 [0,4194304] 0 2024-07-03T15:44:01.979 INFO:tasks.workunit.client.1.smithi148.stdout:2/992: chown d5/d67/dc3/f123 472 1 2024-07-03T15:44:01.980 INFO:tasks.workunit.client.1.smithi148.stdout:2/993: link d5/d14/d8b/dd5/f10b d5/da8/dde/d11b/f13a 0 2024-07-03T15:44:01.981 INFO:tasks.workunit.client.1.smithi148.stdout:2/994: mknod d5/d14/d22/dc0/dfb/d7e/d117/c13b 0 2024-07-03T15:44:01.982 INFO:tasks.workunit.client.1.smithi148.stdout:1/913: read d0/d2/d11/d3a/dc5/d116/fa [3916211,84007] 0 2024-07-03T15:44:01.982 INFO:tasks.workunit.client.0.smithi028.stdout:9/176: dwrite d1/f2d [0,4194304] 0 2024-07-03T15:44:01.983 INFO:tasks.workunit.client.0.smithi028.stdout:5/194: dwrite d0/dd/d13/f39 [0,4194304] 0 2024-07-03T15:44:01.983 INFO:tasks.workunit.client.0.smithi028.stdout:5/195: write d0/dd/f1b [744843,26450] 0 2024-07-03T15:44:01.984 INFO:tasks.workunit.client.1.smithi148.stdout:1/914: symlink d0/d2/d11/d1e/d9d/dc2/de2/d77/da3/l12b 0 2024-07-03T15:44:01.984 INFO:tasks.workunit.client.1.smithi148.stdout:1/915: chown d0/d2/d11/d3a/dc5/d119/d44/d54/fb6 2 1 2024-07-03T15:44:01.989 INFO:tasks.workunit.client.1.smithi148.stdout:1/916: mkdir d0/d2/d11/d3a/dc5/d119/d2f/d72/df9/d12c 0 2024-07-03T15:44:01.989 INFO:tasks.workunit.client.1.smithi148.stdout:0/803: write da/d58/dbd/d13/d14/f6a [1171819,128735] 0 2024-07-03T15:44:01.992 INFO:tasks.workunit.client.1.smithi148.stdout:1/917: mkdir d0/d2/d11/d1e/d9d/dc2/de2/d12d 0 2024-07-03T15:44:01.993 INFO:tasks.workunit.client.1.smithi148.stdout:2/995: link d5/d14/d86/db9/dbb/d13/d41/d7b/d6a/f139 d5/d14/d22/dc0/dfb/d79/de4/f13c 0 2024-07-03T15:44:01.994 INFO:tasks.workunit.client.1.smithi148.stdout:2/996: chown d5/d14/d86/db9/dbb/d13/d2d/f3b 585005058 1 2024-07-03T15:44:01.996 INFO:tasks.workunit.client.1.smithi148.stdout:1/918: symlink d0/d2/d11/d3a/dc5/d119/d2f/d72/df9/d42/db7/l12e 0 2024-07-03T15:44:02.004 INFO:tasks.workunit.client.1.smithi148.stdout:1/919: write d0/d2/d11/d1e/d9d/dc2/de2/fcd [472464,89988] 0 2024-07-03T15:44:02.007 INFO:tasks.workunit.client.1.smithi148.stdout:1/920: mknod d0/d2/d11/d3a/dc5/d119/d44/d54/d5d/d10d/c12f 0 2024-07-03T15:44:02.008 INFO:tasks.workunit.client.1.smithi148.stdout:1/921: getdents d0/d2/d11/d3a/dc5/d119/d44/d54/d105/dc7 0 2024-07-03T15:44:02.011 INFO:tasks.workunit.client.1.smithi148.stdout:1/922: link d0/d2/d11/d3a/dc5/d119/d2f/d72/cd1 d0/d2/d11/d3a/dc5/d119/d44/d54/c130 0 2024-07-03T15:44:02.013 INFO:tasks.workunit.client.0.smithi028.stdout:9/177: dwrite d1/dd/f35 [4194304,4194304] 0 2024-07-03T15:44:02.013 INFO:tasks.workunit.client.0.smithi028.stdout:9/178: readlink d1/dd/l23 0 2024-07-03T15:44:02.013 INFO:tasks.workunit.client.0.smithi028.stdout:9/179: stat d1/dd/f35 0 2024-07-03T15:44:02.014 INFO:tasks.workunit.client.0.smithi028.stdout:9/180: symlink d1/dd/l37 0 2024-07-03T15:44:02.017 INFO:tasks.workunit.client.1.smithi148.stdout:0/804: dwrite da/d58/d90/db4/fcc [0,4194304] 0 2024-07-03T15:44:02.018 INFO:tasks.workunit.client.0.smithi028.stdout:5/196: dwrite d0/dd/d13/d19/f21 [0,4194304] 0 2024-07-03T15:44:02.019 INFO:tasks.workunit.client.0.smithi028.stdout:5/197: link d0/dd/df/f16 d0/dd/d13/d19/d26/f42 0 2024-07-03T15:44:02.019 INFO:tasks.workunit.client.0.smithi028.stdout:5/198: write d0/dd/d13/f17 [4878573,81934] 0 2024-07-03T15:44:02.019 INFO:tasks.workunit.client.1.smithi148.stdout:2/997: dwrite d5/d14/d86/fbc [0,4194304] 0 2024-07-03T15:44:02.022 INFO:tasks.workunit.client.0.smithi028.stdout:5/199: creat d0/dd/d13/d19/d26/d41/f43 x:0 0 0 2024-07-03T15:44:02.022 INFO:tasks.workunit.client.0.smithi028.stdout:5/200: creat d0/dd/d13/d19/d26/d41/f44 x:0 0 0 2024-07-03T15:44:02.022 INFO:tasks.workunit.client.0.smithi028.stdout:5/201: truncate d0/dd/d13/d19/d26/d41/f43 724737 0 2024-07-03T15:44:02.023 INFO:tasks.workunit.client.0.smithi028.stdout:5/202: truncate d0/dd/df/f16 1148120 0 2024-07-03T15:44:02.027 INFO:tasks.workunit.client.0.smithi028.stdout:5/203: mknod d0/dd/d1e/d3b/c45 0 2024-07-03T15:44:02.029 INFO:tasks.workunit.client.1.smithi148.stdout:0/805: getdents da/d58/dbd/d13/d1d/da1 0 2024-07-03T15:44:02.029 INFO:tasks.workunit.client.1.smithi148.stdout:0/806: fdatasync da/d58/dbd/d24/dc8/f5e 0 2024-07-03T15:44:02.029 INFO:tasks.workunit.client.1.smithi148.stdout:0/807: chown da/d58/dbd/d4a/f103 7842 1 2024-07-03T15:44:02.029 INFO:tasks.workunit.client.1.smithi148.stdout:0/808: fdatasync da/d58/fed 0 2024-07-03T15:44:02.029 INFO:tasks.workunit.client.1.smithi148.stdout:0/809: chown da/d17/d2a/f59 44963936 1 2024-07-03T15:44:02.029 INFO:tasks.workunit.client.1.smithi148.stdout:0/810: truncate da/d17/d2a/d2f/d93/da3/fc7 414262 0 2024-07-03T15:44:02.030 INFO:tasks.workunit.client.1.smithi148.stdout:0/811: chown da/d58/dbd/d13/d14/f61 342 1 2024-07-03T15:44:02.030 INFO:tasks.workunit.client.1.smithi148.stdout:0/812: chown da/d58/d90/db4/l101 0 1 2024-07-03T15:44:02.030 INFO:tasks.workunit.client.1.smithi148.stdout:0/813: chown da/d58/dbd/d4a 3528 1 2024-07-03T15:44:02.030 INFO:tasks.workunit.client.1.smithi148.stdout:2/998: mknod d5/d14/d86/db9/dbb/d68/c13d 0 2024-07-03T15:44:02.030 INFO:tasks.workunit.client.1.smithi148.stdout:2/999: fsync d5/d14/d86/db9/dbb/f6d 0 2024-07-03T15:44:02.031 INFO:tasks.workunit.client.1.smithi148.stdout:0/814: creat da/d58/dbd/d24/f10b x:0 0 0 2024-07-03T15:44:02.031 INFO:tasks.workunit.client.1.smithi148.stdout:0/815: write da/d58/dbd/d24/d84/fe7 [184621,70457] 0 2024-07-03T15:44:02.031 INFO:tasks.workunit.client.1.smithi148.stdout:4/932: sync 2024-07-03T15:44:02.031 INFO:tasks.workunit.client.1.smithi148.stdout:8/961: sync 2024-07-03T15:44:02.034 INFO:tasks.workunit.client.1.smithi148.stdout:4/933: symlink d0/d8/d3a/d9d/l147 0 2024-07-03T15:44:02.036 INFO:tasks.workunit.client.0.smithi028.stdout:9/181: dwrite d1/dd/f1b [0,4194304] 0 2024-07-03T15:44:02.039 INFO:tasks.workunit.client.1.smithi148.stdout:4/934: rename d0/d28/d7d/d6f/d7c/da3 to d0/d28/da7/d2f/d41/d148 0 2024-07-03T15:44:02.039 INFO:tasks.workunit.client.1.smithi148.stdout:4/935: truncate d0/d8/d3a/dce/d106/f136 5050458 0 2024-07-03T15:44:02.040 INFO:tasks.workunit.client.0.smithi028.stdout:9/182: truncate d1/f2 2624714 0 2024-07-03T15:44:02.040 INFO:tasks.workunit.client.0.smithi028.stdout:9/183: write d1/d16/f2e [510487,113969] 0 2024-07-03T15:44:02.041 INFO:tasks.workunit.client.1.smithi148.stdout:8/962: dread d4/da/d14/d1a/d32/d89/fee [4194304,4194304] 0 2024-07-03T15:44:02.041 INFO:tasks.workunit.client.1.smithi148.stdout:8/963: stat d4/da/d14/d1a/d32 0 2024-07-03T15:44:02.041 INFO:tasks.workunit.client.1.smithi148.stdout:8/964: chown d4/da/d14/d3a/c5d 10648131 1 2024-07-03T15:44:02.041 INFO:tasks.workunit.client.1.smithi148.stdout:4/936: rename d0/d28/da7/d2f/d52/lbe to d0/l149 0 2024-07-03T15:44:02.041 INFO:tasks.workunit.client.0.smithi028.stdout:5/204: write d0/dd/d13/d19/f21 [1393487,98738] 0 2024-07-03T15:44:02.044 INFO:tasks.workunit.client.0.smithi028.stdout:4/121: sync 2024-07-03T15:44:02.044 INFO:tasks.workunit.client.0.smithi028.stdout:3/203: sync 2024-07-03T15:44:02.044 INFO:tasks.workunit.client.0.smithi028.stdout:3/204: read - d18/d1a/d1e/f3e zero size 2024-07-03T15:44:02.044 INFO:tasks.workunit.client.1.smithi148.stdout:8/965: rename d4/da/d51/d18/d20/d82/fa7 to d4/da/d14/d3a/da8/f14a 0 2024-07-03T15:44:02.044 INFO:tasks.workunit.client.1.smithi148.stdout:0/816: dwrite da/d58/dbd/d4a/f103 [0,4194304] 0 2024-07-03T15:44:02.045 INFO:tasks.workunit.client.1.smithi148.stdout:4/937: creat d0/d28/de8/d131/f14a x:0 0 0 2024-07-03T15:44:02.047 INFO:tasks.workunit.client.1.smithi148.stdout:8/966: truncate d4/da/d14/d1a/d32/d38/dba/f77 1347723 0 2024-07-03T15:44:02.053 INFO:tasks.workunit.client.0.smithi028.stdout:3/205: read d18/f2d [2400458,85271] 0 2024-07-03T15:44:02.058 INFO:tasks.workunit.client.1.smithi148.stdout:8/967: rename d4/da/d14/d17/fd5 to d4/da/d14/d1a/d32/f14b 0 2024-07-03T15:44:02.059 INFO:tasks.workunit.client.1.smithi148.stdout:0/817: write da/d58/dbd/d13/d14/f49 [1585703,2602] 0 2024-07-03T15:44:02.059 INFO:tasks.workunit.client.1.smithi148.stdout:8/968: symlink d4/da/d14/l14c 0 2024-07-03T15:44:02.061 INFO:tasks.workunit.client.1.smithi148.stdout:0/818: rmdir da/d58/dbd/d13/d14/d50/de4 39 2024-07-03T15:44:02.062 INFO:tasks.workunit.client.0.smithi028.stdout:9/184: rmdir d1/dd 39 2024-07-03T15:44:02.063 INFO:tasks.workunit.client.1.smithi148.stdout:8/969: mknod d4/da/d14/d3a/dab/c14d 0 2024-07-03T15:44:02.064 INFO:tasks.workunit.client.0.smithi028.stdout:5/205: mknod d0/dd/d13/d19/d26/c46 0 2024-07-03T15:44:02.066 INFO:tasks.workunit.client.1.smithi148.stdout:0/819: creat da/d58/d90/dd1/f10c x:0 0 0 2024-07-03T15:44:02.067 INFO:tasks.workunit.client.0.smithi028.stdout:3/206: unlink f15 0 2024-07-03T15:44:02.067 INFO:tasks.workunit.client.0.smithi028.stdout:3/207: chown d18 69650 1 2024-07-03T15:44:02.067 INFO:tasks.workunit.client.0.smithi028.stdout:3/208: write d18/d3a/f39 [213019,35135] 0 2024-07-03T15:44:02.068 INFO:tasks.workunit.client.1.smithi148.stdout:4/938: dwrite d0/d8/d3a/db8/dd3/df2/ff8 [0,4194304] 0 2024-07-03T15:44:02.068 INFO:tasks.workunit.client.1.smithi148.stdout:4/939: write d0/d8/d3a/db8/dd3/f12f [909274,99466] 0 2024-07-03T15:44:02.068 INFO:tasks.workunit.client.1.smithi148.stdout:8/970: getdents d4/da/d51 0 2024-07-03T15:44:02.068 INFO:tasks.workunit.client.0.smithi028.stdout:4/122: dwrite d2/f27 [0,4194304] 0 2024-07-03T15:44:02.070 INFO:tasks.workunit.client.0.smithi028.stdout:9/185: rename d1/dd/l37 to d1/d10/l38 0 2024-07-03T15:44:02.070 INFO:tasks.workunit.client.0.smithi028.stdout:9/186: chown d1/dd/f1b 2432436 1 2024-07-03T15:44:02.070 INFO:tasks.workunit.client.0.smithi028.stdout:9/187: write d1/f9 [589843,62137] 0 2024-07-03T15:44:02.072 INFO:tasks.workunit.client.1.smithi148.stdout:0/820: rename da/d17/d2a/d2f/d93/da3/fe3 to da/d17/d2a/f10d 0 2024-07-03T15:44:02.072 INFO:tasks.workunit.client.1.smithi148.stdout:0/821: chown da/d58/dbd/d4a/c7f 8281371 1 2024-07-03T15:44:02.072 INFO:tasks.workunit.client.1.smithi148.stdout:0/822: write da/d17/d2a/d2f/ff5 [325828,114411] 0 2024-07-03T15:44:02.073 INFO:tasks.workunit.client.0.smithi028.stdout:5/206: mkdir d0/dd/d13/d47 0 2024-07-03T15:44:02.075 INFO:tasks.workunit.client.0.smithi028.stdout:3/209: mkdir d18/d43 0 2024-07-03T15:44:02.075 INFO:tasks.workunit.client.0.smithi028.stdout:3/210: fdatasync f14 0 2024-07-03T15:44:02.077 INFO:tasks.workunit.client.0.smithi028.stdout:2/181: sync 2024-07-03T15:44:02.077 INFO:tasks.workunit.client.0.smithi028.stdout:6/126: sync 2024-07-03T15:44:02.077 INFO:tasks.workunit.client.0.smithi028.stdout:6/127: chown d4/f29 551019 1 2024-07-03T15:44:02.077 INFO:tasks.workunit.client.0.smithi028.stdout:6/128: write d4/f29 [913487,57820] 0 2024-07-03T15:44:02.077 INFO:tasks.workunit.client.1.smithi148.stdout:4/940: mknod d0/d8/d13/d48/c14b 0 2024-07-03T15:44:02.081 INFO:tasks.workunit.client.0.smithi028.stdout:4/123: symlink d2/l2a 0 2024-07-03T15:44:02.081 INFO:tasks.workunit.client.0.smithi028.stdout:4/124: chown d2/cd 539 1 2024-07-03T15:44:02.081 INFO:tasks.workunit.client.0.smithi028.stdout:4/125: truncate d2/f7 645271 0 2024-07-03T15:44:02.081 INFO:tasks.workunit.client.0.smithi028.stdout:4/126: creat d2/f2b x:0 0 0 2024-07-03T15:44:02.081 INFO:tasks.workunit.client.1.smithi148.stdout:0/823: symlink da/d58/d90/db4/l10e 0 2024-07-03T15:44:02.086 INFO:tasks.workunit.client.1.smithi148.stdout:8/971: rename d4/da/d47 to d4/da/d14/d1a/d32/d14e 0 2024-07-03T15:44:02.086 INFO:tasks.workunit.client.1.smithi148.stdout:8/972: write d4/da/d14/f9b [877141,69489] 0 2024-07-03T15:44:02.086 INFO:tasks.workunit.client.1.smithi148.stdout:8/973: fsync d4/da/d51/f57 0 2024-07-03T15:44:02.087 INFO:tasks.workunit.client.1.smithi148.stdout:8/974: write d4/da/d51/d18/f8c [291071,46352] 0 2024-07-03T15:44:02.087 INFO:tasks.workunit.client.0.smithi028.stdout:3/211: mknod d18/c44 0 2024-07-03T15:44:02.088 INFO:tasks.workunit.client.1.smithi148.stdout:0/824: mkdir da/d58/dbd/d24/dc8/d10f 0 2024-07-03T15:44:02.088 INFO:tasks.workunit.client.1.smithi148.stdout:0/825: chown da/d17/d2a/d2f 0 1 2024-07-03T15:44:02.089 INFO:tasks.workunit.client.0.smithi028.stdout:2/182: mkdir da/d42 0 2024-07-03T15:44:02.090 INFO:tasks.workunit.client.1.smithi148.stdout:4/941: rename d0/d28/db0/fcb to d0/d28/d7d/d6f/da6/f14c 0 2024-07-03T15:44:02.090 INFO:tasks.workunit.client.0.smithi028.stdout:6/129: mkdir d4/d12/d2c 0 2024-07-03T15:44:02.091 INFO:tasks.workunit.client.0.smithi028.stdout:6/130: dread - d4/d12/f21 zero size 2024-07-03T15:44:02.091 INFO:tasks.workunit.client.0.smithi028.stdout:6/131: chown d4/d12/d2c 3296732 1 2024-07-03T15:44:02.091 INFO:tasks.workunit.client.0.smithi028.stdout:6/132: readlink d4/da/l26 0 2024-07-03T15:44:02.096 INFO:tasks.workunit.client.1.smithi148.stdout:8/975: dread d4/da/d14/f67 [0,4194304] 0 2024-07-03T15:44:02.096 INFO:tasks.workunit.client.1.smithi148.stdout:8/976: read - d4/da/d51/fe8 zero size 2024-07-03T15:44:02.096 INFO:tasks.workunit.client.1.smithi148.stdout:8/977: getdents d4/da/d14/d1a/d32/d14e/d88/dce/d12e 0 2024-07-03T15:44:02.097 INFO:tasks.workunit.client.1.smithi148.stdout:8/978: truncate d4/da/d14/d1a/d32/d38/d58/dcb/f10e 957327 0 2024-07-03T15:44:02.097 INFO:tasks.workunit.client.1.smithi148.stdout:8/979: truncate d4/da/d14/d9e/f7d 4520036 0 2024-07-03T15:44:02.107 INFO:tasks.workunit.client.0.smithi028.stdout:6/133: dwrite d4/f11 [4194304,4194304] 0 2024-07-03T15:44:02.184 INFO:tasks.workunit.client.1.smithi148.stdout:0/826: rename da/d58/dbd/d13/d14/d50/de4/lff to da/d17/d2a/d2f/d93/da3/da7/l110 0 2024-07-03T15:44:02.184 INFO:tasks.workunit.client.0.smithi028.stdout:4/127: mkdir d2/d12/d2c 0 2024-07-03T15:44:02.185 INFO:tasks.workunit.client.0.smithi028.stdout:4/128: readlink d2/l2a 0 2024-07-03T15:44:02.185 INFO:tasks.workunit.client.0.smithi028.stdout:4/129: fsync d2/f7 0 2024-07-03T15:44:02.185 INFO:tasks.workunit.client.1.smithi148.stdout:4/942: truncate d0/d8/d13/d48/f2c 130333 0 2024-07-03T15:44:02.186 INFO:tasks.workunit.client.0.smithi028.stdout:9/188: getdents d1/d17 0 2024-07-03T15:44:02.186 INFO:tasks.workunit.client.0.smithi028.stdout:9/189: fsync d1/dd/f13 0 2024-07-03T15:44:02.188 INFO:tasks.workunit.client.1.smithi148.stdout:8/980: truncate d4/da/d14/d1a/d32/d14e/d88/faa 450238 0 2024-07-03T15:44:02.188 INFO:tasks.workunit.client.1.smithi148.stdout:8/981: fsync d4/da/d14/d1a/d32/d38/dba/f77 0 2024-07-03T15:44:02.188 INFO:tasks.workunit.client.1.smithi148.stdout:8/982: truncate d4/da/d14/d1a/d32/d38/d58/d8e/db9/fdd 4351618 0 2024-07-03T15:44:02.189 INFO:tasks.workunit.client.0.smithi028.stdout:5/207: rmdir d0/dd/d13/d19/d26/d41 39 2024-07-03T15:44:02.189 INFO:tasks.workunit.client.0.smithi028.stdout:5/208: write d0/dd/df/f16 [35556,98494] 0 2024-07-03T15:44:02.189 INFO:tasks.workunit.client.0.smithi028.stdout:5/209: write d0/dd/df/f16 [561211,78434] 0 2024-07-03T15:44:02.201 INFO:tasks.workunit.client.0.smithi028.stdout:2/183: unlink da/f26 0 2024-07-03T15:44:02.203 INFO:tasks.workunit.client.1.smithi148.stdout:0/827: dwrite da/d58/dbd/d4a/feb [0,4194304] 0 2024-07-03T15:44:02.203 INFO:tasks.workunit.client.1.smithi148.stdout:0/828: fsync da/d17/fe8 0 2024-07-03T15:44:02.203 INFO:tasks.workunit.client.0.smithi028.stdout:4/130: dwrite d2/f22 [0,4194304] 0 2024-07-03T15:44:02.205 INFO:tasks.workunit.client.0.smithi028.stdout:9/190: getdents d1/dd 0 2024-07-03T15:44:02.205 INFO:tasks.workunit.client.0.smithi028.stdout:9/191: creat d1/f39 x:0 0 0 2024-07-03T15:44:02.207 INFO:tasks.workunit.client.0.smithi028.stdout:5/210: rename d0/dd/d13/d29/l33 to d0/dd/d13/d19/l48 0 2024-07-03T15:44:02.208 INFO:tasks.workunit.client.1.smithi148.stdout:4/943: mknod d0/d8/d13/d67/db1/c14d 0 2024-07-03T15:44:02.209 INFO:tasks.workunit.client.0.smithi028.stdout:2/184: mknod da/d23/d30/c43 0 2024-07-03T15:44:02.210 INFO:tasks.workunit.client.0.smithi028.stdout:6/134: getdents d4/d12 0 2024-07-03T15:44:02.214 INFO:tasks.workunit.client.1.smithi148.stdout:8/983: mkdir d4/da/d51/d14f 0 2024-07-03T15:44:02.214 INFO:tasks.workunit.client.1.smithi148.stdout:8/984: write d4/da/d14/d1a/d32/d38/d58/dcb/f12d [661454,60709] 0 2024-07-03T15:44:02.214 INFO:tasks.workunit.client.1.smithi148.stdout:8/985: write d4/da/d14/d1a/d32/d38/fd1 [2566274,109391] 0 2024-07-03T15:44:02.215 INFO:tasks.workunit.client.0.smithi028.stdout:4/131: mkdir d2/d12/d2d 0 2024-07-03T15:44:02.219 INFO:tasks.workunit.client.1.smithi148.stdout:0/829: rmdir da/d17/d2a/d2f/d93 39 2024-07-03T15:44:02.220 INFO:tasks.workunit.client.0.smithi028.stdout:6/135: mknod d4/d15/c2d 0 2024-07-03T15:44:02.222 INFO:tasks.workunit.client.0.smithi028.stdout:9/192: rename d1/f39 to d1/d16/f3a 0 2024-07-03T15:44:02.222 INFO:tasks.workunit.client.0.smithi028.stdout:9/193: chown d1/d16/f1c 21083 1 2024-07-03T15:44:02.222 INFO:tasks.workunit.client.0.smithi028.stdout:9/194: chown d1/l8 44389388 1 2024-07-03T15:44:02.224 INFO:tasks.workunit.client.0.smithi028.stdout:5/211: dwrite d0/dd/d13/d19/d26/f2e [0,4194304] 0 2024-07-03T15:44:02.224 INFO:tasks.workunit.client.0.smithi028.stdout:5/212: truncate d0/dd/d13/d29/f35 1502639 0 2024-07-03T15:44:02.224 INFO:tasks.workunit.client.0.smithi028.stdout:4/132: read d2/f27 [1604833,101030] 0 2024-07-03T15:44:02.225 INFO:tasks.workunit.client.0.smithi028.stdout:4/133: chown d2/f7 229 1 2024-07-03T15:44:02.225 INFO:tasks.workunit.client.1.smithi148.stdout:4/944: rename d0/d28/da7/d2f/d41/ff5 to d0/d28/d7d/d6f/da6/f14e 0 2024-07-03T15:44:02.225 INFO:tasks.workunit.client.1.smithi148.stdout:4/945: readlink d0/d8/d3a/l72 0 2024-07-03T15:44:02.225 INFO:tasks.workunit.client.1.smithi148.stdout:1/923: sync 2024-07-03T15:44:02.225 INFO:tasks.workunit.client.1.smithi148.stdout:1/924: chown d0/d2/d11/d3a/dc5/d116/d14/d29/d94 22096 1 2024-07-03T15:44:02.225 INFO:tasks.workunit.client.1.smithi148.stdout:1/925: chown d0/d2/d11/d3a/dc5/d116/d14/d29/de7/l122 25 1 2024-07-03T15:44:02.226 INFO:tasks.workunit.client.1.smithi148.stdout:8/986: mknod d4/da/d14/d3a/da8/c150 0 2024-07-03T15:44:02.226 INFO:tasks.workunit.client.1.smithi148.stdout:8/987: fdatasync d4/da/d51/d18/d20/f5a 0 2024-07-03T15:44:02.226 INFO:tasks.workunit.client.0.smithi028.stdout:2/185: rename da/l2a to da/d23/d30/l44 0 2024-07-03T15:44:02.227 INFO:tasks.workunit.client.1.smithi148.stdout:0/830: symlink da/d58/dbd/d13/d14/d50/l111 0 2024-07-03T15:44:02.227 INFO:tasks.workunit.client.1.smithi148.stdout:0/831: chown da/d58/d90/dd1/d109 624 1 2024-07-03T15:44:02.227 INFO:tasks.workunit.client.1.smithi148.stdout:0/832: chown da/d58/dbd/d24/dc8/d8a/dab/lef 86 1 2024-07-03T15:44:02.229 INFO:tasks.workunit.client.0.smithi028.stdout:5/213: symlink d0/dd/d13/d29/l49 0 2024-07-03T15:44:02.229 INFO:tasks.workunit.client.1.smithi148.stdout:4/946: creat d0/d8/d13/f14f x:0 0 0 2024-07-03T15:44:02.230 INFO:tasks.workunit.client.0.smithi028.stdout:4/134: mkdir d2/d12/d2e 0 2024-07-03T15:44:02.230 INFO:tasks.workunit.client.0.smithi028.stdout:4/135: chown d2/d12 96899 1 2024-07-03T15:44:02.230 INFO:tasks.workunit.client.0.smithi028.stdout:4/136: readlink d2/d12/d19/l29 0 2024-07-03T15:44:02.230 INFO:tasks.workunit.client.1.smithi148.stdout:1/926: symlink d0/d2/d11/d1e/d9d/dc0/l131 0 2024-07-03T15:44:02.231 INFO:tasks.workunit.client.0.smithi028.stdout:6/136: mknod d4/c2e 0 2024-07-03T15:44:02.231 INFO:tasks.workunit.client.0.smithi028.stdout:6/137: dread - d4/d12/f21 zero size 2024-07-03T15:44:02.235 INFO:tasks.workunit.client.1.smithi148.stdout:0/833: getdents da/d17 0 2024-07-03T15:44:02.236 INFO:tasks.workunit.client.0.smithi028.stdout:9/195: rename d1/d17/f29 to d1/d10/d28/f3b 0 2024-07-03T15:44:02.237 INFO:tasks.workunit.client.0.smithi028.stdout:2/186: unlink da/f22 0 2024-07-03T15:44:02.240 INFO:tasks.workunit.client.0.smithi028.stdout:4/137: mkdir d2/d12/d2d/d2f 0 2024-07-03T15:44:02.242 INFO:tasks.workunit.client.0.smithi028.stdout:6/138: mknod d4/d15/c2f 0 2024-07-03T15:44:02.242 INFO:tasks.workunit.client.0.smithi028.stdout:6/139: read - d4/d12/f21 zero size 2024-07-03T15:44:02.243 INFO:tasks.workunit.client.0.smithi028.stdout:9/196: rename d1/dd/f14 to d1/dd/f3c 0 2024-07-03T15:44:02.245 INFO:tasks.workunit.client.0.smithi028.stdout:2/187: creat da/d1e/f45 x:0 0 0 2024-07-03T15:44:02.245 INFO:tasks.workunit.client.0.smithi028.stdout:2/188: readlink l8 0 2024-07-03T15:44:02.245 INFO:tasks.workunit.client.0.smithi028.stdout:2/189: write da/d24/d2c/f3d [852541,56579] 0 2024-07-03T15:44:02.246 INFO:tasks.workunit.client.0.smithi028.stdout:9/197: write d1/dd/f1f [153021,121739] 0 2024-07-03T15:44:02.246 INFO:tasks.workunit.client.0.smithi028.stdout:9/198: chown d1/dd/f3c 7046 1 2024-07-03T15:44:02.247 INFO:tasks.workunit.client.0.smithi028.stdout:4/138: unlink d2/d12/d19/f1a 0 2024-07-03T15:44:02.247 INFO:tasks.workunit.client.0.smithi028.stdout:4/139: fsync d2/f14 0 2024-07-03T15:44:02.247 INFO:tasks.workunit.client.0.smithi028.stdout:2/190: stat da/d23/d30/l3a 0 2024-07-03T15:44:02.249 INFO:tasks.workunit.client.0.smithi028.stdout:9/199: mkdir d1/d3d 0 2024-07-03T15:44:02.249 INFO:tasks.workunit.client.0.smithi028.stdout:9/200: dread - d1/d16/f2f zero size 2024-07-03T15:44:02.249 INFO:tasks.workunit.client.0.smithi028.stdout:9/201: creat d1/d10/f3e x:0 0 0 2024-07-03T15:44:02.249 INFO:tasks.workunit.client.0.smithi028.stdout:9/202: read - d1/f31 zero size 2024-07-03T15:44:02.251 INFO:tasks.workunit.client.0.smithi028.stdout:2/191: stat da/d1e/c2e 0 2024-07-03T15:44:02.255 INFO:tasks.workunit.client.0.smithi028.stdout:4/140: getdents d2 0 2024-07-03T15:44:02.256 INFO:tasks.workunit.client.0.smithi028.stdout:2/192: creat da/d42/f46 x:0 0 0 2024-07-03T15:44:02.259 INFO:tasks.workunit.client.0.smithi028.stdout:4/141: mknod d2/d12/d2e/c30 0 2024-07-03T15:44:02.268 INFO:tasks.workunit.client.0.smithi028.stdout:4/142: write d2/f14 [1462182,110905] 0 2024-07-03T15:44:02.269 INFO:tasks.workunit.client.1.smithi148.stdout:8/988: write d4/da/d51/f8 [42324,40524] 0 2024-07-03T15:44:02.269 INFO:tasks.workunit.client.1.smithi148.stdout:1/927: dwrite d0/d2/d11/d1e/fb3 [0,4194304] 0 2024-07-03T15:44:02.269 INFO:tasks.workunit.client.1.smithi148.stdout:1/928: write d0/d2/d11/d3a/dc5/d116/d14/f8e [1104347,67404] 0 2024-07-03T15:44:02.269 INFO:tasks.workunit.client.1.smithi148.stdout:1/929: dread - d0/d2/d11/d3a/dc5/d116/d14/ff6 zero size 2024-07-03T15:44:02.270 INFO:tasks.workunit.client.1.smithi148.stdout:1/930: creat d0/d2/dd0/f132 x:0 0 0 2024-07-03T15:44:02.270 INFO:tasks.workunit.client.0.smithi028.stdout:4/143: fdatasync d2/d12/d19/f1f 0 2024-07-03T15:44:02.271 INFO:tasks.workunit.client.1.smithi148.stdout:8/989: rename d4/da/d51/d18/l24 to d4/da/d14/d1a/d32/d14e/df9/l151 0 2024-07-03T15:44:02.271 INFO:tasks.workunit.client.1.smithi148.stdout:8/990: read d4/da/d14/d1a/d32/d38/d58/dcb/f10e [688867,66339] 0 2024-07-03T15:44:02.272 INFO:tasks.workunit.client.0.smithi028.stdout:4/144: rename d2/d12/l25 to d2/d12/d2d/l31 0 2024-07-03T15:44:02.273 INFO:tasks.workunit.client.0.smithi028.stdout:4/145: chown d2/d12/f26 23434889 1 2024-07-03T15:44:02.273 INFO:tasks.workunit.client.0.smithi028.stdout:4/146: write d2/f20 [54266,16688] 0 2024-07-03T15:44:02.274 INFO:tasks.workunit.client.0.smithi028.stdout:4/147: write d2/f7 [759995,51582] 0 2024-07-03T15:44:02.274 INFO:tasks.workunit.client.0.smithi028.stdout:4/148: creat d2/d12/d2d/f32 x:0 0 0 2024-07-03T15:44:02.275 INFO:tasks.workunit.client.1.smithi148.stdout:1/931: creat d0/d2/d11/d3a/dc5/d116/d14/d29/f133 x:0 0 0 2024-07-03T15:44:02.275 INFO:tasks.workunit.client.1.smithi148.stdout:8/991: truncate d4/da/d14/d1a/d32/d38/d58/f8b 458154 0 2024-07-03T15:44:02.275 INFO:tasks.workunit.client.1.smithi148.stdout:1/932: link d0/d2/d11/d3a/dc5/d116/d14/d29/d94/cb1 d0/d2/d11/d3a/c134 0 2024-07-03T15:44:02.275 INFO:tasks.workunit.client.1.smithi148.stdout:8/992: mkdir d4/da/d14/d3a/da8/dd3/d152 0 2024-07-03T15:44:02.276 INFO:tasks.workunit.client.1.smithi148.stdout:1/933: dread d0/d2/d11/d3a/dc5/d116/f5a [0,4194304] 0 2024-07-03T15:44:02.276 INFO:tasks.workunit.client.1.smithi148.stdout:8/993: mkdir d4/da/d14/d3a/da8/d149/d153 0 2024-07-03T15:44:02.277 INFO:tasks.workunit.client.1.smithi148.stdout:1/934: symlink d0/d2/d11/d3a/dc5/d119/d44/d54/d5d/d10d/l135 0 2024-07-03T15:44:02.277 INFO:tasks.workunit.client.1.smithi148.stdout:1/935: write d0/d2/d10b/f123 [347801,117762] 0 2024-07-03T15:44:02.277 INFO:tasks.workunit.client.1.smithi148.stdout:8/994: mkdir d4/da/d14/d1a/d32/d89/d154 0 2024-07-03T15:44:02.277 INFO:tasks.workunit.client.1.smithi148.stdout:8/995: readlink d4/da/d14/d3a/lfb 0 2024-07-03T15:44:02.278 INFO:tasks.workunit.client.1.smithi148.stdout:1/936: symlink d0/d2/d11/d3a/dc5/d116/d14/d29/d94/l136 0 2024-07-03T15:44:02.280 INFO:tasks.workunit.client.1.smithi148.stdout:1/937: unlink d0/d2/cc 0 2024-07-03T15:44:02.282 INFO:tasks.workunit.client.1.smithi148.stdout:1/938: link d0/d2/d11/d3a/dc5/d119/d2f/d72/df9/fdc d0/d2/d11/d3a/dc5/d116/d14/d29/de7/f137 0 2024-07-03T15:44:02.282 INFO:tasks.workunit.client.1.smithi148.stdout:1/939: readlink d0/d2/d11/d1e/d9d/dc2/de2/d77/da3/l101 0 2024-07-03T15:44:02.282 INFO:tasks.workunit.client.1.smithi148.stdout:1/940: fsync d0/d2/d11/d3a/dc5/d119/d2f/d72/df9/d42/daf/fd6 0 2024-07-03T15:44:02.282 INFO:tasks.workunit.client.1.smithi148.stdout:1/941: chown d0/d2/d11/d3a/dc5/d119/lf 55906 1 2024-07-03T15:44:02.284 INFO:tasks.workunit.client.0.smithi028.stdout:5/214: dwrite d0/dd/d13/d19/d26/f42 [0,4194304] 0 2024-07-03T15:44:02.290 INFO:tasks.workunit.client.1.smithi148.stdout:8/996: dread d4/da/d14/d1a/d32/d14e/d88/f113 [0,4194304] 0 2024-07-03T15:44:02.290 INFO:tasks.workunit.client.1.smithi148.stdout:8/997: creat d4/da/d14/d9e/f155 x:0 0 0 2024-07-03T15:44:02.292 INFO:tasks.workunit.client.1.smithi148.stdout:4/947: dwrite d0/d28/da7/d2f/d41/d148/fd8 [0,4194304] 0 2024-07-03T15:44:02.292 INFO:tasks.workunit.client.1.smithi148.stdout:4/948: chown d0/d28/d7d/c11a 83547554 1 2024-07-03T15:44:02.292 INFO:tasks.workunit.client.1.smithi148.stdout:4/949: creat d0/d28/d6e/dc7/f150 x:0 0 0 2024-07-03T15:44:02.294 INFO:tasks.workunit.client.0.smithi028.stdout:9/203: rmdir d1 39 2024-07-03T15:44:02.296 INFO:tasks.workunit.client.0.smithi028.stdout:9/204: rename d1/d10/d28/l33 to d1/l3f 0 2024-07-03T15:44:02.297 INFO:tasks.workunit.client.1.smithi148.stdout:4/950: creat d0/d8/d13/d8b/f151 x:0 0 0 2024-07-03T15:44:02.299 INFO:tasks.workunit.client.1.smithi148.stdout:4/951: write d0/d8/d13/d48/fec [1060994,94963] 0 2024-07-03T15:44:02.300 INFO:tasks.workunit.client.0.smithi028.stdout:6/140: dwrite d4/f11 [8388608,4194304] 0 2024-07-03T15:44:02.301 INFO:tasks.workunit.client.1.smithi148.stdout:0/834: dwrite da/d17/d2a/d2f/ff5 [0,4194304] 0 2024-07-03T15:44:02.301 INFO:tasks.workunit.client.1.smithi148.stdout:0/835: fsync da/d58/dbd/d94/dba/ffd 0 2024-07-03T15:44:02.305 INFO:tasks.workunit.client.1.smithi148.stdout:8/998: dread d4/da/d51/f7 [8388608,4194304] 0 2024-07-03T15:44:02.305 INFO:tasks.workunit.client.0.smithi028.stdout:2/193: dwrite f4 [0,4194304] 0 2024-07-03T15:44:02.305 INFO:tasks.workunit.client.0.smithi028.stdout:6/141: link d4/l1b d4/d15/l30 0 2024-07-03T15:44:02.307 INFO:tasks.workunit.client.1.smithi148.stdout:8/999: rename d4/da/d14/d1a/d32/d38/lac to d4/da/d14/l156 0 2024-07-03T15:44:02.309 INFO:tasks.workunit.client.0.smithi028.stdout:2/194: mkdir da/d42/d47 0 2024-07-03T15:44:02.309 INFO:tasks.workunit.client.0.smithi028.stdout:2/195: chown da/d1e/f21 151 1 2024-07-03T15:44:02.309 INFO:tasks.workunit.client.1.smithi148.stdout:0/836: dread da/d17/d2a/d2f/fcd [0,4194304] 0 2024-07-03T15:44:02.310 INFO:tasks.workunit.client.0.smithi028.stdout:6/142: mkdir d4/d12/d2c/d31 0 2024-07-03T15:44:02.310 INFO:tasks.workunit.client.0.smithi028.stdout:6/143: chown d4/l24 51810064 1 2024-07-03T15:44:02.311 INFO:tasks.workunit.client.1.smithi148.stdout:0/837: read da/d17/d2a/d2f/d93/da3/fc7 [245030,30789] 0 2024-07-03T15:44:02.326 INFO:tasks.workunit.client.0.smithi028.stdout:4/149: dwrite d2/d12/d19/f9 [0,4194304] 0 2024-07-03T15:44:02.326 INFO:tasks.workunit.client.0.smithi028.stdout:4/150: chown d2/d12/d19 95008838 1 2024-07-03T15:44:02.326 INFO:tasks.workunit.client.0.smithi028.stdout:4/151: write d2/d12/f17 [749911,195] 0 2024-07-03T15:44:02.326 INFO:tasks.workunit.client.0.smithi028.stdout:4/152: symlink d2/d12/d2d/l33 0 2024-07-03T15:44:02.326 INFO:tasks.workunit.client.0.smithi028.stdout:4/153: getdents d2 0 2024-07-03T15:44:02.327 INFO:tasks.workunit.client.0.smithi028.stdout:6/144: write d4/f11 [8923070,14502] 0 2024-07-03T15:44:02.327 INFO:tasks.workunit.client.0.smithi028.stdout:6/145: chown d4/d12 11330 1 2024-07-03T15:44:02.327 INFO:tasks.workunit.client.0.smithi028.stdout:4/154: creat d2/d12/d19/f34 x:0 0 0 2024-07-03T15:44:02.327 INFO:tasks.workunit.client.0.smithi028.stdout:4/155: fsync d2/f2b 0 2024-07-03T15:44:02.328 INFO:tasks.workunit.client.1.smithi148.stdout:1/942: dwrite d0/d2/dd0/f132 [0,4194304] 0 2024-07-03T15:44:02.331 INFO:tasks.workunit.client.1.smithi148.stdout:1/943: symlink d0/d2/d11/d3a/dc5/d116/d14/l138 0 2024-07-03T15:44:02.331 INFO:tasks.workunit.client.1.smithi148.stdout:1/944: read d0/d2/d11/d3a/f5b [385301,108388] 0 2024-07-03T15:44:02.332 INFO:tasks.workunit.client.1.smithi148.stdout:1/945: stat d0/d2/d11/d1e/d9d/dc2/de2/fd5 0 2024-07-03T15:44:02.333 INFO:tasks.workunit.client.1.smithi148.stdout:1/946: rename d0/d2/d11/d3a/dc5/d119/cbd to d0/d2/dae/d106/c139 0 2024-07-03T15:44:02.336 INFO:tasks.workunit.client.1.smithi148.stdout:1/947: mknod d0/d2/dd0/c13a 0 2024-07-03T15:44:02.336 INFO:tasks.workunit.client.1.smithi148.stdout:1/948: creat d0/d2/dd0/f13b x:0 0 0 2024-07-03T15:44:02.336 INFO:tasks.workunit.client.1.smithi148.stdout:1/949: fsync d0/d2/d11/d3a/dc5/d119/d2f/d72/df9/f5f 0 2024-07-03T15:44:02.336 INFO:tasks.workunit.client.1.smithi148.stdout:1/950: dread - d0/d2/d11/d1e/d9d/dc2/de2/fd5 zero size 2024-07-03T15:44:02.339 INFO:tasks.workunit.client.1.smithi148.stdout:1/951: mkdir d0/d2/d11/d3a/dc5/d116/d14/d13c 0 2024-07-03T15:44:02.339 INFO:tasks.workunit.client.1.smithi148.stdout:1/952: chown d0/d2/d11/d3a/dc5/d119/d2f/d72/fa5 13 1 2024-07-03T15:44:02.339 INFO:tasks.workunit.client.1.smithi148.stdout:1/953: chown d0/d2/d11/d3a/dc5/d116/d14/d29/fce 116194 1 2024-07-03T15:44:02.340 INFO:tasks.workunit.client.0.smithi028.stdout:2/196: dwrite da/d24/d2c/f32 [0,4194304] 0 2024-07-03T15:44:02.341 INFO:tasks.workunit.client.0.smithi028.stdout:5/215: dwrite d0/dd/df/f16 [0,4194304] 0 2024-07-03T15:44:02.341 INFO:tasks.workunit.client.0.smithi028.stdout:5/216: truncate d0/dd/d13/d19/d26/d41/f43 1291681 0 2024-07-03T15:44:02.341 INFO:tasks.workunit.client.0.smithi028.stdout:9/205: dwrite d1/f9 [0,4194304] 0 2024-07-03T15:44:02.341 INFO:tasks.workunit.client.1.smithi148.stdout:4/952: dwrite d0/d28/d7d/d6f/d7c/dae/dba/dde/f113 [0,4194304] 0 2024-07-03T15:44:02.350 INFO:tasks.workunit.client.1.smithi148.stdout:4/953: mknod d0/d8/d3a/db8/dd3/c152 0 2024-07-03T15:44:02.352 INFO:tasks.workunit.client.0.smithi028.stdout:9/206: getdents d1/d17 0 2024-07-03T15:44:02.354 INFO:tasks.workunit.client.1.smithi148.stdout:4/954: mknod d0/d28/d102/c153 0 2024-07-03T15:44:02.354 INFO:tasks.workunit.client.1.smithi148.stdout:4/955: truncate d0/d28/da7/d2f/d41/d148/fca 490258 0 2024-07-03T15:44:02.354 INFO:tasks.workunit.client.1.smithi148.stdout:4/956: fdatasync d0/d8/d3a/dce/df7/f104 0 2024-07-03T15:44:02.355 INFO:tasks.workunit.client.0.smithi028.stdout:9/207: rename d1/d10/d28/f3b to d1/d10/d28/f40 0 2024-07-03T15:44:02.357 INFO:tasks.workunit.client.0.smithi028.stdout:2/197: write da/d24/d2c/f32 [3183224,70974] 0 2024-07-03T15:44:02.358 INFO:tasks.workunit.client.1.smithi148.stdout:4/957: chown d0/d8/d13/c4a 17648668 1 2024-07-03T15:44:02.361 INFO:tasks.workunit.client.1.smithi148.stdout:4/958: truncate d0/d8/d3a/dce/df7/f108 3505200 0 2024-07-03T15:44:02.361 INFO:tasks.workunit.client.1.smithi148.stdout:4/959: write d0/d28/d7d/d6f/f82 [5178018,128903] 0 2024-07-03T15:44:02.363 INFO:tasks.workunit.client.0.smithi028.stdout:0/128: sync 2024-07-03T15:44:02.363 INFO:tasks.workunit.client.0.smithi028.stdout:1/319: sync 2024-07-03T15:44:02.363 INFO:tasks.workunit.client.0.smithi028.stdout:7/74: sync 2024-07-03T15:44:02.363 INFO:tasks.workunit.client.0.smithi028.stdout:5/217: dread d0/dd/f1b [0,4194304] 0 2024-07-03T15:44:02.364 INFO:tasks.workunit.client.0.smithi028.stdout:7/75: getdents d9 0 2024-07-03T15:44:02.364 INFO:tasks.workunit.client.0.smithi028.stdout:7/76: truncate f0 9145230 0 2024-07-03T15:44:02.365 INFO:tasks.workunit.client.1.smithi148.stdout:4/960: dread d0/fbf [0,4194304] 0 2024-07-03T15:44:02.365 INFO:tasks.workunit.client.1.smithi148.stdout:4/961: chown d0/d28/de8/d132/f122 1583 1 2024-07-03T15:44:02.367 INFO:tasks.workunit.client.0.smithi028.stdout:0/129: symlink d2/l28 0 2024-07-03T15:44:02.367 INFO:tasks.workunit.client.0.smithi028.stdout:0/130: stat d2/d3/c17 0 2024-07-03T15:44:02.367 INFO:tasks.workunit.client.0.smithi028.stdout:8/204: sync 2024-07-03T15:44:02.368 INFO:tasks.workunit.client.0.smithi028.stdout:3/212: sync 2024-07-03T15:44:02.368 INFO:tasks.workunit.client.0.smithi028.stdout:3/213: fsync f6 0 2024-07-03T15:44:02.368 INFO:tasks.workunit.client.0.smithi028.stdout:8/205: write d0/d2/d3/dc/d11/f42 [848347,48508] 0 2024-07-03T15:44:02.368 INFO:tasks.workunit.client.0.smithi028.stdout:8/206: chown d0/d2/c19 6 1 2024-07-03T15:44:02.368 INFO:tasks.workunit.client.0.smithi028.stdout:8/207: fdatasync d0/f20 0 2024-07-03T15:44:02.368 INFO:tasks.workunit.client.0.smithi028.stdout:5/218: rmdir d0/dd/d1e/d3b 39 2024-07-03T15:44:02.371 INFO:tasks.workunit.client.0.smithi028.stdout:7/77: mkdir d9/d19 0 2024-07-03T15:44:02.372 INFO:tasks.workunit.client.0.smithi028.stdout:3/214: write f14 [272241,91308] 0 2024-07-03T15:44:02.372 INFO:tasks.workunit.client.0.smithi028.stdout:3/215: chown d18/d1a/f24 30222 1 2024-07-03T15:44:02.372 INFO:tasks.workunit.client.0.smithi028.stdout:8/208: mkdir d0/d41/d47 0 2024-07-03T15:44:02.374 INFO:tasks.workunit.client.0.smithi028.stdout:5/219: symlink d0/l4a 0 2024-07-03T15:44:02.375 INFO:tasks.workunit.client.1.smithi148.stdout:0/838: write da/d17/d2a/f4b [1127284,40340] 0 2024-07-03T15:44:02.376 INFO:tasks.workunit.client.0.smithi028.stdout:6/146: dwrite f0 [0,4194304] 0 2024-07-03T15:44:02.376 INFO:tasks.workunit.client.0.smithi028.stdout:6/147: truncate d4/d12/f21 258832 0 2024-07-03T15:44:02.378 INFO:tasks.workunit.client.0.smithi028.stdout:4/156: dwrite d2/f1b [0,4194304] 0 2024-07-03T15:44:02.378 INFO:tasks.workunit.client.0.smithi028.stdout:7/78: rmdir d9/d19 0 2024-07-03T15:44:02.379 INFO:tasks.workunit.client.0.smithi028.stdout:0/131: link d2/d1c/l24 d2/d3/l29 0 2024-07-03T15:44:02.380 INFO:tasks.workunit.client.0.smithi028.stdout:3/216: truncate f17 302083 0 2024-07-03T15:44:02.382 INFO:tasks.workunit.client.0.smithi028.stdout:8/209: truncate d0/d2/d3/dc/f23 75306 0 2024-07-03T15:44:02.382 INFO:tasks.workunit.client.0.smithi028.stdout:8/210: stat d0/d2/d3/d35/d46 0 2024-07-03T15:44:02.383 INFO:tasks.workunit.client.0.smithi028.stdout:9/208: dwrite d1/d16/f2f [0,4194304] 0 2024-07-03T15:44:02.387 INFO:tasks.workunit.client.0.smithi028.stdout:6/148: mknod d4/d12/d2c/d31/c32 0 2024-07-03T15:44:02.390 INFO:tasks.workunit.client.0.smithi028.stdout:7/79: read f4 [3069336,46478] 0 2024-07-03T15:44:02.390 INFO:tasks.workunit.client.0.smithi028.stdout:3/217: mknod d18/d1a/d1e/c45 0 2024-07-03T15:44:02.392 INFO:tasks.workunit.client.1.smithi148.stdout:0/839: write da/d58/dbd/d94/dba/ffd [5147272,118645] 0 2024-07-03T15:44:02.394 INFO:tasks.workunit.client.0.smithi028.stdout:9/209: rename d1/d10/f24 to d1/f41 0 2024-07-03T15:44:02.394 INFO:tasks.workunit.client.0.smithi028.stdout:0/132: symlink d2/d3/l2a 0 2024-07-03T15:44:02.396 INFO:tasks.workunit.client.0.smithi028.stdout:3/218: dread f6 [0,4194304] 0 2024-07-03T15:44:02.397 INFO:tasks.workunit.client.1.smithi148.stdout:0/840: unlink da/d58/dbd/d13/d5a/ld2 0 2024-07-03T15:44:02.399 INFO:tasks.workunit.client.0.smithi028.stdout:0/133: creat d2/d1c/f2b x:0 0 0 2024-07-03T15:44:02.399 INFO:tasks.workunit.client.0.smithi028.stdout:0/134: fsync d2/fa 0 2024-07-03T15:44:02.399 INFO:tasks.workunit.client.0.smithi028.stdout:0/135: readlink d2/d3/l2a 0 2024-07-03T15:44:02.399 INFO:tasks.workunit.client.0.smithi028.stdout:0/136: readlink d2/d3/l22 0 2024-07-03T15:44:02.400 INFO:tasks.workunit.client.0.smithi028.stdout:3/219: mknod d18/d40/c46 0 2024-07-03T15:44:02.401 INFO:tasks.workunit.client.1.smithi148.stdout:0/841: mkdir da/d58/dbd/d13/d1d/d3e/dd8/d112 0 2024-07-03T15:44:02.402 INFO:tasks.workunit.client.0.smithi028.stdout:3/220: creat d18/d3a/f47 x:0 0 0 2024-07-03T15:44:02.402 INFO:tasks.workunit.client.0.smithi028.stdout:3/221: creat d18/d1a/d1e/f48 x:0 0 0 2024-07-03T15:44:02.403 INFO:tasks.workunit.client.1.smithi148.stdout:1/954: dwrite d0/d2/d11/d1e/d9d/dc2/de2/f69 [8388608,4194304] 0 2024-07-03T15:44:02.403 INFO:tasks.workunit.client.1.smithi148.stdout:1/955: write d0/d2/d11/d3a/dc5/d119/d2f/d72/df9/d42/f48 [10647706,62303] 0 2024-07-03T15:44:02.404 INFO:tasks.workunit.client.0.smithi028.stdout:0/137: link d2/d1c/f2b d2/d3/f2c 0 2024-07-03T15:44:02.404 INFO:tasks.workunit.client.0.smithi028.stdout:0/138: write d2/d3/f19 [5200552,3919] 0 2024-07-03T15:44:02.405 INFO:tasks.workunit.client.1.smithi148.stdout:0/842: mknod da/d58/dbd/d4a/df6/c113 0 2024-07-03T15:44:02.406 INFO:tasks.workunit.client.0.smithi028.stdout:4/157: dread d2/d12/f26 [0,4194304] 0 2024-07-03T15:44:02.406 INFO:tasks.workunit.client.0.smithi028.stdout:4/158: chown d2/d12/d2d/l31 497748 1 2024-07-03T15:44:02.406 INFO:tasks.workunit.client.0.smithi028.stdout:4/159: readlink d2/d12/d19/l29 0 2024-07-03T15:44:02.406 INFO:tasks.workunit.client.0.smithi028.stdout:4/160: getdents d2/d12/d2d/d2f 0 2024-07-03T15:44:02.407 INFO:tasks.workunit.client.1.smithi148.stdout:1/956: truncate d0/d2/d11/d1e/f9f 2252031 0 2024-07-03T15:44:02.407 INFO:tasks.workunit.client.1.smithi148.stdout:1/957: write d0/d2/d2e/f85 [2356337,127533] 0 2024-07-03T15:44:02.408 INFO:tasks.workunit.client.0.smithi028.stdout:3/222: symlink d18/d1a/l49 0 2024-07-03T15:44:02.410 INFO:tasks.workunit.client.1.smithi148.stdout:0/843: mkdir da/d58/dbd/d24/d84/d114 0 2024-07-03T15:44:02.410 INFO:tasks.workunit.client.1.smithi148.stdout:0/844: chown da/d58/d90/lbb 59212228 1 2024-07-03T15:44:02.414 INFO:tasks.workunit.client.0.smithi028.stdout:9/210: write d1/d10/f18 [1467629,92392] 0 2024-07-03T15:44:02.414 INFO:tasks.workunit.client.0.smithi028.stdout:9/211: chown d1/lc 410 1 2024-07-03T15:44:02.416 INFO:tasks.workunit.client.1.smithi148.stdout:0/845: creat da/d58/dbd/d13/d14/f115 x:0 0 0 2024-07-03T15:44:02.417 INFO:tasks.workunit.client.0.smithi028.stdout:3/223: dread ff [0,4194304] 0 2024-07-03T15:44:02.417 INFO:tasks.workunit.client.0.smithi028.stdout:3/224: chown d18/d40 395030812 1 2024-07-03T15:44:02.417 INFO:tasks.workunit.client.0.smithi028.stdout:3/225: dread - d18/d1a/d1e/f38 zero size 2024-07-03T15:44:02.418 INFO:tasks.workunit.client.0.smithi028.stdout:0/139: rename d2/fa to d2/d1c/f2d 0 2024-07-03T15:44:02.418 INFO:tasks.workunit.client.0.smithi028.stdout:0/140: creat d2/d1c/f2e x:0 0 0 2024-07-03T15:44:02.418 INFO:tasks.workunit.client.0.smithi028.stdout:0/141: dread - d2/d3/fe zero size 2024-07-03T15:44:02.420 INFO:tasks.workunit.client.1.smithi148.stdout:0/846: rename da/d58/dbd/l20 to da/d58/dbd/d13/d1d/d3e/dd8/d112/l116 0 2024-07-03T15:44:02.421 INFO:tasks.workunit.client.1.smithi148.stdout:0/847: creat da/f117 x:0 0 0 2024-07-03T15:44:02.421 INFO:tasks.workunit.client.1.smithi148.stdout:0/848: stat da/d58/dbd/d13/d14/f68 0 2024-07-03T15:44:02.421 INFO:tasks.workunit.client.1.smithi148.stdout:0/849: fsync da/d58/dbd/d13/d14/f61 0 2024-07-03T15:44:02.422 INFO:tasks.workunit.client.0.smithi028.stdout:9/212: symlink d1/d17/d30/l42 0 2024-07-03T15:44:02.422 INFO:tasks.workunit.client.0.smithi028.stdout:9/213: write d1/f2b [397764,15859] 0 2024-07-03T15:44:02.424 INFO:tasks.workunit.client.0.smithi028.stdout:0/142: creat d2/d3/f2f x:0 0 0 2024-07-03T15:44:02.424 INFO:tasks.workunit.client.1.smithi148.stdout:1/958: write d0/d2/dd0/f132 [1146912,24430] 0 2024-07-03T15:44:02.425 INFO:tasks.workunit.client.0.smithi028.stdout:9/214: creat d1/f43 x:0 0 0 2024-07-03T15:44:02.425 INFO:tasks.workunit.client.0.smithi028.stdout:9/215: truncate d1/d17/f25 611852 0 2024-07-03T15:44:02.425 INFO:tasks.workunit.client.0.smithi028.stdout:0/143: symlink d2/d1c/l30 0 2024-07-03T15:44:02.426 INFO:tasks.workunit.client.1.smithi148.stdout:4/962: dwrite d0/d28/da7/d2f/d41/f4d [0,4194304] 0 2024-07-03T15:44:02.426 INFO:tasks.workunit.client.0.smithi028.stdout:9/216: symlink d1/d2a/l44 0 2024-07-03T15:44:02.427 INFO:tasks.workunit.client.0.smithi028.stdout:1/320: dwrite d4/de/f10 [0,4194304] 0 2024-07-03T15:44:02.433 INFO:tasks.workunit.client.0.smithi028.stdout:9/217: truncate d1/dd/f13 3270967 0 2024-07-03T15:44:02.434 INFO:tasks.workunit.client.0.smithi028.stdout:8/211: dwrite d0/d2/d3/dc/d11/d12/f2f [8388608,4194304] 0 2024-07-03T15:44:02.434 INFO:tasks.workunit.client.0.smithi028.stdout:5/220: dwrite d0/dd/d13/d19/f21 [0,4194304] 0 2024-07-03T15:44:02.434 INFO:tasks.workunit.client.0.smithi028.stdout:6/149: dwrite d4/f29 [0,4194304] 0 2024-07-03T15:44:02.434 INFO:tasks.workunit.client.0.smithi028.stdout:6/150: write d4/f23 [35688,2742] 0 2024-07-03T15:44:02.434 INFO:tasks.workunit.client.0.smithi028.stdout:6/151: write d4/d15/f2a [715150,27378] 0 2024-07-03T15:44:02.434 INFO:tasks.workunit.client.1.smithi148.stdout:4/963: symlink d0/d28/d7d/d6f/d7c/dae/dba/dde/l154 0 2024-07-03T15:44:02.435 INFO:tasks.workunit.client.0.smithi028.stdout:1/321: creat d4/d2c/d3a/d5c/d6c/d7d/d59/f82 x:0 0 0 2024-07-03T15:44:02.437 INFO:tasks.workunit.client.0.smithi028.stdout:0/144: mknod d2/d3/c31 0 2024-07-03T15:44:02.440 INFO:tasks.workunit.client.1.smithi148.stdout:4/964: creat d0/d28/da7/d2f/d41/d148/f155 x:0 0 0 2024-07-03T15:44:02.440 INFO:tasks.workunit.client.0.smithi028.stdout:8/212: getdents d0/d41 0 2024-07-03T15:44:02.442 INFO:tasks.workunit.client.0.smithi028.stdout:5/221: mkdir d0/dd/d4b 0 2024-07-03T15:44:02.442 INFO:tasks.workunit.client.0.smithi028.stdout:5/222: stat d0/dd/d13/d19/f21 0 2024-07-03T15:44:02.442 INFO:tasks.workunit.client.0.smithi028.stdout:5/223: fdatasync d0/dd/d13/d19/d26/f42 0 2024-07-03T15:44:02.443 INFO:tasks.workunit.client.0.smithi028.stdout:7/80: dwrite f0 [0,4194304] 0 2024-07-03T15:44:02.443 INFO:tasks.workunit.client.0.smithi028.stdout:4/161: dwrite d2/f14 [0,4194304] 0 2024-07-03T15:44:02.444 INFO:tasks.workunit.client.0.smithi028.stdout:4/162: fsync d2/d12/d2d/f32 0 2024-07-03T15:44:02.444 INFO:tasks.workunit.client.1.smithi148.stdout:4/965: mknod d0/d28/de8/d132/c156 0 2024-07-03T15:44:02.444 INFO:tasks.workunit.client.1.smithi148.stdout:4/966: getdents d0/d8/d44/dc2/d13f 0 2024-07-03T15:44:02.444 INFO:tasks.workunit.client.1.smithi148.stdout:4/967: readlink d0/d28/d7d/d6f/d7c/l8e 0 2024-07-03T15:44:02.446 INFO:tasks.workunit.client.1.smithi148.stdout:0/850: dwrite da/d58/dbd/d24/dc8/d8a/dab/f6c [4194304,4194304] 0 2024-07-03T15:44:02.447 INFO:tasks.workunit.client.1.smithi148.stdout:4/968: creat d0/d28/d102/f157 x:0 0 0 2024-07-03T15:44:02.448 INFO:tasks.workunit.client.0.smithi028.stdout:9/218: write d1/dd/f11 [6211013,25242] 0 2024-07-03T15:44:02.448 INFO:tasks.workunit.client.0.smithi028.stdout:3/226: dwrite d18/d1a/f24 [0,4194304] 0 2024-07-03T15:44:02.448 INFO:tasks.workunit.client.0.smithi028.stdout:6/152: unlink d4/f23 0 2024-07-03T15:44:02.450 INFO:tasks.workunit.client.1.smithi148.stdout:0/851: symlink da/d17/d2a/d2f/d93/da3/dda/l118 0 2024-07-03T15:44:02.450 INFO:tasks.workunit.client.1.smithi148.stdout:1/959: dread d0/d2/dd0/f132 [0,4194304] 0 2024-07-03T15:44:02.451 INFO:tasks.workunit.client.0.smithi028.stdout:9/219: dread d1/dd/f20 [0,4194304] 0 2024-07-03T15:44:02.452 INFO:tasks.workunit.client.1.smithi148.stdout:4/969: unlink d0/d28/d6e/dc7/f150 0 2024-07-03T15:44:02.453 INFO:tasks.workunit.client.0.smithi028.stdout:0/145: mkdir d2/d1c/d32 0 2024-07-03T15:44:02.453 INFO:tasks.workunit.client.0.smithi028.stdout:0/146: getdents d2/d1c/d32 0 2024-07-03T15:44:02.454 INFO:tasks.workunit.client.0.smithi028.stdout:0/147: fsync d2/ff 0 2024-07-03T15:44:02.456 INFO:tasks.workunit.client.1.smithi148.stdout:0/852: rename da/d58/dbd/d24/d84/ff3 to da/d58/dbd/d13/d14/d50/d95/d8c/f119 0 2024-07-03T15:44:02.458 INFO:tasks.workunit.client.0.smithi028.stdout:8/213: mknod d0/dd/c48 0 2024-07-03T15:44:02.458 INFO:tasks.workunit.client.0.smithi028.stdout:8/214: write d0/d2/d3/f3a [651549,103307] 0 2024-07-03T15:44:02.460 INFO:tasks.workunit.client.1.smithi148.stdout:1/960: creat d0/d2/d11/d3a/dc5/d119/d2f/d72/df9/d42/daf/f13d x:0 0 0 2024-07-03T15:44:02.460 INFO:tasks.workunit.client.0.smithi028.stdout:5/224: rmdir d0/dd/d1e 39 2024-07-03T15:44:02.462 INFO:tasks.workunit.client.1.smithi148.stdout:4/970: unlink d0/d8/c2b 0 2024-07-03T15:44:02.464 INFO:tasks.workunit.client.1.smithi148.stdout:0/853: rename da/d17/d2a/d2f/d93/da3/dda/l118 to da/d58/dbd/d94/l11a 0 2024-07-03T15:44:02.470 INFO:tasks.workunit.client.1.smithi148.stdout:4/971: unlink d0/l149 0 2024-07-03T15:44:02.470 INFO:tasks.workunit.client.1.smithi148.stdout:4/972: write d0/d28/da7/d2f/d52/f12b [1052978,127653] 0 2024-07-03T15:44:02.470 INFO:tasks.workunit.client.1.smithi148.stdout:4/973: fsync d0/d8/d3a/dce/df7/f108 0 2024-07-03T15:44:02.470 INFO:tasks.workunit.client.1.smithi148.stdout:4/974: chown d0/d28/da7/d2f/d41/d148/ca4 868660 1 2024-07-03T15:44:02.471 INFO:tasks.workunit.client.0.smithi028.stdout:9/220: dwrite d1/f21 [0,4194304] 0 2024-07-03T15:44:02.472 INFO:tasks.workunit.client.0.smithi028.stdout:4/163: mkdir d2/d12/d2e/d35 0 2024-07-03T15:44:02.474 INFO:tasks.workunit.client.1.smithi148.stdout:0/854: unlink da/d58/dbd/d13/d14/l62 0 2024-07-03T15:44:02.474 INFO:tasks.workunit.client.1.smithi148.stdout:0/855: fsync da/d58/dbd/d13/d14/d50/d95/d8c/fb6 0 2024-07-03T15:44:02.474 INFO:tasks.workunit.client.1.smithi148.stdout:0/856: write da/d58/dbd/d13/d1d/d3e/d85/d8e/f10a [481702,53971] 0 2024-07-03T15:44:02.474 INFO:tasks.workunit.client.1.smithi148.stdout:0/857: dread - da/d58/dbd/f9d zero size 2024-07-03T15:44:02.477 INFO:tasks.workunit.client.1.smithi148.stdout:4/975: dread d0/d28/d7d/d6f/d7c/dae/dba/fdc [0,4194304] 0 2024-07-03T15:44:02.478 INFO:tasks.workunit.client.1.smithi148.stdout:0/858: mknod da/d58/dbd/d24/c11b 0 2024-07-03T15:44:02.488 INFO:tasks.workunit.client.1.smithi148.stdout:4/976: dread d0/d8/d13/d67/f86 [0,4194304] 0 2024-07-03T15:44:02.489 INFO:tasks.workunit.client.1.smithi148.stdout:1/961: dwrite d0/d2/d11/d1e/f3c [0,4194304] 0 2024-07-03T15:44:02.489 INFO:tasks.workunit.client.1.smithi148.stdout:0/859: dread da/d58/dbd/d94/dac/ddd/f6b [0,4194304] 0 2024-07-03T15:44:02.489 INFO:tasks.workunit.client.1.smithi148.stdout:0/860: dread - da/d17/d2a/d2f/d93/f104 zero size 2024-07-03T15:44:02.490 INFO:tasks.workunit.client.1.smithi148.stdout:1/962: dread - d0/d2/d11/d3a/dc5/d119/d2f/d72/df9/d42/db7/feb zero size 2024-07-03T15:44:02.490 INFO:tasks.workunit.client.1.smithi148.stdout:1/963: fsync d0/d2/d11/d3a/dc5/d119/d2f/d72/df9/d42/d62/d7b/fa4 0 2024-07-03T15:44:02.490 INFO:tasks.workunit.client.1.smithi148.stdout:1/964: chown d0/d2/d11/d1e/d9d/dc2/de2/d77 529413720 1 2024-07-03T15:44:02.490 INFO:tasks.workunit.client.1.smithi148.stdout:1/965: chown d0/d2/d11/d3a/dc5/d119/d2f/d72/df9/d42/db7/c111 190 1 2024-07-03T15:44:02.490 INFO:tasks.workunit.client.1.smithi148.stdout:1/966: dread - d0/d2/d11/d3a/f117 zero size 2024-07-03T15:44:02.492 INFO:tasks.workunit.client.1.smithi148.stdout:0/861: dread da/d58/dbd/d24/dc8/d8a/dab/f72 [4194304,4194304] 0 2024-07-03T15:44:02.492 INFO:tasks.workunit.client.1.smithi148.stdout:4/977: symlink d0/d8/dd4/l158 0 2024-07-03T15:44:02.492 INFO:tasks.workunit.client.1.smithi148.stdout:4/978: chown d0/d91/da8/dab/dad/ccc 134306 1 2024-07-03T15:44:02.496 INFO:tasks.workunit.client.1.smithi148.stdout:1/967: write d0/d2/d11/d3a/dc5/d116/d14/f6a [1282301,68779] 0 2024-07-03T15:44:02.496 INFO:tasks.workunit.client.1.smithi148.stdout:1/968: creat d0/d2/d11/d3a/dc5/d116/d14/d29/f13e x:0 0 0 2024-07-03T15:44:02.496 INFO:tasks.workunit.client.1.smithi148.stdout:1/969: chown d0/d2/d11/d3a/dc5/d119/d2f/d72/df9/d42/l97 7141 1 2024-07-03T15:44:02.498 INFO:tasks.workunit.client.0.smithi028.stdout:3/227: rename d18/d1a/d1e/c45 to d18/d3a/c4a 0 2024-07-03T15:44:02.498 INFO:tasks.workunit.client.0.smithi028.stdout:3/228: dread - d18/d3a/f47 zero size 2024-07-03T15:44:02.499 INFO:tasks.workunit.client.1.smithi148.stdout:0/862: symlink da/d58/d90/dd1/l11c 0 2024-07-03T15:44:02.500 INFO:tasks.workunit.client.0.smithi028.stdout:6/153: creat d4/f33 x:0 0 0 2024-07-03T15:44:02.501 INFO:tasks.workunit.client.0.smithi028.stdout:6/154: chown d4/da/l26 4 1 2024-07-03T15:44:02.502 INFO:tasks.workunit.client.1.smithi148.stdout:4/979: rename d0/d28/d6e/dc7/c78 to d0/d28/da7/d2f/d52/c159 0 2024-07-03T15:44:02.503 INFO:tasks.workunit.client.0.smithi028.stdout:0/148: unlink d2/f9 0 2024-07-03T15:44:02.504 INFO:tasks.workunit.client.0.smithi028.stdout:8/215: symlink d0/l49 0 2024-07-03T15:44:02.507 INFO:tasks.workunit.client.0.smithi028.stdout:5/225: mknod d0/dd/d13/d19/d26/d41/c4c 0 2024-07-03T15:44:02.510 INFO:tasks.workunit.client.0.smithi028.stdout:7/81: truncate d9/f13 202198 0 2024-07-03T15:44:02.511 INFO:tasks.workunit.client.1.smithi148.stdout:4/980: dread d0/d28/d6e/dc7/faa [0,4194304] 0 2024-07-03T15:44:02.512 INFO:tasks.workunit.client.0.smithi028.stdout:9/221: mkdir d1/d10/d28/d45 0 2024-07-03T15:44:02.517 INFO:tasks.workunit.client.1.smithi148.stdout:4/981: rename d0/d91/fa2 to d0/d8/d13/d67/f15a 0 2024-07-03T15:44:02.518 INFO:tasks.workunit.client.0.smithi028.stdout:3/229: rename d18/d1a/d1e/c36 to d18/d3a/c4b 0 2024-07-03T15:44:02.519 INFO:tasks.workunit.client.0.smithi028.stdout:4/164: getdents d2/d12/d2e 0 2024-07-03T15:44:02.519 INFO:tasks.workunit.client.0.smithi028.stdout:4/165: fsync d2/d12/d19/f9 0 2024-07-03T15:44:02.519 INFO:tasks.workunit.client.0.smithi028.stdout:4/166: chown d2/f2b 65520542 1 2024-07-03T15:44:02.519 INFO:tasks.workunit.client.0.smithi028.stdout:4/167: chown d2/d12/d2d 13 1 2024-07-03T15:44:02.520 INFO:tasks.workunit.client.0.smithi028.stdout:6/155: symlink d4/da/d13/l34 0 2024-07-03T15:44:02.522 INFO:tasks.workunit.client.1.smithi148.stdout:4/982: creat d0/d28/de8/d131/d144/f15b x:0 0 0 2024-07-03T15:44:02.522 INFO:tasks.workunit.client.1.smithi148.stdout:4/983: write d0/d28/d7d/fdd [2026083,58399] 0 2024-07-03T15:44:02.523 INFO:tasks.workunit.client.0.smithi028.stdout:0/149: mknod d2/d1c/c33 0 2024-07-03T15:44:02.523 INFO:tasks.workunit.client.0.smithi028.stdout:0/150: fdatasync d2/d1c/f21 0 2024-07-03T15:44:02.523 INFO:tasks.workunit.client.0.smithi028.stdout:0/151: write d2/d3/f2c [383640,60343] 0 2024-07-03T15:44:02.523 INFO:tasks.workunit.client.0.smithi028.stdout:0/152: stat d2/d3/f19 0 2024-07-03T15:44:02.527 INFO:tasks.workunit.client.0.smithi028.stdout:8/216: mkdir d0/dd/d22/d40/d4a 0 2024-07-03T15:44:02.533 INFO:tasks.workunit.client.0.smithi028.stdout:5/226: rename d0/dd/d14/l38 to d0/dd/d13/d47/l4d 0 2024-07-03T15:44:02.534 INFO:tasks.workunit.client.0.smithi028.stdout:3/230: symlink d18/d40/l4c 0 2024-07-03T15:44:02.536 INFO:tasks.workunit.client.0.smithi028.stdout:5/227: creat d0/dd/d13/d29/f4e x:0 0 0 2024-07-03T15:44:02.537 INFO:tasks.workunit.client.0.smithi028.stdout:5/228: chown d0/dd/d13/d19/d26/f42 1852963 1 2024-07-03T15:44:02.537 INFO:tasks.workunit.client.0.smithi028.stdout:3/231: creat d18/d43/f4d x:0 0 0 2024-07-03T15:44:02.537 INFO:tasks.workunit.client.0.smithi028.stdout:3/232: write d18/d1a/d1e/f38 [478955,71695] 0 2024-07-03T15:44:02.537 INFO:tasks.workunit.client.0.smithi028.stdout:3/233: write d18/d1a/f2a [350225,93522] 0 2024-07-03T15:44:02.542 INFO:tasks.workunit.client.0.smithi028.stdout:0/153: read d2/d3/f7 [766053,39781] 0 2024-07-03T15:44:02.542 INFO:tasks.workunit.client.0.smithi028.stdout:3/234: symlink d18/l4e 0 2024-07-03T15:44:02.543 INFO:tasks.workunit.client.1.smithi148.stdout:0/863: getdents da/d58/d90/dd1 0 2024-07-03T15:44:02.543 INFO:tasks.workunit.client.0.smithi028.stdout:6/156: write d4/d15/f20 [3034978,24999] 0 2024-07-03T15:44:02.545 INFO:tasks.workunit.client.0.smithi028.stdout:6/157: dread d4/d12/f21 [0,4194304] 0 2024-07-03T15:44:02.545 INFO:tasks.workunit.client.0.smithi028.stdout:6/158: readlink d4/d15/l25 0 2024-07-03T15:44:02.547 INFO:tasks.workunit.client.0.smithi028.stdout:3/235: truncate ff 815671 0 2024-07-03T15:44:02.547 INFO:tasks.workunit.client.0.smithi028.stdout:0/154: creat d2/d1c/d32/f34 x:0 0 0 2024-07-03T15:44:02.547 INFO:tasks.workunit.client.0.smithi028.stdout:6/159: dread d4/d12/f21 [0,4194304] 0 2024-07-03T15:44:02.547 INFO:tasks.workunit.client.0.smithi028.stdout:6/160: chown d4/d15/f2a 153 1 2024-07-03T15:44:02.548 INFO:tasks.workunit.client.0.smithi028.stdout:4/168: dwrite d2/f4 [0,4194304] 0 2024-07-03T15:44:02.548 INFO:tasks.workunit.client.0.smithi028.stdout:4/169: stat c0 0 2024-07-03T15:44:02.548 INFO:tasks.workunit.client.0.smithi028.stdout:4/170: chown d2/d12/d2d/d2f 200 1 2024-07-03T15:44:02.548 INFO:tasks.workunit.client.0.smithi028.stdout:8/217: write d0/d2/d3/dc/f21 [2405365,11545] 0 2024-07-03T15:44:02.549 INFO:tasks.workunit.client.0.smithi028.stdout:8/218: dread d0/d2/f27 [0,4194304] 0 2024-07-03T15:44:02.549 INFO:tasks.workunit.client.0.smithi028.stdout:8/219: write d0/dd/d22/f25 [4629762,14158] 0 2024-07-03T15:44:02.550 INFO:tasks.workunit.client.1.smithi148.stdout:0/864: unlink da/d58/dbd/d4a/c7f 0 2024-07-03T15:44:02.550 INFO:tasks.workunit.client.1.smithi148.stdout:0/865: dread - da/f117 zero size 2024-07-03T15:44:02.551 INFO:tasks.workunit.client.0.smithi028.stdout:0/155: unlink d2/c23 0 2024-07-03T15:44:02.551 INFO:tasks.workunit.client.0.smithi028.stdout:0/156: creat d2/d1c/d32/f35 x:0 0 0 2024-07-03T15:44:02.551 INFO:tasks.workunit.client.0.smithi028.stdout:0/157: fsync d2/d3/f1d 0 2024-07-03T15:44:02.552 INFO:tasks.workunit.client.0.smithi028.stdout:5/229: dread d0/dd/d13/d19/f21 [0,4194304] 0 2024-07-03T15:44:02.552 INFO:tasks.workunit.client.1.smithi148.stdout:4/984: dwrite d0/d28/d6e/dc7/fe3 [0,4194304] 0 2024-07-03T15:44:02.552 INFO:tasks.workunit.client.1.smithi148.stdout:4/985: chown d0/d28/de8/d131/f14a 0 1 2024-07-03T15:44:02.553 INFO:tasks.workunit.client.0.smithi028.stdout:4/171: getdents d2/d12 0 2024-07-03T15:44:02.555 INFO:tasks.workunit.client.0.smithi028.stdout:8/220: unlink d0/dd/l3b 0 2024-07-03T15:44:02.557 INFO:tasks.workunit.client.0.smithi028.stdout:0/158: mkdir d2/d1c/d32/d36 0 2024-07-03T15:44:02.558 INFO:tasks.workunit.client.0.smithi028.stdout:3/236: write d18/d1a/f28 [1170548,97999] 0 2024-07-03T15:44:02.559 INFO:tasks.workunit.client.1.smithi148.stdout:1/970: truncate d0/d2/d2e/fa7 8448669 0 2024-07-03T15:44:02.559 INFO:tasks.workunit.client.1.smithi148.stdout:1/971: fsync d0/d2/d11/d3a/dc5/d119/d2f/d72/df9/d42/d62/f4f 0 2024-07-03T15:44:02.561 INFO:tasks.workunit.client.0.smithi028.stdout:5/230: creat d0/dd/d1e/d3b/f4f x:0 0 0 2024-07-03T15:44:02.561 INFO:tasks.workunit.client.0.smithi028.stdout:5/231: chown d0/dd/d14/l18 1274 1 2024-07-03T15:44:02.562 INFO:tasks.workunit.client.1.smithi148.stdout:4/986: read d0/d28/da7/d2f/d41/f124 [132954,102386] 0 2024-07-03T15:44:02.564 INFO:tasks.workunit.client.0.smithi028.stdout:4/172: creat d2/d12/d19/f36 x:0 0 0 2024-07-03T15:44:02.564 INFO:tasks.workunit.client.0.smithi028.stdout:4/173: chown d2/d12/d2c 1363705 1 2024-07-03T15:44:02.567 INFO:tasks.workunit.client.1.smithi148.stdout:0/866: dwrite da/d17/d2a/fa9 [0,4194304] 0 2024-07-03T15:44:02.568 INFO:tasks.workunit.client.1.smithi148.stdout:4/987: creat d0/d8/d13/f15c x:0 0 0 2024-07-03T15:44:02.568 INFO:tasks.workunit.client.1.smithi148.stdout:4/988: dread - d0/d28/de8/d132/fd0 zero size 2024-07-03T15:44:02.568 INFO:tasks.workunit.client.0.smithi028.stdout:8/221: mkdir d0/d2/d3/dc/d11/d4b 0 2024-07-03T15:44:02.571 INFO:tasks.workunit.client.0.smithi028.stdout:7/82: unlink d9/f13 0 2024-07-03T15:44:02.578 INFO:tasks.workunit.client.0.smithi028.stdout:3/237: creat d18/d1a/f4f x:0 0 0 2024-07-03T15:44:02.578 INFO:tasks.workunit.client.1.smithi148.stdout:1/972: dwrite d0/d2/d11/d3a/dc5/d119/d2f/d72/df9/fdc [0,4194304] 0 2024-07-03T15:44:02.579 INFO:tasks.workunit.client.1.smithi148.stdout:1/973: dread - d0/d2/d11/d3a/dc5/d119/d2f/d72/fe3 zero size 2024-07-03T15:44:02.580 INFO:tasks.workunit.client.1.smithi148.stdout:0/867: creat da/d58/dbd/d24/dc8/f11d x:0 0 0 2024-07-03T15:44:02.582 INFO:tasks.workunit.client.1.smithi148.stdout:4/989: symlink d0/d8/d3a/db8/dd3/l15d 0 2024-07-03T15:44:02.587 INFO:tasks.workunit.client.0.smithi028.stdout:7/83: dwrite f2 [0,4194304] 0 2024-07-03T15:44:02.587 INFO:tasks.workunit.client.0.smithi028.stdout:7/84: write f6 [591658,66162] 0 2024-07-03T15:44:02.590 INFO:tasks.workunit.client.1.smithi148.stdout:1/974: link d0/d2/d11/d3a/dc5/d119/d2f/d72/df9/d42/l10f d0/d2/d11/d3a/dc5/d119/d44/d54/d5d/d10d/l13f 0 2024-07-03T15:44:02.594 INFO:tasks.workunit.client.1.smithi148.stdout:0/868: rename da/d58/dbd/d13/fd9 to da/d58/dbd/d94/f11e 0 2024-07-03T15:44:02.594 INFO:tasks.workunit.client.1.smithi148.stdout:0/869: write da/d58/dbd/d24/dc8/d8a/dab/f6c [3139039,27871] 0 2024-07-03T15:44:02.595 INFO:tasks.workunit.client.0.smithi028.stdout:5/232: mkdir d0/dd/d13/d47/d50 0 2024-07-03T15:44:02.595 INFO:tasks.workunit.client.0.smithi028.stdout:5/233: fdatasync d0/dd/d13/f39 0 2024-07-03T15:44:02.598 INFO:tasks.workunit.client.0.smithi028.stdout:8/222: mkdir d0/d2/d3/dc/d11/d4c 0 2024-07-03T15:44:02.598 INFO:tasks.workunit.client.1.smithi148.stdout:0/870: creat da/d17/d2a/d2f/d93/da3/dda/f11f x:0 0 0 2024-07-03T15:44:02.598 INFO:tasks.workunit.client.1.smithi148.stdout:0/871: dread - da/d58/dbd/d24/d84/ff7 zero size 2024-07-03T15:44:02.600 INFO:tasks.workunit.client.1.smithi148.stdout:4/990: fsync d0/d8/d13/f15c 0 2024-07-03T15:44:02.602 INFO:tasks.workunit.client.1.smithi148.stdout:1/975: write d0/d2/d11/d3a/dc5/f11d [3162370,50025] 0 2024-07-03T15:44:02.602 INFO:tasks.workunit.client.0.smithi028.stdout:0/159: rmdir d2 39 2024-07-03T15:44:02.602 INFO:tasks.workunit.client.0.smithi028.stdout:0/160: readlink d2/d3/l13 0 2024-07-03T15:44:02.603 INFO:tasks.workunit.client.0.smithi028.stdout:3/238: mknod d18/c50 0 2024-07-03T15:44:02.603 INFO:tasks.workunit.client.0.smithi028.stdout:3/239: truncate d18/d3a/f3f 223519 0 2024-07-03T15:44:02.603 INFO:tasks.workunit.client.0.smithi028.stdout:3/240: readlink d18/l4e 0 2024-07-03T15:44:02.603 INFO:tasks.workunit.client.0.smithi028.stdout:3/241: fdatasync d18/f2d 0 2024-07-03T15:44:02.604 INFO:tasks.workunit.client.0.smithi028.stdout:3/242: creat d18/f51 x:0 0 0 2024-07-03T15:44:02.604 INFO:tasks.workunit.client.0.smithi028.stdout:3/243: creat d18/f52 x:0 0 0 2024-07-03T15:44:02.604 INFO:tasks.workunit.client.0.smithi028.stdout:3/244: creat d18/d1a/d1e/f53 x:0 0 0 2024-07-03T15:44:02.605 INFO:tasks.workunit.client.0.smithi028.stdout:5/234: mknod d0/dd/d1e/d3b/d3f/c51 0 2024-07-03T15:44:02.605 INFO:tasks.workunit.client.1.smithi148.stdout:0/872: dread da/d17/d2a/f4b [0,4194304] 0 2024-07-03T15:44:02.609 INFO:tasks.workunit.client.0.smithi028.stdout:8/223: unlink d0/f20 0 2024-07-03T15:44:02.611 INFO:tasks.workunit.client.0.smithi028.stdout:7/85: dwrite f4 [0,4194304] 0 2024-07-03T15:44:02.613 INFO:tasks.workunit.client.0.smithi028.stdout:8/224: dread d0/d2/d3/dc/f1f [0,4194304] 0 2024-07-03T15:44:02.614 INFO:tasks.workunit.client.0.smithi028.stdout:3/245: symlink d18/d1a/d1e/l54 0 2024-07-03T15:44:02.616 INFO:tasks.workunit.client.0.smithi028.stdout:5/235: creat d0/dd/df/f52 x:0 0 0 2024-07-03T15:44:02.616 INFO:tasks.workunit.client.0.smithi028.stdout:5/236: write d0/dd/df/f52 [660654,85057] 0 2024-07-03T15:44:02.616 INFO:tasks.workunit.client.0.smithi028.stdout:3/246: write d18/d1a/f2a [804069,122849] 0 2024-07-03T15:44:02.616 INFO:tasks.workunit.client.0.smithi028.stdout:3/247: creat d18/f55 x:0 0 0 2024-07-03T15:44:02.616 INFO:tasks.workunit.client.0.smithi028.stdout:3/248: stat d18/d1a/l49 0 2024-07-03T15:44:02.617 INFO:tasks.workunit.client.0.smithi028.stdout:0/161: rename d2/d3/c12 to d2/d3/c37 0 2024-07-03T15:44:02.618 INFO:tasks.workunit.client.0.smithi028.stdout:0/162: chown d2/f15 1273229 1 2024-07-03T15:44:02.618 INFO:tasks.workunit.client.0.smithi028.stdout:0/163: chown d2/d1c/l30 60300 1 2024-07-03T15:44:02.618 INFO:tasks.workunit.client.0.smithi028.stdout:0/164: write d2/d3/f1d [247321,57743] 0 2024-07-03T15:44:02.619 INFO:tasks.workunit.client.0.smithi028.stdout:5/237: creat d0/dd/f53 x:0 0 0 2024-07-03T15:44:02.619 INFO:tasks.workunit.client.1.smithi148.stdout:0/873: dread da/d17/d2a/d2f/f87 [0,4194304] 0 2024-07-03T15:44:02.623 INFO:tasks.workunit.client.0.smithi028.stdout:3/249: rename l11 to d18/d1a/l56 0 2024-07-03T15:44:02.623 INFO:tasks.workunit.client.0.smithi028.stdout:3/250: write d18/d1a/d1e/f3e [402986,4565] 0 2024-07-03T15:44:02.623 INFO:tasks.workunit.client.0.smithi028.stdout:3/251: truncate d18/d3a/f39 900007 0 2024-07-03T15:44:02.624 INFO:tasks.workunit.client.1.smithi148.stdout:0/874: truncate da/d58/dbd/d94/dac/ddd/f6b 2320223 0 2024-07-03T15:44:02.626 INFO:tasks.workunit.client.0.smithi028.stdout:7/86: dread d9/f10 [0,4194304] 0 2024-07-03T15:44:02.626 INFO:tasks.workunit.client.0.smithi028.stdout:0/165: creat d2/d1c/d32/f38 x:0 0 0 2024-07-03T15:44:02.627 INFO:tasks.workunit.client.0.smithi028.stdout:0/166: chown d2/f15 1 1 2024-07-03T15:44:02.627 INFO:tasks.workunit.client.0.smithi028.stdout:0/167: truncate d2/d3/f27 921787 0 2024-07-03T15:44:02.627 INFO:tasks.workunit.client.0.smithi028.stdout:0/168: chown d2/c4 5407906 1 2024-07-03T15:44:02.637 INFO:tasks.workunit.client.0.smithi028.stdout:8/225: write d0/d2/d3/dc/d11/d12/f2f [6836526,66316] 0 2024-07-03T15:44:02.642 INFO:tasks.workunit.client.1.smithi148.stdout:0/875: write da/d58/dbd/d13/d1d/d3e/f6e [2274227,4087] 0 2024-07-03T15:44:02.645 INFO:tasks.workunit.client.1.smithi148.stdout:1/976: dwrite d0/d2/d11/d3a/dc5/d116/f9e [0,4194304] 0 2024-07-03T15:44:02.645 INFO:tasks.workunit.client.1.smithi148.stdout:1/977: read d0/d2/d11/d3a/dc5/d119/d89/f95 [1579302,129747] 0 2024-07-03T15:44:02.645 INFO:tasks.workunit.client.1.smithi148.stdout:4/991: dwrite d0/d8/d13/db4/fcd [4194304,4194304] 0 2024-07-03T15:44:02.646 INFO:tasks.workunit.client.0.smithi028.stdout:0/169: dwrite d2/ff [0,4194304] 0 2024-07-03T15:44:02.646 INFO:tasks.workunit.client.0.smithi028.stdout:0/170: dread - d2/d1c/f20 zero size 2024-07-03T15:44:02.646 INFO:tasks.workunit.client.0.smithi028.stdout:0/171: creat d2/d1c/d32/f39 x:0 0 0 2024-07-03T15:44:02.658 INFO:tasks.workunit.client.1.smithi148.stdout:4/992: write d0/d28/f69 [7664424,15979] 0 2024-07-03T15:44:02.669 INFO:tasks.workunit.client.0.smithi028.stdout:0/172: dwrite d2/d1c/d32/f38 [0,4194304] 0 2024-07-03T15:44:02.671 INFO:tasks.workunit.client.0.smithi028.stdout:0/173: fdatasync d2/d3/f1d 0 2024-07-03T15:44:02.676 INFO:tasks.workunit.client.1.smithi148.stdout:0/876: dwrite da/d58/dbd/d13/d1d/d3e/d85/d8e/f10a [0,4194304] 0 2024-07-03T15:44:02.690 INFO:tasks.workunit.client.0.smithi028.stdout:5/238: truncate d0/dd/df/f16 1976676 0 2024-07-03T15:44:02.690 INFO:tasks.workunit.client.0.smithi028.stdout:5/239: write d0/f9 [9254791,42936] 0 2024-07-03T15:44:02.694 INFO:tasks.workunit.client.0.smithi028.stdout:7/87: creat d9/f1a x:0 0 0 2024-07-03T15:44:02.694 INFO:tasks.workunit.client.0.smithi028.stdout:7/88: fsync f7 0 2024-07-03T15:44:02.695 INFO:tasks.workunit.client.0.smithi028.stdout:3/252: fsync d18/f55 0 2024-07-03T15:44:02.704 INFO:tasks.workunit.client.0.smithi028.stdout:1/322: sync 2024-07-03T15:44:02.704 INFO:tasks.workunit.client.0.smithi028.stdout:2/198: sync 2024-07-03T15:44:02.704 INFO:tasks.workunit.client.0.smithi028.stdout:2/199: chown da/d1e 18628820 1 2024-07-03T15:44:02.705 INFO:tasks.workunit.client.0.smithi028.stdout:1/323: dread d4/d2c/d3a/d5c/d6c/d7d/d59/f2d [0,4194304] 0 2024-07-03T15:44:02.715 INFO:tasks.workunit.client.0.smithi028.stdout:5/240: creat d0/dd/d13/d19/d26/d41/f54 x:0 0 0 2024-07-03T15:44:02.717 INFO:tasks.workunit.client.0.smithi028.stdout:8/226: rmdir d0/d2/d3/dc/d11 39 2024-07-03T15:44:02.722 INFO:tasks.workunit.client.0.smithi028.stdout:2/200: symlink da/d24/d2c/l48 0 2024-07-03T15:44:02.723 INFO:tasks.workunit.client.0.smithi028.stdout:1/324: mkdir d4/d83 0 2024-07-03T15:44:02.726 INFO:tasks.workunit.client.0.smithi028.stdout:0/174: getdents d2/d1c/d32 0 2024-07-03T15:44:02.726 INFO:tasks.workunit.client.0.smithi028.stdout:0/175: fsync d2/f15 0 2024-07-03T15:44:02.726 INFO:tasks.workunit.client.0.smithi028.stdout:2/201: dread da/ff [0,4194304] 0 2024-07-03T15:44:02.727 INFO:tasks.workunit.client.0.smithi028.stdout:0/176: write d2/d1c/f2b [369417,102228] 0 2024-07-03T15:44:02.727 INFO:tasks.workunit.client.0.smithi028.stdout:0/177: readlink d2/l1b 0 2024-07-03T15:44:02.727 INFO:tasks.workunit.client.0.smithi028.stdout:0/178: readlink d2/d1c/l24 0 2024-07-03T15:44:02.729 INFO:tasks.workunit.client.0.smithi028.stdout:5/241: symlink d0/dd/df/l55 0 2024-07-03T15:44:02.729 INFO:tasks.workunit.client.0.smithi028.stdout:5/242: fdatasync d0/dd/d13/d29/f35 0 2024-07-03T15:44:02.729 INFO:tasks.workunit.client.0.smithi028.stdout:5/243: write d0/dd/d13/d19/f1f [529948,111882] 0 2024-07-03T15:44:02.729 INFO:tasks.workunit.client.0.smithi028.stdout:5/244: dread - d0/dd/d1e/f36 zero size 2024-07-03T15:44:02.729 INFO:tasks.workunit.client.0.smithi028.stdout:5/245: chown d0/l1d 1 1 2024-07-03T15:44:02.730 INFO:tasks.workunit.client.0.smithi028.stdout:8/227: dwrite d0/d2/d3/dc/d11/d12/f15 [4194304,4194304] 0 2024-07-03T15:44:02.730 INFO:tasks.workunit.client.0.smithi028.stdout:8/228: readlink d0/d2/d3/dc/d11/d12/l14 0 2024-07-03T15:44:02.730 INFO:tasks.workunit.client.0.smithi028.stdout:7/89: unlink d9/c18 0 2024-07-03T15:44:02.731 INFO:tasks.workunit.client.0.smithi028.stdout:1/325: mkdir d4/d3c/d54/d84 0 2024-07-03T15:44:02.732 INFO:tasks.workunit.client.1.smithi148.stdout:0/877: link da/d58/dbd/d13/d1d/d3e/d85/d8e/lf4 da/d17/d2a/d2f/l120 0 2024-07-03T15:44:02.734 INFO:tasks.workunit.client.0.smithi028.stdout:0/179: creat d2/f3a x:0 0 0 2024-07-03T15:44:02.734 INFO:tasks.workunit.client.0.smithi028.stdout:0/180: read - d2/d1c/f20 zero size 2024-07-03T15:44:02.734 INFO:tasks.workunit.client.1.smithi148.stdout:0/878: mknod da/d17/d2a/d2f/d93/da3/dda/c121 0 2024-07-03T15:44:02.735 INFO:tasks.workunit.client.0.smithi028.stdout:5/246: creat d0/dd/d1e/f56 x:0 0 0 2024-07-03T15:44:02.737 INFO:tasks.workunit.client.0.smithi028.stdout:8/229: rename d0/d2/d3/dc/d11/d4b to d0/d2/d3/dc/d4d 0 2024-07-03T15:44:02.737 INFO:tasks.workunit.client.0.smithi028.stdout:8/230: stat d0/d41/d47 0 2024-07-03T15:44:02.740 INFO:tasks.workunit.client.0.smithi028.stdout:2/202: symlink da/d23/l49 0 2024-07-03T15:44:02.740 INFO:tasks.workunit.client.1.smithi148.stdout:0/879: dread da/d58/dbd/d13/fb8 [0,4194304] 0 2024-07-03T15:44:02.743 INFO:tasks.workunit.client.0.smithi028.stdout:8/231: symlink d0/l4e 0 2024-07-03T15:44:02.745 INFO:tasks.workunit.client.0.smithi028.stdout:8/232: dread d0/f2a [0,4194304] 0 2024-07-03T15:44:02.746 INFO:tasks.workunit.client.0.smithi028.stdout:7/90: rename d9/f17 to d9/f1b 0 2024-07-03T15:44:02.749 INFO:tasks.workunit.client.0.smithi028.stdout:2/203: link da/d23/d30/c31 da/d1e/c4a 0 2024-07-03T15:44:02.751 INFO:tasks.workunit.client.0.smithi028.stdout:8/233: mknod d0/d2/d3/c4f 0 2024-07-03T15:44:02.751 INFO:tasks.workunit.client.0.smithi028.stdout:7/91: mkdir d9/d1c 0 2024-07-03T15:44:02.752 INFO:tasks.workunit.client.0.smithi028.stdout:7/92: readlink d9/ld 0 2024-07-03T15:44:02.765 INFO:tasks.workunit.client.0.smithi028.stdout:2/204: write f9 [126899,102350] 0 2024-07-03T15:44:02.766 INFO:tasks.workunit.client.0.smithi028.stdout:2/205: fdatasync da/f17 0 2024-07-03T15:44:02.766 INFO:tasks.workunit.client.0.smithi028.stdout:2/206: creat da/d42/f4b x:0 0 0 2024-07-03T15:44:02.768 INFO:tasks.workunit.client.0.smithi028.stdout:2/207: mknod da/d24/d2c/c4c 0 2024-07-03T15:44:02.769 INFO:tasks.workunit.client.0.smithi028.stdout:1/326: dwrite d4/d3c/d54/d79/d16/d57/d66/f67 [0,4194304] 0 2024-07-03T15:44:02.769 INFO:tasks.workunit.client.0.smithi028.stdout:7/93: write f2 [3728495,77166] 0 2024-07-03T15:44:02.769 INFO:tasks.workunit.client.0.smithi028.stdout:5/247: dwrite d0/dd/d1e/f56 [0,4194304] 0 2024-07-03T15:44:02.770 INFO:tasks.workunit.client.0.smithi028.stdout:8/234: dwrite d0/d2/f27 [0,4194304] 0 2024-07-03T15:44:02.771 INFO:tasks.workunit.client.1.smithi148.stdout:0/880: dwrite da/d58/dbd/d24/dc8/d8a/dab/f6c [4194304,4194304] 0 2024-07-03T15:44:02.771 INFO:tasks.workunit.client.1.smithi148.stdout:0/881: fsync da/d58/dbd/d94/dac/ddd/f4e 0 2024-07-03T15:44:02.772 INFO:tasks.workunit.client.0.smithi028.stdout:1/327: rename d4/de/f52 to d4/de/f85 0 2024-07-03T15:44:02.774 INFO:tasks.workunit.client.1.smithi148.stdout:0/882: unlink da/d58/dbd/d13/d1d/d3e/d85/d8e/lf4 0 2024-07-03T15:44:02.776 INFO:tasks.workunit.client.0.smithi028.stdout:2/208: write da/d1e/f21 [2378757,50866] 0 2024-07-03T15:44:02.777 INFO:tasks.workunit.client.0.smithi028.stdout:5/248: unlink d0/dd/d13/d29/l49 0 2024-07-03T15:44:02.778 INFO:tasks.workunit.client.1.smithi148.stdout:0/883: creat da/d58/dbd/d24/d84/d114/f122 x:0 0 0 2024-07-03T15:44:02.784 INFO:tasks.workunit.client.0.smithi028.stdout:1/328: dread f2 [0,4194304] 0 2024-07-03T15:44:02.785 INFO:tasks.workunit.client.0.smithi028.stdout:1/329: write d4/d2c/d3a/d5c/d6c/d7d/d59/f2d [27320,25322] 0 2024-07-03T15:44:02.789 INFO:tasks.workunit.client.0.smithi028.stdout:1/330: symlink d4/d2c/d3a/d5c/l86 0 2024-07-03T15:44:02.789 INFO:tasks.workunit.client.1.smithi148.stdout:0/884: write da/d58/dbd/d13/d14/d50/d95/f97 [270497,13537] 0 2024-07-03T15:44:02.794 INFO:tasks.workunit.client.0.smithi028.stdout:2/209: dread da/f17 [0,4194304] 0 2024-07-03T15:44:02.802 INFO:tasks.workunit.client.0.smithi028.stdout:9/222: sync 2024-07-03T15:44:02.803 INFO:tasks.workunit.client.0.smithi028.stdout:9/223: rename d1/c1a to d1/d2a/c46 0 2024-07-03T15:44:02.805 INFO:tasks.workunit.client.0.smithi028.stdout:9/224: dread d1/d16/f2e [0,4194304] 0 2024-07-03T15:44:02.807 INFO:tasks.workunit.client.0.smithi028.stdout:2/210: read f4 [375515,70309] 0 2024-07-03T15:44:02.808 INFO:tasks.workunit.client.1.smithi148.stdout:0/885: dwrite da/ff8 [0,4194304] 0 2024-07-03T15:44:02.808 INFO:tasks.workunit.client.0.smithi028.stdout:2/211: truncate da/d24/d2c/f32 91944 0 2024-07-03T15:44:02.808 INFO:tasks.workunit.client.0.smithi028.stdout:2/212: creat da/d42/f4d x:0 0 0 2024-07-03T15:44:02.822 INFO:tasks.workunit.client.0.smithi028.stdout:9/225: dwrite d1/dd/f1b [0,4194304] 0 2024-07-03T15:44:02.822 INFO:tasks.workunit.client.0.smithi028.stdout:9/226: dread - d1/f43 zero size 2024-07-03T15:44:02.822 INFO:tasks.workunit.client.0.smithi028.stdout:9/227: stat d1/d16/f1c 0 2024-07-03T15:44:02.873 INFO:tasks.workunit.client.1.smithi148.stdout:0/886: mknod da/d58/dbd/d13/d14/c123 0 2024-07-03T15:44:02.879 INFO:tasks.workunit.client.1.smithi148.stdout:1/978: sync 2024-07-03T15:44:02.883 INFO:tasks.workunit.client.0.smithi028.stdout:9/228: symlink d1/d10/l47 0 2024-07-03T15:44:02.888 INFO:tasks.workunit.client.1.smithi148.stdout:0/887: rename da/d17/f8b to da/d58/dbd/d13/d1d/d3e/d85/d8e/dd4/f124 0 2024-07-03T15:44:02.888 INFO:tasks.workunit.client.1.smithi148.stdout:4/993: sync 2024-07-03T15:44:02.890 INFO:tasks.workunit.client.1.smithi148.stdout:1/979: mknod d0/d2/dd0/c140 0 2024-07-03T15:44:02.890 INFO:tasks.workunit.client.0.smithi028.stdout:2/213: link da/d24/d2c/c29 da/d1e/c4e 0 2024-07-03T15:44:02.891 INFO:tasks.workunit.client.0.smithi028.stdout:2/214: creat da/d24/d2c/f4f x:0 0 0 2024-07-03T15:44:02.891 INFO:tasks.workunit.client.0.smithi028.stdout:2/215: creat da/d42/f50 x:0 0 0 2024-07-03T15:44:02.894 INFO:tasks.workunit.client.1.smithi148.stdout:0/888: rename da/d58/dbd/d24/dc8/d8a/dab to da/d17/d125 0 2024-07-03T15:44:02.896 INFO:tasks.workunit.client.1.smithi148.stdout:4/994: symlink d0/d28/d7d/l15e 0 2024-07-03T15:44:02.897 INFO:tasks.workunit.client.1.smithi148.stdout:0/889: dread da/d58/dbd/d13/d14/d50/d95/ffb [0,4194304] 0 2024-07-03T15:44:02.897 INFO:tasks.workunit.client.1.smithi148.stdout:0/890: creat da/d58/dbd/d94/dac/ddd/d88/f126 x:0 0 0 2024-07-03T15:44:02.897 INFO:tasks.workunit.client.1.smithi148.stdout:1/980: creat d0/d2/dae/d114/f141 x:0 0 0 2024-07-03T15:44:02.898 INFO:tasks.workunit.client.0.smithi028.stdout:2/216: creat da/d23/d30/f51 x:0 0 0 2024-07-03T15:44:02.898 INFO:tasks.workunit.client.0.smithi028.stdout:2/217: dread - da/d23/d30/f51 zero size 2024-07-03T15:44:02.898 INFO:tasks.workunit.client.1.smithi148.stdout:4/995: truncate d0/d8/d3a/db8/dd3/f111 496673 0 2024-07-03T15:44:02.901 INFO:tasks.workunit.client.0.smithi028.stdout:2/218: mknod da/d1e/c52 0 2024-07-03T15:44:02.901 INFO:tasks.workunit.client.0.smithi028.stdout:9/229: dwrite d1/f2b [0,4194304] 0 2024-07-03T15:44:02.902 INFO:tasks.workunit.client.0.smithi028.stdout:9/230: fsync d1/dd/f1b 0 2024-07-03T15:44:02.902 INFO:tasks.workunit.client.0.smithi028.stdout:9/231: readlink d1/l2c 0 2024-07-03T15:44:02.903 INFO:tasks.workunit.client.1.smithi148.stdout:1/981: dread d0/d2/d11/d3a/dc5/d116/ff2 [0,4194304] 0 2024-07-03T15:44:02.903 INFO:tasks.workunit.client.1.smithi148.stdout:4/996: link d0/d8/d44/f6a d0/d28/da7/f15f 0 2024-07-03T15:44:02.903 INFO:tasks.workunit.client.1.smithi148.stdout:4/997: readlink d0/d8/d44/l83 0 2024-07-03T15:44:02.903 INFO:tasks.workunit.client.1.smithi148.stdout:4/998: chown d0/d28/da7/d2f/d41/l77 4 1 2024-07-03T15:44:02.903 INFO:tasks.workunit.client.1.smithi148.stdout:4/999: creat d0/d28/d102/f160 x:0 0 0 2024-07-03T15:44:02.904 INFO:tasks.workunit.client.1.smithi148.stdout:1/982: truncate d0/d2/d11/d3a/dc5/d119/d2f/d72/df9/d42/d62/f71 558592 0 2024-07-03T15:44:02.908 INFO:tasks.workunit.client.0.smithi028.stdout:9/232: dread d1/dd/f26 [0,4194304] 0 2024-07-03T15:44:02.909 INFO:tasks.workunit.client.0.smithi028.stdout:2/219: link da/d23/l49 da/d23/l53 0 2024-07-03T15:44:02.910 INFO:tasks.workunit.client.0.smithi028.stdout:2/220: write da/d24/d2c/f4f [195608,108868] 0 2024-07-03T15:44:02.910 INFO:tasks.workunit.client.0.smithi028.stdout:2/221: readlink da/d23/d30/l40 0 2024-07-03T15:44:02.910 INFO:tasks.workunit.client.0.smithi028.stdout:2/222: write da/d42/f4d [347763,114999] 0 2024-07-03T15:44:02.910 INFO:tasks.workunit.client.1.smithi148.stdout:1/983: symlink d0/d2/dae/d114/l142 0 2024-07-03T15:44:02.911 INFO:tasks.workunit.client.0.smithi028.stdout:2/223: rename da/d24/f37 to da/d24/d2c/f54 0 2024-07-03T15:44:02.911 INFO:tasks.workunit.client.0.smithi028.stdout:2/224: creat da/d24/f55 x:0 0 0 2024-07-03T15:44:02.912 INFO:tasks.workunit.client.0.smithi028.stdout:2/225: write da/f1d [938294,52534] 0 2024-07-03T15:44:02.912 INFO:tasks.workunit.client.0.smithi028.stdout:2/226: stat da/d24/d2c/l3c 0 2024-07-03T15:44:02.912 INFO:tasks.workunit.client.0.smithi028.stdout:2/227: readlink da/d23/l49 0 2024-07-03T15:44:02.913 INFO:tasks.workunit.client.1.smithi148.stdout:1/984: mkdir d0/d2/d11/d3a/dc5/d119/d89/d143 0 2024-07-03T15:44:02.913 INFO:tasks.workunit.client.1.smithi148.stdout:1/985: truncate d0/d2/d11/d3a/dc5/d119/d2f/d72/df9/d42/fb0 151642 0 2024-07-03T15:44:02.915 INFO:tasks.workunit.client.0.smithi028.stdout:2/228: mkdir da/d1e/d56 0 2024-07-03T15:44:02.916 INFO:tasks.workunit.client.1.smithi148.stdout:1/986: dread d0/d2/dd0/f132 [0,4194304] 0 2024-07-03T15:44:02.917 INFO:tasks.workunit.client.1.smithi148.stdout:1/987: rmdir d0/d2/d11/d3a/dc5/d119 39 2024-07-03T15:44:02.919 INFO:tasks.workunit.client.1.smithi148.stdout:1/988: creat d0/d2/d11/d1e/d9d/dc0/f144 x:0 0 0 2024-07-03T15:44:02.920 INFO:tasks.workunit.client.1.smithi148.stdout:1/989: dread d0/d2/d11/d1e/d9d/dc2/fd8 [0,4194304] 0 2024-07-03T15:44:02.920 INFO:tasks.workunit.client.1.smithi148.stdout:1/990: fdatasync d0/d2/dd0/f13b 0 2024-07-03T15:44:02.921 INFO:tasks.workunit.client.1.smithi148.stdout:1/991: dread d0/d2/d11/d1e/d9d/dc0/ff1 [0,4194304] 0 2024-07-03T15:44:02.921 INFO:tasks.workunit.client.1.smithi148.stdout:1/992: dread - d0/d2/dd0/f13b zero size 2024-07-03T15:44:02.922 INFO:tasks.workunit.client.1.smithi148.stdout:1/993: rmdir d0/d2/d11/d3a/dc5/d119/d2f 39 2024-07-03T15:44:02.924 INFO:tasks.workunit.client.1.smithi148.stdout:1/994: creat d0/d2/d11/d3a/dc5/d119/d44/d54/d105/f145 x:0 0 0 2024-07-03T15:44:02.924 INFO:tasks.workunit.client.1.smithi148.stdout:1/995: write d0/d2/d11/d3a/dc5/d116/d14/d29/de7/f137 [5011437,123694] 0 2024-07-03T15:44:02.925 INFO:tasks.workunit.client.1.smithi148.stdout:1/996: dread - d0/d2/d11/d3a/dc5/d119/d2f/d72/df9/d42/d62/d7b/f112 zero size 2024-07-03T15:44:02.927 INFO:tasks.workunit.client.1.smithi148.stdout:1/997: link d0/d2/d11/d3a/dc5/d119/f83 d0/d2/d11/d3a/dc5/d119/d44/d54/d105/dc7/f146 0 2024-07-03T15:44:02.928 INFO:tasks.workunit.client.1.smithi148.stdout:1/998: creat d0/d2/d11/d1e/d9d/dc2/de2/d12d/f147 x:0 0 0 2024-07-03T15:44:02.929 INFO:tasks.workunit.client.0.smithi028.stdout:2/229: write f4 [6203935,86834] 0 2024-07-03T15:44:02.929 INFO:tasks.workunit.client.0.smithi028.stdout:2/230: creat da/d42/f57 x:0 0 0 2024-07-03T15:44:02.933 INFO:tasks.workunit.client.0.smithi028.stdout:2/231: creat da/d1e/f58 x:0 0 0 2024-07-03T15:44:02.934 INFO:tasks.workunit.client.0.smithi028.stdout:2/232: symlink da/l59 0 2024-07-03T15:44:02.936 INFO:tasks.workunit.client.0.smithi028.stdout:2/233: symlink da/d24/l5a 0 2024-07-03T15:44:02.936 INFO:tasks.workunit.client.0.smithi028.stdout:2/234: fsync da/d24/f39 0 2024-07-03T15:44:02.936 INFO:tasks.workunit.client.0.smithi028.stdout:2/235: write da/d24/f39 [908352,98803] 0 2024-07-03T15:44:02.953 INFO:tasks.workunit.client.0.smithi028.stdout:6/161: sync 2024-07-03T15:44:02.953 INFO:tasks.workunit.client.0.smithi028.stdout:2/236: dwrite da/d24/d2c/f4f [0,4194304] 0 2024-07-03T15:44:02.954 INFO:tasks.workunit.client.0.smithi028.stdout:6/162: rename d4/da/l26 to d4/d12/d2c/l35 0 2024-07-03T15:44:02.956 INFO:tasks.workunit.client.0.smithi028.stdout:2/237: mknod da/d42/d47/c5b 0 2024-07-03T15:44:02.957 INFO:tasks.workunit.client.0.smithi028.stdout:6/163: fdatasync d4/da/d1c/f1e 0 2024-07-03T15:44:02.961 INFO:tasks.workunit.client.0.smithi028.stdout:2/238: dread da/d23/f3b [0,4194304] 0 2024-07-03T15:44:02.962 INFO:tasks.workunit.client.0.smithi028.stdout:2/239: chown f9 36346 1 2024-07-03T15:44:02.962 INFO:tasks.workunit.client.0.smithi028.stdout:6/164: creat d4/d12/f36 x:0 0 0 2024-07-03T15:44:02.963 INFO:tasks.workunit.client.0.smithi028.stdout:2/240: dread da/f12 [0,4194304] 0 2024-07-03T15:44:02.964 INFO:tasks.workunit.client.0.smithi028.stdout:6/165: rename d4/d12/d2c/d31/c32 to d4/da/c37 0 2024-07-03T15:44:02.964 INFO:tasks.workunit.client.0.smithi028.stdout:2/241: truncate da/f10 5154828 0 2024-07-03T15:44:02.966 INFO:tasks.workunit.client.0.smithi028.stdout:6/166: mkdir d4/da/d38 0 2024-07-03T15:44:02.967 INFO:tasks.workunit.client.0.smithi028.stdout:6/167: symlink d4/d12/d2c/d31/l39 0 2024-07-03T15:44:02.970 INFO:tasks.workunit.client.0.smithi028.stdout:2/242: dread f4 [4194304,4194304] 0 2024-07-03T15:44:02.970 INFO:tasks.workunit.client.0.smithi028.stdout:2/243: creat da/d23/d30/f5c x:0 0 0 2024-07-03T15:44:02.974 INFO:tasks.workunit.client.0.smithi028.stdout:6/168: link d4/f18 d4/f3a 0 2024-07-03T15:44:02.986 INFO:tasks.workunit.client.0.smithi028.stdout:2/244: dwrite da/d42/f4d [0,4194304] 0 2024-07-03T15:44:02.986 INFO:tasks.workunit.client.0.smithi028.stdout:2/245: fsync da/d23/d30/f3e 0 2024-07-03T15:44:03.045 INFO:tasks.workunit.client.1.smithi148.stdout:0/891: sync 2024-07-03T15:44:03.046 INFO:tasks.workunit.client.1.smithi148.stdout:0/892: rename da/d17/d2a/d2f/d93/da3/da7 to da/d58/dbd/d13/d14/d50/d95/d127 0 2024-07-03T15:44:03.046 INFO:tasks.workunit.client.1.smithi148.stdout:0/893: chown da/d58/dbd/d13/d14/d50/d95/d127 2166 1 2024-07-03T15:44:03.047 INFO:tasks.workunit.client.1.smithi148.stdout:0/894: dread da/d58/dbd/d94/dac/ddd/f6b [0,4194304] 0 2024-07-03T15:44:03.048 INFO:tasks.workunit.client.1.smithi148.stdout:0/895: write da/d17/d2a/d2f/f37 [194759,93837] 0 2024-07-03T15:44:03.048 INFO:tasks.workunit.client.1.smithi148.stdout:0/896: mkdir da/d58/dbd/d13/d1d/d3e/d85/d8e/db2/d128 0 2024-07-03T15:44:03.049 INFO:tasks.workunit.client.1.smithi148.stdout:0/897: getdents da/d58/dbd/d13/d14/d50/d95/d8c 0 2024-07-03T15:44:03.050 INFO:tasks.workunit.client.1.smithi148.stdout:0/898: write da/d58/dbd/d13/d14/d50/d95/d8c/f119 [75302,4195] 0 2024-07-03T15:44:03.050 INFO:tasks.workunit.client.1.smithi148.stdout:0/899: write da/d17/fe8 [789836,86586] 0 2024-07-03T15:44:03.072 INFO:tasks.workunit.client.0.smithi028.stdout:4/174: sync 2024-07-03T15:44:03.073 INFO:tasks.workunit.client.0.smithi028.stdout:4/175: stat d2/d12/d19/f1f 0 2024-07-03T15:44:03.080 INFO:tasks.workunit.client.0.smithi028.stdout:4/176: dread d2/f22 [0,4194304] 0 2024-07-03T15:44:03.095 INFO:tasks.workunit.client.0.smithi028.stdout:4/177: dwrite d2/f22 [0,4194304] 0 2024-07-03T15:44:03.097 INFO:tasks.workunit.client.0.smithi028.stdout:4/178: mknod d2/d12/d2d/d2f/c37 0 2024-07-03T15:44:03.098 INFO:tasks.workunit.client.1.smithi148.stdout:1/999: sync 2024-07-03T15:44:03.112 INFO:tasks.workunit.client.0.smithi028.stdout:4/179: dwrite d2/f14 [0,4194304] 0 2024-07-03T15:44:03.112 INFO:tasks.workunit.client.0.smithi028.stdout:4/180: dread - d2/f13 zero size 2024-07-03T15:44:03.130 INFO:tasks.workunit.client.0.smithi028.stdout:4/181: dwrite d2/d12/f26 [0,4194304] 0 2024-07-03T15:44:03.130 INFO:tasks.workunit.client.0.smithi028.stdout:4/182: read - d2/d12/f28 zero size 2024-07-03T15:44:03.131 INFO:tasks.workunit.client.0.smithi028.stdout:4/183: unlink d2/f14 0 2024-07-03T15:44:03.135 INFO:tasks.workunit.client.0.smithi028.stdout:4/184: dread d2/f7 [0,4194304] 0 2024-07-03T15:44:03.135 INFO:tasks.workunit.client.0.smithi028.stdout:4/185: stat d2/d12/d2d/d2f/c37 0 2024-07-03T15:44:03.135 INFO:tasks.workunit.client.0.smithi028.stdout:4/186: creat d2/d12/d2e/f38 x:0 0 0 2024-07-03T15:44:03.136 INFO:tasks.workunit.client.0.smithi028.stdout:4/187: mknod d2/d12/d2e/c39 0 2024-07-03T15:44:03.137 INFO:tasks.workunit.client.1.smithi148.stdout:0/900: sync 2024-07-03T15:44:03.138 INFO:tasks.workunit.client.1.smithi148.stdout:0/901: creat da/f129 x:0 0 0 2024-07-03T15:44:03.138 INFO:tasks.workunit.client.0.smithi028.stdout:4/188: mkdir d2/d12/d2d/d2f/d3a 0 2024-07-03T15:44:03.138 INFO:tasks.workunit.client.0.smithi028.stdout:4/189: chown d2/d12/d19 1 1 2024-07-03T15:44:03.139 INFO:tasks.workunit.client.1.smithi148.stdout:0/902: rename da/d58/dbd/d13/d1d/d3e/d85/d8e/dd4 to da/d58/dbd/d13/d14/d50/d95/d8c/d12a 0 2024-07-03T15:44:03.140 INFO:tasks.workunit.client.1.smithi148.stdout:0/903: readlink da/d58/dbd/d94/dac/ddd/l3d 0 2024-07-03T15:44:03.157 INFO:tasks.workunit.client.1.smithi148.stdout:0/904: dwrite da/d58/dbd/d24/dc8/d8a/fa8 [0,4194304] 0 2024-07-03T15:44:03.158 INFO:tasks.workunit.client.0.smithi028.stdout:4/190: dwrite d2/d12/d19/f24 [0,4194304] 0 2024-07-03T15:44:03.158 INFO:tasks.workunit.client.0.smithi028.stdout:4/191: read - d2/f2b zero size 2024-07-03T15:44:03.162 INFO:tasks.workunit.client.1.smithi148.stdout:0/905: link da/d58/dbd/d13/d1d/da1/cc5 da/d58/dbd/d13/d14/d50/d95/c12b 0 2024-07-03T15:44:03.162 INFO:tasks.workunit.client.1.smithi148.stdout:0/906: stat da/d58/dbd/d13/d14/d50/d95/de2/c6f 0 2024-07-03T15:44:03.162 INFO:tasks.workunit.client.1.smithi148.stdout:0/907: write da/d58/dbd/d13/fb8 [955959,80212] 0 2024-07-03T15:44:03.163 INFO:tasks.workunit.client.1.smithi148.stdout:0/908: getdents da/d17/d2a/d2f/d93/da3 0 2024-07-03T15:44:03.166 INFO:tasks.workunit.client.1.smithi148.stdout:0/909: mkdir da/d58/d90/d12c 0 2024-07-03T15:44:03.166 INFO:tasks.workunit.client.1.smithi148.stdout:0/910: truncate da/d58/dbd/f96 229665 0 2024-07-03T15:44:03.166 INFO:tasks.workunit.client.1.smithi148.stdout:0/911: truncate da/d58/dbd/f75 5105521 0 2024-07-03T15:44:03.166 INFO:tasks.workunit.client.1.smithi148.stdout:0/912: chown da/d58/dbd/d13/d14/d50/l102 126503921 1 2024-07-03T15:44:03.167 INFO:tasks.workunit.client.1.smithi148.stdout:0/913: mknod da/d58/dbd/d13/d14/dbe/c12d 0 2024-07-03T15:44:03.167 INFO:tasks.workunit.client.1.smithi148.stdout:0/914: read da/f73 [1089371,101687] 0 2024-07-03T15:44:03.168 INFO:tasks.workunit.client.1.smithi148.stdout:0/915: creat da/d58/dbd/d13/d14/d50/d95/d8c/d12a/f12e x:0 0 0 2024-07-03T15:44:03.169 INFO:tasks.workunit.client.1.smithi148.stdout:0/916: chown da/d58/dbd/d13/d14/d50/d95/de2/f63 234768 1 2024-07-03T15:44:03.169 INFO:tasks.workunit.client.1.smithi148.stdout:0/917: creat da/d58/d90/db4/f12f x:0 0 0 2024-07-03T15:44:03.170 INFO:tasks.workunit.client.0.smithi028.stdout:4/192: dwrite d2/f13 [0,4194304] 0 2024-07-03T15:44:03.172 INFO:tasks.workunit.client.0.smithi028.stdout:4/193: unlink c0 0 2024-07-03T15:44:03.174 INFO:tasks.workunit.client.0.smithi028.stdout:4/194: mkdir d2/d12/d3b 0 2024-07-03T15:44:03.177 INFO:tasks.workunit.client.0.smithi028.stdout:0/181: sync 2024-07-03T15:44:03.177 INFO:tasks.workunit.client.0.smithi028.stdout:7/94: sync 2024-07-03T15:44:03.177 INFO:tasks.workunit.client.0.smithi028.stdout:3/253: sync 2024-07-03T15:44:03.177 INFO:tasks.workunit.client.0.smithi028.stdout:7/95: rename d9 to d9/d1c/d1d 22 2024-07-03T15:44:03.177 INFO:tasks.workunit.client.0.smithi028.stdout:3/254: creat d18/d1a/d1e/f57 x:0 0 0 2024-07-03T15:44:03.177 INFO:tasks.workunit.client.0.smithi028.stdout:3/255: truncate d18/d1a/d1e/f57 515700 0 2024-07-03T15:44:03.178 INFO:tasks.workunit.client.0.smithi028.stdout:4/195: dread d2/d12/f17 [0,4194304] 0 2024-07-03T15:44:03.178 INFO:tasks.workunit.client.0.smithi028.stdout:0/182: link d2/d1c/f1e d2/f3b 0 2024-07-03T15:44:03.178 INFO:tasks.workunit.client.0.smithi028.stdout:0/183: read d2/d3/f7 [1980583,45231] 0 2024-07-03T15:44:03.179 INFO:tasks.workunit.client.0.smithi028.stdout:0/184: truncate d2/d1c/f20 423238 0 2024-07-03T15:44:03.179 INFO:tasks.workunit.client.0.smithi028.stdout:7/96: write d9/fe [1406587,5159] 0 2024-07-03T15:44:03.183 INFO:tasks.workunit.client.0.smithi028.stdout:3/256: mkdir d18/d1a/d1e/d58 0 2024-07-03T15:44:03.187 INFO:tasks.workunit.client.0.smithi028.stdout:7/97: write f7 [2148657,64706] 0 2024-07-03T15:44:03.187 INFO:tasks.workunit.client.0.smithi028.stdout:0/185: dread d2/d1c/d32/f38 [0,4194304] 0 2024-07-03T15:44:03.187 INFO:tasks.workunit.client.0.smithi028.stdout:0/186: chown d2/d1c/f2b 4570034 1 2024-07-03T15:44:03.187 INFO:tasks.workunit.client.0.smithi028.stdout:0/187: chown d2/d3/l22 0 1 2024-07-03T15:44:03.188 INFO:tasks.workunit.client.0.smithi028.stdout:0/188: truncate d2/d1c/d32/f35 833542 0 2024-07-03T15:44:03.189 INFO:tasks.workunit.client.0.smithi028.stdout:7/98: link d9/f1a d9/d1c/f1e 0 2024-07-03T15:44:03.191 INFO:tasks.workunit.client.0.smithi028.stdout:0/189: link d2/d3/c26 d2/d3/c3c 0 2024-07-03T15:44:03.192 INFO:tasks.workunit.client.0.smithi028.stdout:4/196: dwrite d2/f2b [0,4194304] 0 2024-07-03T15:44:03.193 INFO:tasks.workunit.client.0.smithi028.stdout:7/99: link f6 d9/f1f 0 2024-07-03T15:44:03.194 INFO:tasks.workunit.client.0.smithi028.stdout:7/100: stat d9/l14 0 2024-07-03T15:44:03.199 INFO:tasks.workunit.client.0.smithi028.stdout:4/197: dread d2/d12/d19/f1f [0,4194304] 0 2024-07-03T15:44:03.201 INFO:tasks.workunit.client.0.smithi028.stdout:4/198: symlink d2/d12/d3b/l3c 0 2024-07-03T15:44:03.202 INFO:tasks.workunit.client.0.smithi028.stdout:4/199: chown d2/l1d 1 1 2024-07-03T15:44:03.202 INFO:tasks.workunit.client.0.smithi028.stdout:4/200: stat d2/f7 0 2024-07-03T15:44:03.202 INFO:tasks.workunit.client.0.smithi028.stdout:4/201: write d2/d12/f17 [1256061,111488] 0 2024-07-03T15:44:03.202 INFO:tasks.workunit.client.0.smithi028.stdout:4/202: chown d2/l2a 6 1 2024-07-03T15:44:03.202 INFO:tasks.workunit.client.0.smithi028.stdout:4/203: read d2/d12/f26 [5185787,83671] 0 2024-07-03T15:44:03.203 INFO:tasks.workunit.client.0.smithi028.stdout:4/204: rmdir d2/d12/d2e/d35 0 2024-07-03T15:44:03.204 INFO:tasks.workunit.client.0.smithi028.stdout:4/205: write d2/d12/d2e/f38 [141140,115122] 0 2024-07-03T15:44:03.204 INFO:tasks.workunit.client.0.smithi028.stdout:3/257: dwrite d18/d1a/d1e/f41 [0,4194304] 0 2024-07-03T15:44:03.205 INFO:tasks.workunit.client.0.smithi028.stdout:4/206: getdents d2/d12 0 2024-07-03T15:44:03.207 INFO:tasks.workunit.client.0.smithi028.stdout:3/258: unlink d18/l25 0 2024-07-03T15:44:03.207 INFO:tasks.workunit.client.0.smithi028.stdout:3/259: truncate d18/d43/f4d 509284 0 2024-07-03T15:44:03.208 INFO:tasks.workunit.client.0.smithi028.stdout:0/190: dread d2/d3/f7 [0,4194304] 0 2024-07-03T15:44:03.210 INFO:tasks.workunit.client.0.smithi028.stdout:7/101: fsync d9/f1f 0 2024-07-03T15:44:03.211 INFO:tasks.workunit.client.0.smithi028.stdout:7/102: truncate d9/d1c/f1e 981649 0 2024-07-03T15:44:03.211 INFO:tasks.workunit.client.0.smithi028.stdout:7/103: read f4 [819523,47578] 0 2024-07-03T15:44:03.211 INFO:tasks.workunit.client.0.smithi028.stdout:7/104: truncate d9/f1f 1637126 0 2024-07-03T15:44:03.214 INFO:tasks.workunit.client.0.smithi028.stdout:4/207: unlink d2/f4 0 2024-07-03T15:44:03.214 INFO:tasks.workunit.client.0.smithi028.stdout:4/208: write d2/d12/d19/f9 [4526662,43460] 0 2024-07-03T15:44:03.214 INFO:tasks.workunit.client.0.smithi028.stdout:4/209: rename d2 to d2/d3d 22 2024-07-03T15:44:03.215 INFO:tasks.workunit.client.0.smithi028.stdout:3/260: link d18/d1a/l49 d18/d1a/d1e/d58/l59 0 2024-07-03T15:44:03.216 INFO:tasks.workunit.client.0.smithi028.stdout:3/261: truncate ff 857780 0 2024-07-03T15:44:03.216 INFO:tasks.workunit.client.0.smithi028.stdout:3/262: creat d18/d3a/f5a x:0 0 0 2024-07-03T15:44:03.216 INFO:tasks.workunit.client.0.smithi028.stdout:3/263: chown d18/d43/f4d 38552533 1 2024-07-03T15:44:03.217 INFO:tasks.workunit.client.0.smithi028.stdout:0/191: creat d2/d1c/d32/d36/f3d x:0 0 0 2024-07-03T15:44:03.217 INFO:tasks.workunit.client.0.smithi028.stdout:0/192: dread - d2/d1c/d32/d36/f3d zero size 2024-07-03T15:44:03.217 INFO:tasks.workunit.client.0.smithi028.stdout:0/193: write d2/d3/fe [587685,63895] 0 2024-07-03T15:44:03.218 INFO:tasks.workunit.client.0.smithi028.stdout:7/105: creat d9/d1c/f20 x:0 0 0 2024-07-03T15:44:03.220 INFO:tasks.workunit.client.1.smithi148.stdout:0/918: sync 2024-07-03T15:44:03.222 INFO:tasks.workunit.client.0.smithi028.stdout:4/210: dread d2/d12/f17 [0,4194304] 0 2024-07-03T15:44:03.223 INFO:tasks.workunit.client.1.smithi148.stdout:0/919: creat da/d58/d90/dd1/f130 x:0 0 0 2024-07-03T15:44:03.224 INFO:tasks.workunit.client.0.smithi028.stdout:3/264: dread d18/d1a/f2a [0,4194304] 0 2024-07-03T15:44:03.225 INFO:tasks.workunit.client.0.smithi028.stdout:8/235: sync 2024-07-03T15:44:03.227 INFO:tasks.workunit.client.1.smithi148.stdout:0/920: rename da/d58/dbd/d4a/la6 to da/d58/dbd/d94/dac/l131 0 2024-07-03T15:44:03.227 INFO:tasks.workunit.client.1.smithi148.stdout:0/921: chown da/d17/d2a/d2f/d93/f104 652785 1 2024-07-03T15:44:03.230 INFO:tasks.workunit.client.1.smithi148.stdout:0/922: creat da/d58/dbd/d24/dc8/d10f/f132 x:0 0 0 2024-07-03T15:44:03.232 INFO:tasks.workunit.client.1.smithi148.stdout:0/923: creat da/d58/dbd/d13/d1d/f133 x:0 0 0 2024-07-03T15:44:03.233 INFO:tasks.workunit.client.1.smithi148.stdout:0/924: dread - da/d58/dbd/d94/f11e zero size 2024-07-03T15:44:03.233 INFO:tasks.workunit.client.0.smithi028.stdout:3/265: dread d18/d1a/d1e/f41 [0,4194304] 0 2024-07-03T15:44:03.233 INFO:tasks.workunit.client.0.smithi028.stdout:7/106: dwrite f2 [4194304,4194304] 0 2024-07-03T15:44:03.233 INFO:tasks.workunit.client.0.smithi028.stdout:7/107: creat d9/d1c/f21 x:0 0 0 2024-07-03T15:44:03.234 INFO:tasks.workunit.client.0.smithi028.stdout:4/211: mkdir d2/d3e 0 2024-07-03T15:44:03.237 INFO:tasks.workunit.client.1.smithi148.stdout:0/925: creat da/d58/dbd/d13/d1d/d3e/d85/d8e/db2/f134 x:0 0 0 2024-07-03T15:44:03.237 INFO:tasks.workunit.client.1.smithi148.stdout:0/926: stat da/d58/dbd/d94/dac/ddd/c40 0 2024-07-03T15:44:03.239 INFO:tasks.workunit.client.0.smithi028.stdout:3/266: symlink d18/l5b 0 2024-07-03T15:44:03.239 INFO:tasks.workunit.client.0.smithi028.stdout:3/267: creat d18/d1a/f5c x:0 0 0 2024-07-03T15:44:03.239 INFO:tasks.workunit.client.1.smithi148.stdout:0/927: dread da/d17/d2a/d2f/d93/f104 [0,4194304] 0 2024-07-03T15:44:03.239 INFO:tasks.workunit.client.0.smithi028.stdout:3/268: read d18/d3a/f39 [338825,118814] 0 2024-07-03T15:44:03.243 INFO:tasks.workunit.client.1.smithi148.stdout:0/928: link da/d58/dbd/d13/fce da/d58/dbd/d24/f135 0 2024-07-03T15:44:03.245 INFO:tasks.workunit.client.0.smithi028.stdout:7/108: creat d9/f22 x:0 0 0 2024-07-03T15:44:03.248 INFO:tasks.workunit.client.0.smithi028.stdout:8/236: rename d0/d2 to d0/d41/d50 0 2024-07-03T15:44:03.248 INFO:tasks.workunit.client.0.smithi028.stdout:8/237: chown d0/d41/d50/d3/dc/f21 14 1 2024-07-03T15:44:03.253 INFO:tasks.workunit.client.0.smithi028.stdout:6/169: sync 2024-07-03T15:44:03.253 INFO:tasks.workunit.client.0.smithi028.stdout:5/249: sync 2024-07-03T15:44:03.253 INFO:tasks.workunit.client.0.smithi028.stdout:5/250: readlink d0/dd/d13/d19/l3c 0 2024-07-03T15:44:03.253 INFO:tasks.workunit.client.0.smithi028.stdout:6/170: fdatasync d4/d15/f2a 0 2024-07-03T15:44:03.253 INFO:tasks.workunit.client.0.smithi028.stdout:6/171: fdatasync d4/d12/f36 0 2024-07-03T15:44:03.253 INFO:tasks.workunit.client.0.smithi028.stdout:5/251: dread - d0/dd/d1e/d3b/f4f zero size 2024-07-03T15:44:03.253 INFO:tasks.workunit.client.0.smithi028.stdout:5/252: readlink d0/lb 0 2024-07-03T15:44:03.253 INFO:tasks.workunit.client.0.smithi028.stdout:1/331: sync 2024-07-03T15:44:03.255 INFO:tasks.workunit.client.0.smithi028.stdout:1/332: write d4/de/d1c/f58 [972678,82191] 0 2024-07-03T15:44:03.255 INFO:tasks.workunit.client.0.smithi028.stdout:9/233: sync 2024-07-03T15:44:03.255 INFO:tasks.workunit.client.0.smithi028.stdout:2/246: sync 2024-07-03T15:44:03.256 INFO:tasks.workunit.client.0.smithi028.stdout:8/238: rename d0/d41/d50/d3/dc/d33/l43 to d0/d41/d50/l51 0 2024-07-03T15:44:03.260 INFO:tasks.workunit.client.0.smithi028.stdout:1/333: dread d4/d3c/d54/d79/f12 [0,4194304] 0 2024-07-03T15:44:03.260 INFO:tasks.workunit.client.0.smithi028.stdout:1/334: readlink d4/de/d1c/l36 0 2024-07-03T15:44:03.261 INFO:tasks.workunit.client.0.smithi028.stdout:1/335: fdatasync d4/f19 0 2024-07-03T15:44:03.261 INFO:tasks.workunit.client.0.smithi028.stdout:5/253: stat d0/dd/d1e/c2a 0 2024-07-03T15:44:03.262 INFO:tasks.workunit.client.0.smithi028.stdout:9/234: dread d1/f9 [0,4194304] 0 2024-07-03T15:44:03.263 INFO:tasks.workunit.client.0.smithi028.stdout:3/269: dwrite d18/d1a/f24 [0,4194304] 0 2024-07-03T15:44:03.263 INFO:tasks.workunit.client.0.smithi028.stdout:3/270: stat d18/d1a/l21 0 2024-07-03T15:44:03.264 INFO:tasks.workunit.client.0.smithi028.stdout:2/247: mknod da/d42/c5d 0 2024-07-03T15:44:03.264 INFO:tasks.workunit.client.0.smithi028.stdout:8/239: mknod d0/d41/c52 0 2024-07-03T15:44:03.264 INFO:tasks.workunit.client.0.smithi028.stdout:8/240: getdents d0/d41/d50/d3/d35/d46 0 2024-07-03T15:44:03.265 INFO:tasks.workunit.client.0.smithi028.stdout:6/172: mknod d4/c3b 0 2024-07-03T15:44:03.266 INFO:tasks.workunit.client.0.smithi028.stdout:9/235: dread d1/d16/f1c [0,4194304] 0 2024-07-03T15:44:03.266 INFO:tasks.workunit.client.0.smithi028.stdout:9/236: truncate d1/d10/f3e 745632 0 2024-07-03T15:44:03.268 INFO:tasks.workunit.client.0.smithi028.stdout:1/336: rename d4/de/d1c/f58 to d4/d2c/d3a/d5c/d6c/d7d/d59/d7f/f87 0 2024-07-03T15:44:03.272 INFO:tasks.workunit.client.0.smithi028.stdout:2/248: mknod da/d42/c5e 0 2024-07-03T15:44:03.272 INFO:tasks.workunit.client.1.smithi148.stdout:0/929: dwrite da/d58/dbd/d24/d84/d114/f122 [0,4194304] 0 2024-07-03T15:44:03.272 INFO:tasks.workunit.client.1.smithi148.stdout:0/930: read - da/d58/dbd/f106 zero size 2024-07-03T15:44:03.275 INFO:tasks.workunit.client.0.smithi028.stdout:8/241: symlink d0/d41/d50/d3/dc/d4d/l53 0 2024-07-03T15:44:03.275 INFO:tasks.workunit.client.0.smithi028.stdout:8/242: chown d0/d41/d50/d3/dc/d11 188 1 2024-07-03T15:44:03.275 INFO:tasks.workunit.client.0.smithi028.stdout:6/173: mknod d4/d15/c3c 0 2024-07-03T15:44:03.277 INFO:tasks.workunit.client.0.smithi028.stdout:9/237: rmdir d1/d3d 0 2024-07-03T15:44:03.279 INFO:tasks.workunit.client.1.smithi148.stdout:0/931: creat da/d58/dbd/d13/f136 x:0 0 0 2024-07-03T15:44:03.280 INFO:tasks.workunit.client.0.smithi028.stdout:2/249: mknod da/d42/c5f 0 2024-07-03T15:44:03.281 INFO:tasks.workunit.client.0.smithi028.stdout:3/271: dwrite d18/f2d [4194304,4194304] 0 2024-07-03T15:44:03.281 INFO:tasks.workunit.client.0.smithi028.stdout:3/272: creat d18/d1a/d1e/f5d x:0 0 0 2024-07-03T15:44:03.281 INFO:tasks.workunit.client.0.smithi028.stdout:4/212: truncate d2/d12/f26 5508723 0 2024-07-03T15:44:03.281 INFO:tasks.workunit.client.0.smithi028.stdout:3/273: truncate d18/d3a/f39 1160041 0 2024-07-03T15:44:03.281 INFO:tasks.workunit.client.0.smithi028.stdout:3/274: dread - d18/f52 zero size 2024-07-03T15:44:03.281 INFO:tasks.workunit.client.0.smithi028.stdout:3/275: write d18/d3a/f47 [361791,86610] 0 2024-07-03T15:44:03.284 INFO:tasks.workunit.client.0.smithi028.stdout:8/243: mknod d0/dd/d22/d40/d45/c54 0 2024-07-03T15:44:03.286 INFO:tasks.workunit.client.0.smithi028.stdout:7/109: dwrite d9/d1c/f21 [0,4194304] 0 2024-07-03T15:44:03.286 INFO:tasks.workunit.client.0.smithi028.stdout:7/110: fdatasync d9/f1f 0 2024-07-03T15:44:03.287 INFO:tasks.workunit.client.0.smithi028.stdout:6/174: mkdir d4/da/d38/d3d 0 2024-07-03T15:44:03.288 INFO:tasks.workunit.client.0.smithi028.stdout:5/254: rename d0/dd/d13/d19 to d0/dd/d13/d57 0 2024-07-03T15:44:03.290 INFO:tasks.workunit.client.1.smithi148.stdout:0/932: dread da/d17/d2a/f59 [0,4194304] 0 2024-07-03T15:44:03.291 INFO:tasks.workunit.client.0.smithi028.stdout:2/250: unlink da/d24/f39 0 2024-07-03T15:44:03.295 INFO:tasks.workunit.client.0.smithi028.stdout:3/276: dwrite fe [4194304,4194304] 0 2024-07-03T15:44:03.295 INFO:tasks.workunit.client.1.smithi148.stdout:0/933: creat da/d58/dbd/d13/d1d/da1/f137 x:0 0 0 2024-07-03T15:44:03.297 INFO:tasks.workunit.client.0.smithi028.stdout:4/213: creat d2/d3e/f3f x:0 0 0 2024-07-03T15:44:03.298 INFO:tasks.workunit.client.0.smithi028.stdout:9/238: dwrite d1/d16/f3a [0,4194304] 0 2024-07-03T15:44:03.298 INFO:tasks.workunit.client.0.smithi028.stdout:9/239: fdatasync d1/d17/f25 0 2024-07-03T15:44:03.299 INFO:tasks.workunit.client.1.smithi148.stdout:0/934: link da/d17/d125/lef da/d58/dbd/d24/dc8/d10f/l138 0 2024-07-03T15:44:03.299 INFO:tasks.workunit.client.1.smithi148.stdout:0/935: readlink da/d58/dbd/d13/d1d/d3e/lb5 0 2024-07-03T15:44:03.300 INFO:tasks.workunit.client.0.smithi028.stdout:8/244: mknod d0/c55 0 2024-07-03T15:44:03.302 INFO:tasks.workunit.client.0.smithi028.stdout:3/277: write d18/d1a/d1e/f41 [765193,2478] 0 2024-07-03T15:44:03.302 INFO:tasks.workunit.client.0.smithi028.stdout:3/278: truncate d18/f1d 2000463 0 2024-07-03T15:44:03.303 INFO:tasks.workunit.client.0.smithi028.stdout:3/279: write d18/f55 [328748,117541] 0 2024-07-03T15:44:03.303 INFO:tasks.workunit.client.0.smithi028.stdout:6/175: rmdir d4/da/d38 39 2024-07-03T15:44:03.305 INFO:tasks.workunit.client.0.smithi028.stdout:8/245: dread d0/d41/d50/d3/f3a [0,4194304] 0 2024-07-03T15:44:03.309 INFO:tasks.workunit.client.1.smithi148.stdout:0/936: dread da/f11 [4194304,4194304] 0 2024-07-03T15:44:03.309 INFO:tasks.workunit.client.0.smithi028.stdout:5/255: dwrite d0/dd/d13/d29/f4e [0,4194304] 0 2024-07-03T15:44:03.309 INFO:tasks.workunit.client.0.smithi028.stdout:5/256: stat d0/c8 0 2024-07-03T15:44:03.311 INFO:tasks.workunit.client.1.smithi148.stdout:0/937: mkdir da/d139 0 2024-07-03T15:44:03.313 INFO:tasks.workunit.client.0.smithi028.stdout:5/257: dread d0/dd/df/f16 [0,4194304] 0 2024-07-03T15:44:03.314 INFO:tasks.workunit.client.0.smithi028.stdout:6/176: write f2 [3057853,18849] 0 2024-07-03T15:44:03.314 INFO:tasks.workunit.client.0.smithi028.stdout:9/240: creat d1/d17/f48 x:0 0 0 2024-07-03T15:44:03.314 INFO:tasks.workunit.client.0.smithi028.stdout:9/241: chown d1/d10/d28/f40 29938 1 2024-07-03T15:44:03.316 INFO:tasks.workunit.client.0.smithi028.stdout:6/177: dread d4/d15/f2a [0,4194304] 0 2024-07-03T15:44:03.317 INFO:tasks.workunit.client.0.smithi028.stdout:3/280: rename d18/d40 to d18/d1a/d5e 0 2024-07-03T15:44:03.319 INFO:tasks.workunit.client.0.smithi028.stdout:3/281: write d18/d1a/f2a [1506188,94971] 0 2024-07-03T15:44:03.320 INFO:tasks.workunit.client.1.smithi148.stdout:0/938: write da/d58/dbd/d24/f135 [1877270,30648] 0 2024-07-03T15:44:03.320 INFO:tasks.workunit.client.1.smithi148.stdout:0/939: fdatasync da/d58/dbd/d94/dba/ffd 0 2024-07-03T15:44:03.321 INFO:tasks.workunit.client.0.smithi028.stdout:8/246: unlink d0/d41/d50/d3/dc/f21 0 2024-07-03T15:44:03.321 INFO:tasks.workunit.client.0.smithi028.stdout:2/251: symlink da/d1e/l60 0 2024-07-03T15:44:03.323 INFO:tasks.workunit.client.0.smithi028.stdout:8/247: dread d0/f2a [0,4194304] 0 2024-07-03T15:44:03.323 INFO:tasks.workunit.client.0.smithi028.stdout:8/248: chown d0/d41/d50/d3/dc/d11/d12/l1b 2 1 2024-07-03T15:44:03.323 INFO:tasks.workunit.client.0.smithi028.stdout:8/249: write d0/d41/d50/d3/dc/d11/f42 [1929957,93014] 0 2024-07-03T15:44:03.324 INFO:tasks.workunit.client.0.smithi028.stdout:8/250: readlink d0/d41/d50/l51 0 2024-07-03T15:44:03.326 INFO:tasks.workunit.client.0.smithi028.stdout:4/214: dread d2/d12/d19/f24 [0,4194304] 0 2024-07-03T15:44:03.327 INFO:tasks.workunit.client.1.smithi148.stdout:0/940: rename da/d58/dbd/d24/f135 to da/d58/dbd/d13/f13a 0 2024-07-03T15:44:03.330 INFO:tasks.workunit.client.0.smithi028.stdout:5/258: unlink d0/dd/df/c2f 0 2024-07-03T15:44:03.332 INFO:tasks.workunit.client.0.smithi028.stdout:9/242: link d1/d10/l32 d1/dd/l49 0 2024-07-03T15:44:03.333 INFO:tasks.workunit.client.1.smithi148.stdout:0/941: symlink da/d58/dbd/d13/d1d/d3e/d85/d8e/db2/d128/l13b 0 2024-07-03T15:44:03.333 INFO:tasks.workunit.client.1.smithi148.stdout:0/942: creat da/d58/dbd/d13/d14/f13c x:0 0 0 2024-07-03T15:44:03.333 INFO:tasks.workunit.client.0.smithi028.stdout:9/243: dread d1/d10/f3e [0,4194304] 0 2024-07-03T15:44:03.333 INFO:tasks.workunit.client.0.smithi028.stdout:5/259: dread d0/dd/df/f16 [0,4194304] 0 2024-07-03T15:44:03.333 INFO:tasks.workunit.client.0.smithi028.stdout:5/260: rename d0 to d0/d58 22 2024-07-03T15:44:03.334 INFO:tasks.workunit.client.0.smithi028.stdout:8/251: write d0/f44 [2955689,8413] 0 2024-07-03T15:44:03.334 INFO:tasks.workunit.client.0.smithi028.stdout:7/111: rename d9/l15 to d9/l23 0 2024-07-03T15:44:03.336 INFO:tasks.workunit.client.0.smithi028.stdout:2/252: dwrite da/d24/d2c/f54 [0,4194304] 0 2024-07-03T15:44:03.336 INFO:tasks.workunit.client.0.smithi028.stdout:2/253: creat da/d24/d2c/f61 x:0 0 0 2024-07-03T15:44:03.336 INFO:tasks.workunit.client.0.smithi028.stdout:2/254: creat da/d23/d30/f62 x:0 0 0 2024-07-03T15:44:03.337 INFO:tasks.workunit.client.1.smithi148.stdout:0/943: creat da/d17/d2a/d2f/f13d x:0 0 0 2024-07-03T15:44:03.338 INFO:tasks.workunit.client.0.smithi028.stdout:3/282: getdents d18/d1a/d5e 0 2024-07-03T15:44:03.339 INFO:tasks.workunit.client.1.smithi148.stdout:0/944: mknod da/d58/dbd/c13e 0 2024-07-03T15:44:03.340 INFO:tasks.workunit.client.1.smithi148.stdout:0/945: creat da/d58/dbd/d13/d1d/da1/f13f x:0 0 0 2024-07-03T15:44:03.340 INFO:tasks.workunit.client.0.smithi028.stdout:9/244: mkdir d1/d4a 0 2024-07-03T15:44:03.341 INFO:tasks.workunit.client.0.smithi028.stdout:9/245: dread d1/d17/f25 [0,4194304] 0 2024-07-03T15:44:03.342 INFO:tasks.workunit.client.0.smithi028.stdout:9/246: creat d1/f4b x:0 0 0 2024-07-03T15:44:03.342 INFO:tasks.workunit.client.0.smithi028.stdout:9/247: chown d1/d16 0 1 2024-07-03T15:44:03.342 INFO:tasks.workunit.client.0.smithi028.stdout:5/261: unlink d0/dd/d13/d57/l3c 0 2024-07-03T15:44:03.343 INFO:tasks.workunit.client.0.smithi028.stdout:6/178: symlink d4/da/d1c/l3e 0 2024-07-03T15:44:03.345 INFO:tasks.workunit.client.0.smithi028.stdout:6/179: write d4/d15/f2a [290871,65256] 0 2024-07-03T15:44:03.345 INFO:tasks.workunit.client.0.smithi028.stdout:6/180: fsync d4/f29 0 2024-07-03T15:44:03.351 INFO:tasks.workunit.client.0.smithi028.stdout:2/255: symlink da/d1e/d56/l63 0 2024-07-03T15:44:03.353 INFO:tasks.workunit.client.0.smithi028.stdout:9/248: rename d1/dd/f1f to d1/dd/f4c 0 2024-07-03T15:44:03.354 INFO:tasks.workunit.client.0.smithi028.stdout:5/262: truncate d0/dd/d1e/f56 1274050 0 2024-07-03T15:44:03.354 INFO:tasks.workunit.client.0.smithi028.stdout:8/252: rmdir d0/dd 39 2024-07-03T15:44:03.355 INFO:tasks.workunit.client.0.smithi028.stdout:7/112: mknod d9/c24 0 2024-07-03T15:44:03.355 INFO:tasks.workunit.client.0.smithi028.stdout:7/113: fsync d9/f1a 0 2024-07-03T15:44:03.356 INFO:tasks.workunit.client.0.smithi028.stdout:2/256: mknod da/d1e/d56/c64 0 2024-07-03T15:44:03.357 INFO:tasks.workunit.client.0.smithi028.stdout:4/215: link d2/d12/f26 d2/d12/f40 0 2024-07-03T15:44:03.358 INFO:tasks.workunit.client.0.smithi028.stdout:4/216: chown d2/l16 283 1 2024-07-03T15:44:03.358 INFO:tasks.workunit.client.0.smithi028.stdout:4/217: write d2/d12/d19/f34 [716954,73016] 0 2024-07-03T15:44:03.359 INFO:tasks.workunit.client.0.smithi028.stdout:9/249: symlink d1/d17/d30/l4d 0 2024-07-03T15:44:03.359 INFO:tasks.workunit.client.0.smithi028.stdout:8/253: getdents d0/d41/d50/d3/dc/d4d 0 2024-07-03T15:44:03.359 INFO:tasks.workunit.client.0.smithi028.stdout:7/114: creat d9/f25 x:0 0 0 2024-07-03T15:44:03.361 INFO:tasks.workunit.client.0.smithi028.stdout:2/257: unlink da/d24/d2c/f4f 0 2024-07-03T15:44:03.361 INFO:tasks.workunit.client.0.smithi028.stdout:2/258: fdatasync da/d42/f50 0 2024-07-03T15:44:03.362 INFO:tasks.workunit.client.0.smithi028.stdout:2/259: creat da/d42/f65 x:0 0 0 2024-07-03T15:44:03.362 INFO:tasks.workunit.client.0.smithi028.stdout:2/260: creat da/d24/f66 x:0 0 0 2024-07-03T15:44:03.363 INFO:tasks.workunit.client.0.smithi028.stdout:5/263: read d0/dd/d13/f39 [2900760,76519] 0 2024-07-03T15:44:03.363 INFO:tasks.workunit.client.0.smithi028.stdout:3/283: dwrite d18/f42 [0,4194304] 0 2024-07-03T15:44:03.364 INFO:tasks.workunit.client.1.smithi148.stdout:0/946: dwrite da/d58/dbd/d94/f11e [0,4194304] 0 2024-07-03T15:44:03.364 INFO:tasks.workunit.client.0.smithi028.stdout:9/250: mkdir d1/d10/d28/d4e 0 2024-07-03T15:44:03.365 INFO:tasks.workunit.client.0.smithi028.stdout:7/115: creat d9/f26 x:0 0 0 2024-07-03T15:44:03.365 INFO:tasks.workunit.client.0.smithi028.stdout:7/116: chown d9/c24 1 1 2024-07-03T15:44:03.367 INFO:tasks.workunit.client.0.smithi028.stdout:3/284: dread d18/d1a/d1e/f3e [0,4194304] 0 2024-07-03T15:44:03.374 INFO:tasks.workunit.client.1.smithi148.stdout:0/947: symlink da/d58/dbd/d24/d84/l140 0 2024-07-03T15:44:03.374 INFO:tasks.workunit.client.1.smithi148.stdout:0/948: chown da/d58/dbd/d13/c42 0 1 2024-07-03T15:44:03.374 INFO:tasks.workunit.client.1.smithi148.stdout:0/949: write da/d58/dbd/d4a/f108 [388441,89259] 0 2024-07-03T15:44:03.377 INFO:tasks.workunit.client.0.smithi028.stdout:4/218: creat d2/d12/d2d/d2f/f41 x:0 0 0 2024-07-03T15:44:03.378 INFO:tasks.workunit.client.0.smithi028.stdout:4/219: write d2/d12/d2d/f32 [47903,17498] 0 2024-07-03T15:44:03.379 INFO:tasks.workunit.client.1.smithi148.stdout:0/950: mknod da/d58/dbd/d13/d1d/c141 0 2024-07-03T15:44:03.381 INFO:tasks.workunit.client.0.smithi028.stdout:5/264: link d0/dd/d1e/d3b/c3d d0/dd/d4b/c59 0 2024-07-03T15:44:03.383 INFO:tasks.workunit.client.0.smithi028.stdout:5/265: dread d0/dd/d13/d29/f35 [0,4194304] 0 2024-07-03T15:44:03.383 INFO:tasks.workunit.client.0.smithi028.stdout:5/266: dread - d0/dd/d13/d57/d26/d41/f44 zero size 2024-07-03T15:44:03.383 INFO:tasks.workunit.client.1.smithi148.stdout:0/951: mknod da/d58/dbd/d13/d14/d50/d95/d8c/c142 0 2024-07-03T15:44:03.384 INFO:tasks.workunit.client.0.smithi028.stdout:8/254: dwrite d0/d41/d50/d3/dc/d11/d12/f39 [0,4194304] 0 2024-07-03T15:44:03.385 INFO:tasks.workunit.client.1.smithi148.stdout:0/952: mknod da/d58/d90/dd1/d109/c143 0 2024-07-03T15:44:03.385 INFO:tasks.workunit.client.0.smithi028.stdout:9/251: dwrite d1/d16/f1c [0,4194304] 0 2024-07-03T15:44:03.386 INFO:tasks.workunit.client.0.smithi028.stdout:7/117: unlink f7 0 2024-07-03T15:44:03.390 INFO:tasks.workunit.client.0.smithi028.stdout:3/285: creat d18/d1a/d5e/f5f x:0 0 0 2024-07-03T15:44:03.390 INFO:tasks.workunit.client.0.smithi028.stdout:3/286: read - d18/d1a/d5e/f5f zero size 2024-07-03T15:44:03.390 INFO:tasks.workunit.client.0.smithi028.stdout:3/287: fdatasync d18/d3a/f47 0 2024-07-03T15:44:03.390 INFO:tasks.workunit.client.0.smithi028.stdout:3/288: fdatasync d18/f52 0 2024-07-03T15:44:03.398 INFO:tasks.workunit.client.0.smithi028.stdout:3/289: dread d18/f2d [0,4194304] 0 2024-07-03T15:44:03.400 INFO:tasks.workunit.client.0.smithi028.stdout:9/252: mkdir d1/d10/d28/d4e/d4f 0 2024-07-03T15:44:03.400 INFO:tasks.workunit.client.0.smithi028.stdout:9/253: truncate d1/dd/f20 1708195 0 2024-07-03T15:44:03.401 INFO:tasks.workunit.client.0.smithi028.stdout:5/267: dread d0/dd/d1e/f56 [0,4194304] 0 2024-07-03T15:44:03.401 INFO:tasks.workunit.client.0.smithi028.stdout:5/268: stat d0/dd/df/f3e 0 2024-07-03T15:44:03.401 INFO:tasks.workunit.client.0.smithi028.stdout:4/220: mkdir d2/d12/d42 0 2024-07-03T15:44:03.404 INFO:tasks.workunit.client.0.smithi028.stdout:9/254: creat d1/d10/d28/d4e/f50 x:0 0 0 2024-07-03T15:44:03.405 INFO:tasks.workunit.client.0.smithi028.stdout:5/269: mkdir d0/dd/d1e/d3b/d3f/d5a 0 2024-07-03T15:44:03.407 INFO:tasks.workunit.client.0.smithi028.stdout:3/290: getdents d18 0 2024-07-03T15:44:03.407 INFO:tasks.workunit.client.0.smithi028.stdout:3/291: write f14 [737606,63916] 0 2024-07-03T15:44:03.409 INFO:tasks.workunit.client.0.smithi028.stdout:9/255: creat d1/d16/f51 x:0 0 0 2024-07-03T15:44:03.410 INFO:tasks.workunit.client.0.smithi028.stdout:4/221: creat d2/d12/d3b/f43 x:0 0 0 2024-07-03T15:44:03.410 INFO:tasks.workunit.client.0.smithi028.stdout:4/222: write d2/d12/d3b/f43 [890441,34735] 0 2024-07-03T15:44:03.411 INFO:tasks.workunit.client.0.smithi028.stdout:8/255: dwrite d0/d41/d50/d3/dc/f1f [0,4194304] 0 2024-07-03T15:44:03.412 INFO:tasks.workunit.client.0.smithi028.stdout:8/256: readlink d0/d41/d50/d3/dc/d11/d12/l14 0 2024-07-03T15:44:03.412 INFO:tasks.workunit.client.0.smithi028.stdout:8/257: dread - d0/d41/d50/d3/dc/f34 zero size 2024-07-03T15:44:03.413 INFO:tasks.workunit.client.0.smithi028.stdout:7/118: dwrite f0 [0,4194304] 0 2024-07-03T15:44:03.413 INFO:tasks.workunit.client.0.smithi028.stdout:9/256: creat d1/d4a/f52 x:0 0 0 2024-07-03T15:44:03.416 INFO:tasks.workunit.client.0.smithi028.stdout:5/270: dread d0/dd/d13/d57/d26/f2e [0,4194304] 0 2024-07-03T15:44:03.420 INFO:tasks.workunit.client.0.smithi028.stdout:9/257: symlink d1/l53 0 2024-07-03T15:44:03.420 INFO:tasks.workunit.client.0.smithi028.stdout:9/258: write d1/f2 [1401914,105511] 0 2024-07-03T15:44:03.420 INFO:tasks.workunit.client.1.smithi148.stdout:0/953: getdents da/d58/dbd/d13/d14/d50/d95/d8c 0 2024-07-03T15:44:03.420 INFO:tasks.workunit.client.1.smithi148.stdout:0/954: read - da/f117 zero size 2024-07-03T15:44:03.420 INFO:tasks.workunit.client.1.smithi148.stdout:0/955: truncate da/d58/dbd/d24/dc8/f11d 498627 0 2024-07-03T15:44:03.423 INFO:tasks.workunit.client.0.smithi028.stdout:7/119: rename c1 to d9/d1c/c27 0 2024-07-03T15:44:03.425 INFO:tasks.workunit.client.0.smithi028.stdout:3/292: dwrite d18/d3a/f47 [0,4194304] 0 2024-07-03T15:44:03.425 INFO:tasks.workunit.client.1.smithi148.stdout:0/956: truncate da/d17/d125/f6c 1599258 0 2024-07-03T15:44:03.428 INFO:tasks.workunit.client.0.smithi028.stdout:9/259: unlink d1/d17/c1e 0 2024-07-03T15:44:03.431 INFO:tasks.workunit.client.1.smithi148.stdout:0/957: mknod da/d58/dbd/d4a/c144 0 2024-07-03T15:44:03.433 INFO:tasks.workunit.client.0.smithi028.stdout:8/258: dwrite d0/d41/d50/d3/dc/f23 [0,4194304] 0 2024-07-03T15:44:03.433 INFO:tasks.workunit.client.0.smithi028.stdout:8/259: chown d0/dd/d22/d40/d45/c54 2 1 2024-07-03T15:44:03.434 INFO:tasks.workunit.client.0.smithi028.stdout:3/293: dread d18/d1a/f20 [0,4194304] 0 2024-07-03T15:44:03.435 INFO:tasks.workunit.client.1.smithi148.stdout:0/958: creat da/d58/dbd/d24/dc8/d8a/f145 x:0 0 0 2024-07-03T15:44:03.435 INFO:tasks.workunit.client.1.smithi148.stdout:0/959: readlink da/d58/dbd/d94/dac/l131 0 2024-07-03T15:44:03.438 INFO:tasks.workunit.client.1.smithi148.stdout:0/960: creat da/d58/dbd/d13/d14/d50/d95/de2/f146 x:0 0 0 2024-07-03T15:44:03.441 INFO:tasks.workunit.client.0.smithi028.stdout:8/260: dread d0/d41/d50/d3/dc/d11/d12/f2f [0,4194304] 0 2024-07-03T15:44:03.443 INFO:tasks.workunit.client.0.smithi028.stdout:5/271: rmdir d0/dd 39 2024-07-03T15:44:03.446 INFO:tasks.workunit.client.0.smithi028.stdout:7/120: dwrite d9/f26 [0,4194304] 0 2024-07-03T15:44:03.446 INFO:tasks.workunit.client.0.smithi028.stdout:7/121: write d9/f1a [1564774,33590] 0 2024-07-03T15:44:03.446 INFO:tasks.workunit.client.0.smithi028.stdout:7/122: fdatasync d9/f26 0 2024-07-03T15:44:03.446 INFO:tasks.workunit.client.0.smithi028.stdout:7/123: fdatasync d9/d1c/f21 0 2024-07-03T15:44:03.447 INFO:tasks.workunit.client.0.smithi028.stdout:9/260: dwrite d1/d17/f48 [0,4194304] 0 2024-07-03T15:44:03.454 INFO:tasks.workunit.client.0.smithi028.stdout:5/272: write d0/dd/d13/f17 [1427484,75025] 0 2024-07-03T15:44:03.454 INFO:tasks.workunit.client.0.smithi028.stdout:5/273: read d0/dd/d13/f17 [530979,18684] 0 2024-07-03T15:44:03.455 INFO:tasks.workunit.client.0.smithi028.stdout:3/294: mkdir d18/d1a/d5e/d60 0 2024-07-03T15:44:03.455 INFO:tasks.workunit.client.0.smithi028.stdout:3/295: stat d18/d1a/d1e/f53 0 2024-07-03T15:44:03.456 INFO:tasks.workunit.client.0.smithi028.stdout:3/296: read d18/d1a/d1e/f3e [180716,66504] 0 2024-07-03T15:44:03.457 INFO:tasks.workunit.client.0.smithi028.stdout:8/261: link d0/d41/d50/d3/dc/f2b d0/d41/d50/d3/dc/d11/d12/f56 0 2024-07-03T15:44:03.460 INFO:tasks.workunit.client.0.smithi028.stdout:4/223: rename d2/d12/d19/c1c to d2/d12/d2c/c44 0 2024-07-03T15:44:03.463 INFO:tasks.workunit.client.1.smithi148.stdout:0/961: rmdir da/d58/dbd/d13 39 2024-07-03T15:44:03.467 INFO:tasks.workunit.client.0.smithi028.stdout:9/261: symlink d1/d10/d28/l54 0 2024-07-03T15:44:03.467 INFO:tasks.workunit.client.0.smithi028.stdout:9/262: readlink d1/l12 0 2024-07-03T15:44:03.467 INFO:tasks.workunit.client.0.smithi028.stdout:9/263: chown d1/d16/f2f 193517 1 2024-07-03T15:44:03.467 INFO:tasks.workunit.client.0.smithi028.stdout:3/297: dread d18/f42 [0,4194304] 0 2024-07-03T15:44:03.468 INFO:tasks.workunit.client.0.smithi028.stdout:3/298: creat d18/d43/f61 x:0 0 0 2024-07-03T15:44:03.468 INFO:tasks.workunit.client.0.smithi028.stdout:3/299: chown d18/f51 16185 1 2024-07-03T15:44:03.468 INFO:tasks.workunit.client.0.smithi028.stdout:5/274: creat d0/dd/d13/d47/f5b x:0 0 0 2024-07-03T15:44:03.470 INFO:tasks.workunit.client.0.smithi028.stdout:8/262: symlink d0/d41/d50/d3/dc/d3f/l57 0 2024-07-03T15:44:03.470 INFO:tasks.workunit.client.0.smithi028.stdout:8/263: chown d0/d41/d50/d3/dc/d11/d12/c31 1281 1 2024-07-03T15:44:03.472 INFO:tasks.workunit.client.1.smithi148.stdout:0/962: write da/d58/dbd/d94/f11e [4060632,127657] 0 2024-07-03T15:44:03.473 INFO:tasks.workunit.client.0.smithi028.stdout:9/264: dread d1/f9 [0,4194304] 0 2024-07-03T15:44:03.474 INFO:tasks.workunit.client.0.smithi028.stdout:4/224: truncate d2/f13 1660149 0 2024-07-03T15:44:03.478 INFO:tasks.workunit.client.0.smithi028.stdout:5/275: dread d0/dd/d13/d57/d26/f42 [0,4194304] 0 2024-07-03T15:44:03.479 INFO:tasks.workunit.client.0.smithi028.stdout:9/265: creat d1/dd/f55 x:0 0 0 2024-07-03T15:44:03.480 INFO:tasks.workunit.client.1.smithi148.stdout:0/963: getdents da 0 2024-07-03T15:44:03.480 INFO:tasks.workunit.client.1.smithi148.stdout:0/964: write da/d17/d2a/f59 [2776662,15144] 0 2024-07-03T15:44:03.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:03 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:44:03.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:03 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:44:03.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:03 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:44:03.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:03 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:44:03.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:03 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:44:03.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:03 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:44:03.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:03 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mgr fail", "who": "smithi028.vtqrtz"}]: dispatch 2024-07-03T15:44:03.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:03 smithi148 ceph-mon[36056]: Activating manager daemon smithi148.rjzsar 2024-07-03T15:44:03.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:03 smithi148 ceph-mon[36056]: osdmap e46: 6 total, 6 up, 6 in 2024-07-03T15:44:03.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:03 smithi148 ceph-mon[36056]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "mgr fail", "who": "smithi028.vtqrtz"}]': finished 2024-07-03T15:44:03.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:03 smithi148 ceph-mon[36056]: mgrmap e21: smithi148.rjzsar(active, starting, since 0.0133711s) 2024-07-03T15:44:03.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:03 smithi148 ceph-mon[36056]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "mon metadata", "id": "smithi028"}]: dispatch 2024-07-03T15:44:03.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:03 smithi148 ceph-mon[36056]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-07-03T15:44:03.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:03 smithi148 ceph-mon[36056]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi028.rhpvve"}]: dispatch 2024-07-03T15:44:03.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:03 smithi148 ceph-mon[36056]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi028.wmbxew"}]: dispatch 2024-07-03T15:44:03.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:03 smithi148 ceph-mon[36056]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.szomfl"}]: dispatch 2024-07-03T15:44:03.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:03 smithi148 ceph-mon[36056]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.xgpuld"}]: dispatch 2024-07-03T15:44:03.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:03 smithi148 ceph-mon[36056]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "mgr metadata", "who": "smithi148.rjzsar", "id": "smithi148.rjzsar"}]: dispatch 2024-07-03T15:44:03.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:03 smithi148 ceph-mon[36056]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-03T15:44:03.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:03 smithi148 ceph-mon[36056]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-03T15:44:03.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:03 smithi148 ceph-mon[36056]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-03T15:44:03.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:03 smithi148 ceph-mon[36056]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-03T15:44:03.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:03 smithi148 ceph-mon[36056]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-03T15:44:03.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:03 smithi148 ceph-mon[36056]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-03T15:44:03.485 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:03 smithi148 ceph-mon[36056]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-03T15:44:03.485 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:03 smithi148 ceph-mon[36056]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-03T15:44:03.485 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:03 smithi148 ceph-mon[36056]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-03T15:44:03.485 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:03 smithi148 ceph-mon[36056]: Manager daemon smithi148.rjzsar is now available 2024-07-03T15:44:03.485 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:03 smithi148 ceph-mon[36056]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:44:03.485 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:03 smithi148 ceph-mon[36056]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:44:03.485 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:03 smithi148 ceph-mon[36056]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi148.rjzsar/mirror_snapshot_schedule"}]: dispatch 2024-07-03T15:44:03.485 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:03 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi148.rjzsar/mirror_snapshot_schedule"}]: dispatch 2024-07-03T15:44:03.485 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:03 smithi148 ceph-mon[36056]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi148.rjzsar/trash_purge_schedule"}]: dispatch 2024-07-03T15:44:03.485 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:03 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi148.rjzsar/trash_purge_schedule"}]: dispatch 2024-07-03T15:44:03.486 INFO:tasks.workunit.client.1.smithi148.stdout:0/965: link da/d58/dbd/d94/dac/ddd/d88/f105 da/d58/dbd/d94/dac/f147 0 2024-07-03T15:44:03.486 INFO:tasks.workunit.client.0.smithi028.stdout:5/276: link d0/dd/c15 d0/dd/d1e/d3b/d3f/c5c 0 2024-07-03T15:44:03.486 INFO:tasks.workunit.client.0.smithi028.stdout:5/277: write d0/dd/d1e/d3b/f4f [862509,58288] 0 2024-07-03T15:44:03.486 INFO:tasks.workunit.client.0.smithi028.stdout:5/278: stat d0/dd/df/f16 0 2024-07-03T15:44:03.486 INFO:tasks.workunit.client.0.smithi028.stdout:9/266: rename d1/d17/d30 to d1/d10/d28/d45/d56 0 2024-07-03T15:44:03.487 INFO:tasks.workunit.client.0.smithi028.stdout:4/225: mkdir d2/d12/d19/d45 0 2024-07-03T15:44:03.487 INFO:tasks.workunit.client.0.smithi028.stdout:4/226: write d2/d12/d19/f36 [12108,75839] 0 2024-07-03T15:44:03.488 INFO:tasks.workunit.client.0.smithi028.stdout:5/279: getdents d0/dd/d13 0 2024-07-03T15:44:03.488 INFO:tasks.workunit.client.0.smithi028.stdout:5/280: chown d0/dd/d14 3263 1 2024-07-03T15:44:03.489 INFO:tasks.workunit.client.0.smithi028.stdout:8/264: dread d0/d41/d50/d3/dc/d11/d12/f56 [0,4194304] 0 2024-07-03T15:44:03.489 INFO:tasks.workunit.client.0.smithi028.stdout:4/227: symlink d2/d12/d2d/d2f/l46 0 2024-07-03T15:44:03.489 INFO:tasks.workunit.client.0.smithi028.stdout:4/228: fsync d2/d12/f40 0 2024-07-03T15:44:03.492 INFO:tasks.workunit.client.0.smithi028.stdout:5/281: rename d0/c8 to d0/dd/d13/d57/d26/d41/c5d 0 2024-07-03T15:44:03.498 INFO:tasks.workunit.client.0.smithi028.stdout:9/267: read d1/f21 [877213,54523] 0 2024-07-03T15:44:03.498 INFO:tasks.workunit.client.0.smithi028.stdout:8/265: truncate d0/d41/d50/d3/dc/d11/d12/f15 2134095 0 2024-07-03T15:44:03.498 INFO:tasks.workunit.client.0.smithi028.stdout:9/268: stat d1/d10/d28/d45/d56/l4d 0 2024-07-03T15:44:03.498 INFO:tasks.workunit.client.0.smithi028.stdout:4/229: creat d2/d12/d2e/f47 x:0 0 0 2024-07-03T15:44:03.500 INFO:tasks.workunit.client.0.smithi028.stdout:9/269: mkdir d1/d16/d57 0 2024-07-03T15:44:03.501 INFO:tasks.workunit.client.0.smithi028.stdout:5/282: dread d0/f9 [4194304,4194304] 0 2024-07-03T15:44:03.503 INFO:tasks.workunit.client.0.smithi028.stdout:9/270: mknod d1/d10/d28/d45/d56/c58 0 2024-07-03T15:44:03.503 INFO:tasks.workunit.client.0.smithi028.stdout:9/271: creat d1/dd/f59 x:0 0 0 2024-07-03T15:44:03.504 INFO:tasks.workunit.client.0.smithi028.stdout:3/300: dwrite d18/d43/f61 [0,4194304] 0 2024-07-03T15:44:03.504 INFO:tasks.workunit.client.0.smithi028.stdout:5/283: write d0/dd/d13/d57/d26/f2e [1574902,39546] 0 2024-07-03T15:44:03.504 INFO:tasks.workunit.client.0.smithi028.stdout:7/124: dwrite f4 [0,4194304] 0 2024-07-03T15:44:03.505 INFO:tasks.workunit.client.0.smithi028.stdout:9/272: rename d1/l4 to d1/d16/d57/l5a 0 2024-07-03T15:44:03.505 INFO:tasks.workunit.client.0.smithi028.stdout:9/273: truncate d1/f4b 401702 0 2024-07-03T15:44:03.505 INFO:tasks.workunit.client.0.smithi028.stdout:9/274: chown d1/dd/f59 182 1 2024-07-03T15:44:03.505 INFO:tasks.workunit.client.0.smithi028.stdout:9/275: write d1/dd/f4c [2072298,57377] 0 2024-07-03T15:44:03.509 INFO:tasks.workunit.client.0.smithi028.stdout:3/301: stat d18/l34 0 2024-07-03T15:44:03.510 INFO:tasks.workunit.client.0.smithi028.stdout:3/302: dread f9 [0,4194304] 0 2024-07-03T15:44:03.513 INFO:tasks.workunit.client.1.smithi148.stdout:0/966: dwrite da/d17/d2a/d2f/d93/f104 [0,4194304] 0 2024-07-03T15:44:03.514 INFO:tasks.workunit.client.1.smithi148.stdout:0/967: fsync da/d58/dbd/d13/f71 0 2024-07-03T15:44:03.514 INFO:tasks.workunit.client.0.smithi028.stdout:8/266: write d0/d41/d50/d3/dc/d11/d12/f39 [1400530,39214] 0 2024-07-03T15:44:03.514 INFO:tasks.workunit.client.0.smithi028.stdout:8/267: fsync d0/d41/d50/d3/dc/f1f 0 2024-07-03T15:44:03.514 INFO:tasks.workunit.client.1.smithi148.stdout:0/968: dread da/d58/dbd/d13/d14/d50/d95/d8c/f119 [0,4194304] 0 2024-07-03T15:44:03.521 INFO:tasks.workunit.client.0.smithi028.stdout:4/230: dwrite f1 [0,4194304] 0 2024-07-03T15:44:03.522 INFO:tasks.workunit.client.0.smithi028.stdout:4/231: read d2/d12/d19/f34 [706233,55723] 0 2024-07-03T15:44:03.524 INFO:tasks.workunit.client.1.smithi148.stdout:0/969: creat da/d58/d90/d12c/f148 x:0 0 0 2024-07-03T15:44:03.527 INFO:tasks.workunit.client.1.smithi148.stdout:0/970: getdents da/d58/dbd/d94/dac/ddd 0 2024-07-03T15:44:03.529 INFO:tasks.workunit.client.0.smithi028.stdout:5/284: creat d0/dd/d1e/d3b/f5e x:0 0 0 2024-07-03T15:44:03.530 INFO:tasks.workunit.client.0.smithi028.stdout:7/125: dread - d9/fa zero size 2024-07-03T15:44:03.531 INFO:tasks.workunit.client.1.smithi148.stdout:0/971: getdents da/d58/dbd/d13/d14/d50/d95/de2 0 2024-07-03T15:44:03.531 INFO:tasks.workunit.client.0.smithi028.stdout:7/126: write f6 [2455250,129733] 0 2024-07-03T15:44:03.531 INFO:tasks.workunit.client.0.smithi028.stdout:7/127: write d9/f26 [4625411,52312] 0 2024-07-03T15:44:03.535 INFO:tasks.workunit.client.0.smithi028.stdout:7/128: dread d9/f10 [4194304,4194304] 0 2024-07-03T15:44:03.535 INFO:tasks.workunit.client.0.smithi028.stdout:9/276: truncate d1/f9 508867 0 2024-07-03T15:44:03.536 INFO:tasks.workunit.client.0.smithi028.stdout:3/303: mknod d18/d1a/d5e/d60/c62 0 2024-07-03T15:44:03.538 INFO:tasks.workunit.client.0.smithi028.stdout:8/268: rename d0/c55 to d0/dd/d22/c58 0 2024-07-03T15:44:03.540 INFO:tasks.workunit.client.0.smithi028.stdout:5/285: creat d0/dd/d13/d57/d26/d41/f5f x:0 0 0 2024-07-03T15:44:03.542 INFO:tasks.workunit.client.0.smithi028.stdout:9/277: unlink d1/d17/f48 0 2024-07-03T15:44:03.542 INFO:tasks.workunit.client.0.smithi028.stdout:9/278: write d1/f31 [119068,34989] 0 2024-07-03T15:44:03.543 INFO:tasks.workunit.client.0.smithi028.stdout:7/129: symlink d9/d1c/l28 0 2024-07-03T15:44:03.545 INFO:tasks.workunit.client.0.smithi028.stdout:8/269: dread d0/d41/d50/d3/dc/d11/d12/f15 [0,4194304] 0 2024-07-03T15:44:03.545 INFO:tasks.workunit.client.0.smithi028.stdout:3/304: symlink d18/d3a/l63 0 2024-07-03T15:44:03.545 INFO:tasks.workunit.client.0.smithi028.stdout:3/305: chown d18/d1a/d1e/l27 0 1 2024-07-03T15:44:03.546 INFO:tasks.workunit.client.0.smithi028.stdout:5/286: mknod d0/dd/d13/d47/c60 0 2024-07-03T15:44:03.549 INFO:tasks.workunit.client.0.smithi028.stdout:9/279: rmdir d1/d2a 39 2024-07-03T15:44:03.549 INFO:tasks.workunit.client.0.smithi028.stdout:9/280: write d1/d4a/f52 [300663,56905] 0 2024-07-03T15:44:03.549 INFO:tasks.workunit.client.0.smithi028.stdout:9/281: stat d1/dd/f13 0 2024-07-03T15:44:03.550 INFO:tasks.workunit.client.0.smithi028.stdout:9/282: readlink d1/l12 0 2024-07-03T15:44:03.562 INFO:tasks.workunit.client.0.smithi028.stdout:7/130: mkdir d9/d29 0 2024-07-03T15:44:03.563 INFO:tasks.workunit.client.0.smithi028.stdout:0/194: sync 2024-07-03T15:44:03.565 INFO:tasks.workunit.client.0.smithi028.stdout:4/232: dwrite d2/d12/f26 [0,4194304] 0 2024-07-03T15:44:03.566 INFO:tasks.workunit.client.0.smithi028.stdout:4/233: chown d2/d12/d2d/d2f/c37 42429877 1 2024-07-03T15:44:03.573 INFO:tasks.workunit.client.1.smithi148.stdout:0/972: dwrite da/f16 [0,4194304] 0 2024-07-03T15:44:03.573 INFO:tasks.workunit.client.1.smithi148.stdout:0/973: write da/d58/dbd/d13/d14/f49 [669193,45742] 0 2024-07-03T15:44:03.573 INFO:tasks.workunit.client.1.smithi148.stdout:0/974: chown da/d58/dbd/d13/d14/d50/d95 182 1 2024-07-03T15:44:03.574 INFO:tasks.workunit.client.0.smithi028.stdout:0/195: rename d2/d3/c16 to d2/d3/c3e 0 2024-07-03T15:44:03.574 INFO:tasks.workunit.client.0.smithi028.stdout:0/196: chown d2/d3/c17 3 1 2024-07-03T15:44:03.574 INFO:tasks.workunit.client.0.smithi028.stdout:4/234: getdents d2/d3e 0 2024-07-03T15:44:03.574 INFO:tasks.workunit.client.0.smithi028.stdout:4/235: creat d2/d12/d3b/f48 x:0 0 0 2024-07-03T15:44:03.574 INFO:tasks.workunit.client.0.smithi028.stdout:4/236: write d2/d12/d3b/f43 [1923713,120517] 0 2024-07-03T15:44:03.576 INFO:tasks.workunit.client.1.smithi148.stdout:0/975: mknod da/d17/d125/c149 0 2024-07-03T15:44:03.576 INFO:tasks.workunit.client.0.smithi028.stdout:9/283: dwrite d1/dd/f11 [0,4194304] 0 2024-07-03T15:44:03.577 INFO:tasks.workunit.client.0.smithi028.stdout:9/284: chown d1/d10/d28/d4e 3913158 1 2024-07-03T15:44:03.579 INFO:tasks.workunit.client.0.smithi028.stdout:5/287: dwrite d0/dd/d13/d47/f5b [0,4194304] 0 2024-07-03T15:44:03.579 INFO:tasks.workunit.client.0.smithi028.stdout:5/288: write d0/dd/d13/d57/d26/f2e [1156867,41556] 0 2024-07-03T15:44:03.579 INFO:tasks.workunit.client.0.smithi028.stdout:3/306: dwrite d18/d3a/f5a [0,4194304] 0 2024-07-03T15:44:03.581 INFO:tasks.workunit.client.0.smithi028.stdout:8/270: dwrite d0/d41/d50/d3/dc/d11/d12/f2f [0,4194304] 0 2024-07-03T15:44:03.582 INFO:tasks.workunit.client.0.smithi028.stdout:5/289: symlink d0/dd/d13/d57/d26/d41/l61 0 2024-07-03T15:44:03.588 INFO:tasks.workunit.client.0.smithi028.stdout:8/271: rmdir d0/d41/d50/d3/dc/d4d 39 2024-07-03T15:44:03.592 INFO:tasks.workunit.client.0.smithi028.stdout:4/237: dread d2/d12/d19/f9 [0,4194304] 0 2024-07-03T15:44:03.592 INFO:tasks.workunit.client.0.smithi028.stdout:3/307: dread d18/d3a/f47 [0,4194304] 0 2024-07-03T15:44:03.592 INFO:tasks.workunit.client.0.smithi028.stdout:3/308: write d18/d1a/f4f [948437,80841] 0 2024-07-03T15:44:03.592 INFO:tasks.workunit.client.0.smithi028.stdout:7/131: getdents d9/d1c 0 2024-07-03T15:44:03.596 INFO:tasks.workunit.client.0.smithi028.stdout:5/290: link d0/dd/l24 d0/dd/d1e/d3b/d3f/d5a/l62 0 2024-07-03T15:44:03.599 INFO:tasks.workunit.client.0.smithi028.stdout:4/238: symlink d2/d12/d2e/l49 0 2024-07-03T15:44:03.601 INFO:tasks.workunit.client.0.smithi028.stdout:7/132: rename d9/f26 to d9/d1c/f2a 0 2024-07-03T15:44:03.601 INFO:tasks.workunit.client.0.smithi028.stdout:7/133: truncate d9/f1b 345796 0 2024-07-03T15:44:03.602 INFO:tasks.workunit.client.0.smithi028.stdout:7/134: write f0 [9947977,43549] 0 2024-07-03T15:44:03.602 INFO:tasks.workunit.client.0.smithi028.stdout:7/135: getdents d9/d29 0 2024-07-03T15:44:03.602 INFO:tasks.workunit.client.0.smithi028.stdout:7/136: chown f2 27493 1 2024-07-03T15:44:03.602 INFO:tasks.workunit.client.0.smithi028.stdout:7/137: chown d9/f10 25 1 2024-07-03T15:44:03.604 INFO:tasks.workunit.client.0.smithi028.stdout:7/138: symlink d9/d29/l2b 0 2024-07-03T15:44:03.608 INFO:tasks.workunit.client.0.smithi028.stdout:7/139: link d9/d1c/c27 d9/c2c 0 2024-07-03T15:44:03.608 INFO:tasks.workunit.client.0.smithi028.stdout:1/337: sync 2024-07-03T15:44:03.608 INFO:tasks.workunit.client.0.smithi028.stdout:1/338: chown d4/de/f85 447250 1 2024-07-03T15:44:03.608 INFO:tasks.workunit.client.0.smithi028.stdout:1/339: creat d4/d2c/d3a/d43/f88 x:0 0 0 2024-07-03T15:44:03.611 INFO:tasks.workunit.client.0.smithi028.stdout:7/140: symlink d9/l2d 0 2024-07-03T15:44:03.613 INFO:tasks.workunit.client.0.smithi028.stdout:7/141: creat d9/d29/f2e x:0 0 0 2024-07-03T15:44:03.614 INFO:tasks.workunit.client.0.smithi028.stdout:7/142: symlink d9/d1c/l2f 0 2024-07-03T15:44:03.615 INFO:tasks.workunit.client.0.smithi028.stdout:7/143: truncate d9/fe 1292513 0 2024-07-03T15:44:03.616 INFO:tasks.workunit.client.0.smithi028.stdout:7/144: mknod d9/c30 0 2024-07-03T15:44:03.621 INFO:tasks.workunit.client.1.smithi148.stdout:0/976: dwrite da/d58/dbd/d13/d14/d50/d95/d8c/d12a/f12e [0,4194304] 0 2024-07-03T15:44:03.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:03 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:44:03.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:03 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:44:03.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:03 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:44:03.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:03 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:44:03.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:03 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:44:03.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:03 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:44:03.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:03 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mgr fail", "who": "smithi028.vtqrtz"}]: dispatch 2024-07-03T15:44:03.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:03 smithi028 ceph-mon[29148]: Activating manager daemon smithi148.rjzsar 2024-07-03T15:44:03.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:03 smithi028 ceph-mon[29148]: osdmap e46: 6 total, 6 up, 6 in 2024-07-03T15:44:03.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:03 smithi028 ceph-mon[29148]: from='mgr.14210 172.21.15.28:0/3635417239' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "mgr fail", "who": "smithi028.vtqrtz"}]': finished 2024-07-03T15:44:03.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:03 smithi028 ceph-mon[29148]: mgrmap e21: smithi148.rjzsar(active, starting, since 0.0133711s) 2024-07-03T15:44:03.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:03 smithi028 ceph-mon[29148]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "mon metadata", "id": "smithi028"}]: dispatch 2024-07-03T15:44:03.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:03 smithi028 ceph-mon[29148]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-07-03T15:44:03.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:03 smithi028 ceph-mon[29148]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi028.rhpvve"}]: dispatch 2024-07-03T15:44:03.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:03 smithi028 ceph-mon[29148]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi028.wmbxew"}]: dispatch 2024-07-03T15:44:03.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:03 smithi028 ceph-mon[29148]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.szomfl"}]: dispatch 2024-07-03T15:44:03.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:03 smithi028 ceph-mon[29148]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.xgpuld"}]: dispatch 2024-07-03T15:44:03.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:03 smithi028 ceph-mon[29148]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "mgr metadata", "who": "smithi148.rjzsar", "id": "smithi148.rjzsar"}]: dispatch 2024-07-03T15:44:03.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:03 smithi028 ceph-mon[29148]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-03T15:44:03.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:03 smithi028 ceph-mon[29148]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-03T15:44:03.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:03 smithi028 ceph-mon[29148]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-03T15:44:03.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:03 smithi028 ceph-mon[29148]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-03T15:44:03.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:03 smithi028 ceph-mon[29148]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-03T15:44:03.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:03 smithi028 ceph-mon[29148]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-03T15:44:03.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:03 smithi028 ceph-mon[29148]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-03T15:44:03.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:03 smithi028 ceph-mon[29148]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-03T15:44:03.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:03 smithi028 ceph-mon[29148]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-03T15:44:03.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:03 smithi028 ceph-mon[29148]: Manager daemon smithi148.rjzsar is now available 2024-07-03T15:44:03.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:03 smithi028 ceph-mon[29148]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:44:03.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:03 smithi028 ceph-mon[29148]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:44:03.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:03 smithi028 ceph-mon[29148]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi148.rjzsar/mirror_snapshot_schedule"}]: dispatch 2024-07-03T15:44:03.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:03 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi148.rjzsar/mirror_snapshot_schedule"}]: dispatch 2024-07-03T15:44:03.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:03 smithi028 ceph-mon[29148]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi148.rjzsar/trash_purge_schedule"}]: dispatch 2024-07-03T15:44:03.625 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:03 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi148.rjzsar/trash_purge_schedule"}]: dispatch 2024-07-03T15:44:03.632 INFO:tasks.workunit.client.0.smithi028.stdout:7/145: fsync d9/f1f 0 2024-07-03T15:44:03.632 INFO:tasks.workunit.client.0.smithi028.stdout:7/146: chown d9/d1c/f2a 95885 1 2024-07-03T15:44:03.633 INFO:tasks.workunit.client.0.smithi028.stdout:5/291: dwrite d0/dd/d1e/d3b/f5e [0,4194304] 0 2024-07-03T15:44:03.633 INFO:tasks.workunit.client.0.smithi028.stdout:5/292: readlink d0/l1d 0 2024-07-03T15:44:03.633 INFO:tasks.workunit.client.0.smithi028.stdout:5/293: write d0/f2c [3600026,23096] 0 2024-07-03T15:44:03.634 INFO:tasks.workunit.client.0.smithi028.stdout:1/340: dread d4/d2c/d3a/f68 [0,4194304] 0 2024-07-03T15:44:03.634 INFO:tasks.workunit.client.0.smithi028.stdout:8/272: dwrite d0/d41/d50/d3/dc/f34 [0,4194304] 0 2024-07-03T15:44:03.634 INFO:tasks.workunit.client.0.smithi028.stdout:7/147: read d9/f1b [134959,115566] 0 2024-07-03T15:44:03.634 INFO:tasks.workunit.client.0.smithi028.stdout:9/285: dwrite d1/dd/f59 [0,4194304] 0 2024-07-03T15:44:03.635 INFO:tasks.workunit.client.0.smithi028.stdout:4/239: dwrite d2/d12/d19/f24 [0,4194304] 0 2024-07-03T15:44:03.635 INFO:tasks.workunit.client.0.smithi028.stdout:4/240: readlink d2/d12/d2d/l31 0 2024-07-03T15:44:03.635 INFO:tasks.workunit.client.0.smithi028.stdout:4/241: creat d2/d12/d3b/f4a x:0 0 0 2024-07-03T15:44:03.637 INFO:tasks.workunit.client.0.smithi028.stdout:0/197: dwrite d2/d1c/f2d [0,4194304] 0 2024-07-03T15:44:03.637 INFO:tasks.workunit.client.0.smithi028.stdout:0/198: dread - d2/d3/f2f zero size 2024-07-03T15:44:03.637 INFO:tasks.workunit.client.0.smithi028.stdout:6/181: sync 2024-07-03T15:44:03.637 INFO:tasks.workunit.client.0.smithi028.stdout:2/261: sync 2024-07-03T15:44:03.638 INFO:tasks.workunit.client.0.smithi028.stdout:2/262: truncate da/d42/f4b 893710 0 2024-07-03T15:44:03.638 INFO:tasks.workunit.client.0.smithi028.stdout:2/263: dread - da/d23/d30/f62 zero size 2024-07-03T15:44:03.638 INFO:tasks.workunit.client.0.smithi028.stdout:6/182: write d4/d15/f20 [4704529,27983] 0 2024-07-03T15:44:03.638 INFO:tasks.workunit.client.0.smithi028.stdout:2/264: creat da/d23/d30/f67 x:0 0 0 2024-07-03T15:44:03.638 INFO:tasks.workunit.client.0.smithi028.stdout:6/183: write d4/d15/f20 [582838,59895] 0 2024-07-03T15:44:03.638 INFO:tasks.workunit.client.0.smithi028.stdout:6/184: chown d4/d15/l25 19 1 2024-07-03T15:44:03.638 INFO:tasks.workunit.client.0.smithi028.stdout:5/294: dread d0/dd/d13/d57/f1f [0,4194304] 0 2024-07-03T15:44:03.639 INFO:tasks.workunit.client.0.smithi028.stdout:8/273: truncate d0/d41/d50/d3/dc/f23 1387082 0 2024-07-03T15:44:03.642 INFO:tasks.workunit.client.0.smithi028.stdout:1/341: link d4/d3c/d54/d79/f1b d4/d2c/d3a/d5c/d6c/d7d/f89 0 2024-07-03T15:44:03.642 INFO:tasks.workunit.client.0.smithi028.stdout:1/342: write d4/d2c/f75 [1545001,20845] 0 2024-07-03T15:44:03.648 INFO:tasks.workunit.client.0.smithi028.stdout:9/286: truncate d1/d16/f19 1432365 0 2024-07-03T15:44:03.648 INFO:tasks.workunit.client.1.smithi148.stdout:0/977: dwrite da/d17/d2a/d2f/ff5 [0,4194304] 0 2024-07-03T15:44:03.654 INFO:tasks.workunit.client.0.smithi028.stdout:0/199: truncate d2/d3/f2c 175666 0 2024-07-03T15:44:03.655 INFO:tasks.workunit.client.0.smithi028.stdout:7/148: dread f0 [0,4194304] 0 2024-07-03T15:44:03.655 INFO:tasks.workunit.client.0.smithi028.stdout:7/149: creat d9/d29/f31 x:0 0 0 2024-07-03T15:44:03.656 INFO:tasks.workunit.client.0.smithi028.stdout:3/309: dwrite d18/d1a/f4f [0,4194304] 0 2024-07-03T15:44:03.657 INFO:tasks.workunit.client.0.smithi028.stdout:3/310: readlink d18/l34 0 2024-07-03T15:44:03.660 INFO:tasks.workunit.client.0.smithi028.stdout:2/265: truncate da/f10 3045058 0 2024-07-03T15:44:03.662 INFO:tasks.workunit.client.0.smithi028.stdout:6/185: dwrite f0 [4194304,4194304] 0 2024-07-03T15:44:03.662 INFO:tasks.workunit.client.0.smithi028.stdout:5/295: symlink d0/dd/d13/d47/l63 0 2024-07-03T15:44:03.664 INFO:tasks.workunit.client.0.smithi028.stdout:8/274: rename d0/d41/d50/d3/dc/d11/l2c to d0/d41/d50/d3/dc/d33/l59 0 2024-07-03T15:44:03.664 INFO:tasks.workunit.client.0.smithi028.stdout:8/275: readlink d0/d41/d50/l51 0 2024-07-03T15:44:03.665 INFO:tasks.workunit.client.0.smithi028.stdout:1/343: symlink d4/de/d1c/d55/l8a 0 2024-07-03T15:44:03.665 INFO:tasks.workunit.client.0.smithi028.stdout:1/344: fdatasync d4/d2c/d3a/d5c/d6c/d7d/d59/f82 0 2024-07-03T15:44:03.666 INFO:tasks.workunit.client.0.smithi028.stdout:9/287: mkdir d1/d10/d28/d4e/d5b 0 2024-07-03T15:44:03.667 INFO:tasks.workunit.client.1.smithi148.stdout:0/978: dread da/d58/dbd/d13/d1d/d3e/d85/d8e/f10a [0,4194304] 0 2024-07-03T15:44:03.668 INFO:tasks.workunit.client.0.smithi028.stdout:4/242: truncate d2/d12/f17 349916 0 2024-07-03T15:44:03.677 INFO:tasks.workunit.client.1.smithi148.stdout:0/979: getdents da/d58/dbd/d94/dba 0 2024-07-03T15:44:03.679 INFO:tasks.workunit.client.0.smithi028.stdout:5/296: rename d0/l11 to d0/dd/d13/d57/d26/d41/l64 0 2024-07-03T15:44:03.681 INFO:tasks.workunit.client.0.smithi028.stdout:1/345: dread d4/de/f31 [4194304,4194304] 0 2024-07-03T15:44:03.681 INFO:tasks.workunit.client.0.smithi028.stdout:1/346: chown d4/d3c/d54/d79/l28 1 1 2024-07-03T15:44:03.681 INFO:tasks.workunit.client.0.smithi028.stdout:4/243: link d2/f1b d2/d12/d3b/f4b 0 2024-07-03T15:44:03.682 INFO:tasks.workunit.client.0.smithi028.stdout:7/150: symlink d9/l32 0 2024-07-03T15:44:03.683 INFO:tasks.workunit.client.0.smithi028.stdout:6/186: link d4/ce d4/d12/d2c/c3f 0 2024-07-03T15:44:03.687 INFO:tasks.workunit.client.0.smithi028.stdout:0/200: dwrite d2/d1c/d32/d36/f3d [0,4194304] 0 2024-07-03T15:44:03.687 INFO:tasks.workunit.client.0.smithi028.stdout:0/201: chown d2/d3/c10 418611 1 2024-07-03T15:44:03.688 INFO:tasks.workunit.client.0.smithi028.stdout:1/347: unlink d4/d2c/d3a/d5c/d6c/d7d/f6e 0 2024-07-03T15:44:03.688 INFO:tasks.workunit.client.0.smithi028.stdout:7/151: dread d9/f1a [0,4194304] 0 2024-07-03T15:44:03.688 INFO:tasks.workunit.client.0.smithi028.stdout:7/152: write d9/f25 [882649,52873] 0 2024-07-03T15:44:03.688 INFO:tasks.workunit.client.0.smithi028.stdout:0/202: dread d2/d3/f1d [0,4194304] 0 2024-07-03T15:44:03.688 INFO:tasks.workunit.client.0.smithi028.stdout:0/203: dread - d2/d1c/d32/f39 zero size 2024-07-03T15:44:03.688 INFO:tasks.workunit.client.0.smithi028.stdout:6/187: creat d4/da/d38/f40 x:0 0 0 2024-07-03T15:44:03.690 INFO:tasks.workunit.client.1.smithi148.stdout:0/980: write da/d58/dbd/d13/d14/d50/f60 [4507917,80285] 0 2024-07-03T15:44:03.692 INFO:tasks.workunit.client.0.smithi028.stdout:8/276: rmdir d0/d41/d50/d3/d35/d46 0 2024-07-03T15:44:03.692 INFO:tasks.workunit.client.0.smithi028.stdout:3/311: dwrite d18/d3a/f39 [0,4194304] 0 2024-07-03T15:44:03.696 INFO:tasks.workunit.client.0.smithi028.stdout:1/348: creat d4/d83/f8b x:0 0 0 2024-07-03T15:44:03.696 INFO:tasks.workunit.client.0.smithi028.stdout:9/288: dwrite d1/d16/f2e [0,4194304] 0 2024-07-03T15:44:03.698 INFO:tasks.workunit.client.0.smithi028.stdout:2/266: dread da/f10 [0,4194304] 0 2024-07-03T15:44:03.698 INFO:tasks.workunit.client.0.smithi028.stdout:2/267: dread - da/d42/f46 zero size 2024-07-03T15:44:03.698 INFO:tasks.workunit.client.0.smithi028.stdout:7/153: unlink d9/f1b 0 2024-07-03T15:44:03.699 INFO:tasks.workunit.client.0.smithi028.stdout:7/154: creat d9/d29/f33 x:0 0 0 2024-07-03T15:44:03.704 INFO:tasks.workunit.client.0.smithi028.stdout:3/312: dread f14 [0,4194304] 0 2024-07-03T15:44:03.704 INFO:tasks.workunit.client.0.smithi028.stdout:1/349: write d4/d2c/d3a/f68 [298702,42869] 0 2024-07-03T15:44:03.704 INFO:tasks.workunit.client.1.smithi148.stdout:0/981: dread da/d58/d90/db4/fcc [0,4194304] 0 2024-07-03T15:44:03.704 INFO:tasks.workunit.client.1.smithi148.stdout:0/982: stat da/d58/dbd/d13/d1d/d3e 0 2024-07-03T15:44:03.706 INFO:tasks.workunit.client.0.smithi028.stdout:0/204: link d2/d1c/f21 d2/d1c/d32/d36/f3f 0 2024-07-03T15:44:03.708 INFO:tasks.workunit.client.0.smithi028.stdout:3/313: dread d18/d3a/f47 [0,4194304] 0 2024-07-03T15:44:03.710 INFO:tasks.workunit.client.1.smithi148.stdout:0/983: rename da/d58/dbd/d13/f27 to da/d58/d90/dd1/f14a 0 2024-07-03T15:44:03.714 INFO:tasks.workunit.client.0.smithi028.stdout:1/350: dread d4/de/ff [0,4194304] 0 2024-07-03T15:44:03.714 INFO:tasks.workunit.client.1.smithi148.stdout:0/984: mknod da/d58/dbd/d24/dc8/c14b 0 2024-07-03T15:44:03.715 INFO:tasks.workunit.client.0.smithi028.stdout:2/268: dread f4 [4194304,4194304] 0 2024-07-03T15:44:03.715 INFO:tasks.workunit.client.0.smithi028.stdout:2/269: dread - da/d24/f55 zero size 2024-07-03T15:44:03.720 INFO:tasks.workunit.client.1.smithi148.stdout:0/985: read da/d17/d2a/d2f/fcd [160314,78040] 0 2024-07-03T15:44:03.720 INFO:tasks.workunit.client.1.smithi148.stdout:0/986: fdatasync da/d58/dbd/d94/dac/ddd/d88/f105 0 2024-07-03T15:44:03.720 INFO:tasks.workunit.client.0.smithi028.stdout:7/155: creat d9/f34 x:0 0 0 2024-07-03T15:44:03.721 INFO:tasks.workunit.client.0.smithi028.stdout:5/297: dwrite d0/dd/d1e/f56 [0,4194304] 0 2024-07-03T15:44:03.721 INFO:tasks.workunit.client.0.smithi028.stdout:5/298: chown d0/dd/df 959223970 1 2024-07-03T15:44:03.721 INFO:tasks.workunit.client.0.smithi028.stdout:5/299: dread - d0/dd/d13/d57/d26/d41/f54 zero size 2024-07-03T15:44:03.721 INFO:tasks.workunit.client.0.smithi028.stdout:5/300: readlink d0/dd/d13/d57/l3a 0 2024-07-03T15:44:03.721 INFO:tasks.workunit.client.0.smithi028.stdout:5/301: write d0/f2c [7309374,110123] 0 2024-07-03T15:44:03.722 INFO:tasks.workunit.client.0.smithi028.stdout:5/302: write d0/f2c [3670188,69161] 0 2024-07-03T15:44:03.722 INFO:tasks.workunit.client.0.smithi028.stdout:5/303: chown d0/dd/d13/d47/l4d 123214 1 2024-07-03T15:44:03.722 INFO:tasks.workunit.client.0.smithi028.stdout:5/304: fdatasync d0/dd/df/f16 0 2024-07-03T15:44:03.723 INFO:tasks.workunit.client.0.smithi028.stdout:6/188: rename d4/da/d1c/l3e to d4/l41 0 2024-07-03T15:44:03.723 INFO:tasks.workunit.client.0.smithi028.stdout:4/244: dwrite d2/d12/d2e/f47 [0,4194304] 0 2024-07-03T15:44:03.724 INFO:tasks.workunit.client.0.smithi028.stdout:5/305: dread d0/dd/d13/f17 [4194304,4194304] 0 2024-07-03T15:44:03.725 INFO:tasks.workunit.client.0.smithi028.stdout:1/351: rmdir d4/d2c/d3a/d5c/d6c/d7d/d4e 39 2024-07-03T15:44:03.727 INFO:tasks.workunit.client.0.smithi028.stdout:8/277: dwrite d0/f44 [0,4194304] 0 2024-07-03T15:44:03.727 INFO:tasks.workunit.client.0.smithi028.stdout:0/205: dread d2/ff [0,4194304] 0 2024-07-03T15:44:03.727 INFO:tasks.workunit.client.0.smithi028.stdout:8/278: write d0/d41/d50/d3/dc/d11/d12/f15 [1709832,16108] 0 2024-07-03T15:44:03.727 INFO:tasks.workunit.client.0.smithi028.stdout:8/279: fdatasync d0/dd/d22/f25 0 2024-07-03T15:44:03.730 INFO:tasks.workunit.client.0.smithi028.stdout:6/189: symlink d4/d15/l42 0 2024-07-03T15:44:03.730 INFO:tasks.workunit.client.0.smithi028.stdout:9/289: rename d1/c1d to d1/dd/c5c 0 2024-07-03T15:44:03.736 INFO:tasks.workunit.client.0.smithi028.stdout:4/245: creat d2/d12/d19/d45/f4c x:0 0 0 2024-07-03T15:44:03.736 INFO:tasks.workunit.client.0.smithi028.stdout:4/246: chown d2/d12/d2c/c44 0 1 2024-07-03T15:44:03.738 INFO:tasks.workunit.client.0.smithi028.stdout:3/314: dwrite d18/f55 [0,4194304] 0 2024-07-03T15:44:03.738 INFO:tasks.workunit.client.0.smithi028.stdout:3/315: readlink la 0 2024-07-03T15:44:03.738 INFO:tasks.workunit.client.0.smithi028.stdout:3/316: chown d18/d1a/f20 29 1 2024-07-03T15:44:03.739 INFO:tasks.workunit.client.0.smithi028.stdout:3/317: dread d18/d3a/f3f [0,4194304] 0 2024-07-03T15:44:03.745 INFO:tasks.workunit.client.0.smithi028.stdout:8/280: dread d0/dd/d22/f25 [0,4194304] 0 2024-07-03T15:44:03.747 INFO:tasks.workunit.client.0.smithi028.stdout:4/247: dread d2/d12/d19/f9 [0,4194304] 0 2024-07-03T15:44:03.748 INFO:tasks.workunit.client.0.smithi028.stdout:2/270: mkdir da/d1e/d68 0 2024-07-03T15:44:03.748 INFO:tasks.workunit.client.0.smithi028.stdout:2/271: write f4 [12631843,45567] 0 2024-07-03T15:44:03.749 INFO:tasks.workunit.client.0.smithi028.stdout:5/306: dwrite d0/dd/d1e/d3b/f4f [0,4194304] 0 2024-07-03T15:44:03.749 INFO:tasks.workunit.client.1.smithi148.stdout:0/987: dwrite da/d17/d2a/f10d [0,4194304] 0 2024-07-03T15:44:03.749 INFO:tasks.workunit.client.1.smithi148.stdout:0/988: creat da/d58/dbd/d24/dc8/d8a/f14c x:0 0 0 2024-07-03T15:44:03.750 INFO:tasks.workunit.client.0.smithi028.stdout:1/352: truncate d4/d3c/d54/d79/f3e 128285 0 2024-07-03T15:44:03.750 INFO:tasks.workunit.client.0.smithi028.stdout:1/353: read d4/d2c/d3a/d5c/d6c/d7d/d59/d7f/f87 [118003,92092] 0 2024-07-03T15:44:03.751 INFO:tasks.workunit.client.0.smithi028.stdout:1/354: readlink d4/d3c/d54/d79/l2b 0 2024-07-03T15:44:03.757 INFO:tasks.workunit.client.0.smithi028.stdout:1/355: read d4/f37 [990176,96223] 0 2024-07-03T15:44:03.757 INFO:tasks.workunit.client.0.smithi028.stdout:1/356: dread - d4/d2c/d3a/d43/f88 zero size 2024-07-03T15:44:03.757 INFO:tasks.workunit.client.0.smithi028.stdout:7/156: dwrite d9/d1c/f20 [0,4194304] 0 2024-07-03T15:44:03.757 INFO:tasks.workunit.client.0.smithi028.stdout:7/157: chown d9/f1f 577 1 2024-07-03T15:44:03.758 INFO:tasks.workunit.client.0.smithi028.stdout:6/190: dwrite f0 [4194304,4194304] 0 2024-07-03T15:44:03.758 INFO:tasks.workunit.client.0.smithi028.stdout:6/191: fsync f0 0 2024-07-03T15:44:03.761 INFO:tasks.workunit.client.0.smithi028.stdout:9/290: rename d1/d10/d28/f40 to d1/d10/f5d 0 2024-07-03T15:44:03.764 INFO:tasks.workunit.client.0.smithi028.stdout:3/318: mkdir d18/d1a/d64 0 2024-07-03T15:44:03.764 INFO:tasks.workunit.client.0.smithi028.stdout:8/281: mknod d0/d41/d50/d3/dc/c5a 0 2024-07-03T15:44:03.765 INFO:tasks.workunit.client.0.smithi028.stdout:3/319: chown d18/d1a/f28 11798446 1 2024-07-03T15:44:03.765 INFO:tasks.workunit.client.0.smithi028.stdout:8/282: readlink d0/d41/d50/d3/dc/d4d/l53 0 2024-07-03T15:44:03.766 INFO:tasks.workunit.client.0.smithi028.stdout:7/158: write d9/f1f [2343102,81214] 0 2024-07-03T15:44:03.767 INFO:tasks.workunit.client.0.smithi028.stdout:4/248: read d2/d12/f40 [1023820,114653] 0 2024-07-03T15:44:03.767 INFO:tasks.workunit.client.0.smithi028.stdout:4/249: fsync d2/f2b 0 2024-07-03T15:44:03.768 INFO:tasks.workunit.client.0.smithi028.stdout:2/272: unlink da/d1e/c52 0 2024-07-03T15:44:03.768 INFO:tasks.workunit.client.0.smithi028.stdout:2/273: fsync da/d24/d2c/f54 0 2024-07-03T15:44:03.768 INFO:tasks.workunit.client.0.smithi028.stdout:2/274: chown da/f15 1929 1 2024-07-03T15:44:03.769 INFO:tasks.workunit.client.0.smithi028.stdout:0/206: getdents d2/d1c/d32/d36 0 2024-07-03T15:44:03.773 INFO:tasks.workunit.client.0.smithi028.stdout:5/307: dwrite d0/dd/d13/d29/f35 [0,4194304] 0 2024-07-03T15:44:03.776 INFO:tasks.workunit.client.0.smithi028.stdout:3/320: rename d18/d3a/c37 to d18/d1a/d1e/d58/c65 0 2024-07-03T15:44:03.777 INFO:tasks.workunit.client.0.smithi028.stdout:6/192: dread d4/d15/f20 [4194304,4194304] 0 2024-07-03T15:44:03.778 INFO:tasks.workunit.client.0.smithi028.stdout:6/193: dread - d4/d12/f36 zero size 2024-07-03T15:44:03.778 INFO:tasks.workunit.client.0.smithi028.stdout:6/194: write d4/d15/f2a [1455266,97325] 0 2024-07-03T15:44:03.778 INFO:tasks.workunit.client.0.smithi028.stdout:6/195: chown d4/f29 982 1 2024-07-03T15:44:03.779 INFO:tasks.workunit.client.0.smithi028.stdout:3/321: dread d18/d1a/d1e/f57 [0,4194304] 0 2024-07-03T15:44:03.779 INFO:tasks.workunit.client.0.smithi028.stdout:3/322: fsync d18/d1a/d1e/f53 0 2024-07-03T15:44:03.779 INFO:tasks.workunit.client.0.smithi028.stdout:8/283: creat d0/f5b x:0 0 0 2024-07-03T15:44:03.779 INFO:tasks.workunit.client.0.smithi028.stdout:7/159: truncate d9/d1c/f1e 1850191 0 2024-07-03T15:44:03.780 INFO:tasks.workunit.client.0.smithi028.stdout:7/160: write d9/d29/f33 [721877,71017] 0 2024-07-03T15:44:03.780 INFO:tasks.workunit.client.0.smithi028.stdout:4/250: symlink d2/d12/d2d/d2f/d3a/l4d 0 2024-07-03T15:44:03.781 INFO:tasks.workunit.client.0.smithi028.stdout:4/251: chown d2/d12/d19/l29 2131004553 1 2024-07-03T15:44:03.782 INFO:tasks.workunit.client.0.smithi028.stdout:2/275: link da/l59 da/d23/d30/l69 0 2024-07-03T15:44:03.782 INFO:tasks.workunit.client.0.smithi028.stdout:2/276: readlink da/d23/l53 0 2024-07-03T15:44:03.786 INFO:tasks.workunit.client.0.smithi028.stdout:5/308: creat d0/dd/d1e/d3b/d3f/d5a/f65 x:0 0 0 2024-07-03T15:44:03.786 INFO:tasks.workunit.client.0.smithi028.stdout:1/357: dwrite d4/d2c/d3a/d43/f77 [0,4194304] 0 2024-07-03T15:44:03.787 INFO:tasks.workunit.client.1.smithi148.stdout:0/989: dwrite da/d58/dbd/d94/dac/ddd/f4e [0,4194304] 0 2024-07-03T15:44:03.787 INFO:tasks.workunit.client.1.smithi148.stdout:0/990: chown da/d58/dbd/d94/dac/ddd 8994 1 2024-07-03T15:44:03.787 INFO:tasks.workunit.client.0.smithi028.stdout:7/161: dread d9/fe [0,4194304] 0 2024-07-03T15:44:03.788 INFO:tasks.workunit.client.0.smithi028.stdout:9/291: rmdir d1/d17 39 2024-07-03T15:44:03.790 INFO:tasks.workunit.client.0.smithi028.stdout:0/207: rename d2/d3/l2a to d2/d1c/d32/d36/l40 0 2024-07-03T15:44:03.791 INFO:tasks.workunit.client.0.smithi028.stdout:0/208: fsync d2/f3a 0 2024-07-03T15:44:03.791 INFO:tasks.workunit.client.0.smithi028.stdout:0/209: dread - d2/d1c/d32/f34 zero size 2024-07-03T15:44:03.791 INFO:tasks.workunit.client.0.smithi028.stdout:0/210: chown d2/f15 570511 1 2024-07-03T15:44:03.791 INFO:tasks.workunit.client.1.smithi148.stdout:0/991: creat da/d17/d2a/d2f/d93/f14d x:0 0 0 2024-07-03T15:44:03.793 INFO:tasks.workunit.client.1.smithi148.stdout:0/992: dread da/d58/dbd/d13/f18 [0,4194304] 0 2024-07-03T15:44:03.796 INFO:tasks.workunit.client.0.smithi028.stdout:3/323: mkdir d18/d1a/d5e/d66 0 2024-07-03T15:44:03.797 INFO:tasks.workunit.client.0.smithi028.stdout:3/324: fsync d18/d1a/d5e/f5f 0 2024-07-03T15:44:03.797 INFO:tasks.workunit.client.0.smithi028.stdout:6/196: dwrite d4/da/d13/f19 [0,4194304] 0 2024-07-03T15:44:03.797 INFO:tasks.workunit.client.0.smithi028.stdout:4/252: mknod d2/d12/d2c/c4e 0 2024-07-03T15:44:03.799 INFO:tasks.workunit.client.0.smithi028.stdout:2/277: mknod da/d1e/d68/c6a 0 2024-07-03T15:44:03.801 INFO:tasks.workunit.client.0.smithi028.stdout:5/309: symlink d0/dd/df/l66 0 2024-07-03T15:44:03.801 INFO:tasks.workunit.client.0.smithi028.stdout:5/310: fdatasync d0/dd/d13/d57/d26/f42 0 2024-07-03T15:44:03.807 INFO:tasks.workunit.client.0.smithi028.stdout:1/358: unlink d4/d2c/d3a/d43/f88 0 2024-07-03T15:44:03.810 INFO:tasks.workunit.client.0.smithi028.stdout:9/292: mknod d1/d10/d28/d4e/c5e 0 2024-07-03T15:44:03.810 INFO:tasks.workunit.client.0.smithi028.stdout:7/162: dwrite d9/f25 [0,4194304] 0 2024-07-03T15:44:03.812 INFO:tasks.workunit.client.0.smithi028.stdout:8/284: rename d0/dd/c48 to d0/d41/d50/d3/dc/d11/c5c 0 2024-07-03T15:44:03.814 INFO:tasks.workunit.client.0.smithi028.stdout:0/211: creat d2/d1c/f41 x:0 0 0 2024-07-03T15:44:03.816 INFO:tasks.workunit.client.0.smithi028.stdout:6/197: mknod d4/da/d38/c43 0 2024-07-03T15:44:03.816 INFO:tasks.workunit.client.0.smithi028.stdout:6/198: chown d4/d12/d2c/d31 1 1 2024-07-03T15:44:03.818 INFO:tasks.workunit.client.0.smithi028.stdout:4/253: symlink d2/d12/d2d/d2f/d3a/l4f 0 2024-07-03T15:44:03.819 INFO:tasks.workunit.client.0.smithi028.stdout:0/212: dread d2/d1c/f21 [0,4194304] 0 2024-07-03T15:44:03.823 INFO:tasks.workunit.client.0.smithi028.stdout:9/293: mkdir d1/d16/d57/d5f 0 2024-07-03T15:44:03.823 INFO:tasks.workunit.client.0.smithi028.stdout:8/285: write d0/d41/d50/d3/dc/f2b [2435930,57787] 0 2024-07-03T15:44:03.830 INFO:tasks.workunit.client.0.smithi028.stdout:3/325: rename d18/d1a/c2f to d18/d1a/d5e/d60/c67 0 2024-07-03T15:44:03.830 INFO:tasks.workunit.client.0.smithi028.stdout:3/326: truncate d18/d1a/d1e/f48 733326 0 2024-07-03T15:44:03.830 INFO:tasks.workunit.client.0.smithi028.stdout:3/327: write d18/d1a/d5e/f5f [352676,70156] 0 2024-07-03T15:44:03.832 INFO:tasks.workunit.client.0.smithi028.stdout:2/278: getdents da/d24/d2c 0 2024-07-03T15:44:03.838 INFO:tasks.workunit.client.0.smithi028.stdout:1/359: getdents d4/d2c 0 2024-07-03T15:44:03.838 INFO:tasks.workunit.client.0.smithi028.stdout:1/360: chown c3 95285957 1 2024-07-03T15:44:03.838 INFO:tasks.workunit.client.0.smithi028.stdout:1/361: write d4/de/f5a [116539,84909] 0 2024-07-03T15:44:03.838 INFO:tasks.workunit.client.0.smithi028.stdout:1/362: chown d4/d83/f8b 107 1 2024-07-03T15:44:03.839 INFO:tasks.workunit.client.0.smithi028.stdout:9/294: mkdir d1/d16/d57/d60 0 2024-07-03T15:44:03.839 INFO:tasks.workunit.client.0.smithi028.stdout:9/295: chown d1/f21 343410 1 2024-07-03T15:44:03.841 INFO:tasks.workunit.client.0.smithi028.stdout:7/163: getdents d9 0 2024-07-03T15:44:03.841 INFO:tasks.workunit.client.0.smithi028.stdout:7/164: chown d9/d1c/f1e 6130 1 2024-07-03T15:44:03.843 INFO:tasks.workunit.client.0.smithi028.stdout:4/254: rename d2/d12/f40 to d2/d3e/f50 0 2024-07-03T15:44:03.844 INFO:tasks.workunit.client.0.smithi028.stdout:2/279: getdents da/d1e/d56 0 2024-07-03T15:44:03.845 INFO:tasks.workunit.client.0.smithi028.stdout:1/363: truncate d4/d3c/d54/d79/f1b 3671645 0 2024-07-03T15:44:03.847 INFO:tasks.workunit.client.0.smithi028.stdout:9/296: mkdir d1/d10/d28/d45/d61 0 2024-07-03T15:44:03.850 INFO:tasks.workunit.client.0.smithi028.stdout:7/165: rmdir d9 39 2024-07-03T15:44:03.850 INFO:tasks.workunit.client.0.smithi028.stdout:7/166: creat d9/d29/f35 x:0 0 0 2024-07-03T15:44:03.852 INFO:tasks.workunit.client.0.smithi028.stdout:2/280: mknod da/d42/d47/c6b 0 2024-07-03T15:44:03.853 INFO:tasks.workunit.client.0.smithi028.stdout:1/364: mkdir d4/d3c/d54/d79/d8c 0 2024-07-03T15:44:03.855 INFO:tasks.workunit.client.0.smithi028.stdout:3/328: dwrite d18/d1a/d1e/f38 [0,4194304] 0 2024-07-03T15:44:03.855 INFO:tasks.workunit.client.0.smithi028.stdout:9/297: mkdir d1/d16/d62 0 2024-07-03T15:44:03.855 INFO:tasks.workunit.client.0.smithi028.stdout:9/298: getdents d1/d10/d28/d4e/d4f 0 2024-07-03T15:44:03.855 INFO:tasks.workunit.client.0.smithi028.stdout:6/199: dwrite d4/da/d13/f19 [0,4194304] 0 2024-07-03T15:44:03.855 INFO:tasks.workunit.client.0.smithi028.stdout:0/213: dwrite d2/d1c/f21 [0,4194304] 0 2024-07-03T15:44:03.856 INFO:tasks.workunit.client.0.smithi028.stdout:0/214: fdatasync d2/d1c/f2d 0 2024-07-03T15:44:03.856 INFO:tasks.workunit.client.0.smithi028.stdout:6/200: creat d4/da/f44 x:0 0 0 2024-07-03T15:44:03.856 INFO:tasks.workunit.client.0.smithi028.stdout:6/201: fdatasync d4/f33 0 2024-07-03T15:44:03.856 INFO:tasks.workunit.client.0.smithi028.stdout:6/202: chown d4/d12/d2c/c3f 92289 1 2024-07-03T15:44:03.856 INFO:tasks.workunit.client.0.smithi028.stdout:6/203: fsync d4/f11 0 2024-07-03T15:44:03.861 INFO:tasks.workunit.client.0.smithi028.stdout:7/167: truncate d9/d1c/f20 428376 0 2024-07-03T15:44:03.861 INFO:tasks.workunit.client.0.smithi028.stdout:6/204: dread f2 [0,4194304] 0 2024-07-03T15:44:03.862 INFO:tasks.workunit.client.0.smithi028.stdout:4/255: rmdir d2/d12/d19/d45 39 2024-07-03T15:44:03.862 INFO:tasks.workunit.client.0.smithi028.stdout:4/256: write d2/f27 [4377144,73704] 0 2024-07-03T15:44:03.862 INFO:tasks.workunit.client.0.smithi028.stdout:2/281: rename da/d42/f4b to da/d1e/d68/f6c 0 2024-07-03T15:44:03.864 INFO:tasks.workunit.client.0.smithi028.stdout:1/365: creat d4/d3c/d54/f8d x:0 0 0 2024-07-03T15:44:03.864 INFO:tasks.workunit.client.0.smithi028.stdout:1/366: chown d4/d2c/d3a/d5c/d6c/d7d/d4e/d51/f5e 1 1 2024-07-03T15:44:03.867 INFO:tasks.workunit.client.0.smithi028.stdout:3/329: symlink d18/d1a/d5e/d60/l68 0 2024-07-03T15:44:03.873 INFO:tasks.workunit.client.0.smithi028.stdout:0/215: symlink d2/d1c/l42 0 2024-07-03T15:44:03.873 INFO:tasks.workunit.client.0.smithi028.stdout:0/216: readlink d2/d3/l13 0 2024-07-03T15:44:03.876 INFO:tasks.workunit.client.0.smithi028.stdout:7/168: unlink d9/d1c/f2a 0 2024-07-03T15:44:03.877 INFO:tasks.workunit.client.0.smithi028.stdout:6/205: creat d4/f45 x:0 0 0 2024-07-03T15:44:03.878 INFO:tasks.workunit.client.0.smithi028.stdout:4/257: mknod d2/d12/d42/c51 0 2024-07-03T15:44:03.880 INFO:tasks.workunit.client.0.smithi028.stdout:5/311: truncate d0/f2c 9170866 0 2024-07-03T15:44:03.880 INFO:tasks.workunit.client.0.smithi028.stdout:5/312: write d0/dd/d13/f39 [3562241,59249] 0 2024-07-03T15:44:03.881 INFO:tasks.workunit.client.0.smithi028.stdout:2/282: creat da/d42/d47/f6d x:0 0 0 2024-07-03T15:44:03.882 INFO:tasks.workunit.client.0.smithi028.stdout:1/367: creat d4/d3c/d54/d79/f8e x:0 0 0 2024-07-03T15:44:03.882 INFO:tasks.workunit.client.0.smithi028.stdout:1/368: write d4/f26 [1034615,55814] 0 2024-07-03T15:44:03.885 INFO:tasks.workunit.client.0.smithi028.stdout:9/299: dwrite d1/f2b [0,4194304] 0 2024-07-03T15:44:03.886 INFO:tasks.workunit.client.0.smithi028.stdout:3/330: symlink d18/d43/l69 0 2024-07-03T15:44:03.890 INFO:tasks.workunit.client.0.smithi028.stdout:6/206: rename f2 to d4/da/d38/f46 0 2024-07-03T15:44:03.896 INFO:tasks.workunit.client.0.smithi028.stdout:3/331: fsync d18/d1a/d5e/f5f 0 2024-07-03T15:44:03.896 INFO:tasks.workunit.client.0.smithi028.stdout:3/332: write d18/f51 [665273,36121] 0 2024-07-03T15:44:03.896 INFO:tasks.workunit.client.0.smithi028.stdout:3/333: chown d18/d1a/d1e/f57 379836 1 2024-07-03T15:44:03.896 INFO:tasks.workunit.client.0.smithi028.stdout:6/207: dread d4/f18 [4194304,4194304] 0 2024-07-03T15:44:03.897 INFO:tasks.workunit.client.0.smithi028.stdout:4/258: creat d2/f52 x:0 0 0 2024-07-03T15:44:03.897 INFO:tasks.workunit.client.0.smithi028.stdout:1/369: fdatasync d4/f26 0 2024-07-03T15:44:03.898 INFO:tasks.workunit.client.0.smithi028.stdout:1/370: truncate d4/d2c/d3a/d5c/d6c/d7d/d59/f6a 192542 0 2024-07-03T15:44:03.898 INFO:tasks.workunit.client.0.smithi028.stdout:1/371: getdents d4/d3c/d54/d84 0 2024-07-03T15:44:03.899 INFO:tasks.workunit.client.0.smithi028.stdout:1/372: write d4/d2c/f75 [386614,923] 0 2024-07-03T15:44:03.899 INFO:tasks.workunit.client.0.smithi028.stdout:1/373: fdatasync d4/de/ff 0 2024-07-03T15:44:03.899 INFO:tasks.workunit.client.0.smithi028.stdout:1/374: write d4/fd [2555893,126469] 0 2024-07-03T15:44:03.900 INFO:tasks.workunit.client.0.smithi028.stdout:1/375: creat d4/d2c/d3a/f8f x:0 0 0 2024-07-03T15:44:03.902 INFO:tasks.workunit.client.0.smithi028.stdout:7/169: dwrite d9/d1c/f1e [0,4194304] 0 2024-07-03T15:44:03.902 INFO:tasks.workunit.client.0.smithi028.stdout:7/170: chown d9/d1c/f1e 10 1 2024-07-03T15:44:03.902 INFO:tasks.workunit.client.0.smithi028.stdout:3/334: truncate d18/d1a/f24 2507365 0 2024-07-03T15:44:03.904 INFO:tasks.workunit.client.0.smithi028.stdout:6/208: mkdir d4/da/d47 0 2024-07-03T15:44:03.908 INFO:tasks.workunit.client.0.smithi028.stdout:6/209: dread d4/d15/f2a [0,4194304] 0 2024-07-03T15:44:03.908 INFO:tasks.workunit.client.0.smithi028.stdout:6/210: readlink d4/d12/d2c/l35 0 2024-07-03T15:44:03.908 INFO:tasks.workunit.client.0.smithi028.stdout:5/313: dwrite d0/dd/d13/d57/d26/d41/f44 [0,4194304] 0 2024-07-03T15:44:03.910 INFO:tasks.workunit.client.0.smithi028.stdout:9/300: dwrite d1/dd/f20 [0,4194304] 0 2024-07-03T15:44:03.912 INFO:tasks.workunit.client.0.smithi028.stdout:1/376: truncate d4/de/f10 3050001 0 2024-07-03T15:44:03.912 INFO:tasks.workunit.client.0.smithi028.stdout:1/377: write d4/de/f1f [3570863,17708] 0 2024-07-03T15:44:03.918 INFO:tasks.workunit.client.1.smithi148.stdout:0/993: sync 2024-07-03T15:44:03.918 INFO:tasks.workunit.client.0.smithi028.stdout:3/335: symlink d18/d1a/d1e/l6a 0 2024-07-03T15:44:03.918 INFO:tasks.workunit.client.0.smithi028.stdout:3/336: chown d18/d3a/f3f 1010 1 2024-07-03T15:44:03.920 INFO:tasks.workunit.client.0.smithi028.stdout:4/259: mknod d2/d12/d19/c53 0 2024-07-03T15:44:03.920 INFO:tasks.workunit.client.0.smithi028.stdout:4/260: chown d2/l1d 6635484 1 2024-07-03T15:44:03.920 INFO:tasks.workunit.client.0.smithi028.stdout:4/261: chown d2/d12/d2d/d2f/l46 15740551 1 2024-07-03T15:44:03.920 INFO:tasks.workunit.client.0.smithi028.stdout:4/262: fdatasync d2/d12/d19/f9 0 2024-07-03T15:44:03.922 INFO:tasks.workunit.client.1.smithi148.stdout:0/994: symlink da/d58/dbd/d94/dac/ddd/l14e 0 2024-07-03T15:44:03.923 INFO:tasks.workunit.client.0.smithi028.stdout:6/211: getdents d4/d15 0 2024-07-03T15:44:03.927 INFO:tasks.workunit.client.1.smithi148.stdout:0/995: creat da/d58/dbd/d13/d1d/d3e/f14f x:0 0 0 2024-07-03T15:44:03.927 INFO:tasks.workunit.client.1.smithi148.stdout:0/996: getdents da 0 2024-07-03T15:44:03.928 INFO:tasks.workunit.client.0.smithi028.stdout:0/217: getdents d2/d1c 0 2024-07-03T15:44:03.928 INFO:tasks.workunit.client.0.smithi028.stdout:0/218: truncate d2/d1c/f41 462729 0 2024-07-03T15:44:03.928 INFO:tasks.workunit.client.0.smithi028.stdout:0/219: readlink d2/d3/l13 0 2024-07-03T15:44:03.931 INFO:tasks.workunit.client.1.smithi148.stdout:0/997: creat da/d17/d2a/d2f/f150 x:0 0 0 2024-07-03T15:44:03.931 INFO:tasks.workunit.client.0.smithi028.stdout:1/378: symlink d4/de/d1c/d81/l90 0 2024-07-03T15:44:03.932 INFO:tasks.workunit.client.0.smithi028.stdout:7/171: rename d9/cc to d9/d1c/c36 0 2024-07-03T15:44:03.933 INFO:tasks.workunit.client.0.smithi028.stdout:4/263: symlink d2/d12/d42/l54 0 2024-07-03T15:44:03.935 INFO:tasks.workunit.client.1.smithi148.stdout:0/998: unlink da/d17/d125/c149 0 2024-07-03T15:44:03.936 INFO:tasks.workunit.client.1.smithi148.stdout:0/999: chown da/d58/dbd/d94/dac/l131 357773 1 2024-07-03T15:44:03.936 INFO:tasks.workunit.client.0.smithi028.stdout:2/283: rmdir da 39 2024-07-03T15:44:03.936 INFO:tasks.workunit.client.0.smithi028.stdout:2/284: dread - da/d23/d30/f67 zero size 2024-07-03T15:44:03.936 INFO:tasks.workunit.client.0.smithi028.stdout:5/314: dwrite d0/dd/d13/d29/f35 [4194304,4194304] 0 2024-07-03T15:44:03.936 INFO:tasks.workunit.client.0.smithi028.stdout:5/315: rename d0/dd/d13 to d0/dd/d13/d57/d30/d67 22 2024-07-03T15:44:03.947 INFO:tasks.workunit.client.1.smithi148.stderr:+ rm -rf -- ./tmp.T0EhYNVc6z 2024-07-03T15:44:03.949 INFO:tasks.workunit.client.0.smithi028.stdout:1/379: mknod d4/d2c/d3a/d5c/d6c/d7d/c91 0 2024-07-03T15:44:03.949 INFO:tasks.workunit.client.0.smithi028.stdout:1/380: write d4/d3c/d54/f8d [3157,108548] 0 2024-07-03T15:44:03.950 INFO:tasks.workunit.client.0.smithi028.stdout:4/264: mknod d2/d12/d2d/c55 0 2024-07-03T15:44:03.950 INFO:tasks.workunit.client.0.smithi028.stdout:4/265: truncate d2/d12/d3b/f48 628680 0 2024-07-03T15:44:03.952 INFO:tasks.workunit.client.0.smithi028.stdout:2/285: getdents da/d42/d47 0 2024-07-03T15:44:03.952 INFO:tasks.workunit.client.0.smithi028.stdout:2/286: fsync da/ff 0 2024-07-03T15:44:03.955 INFO:tasks.workunit.client.0.smithi028.stdout:0/220: mknod d2/d3/c43 0 2024-07-03T15:44:03.956 INFO:tasks.workunit.client.0.smithi028.stdout:1/381: creat d4/d2c/f92 x:0 0 0 2024-07-03T15:44:03.957 INFO:tasks.workunit.client.0.smithi028.stdout:1/382: readlink d4/d2c/d3a/d5c/d6c/d7d/d59/l2f 0 2024-07-03T15:44:03.959 INFO:tasks.workunit.client.0.smithi028.stdout:4/266: creat d2/d12/d19/f56 x:0 0 0 2024-07-03T15:44:03.960 INFO:tasks.workunit.client.0.smithi028.stdout:2/287: mknod da/d1e/d68/c6e 0 2024-07-03T15:44:03.964 INFO:tasks.workunit.client.0.smithi028.stdout:4/267: dread d2/f20 [0,4194304] 0 2024-07-03T15:44:03.964 INFO:tasks.workunit.client.0.smithi028.stdout:0/221: getdents d2 0 2024-07-03T15:44:03.964 INFO:tasks.workunit.client.0.smithi028.stdout:2/288: mkdir da/d23/d6f 0 2024-07-03T15:44:03.965 INFO:tasks.workunit.client.0.smithi028.stdout:0/222: symlink d2/d1c/d32/d36/l44 0 2024-07-03T15:44:03.965 INFO:tasks.workunit.client.0.smithi028.stdout:6/212: dwrite d4/f10 [0,4194304] 0 2024-07-03T15:44:03.965 INFO:tasks.workunit.client.0.smithi028.stdout:6/213: truncate d4/f11 13242179 0 2024-07-03T15:44:03.967 INFO:tasks.workunit.client.0.smithi028.stdout:7/172: dwrite d9/d1c/f1e [0,4194304] 0 2024-07-03T15:44:03.967 INFO:tasks.workunit.client.0.smithi028.stdout:9/301: dwrite d1/f2 [0,4194304] 0 2024-07-03T15:44:03.974 INFO:tasks.workunit.client.0.smithi028.stdout:6/214: mknod d4/d15/c48 0 2024-07-03T15:44:03.974 INFO:tasks.workunit.client.0.smithi028.stdout:6/215: fsync d4/da/d13/f19 0 2024-07-03T15:44:03.974 INFO:tasks.workunit.client.0.smithi028.stdout:6/216: chown d4/da/c37 303 1 2024-07-03T15:44:03.974 INFO:tasks.workunit.client.0.smithi028.stdout:6/217: fsync d4/da/d1c/f1e 0 2024-07-03T15:44:03.974 INFO:tasks.workunit.client.0.smithi028.stdout:5/316: dwrite d0/dd/df/f16 [0,4194304] 0 2024-07-03T15:44:03.974 INFO:tasks.workunit.client.0.smithi028.stdout:5/317: dread - d0/dd/d13/d57/d26/d41/f54 zero size 2024-07-03T15:44:03.974 INFO:tasks.workunit.client.0.smithi028.stdout:1/383: dread d4/d2c/d3a/d5c/d6c/d7d/f5d [0,4194304] 0 2024-07-03T15:44:03.980 INFO:tasks.workunit.client.0.smithi028.stdout:4/268: read d2/f1b [2882676,121339] 0 2024-07-03T15:44:03.981 INFO:tasks.workunit.client.0.smithi028.stdout:7/173: mknod d9/d1c/c37 0 2024-07-03T15:44:03.998 INFO:tasks.workunit.client.0.smithi028.stdout:1/384: dread d4/de/f85 [0,4194304] 0 2024-07-03T15:44:03.999 INFO:tasks.workunit.client.0.smithi028.stdout:1/385: write d4/de/f1f [911914,119574] 0 2024-07-03T15:44:04.004 INFO:tasks.workunit.client.0.smithi028.stdout:4/269: mknod d2/d12/d2d/d2f/c57 0 2024-07-03T15:44:04.007 INFO:tasks.workunit.client.0.smithi028.stdout:7/174: link d9/d1c/c36 d9/d29/c38 0 2024-07-03T15:44:04.007 INFO:tasks.workunit.client.0.smithi028.stdout:7/175: readlink d9/d29/l2b 0 2024-07-03T15:44:04.007 INFO:tasks.workunit.client.0.smithi028.stdout:9/302: dwrite d1/f31 [0,4194304] 0 2024-07-03T15:44:04.007 INFO:tasks.workunit.client.0.smithi028.stdout:9/303: creat d1/d16/f63 x:0 0 0 2024-07-03T15:44:04.007 INFO:tasks.workunit.client.0.smithi028.stdout:9/304: chown d1/d17 6502 1 2024-07-03T15:44:04.007 INFO:tasks.workunit.client.0.smithi028.stdout:2/289: dwrite da/d42/d47/f6d [0,4194304] 0 2024-07-03T15:44:04.008 INFO:tasks.workunit.client.0.smithi028.stdout:6/218: dwrite d4/d12/f36 [0,4194304] 0 2024-07-03T15:44:04.008 INFO:tasks.workunit.client.0.smithi028.stdout:1/386: rmdir d4/de/d1c/d81 39 2024-07-03T15:44:04.009 INFO:tasks.workunit.client.0.smithi028.stdout:5/318: dwrite d0/f2c [0,4194304] 0 2024-07-03T15:44:04.009 INFO:tasks.workunit.client.0.smithi028.stdout:5/319: chown d0/dd/d1e/d3b/d3f/c51 136541 1 2024-07-03T15:44:04.010 INFO:tasks.workunit.client.0.smithi028.stdout:4/270: mkdir d2/d12/d2d/d58 0 2024-07-03T15:44:04.010 INFO:tasks.workunit.client.0.smithi028.stdout:4/271: creat d2/d12/d3b/f59 x:0 0 0 2024-07-03T15:44:04.010 INFO:tasks.workunit.client.0.smithi028.stdout:4/272: write d2/f1b [2092711,12976] 0 2024-07-03T15:44:04.013 INFO:tasks.workunit.client.0.smithi028.stdout:4/273: dread d2/d12/d19/f34 [0,4194304] 0 2024-07-03T15:44:04.016 INFO:tasks.workunit.client.0.smithi028.stdout:7/176: mknod d9/d29/c39 0 2024-07-03T15:44:04.019 INFO:tasks.workunit.client.0.smithi028.stdout:9/305: mkdir d1/d16/d57/d5f/d64 0 2024-07-03T15:44:04.020 INFO:tasks.workunit.client.0.smithi028.stdout:9/306: fsync d1/d16/f2f 0 2024-07-03T15:44:04.020 INFO:tasks.workunit.client.0.smithi028.stdout:9/307: creat d1/d4a/f65 x:0 0 0 2024-07-03T15:44:04.020 INFO:tasks.workunit.client.0.smithi028.stdout:9/308: stat d1/d10/l47 0 2024-07-03T15:44:04.020 INFO:tasks.workunit.client.0.smithi028.stdout:9/309: write d1/f4b [947995,117596] 0 2024-07-03T15:44:04.020 INFO:tasks.workunit.client.0.smithi028.stdout:9/310: chown d1/f2b 104 1 2024-07-03T15:44:04.022 INFO:tasks.workunit.client.0.smithi028.stdout:2/290: truncate da/d24/d2c/f3d 547978 0 2024-07-03T15:44:04.024 INFO:tasks.workunit.client.0.smithi028.stdout:2/291: dread da/f1d [0,4194304] 0 2024-07-03T15:44:04.026 INFO:tasks.workunit.client.0.smithi028.stdout:6/219: rename d4 to d4/da/d1c/d49 22 2024-07-03T15:44:04.032 INFO:tasks.workunit.client.0.smithi028.stdout:1/387: dwrite d4/d3c/f7b [0,4194304] 0 2024-07-03T15:44:04.040 INFO:tasks.workunit.client.0.smithi028.stdout:2/292: rmdir da 39 2024-07-03T15:44:04.044 INFO:tasks.workunit.client.0.smithi028.stdout:5/320: dwrite d0/dd/d1e/d3b/f5e [4194304,4194304] 0 2024-07-03T15:44:04.045 INFO:tasks.workunit.client.0.smithi028.stdout:1/388: mknod d4/d3c/c93 0 2024-07-03T15:44:04.045 INFO:tasks.workunit.client.0.smithi028.stdout:1/389: fsync d4/d3c/d54/d79/f3e 0 2024-07-03T15:44:04.045 INFO:tasks.workunit.client.0.smithi028.stdout:1/390: write d4/d2c/f78 [833609,129387] 0 2024-07-03T15:44:04.045 INFO:tasks.workunit.client.0.smithi028.stdout:6/220: dwrite d4/d15/f2a [0,4194304] 0 2024-07-03T15:44:04.045 INFO:tasks.workunit.client.0.smithi028.stdout:6/221: creat d4/da/d38/f4a x:0 0 0 2024-07-03T15:44:04.046 INFO:tasks.workunit.client.0.smithi028.stdout:7/177: dwrite d9/f34 [0,4194304] 0 2024-07-03T15:44:04.048 INFO:tasks.workunit.client.0.smithi028.stdout:7/178: dread d9/d29/f33 [0,4194304] 0 2024-07-03T15:44:04.052 INFO:tasks.workunit.client.0.smithi028.stdout:5/321: creat d0/dd/d13/d29/f68 x:0 0 0 2024-07-03T15:44:04.058 INFO:tasks.workunit.client.0.smithi028.stdout:6/222: symlink d4/d12/d2c/d31/l4b 0 2024-07-03T15:44:04.058 INFO:tasks.workunit.client.0.smithi028.stdout:6/223: fsync d4/da/d1c/f1e 0 2024-07-03T15:44:04.058 INFO:tasks.workunit.client.0.smithi028.stdout:6/224: chown d4/da/f44 1 1 2024-07-03T15:44:04.059 INFO:tasks.workunit.client.0.smithi028.stdout:1/391: dwrite d4/de/d1c/f3d [0,4194304] 0 2024-07-03T15:44:04.064 INFO:tasks.workunit.client.0.smithi028.stdout:2/293: unlink da/d23/l33 0 2024-07-03T15:44:04.069 INFO:tasks.workunit.client.0.smithi028.stdout:4/274: rename d2/d12/d19 to d2/d5a 0 2024-07-03T15:44:04.071 INFO:tasks.workunit.client.0.smithi028.stdout:5/322: symlink d0/dd/d13/d57/d26/l69 0 2024-07-03T15:44:04.071 INFO:tasks.workunit.client.0.smithi028.stdout:5/323: chown d0/lb 643 1 2024-07-03T15:44:04.071 INFO:tasks.workunit.client.0.smithi028.stdout:5/324: chown d0/dd/df/l66 518980 1 2024-07-03T15:44:04.071 INFO:tasks.workunit.client.0.smithi028.stdout:5/325: chown d0/dd/d1e 12 1 2024-07-03T15:44:04.078 INFO:tasks.workunit.client.0.smithi028.stdout:2/294: symlink da/d23/d30/l70 0 2024-07-03T15:44:04.082 INFO:tasks.workunit.client.0.smithi028.stdout:6/225: dwrite d4/d15/f20 [0,4194304] 0 2024-07-03T15:44:04.082 INFO:tasks.workunit.client.0.smithi028.stdout:5/326: rename d0/dd/d13/d57/d26/d41/f54 to d0/dd/d4b/f6a 0 2024-07-03T15:44:04.086 INFO:tasks.workunit.client.0.smithi028.stdout:4/275: mkdir d2/d12/d5b 0 2024-07-03T15:44:04.087 INFO:tasks.workunit.client.0.smithi028.stdout:1/392: dwrite d4/d2c/d3a/d5c/d6c/d7d/d59/f2a [0,4194304] 0 2024-07-03T15:44:04.087 INFO:tasks.workunit.client.0.smithi028.stdout:1/393: chown d4/d3c/d54/d79/d16/d57 3468 1 2024-07-03T15:44:04.087 INFO:tasks.workunit.client.0.smithi028.stdout:1/394: write d4/d3c/d54/d79/f8e [334283,50863] 0 2024-07-03T15:44:04.088 INFO:tasks.workunit.client.0.smithi028.stdout:6/226: creat d4/d15/f4c x:0 0 0 2024-07-03T15:44:04.090 INFO:tasks.workunit.client.0.smithi028.stdout:2/295: rename da/d42/d47/c5b to da/d23/d30/c71 0 2024-07-03T15:44:04.093 INFO:tasks.workunit.client.0.smithi028.stdout:4/276: write d2/d5a/f9 [2613111,24648] 0 2024-07-03T15:44:04.093 INFO:tasks.workunit.client.0.smithi028.stdout:4/277: fdatasync d2/d12/d3b/f59 0 2024-07-03T15:44:04.093 INFO:tasks.workunit.client.0.smithi028.stdout:4/278: readlink d2/l16 0 2024-07-03T15:44:04.093 INFO:tasks.workunit.client.0.smithi028.stdout:4/279: stat d2/d12/f17 0 2024-07-03T15:44:04.093 INFO:tasks.workunit.client.0.smithi028.stdout:1/395: dread d4/d3c/d54/d79/f8e [0,4194304] 0 2024-07-03T15:44:04.099 INFO:tasks.workunit.client.0.smithi028.stdout:5/327: rename d0/dd/d13/d57/d26/d41/c5d to d0/dd/df/c6b 0 2024-07-03T15:44:04.099 INFO:tasks.workunit.client.0.smithi028.stdout:5/328: write d0/dd/d13/d29/f68 [728354,63155] 0 2024-07-03T15:44:04.103 INFO:tasks.workunit.client.0.smithi028.stdout:4/280: getdents d2/d12/d2d/d2f/d3a 0 2024-07-03T15:44:04.103 INFO:tasks.workunit.client.0.smithi028.stdout:4/281: readlink d2/d12/d2d/l31 0 2024-07-03T15:44:04.105 INFO:tasks.workunit.client.0.smithi028.stdout:1/396: mkdir d4/d2c/d3a/d5c/d6c/d7d/d59/d94 0 2024-07-03T15:44:04.105 INFO:tasks.workunit.client.0.smithi028.stdout:1/397: fdatasync d4/f26 0 2024-07-03T15:44:04.107 INFO:tasks.workunit.client.0.smithi028.stdout:6/227: rmdir d4/da/d38 39 2024-07-03T15:44:04.107 INFO:tasks.workunit.client.0.smithi028.stdout:6/228: chown d4/l24 1926 1 2024-07-03T15:44:04.107 INFO:tasks.workunit.client.0.smithi028.stdout:4/282: dread d2/f13 [0,4194304] 0 2024-07-03T15:44:04.112 INFO:tasks.workunit.client.0.smithi028.stdout:1/398: dread d4/d2c/f30 [0,4194304] 0 2024-07-03T15:44:04.114 INFO:tasks.workunit.client.0.smithi028.stdout:5/329: link d0/dd/d13/d57/f1f d0/dd/d4b/f6c 0 2024-07-03T15:44:04.114 INFO:tasks.workunit.client.0.smithi028.stdout:5/330: readlink d0/dd/df/l23 0 2024-07-03T15:44:04.116 INFO:tasks.workunit.client.0.smithi028.stdout:2/296: symlink da/d1e/l72 0 2024-07-03T15:44:04.116 INFO:tasks.workunit.client.0.smithi028.stdout:2/297: chown da/d24/d2c/f32 1938 1 2024-07-03T15:44:04.116 INFO:tasks.workunit.client.0.smithi028.stdout:2/298: dread - da/d23/d30/f67 zero size 2024-07-03T15:44:04.116 INFO:tasks.workunit.client.0.smithi028.stdout:2/299: write da/ff [1004254,131064] 0 2024-07-03T15:44:04.117 INFO:tasks.workunit.client.0.smithi028.stdout:2/300: dread - da/d1e/f58 zero size 2024-07-03T15:44:04.123 INFO:tasks.workunit.client.0.smithi028.stdout:1/399: truncate d4/d3c/d54/d79/d16/d57/d66/f67 2571362 0 2024-07-03T15:44:04.124 INFO:tasks.workunit.client.0.smithi028.stdout:5/331: dread d0/dd/df/f16 [0,4194304] 0 2024-07-03T15:44:04.124 INFO:tasks.workunit.client.0.smithi028.stdout:5/332: write d0/dd/d13/d57/d26/d41/f5f [866399,104692] 0 2024-07-03T15:44:04.125 INFO:tasks.workunit.client.0.smithi028.stdout:6/229: dwrite d4/f33 [0,4194304] 0 2024-07-03T15:44:04.128 INFO:tasks.workunit.client.0.smithi028.stdout:2/301: mknod da/d1e/d68/c73 0 2024-07-03T15:44:04.131 INFO:tasks.workunit.client.0.smithi028.stdout:1/400: mknod d4/d3c/d54/d79/d16/d57/d66/c95 0 2024-07-03T15:44:04.132 INFO:tasks.workunit.client.0.smithi028.stdout:1/401: write d4/d2c/f75 [928949,108895] 0 2024-07-03T15:44:04.136 INFO:tasks.workunit.client.0.smithi028.stdout:5/333: dwrite d0/dd/df/f16 [4194304,4194304] 0 2024-07-03T15:44:04.140 INFO:tasks.workunit.client.0.smithi028.stdout:2/302: mkdir da/d74 0 2024-07-03T15:44:04.145 INFO:tasks.workunit.client.0.smithi028.stdout:6/230: rename d4/f18 to d4/da/d1c/f4d 0 2024-07-03T15:44:04.145 INFO:tasks.workunit.client.0.smithi028.stdout:6/231: chown d4/d15/c48 193183341 1 2024-07-03T15:44:04.145 INFO:tasks.workunit.client.0.smithi028.stdout:5/334: write d0/dd/d13/d57/d26/d41/f44 [812727,3808] 0 2024-07-03T15:44:04.146 INFO:tasks.workunit.client.0.smithi028.stdout:2/303: creat da/d23/f75 x:0 0 0 2024-07-03T15:44:04.149 INFO:tasks.workunit.client.0.smithi028.stdout:6/232: dread d4/da/d38/f46 [0,4194304] 0 2024-07-03T15:44:04.149 INFO:tasks.workunit.client.0.smithi028.stdout:6/233: creat d4/da/f4e x:0 0 0 2024-07-03T15:44:04.160 INFO:tasks.workunit.client.0.smithi028.stdout:1/402: rename d4/d2c/f30 to d4/d2c/d3a/d5c/d6c/d7d/d59/d94/f96 0 2024-07-03T15:44:04.160 INFO:tasks.workunit.client.0.smithi028.stdout:1/403: chown d4/d2c/l45 521651 1 2024-07-03T15:44:04.160 INFO:tasks.workunit.client.0.smithi028.stdout:1/404: write d4/d2c/d3a/d5c/d6c/d7d/d4e/d51/f5f [778198,76475] 0 2024-07-03T15:44:04.165 INFO:tasks.workunit.client.0.smithi028.stdout:5/335: link d0/dd/d13/d57/d30/c32 d0/dd/d13/c6d 0 2024-07-03T15:44:04.166 INFO:tasks.workunit.client.0.smithi028.stdout:2/304: creat da/f76 x:0 0 0 2024-07-03T15:44:04.171 INFO:tasks.workunit.client.0.smithi028.stdout:6/234: dwrite d4/f11 [8388608,4194304] 0 2024-07-03T15:44:04.172 INFO:tasks.workunit.client.0.smithi028.stdout:1/405: rename d4/de/c21 to d4/de/d1c/c97 0 2024-07-03T15:44:04.174 INFO:tasks.workunit.client.0.smithi028.stdout:5/336: mknod d0/dd/d13/c6e 0 2024-07-03T15:44:04.178 INFO:tasks.workunit.client.0.smithi028.stdout:6/235: unlink d4/l24 0 2024-07-03T15:44:04.178 INFO:tasks.workunit.client.0.smithi028.stdout:1/406: truncate d4/d2c/d3a/d5c/d6c/d7d/d59/d94/f96 291310 0 2024-07-03T15:44:04.178 INFO:tasks.workunit.client.0.smithi028.stdout:1/407: readlink d4/d2c/l45 0 2024-07-03T15:44:04.178 INFO:tasks.workunit.client.0.smithi028.stdout:1/408: chown d4/de/f10 35258967 1 2024-07-03T15:44:04.180 INFO:tasks.workunit.client.0.smithi028.stdout:1/409: write d4/d2c/d3a/d5c/d6c/d7d/d4e/d51/f5f [620422,75765] 0 2024-07-03T15:44:04.180 INFO:tasks.workunit.client.0.smithi028.stdout:1/410: write d4/d2c/d3a/d5c/d6c/d7d/d59/f6a [595409,31309] 0 2024-07-03T15:44:04.182 INFO:tasks.workunit.client.0.smithi028.stdout:1/411: creat d4/d2c/d3a/d5c/f98 x:0 0 0 2024-07-03T15:44:04.185 INFO:tasks.workunit.client.0.smithi028.stdout:6/236: dread f0 [0,4194304] 0 2024-07-03T15:44:04.186 INFO:tasks.workunit.client.0.smithi028.stdout:1/412: truncate d4/de/f31 425954 0 2024-07-03T15:44:04.189 INFO:tasks.workunit.client.0.smithi028.stdout:1/413: unlink d4/d2c/d3a/d5c/d6c/d7d/c91 0 2024-07-03T15:44:04.191 INFO:tasks.workunit.client.0.smithi028.stdout:1/414: symlink d4/de/d1c/d44/l99 0 2024-07-03T15:44:04.212 INFO:tasks.workunit.client.0.smithi028.stdout:6/237: dwrite d4/f29 [0,4194304] 0 2024-07-03T15:44:04.212 INFO:tasks.workunit.client.0.smithi028.stdout:6/238: creat d4/d12/f4f x:0 0 0 2024-07-03T15:44:04.213 INFO:tasks.workunit.client.0.smithi028.stdout:1/415: dwrite d4/de/f1f [0,4194304] 0 2024-07-03T15:44:04.213 INFO:tasks.workunit.client.0.smithi028.stdout:1/416: write d4/d2c/d3a/d5c/d6c/d7d/f5d [4978078,32087] 0 2024-07-03T15:44:04.216 INFO:tasks.workunit.client.0.smithi028.stdout:6/239: dread d4/da/d38/f46 [0,4194304] 0 2024-07-03T15:44:04.220 INFO:tasks.workunit.client.0.smithi028.stdout:6/240: link d4/f45 d4/da/d1c/f50 0 2024-07-03T15:44:04.222 INFO:tasks.workunit.client.0.smithi028.stdout:6/241: chown d4/da/d1c/l27 143387 1 2024-07-03T15:44:04.222 INFO:tasks.workunit.client.0.smithi028.stdout:6/242: write d4/d12/f36 [4386469,63881] 0 2024-07-03T15:44:04.222 INFO:tasks.workunit.client.0.smithi028.stdout:6/243: dread - d4/d12/f4f zero size 2024-07-03T15:44:04.229 INFO:tasks.workunit.client.0.smithi028.stdout:6/244: dread d4/d15/f2a [0,4194304] 0 2024-07-03T15:44:04.233 INFO:tasks.workunit.client.0.smithi028.stdout:1/417: dwrite d4/de/f5a [0,4194304] 0 2024-07-03T15:44:04.237 INFO:tasks.workunit.client.0.smithi028.stdout:1/418: mknod d4/de/d1c/d81/c9a 0 2024-07-03T15:44:04.237 INFO:tasks.workunit.client.0.smithi028.stdout:1/419: write d4/de/f85 [4632927,44136] 0 2024-07-03T15:44:04.239 INFO:tasks.workunit.client.0.smithi028.stdout:1/420: creat d4/d3c/d54/d84/f9b x:0 0 0 2024-07-03T15:44:04.239 INFO:tasks.workunit.client.0.smithi028.stdout:1/421: write d4/d2c/f78 [1404682,68123] 0 2024-07-03T15:44:04.243 INFO:tasks.workunit.client.0.smithi028.stdout:1/422: symlink d4/d3c/d54/d79/l9c 0 2024-07-03T15:44:04.246 INFO:tasks.workunit.client.0.smithi028.stdout:6/245: dwrite d4/d12/f4f [0,4194304] 0 2024-07-03T15:44:04.246 INFO:tasks.workunit.client.0.smithi028.stdout:6/246: read d4/da/d38/f46 [1700258,18018] 0 2024-07-03T15:44:04.246 INFO:tasks.workunit.client.0.smithi028.stdout:1/423: symlink d4/de/d1c/d81/l9d 0 2024-07-03T15:44:04.246 INFO:tasks.workunit.client.0.smithi028.stdout:1/424: creat d4/d3c/f9e x:0 0 0 2024-07-03T15:44:04.246 INFO:tasks.workunit.client.0.smithi028.stdout:1/425: chown d4/de/d1c/d81/c9a 0 1 2024-07-03T15:44:04.317 INFO:tasks.workunit.client.0.smithi028.stdout:6/247: creat d4/da/d47/f51 x:0 0 0 2024-07-03T15:44:04.317 INFO:tasks.workunit.client.0.smithi028.stdout:6/248: fsync d4/d12/f21 0 2024-07-03T15:44:04.321 INFO:tasks.workunit.client.0.smithi028.stdout:1/426: link d4/d3c/f7b d4/d3c/d54/d79/d16/d7c/f9f 0 2024-07-03T15:44:04.338 INFO:tasks.workunit.client.0.smithi028.stdout:6/249: mknod d4/d12/d2c/c52 0 2024-07-03T15:44:04.345 INFO:tasks.workunit.client.0.smithi028.stdout:1/427: rename d4/d3c/d54/d79/l2b to d4/d3c/d54/d79/d16/d57/d66/la0 0 2024-07-03T15:44:04.361 INFO:tasks.workunit.client.0.smithi028.stdout:6/250: dwrite d4/da/d38/f40 [0,4194304] 0 2024-07-03T15:44:04.361 INFO:tasks.workunit.client.0.smithi028.stdout:1/428: dwrite d4/d2c/f75 [0,4194304] 0 2024-07-03T15:44:04.370 INFO:tasks.workunit.client.0.smithi028.stdout:1/429: mknod d4/de/d1c/d81/ca1 0 2024-07-03T15:44:04.373 INFO:tasks.workunit.client.0.smithi028.stdout:1/430: link d4/f19 d4/d2c/d3a/d5c/d6c/d7d/d4e/d51/fa2 0 2024-07-03T15:44:04.377 INFO:tasks.workunit.client.0.smithi028.stdout:6/251: dwrite d4/d15/f2a [0,4194304] 0 2024-07-03T15:44:04.379 INFO:tasks.workunit.client.0.smithi028.stdout:6/252: mknod d4/d15/c53 0 2024-07-03T15:44:04.379 INFO:tasks.workunit.client.0.smithi028.stdout:6/253: chown d4/d15/l1d 1 1 2024-07-03T15:44:04.385 INFO:tasks.workunit.client.0.smithi028.stdout:1/431: write d4/de/f5a [3777204,97852] 0 2024-07-03T15:44:04.386 INFO:tasks.workunit.client.0.smithi028.stdout:6/254: dread d4/da/d38/f40 [0,4194304] 0 2024-07-03T15:44:04.388 INFO:tasks.workunit.client.0.smithi028.stdout:1/432: dread d4/d3c/d54/d79/d16/f27 [0,4194304] 0 2024-07-03T15:44:04.389 INFO:tasks.workunit.client.0.smithi028.stdout:6/255: creat d4/da/d1c/f54 x:0 0 0 2024-07-03T15:44:04.390 INFO:tasks.workunit.client.0.smithi028.stdout:1/433: link d4/d2c/d3a/d5c/d6c/d7d/d59/f82 d4/d3c/fa3 0 2024-07-03T15:44:04.390 INFO:tasks.workunit.client.0.smithi028.stdout:1/434: chown d4/de/f85 25001 1 2024-07-03T15:44:04.391 INFO:tasks.workunit.client.0.smithi028.stdout:6/256: mknod d4/d12/c55 0 2024-07-03T15:44:04.393 INFO:tasks.workunit.client.0.smithi028.stdout:6/257: mkdir d4/da/d56 0 2024-07-03T15:44:04.395 INFO:tasks.workunit.client.0.smithi028.stdout:1/435: getdents d4/de/d1c 0 2024-07-03T15:44:04.398 INFO:tasks.workunit.client.0.smithi028.stdout:6/258: link d4/d12/d2c/d31/l4b d4/d12/l57 0 2024-07-03T15:44:04.399 INFO:tasks.workunit.client.0.smithi028.stdout:1/436: mkdir d4/da4 0 2024-07-03T15:44:04.401 INFO:tasks.workunit.client.0.smithi028.stdout:1/437: mknod d4/da4/ca5 0 2024-07-03T15:44:04.403 INFO:tasks.workunit.client.0.smithi028.stdout:1/438: truncate d4/d3c/d54/d79/d16/d57/d66/f67 1885486 0 2024-07-03T15:44:04.404 INFO:tasks.workunit.client.0.smithi028.stdout:1/439: rmdir d4/d3c/d54/d79/d8c 0 2024-07-03T15:44:04.404 INFO:tasks.workunit.client.0.smithi028.stdout:6/259: write d4/d15/f2a [1336772,37728] 0 2024-07-03T15:44:04.406 INFO:tasks.workunit.client.0.smithi028.stdout:1/440: mkdir d4/de/d1c/da6 0 2024-07-03T15:44:04.407 INFO:tasks.workunit.client.0.smithi028.stdout:6/260: creat d4/da/d38/d3d/f58 x:0 0 0 2024-07-03T15:44:04.409 INFO:tasks.workunit.client.0.smithi028.stdout:1/441: rename d4/de/d1c/da6 to d4/de/da7 0 2024-07-03T15:44:04.409 INFO:tasks.workunit.client.0.smithi028.stdout:1/442: chown d4/d3c/d54/d79/f3e 954191209 1 2024-07-03T15:44:04.409 INFO:tasks.workunit.client.0.smithi028.stdout:1/443: write d4/f37 [2367521,6503] 0 2024-07-03T15:44:04.409 INFO:tasks.workunit.client.0.smithi028.stdout:1/444: write d4/d2c/f78 [1710701,83236] 0 2024-07-03T15:44:04.410 INFO:tasks.workunit.client.0.smithi028.stdout:1/445: fdatasync d4/d2c/d3a/d5c/d6c/d7d/d59/f2d 0 2024-07-03T15:44:04.411 INFO:tasks.workunit.client.0.smithi028.stdout:6/261: mkdir d4/da/d38/d59 0 2024-07-03T15:44:04.411 INFO:tasks.workunit.client.0.smithi028.stdout:6/262: read - d4/da/d1c/f50 zero size 2024-07-03T15:44:04.415 INFO:tasks.workunit.client.0.smithi028.stdout:1/446: unlink d4/d2c/l48 0 2024-07-03T15:44:04.415 INFO:tasks.workunit.client.0.smithi028.stdout:1/447: chown d4/d2c/l62 47125410 1 2024-07-03T15:44:04.415 INFO:tasks.workunit.client.0.smithi028.stdout:1/448: creat d4/d2c/d3a/d43/fa8 x:0 0 0 2024-07-03T15:44:04.418 INFO:tasks.workunit.client.0.smithi028.stdout:6/263: rmdir d4/d12 39 2024-07-03T15:44:04.422 INFO:tasks.workunit.client.0.smithi028.stdout:8/286: sync 2024-07-03T15:44:04.422 INFO:tasks.workunit.client.0.smithi028.stdout:0/223: sync 2024-07-03T15:44:04.423 INFO:tasks.workunit.client.0.smithi028.stdout:3/337: sync 2024-07-03T15:44:04.425 INFO:tasks.workunit.client.0.smithi028.stdout:0/224: mkdir d2/d1c/d32/d36/d45 0 2024-07-03T15:44:04.425 INFO:tasks.workunit.client.0.smithi028.stdout:0/225: readlink d2/d1c/d32/d36/l44 0 2024-07-03T15:44:04.425 INFO:tasks.workunit.client.0.smithi028.stdout:0/226: truncate d2/d3/f2f 405282 0 2024-07-03T15:44:04.426 INFO:tasks.workunit.client.0.smithi028.stdout:3/338: mknod d18/d1a/d5e/c6b 0 2024-07-03T15:44:04.429 INFO:tasks.workunit.client.0.smithi028.stdout:0/227: unlink d2/d3/c3e 0 2024-07-03T15:44:04.433 INFO:tasks.workunit.client.0.smithi028.stdout:8/287: dread d0/d41/d50/d3/dc/d11/d12/f39 [0,4194304] 0 2024-07-03T15:44:04.433 INFO:tasks.workunit.client.0.smithi028.stdout:8/288: getdents d0/dd/d22/d40/d4a 0 2024-07-03T15:44:04.434 INFO:tasks.workunit.client.0.smithi028.stdout:8/289: mkdir d0/d41/d50/d3/dc/d11/d5d 0 2024-07-03T15:44:04.435 INFO:tasks.workunit.client.0.smithi028.stdout:1/449: dwrite d4/d2c/f92 [0,4194304] 0 2024-07-03T15:44:04.437 INFO:tasks.workunit.client.0.smithi028.stdout:8/290: dread d0/d41/d50/d3/dc/d11/f42 [0,4194304] 0 2024-07-03T15:44:04.438 INFO:tasks.workunit.client.0.smithi028.stdout:1/450: symlink d4/de/d1c/d81/la9 0 2024-07-03T15:44:04.439 INFO:tasks.workunit.client.0.smithi028.stdout:1/451: write d4/d2c/d3a/d5c/d6c/d7d/d59/d94/f96 [68178,43350] 0 2024-07-03T15:44:04.439 INFO:tasks.workunit.client.0.smithi028.stdout:8/291: symlink d0/d41/d50/d3/dc/d11/d12/l5e 0 2024-07-03T15:44:04.440 INFO:tasks.workunit.client.0.smithi028.stdout:8/292: fdatasync d0/d41/d50/d3/dc/f23 0 2024-07-03T15:44:04.440 INFO:tasks.workunit.client.0.smithi028.stdout:1/452: truncate f2 2517868 0 2024-07-03T15:44:04.441 INFO:tasks.workunit.client.0.smithi028.stdout:8/293: mknod d0/dd/c5f 0 2024-07-03T15:44:04.442 INFO:tasks.workunit.client.0.smithi028.stdout:1/453: mkdir d4/d2c/d3a/d5c/daa 0 2024-07-03T15:44:04.443 INFO:tasks.workunit.client.0.smithi028.stdout:1/454: truncate d4/d2c/d3a/d5c/d6c/d7d/d59/d94/f96 437653 0 2024-07-03T15:44:04.443 INFO:tasks.workunit.client.0.smithi028.stdout:1/455: chown d4/d2c/d3a/d5c/d6c/d7d/d59/f46 0 1 2024-07-03T15:44:04.443 INFO:tasks.workunit.client.0.smithi028.stdout:8/294: truncate d0/d41/d50/d3/dc/d11/d12/f15 1232456 0 2024-07-03T15:44:04.444 INFO:tasks.workunit.client.0.smithi028.stdout:8/295: creat d0/dd/d22/d40/f60 x:0 0 0 2024-07-03T15:44:04.446 INFO:tasks.workunit.client.0.smithi028.stdout:8/296: creat d0/d41/d50/d3/dc/d3f/f61 x:0 0 0 2024-07-03T15:44:04.446 INFO:tasks.workunit.client.0.smithi028.stdout:8/297: write d0/f44 [2482164,56528] 0 2024-07-03T15:44:04.449 INFO:tasks.workunit.client.0.smithi028.stdout:6/264: dwrite d4/da/d38/f40 [0,4194304] 0 2024-07-03T15:44:04.449 INFO:tasks.workunit.client.0.smithi028.stdout:1/456: dread d4/de/f6f [0,4194304] 0 2024-07-03T15:44:04.449 INFO:tasks.workunit.client.0.smithi028.stdout:1/457: write d4/de/f5a [4794027,88434] 0 2024-07-03T15:44:04.449 INFO:tasks.workunit.client.0.smithi028.stdout:1/458: write d4/d2c/d3a/f68 [2940659,74152] 0 2024-07-03T15:44:04.452 INFO:tasks.workunit.client.0.smithi028.stdout:1/459: link d4/de/d1c/l36 d4/d2c/lab 0 2024-07-03T15:44:04.452 INFO:tasks.workunit.client.0.smithi028.stdout:1/460: chown d4/de/f31 601474 1 2024-07-03T15:44:04.452 INFO:tasks.workunit.client.0.smithi028.stdout:1/461: creat d4/d83/fac x:0 0 0 2024-07-03T15:44:04.454 INFO:tasks.workunit.client.0.smithi028.stdout:1/462: rename d4/d2c/d3a/d5c/d6c/d7d/d59/l2f to d4/d83/lad 0 2024-07-03T15:44:04.457 INFO:tasks.workunit.client.0.smithi028.stdout:1/463: rmdir d4/d3c/d54 39 2024-07-03T15:44:04.459 INFO:tasks.workunit.client.0.smithi028.stdout:1/464: mknod d4/cae 0 2024-07-03T15:44:04.459 INFO:tasks.workunit.client.0.smithi028.stdout:1/465: creat d4/d2c/faf x:0 0 0 2024-07-03T15:44:04.459 INFO:tasks.workunit.client.0.smithi028.stdout:1/466: chown d4/d2c/d3a/d5c/d6c/d7d/f89 660400 1 2024-07-03T15:44:04.459 INFO:tasks.workunit.client.0.smithi028.stdout:1/467: chown d4/d2c/d3a/d5c/d6c/d7d 31 1 2024-07-03T15:44:04.461 INFO:tasks.workunit.client.0.smithi028.stdout:1/468: rename d4/d2c/d3a/d43/fa8 to d4/da4/fb0 0 2024-07-03T15:44:04.463 INFO:tasks.workunit.client.0.smithi028.stdout:1/469: symlink d4/de/da7/lb1 0 2024-07-03T15:44:04.464 INFO:tasks.workunit.client.0.smithi028.stdout:1/470: truncate d4/de/f85 3459545 0 2024-07-03T15:44:04.466 INFO:tasks.workunit.client.0.smithi028.stdout:6/265: dread d4/f10 [0,4194304] 0 2024-07-03T15:44:04.466 INFO:tasks.workunit.client.0.smithi028.stdout:8/298: dwrite d0/d41/d50/d3/dc/d11/d12/f2f [0,4194304] 0 2024-07-03T15:44:04.466 INFO:tasks.workunit.client.0.smithi028.stdout:6/266: fdatasync d4/f45 0 2024-07-03T15:44:04.466 INFO:tasks.workunit.client.0.smithi028.stdout:8/299: stat d0/d41/d50/c10 0 2024-07-03T15:44:04.466 INFO:tasks.workunit.client.0.smithi028.stdout:8/300: fsync d0/dd/d22/f25 0 2024-07-03T15:44:04.467 INFO:tasks.workunit.client.0.smithi028.stdout:0/228: dwrite d2/d3/f19 [0,4194304] 0 2024-07-03T15:44:04.467 INFO:tasks.workunit.client.0.smithi028.stdout:0/229: truncate d2/d1c/f1e 482497 0 2024-07-03T15:44:04.468 INFO:tasks.workunit.client.0.smithi028.stdout:3/339: dwrite d18/d3a/f3f [0,4194304] 0 2024-07-03T15:44:04.482 INFO:tasks.workunit.client.0.smithi028.stdout:1/471: read d4/d2c/d3a/d5c/d6c/d7d/d59/f46 [4038171,20986] 0 2024-07-03T15:44:04.486 INFO:tasks.workunit.client.0.smithi028.stdout:8/301: write d0/dd/d22/f25 [520454,27258] 0 2024-07-03T15:44:04.487 INFO:tasks.workunit.client.0.smithi028.stdout:3/340: creat d18/d1a/d1e/d58/f6c x:0 0 0 2024-07-03T15:44:04.487 INFO:tasks.workunit.client.0.smithi028.stdout:3/341: write d18/f1d [863527,50270] 0 2024-07-03T15:44:04.488 INFO:tasks.workunit.client.0.smithi028.stdout:8/302: write d0/d41/d50/d3/f3a [197869,68483] 0 2024-07-03T15:44:04.488 INFO:tasks.workunit.client.0.smithi028.stdout:8/303: dread - d0/dd/d22/d40/f60 zero size 2024-07-03T15:44:04.490 INFO:tasks.workunit.client.0.smithi028.stdout:1/472: unlink d4/d2c/faf 0 2024-07-03T15:44:04.495 INFO:tasks.workunit.client.0.smithi028.stdout:3/342: unlink d18/d1a/d1e/f48 0 2024-07-03T15:44:04.495 INFO:tasks.workunit.client.0.smithi028.stdout:3/343: chown ld 1752672 1 2024-07-03T15:44:04.496 INFO:tasks.workunit.client.0.smithi028.stdout:1/473: mkdir d4/d2c/d3a/d5c/d6c/db2 0 2024-07-03T15:44:04.498 INFO:tasks.workunit.client.0.smithi028.stdout:6/267: dwrite d4/da/f4e [0,4194304] 0 2024-07-03T15:44:04.498 INFO:tasks.workunit.client.0.smithi028.stdout:6/268: fdatasync d4/da/d13/f19 0 2024-07-03T15:44:04.498 INFO:tasks.workunit.client.0.smithi028.stdout:0/230: dwrite d2/d3/f27 [0,4194304] 0 2024-07-03T15:44:04.504 INFO:tasks.workunit.client.0.smithi028.stdout:6/269: mkdir d4/da/d47/d5a 0 2024-07-03T15:44:04.504 INFO:tasks.workunit.client.0.smithi028.stdout:3/344: write d18/f55 [333127,31959] 0 2024-07-03T15:44:04.505 INFO:tasks.workunit.client.0.smithi028.stdout:6/270: creat d4/d15/f5b x:0 0 0 2024-07-03T15:44:04.505 INFO:tasks.workunit.client.0.smithi028.stdout:6/271: chown d4/da/c1a 12157513 1 2024-07-03T15:44:04.513 INFO:tasks.workunit.client.0.smithi028.stdout:6/272: getdents d4/d12/d2c/d31 0 2024-07-03T15:44:04.515 INFO:tasks.workunit.client.0.smithi028.stdout:8/304: dwrite d0/d41/d50/d3/dc/d11/d12/f2f [8388608,4194304] 0 2024-07-03T15:44:04.515 INFO:tasks.workunit.client.0.smithi028.stdout:8/305: stat d0/d41/d50/le 0 2024-07-03T15:44:04.515 INFO:tasks.workunit.client.0.smithi028.stdout:8/306: fsync d0/d41/d50/f27 0 2024-07-03T15:44:04.516 INFO:tasks.workunit.client.0.smithi028.stdout:9/311: sync 2024-07-03T15:44:04.517 INFO:tasks.workunit.client.0.smithi028.stdout:8/307: mkdir d0/dd/d22/d40/d62 0 2024-07-03T15:44:04.518 INFO:tasks.workunit.client.0.smithi028.stdout:8/308: read d0/d41/d50/d3/dc/d11/f42 [1615772,115497] 0 2024-07-03T15:44:04.520 INFO:tasks.workunit.client.0.smithi028.stdout:9/312: creat d1/d16/d57/f66 x:0 0 0 2024-07-03T15:44:04.522 INFO:tasks.workunit.client.0.smithi028.stdout:8/309: creat d0/d41/d50/d3/dc/f63 x:0 0 0 2024-07-03T15:44:04.522 INFO:tasks.workunit.client.0.smithi028.stdout:8/310: readlink d0/l49 0 2024-07-03T15:44:04.524 INFO:tasks.workunit.client.0.smithi028.stdout:8/311: mkdir d0/d41/d64 0 2024-07-03T15:44:04.524 INFO:tasks.workunit.client.0.smithi028.stdout:8/312: readlink d0/d41/d50/d3/l3d 0 2024-07-03T15:44:04.524 INFO:tasks.workunit.client.0.smithi028.stdout:3/345: fdatasync d18/f55 0 2024-07-03T15:44:04.527 INFO:tasks.workunit.client.0.smithi028.stdout:8/313: creat d0/dd/d22/d40/d62/f65 x:0 0 0 2024-07-03T15:44:04.527 INFO:tasks.workunit.client.0.smithi028.stdout:8/314: fdatasync d0/f5b 0 2024-07-03T15:44:04.530 INFO:tasks.workunit.client.0.smithi028.stdout:8/315: mkdir d0/d41/d50/d3/dc/d33/d66 0 2024-07-03T15:44:04.531 INFO:tasks.workunit.client.0.smithi028.stdout:6/273: dread d4/da/f4e [0,4194304] 0 2024-07-03T15:44:04.534 INFO:tasks.workunit.client.0.smithi028.stdout:1/474: dwrite d4/de/f85 [0,4194304] 0 2024-07-03T15:44:04.534 INFO:tasks.workunit.client.0.smithi028.stdout:3/346: dread d18/d3a/f5a [0,4194304] 0 2024-07-03T15:44:04.534 INFO:tasks.workunit.client.0.smithi028.stdout:1/475: chown d4/de/f10 735471 1 2024-07-03T15:44:04.534 INFO:tasks.workunit.client.0.smithi028.stdout:1/476: truncate d4/d2c/d3a/d5c/d6c/d7d/d4e/d51/f5f 1712910 0 2024-07-03T15:44:04.535 INFO:tasks.workunit.client.0.smithi028.stdout:0/231: dwrite d2/d3/f1d [0,4194304] 0 2024-07-03T15:44:04.535 INFO:tasks.workunit.client.0.smithi028.stdout:9/313: dread d1/f41 [0,4194304] 0 2024-07-03T15:44:04.540 INFO:tasks.workunit.client.0.smithi028.stdout:6/274: symlink d4/d15/l5c 0 2024-07-03T15:44:04.546 INFO:tasks.workunit.client.0.smithi028.stdout:9/314: dread d1/f21 [0,4194304] 0 2024-07-03T15:44:04.546 INFO:tasks.workunit.client.0.smithi028.stdout:0/232: unlink d2/d3/c31 0 2024-07-03T15:44:04.547 INFO:tasks.workunit.client.0.smithi028.stdout:5/337: sync 2024-07-03T15:44:04.547 INFO:tasks.workunit.client.0.smithi028.stdout:7/179: sync 2024-07-03T15:44:04.547 INFO:tasks.workunit.client.0.smithi028.stdout:2/305: sync 2024-07-03T15:44:04.547 INFO:tasks.workunit.client.0.smithi028.stdout:2/306: creat da/f77 x:0 0 0 2024-07-03T15:44:04.547 INFO:tasks.workunit.client.0.smithi028.stdout:2/307: dread - da/d24/f55 zero size 2024-07-03T15:44:04.547 INFO:tasks.workunit.client.0.smithi028.stdout:2/308: dread - da/d42/f65 zero size 2024-07-03T15:44:04.547 INFO:tasks.workunit.client.0.smithi028.stdout:2/309: chown da/f1d 5369603 1 2024-07-03T15:44:04.547 INFO:tasks.workunit.client.0.smithi028.stdout:4/283: sync 2024-07-03T15:44:04.548 INFO:tasks.workunit.client.0.smithi028.stdout:8/316: mknod d0/d41/d50/d3/c67 0 2024-07-03T15:44:04.548 INFO:tasks.workunit.client.0.smithi028.stdout:8/317: fsync d0/d41/d50/f27 0 2024-07-03T15:44:04.548 INFO:tasks.workunit.client.0.smithi028.stdout:0/233: write d2/d1c/d32/f35 [707034,36798] 0 2024-07-03T15:44:04.548 INFO:tasks.workunit.client.0.smithi028.stdout:0/234: fsync d2/d3/f2f 0 2024-07-03T15:44:04.550 INFO:tasks.workunit.client.0.smithi028.stdout:9/315: creat d1/d10/d28/d4e/d4f/f67 x:0 0 0 2024-07-03T15:44:04.550 INFO:tasks.workunit.client.0.smithi028.stdout:9/316: write d1/d16/f3a [4654054,109692] 0 2024-07-03T15:44:04.551 INFO:tasks.workunit.client.0.smithi028.stdout:9/317: creat d1/dd/f68 x:0 0 0 2024-07-03T15:44:04.551 INFO:tasks.workunit.client.0.smithi028.stdout:9/318: chown d1/d10/d28/l54 664 1 2024-07-03T15:44:04.551 INFO:tasks.workunit.client.0.smithi028.stdout:5/338: dread d0/dd/d13/d57/d26/f2e [0,4194304] 0 2024-07-03T15:44:04.551 INFO:tasks.workunit.client.0.smithi028.stdout:7/180: creat d9/f3a x:0 0 0 2024-07-03T15:44:04.554 INFO:tasks.workunit.client.0.smithi028.stdout:2/310: mkdir da/d42/d47/d78 0 2024-07-03T15:44:04.556 INFO:tasks.workunit.client.0.smithi028.stdout:4/284: mknod d2/d12/d3b/c5c 0 2024-07-03T15:44:04.557 INFO:tasks.workunit.client.0.smithi028.stdout:6/275: write d4/d15/f20 [2659611,87195] 0 2024-07-03T15:44:04.560 INFO:tasks.workunit.client.0.smithi028.stdout:9/319: mknod d1/c69 0 2024-07-03T15:44:04.561 INFO:tasks.workunit.client.0.smithi028.stdout:2/311: unlink da/l1b 0 2024-07-03T15:44:04.561 INFO:tasks.workunit.client.0.smithi028.stdout:2/312: chown da/f17 10 1 2024-07-03T15:44:04.561 INFO:tasks.workunit.client.0.smithi028.stdout:2/313: dread - da/d23/f75 zero size 2024-07-03T15:44:04.562 INFO:tasks.workunit.client.0.smithi028.stdout:2/314: chown da/d42/f46 7452 1 2024-07-03T15:44:04.562 INFO:tasks.workunit.client.0.smithi028.stdout:2/315: read - da/d24/f66 zero size 2024-07-03T15:44:04.562 INFO:tasks.workunit.client.0.smithi028.stdout:1/477: dwrite d4/da4/fb0 [0,4194304] 0 2024-07-03T15:44:04.563 INFO:tasks.workunit.client.0.smithi028.stdout:6/276: symlink d4/d15/l5d 0 2024-07-03T15:44:04.567 INFO:tasks.workunit.client.0.smithi028.stdout:7/181: read d9/f34 [815219,64105] 0 2024-07-03T15:44:04.567 INFO:tasks.workunit.client.0.smithi028.stdout:3/347: dwrite d18/f1d [0,4194304] 0 2024-07-03T15:44:04.567 INFO:tasks.workunit.client.0.smithi028.stdout:3/348: stat d18/d1a/d1e 0 2024-07-03T15:44:04.569 INFO:tasks.workunit.client.0.smithi028.stdout:8/318: rename d0/dd to d0/d41/d50/d3/d35/d68 0 2024-07-03T15:44:04.574 INFO:tasks.workunit.client.0.smithi028.stdout:2/316: symlink da/d24/l79 0 2024-07-03T15:44:04.574 INFO:tasks.workunit.client.0.smithi028.stdout:1/478: dread d4/de/f85 [0,4194304] 0 2024-07-03T15:44:04.575 INFO:tasks.workunit.client.0.smithi028.stdout:1/479: write d4/d2c/f78 [409199,102330] 0 2024-07-03T15:44:04.579 INFO:tasks.workunit.client.0.smithi028.stdout:4/285: symlink d2/d12/l5d 0 2024-07-03T15:44:04.581 INFO:tasks.workunit.client.0.smithi028.stdout:2/317: dread da/d24/d2c/f54 [0,4194304] 0 2024-07-03T15:44:04.585 INFO:tasks.workunit.client.0.smithi028.stdout:8/319: stat d0/d41/d50/d3/dc/d11/c5c 0 2024-07-03T15:44:04.585 INFO:tasks.workunit.client.0.smithi028.stdout:8/320: write d0/f5b [293867,72543] 0 2024-07-03T15:44:04.585 INFO:tasks.workunit.client.0.smithi028.stdout:8/321: write d0/d41/d50/d3/d35/d68/d22/f25 [5533641,43105] 0 2024-07-03T15:44:04.585 INFO:tasks.workunit.client.0.smithi028.stdout:8/322: write d0/d41/d50/d3/dc/d11/d12/f39 [3091117,24830] 0 2024-07-03T15:44:04.586 INFO:tasks.workunit.client.0.smithi028.stdout:8/323: dread - d0/d41/d50/d3/d35/d68/d22/d40/d62/f65 zero size 2024-07-03T15:44:04.586 INFO:tasks.workunit.client.0.smithi028.stdout:8/324: write d0/d41/d50/d3/f3a [1447190,121966] 0 2024-07-03T15:44:04.586 INFO:tasks.workunit.client.0.smithi028.stdout:8/325: chown d0/d41/d50/d3/dc/d11/d4c 0 1 2024-07-03T15:44:04.588 INFO:tasks.workunit.client.0.smithi028.stdout:0/235: rename d2/d3/f19 to d2/d1c/d32/d36/d45/f46 0 2024-07-03T15:44:04.588 INFO:tasks.workunit.client.0.smithi028.stdout:5/339: rename d0/dd/d13 to d0/dd/d13/d57/d6f 22 2024-07-03T15:44:04.594 INFO:tasks.workunit.client.0.smithi028.stdout:1/480: creat d4/d3c/d54/d79/d16/fb3 x:0 0 0 2024-07-03T15:44:04.595 INFO:tasks.workunit.client.0.smithi028.stdout:4/286: creat d2/d12/d2d/d58/f5e x:0 0 0 2024-07-03T15:44:04.597 INFO:tasks.workunit.client.0.smithi028.stdout:2/318: creat da/d1e/d56/f7a x:0 0 0 2024-07-03T15:44:04.603 INFO:tasks.workunit.client.0.smithi028.stdout:9/320: rename d1/dd/f26 to d1/d2a/f6a 0 2024-07-03T15:44:04.605 INFO:tasks.workunit.client.0.smithi028.stdout:5/340: mkdir d0/dd/d13/d57/d30/d70 0 2024-07-03T15:44:04.606 INFO:tasks.workunit.client.0.smithi028.stdout:0/236: symlink d2/d1c/d32/d36/d45/l47 0 2024-07-03T15:44:04.610 INFO:tasks.workunit.client.0.smithi028.stdout:1/481: rmdir d4/d2c/d3a/d5c/d6c/d7d/d4e/d51 39 2024-07-03T15:44:04.611 INFO:tasks.workunit.client.0.smithi028.stdout:4/287: mknod d2/d12/d2d/d58/c5f 0 2024-07-03T15:44:04.615 INFO:tasks.workunit.client.0.smithi028.stdout:2/319: stat da/d23/d30/l44 0 2024-07-03T15:44:04.618 INFO:tasks.workunit.client.0.smithi028.stdout:6/277: rename d4/da/d1c/f4d to d4/da/d38/f5e 0 2024-07-03T15:44:04.618 INFO:tasks.workunit.client.0.smithi028.stdout:6/278: chown d4/l1b 63 1 2024-07-03T15:44:04.621 INFO:tasks.workunit.client.0.smithi028.stdout:9/321: truncate d1/d16/f1c 2032765 0 2024-07-03T15:44:04.622 INFO:tasks.workunit.client.0.smithi028.stdout:5/341: dwrite d0/dd/d1e/f56 [0,4194304] 0 2024-07-03T15:44:04.622 INFO:tasks.workunit.client.0.smithi028.stdout:5/342: fdatasync d0/dd/f53 0 2024-07-03T15:44:04.624 INFO:tasks.workunit.client.0.smithi028.stdout:0/237: rmdir d2 39 2024-07-03T15:44:04.626 INFO:tasks.workunit.client.0.smithi028.stdout:4/288: rmdir d2/d3e 39 2024-07-03T15:44:04.629 INFO:tasks.workunit.client.0.smithi028.stdout:7/182: rename d9/d1c/f1e to d9/f3b 0 2024-07-03T15:44:04.631 INFO:tasks.workunit.client.0.smithi028.stdout:9/322: symlink d1/d10/d28/d45/d61/l6b 0 2024-07-03T15:44:04.631 INFO:tasks.workunit.client.0.smithi028.stdout:9/323: truncate d1/d10/d28/d4e/f50 954021 0 2024-07-03T15:44:04.632 INFO:tasks.workunit.client.0.smithi028.stdout:5/343: dread d0/dd/d1e/d3b/f5e [4194304,4194304] 0 2024-07-03T15:44:04.632 INFO:tasks.workunit.client.0.smithi028.stdout:2/320: dwrite da/d24/d2c/f54 [0,4194304] 0 2024-07-03T15:44:04.636 INFO:tasks.workunit.client.0.smithi028.stdout:0/238: rmdir d2/d1c/d32/d36 39 2024-07-03T15:44:04.639 INFO:tasks.workunit.client.0.smithi028.stdout:1/482: getdents d4/d2c/d3a/d5c/d6c/d7d 0 2024-07-03T15:44:04.639 INFO:tasks.workunit.client.0.smithi028.stdout:1/483: creat d4/d3c/d54/d79/d16/fb4 x:0 0 0 2024-07-03T15:44:04.640 INFO:tasks.workunit.client.0.smithi028.stdout:8/326: rename d0/d41/d64 to d0/d41/d50/d3/dc/d11/d69 0 2024-07-03T15:44:04.643 INFO:tasks.workunit.client.0.smithi028.stdout:0/239: rmdir d2/d1c 39 2024-07-03T15:44:04.644 INFO:tasks.workunit.client.0.smithi028.stdout:7/183: write d9/f3b [3839403,2158] 0 2024-07-03T15:44:04.644 INFO:tasks.workunit.client.0.smithi028.stdout:7/184: write d9/d1c/f20 [1206016,4986] 0 2024-07-03T15:44:04.644 INFO:tasks.workunit.client.0.smithi028.stdout:7/185: stat d9/f34 0 2024-07-03T15:44:04.646 INFO:tasks.workunit.client.0.smithi028.stdout:1/484: creat d4/d2c/d3a/d5c/daa/fb5 x:0 0 0 2024-07-03T15:44:04.648 INFO:tasks.workunit.client.0.smithi028.stdout:8/327: creat d0/d41/d50/d3/dc/d11/f6a x:0 0 0 2024-07-03T15:44:04.648 INFO:tasks.workunit.client.0.smithi028.stdout:8/328: write d0/d41/d50/d3/dc/f2b [4992700,90626] 0 2024-07-03T15:44:04.648 INFO:tasks.workunit.client.0.smithi028.stdout:9/324: dread d1/d16/f2f [0,4194304] 0 2024-07-03T15:44:04.648 INFO:tasks.workunit.client.0.smithi028.stdout:0/240: symlink d2/d1c/l48 0 2024-07-03T15:44:04.650 INFO:tasks.workunit.client.0.smithi028.stdout:1/485: rmdir d4/d3c/d54/d79/d16/d57 39 2024-07-03T15:44:04.653 INFO:tasks.workunit.client.0.smithi028.stdout:2/321: rename da/d24/d2c/f54 to da/d1e/f7b 0 2024-07-03T15:44:04.656 INFO:tasks.workunit.client.0.smithi028.stdout:8/329: link d0/d41/d50/d3/f3c d0/d41/d50/d3/d35/d68/d22/f6b 0 2024-07-03T15:44:04.657 INFO:tasks.workunit.client.0.smithi028.stdout:4/289: dwrite d2/d12/d3b/f4b [0,4194304] 0 2024-07-03T15:44:04.657 INFO:tasks.workunit.client.0.smithi028.stdout:4/290: dread - d2/f52 zero size 2024-07-03T15:44:04.658 INFO:tasks.workunit.client.0.smithi028.stdout:5/344: dwrite d0/dd/d13/d29/f35 [4194304,4194304] 0 2024-07-03T15:44:04.659 INFO:tasks.workunit.client.0.smithi028.stdout:2/322: truncate da/d42/f4d 709861 0 2024-07-03T15:44:04.663 INFO:tasks.workunit.client.0.smithi028.stdout:4/291: rmdir d2/d12/d42 39 2024-07-03T15:44:04.667 INFO:tasks.workunit.client.0.smithi028.stdout:5/345: mkdir d0/dd/d1e/d3b/d3f/d71 0 2024-07-03T15:44:04.667 INFO:tasks.workunit.client.0.smithi028.stdout:5/346: write d0/dd/d1e/d3b/d3f/d5a/f65 [112497,44836] 0 2024-07-03T15:44:04.668 INFO:tasks.workunit.client.0.smithi028.stdout:2/323: creat da/d23/d6f/f7c x:0 0 0 2024-07-03T15:44:04.670 INFO:tasks.workunit.client.0.smithi028.stdout:9/325: write d1/dd/f1b [3063749,64718] 0 2024-07-03T15:44:04.672 INFO:tasks.workunit.client.0.smithi028.stdout:4/292: creat d2/d12/f60 x:0 0 0 2024-07-03T15:44:04.674 INFO:tasks.workunit.client.0.smithi028.stdout:5/347: rename d0/dd/df/l23 to d0/dd/d13/d29/l72 0 2024-07-03T15:44:04.674 INFO:tasks.workunit.client.0.smithi028.stdout:5/348: chown d0/dd/d1e/c2a 737760524 1 2024-07-03T15:44:04.678 INFO:tasks.workunit.client.0.smithi028.stdout:8/330: dread d0/d41/d50/d3/dc/d11/d12/f56 [0,4194304] 0 2024-07-03T15:44:04.678 INFO:tasks.workunit.client.0.smithi028.stdout:2/324: mknod da/d24/c7d 0 2024-07-03T15:44:04.679 INFO:tasks.workunit.client.0.smithi028.stdout:2/325: creat da/d1e/d56/f7e x:0 0 0 2024-07-03T15:44:04.679 INFO:tasks.workunit.client.0.smithi028.stdout:2/326: read - da/f77 zero size 2024-07-03T15:44:04.679 INFO:tasks.workunit.client.0.smithi028.stdout:2/327: chown l1 798096164 1 2024-07-03T15:44:04.679 INFO:tasks.workunit.client.0.smithi028.stdout:2/328: chown da/d23/d30/c31 127236 1 2024-07-03T15:44:04.683 INFO:tasks.workunit.client.0.smithi028.stdout:4/293: mkdir d2/d61 0 2024-07-03T15:44:04.684 INFO:tasks.workunit.client.0.smithi028.stdout:0/241: dwrite d2/d1c/f21 [0,4194304] 0 2024-07-03T15:44:04.684 INFO:tasks.workunit.client.0.smithi028.stdout:0/242: chown d2/c1f 2939104 1 2024-07-03T15:44:04.684 INFO:tasks.workunit.client.0.smithi028.stdout:0/243: fdatasync d2/f3a 0 2024-07-03T15:44:04.684 INFO:tasks.workunit.client.0.smithi028.stdout:7/186: dwrite d9/f1a [0,4194304] 0 2024-07-03T15:44:04.685 INFO:tasks.workunit.client.0.smithi028.stdout:9/326: rename d1/dd/f1b to d1/d16/d57/d60/f6c 0 2024-07-03T15:44:04.689 INFO:tasks.workunit.client.0.smithi028.stdout:8/331: unlink d0/d41/d50/d3/dc/c16 0 2024-07-03T15:44:04.690 INFO:tasks.workunit.client.0.smithi028.stdout:5/349: dwrite d0/dd/d1e/d3b/f4f [0,4194304] 0 2024-07-03T15:44:04.690 INFO:tasks.workunit.client.0.smithi028.stdout:5/350: stat d0/dd 0 2024-07-03T15:44:04.694 INFO:tasks.workunit.client.0.smithi028.stdout:8/332: dread d0/d41/d50/d3/dc/d11/d12/f15 [0,4194304] 0 2024-07-03T15:44:04.694 INFO:tasks.workunit.client.0.smithi028.stdout:8/333: truncate d0/d41/d50/d3/d35/d68/d22/d40/f60 699714 0 2024-07-03T15:44:04.695 INFO:tasks.workunit.client.0.smithi028.stdout:8/334: write d0/f2a [442048,95499] 0 2024-07-03T15:44:04.695 INFO:tasks.workunit.client.0.smithi028.stdout:8/335: stat d0/d41/d50/d3/l3d 0 2024-07-03T15:44:04.696 INFO:tasks.workunit.client.0.smithi028.stdout:7/187: unlink d9/d1c/c27 0 2024-07-03T15:44:04.698 INFO:tasks.workunit.client.0.smithi028.stdout:2/329: rename da/d1e/l60 to da/d1e/d68/l7f 0 2024-07-03T15:44:04.699 INFO:tasks.workunit.client.0.smithi028.stdout:4/294: dwrite d2/f52 [0,4194304] 0 2024-07-03T15:44:04.699 INFO:tasks.workunit.client.0.smithi028.stdout:4/295: dread - d2/d5a/d45/f4c zero size 2024-07-03T15:44:04.699 INFO:tasks.workunit.client.0.smithi028.stdout:4/296: readlink d2/d12/d2d/d2f/d3a/l4f 0 2024-07-03T15:44:04.699 INFO:tasks.workunit.client.0.smithi028.stdout:4/297: stat d2/d12/d3b/f59 0 2024-07-03T15:44:04.701 INFO:tasks.workunit.client.0.smithi028.stdout:9/327: link d1/d2a/f6a d1/d2a/f6d 0 2024-07-03T15:44:04.701 INFO:tasks.workunit.client.0.smithi028.stdout:5/351: rmdir d0/dd/d13 39 2024-07-03T15:44:04.702 INFO:tasks.workunit.client.0.smithi028.stdout:5/352: fdatasync d0/dd/d4b/f6c 0 2024-07-03T15:44:04.702 INFO:tasks.workunit.client.0.smithi028.stdout:5/353: readlink d0/dd/d13/d57/l48 0 2024-07-03T15:44:04.703 INFO:tasks.workunit.client.0.smithi028.stdout:0/244: rename d2/d1c/d32/f34 to d2/f49 0 2024-07-03T15:44:04.710 INFO:tasks.workunit.client.0.smithi028.stdout:5/354: creat d0/dd/d13/d57/d30/f73 x:0 0 0 2024-07-03T15:44:04.722 INFO:tasks.workunit.client.0.smithi028.stdout:4/298: dwrite d2/d5a/f56 [0,4194304] 0 2024-07-03T15:44:04.723 INFO:tasks.workunit.client.0.smithi028.stdout:9/328: dwrite d1/dd/f3c [0,4194304] 0 2024-07-03T15:44:04.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:04 smithi148 ceph-mon[36056]: mgrmap e22: smithi148.rjzsar(active, since 1.01925s) 2024-07-03T15:44:04.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:04 smithi148 ceph-mon[36056]: Deploying cephadm binary to smithi148 2024-07-03T15:44:04.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:04 smithi148 ceph-mon[36056]: Deploying cephadm binary to smithi028 2024-07-03T15:44:04.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:04 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:04.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:04 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:04.799 INFO:tasks.workunit.client.0.smithi028.stdout:8/336: rename d0/d41/d47 to d0/d6c 0 2024-07-03T15:44:04.799 INFO:tasks.workunit.client.0.smithi028.stdout:8/337: readlink d0/d41/d50/d3/l3d 0 2024-07-03T15:44:04.799 INFO:tasks.workunit.client.0.smithi028.stdout:8/338: chown d0/d41/d50/d3/dc/d11/d12 0 1 2024-07-03T15:44:04.799 INFO:tasks.workunit.client.0.smithi028.stdout:8/339: write d0/d41/d50/d3/dc/d11/d12/f56 [893494,72296] 0 2024-07-03T15:44:04.816 INFO:tasks.workunit.client.0.smithi028.stdout:8/340: dwrite d0/d41/d50/d3/f3a [0,4194304] 0 2024-07-03T15:44:04.819 INFO:tasks.workunit.client.0.smithi028.stdout:2/330: rmdir da/d23/d6f 39 2024-07-03T15:44:04.819 INFO:tasks.workunit.client.0.smithi028.stdout:2/331: chown da/d1e/d68/c6a 1625524 1 2024-07-03T15:44:04.819 INFO:tasks.workunit.client.0.smithi028.stdout:2/332: read da/ff [978044,54728] 0 2024-07-03T15:44:04.819 INFO:tasks.workunit.client.0.smithi028.stdout:2/333: stat da/d1e/d68/f6c 0 2024-07-03T15:44:04.821 INFO:tasks.workunit.client.0.smithi028.stdout:5/355: link d0/dd/df/c12 d0/dd/d1e/c74 0 2024-07-03T15:44:04.825 INFO:tasks.workunit.client.0.smithi028.stdout:8/341: dread d0/d41/d50/d3/dc/d11/d12/f39 [0,4194304] 0 2024-07-03T15:44:04.825 INFO:tasks.workunit.client.0.smithi028.stdout:8/342: creat d0/d41/d50/d3/dc/d11/f6d x:0 0 0 2024-07-03T15:44:04.829 INFO:tasks.workunit.client.0.smithi028.stdout:4/299: mkdir d2/d61/d62 0 2024-07-03T15:44:04.829 INFO:tasks.workunit.client.0.smithi028.stdout:4/300: fsync d2/d12/f26 0 2024-07-03T15:44:04.833 INFO:tasks.workunit.client.0.smithi028.stdout:9/329: mknod d1/d10/d28/d4e/d5b/c6e 0 2024-07-03T15:44:04.833 INFO:tasks.workunit.client.0.smithi028.stdout:9/330: creat d1/dd/f6f x:0 0 0 2024-07-03T15:44:04.833 INFO:tasks.workunit.client.0.smithi028.stdout:9/331: truncate d1/d16/f1c 2086588 0 2024-07-03T15:44:04.845 INFO:tasks.workunit.client.0.smithi028.stdout:5/356: dwrite d0/dd/f1b [0,4194304] 0 2024-07-03T15:44:04.850 INFO:tasks.workunit.client.0.smithi028.stdout:2/334: mkdir da/d74/d80 0 2024-07-03T15:44:04.853 INFO:tasks.workunit.client.0.smithi028.stdout:8/343: dwrite d0/d41/d50/d3/d35/d68/d22/f25 [4194304,4194304] 0 2024-07-03T15:44:04.853 INFO:tasks.workunit.client.0.smithi028.stdout:8/344: stat d0/d41/d50/c10 0 2024-07-03T15:44:04.855 INFO:tasks.workunit.client.0.smithi028.stdout:9/332: dwrite d1/dd/f20 [0,4194304] 0 2024-07-03T15:44:04.856 INFO:tasks.workunit.client.0.smithi028.stdout:4/301: mkdir d2/d12/d42/d63 0 2024-07-03T15:44:04.861 INFO:tasks.workunit.client.0.smithi028.stdout:5/357: dwrite d0/dd/d1e/f36 [0,4194304] 0 2024-07-03T15:44:04.864 INFO:tasks.workunit.client.0.smithi028.stdout:2/335: truncate f9 34327 0 2024-07-03T15:44:04.864 INFO:tasks.workunit.client.0.smithi028.stdout:2/336: write da/d42/f50 [121154,107823] 0 2024-07-03T15:44:04.865 INFO:tasks.workunit.client.0.smithi028.stdout:8/345: mknod d0/d41/d50/d3/dc/c6e 0 2024-07-03T15:44:04.865 INFO:tasks.workunit.client.0.smithi028.stdout:8/346: stat d0/d41/d50/d3/d35/d68/d22/f25 0 2024-07-03T15:44:04.871 INFO:tasks.workunit.client.0.smithi028.stdout:2/337: link da/d23/d30/f5c da/d42/d47/f81 0 2024-07-03T15:44:04.871 INFO:tasks.workunit.client.0.smithi028.stdout:8/347: mknod d0/d41/d50/d3/dc/d11/d69/c6f 0 2024-07-03T15:44:04.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:04 smithi028 ceph-mon[29148]: mgrmap e22: smithi148.rjzsar(active, since 1.01925s) 2024-07-03T15:44:04.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:04 smithi028 ceph-mon[29148]: Deploying cephadm binary to smithi148 2024-07-03T15:44:04.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:04 smithi028 ceph-mon[29148]: Deploying cephadm binary to smithi028 2024-07-03T15:44:04.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:04 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:04.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:04 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:04.873 INFO:tasks.workunit.client.0.smithi028.stdout:4/302: mknod d2/d5a/c64 0 2024-07-03T15:44:04.873 INFO:tasks.workunit.client.0.smithi028.stdout:4/303: truncate d2/d12/f60 600863 0 2024-07-03T15:44:04.873 INFO:tasks.workunit.client.0.smithi028.stdout:4/304: chown d2/d5a/f56 0 1 2024-07-03T15:44:04.874 INFO:tasks.workunit.client.0.smithi028.stdout:4/305: write d2/d12/d3b/f59 [290876,93534] 0 2024-07-03T15:44:04.874 INFO:tasks.workunit.client.0.smithi028.stdout:4/306: truncate d2/f27 5007307 0 2024-07-03T15:44:04.874 INFO:tasks.workunit.client.0.smithi028.stdout:4/307: fsync d2/f20 0 2024-07-03T15:44:04.874 INFO:tasks.workunit.client.0.smithi028.stdout:5/358: getdents d0/dd/d1e/d3b/d3f/d5a 0 2024-07-03T15:44:04.875 INFO:tasks.workunit.client.0.smithi028.stdout:5/359: dread d0/dd/d13/d57/d26/d41/f43 [0,4194304] 0 2024-07-03T15:44:04.875 INFO:tasks.workunit.client.0.smithi028.stdout:2/338: mknod da/d42/d47/d78/c82 0 2024-07-03T15:44:04.875 INFO:tasks.workunit.client.0.smithi028.stdout:8/348: mknod d0/d41/d50/d3/dc/c70 0 2024-07-03T15:44:04.876 INFO:tasks.workunit.client.0.smithi028.stdout:4/308: getdents d2/d61 0 2024-07-03T15:44:04.878 INFO:tasks.workunit.client.0.smithi028.stdout:2/339: symlink da/d42/d47/l83 0 2024-07-03T15:44:04.878 INFO:tasks.workunit.client.0.smithi028.stdout:2/340: fdatasync da/d23/d30/f67 0 2024-07-03T15:44:04.882 INFO:tasks.workunit.client.0.smithi028.stdout:9/333: dwrite d1/f2 [0,4194304] 0 2024-07-03T15:44:04.891 INFO:tasks.workunit.client.0.smithi028.stdout:5/360: read d0/dd/d13/d29/f4e [996645,12971] 0 2024-07-03T15:44:04.892 INFO:tasks.workunit.client.0.smithi028.stdout:5/361: symlink d0/dd/d13/d57/d26/d41/l75 0 2024-07-03T15:44:04.895 INFO:tasks.workunit.client.0.smithi028.stdout:5/362: write d0/dd/d13/d57/f1f [503105,50645] 0 2024-07-03T15:44:04.895 INFO:tasks.workunit.client.0.smithi028.stdout:5/363: fdatasync d0/dd/d13/d57/d26/f42 0 2024-07-03T15:44:04.898 INFO:tasks.workunit.client.0.smithi028.stdout:5/364: getdents d0/dd/d13/d57/d30 0 2024-07-03T15:44:04.901 INFO:tasks.workunit.client.0.smithi028.stdout:5/365: rename d0/dd/d13/d29/l72 to d0/dd/d13/d57/d26/d41/l76 0 2024-07-03T15:44:04.904 INFO:tasks.workunit.client.0.smithi028.stdout:8/349: dread d0/d41/d50/d3/f3a [0,4194304] 0 2024-07-03T15:44:04.904 INFO:tasks.workunit.client.0.smithi028.stdout:8/350: chown d0/d41/c52 689 1 2024-07-03T15:44:04.904 INFO:tasks.workunit.client.0.smithi028.stdout:5/366: mkdir d0/dd/d13/d57/d77 0 2024-07-03T15:44:04.908 INFO:tasks.workunit.client.0.smithi028.stdout:5/367: creat d0/dd/d1e/d3b/d3f/d5a/f78 x:0 0 0 2024-07-03T15:44:04.908 INFO:tasks.workunit.client.0.smithi028.stdout:4/309: dwrite d2/d12/d2d/d58/f5e [0,4194304] 0 2024-07-03T15:44:04.908 INFO:tasks.workunit.client.0.smithi028.stdout:4/310: chown d2/d5a/d45/f4c 0 1 2024-07-03T15:44:04.908 INFO:tasks.workunit.client.0.smithi028.stdout:4/311: fsync d2/d5a/f34 0 2024-07-03T15:44:04.910 INFO:tasks.workunit.client.0.smithi028.stdout:5/368: unlink d0/dd/d13/d57/d26/d41/l64 0 2024-07-03T15:44:04.910 INFO:tasks.workunit.client.0.smithi028.stdout:5/369: chown d0/dd/d13/d57/c27 0 1 2024-07-03T15:44:04.911 INFO:tasks.workunit.client.0.smithi028.stdout:8/351: dread d0/d41/d50/d3/dc/d11/d12/f56 [0,4194304] 0 2024-07-03T15:44:04.911 INFO:tasks.workunit.client.0.smithi028.stdout:9/334: dwrite d1/f2b [0,4194304] 0 2024-07-03T15:44:04.911 INFO:tasks.workunit.client.0.smithi028.stdout:4/312: dread d2/f20 [0,4194304] 0 2024-07-03T15:44:04.912 INFO:tasks.workunit.client.0.smithi028.stdout:2/341: dwrite da/d23/d30/f51 [0,4194304] 0 2024-07-03T15:44:04.913 INFO:tasks.workunit.client.0.smithi028.stdout:3/349: sync 2024-07-03T15:44:04.913 INFO:tasks.workunit.client.0.smithi028.stdout:3/350: truncate d18/d1a/f4f 5208832 0 2024-07-03T15:44:04.918 INFO:tasks.workunit.client.0.smithi028.stdout:9/335: creat d1/d10/d28/d45/d61/f70 x:0 0 0 2024-07-03T15:44:04.922 INFO:tasks.workunit.client.0.smithi028.stdout:4/313: creat d2/d12/d42/d63/f65 x:0 0 0 2024-07-03T15:44:04.922 INFO:tasks.workunit.client.0.smithi028.stdout:4/314: read d2/f20 [59882,45403] 0 2024-07-03T15:44:04.922 INFO:tasks.workunit.client.0.smithi028.stdout:4/315: creat d2/d12/d2d/d2f/f66 x:0 0 0 2024-07-03T15:44:04.928 INFO:tasks.workunit.client.0.smithi028.stdout:2/342: mkdir da/d23/d6f/d84 0 2024-07-03T15:44:04.929 INFO:tasks.workunit.client.0.smithi028.stdout:3/351: mknod d18/d1a/d5e/d66/c6d 0 2024-07-03T15:44:04.930 INFO:tasks.workunit.client.0.smithi028.stdout:8/352: mkdir d0/d41/d50/d3/d71 0 2024-07-03T15:44:04.931 INFO:tasks.workunit.client.0.smithi028.stdout:8/353: read d0/f5b [188367,4136] 0 2024-07-03T15:44:04.932 INFO:tasks.workunit.client.0.smithi028.stdout:4/316: creat d2/d61/f67 x:0 0 0 2024-07-03T15:44:04.935 INFO:tasks.workunit.client.0.smithi028.stdout:2/343: unlink da/lb 0 2024-07-03T15:44:04.935 INFO:tasks.workunit.client.0.smithi028.stdout:5/370: dwrite d0/dd/d4b/f6c [0,4194304] 0 2024-07-03T15:44:04.936 INFO:tasks.workunit.client.0.smithi028.stdout:3/352: rename d18/d1a/f5c to d18/d1a/d64/f6e 0 2024-07-03T15:44:04.937 INFO:tasks.workunit.client.0.smithi028.stdout:9/336: dwrite d1/dd/f35 [0,4194304] 0 2024-07-03T15:44:04.937 INFO:tasks.workunit.client.0.smithi028.stdout:9/337: chown d1/l8 23074326 1 2024-07-03T15:44:04.942 INFO:tasks.workunit.client.0.smithi028.stdout:5/371: mknod d0/dd/d13/d29/c79 0 2024-07-03T15:44:04.943 INFO:tasks.workunit.client.0.smithi028.stdout:5/372: write d0/dd/d4b/f6a [1013382,82445] 0 2024-07-03T15:44:04.945 INFO:tasks.workunit.client.0.smithi028.stdout:3/353: mkdir d18/d6f 0 2024-07-03T15:44:04.947 INFO:tasks.workunit.client.0.smithi028.stdout:4/317: truncate d2/d12/d2d/d58/f5e 821885 0 2024-07-03T15:44:04.949 INFO:tasks.workunit.client.0.smithi028.stdout:9/338: rename d1/dd/f11 to d1/d16/d57/f71 0 2024-07-03T15:44:04.950 INFO:tasks.workunit.client.0.smithi028.stdout:2/344: dwrite da/d1e/f45 [0,4194304] 0 2024-07-03T15:44:04.950 INFO:tasks.workunit.client.0.smithi028.stdout:2/345: creat da/f85 x:0 0 0 2024-07-03T15:44:04.951 INFO:tasks.workunit.client.0.smithi028.stdout:5/373: creat d0/dd/d1e/f7a x:0 0 0 2024-07-03T15:44:04.953 INFO:tasks.workunit.client.0.smithi028.stdout:2/346: dread da/f1d [0,4194304] 0 2024-07-03T15:44:04.954 INFO:tasks.workunit.client.0.smithi028.stdout:2/347: dread - da/d23/d30/f5c zero size 2024-07-03T15:44:04.957 INFO:tasks.workunit.client.0.smithi028.stdout:4/318: unlink d2/d12/d3b/f4b 0 2024-07-03T15:44:04.963 INFO:tasks.workunit.client.0.smithi028.stdout:9/339: dwrite d1/f41 [4194304,4194304] 0 2024-07-03T15:44:04.969 INFO:tasks.workunit.client.0.smithi028.stdout:5/374: link d0/dd/df/c12 d0/dd/d1e/c7b 0 2024-07-03T15:44:04.971 INFO:tasks.workunit.client.0.smithi028.stdout:3/354: getdents d18/d3a 0 2024-07-03T15:44:04.973 INFO:tasks.workunit.client.0.smithi028.stdout:2/348: getdents da/d23/d6f 0 2024-07-03T15:44:04.975 INFO:tasks.workunit.client.0.smithi028.stdout:4/319: creat d2/d12/f68 x:0 0 0 2024-07-03T15:44:04.975 INFO:tasks.workunit.client.0.smithi028.stdout:5/375: dread d0/dd/d13/f17 [0,4194304] 0 2024-07-03T15:44:04.975 INFO:tasks.workunit.client.0.smithi028.stdout:5/376: write d0/dd/d13/d57/d26/d41/f43 [2105763,15065] 0 2024-07-03T15:44:04.975 INFO:tasks.workunit.client.0.smithi028.stdout:5/377: chown d0/dd/d13/d57/d26/d41/l61 1 1 2024-07-03T15:44:04.976 INFO:tasks.workunit.client.0.smithi028.stdout:9/340: dwrite d1/d10/d28/d4e/d4f/f67 [0,4194304] 0 2024-07-03T15:44:04.986 INFO:tasks.workunit.client.0.smithi028.stdout:3/355: creat d18/d3a/f70 x:0 0 0 2024-07-03T15:44:04.986 INFO:tasks.workunit.client.0.smithi028.stdout:3/356: write f9 [385873,88632] 0 2024-07-03T15:44:04.987 INFO:tasks.workunit.client.0.smithi028.stdout:2/349: mknod da/d24/c86 0 2024-07-03T15:44:04.991 INFO:tasks.workunit.client.0.smithi028.stdout:4/320: rename d2/d12/d2d/d2f/f41 to d2/d12/d2d/d58/f69 0 2024-07-03T15:44:04.992 INFO:tasks.workunit.client.0.smithi028.stdout:4/321: fsync d2/d12/d3b/f48 0 2024-07-03T15:44:04.992 INFO:tasks.workunit.client.0.smithi028.stdout:4/322: fdatasync d2/f7 0 2024-07-03T15:44:05.002 INFO:tasks.workunit.client.0.smithi028.stdout:9/341: creat d1/d2a/f72 x:0 0 0 2024-07-03T15:44:05.003 INFO:tasks.workunit.client.0.smithi028.stdout:2/350: dwrite da/d23/d6f/f7c [0,4194304] 0 2024-07-03T15:44:05.003 INFO:tasks.workunit.client.0.smithi028.stdout:2/351: write da/d23/f3b [1066749,110180] 0 2024-07-03T15:44:05.004 INFO:tasks.workunit.client.0.smithi028.stdout:2/352: readlink da/d24/d2c/l48 0 2024-07-03T15:44:05.004 INFO:tasks.workunit.client.0.smithi028.stdout:2/353: write da/d24/d2c/f32 [1126116,42558] 0 2024-07-03T15:44:05.005 INFO:tasks.workunit.client.0.smithi028.stdout:5/378: rename d0/dd/df/l55 to d0/dd/d13/d47/d50/l7c 0 2024-07-03T15:44:05.005 INFO:tasks.workunit.client.0.smithi028.stdout:2/354: read da/d24/d2c/f3d [433727,43475] 0 2024-07-03T15:44:05.006 INFO:tasks.workunit.client.0.smithi028.stdout:4/323: mkdir d2/d12/d2d/d2f/d3a/d6a 0 2024-07-03T15:44:05.006 INFO:tasks.workunit.client.0.smithi028.stdout:4/324: write d2/d12/f60 [877591,64377] 0 2024-07-03T15:44:05.006 INFO:tasks.workunit.client.0.smithi028.stdout:4/325: fsync d2/d5a/f56 0 2024-07-03T15:44:05.016 INFO:tasks.workunit.client.0.smithi028.stdout:2/355: mknod da/d23/d6f/d84/c87 0 2024-07-03T15:44:05.017 INFO:tasks.workunit.client.0.smithi028.stdout:4/326: rmdir d2/d61/d62 0 2024-07-03T15:44:05.019 INFO:tasks.workunit.client.0.smithi028.stdout:3/357: rename d18/d3a/f47 to d18/d6f/f71 0 2024-07-03T15:44:05.021 INFO:tasks.workunit.client.0.smithi028.stdout:2/356: rename da/l41 to da/d23/d6f/l88 0 2024-07-03T15:44:05.021 INFO:tasks.workunit.client.0.smithi028.stdout:9/342: dwrite d1/d10/f18 [0,4194304] 0 2024-07-03T15:44:05.023 INFO:tasks.workunit.client.0.smithi028.stdout:5/379: dwrite d0/dd/d4b/f6a [0,4194304] 0 2024-07-03T15:44:05.023 INFO:tasks.workunit.client.0.smithi028.stdout:4/327: rename d2/f22 to d2/d12/d2e/f6b 0 2024-07-03T15:44:05.024 INFO:tasks.workunit.client.0.smithi028.stdout:2/357: truncate da/f15 1186660 0 2024-07-03T15:44:05.024 INFO:tasks.workunit.client.0.smithi028.stdout:2/358: readlink da/d1e/d56/l63 0 2024-07-03T15:44:05.024 INFO:tasks.workunit.client.0.smithi028.stdout:2/359: chown da/d23/d30 25 1 2024-07-03T15:44:05.029 INFO:tasks.workunit.client.0.smithi028.stdout:9/343: unlink d1/dd/f13 0 2024-07-03T15:44:05.029 INFO:tasks.workunit.client.0.smithi028.stdout:9/344: fdatasync d1/d17/f25 0 2024-07-03T15:44:05.034 INFO:tasks.workunit.client.0.smithi028.stdout:5/380: creat d0/dd/d13/d57/d77/f7d x:0 0 0 2024-07-03T15:44:05.035 INFO:tasks.workunit.client.0.smithi028.stdout:4/328: mkdir d2/d12/d42/d63/d6c 0 2024-07-03T15:44:05.035 INFO:tasks.workunit.client.0.smithi028.stdout:4/329: write d2/d12/f17 [931570,81572] 0 2024-07-03T15:44:05.039 INFO:tasks.workunit.client.0.smithi028.stdout:9/345: mknod d1/d16/d57/d5f/d64/c73 0 2024-07-03T15:44:05.040 INFO:tasks.workunit.client.0.smithi028.stdout:2/360: dwrite da/f10 [0,4194304] 0 2024-07-03T15:44:05.041 INFO:tasks.workunit.client.0.smithi028.stdout:5/381: rename d0/dd/f1b to d0/dd/f7e 0 2024-07-03T15:44:05.047 INFO:tasks.workunit.client.0.smithi028.stdout:3/358: dwrite d18/d3a/f3d [0,4194304] 0 2024-07-03T15:44:05.047 INFO:tasks.workunit.client.0.smithi028.stdout:3/359: chown d18/d1a/f28 58610 1 2024-07-03T15:44:05.047 INFO:tasks.workunit.client.0.smithi028.stdout:3/360: rename d18/d1a/d5e to d18/d1a/d5e/d72 22 2024-07-03T15:44:05.047 INFO:tasks.workunit.client.0.smithi028.stdout:3/361: dread - d18/f52 zero size 2024-07-03T15:44:05.047 INFO:tasks.workunit.client.0.smithi028.stdout:5/382: mkdir d0/dd/d13/d57/d30/d70/d7f 0 2024-07-03T15:44:05.053 INFO:tasks.workunit.client.0.smithi028.stdout:3/362: mkdir d18/d1a/d1e/d73 0 2024-07-03T15:44:05.057 INFO:tasks.workunit.client.0.smithi028.stdout:5/383: dread d0/dd/d13/d57/d26/f42 [0,4194304] 0 2024-07-03T15:44:05.058 INFO:tasks.workunit.client.0.smithi028.stdout:5/384: link d0/dd/df/f3e d0/dd/d1e/d3b/f80 0 2024-07-03T15:44:05.059 INFO:tasks.workunit.client.0.smithi028.stdout:6/279: sync 2024-07-03T15:44:05.059 INFO:tasks.workunit.client.0.smithi028.stdout:6/280: chown d4/d15/c53 6 1 2024-07-03T15:44:05.060 INFO:tasks.workunit.client.0.smithi028.stdout:5/385: creat d0/dd/d1e/d3b/d3f/d71/f81 x:0 0 0 2024-07-03T15:44:05.060 INFO:tasks.workunit.client.0.smithi028.stdout:5/386: stat d0/dd/d13/d57/d26/d41/f43 0 2024-07-03T15:44:05.060 INFO:tasks.workunit.client.0.smithi028.stdout:5/387: read - d0/dd/d13/d57/d77/f7d zero size 2024-07-03T15:44:05.061 INFO:tasks.workunit.client.0.smithi028.stdout:4/330: dwrite d2/f52 [4194304,4194304] 0 2024-07-03T15:44:05.061 INFO:tasks.workunit.client.0.smithi028.stdout:6/281: creat d4/da/d38/f5f x:0 0 0 2024-07-03T15:44:05.063 INFO:tasks.workunit.client.0.smithi028.stdout:4/331: rmdir d2/d5a 39 2024-07-03T15:44:05.063 INFO:tasks.workunit.client.0.smithi028.stdout:4/332: chown d2/d3e/f3f 0 1 2024-07-03T15:44:05.063 INFO:tasks.workunit.client.0.smithi028.stdout:5/388: dread d0/dd/d13/d57/d26/d41/f43 [0,4194304] 0 2024-07-03T15:44:05.063 INFO:tasks.workunit.client.0.smithi028.stdout:5/389: chown d0/f2c 59285146 1 2024-07-03T15:44:05.064 INFO:tasks.workunit.client.0.smithi028.stdout:2/361: dwrite da/d42/f46 [0,4194304] 0 2024-07-03T15:44:05.064 INFO:tasks.workunit.client.0.smithi028.stdout:2/362: chown da/d42/d47/f81 25702757 1 2024-07-03T15:44:05.065 INFO:tasks.workunit.client.0.smithi028.stdout:6/282: dread d4/d15/f2a [0,4194304] 0 2024-07-03T15:44:05.067 INFO:tasks.workunit.client.0.smithi028.stdout:2/363: dread da/f12 [0,4194304] 0 2024-07-03T15:44:05.067 INFO:tasks.workunit.client.0.smithi028.stdout:2/364: dread - da/d24/f66 zero size 2024-07-03T15:44:05.072 INFO:tasks.workunit.client.0.smithi028.stdout:3/363: dwrite d18/d6f/f71 [4194304,4194304] 0 2024-07-03T15:44:05.072 INFO:tasks.workunit.client.0.smithi028.stdout:2/365: dread da/d42/f50 [0,4194304] 0 2024-07-03T15:44:05.076 INFO:tasks.workunit.client.0.smithi028.stdout:4/333: dread d2/d5a/f24 [0,4194304] 0 2024-07-03T15:44:05.079 INFO:tasks.workunit.client.0.smithi028.stdout:6/283: read d4/da/d38/f40 [563773,68813] 0 2024-07-03T15:44:05.095 INFO:tasks.workunit.client.0.smithi028.stdout:3/364: dwrite d18/d1a/f4f [4194304,4194304] 0 2024-07-03T15:44:05.153 INFO:tasks.workunit.client.0.smithi028.stdout:6/284: mkdir d4/da/d38/d60 0 2024-07-03T15:44:05.155 INFO:tasks.workunit.client.0.smithi028.stdout:3/365: creat d18/d3a/f74 x:0 0 0 2024-07-03T15:44:05.179 INFO:tasks.workunit.client.0.smithi028.stdout:3/366: dwrite d18/d1a/f28 [0,4194304] 0 2024-07-03T15:44:05.180 INFO:tasks.workunit.client.0.smithi028.stdout:3/367: stat d18/d6f 0 2024-07-03T15:44:05.180 INFO:tasks.workunit.client.0.smithi028.stdout:3/368: fdatasync d18/f2d 0 2024-07-03T15:44:05.210 INFO:tasks.workunit.client.0.smithi028.stdout:4/334: unlink d2/d5a/c23 0 2024-07-03T15:44:05.212 INFO:tasks.workunit.client.0.smithi028.stdout:6/285: creat d4/da/d1c/f61 x:0 0 0 2024-07-03T15:44:05.218 INFO:tasks.workunit.client.0.smithi028.stdout:3/369: mknod d18/d1a/d64/c75 0 2024-07-03T15:44:05.218 INFO:tasks.workunit.client.0.smithi028.stdout:3/370: chown d18/d1a/d5e 23230777 1 2024-07-03T15:44:05.234 INFO:tasks.workunit.client.0.smithi028.stdout:2/366: rename da/d23/c3f to da/d23/c89 0 2024-07-03T15:44:05.235 INFO:tasks.workunit.client.0.smithi028.stdout:4/335: mkdir d2/d12/d42/d63/d6d 0 2024-07-03T15:44:05.238 INFO:tasks.workunit.client.0.smithi028.stdout:6/286: unlink d4/da/c1a 0 2024-07-03T15:44:05.238 INFO:tasks.workunit.client.0.smithi028.stdout:6/287: rename d4/da/d56 to d4/da/d56/d62 22 2024-07-03T15:44:05.239 INFO:tasks.workunit.client.0.smithi028.stdout:3/371: creat d18/d3a/f76 x:0 0 0 2024-07-03T15:44:05.241 INFO:tasks.workunit.client.0.smithi028.stdout:2/367: creat da/d23/f8a x:0 0 0 2024-07-03T15:44:05.241 INFO:tasks.workunit.client.0.smithi028.stdout:2/368: chown da/d1e/d56/f7e 125867539 1 2024-07-03T15:44:05.244 INFO:tasks.workunit.client.0.smithi028.stdout:6/288: rmdir d4/d12/d2c/d31 39 2024-07-03T15:44:05.248 INFO:tasks.workunit.client.0.smithi028.stdout:2/369: creat da/d23/d30/f8b x:0 0 0 2024-07-03T15:44:05.248 INFO:tasks.workunit.client.0.smithi028.stdout:2/370: read da/f12 [531030,11251] 0 2024-07-03T15:44:05.250 INFO:tasks.workunit.client.0.smithi028.stdout:6/289: mknod d4/d12/d2c/c63 0 2024-07-03T15:44:05.250 INFO:tasks.workunit.client.0.smithi028.stdout:6/290: write d4/d15/f5b [190543,63525] 0 2024-07-03T15:44:05.252 INFO:tasks.workunit.client.0.smithi028.stdout:2/371: creat da/d24/f8c x:0 0 0 2024-07-03T15:44:05.252 INFO:tasks.workunit.client.0.smithi028.stdout:2/372: dread - da/d23/f8a zero size 2024-07-03T15:44:05.254 INFO:tasks.workunit.client.0.smithi028.stdout:4/336: dwrite d2/d12/d3b/f48 [0,4194304] 0 2024-07-03T15:44:05.257 INFO:tasks.workunit.client.0.smithi028.stdout:2/373: creat da/d74/f8d x:0 0 0 2024-07-03T15:44:05.257 INFO:tasks.workunit.client.0.smithi028.stdout:2/374: dread - da/d42/d47/f81 zero size 2024-07-03T15:44:05.261 INFO:tasks.workunit.client.0.smithi028.stdout:4/337: write d2/d5a/f1f [2435431,94637] 0 2024-07-03T15:44:05.264 INFO:tasks.workunit.client.0.smithi028.stdout:6/291: getdents d4/da/d38/d3d 0 2024-07-03T15:44:05.267 INFO:tasks.workunit.client.0.smithi028.stdout:2/375: mknod da/d24/c8e 0 2024-07-03T15:44:05.269 INFO:tasks.workunit.client.0.smithi028.stdout:2/376: dread da/f1d [0,4194304] 0 2024-07-03T15:44:05.269 INFO:tasks.workunit.client.0.smithi028.stdout:2/377: read - da/d23/d30/f62 zero size 2024-07-03T15:44:05.270 INFO:tasks.workunit.client.0.smithi028.stdout:4/338: mknod d2/d61/c6e 0 2024-07-03T15:44:05.276 INFO:tasks.workunit.client.0.smithi028.stdout:6/292: creat d4/d12/d2c/d31/f64 x:0 0 0 2024-07-03T15:44:05.277 INFO:tasks.workunit.client.0.smithi028.stdout:2/378: getdents da/d24/d2c 0 2024-07-03T15:44:05.277 INFO:tasks.workunit.client.0.smithi028.stdout:2/379: write da/d23/d30/f5c [482684,125368] 0 2024-07-03T15:44:05.277 INFO:tasks.workunit.client.0.smithi028.stdout:2/380: stat da/d1e/d68/c6e 0 2024-07-03T15:44:05.279 INFO:tasks.workunit.client.0.smithi028.stdout:4/339: mknod d2/d12/d42/d63/d6c/c6f 0 2024-07-03T15:44:05.280 INFO:tasks.workunit.client.0.smithi028.stdout:4/340: chown d2/d12/d3b/f59 77 1 2024-07-03T15:44:05.280 INFO:tasks.workunit.client.0.smithi028.stdout:4/341: creat d2/d12/d2d/d58/f70 x:0 0 0 2024-07-03T15:44:05.284 INFO:tasks.workunit.client.0.smithi028.stdout:4/342: truncate d2/d3e/f50 1864737 0 2024-07-03T15:44:05.287 INFO:tasks.workunit.client.0.smithi028.stdout:2/381: mknod da/d23/c8f 0 2024-07-03T15:44:05.292 INFO:tasks.workunit.client.0.smithi028.stdout:4/343: dread d2/d12/d2e/f38 [0,4194304] 0 2024-07-03T15:44:05.292 INFO:tasks.workunit.client.0.smithi028.stdout:2/382: rmdir da/d74/d80 0 2024-07-03T15:44:05.292 INFO:tasks.workunit.client.0.smithi028.stdout:2/383: fsync da/d23/d30/f3e 0 2024-07-03T15:44:05.292 INFO:tasks.workunit.client.0.smithi028.stdout:4/344: creat d2/d12/d2d/d2f/d3a/d6a/f71 x:0 0 0 2024-07-03T15:44:05.294 INFO:tasks.workunit.client.0.smithi028.stdout:4/345: symlink d2/d5a/d45/l72 0 2024-07-03T15:44:05.294 INFO:tasks.workunit.client.0.smithi028.stdout:4/346: stat d2/l2a 0 2024-07-03T15:44:05.296 INFO:tasks.workunit.client.0.smithi028.stdout:4/347: write d2/d12/f28 [385471,129884] 0 2024-07-03T15:44:05.298 INFO:tasks.workunit.client.0.smithi028.stdout:6/293: dwrite d4/da/d13/f19 [0,4194304] 0 2024-07-03T15:44:05.305 INFO:tasks.workunit.client.0.smithi028.stdout:6/294: rmdir d4/da/d47/d5a 0 2024-07-03T15:44:05.308 INFO:tasks.workunit.client.0.smithi028.stdout:6/295: creat d4/da/d38/d59/f65 x:0 0 0 2024-07-03T15:44:05.311 INFO:tasks.workunit.client.0.smithi028.stdout:2/384: dwrite da/d23/d30/f62 [0,4194304] 0 2024-07-03T15:44:05.311 INFO:tasks.workunit.client.0.smithi028.stdout:2/385: readlink da/d23/d30/l70 0 2024-07-03T15:44:05.311 INFO:tasks.workunit.client.0.smithi028.stdout:2/386: dread - da/f85 zero size 2024-07-03T15:44:05.311 INFO:tasks.workunit.client.0.smithi028.stdout:2/387: readlink da/d24/d2c/l3c 0 2024-07-03T15:44:05.316 INFO:tasks.workunit.client.0.smithi028.stdout:4/348: dwrite d2/d12/d2d/d2f/d3a/d6a/f71 [0,4194304] 0 2024-07-03T15:44:05.327 INFO:tasks.workunit.client.0.smithi028.stdout:6/296: dwrite d4/da/d38/f46 [0,4194304] 0 2024-07-03T15:44:05.349 INFO:tasks.workunit.client.0.smithi028.stdout:6/297: dwrite f0 [0,4194304] 0 2024-07-03T15:44:05.349 INFO:tasks.workunit.client.0.smithi028.stdout:6/298: dread - d4/f45 zero size 2024-07-03T15:44:05.350 INFO:tasks.workunit.client.0.smithi028.stdout:6/299: mkdir d4/da/d47/d66 0 2024-07-03T15:44:05.350 INFO:tasks.workunit.client.0.smithi028.stdout:6/300: chown d4/d15/l25 200381 1 2024-07-03T15:44:05.351 INFO:tasks.workunit.client.0.smithi028.stdout:6/301: mknod d4/d15/c67 0 2024-07-03T15:44:05.353 INFO:tasks.workunit.client.0.smithi028.stdout:6/302: mkdir d4/d15/d68 0 2024-07-03T15:44:05.354 INFO:tasks.workunit.client.0.smithi028.stdout:6/303: mkdir d4/da/d38/d3d/d69 0 2024-07-03T15:44:05.356 INFO:tasks.workunit.client.0.smithi028.stdout:3/372: getdents d18/d1a/d64 0 2024-07-03T15:44:05.361 INFO:tasks.workunit.client.0.smithi028.stdout:7/188: sync 2024-07-03T15:44:05.361 INFO:tasks.workunit.client.0.smithi028.stdout:1/486: sync 2024-07-03T15:44:05.361 INFO:tasks.workunit.client.0.smithi028.stdout:0/245: sync 2024-07-03T15:44:05.363 INFO:tasks.workunit.client.0.smithi028.stdout:1/487: mkdir d4/d3c/d54/d79/d16/db6 0 2024-07-03T15:44:05.364 INFO:tasks.workunit.client.0.smithi028.stdout:1/488: write d4/d2c/f78 [1672414,29757] 0 2024-07-03T15:44:05.364 INFO:tasks.workunit.client.0.smithi028.stdout:1/489: readlink d4/d3c/d54/d79/l28 0 2024-07-03T15:44:05.364 INFO:tasks.workunit.client.0.smithi028.stdout:0/246: mkdir d2/d1c/d32/d36/d4a 0 2024-07-03T15:44:05.366 INFO:tasks.workunit.client.0.smithi028.stdout:3/373: write d18/d3a/f5a [633685,5714] 0 2024-07-03T15:44:05.366 INFO:tasks.workunit.client.0.smithi028.stdout:1/490: creat d4/de/d1c/fb7 x:0 0 0 2024-07-03T15:44:05.366 INFO:tasks.workunit.client.0.smithi028.stdout:1/491: fsync d4/de/f6f 0 2024-07-03T15:44:05.367 INFO:tasks.workunit.client.0.smithi028.stdout:3/374: creat d18/d3a/f77 x:0 0 0 2024-07-03T15:44:05.369 INFO:tasks.workunit.client.0.smithi028.stdout:1/492: symlink d4/de/d1c/d44/lb8 0 2024-07-03T15:44:05.369 INFO:tasks.workunit.client.0.smithi028.stdout:7/189: read f2 [6076149,51670] 0 2024-07-03T15:44:05.370 INFO:tasks.workunit.client.0.smithi028.stdout:6/304: dwrite d4/f45 [0,4194304] 0 2024-07-03T15:44:05.370 INFO:tasks.workunit.client.0.smithi028.stdout:6/305: stat d4/da/d38/d60 0 2024-07-03T15:44:05.370 INFO:tasks.workunit.client.0.smithi028.stdout:6/306: creat d4/d15/f6a x:0 0 0 2024-07-03T15:44:05.370 INFO:tasks.workunit.client.0.smithi028.stdout:7/190: chown d9/l14 416545 1 2024-07-03T15:44:05.373 INFO:tasks.workunit.client.0.smithi028.stdout:0/247: dread d2/d1c/f2d [0,4194304] 0 2024-07-03T15:44:05.373 INFO:tasks.workunit.client.0.smithi028.stdout:0/248: dread - d2/d1c/f2e zero size 2024-07-03T15:44:05.373 INFO:tasks.workunit.client.0.smithi028.stdout:7/191: creat d9/d29/f3c x:0 0 0 2024-07-03T15:44:05.380 INFO:tasks.workunit.client.0.smithi028.stdout:3/375: dread d18/d1a/f2a [0,4194304] 0 2024-07-03T15:44:05.382 INFO:tasks.workunit.client.0.smithi028.stdout:1/493: write d4/d2c/f92 [2820712,106395] 0 2024-07-03T15:44:05.383 INFO:tasks.workunit.client.0.smithi028.stdout:1/494: write d4/d3c/d54/d79/f8e [223810,82434] 0 2024-07-03T15:44:05.384 INFO:tasks.workunit.client.0.smithi028.stdout:0/249: dread d2/d1c/d32/f38 [0,4194304] 0 2024-07-03T15:44:05.385 INFO:tasks.workunit.client.0.smithi028.stdout:0/250: write d2/d3/f25 [255784,60004] 0 2024-07-03T15:44:05.387 INFO:tasks.workunit.client.0.smithi028.stdout:8/354: sync 2024-07-03T15:44:05.387 INFO:tasks.workunit.client.0.smithi028.stdout:6/307: read d4/da/d38/f5e [163187,92121] 0 2024-07-03T15:44:05.387 INFO:tasks.workunit.client.0.smithi028.stdout:6/308: creat d4/da/d47/f6b x:0 0 0 2024-07-03T15:44:05.387 INFO:tasks.workunit.client.0.smithi028.stdout:6/309: chown d4/da/d1c/l27 60021 1 2024-07-03T15:44:05.388 INFO:tasks.workunit.client.0.smithi028.stdout:8/355: creat d0/d41/d50/d3/dc/d11/d5d/f72 x:0 0 0 2024-07-03T15:44:05.390 INFO:tasks.workunit.client.0.smithi028.stdout:7/192: dread d9/f1a [0,4194304] 0 2024-07-03T15:44:05.390 INFO:tasks.workunit.client.0.smithi028.stdout:7/193: write d9/d29/f31 [381843,75031] 0 2024-07-03T15:44:05.393 INFO:tasks.workunit.client.0.smithi028.stdout:7/194: stat d9/l32 0 2024-07-03T15:44:05.395 INFO:tasks.workunit.client.0.smithi028.stdout:3/376: rmdir d18/d1a 39 2024-07-03T15:44:05.396 INFO:tasks.workunit.client.0.smithi028.stdout:1/495: write d4/de/f1f [805929,69171] 0 2024-07-03T15:44:05.396 INFO:tasks.workunit.client.0.smithi028.stdout:1/496: chown d4/d2c/d3a/f8f 1181 1 2024-07-03T15:44:05.404 INFO:tasks.workunit.client.0.smithi028.stdout:1/497: rename f2 to d4/d2c/d3a/d5c/d6c/d7d/d4e/fb9 0 2024-07-03T15:44:05.406 INFO:tasks.workunit.client.0.smithi028.stdout:1/498: dread d4/de/f6f [0,4194304] 0 2024-07-03T15:44:05.407 INFO:tasks.workunit.client.0.smithi028.stdout:7/195: dread d9/f10 [0,4194304] 0 2024-07-03T15:44:05.407 INFO:tasks.workunit.client.0.smithi028.stdout:7/196: chown d9/l14 22482 1 2024-07-03T15:44:05.411 INFO:tasks.workunit.client.0.smithi028.stdout:1/499: symlink d4/de/d1c/d81/lba 0 2024-07-03T15:44:05.414 INFO:tasks.workunit.client.0.smithi028.stdout:0/251: dwrite d2/d3/f11 [0,4194304] 0 2024-07-03T15:44:05.414 INFO:tasks.workunit.client.0.smithi028.stdout:0/252: chown d2 277378292 1 2024-07-03T15:44:05.414 INFO:tasks.workunit.client.0.smithi028.stdout:5/390: sync 2024-07-03T15:44:05.415 INFO:tasks.workunit.client.0.smithi028.stdout:9/346: sync 2024-07-03T15:44:05.415 INFO:tasks.workunit.client.0.smithi028.stdout:5/391: write d0/dd/df/f52 [787403,87112] 0 2024-07-03T15:44:05.415 INFO:tasks.workunit.client.0.smithi028.stdout:5/392: fsync d0/dd/d13/d47/f5b 0 2024-07-03T15:44:05.416 INFO:tasks.workunit.client.0.smithi028.stdout:9/347: symlink d1/d10/d28/d45/l74 0 2024-07-03T15:44:05.419 INFO:tasks.workunit.client.0.smithi028.stdout:9/348: creat d1/d16/d57/d5f/d64/f75 x:0 0 0 2024-07-03T15:44:05.420 INFO:tasks.workunit.client.0.smithi028.stdout:9/349: mkdir d1/d10/d76 0 2024-07-03T15:44:05.421 INFO:tasks.workunit.client.0.smithi028.stdout:9/350: truncate d1/dd/f55 674239 0 2024-07-03T15:44:05.421 INFO:tasks.workunit.client.0.smithi028.stdout:3/377: dwrite d18/f52 [0,4194304] 0 2024-07-03T15:44:05.422 INFO:tasks.workunit.client.0.smithi028.stdout:5/393: write d0/dd/df/f16 [2441359,120146] 0 2024-07-03T15:44:05.424 INFO:tasks.workunit.client.0.smithi028.stdout:3/378: getdents d18/d1a 0 2024-07-03T15:44:05.425 INFO:tasks.workunit.client.0.smithi028.stdout:0/253: read d2/d1c/f2d [3679659,79558] 0 2024-07-03T15:44:05.425 INFO:tasks.workunit.client.0.smithi028.stdout:3/379: write d18/d3a/f3f [5093273,74686] 0 2024-07-03T15:44:05.428 INFO:tasks.workunit.client.0.smithi028.stdout:0/254: mknod d2/d1c/d32/d36/c4b 0 2024-07-03T15:44:05.428 INFO:tasks.workunit.client.0.smithi028.stdout:0/255: chown d2/d3/l13 7454 1 2024-07-03T15:44:05.428 INFO:tasks.workunit.client.0.smithi028.stdout:0/256: fdatasync d2/d3/f11 0 2024-07-03T15:44:05.428 INFO:tasks.workunit.client.0.smithi028.stdout:0/257: stat d2/d3/fe 0 2024-07-03T15:44:05.435 INFO:tasks.workunit.client.0.smithi028.stdout:7/197: dwrite d9/d29/f35 [0,4194304] 0 2024-07-03T15:44:05.437 INFO:tasks.workunit.client.0.smithi028.stdout:9/351: dread d1/d10/f18 [0,4194304] 0 2024-07-03T15:44:05.438 INFO:tasks.workunit.client.0.smithi028.stdout:3/380: dread d18/d1a/f20 [0,4194304] 0 2024-07-03T15:44:05.439 INFO:tasks.workunit.client.0.smithi028.stdout:1/500: dwrite d4/d2c/d3a/d5c/d6c/d7d/d59/f6a [0,4194304] 0 2024-07-03T15:44:05.442 INFO:tasks.workunit.client.0.smithi028.stdout:5/394: dread d0/dd/df/f52 [0,4194304] 0 2024-07-03T15:44:05.442 INFO:tasks.workunit.client.0.smithi028.stdout:9/352: unlink d1/f9 0 2024-07-03T15:44:05.444 INFO:tasks.workunit.client.0.smithi028.stdout:3/381: mkdir d18/d6f/d78 0 2024-07-03T15:44:05.452 INFO:tasks.workunit.client.0.smithi028.stdout:4/349: rename d2/f20 to d2/f73 0 2024-07-03T15:44:05.453 INFO:tasks.workunit.client.0.smithi028.stdout:1/501: write d4/d2c/d3a/d5c/d6c/d7d/d59/f6a [1695197,51558] 0 2024-07-03T15:44:05.453 INFO:tasks.workunit.client.0.smithi028.stdout:9/353: mkdir d1/d2a/d77 0 2024-07-03T15:44:05.453 INFO:tasks.workunit.client.0.smithi028.stdout:0/258: dwrite d2/d1c/f21 [0,4194304] 0 2024-07-03T15:44:05.454 INFO:tasks.workunit.client.0.smithi028.stdout:3/382: creat d18/d1a/d5e/d66/f79 x:0 0 0 2024-07-03T15:44:05.466 INFO:tasks.workunit.client.0.smithi028.stdout:5/395: dwrite d0/dd/f7e [0,4194304] 0 2024-07-03T15:44:05.467 INFO:tasks.workunit.client.0.smithi028.stdout:7/198: dwrite d9/f25 [0,4194304] 0 2024-07-03T15:44:05.473 INFO:tasks.workunit.client.0.smithi028.stdout:4/350: mkdir d2/d12/d42/d63/d6c/d74 0 2024-07-03T15:44:05.475 INFO:tasks.workunit.client.0.smithi028.stdout:9/354: rename d1/d10/d28/d45/l74 to d1/d10/d76/l78 0 2024-07-03T15:44:05.475 INFO:tasks.workunit.client.0.smithi028.stdout:9/355: dread - d1/dd/f6f zero size 2024-07-03T15:44:05.476 INFO:tasks.workunit.client.0.smithi028.stdout:1/502: getdents d4/d2c/d3a/d5c/d6c 0 2024-07-03T15:44:05.478 INFO:tasks.workunit.client.0.smithi028.stdout:0/259: truncate d2/d1c/d32/d36/d45/f46 3729591 0 2024-07-03T15:44:05.480 INFO:tasks.workunit.client.0.smithi028.stdout:3/383: link d18/c44 d18/d1a/d5e/d66/c7a 0 2024-07-03T15:44:05.480 INFO:tasks.workunit.client.0.smithi028.stdout:3/384: fsync f14 0 2024-07-03T15:44:05.482 INFO:tasks.workunit.client.0.smithi028.stdout:5/396: mknod d0/dd/d1e/d3b/d3f/d5a/c82 0 2024-07-03T15:44:05.489 INFO:tasks.workunit.client.0.smithi028.stdout:1/503: dread d4/d2c/d3a/d43/f77 [0,4194304] 0 2024-07-03T15:44:05.489 INFO:tasks.workunit.client.0.smithi028.stdout:4/351: dwrite f1 [4194304,4194304] 0 2024-07-03T15:44:05.490 INFO:tasks.workunit.client.0.smithi028.stdout:4/352: truncate d2/f73 878859 0 2024-07-03T15:44:05.494 INFO:tasks.workunit.client.0.smithi028.stdout:4/353: dread d2/d5a/f36 [0,4194304] 0 2024-07-03T15:44:05.498 INFO:tasks.workunit.client.0.smithi028.stdout:9/356: rename d1/d10/d28/d45/d61/f70 to d1/d10/d28/f79 0 2024-07-03T15:44:05.500 INFO:tasks.workunit.client.0.smithi028.stdout:7/199: mknod d9/c3d 0 2024-07-03T15:44:05.502 INFO:tasks.workunit.client.0.smithi028.stdout:4/354: mkdir d2/d12/d42/d63/d75 0 2024-07-03T15:44:05.502 INFO:tasks.workunit.client.0.smithi028.stdout:4/355: chown d2/d5a/c64 6680 1 2024-07-03T15:44:05.504 INFO:tasks.workunit.client.0.smithi028.stdout:7/200: rename d9/d29/f33 to d9/d29/f3e 0 2024-07-03T15:44:05.505 INFO:tasks.workunit.client.0.smithi028.stdout:4/356: creat d2/d12/d2c/f76 x:0 0 0 2024-07-03T15:44:05.508 INFO:tasks.workunit.client.0.smithi028.stdout:7/201: rename f2 to d9/d1c/f3f 0 2024-07-03T15:44:05.508 INFO:tasks.workunit.client.0.smithi028.stdout:7/202: chown d9/d29/f2e 3838 1 2024-07-03T15:44:05.509 INFO:tasks.workunit.client.0.smithi028.stdout:7/203: creat d9/d1c/f40 x:0 0 0 2024-07-03T15:44:05.509 INFO:tasks.workunit.client.0.smithi028.stdout:7/204: chown d9/d29/f2e 138 1 2024-07-03T15:44:05.510 INFO:tasks.workunit.client.0.smithi028.stdout:7/205: chown d9/l16 2377820 1 2024-07-03T15:44:05.520 INFO:tasks.workunit.client.0.smithi028.stdout:0/260: dwrite d2/f3b [0,4194304] 0 2024-07-03T15:44:05.524 INFO:tasks.workunit.client.0.smithi028.stdout:4/357: write d2/f52 [4339604,104254] 0 2024-07-03T15:44:05.525 INFO:tasks.workunit.client.0.smithi028.stdout:0/261: rename d2/d3/f27 to d2/d3/f4c 0 2024-07-03T15:44:05.526 INFO:tasks.workunit.client.0.smithi028.stdout:4/358: mknod d2/d61/c77 0 2024-07-03T15:44:05.526 INFO:tasks.workunit.client.0.smithi028.stdout:4/359: write d2/d12/d2d/d2f/d3a/d6a/f71 [5149081,92912] 0 2024-07-03T15:44:05.527 INFO:tasks.workunit.client.0.smithi028.stdout:5/397: dwrite d0/dd/d13/d29/f68 [0,4194304] 0 2024-07-03T15:44:05.527 INFO:tasks.workunit.client.0.smithi028.stdout:5/398: write d0/dd/d13/d57/d77/f7d [475915,9817] 0 2024-07-03T15:44:05.528 INFO:tasks.workunit.client.0.smithi028.stdout:5/399: write d0/dd/d1e/d3b/f80 [4205482,110561] 0 2024-07-03T15:44:05.529 INFO:tasks.workunit.client.0.smithi028.stdout:3/385: dwrite d18/d1a/f20 [0,4194304] 0 2024-07-03T15:44:05.529 INFO:tasks.workunit.client.0.smithi028.stdout:3/386: chown d18/f51 1500 1 2024-07-03T15:44:05.530 INFO:tasks.workunit.client.0.smithi028.stdout:3/387: dread f9 [0,4194304] 0 2024-07-03T15:44:05.531 INFO:tasks.workunit.client.0.smithi028.stdout:3/388: creat d18/d1a/d5e/f7b x:0 0 0 2024-07-03T15:44:05.531 INFO:tasks.workunit.client.0.smithi028.stdout:3/389: readlink d18/d1a/d1e/l6a 0 2024-07-03T15:44:05.533 INFO:tasks.workunit.client.0.smithi028.stdout:7/206: write d9/f1a [1159172,127619] 0 2024-07-03T15:44:05.533 INFO:tasks.workunit.client.0.smithi028.stdout:1/504: dwrite d4/d3c/d54/d79/d16/d57/d66/f67 [0,4194304] 0 2024-07-03T15:44:05.540 INFO:tasks.workunit.client.0.smithi028.stdout:4/360: read d2/d12/d2e/f6b [2613747,1093] 0 2024-07-03T15:44:05.540 INFO:tasks.workunit.client.0.smithi028.stdout:4/361: fdatasync d2/d12/d2d/d2f/d3a/d6a/f71 0 2024-07-03T15:44:05.541 INFO:tasks.workunit.client.0.smithi028.stdout:4/362: write d2/f7 [483893,48875] 0 2024-07-03T15:44:05.542 INFO:tasks.workunit.client.0.smithi028.stdout:0/262: mkdir d2/d3/d4d 0 2024-07-03T15:44:05.547 INFO:tasks.workunit.client.0.smithi028.stdout:3/390: dwrite f17 [0,4194304] 0 2024-07-03T15:44:05.547 INFO:tasks.workunit.client.0.smithi028.stdout:3/391: fdatasync d18/d1a/d1e/f38 0 2024-07-03T15:44:05.547 INFO:tasks.workunit.client.0.smithi028.stdout:1/505: rmdir d4/de/d1c/d44 39 2024-07-03T15:44:05.549 INFO:tasks.workunit.client.0.smithi028.stdout:7/207: creat d9/f41 x:0 0 0 2024-07-03T15:44:05.550 INFO:tasks.workunit.client.0.smithi028.stdout:4/363: dread d2/f52 [0,4194304] 0 2024-07-03T15:44:05.550 INFO:tasks.workunit.client.0.smithi028.stdout:4/364: fdatasync d2/d3e/f3f 0 2024-07-03T15:44:05.550 INFO:tasks.workunit.client.0.smithi028.stdout:4/365: write d2/d3e/f3f [118740,12414] 0 2024-07-03T15:44:05.551 INFO:tasks.workunit.client.0.smithi028.stdout:4/366: chown d2/d5a/f1f 12280345 1 2024-07-03T15:44:05.554 INFO:tasks.workunit.client.0.smithi028.stdout:0/263: rmdir d2 39 2024-07-03T15:44:05.554 INFO:tasks.workunit.client.0.smithi028.stdout:0/264: write d2/d1c/f2b [530449,57803] 0 2024-07-03T15:44:05.554 INFO:tasks.workunit.client.0.smithi028.stdout:1/506: stat d4/d2c/d3a/d5c/d6c/d7d/d4e/d51/l6b 0 2024-07-03T15:44:05.557 INFO:tasks.workunit.client.0.smithi028.stdout:7/208: write d9/fa [615924,60366] 0 2024-07-03T15:44:05.564 INFO:tasks.workunit.client.0.smithi028.stdout:3/392: dwrite d18/d1a/f4f [0,4194304] 0 2024-07-03T15:44:05.565 INFO:tasks.workunit.client.0.smithi028.stdout:1/507: symlink d4/d3c/d54/d84/lbb 0 2024-07-03T15:44:05.565 INFO:tasks.workunit.client.0.smithi028.stdout:1/508: chown d4/f37 99733 1 2024-07-03T15:44:05.566 INFO:tasks.workunit.client.0.smithi028.stdout:4/367: getdents d2/d12/d2d/d58 0 2024-07-03T15:44:05.569 INFO:tasks.workunit.client.0.smithi028.stdout:1/509: rename d4/d2c/d3a/f8f to d4/d2c/d3a/d5c/d6c/d7d/d4e/fbc 0 2024-07-03T15:44:05.571 INFO:tasks.workunit.client.0.smithi028.stdout:4/368: mkdir d2/d12/d5b/d78 0 2024-07-03T15:44:05.573 INFO:tasks.workunit.client.0.smithi028.stdout:4/369: unlink d2/d12/d3b/f59 0 2024-07-03T15:44:05.573 INFO:tasks.workunit.client.0.smithi028.stdout:4/370: stat d2/d5a/f34 0 2024-07-03T15:44:05.580 INFO:tasks.workunit.client.0.smithi028.stdout:0/265: dwrite d2/d3/f4c [0,4194304] 0 2024-07-03T15:44:05.582 INFO:tasks.workunit.client.0.smithi028.stdout:0/266: truncate d2/d3/f11 4055873 0 2024-07-03T15:44:05.583 INFO:tasks.workunit.client.0.smithi028.stdout:0/267: mknod d2/c4e 0 2024-07-03T15:44:05.606 INFO:tasks.workunit.client.0.smithi028.stdout:3/393: dwrite f17 [0,4194304] 0 2024-07-03T15:44:05.606 INFO:tasks.workunit.client.0.smithi028.stdout:1/510: dwrite d4/de/f1f [0,4194304] 0 2024-07-03T15:44:05.606 INFO:tasks.workunit.client.0.smithi028.stdout:0/268: dwrite d2/d1c/f20 [0,4194304] 0 2024-07-03T15:44:05.606 INFO:tasks.workunit.client.0.smithi028.stdout:4/371: dwrite d2/f1b [0,4194304] 0 2024-07-03T15:44:05.606 INFO:tasks.workunit.client.0.smithi028.stdout:3/394: symlink d18/d1a/l7c 0 2024-07-03T15:44:05.611 INFO:tasks.workunit.client.0.smithi028.stdout:1/511: creat d4/de/d1c/d44/fbd x:0 0 0 2024-07-03T15:44:05.616 INFO:tasks.workunit.client.0.smithi028.stdout:3/395: creat d18/d6f/f7d x:0 0 0 2024-07-03T15:44:05.623 INFO:tasks.workunit.client.0.smithi028.stdout:3/396: dread d18/d3a/f3f [4194304,4194304] 0 2024-07-03T15:44:05.632 INFO:tasks.workunit.client.0.smithi028.stdout:3/397: dread f6 [0,4194304] 0 2024-07-03T15:44:05.632 INFO:tasks.workunit.client.0.smithi028.stdout:3/398: truncate f6 2662584 0 2024-07-03T15:44:05.634 INFO:tasks.workunit.client.0.smithi028.stdout:3/399: truncate d18/f52 3402798 0 2024-07-03T15:44:05.634 INFO:tasks.workunit.client.0.smithi028.stdout:3/400: creat d18/d1a/d1e/d58/f7e x:0 0 0 2024-07-03T15:44:05.634 INFO:tasks.workunit.client.0.smithi028.stdout:3/401: creat d18/d1a/d1e/d58/f7f x:0 0 0 2024-07-03T15:44:05.639 INFO:tasks.workunit.client.0.smithi028.stdout:0/269: dwrite d2/d1c/f2b [0,4194304] 0 2024-07-03T15:44:05.639 INFO:tasks.workunit.client.0.smithi028.stdout:3/402: write d18/d1a/f2a [3370415,21884] 0 2024-07-03T15:44:05.641 INFO:tasks.workunit.client.0.smithi028.stdout:4/372: dwrite d2/d5a/f56 [0,4194304] 0 2024-07-03T15:44:05.641 INFO:tasks.workunit.client.0.smithi028.stdout:4/373: dread - d2/d12/d3b/f4a zero size 2024-07-03T15:44:05.641 INFO:tasks.workunit.client.0.smithi028.stdout:1/512: dwrite d4/de/d1c/d44/fbd [0,4194304] 0 2024-07-03T15:44:05.644 INFO:tasks.workunit.client.0.smithi028.stdout:4/374: creat d2/d12/d5b/d78/f79 x:0 0 0 2024-07-03T15:44:05.647 INFO:tasks.workunit.client.0.smithi028.stdout:0/270: link d2/ff d2/d3/f4f 0 2024-07-03T15:44:05.650 INFO:tasks.workunit.client.0.smithi028.stdout:4/375: creat d2/d12/d5b/f7a x:0 0 0 2024-07-03T15:44:05.650 INFO:tasks.workunit.client.0.smithi028.stdout:4/376: read d2/d12/f26 [576981,54304] 0 2024-07-03T15:44:05.652 INFO:tasks.workunit.client.0.smithi028.stdout:3/403: dread d18/d1a/d1e/f38 [0,4194304] 0 2024-07-03T15:44:05.652 INFO:tasks.workunit.client.0.smithi028.stdout:3/404: write d18/f51 [766130,88835] 0 2024-07-03T15:44:05.653 INFO:tasks.workunit.client.0.smithi028.stdout:4/377: creat d2/d12/d42/d63/d6c/f7b x:0 0 0 2024-07-03T15:44:05.656 INFO:tasks.workunit.client.0.smithi028.stdout:0/271: dread d2/d3/f11 [0,4194304] 0 2024-07-03T15:44:05.657 INFO:tasks.workunit.client.0.smithi028.stdout:4/378: creat d2/d5a/d45/f7c x:0 0 0 2024-07-03T15:44:05.657 INFO:tasks.workunit.client.0.smithi028.stdout:4/379: creat d2/d12/d2c/f7d x:0 0 0 2024-07-03T15:44:05.660 INFO:tasks.workunit.client.0.smithi028.stdout:0/272: symlink d2/d1c/d32/d36/l50 0 2024-07-03T15:44:05.660 INFO:tasks.workunit.client.0.smithi028.stdout:0/273: fsync d2/d3/f4c 0 2024-07-03T15:44:05.660 INFO:tasks.workunit.client.0.smithi028.stdout:3/405: dread d18/d1a/f4f [4194304,4194304] 0 2024-07-03T15:44:05.663 INFO:tasks.workunit.client.0.smithi028.stdout:0/274: mkdir d2/d1c/d32/d36/d51 0 2024-07-03T15:44:05.663 INFO:tasks.workunit.client.0.smithi028.stdout:0/275: fsync d2/f15 0 2024-07-03T15:44:05.663 INFO:tasks.workunit.client.0.smithi028.stdout:0/276: fsync d2/d1c/d32/f39 0 2024-07-03T15:44:05.665 INFO:tasks.workunit.client.0.smithi028.stdout:3/406: rename d18/l19 to d18/d6f/d78/l80 0 2024-07-03T15:44:05.667 INFO:tasks.workunit.client.0.smithi028.stdout:3/407: dread d18/d1a/d1e/f3e [0,4194304] 0 2024-07-03T15:44:05.667 INFO:tasks.workunit.client.0.smithi028.stdout:3/408: fdatasync d18/d1a/d1e/f41 0 2024-07-03T15:44:05.667 INFO:tasks.workunit.client.0.smithi028.stdout:0/277: symlink d2/d3/l52 0 2024-07-03T15:44:05.668 INFO:tasks.workunit.client.0.smithi028.stdout:3/409: mknod d18/d1a/d1e/d58/c81 0 2024-07-03T15:44:05.668 INFO:tasks.workunit.client.0.smithi028.stdout:3/410: creat d18/d1a/d1e/f82 x:0 0 0 2024-07-03T15:44:05.668 INFO:tasks.workunit.client.0.smithi028.stdout:3/411: chown d18/c44 33185815 1 2024-07-03T15:44:05.669 INFO:tasks.workunit.client.0.smithi028.stdout:0/278: symlink d2/d1c/d32/d36/l53 0 2024-07-03T15:44:05.670 INFO:tasks.workunit.client.0.smithi028.stdout:3/412: mkdir d18/d1a/d1e/d83 0 2024-07-03T15:44:05.672 INFO:tasks.workunit.client.0.smithi028.stdout:3/413: truncate f14 329195 0 2024-07-03T15:44:05.672 INFO:tasks.workunit.client.0.smithi028.stdout:3/414: fdatasync fe 0 2024-07-03T15:44:05.684 INFO:tasks.workunit.client.0.smithi028.stdout:3/415: getdents d18/d1a/d1e/d58 0 2024-07-03T15:44:05.684 INFO:tasks.workunit.client.0.smithi028.stdout:3/416: fsync d18/d3a/f74 0 2024-07-03T15:44:05.684 INFO:tasks.workunit.client.0.smithi028.stdout:3/417: chown d18/d6f/f7d 2387 1 2024-07-03T15:44:05.684 INFO:tasks.workunit.client.0.smithi028.stdout:3/418: stat d18/d1a/d5e/f7b 0 2024-07-03T15:44:05.684 INFO:tasks.workunit.client.0.smithi028.stdout:3/419: chown d18/d1a/d1e/d58/l59 38560 1 2024-07-03T15:44:05.685 INFO:tasks.workunit.client.0.smithi028.stdout:3/420: rmdir d18/d1a/d1e/d58 39 2024-07-03T15:44:05.687 INFO:tasks.workunit.client.0.smithi028.stdout:3/421: mknod d18/d1a/d5e/d60/c84 0 2024-07-03T15:44:05.687 INFO:tasks.workunit.client.0.smithi028.stdout:3/422: getdents d18/d1a/d1e/d73 0 2024-07-03T15:44:05.689 INFO:tasks.workunit.client.0.smithi028.stdout:3/423: symlink d18/d1a/d1e/d73/l85 0 2024-07-03T15:44:05.690 INFO:tasks.workunit.client.0.smithi028.stdout:0/279: dwrite d2/f15 [0,4194304] 0 2024-07-03T15:44:05.692 INFO:tasks.workunit.client.0.smithi028.stdout:0/280: link d2/d3/l22 d2/d1c/d32/l54 0 2024-07-03T15:44:05.693 INFO:tasks.workunit.client.0.smithi028.stdout:3/424: write d18/d6f/f71 [2898846,33921] 0 2024-07-03T15:44:05.699 INFO:tasks.workunit.client.0.smithi028.stdout:3/425: unlink d18/d1a/d1e/l54 0 2024-07-03T15:44:05.724 INFO:tasks.workunit.client.0.smithi028.stdout:0/281: dwrite d2/d3/f1d [0,4194304] 0 2024-07-03T15:44:05.724 INFO:tasks.workunit.client.0.smithi028.stdout:3/426: dwrite d18/d1a/d1e/f57 [0,4194304] 0 2024-07-03T15:44:05.732 INFO:tasks.workunit.client.0.smithi028.stdout:0/282: write d2/d1c/d32/d36/f3f [3627597,50922] 0 2024-07-03T15:44:05.734 INFO:tasks.workunit.client.0.smithi028.stdout:0/283: getdents d2 0 2024-07-03T15:44:05.734 INFO:tasks.workunit.client.0.smithi028.stdout:0/284: chown d2/d3/c10 7 1 2024-07-03T15:44:05.734 INFO:tasks.workunit.client.0.smithi028.stdout:0/285: readlink d2/d1c/d32/d36/l40 0 2024-07-03T15:44:05.734 INFO:tasks.workunit.client.0.smithi028.stdout:3/427: write d18/f2d [6278316,18657] 0 2024-07-03T15:44:05.737 INFO:tasks.workunit.client.0.smithi028.stdout:0/286: dread d2/d3/fe [0,4194304] 0 2024-07-03T15:44:05.741 INFO:tasks.workunit.client.0.smithi028.stdout:0/287: rename d2/d1c/d32/d36/f3f to d2/f55 0 2024-07-03T15:44:05.743 INFO:tasks.workunit.client.0.smithi028.stdout:0/288: mkdir d2/d3/d4d/d56 0 2024-07-03T15:44:05.745 INFO:tasks.workunit.client.0.smithi028.stdout:0/289: creat d2/d3/d4d/f57 x:0 0 0 2024-07-03T15:44:05.746 INFO:tasks.workunit.client.0.smithi028.stdout:0/290: unlink d2/d3/f4f 0 2024-07-03T15:44:05.747 INFO:tasks.workunit.client.0.smithi028.stdout:0/291: mkdir d2/d1c/d32/d58 0 2024-07-03T15:44:05.749 INFO:tasks.workunit.client.0.smithi028.stdout:0/292: mknod d2/d1c/d32/d58/c59 0 2024-07-03T15:44:05.749 INFO:tasks.workunit.client.0.smithi028.stdout:0/293: chown d2/d1c/l48 2885 1 2024-07-03T15:44:05.749 INFO:tasks.workunit.client.0.smithi028.stdout:0/294: truncate d2/f3b 359967 0 2024-07-03T15:44:05.749 INFO:tasks.workunit.client.0.smithi028.stdout:0/295: getdents d2/d3/d4d/d56 0 2024-07-03T15:44:05.753 INFO:tasks.workunit.client.0.smithi028.stdout:3/428: dwrite d18/d3a/f70 [0,4194304] 0 2024-07-03T15:44:05.755 INFO:tasks.workunit.client.0.smithi028.stdout:3/429: rename d18/d1a/d64/c75 to d18/d6f/c86 0 2024-07-03T15:44:05.757 INFO:tasks.workunit.client.0.smithi028.stdout:3/430: mkdir d18/d6f/d78/d87 0 2024-07-03T15:44:05.758 INFO:tasks.workunit.client.0.smithi028.stdout:3/431: creat d18/d1a/d5e/d60/f88 x:0 0 0 2024-07-03T15:44:05.759 INFO:tasks.workunit.client.0.smithi028.stdout:3/432: getdents d18/d1a/d1e/d73 0 2024-07-03T15:44:05.760 INFO:tasks.workunit.client.0.smithi028.stdout:0/296: rmdir d2/d3/d4d 39 2024-07-03T15:44:05.762 INFO:tasks.workunit.client.0.smithi028.stdout:0/297: creat d2/d1c/d32/d58/f5a x:0 0 0 2024-07-03T15:44:05.768 INFO:tasks.workunit.client.0.smithi028.stdout:3/433: write d18/d1a/f4f [4567031,52523] 0 2024-07-03T15:44:05.771 INFO:tasks.workunit.client.0.smithi028.stdout:3/434: symlink d18/d1a/d1e/d83/l89 0 2024-07-03T15:44:05.773 INFO:tasks.workunit.client.0.smithi028.stdout:0/298: dread d2/ff [0,4194304] 0 2024-07-03T15:44:05.775 INFO:tasks.workunit.client.0.smithi028.stdout:0/299: creat d2/d1c/d32/d36/f5b x:0 0 0 2024-07-03T15:44:05.780 INFO:tasks.workunit.client.0.smithi028.stdout:0/300: rmdir d2/d1c/d32/d58 39 2024-07-03T15:44:05.782 INFO:tasks.workunit.client.0.smithi028.stdout:0/301: dread d2/d1c/f1e [0,4194304] 0 2024-07-03T15:44:05.788 INFO:tasks.workunit.client.0.smithi028.stdout:0/302: rename d2/d1c/l42 to d2/d3/l5c 0 2024-07-03T15:44:05.791 INFO:tasks.workunit.client.0.smithi028.stdout:0/303: stat d2/d1c/d32/l54 0 2024-07-03T15:44:05.791 INFO:tasks.workunit.client.0.smithi028.stdout:0/304: fsync d2/f49 0 2024-07-03T15:44:05.791 INFO:tasks.workunit.client.0.smithi028.stdout:0/305: write d2/d1c/d32/d36/f5b [837522,38439] 0 2024-07-03T15:44:05.795 INFO:tasks.workunit.client.0.smithi028.stdout:0/306: mknod d2/d3/c5d 0 2024-07-03T15:44:05.796 INFO:tasks.workunit.client.0.smithi028.stdout:0/307: write d2/d1c/f2d [4476816,96000] 0 2024-07-03T15:44:05.812 INFO:tasks.workunit.client.0.smithi028.stdout:0/308: dwrite d2/d1c/d32/d36/d45/f46 [0,4194304] 0 2024-07-03T15:44:05.812 INFO:tasks.workunit.client.0.smithi028.stdout:0/309: write d2/f3b [19373,63062] 0 2024-07-03T15:44:05.815 INFO:tasks.workunit.client.0.smithi028.stdout:0/310: link d2/d3/l5c d2/l5e 0 2024-07-03T15:44:05.816 INFO:tasks.workunit.client.0.smithi028.stdout:0/311: creat d2/d1c/d32/d58/f5f x:0 0 0 2024-07-03T15:44:05.816 INFO:tasks.workunit.client.0.smithi028.stdout:0/312: write d2/d3/d4d/f57 [886796,122834] 0 2024-07-03T15:44:05.816 INFO:tasks.workunit.client.0.smithi028.stdout:0/313: dread - d2/d1c/d32/f39 zero size 2024-07-03T15:44:05.818 INFO:tasks.workunit.client.0.smithi028.stdout:0/314: rmdir d2/d1c/d32/d36/d51 0 2024-07-03T15:44:05.818 INFO:tasks.workunit.client.0.smithi028.stdout:0/315: chown d2/d1c/f2b 974 1 2024-07-03T15:44:05.819 INFO:tasks.workunit.client.0.smithi028.stdout:0/316: rmdir d2/d1c/d32 39 2024-07-03T15:44:05.820 INFO:tasks.workunit.client.0.smithi028.stdout:0/317: mknod d2/d1c/c60 0 2024-07-03T15:44:05.820 INFO:tasks.workunit.client.0.smithi028.stdout:0/318: write d2/d1c/f2d [3887281,74852] 0 2024-07-03T15:44:05.826 INFO:tasks.workunit.client.0.smithi028.stdout:0/319: write d2/ff [2993413,95618] 0 2024-07-03T15:44:05.826 INFO:tasks.workunit.client.0.smithi028.stdout:0/320: creat d2/d1c/d32/d36/f61 x:0 0 0 2024-07-03T15:44:05.829 INFO:tasks.workunit.client.0.smithi028.stdout:0/321: symlink d2/d1c/d32/l62 0 2024-07-03T15:44:05.829 INFO:tasks.workunit.client.0.smithi028.stdout:0/322: write d2/f3b [602772,3487] 0 2024-07-03T15:44:05.835 INFO:tasks.workunit.client.0.smithi028.stdout:0/323: dread d2/f55 [0,4194304] 0 2024-07-03T15:44:05.835 INFO:tasks.workunit.client.0.smithi028.stdout:0/324: chown d2/d1c/d32/d36/c4b 2595 1 2024-07-03T15:44:05.838 INFO:tasks.workunit.client.0.smithi028.stdout:0/325: dread d2/d3/d4d/f57 [0,4194304] 0 2024-07-03T15:44:05.839 INFO:tasks.workunit.client.0.smithi028.stdout:0/326: symlink d2/d3/d4d/d56/l63 0 2024-07-03T15:44:05.841 INFO:tasks.workunit.client.0.smithi028.stdout:0/327: rename d2/d1c/d32/d36/f5b to d2/d3/d4d/f64 0 2024-07-03T15:44:05.841 INFO:tasks.workunit.client.0.smithi028.stdout:0/328: chown d2/d1c/d32/d36 68424038 1 2024-07-03T15:44:05.841 INFO:tasks.workunit.client.0.smithi028.stdout:0/329: creat d2/d1c/d32/d36/f65 x:0 0 0 2024-07-03T15:44:05.842 INFO:tasks.workunit.client.0.smithi028.stdout:0/330: getdents d2/d1c/d32/d58 0 2024-07-03T15:44:05.845 INFO:tasks.workunit.client.0.smithi028.stdout:0/331: creat d2/d1c/d32/f66 x:0 0 0 2024-07-03T15:44:05.845 INFO:tasks.workunit.client.0.smithi028.stdout:0/332: fsync d2/d3/f4c 0 2024-07-03T15:44:05.845 INFO:tasks.workunit.client.0.smithi028.stdout:0/333: rename d2/d1c/d32/d36/d4a to d2/d1c/d32/d36/d4a/d67 22 2024-07-03T15:44:05.845 INFO:tasks.workunit.client.0.smithi028.stdout:0/334: truncate d2/d1c/d32/d36/f61 809391 0 2024-07-03T15:44:05.846 INFO:tasks.workunit.client.0.smithi028.stdout:0/335: creat d2/d1c/f68 x:0 0 0 2024-07-03T15:44:05.847 INFO:tasks.workunit.client.0.smithi028.stdout:0/336: rename d2/d3/c5d to d2/d3/d4d/c69 0 2024-07-03T15:44:05.847 INFO:tasks.workunit.client.0.smithi028.stdout:0/337: truncate d2/d1c/f2e 642864 0 2024-07-03T15:44:05.855 INFO:tasks.workunit.client.0.smithi028.stdout:0/338: dread d2/d1c/f2d [0,4194304] 0 2024-07-03T15:44:05.855 INFO:tasks.workunit.client.0.smithi028.stdout:0/339: readlink d2/d1c/d32/d36/l40 0 2024-07-03T15:44:05.855 INFO:tasks.workunit.client.0.smithi028.stdout:0/340: chown d2/d1c/f41 0 1 2024-07-03T15:44:05.863 INFO:tasks.workunit.client.0.smithi028.stdout:0/341: write d2/d3/f1d [173714,6683] 0 2024-07-03T15:44:05.864 INFO:tasks.workunit.client.0.smithi028.stdout:0/342: read d2/d1c/f41 [320978,127265] 0 2024-07-03T15:44:05.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:05 smithi028 ceph-mon[29148]: pgmap v4: 65 pgs: 65 active+clean; 2.7 GiB data, 9.3 GiB used, 527 GiB / 536 GiB avail 2024-07-03T15:44:05.890 INFO:tasks.workunit.client.0.smithi028.stdout:0/343: dwrite d2/d1c/d32/f66 [0,4194304] 0 2024-07-03T15:44:05.890 INFO:tasks.workunit.client.0.smithi028.stdout:0/344: write d2/d1c/d32/f35 [605574,81909] 0 2024-07-03T15:44:05.890 INFO:tasks.workunit.client.0.smithi028.stdout:0/345: write d2/d3/f25 [1097619,17718] 0 2024-07-03T15:44:05.910 INFO:tasks.workunit.client.0.smithi028.stdout:0/346: dwrite d2/d3/f4c [0,4194304] 0 2024-07-03T15:44:05.910 INFO:tasks.workunit.client.0.smithi028.stdout:8/356: sync 2024-07-03T15:44:05.910 INFO:tasks.workunit.client.0.smithi028.stdout:8/357: fsync d0/d41/d50/d3/dc/d11/f42 0 2024-07-03T15:44:05.910 INFO:tasks.workunit.client.0.smithi028.stdout:8/358: chown d0/d41/d50/d3/dc/d11/d12/c3e 1759211826 1 2024-07-03T15:44:05.910 INFO:tasks.workunit.client.0.smithi028.stdout:6/310: sync 2024-07-03T15:44:05.911 INFO:tasks.workunit.client.0.smithi028.stdout:6/311: fdatasync d4/da/d1c/f1e 0 2024-07-03T15:44:05.911 INFO:tasks.workunit.client.0.smithi028.stdout:6/312: creat d4/da/d13/f6c x:0 0 0 2024-07-03T15:44:05.911 INFO:tasks.workunit.client.0.smithi028.stdout:6/313: fdatasync d4/da/d38/f5f 0 2024-07-03T15:44:05.911 INFO:tasks.workunit.client.0.smithi028.stdout:2/388: sync 2024-07-03T15:44:05.911 INFO:tasks.workunit.client.0.smithi028.stdout:2/389: creat da/d74/f90 x:0 0 0 2024-07-03T15:44:05.911 INFO:tasks.workunit.client.0.smithi028.stdout:2/390: chown da/d1e/c4e 300100589 1 2024-07-03T15:44:05.912 INFO:tasks.workunit.client.0.smithi028.stdout:0/347: creat d2/d1c/d32/d58/f6a x:0 0 0 2024-07-03T15:44:05.913 INFO:tasks.workunit.client.0.smithi028.stdout:8/359: mknod d0/d41/d50/d3/dc/d11/d12/c73 0 2024-07-03T15:44:05.917 INFO:tasks.workunit.client.0.smithi028.stdout:6/314: mknod d4/da/d47/d66/c6d 0 2024-07-03T15:44:05.917 INFO:tasks.workunit.client.0.smithi028.stdout:6/315: chown d4/d12/c55 56 1 2024-07-03T15:44:05.920 INFO:tasks.workunit.client.0.smithi028.stdout:2/391: mkdir da/d42/d47/d91 0 2024-07-03T15:44:05.920 INFO:tasks.workunit.client.0.smithi028.stdout:2/392: write da/d42/f57 [138310,96034] 0 2024-07-03T15:44:05.926 INFO:tasks.workunit.client.0.smithi028.stdout:0/348: unlink d2/ff 0 2024-07-03T15:44:05.926 INFO:tasks.workunit.client.0.smithi028.stdout:8/360: getdents d0/d41 0 2024-07-03T15:44:05.929 INFO:tasks.workunit.client.0.smithi028.stdout:6/316: rename d4/d12/l57 to d4/da/d38/d60/l6e 0 2024-07-03T15:44:05.931 INFO:tasks.workunit.client.0.smithi028.stdout:0/349: rename d2/d1c/d32/d36/l44 to d2/d1c/d32/d36/l6b 0 2024-07-03T15:44:05.934 INFO:tasks.workunit.client.0.smithi028.stdout:8/361: rename d0/d41/d50/d3/dc/d11/d69/c6f to d0/d41/d50/d3/d35/c74 0 2024-07-03T15:44:05.936 INFO:tasks.workunit.client.0.smithi028.stdout:8/362: mknod d0/d41/d50/d3/d35/c75 0 2024-07-03T15:44:05.940 INFO:tasks.workunit.client.0.smithi028.stdout:8/363: unlink d0/d41/d50/d3/dc/d11/d12/c31 0 2024-07-03T15:44:05.940 INFO:tasks.workunit.client.0.smithi028.stdout:8/364: chown d0/d41/d50/d3/dc/d11/d12/l1b 45082142 1 2024-07-03T15:44:05.945 INFO:tasks.workunit.client.0.smithi028.stdout:8/365: rename d0/d41/d50/d3/f3c to d0/f76 0 2024-07-03T15:44:05.945 INFO:tasks.workunit.client.0.smithi028.stdout:8/366: dread - d0/d41/d50/d3/dc/d11/f6d zero size 2024-07-03T15:44:05.949 INFO:tasks.workunit.client.0.smithi028.stdout:8/367: truncate d0/d41/d50/d3/d35/d68/d22/f25 7935167 0 2024-07-03T15:44:05.949 INFO:tasks.workunit.client.0.smithi028.stdout:8/368: truncate d0/d41/d50/d3/d35/d68/d22/d40/f60 986428 0 2024-07-03T15:44:05.949 INFO:tasks.workunit.client.0.smithi028.stdout:8/369: fsync d0/f44 0 2024-07-03T15:44:05.949 INFO:tasks.workunit.client.0.smithi028.stdout:8/370: chown d0/d41/d50/d3/dc/d11/f42 33470 1 2024-07-03T15:44:05.950 INFO:tasks.workunit.client.0.smithi028.stdout:8/371: write d0/d41/d50/d3/dc/d11/d12/f39 [3315920,59515] 0 2024-07-03T15:44:05.950 INFO:tasks.workunit.client.0.smithi028.stdout:8/372: chown d0/d41/d50/d3/d35/d68/d22/c28 1305492 1 2024-07-03T15:44:05.954 INFO:tasks.workunit.client.0.smithi028.stdout:2/393: dwrite da/f17 [0,4194304] 0 2024-07-03T15:44:05.954 INFO:tasks.workunit.client.0.smithi028.stdout:2/394: fsync da/d23/d6f/f7c 0 2024-07-03T15:44:05.954 INFO:tasks.workunit.client.0.smithi028.stdout:2/395: readlink l8 0 2024-07-03T15:44:05.956 INFO:tasks.workunit.client.0.smithi028.stdout:2/396: link da/d74/f8d da/d42/d47/d78/f92 0 2024-07-03T15:44:05.956 INFO:tasks.workunit.client.0.smithi028.stdout:2/397: fdatasync da/d24/f55 0 2024-07-03T15:44:05.959 INFO:tasks.workunit.client.0.smithi028.stdout:6/317: dwrite d4/da/d13/f19 [0,4194304] 0 2024-07-03T15:44:05.963 INFO:tasks.workunit.client.0.smithi028.stdout:0/350: dwrite d2/d1c/d32/d58/f5f [0,4194304] 0 2024-07-03T15:44:05.969 INFO:tasks.workunit.client.0.smithi028.stdout:8/373: write d0/d41/d50/d3/dc/f34 [2024815,86728] 0 2024-07-03T15:44:05.969 INFO:tasks.workunit.client.0.smithi028.stdout:8/374: fdatasync d0/d41/d50/f27 0 2024-07-03T15:44:05.973 INFO:tasks.workunit.client.0.smithi028.stdout:0/351: mkdir d2/d1c/d6c 0 2024-07-03T15:44:05.974 INFO:tasks.workunit.client.0.smithi028.stdout:8/375: mknod d0/d41/d50/d3/dc/d33/d66/c77 0 2024-07-03T15:44:05.975 INFO:tasks.workunit.client.0.smithi028.stdout:8/376: write d0/d41/d50/d3/d35/d68/d22/d40/d62/f65 [1956,47957] 0 2024-07-03T15:44:05.976 INFO:tasks.workunit.client.0.smithi028.stdout:0/352: unlink d2/f55 0 2024-07-03T15:44:05.978 INFO:tasks.workunit.client.0.smithi028.stdout:8/377: link d0/d41/d50/d3/dc/f63 d0/d41/d50/f78 0 2024-07-03T15:44:05.980 INFO:tasks.workunit.client.0.smithi028.stdout:0/353: mkdir d2/d6d 0 2024-07-03T15:44:05.980 INFO:tasks.workunit.client.0.smithi028.stdout:0/354: write d2/d1c/f1e [564044,67480] 0 2024-07-03T15:44:05.981 INFO:tasks.workunit.client.0.smithi028.stdout:8/378: rename d0/d41/d50/d3/d35/d68/l1c to d0/d41/d50/d3/d35/d68/d22/d40/d45/l79 0 2024-07-03T15:44:05.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:05 smithi148 ceph-mon[36056]: pgmap v4: 65 pgs: 65 active+clean; 2.7 GiB data, 9.3 GiB used, 527 GiB / 536 GiB avail 2024-07-03T15:44:05.982 INFO:tasks.workunit.client.0.smithi028.stdout:0/355: mkdir d2/d1c/d32/d58/d6e 0 2024-07-03T15:44:05.983 INFO:tasks.workunit.client.0.smithi028.stdout:0/356: stat d2/d3/f1d 0 2024-07-03T15:44:05.985 INFO:tasks.workunit.client.0.smithi028.stdout:0/357: creat d2/d1c/d32/d36/d4a/f6f x:0 0 0 2024-07-03T15:44:05.987 INFO:tasks.workunit.client.0.smithi028.stdout:8/379: dread d0/d41/d50/d3/dc/f2b [0,4194304] 0 2024-07-03T15:44:05.991 INFO:tasks.workunit.client.0.smithi028.stdout:8/380: rename d0/d41/d50/d3/dc/d11/d12 to d0/d41/d50/d3/dc/d33/d7a 0 2024-07-03T15:44:05.993 INFO:tasks.workunit.client.0.smithi028.stdout:8/381: link d0/d41/d50/f27 d0/f7b 0 2024-07-03T15:44:05.993 INFO:tasks.workunit.client.0.smithi028.stdout:6/318: dwrite d4/d12/f36 [0,4194304] 0 2024-07-03T15:44:05.993 INFO:tasks.workunit.client.0.smithi028.stdout:6/319: chown d4/d12 197238524 1 2024-07-03T15:44:05.993 INFO:tasks.workunit.client.0.smithi028.stdout:6/320: creat d4/da/d38/f6f x:0 0 0 2024-07-03T15:44:05.994 INFO:tasks.workunit.client.0.smithi028.stdout:8/382: creat d0/d41/d50/d3/dc/d11/d4c/f7c x:0 0 0 2024-07-03T15:44:05.995 INFO:tasks.workunit.client.0.smithi028.stdout:2/398: dwrite da/d74/f8d [0,4194304] 0 2024-07-03T15:44:05.997 INFO:tasks.workunit.client.0.smithi028.stdout:6/321: getdents d4/d12 0 2024-07-03T15:44:05.998 INFO:tasks.workunit.client.0.smithi028.stdout:0/358: read d2/d1c/d32/d36/f3d [3659793,110631] 0 2024-07-03T15:44:06.003 INFO:tasks.workunit.client.0.smithi028.stdout:8/383: creat d0/d41/d50/d3/dc/d33/f7d x:0 0 0 2024-07-03T15:44:06.004 INFO:tasks.workunit.client.0.smithi028.stdout:8/384: chown d0/d41/d50/c10 6636 1 2024-07-03T15:44:06.004 INFO:tasks.workunit.client.0.smithi028.stdout:8/385: creat d0/d41/d50/d3/dc/f7e x:0 0 0 2024-07-03T15:44:06.005 INFO:tasks.workunit.client.0.smithi028.stdout:2/399: symlink da/d24/d2c/l93 0 2024-07-03T15:44:06.005 INFO:tasks.workunit.client.0.smithi028.stdout:6/322: unlink d4/da/d13/f6c 0 2024-07-03T15:44:06.005 INFO:tasks.workunit.client.0.smithi028.stdout:6/323: write d4/d15/f5b [999668,90951] 0 2024-07-03T15:44:06.005 INFO:tasks.workunit.client.0.smithi028.stdout:0/359: mknod d2/d1c/d32/d58/d6e/c70 0 2024-07-03T15:44:06.005 INFO:tasks.workunit.client.0.smithi028.stdout:0/360: chown d2/d3/l22 8438464 1 2024-07-03T15:44:06.007 INFO:tasks.workunit.client.0.smithi028.stdout:8/386: mkdir d0/d41/d50/d3/dc/d33/d7f 0 2024-07-03T15:44:06.011 INFO:tasks.workunit.client.0.smithi028.stdout:2/400: rename da/d24/c7d to da/d42/c94 0 2024-07-03T15:44:06.011 INFO:tasks.workunit.client.0.smithi028.stdout:2/401: dread - da/d23/f75 zero size 2024-07-03T15:44:06.011 INFO:tasks.workunit.client.0.smithi028.stdout:2/402: truncate da/d24/f8c 625667 0 2024-07-03T15:44:06.016 INFO:tasks.workunit.client.0.smithi028.stdout:6/324: mknod d4/d15/c70 0 2024-07-03T15:44:06.020 INFO:tasks.workunit.client.0.smithi028.stdout:0/361: creat d2/d1c/d6c/f71 x:0 0 0 2024-07-03T15:44:06.020 INFO:tasks.workunit.client.0.smithi028.stdout:0/362: write d2/d1c/d32/d58/f5f [4584701,78359] 0 2024-07-03T15:44:06.024 INFO:tasks.workunit.client.0.smithi028.stdout:2/403: mknod da/d42/d47/d78/c95 0 2024-07-03T15:44:06.032 INFO:tasks.workunit.client.0.smithi028.stdout:6/325: mknod d4/c71 0 2024-07-03T15:44:06.035 INFO:tasks.workunit.client.0.smithi028.stdout:0/363: rmdir d2/d3 39 2024-07-03T15:44:06.037 INFO:tasks.workunit.client.0.smithi028.stdout:2/404: rename da/f17 to da/d23/f96 0 2024-07-03T15:44:06.038 INFO:tasks.workunit.client.0.smithi028.stdout:0/364: unlink d2/d3/f2c 0 2024-07-03T15:44:06.040 INFO:tasks.workunit.client.0.smithi028.stdout:2/405: creat da/d24/d2c/f97 x:0 0 0 2024-07-03T15:44:06.043 INFO:tasks.workunit.client.0.smithi028.stdout:2/406: rmdir da/d42/d47/d91 0 2024-07-03T15:44:06.043 INFO:tasks.workunit.client.0.smithi028.stdout:2/407: creat da/d24/f98 x:0 0 0 2024-07-03T15:44:06.043 INFO:tasks.workunit.client.0.smithi028.stdout:2/408: fdatasync da/f12 0 2024-07-03T15:44:06.047 INFO:tasks.workunit.client.0.smithi028.stdout:6/326: dwrite d4/da/d38/f46 [0,4194304] 0 2024-07-03T15:44:06.049 INFO:tasks.workunit.client.0.smithi028.stdout:2/409: dread da/f15 [0,4194304] 0 2024-07-03T15:44:06.049 INFO:tasks.workunit.client.0.smithi028.stdout:2/410: readlink da/l59 0 2024-07-03T15:44:06.049 INFO:tasks.workunit.client.0.smithi028.stdout:2/411: fsync da/d23/d6f/f7c 0 2024-07-03T15:44:06.055 INFO:tasks.workunit.client.0.smithi028.stdout:6/327: write d4/da/d38/f46 [2594300,103052] 0 2024-07-03T15:44:06.061 INFO:tasks.workunit.client.0.smithi028.stdout:0/365: truncate d2/d1c/d32/d58/f5f 139720 0 2024-07-03T15:44:06.061 INFO:tasks.workunit.client.0.smithi028.stdout:0/366: write d2/d1c/d32/d36/f3d [684829,52864] 0 2024-07-03T15:44:06.064 INFO:tasks.workunit.client.0.smithi028.stdout:0/367: dread d2/d1c/f1e [0,4194304] 0 2024-07-03T15:44:06.072 INFO:tasks.workunit.client.0.smithi028.stdout:6/328: dwrite d4/da/d1c/f54 [0,4194304] 0 2024-07-03T15:44:06.075 INFO:tasks.workunit.client.0.smithi028.stdout:0/368: dread d2/d1c/d32/f66 [0,4194304] 0 2024-07-03T15:44:06.077 INFO:tasks.workunit.client.0.smithi028.stdout:6/329: dread d4/da/d38/f46 [0,4194304] 0 2024-07-03T15:44:06.078 INFO:tasks.workunit.client.0.smithi028.stdout:0/369: creat d2/d1c/d32/d36/d45/f72 x:0 0 0 2024-07-03T15:44:06.078 INFO:tasks.workunit.client.0.smithi028.stdout:0/370: write d2/d1c/d32/f35 [164797,28573] 0 2024-07-03T15:44:06.084 INFO:tasks.workunit.client.0.smithi028.stdout:0/371: link d2/d1c/d32/d36/f65 d2/d3/f73 0 2024-07-03T15:44:06.091 INFO:tasks.workunit.client.0.smithi028.stdout:6/330: dwrite d4/da/d1c/f61 [0,4194304] 0 2024-07-03T15:44:06.092 INFO:tasks.workunit.client.0.smithi028.stdout:6/331: dread - d4/da/d38/f6f zero size 2024-07-03T15:44:06.092 INFO:tasks.workunit.client.0.smithi028.stdout:6/332: chown d4/da/c37 831148575 1 2024-07-03T15:44:06.092 INFO:tasks.workunit.client.0.smithi028.stdout:6/333: chown d4/da/d38/d59 132024906 1 2024-07-03T15:44:06.092 INFO:tasks.workunit.client.0.smithi028.stdout:6/334: write d4/da/d38/d3d/f58 [487576,53632] 0 2024-07-03T15:44:06.092 INFO:tasks.workunit.client.0.smithi028.stdout:6/335: fdatasync d4/d12/d2c/d31/f64 0 2024-07-03T15:44:06.092 INFO:tasks.workunit.client.0.smithi028.stdout:6/336: chown d4/da/cf 1 1 2024-07-03T15:44:06.096 INFO:tasks.workunit.client.0.smithi028.stdout:6/337: link d4/da/c14 d4/da/d38/c72 0 2024-07-03T15:44:06.099 INFO:tasks.workunit.client.0.smithi028.stdout:0/372: dwrite d2/d1c/f20 [0,4194304] 0 2024-07-03T15:44:06.100 INFO:tasks.workunit.client.0.smithi028.stdout:6/338: creat d4/da/d56/f73 x:0 0 0 2024-07-03T15:44:06.110 INFO:tasks.workunit.client.0.smithi028.stdout:0/373: rename d2/d1c/d32/d36/d45/l47 to d2/l74 0 2024-07-03T15:44:06.110 INFO:tasks.workunit.client.0.smithi028.stdout:6/339: dread d4/da/d1c/f61 [0,4194304] 0 2024-07-03T15:44:06.111 INFO:tasks.workunit.client.0.smithi028.stdout:6/340: rmdir d4/da/d38 39 2024-07-03T15:44:06.112 INFO:tasks.workunit.client.0.smithi028.stdout:6/341: chown d4/l1f 10 1 2024-07-03T15:44:06.112 INFO:tasks.workunit.client.0.smithi028.stdout:6/342: chown d4/d15/l1d 782 1 2024-07-03T15:44:06.113 INFO:tasks.workunit.client.0.smithi028.stdout:0/374: dread d2/d1c/f2d [0,4194304] 0 2024-07-03T15:44:06.113 INFO:tasks.workunit.client.0.smithi028.stdout:0/375: chown d2/d1c/l48 22 1 2024-07-03T15:44:06.115 INFO:tasks.workunit.client.0.smithi028.stdout:6/343: rename d4/d15/l1d to d4/da/d47/d66/l74 0 2024-07-03T15:44:06.119 INFO:tasks.workunit.client.0.smithi028.stdout:0/376: rename d2/d3/l5c to d2/l75 0 2024-07-03T15:44:06.119 INFO:tasks.workunit.client.0.smithi028.stdout:0/377: write d2/d3/fe [1553113,21893] 0 2024-07-03T15:44:06.119 INFO:tasks.workunit.client.0.smithi028.stdout:0/378: chown d2/d3/f7 1312 1 2024-07-03T15:44:06.119 INFO:tasks.workunit.client.0.smithi028.stdout:0/379: creat d2/d1c/f76 x:0 0 0 2024-07-03T15:44:06.123 INFO:tasks.workunit.client.0.smithi028.stdout:6/344: link d4/l41 d4/da/d38/l75 0 2024-07-03T15:44:06.127 INFO:tasks.workunit.client.0.smithi028.stdout:0/380: mknod d2/d1c/d32/c77 0 2024-07-03T15:44:06.129 INFO:tasks.workunit.client.0.smithi028.stdout:6/345: mknod d4/da/d13/c76 0 2024-07-03T15:44:06.129 INFO:tasks.workunit.client.0.smithi028.stdout:6/346: write d4/da/f44 [46266,40298] 0 2024-07-03T15:44:06.132 INFO:tasks.workunit.client.0.smithi028.stdout:0/381: write d2/d3/f11 [646275,59681] 0 2024-07-03T15:44:06.132 INFO:tasks.workunit.client.0.smithi028.stdout:0/382: dread - d2/d1c/d32/f39 zero size 2024-07-03T15:44:06.134 INFO:tasks.workunit.client.0.smithi028.stdout:0/383: mkdir d2/d3/d4d/d56/d78 0 2024-07-03T15:44:06.134 INFO:tasks.workunit.client.0.smithi028.stdout:0/384: creat d2/d1c/d6c/f79 x:0 0 0 2024-07-03T15:44:06.135 INFO:tasks.workunit.client.0.smithi028.stdout:0/385: readlink d2/d3/l52 0 2024-07-03T15:44:06.135 INFO:tasks.workunit.client.0.smithi028.stdout:0/386: write d2/f3b [378530,102871] 0 2024-07-03T15:44:06.135 INFO:tasks.workunit.client.0.smithi028.stdout:0/387: truncate d2/d3/fe 2585154 0 2024-07-03T15:44:06.135 INFO:tasks.workunit.client.0.smithi028.stdout:0/388: fdatasync d2/d1c/f41 0 2024-07-03T15:44:06.135 INFO:tasks.workunit.client.0.smithi028.stdout:0/389: chown d2/d1c/f2b 89218 1 2024-07-03T15:44:06.135 INFO:tasks.workunit.client.0.smithi028.stdout:0/390: creat d2/d1c/d32/d36/f7a x:0 0 0 2024-07-03T15:44:06.138 INFO:tasks.workunit.client.0.smithi028.stdout:6/347: dread d4/f10 [0,4194304] 0 2024-07-03T15:44:06.144 INFO:tasks.workunit.client.0.smithi028.stdout:0/391: read d2/d1c/d32/f38 [3609717,118073] 0 2024-07-03T15:44:06.144 INFO:tasks.workunit.client.0.smithi028.stdout:0/392: fdatasync d2/f3b 0 2024-07-03T15:44:06.144 INFO:tasks.workunit.client.0.smithi028.stdout:0/393: creat d2/d1c/d32/d36/d45/f7b x:0 0 0 2024-07-03T15:44:06.152 INFO:tasks.workunit.client.0.smithi028.stdout:0/394: mknod d2/d1c/d32/c7c 0 2024-07-03T15:44:06.154 INFO:tasks.workunit.client.0.smithi028.stdout:0/395: mknod d2/d1c/d32/d36/d4a/c7d 0 2024-07-03T15:44:06.162 INFO:tasks.workunit.client.0.smithi028.stdout:0/396: getdents d2/d1c/d32/d36/d4a 0 2024-07-03T15:44:06.163 INFO:tasks.workunit.client.0.smithi028.stdout:0/397: dread - d2/f49 zero size 2024-07-03T15:44:06.163 INFO:tasks.workunit.client.0.smithi028.stdout:0/398: creat d2/d1c/f7e x:0 0 0 2024-07-03T15:44:06.165 INFO:tasks.workunit.client.0.smithi028.stdout:0/399: dread d2/d1c/f1e [0,4194304] 0 2024-07-03T15:44:06.165 INFO:tasks.workunit.client.0.smithi028.stdout:0/400: write d2/d1c/d32/d58/f6a [444855,29977] 0 2024-07-03T15:44:06.165 INFO:tasks.workunit.client.0.smithi028.stdout:0/401: write d2/d1c/d32/d36/d45/f72 [423618,121246] 0 2024-07-03T15:44:06.168 INFO:tasks.workunit.client.0.smithi028.stdout:0/402: rename d2/d1c/f21 to d2/d3/d4d/d56/f7f 0 2024-07-03T15:44:06.172 INFO:tasks.workunit.client.0.smithi028.stdout:0/403: rename d2/d3/f1d to d2/d1c/d32/d58/d6e/f80 0 2024-07-03T15:44:06.172 INFO:tasks.workunit.client.0.smithi028.stdout:0/404: write d2/d1c/f68 [414733,62022] 0 2024-07-03T15:44:06.175 INFO:tasks.workunit.client.0.smithi028.stdout:0/405: symlink d2/d3/d4d/d56/d78/l81 0 2024-07-03T15:44:06.175 INFO:tasks.workunit.client.0.smithi028.stdout:0/406: write d2/d1c/d32/d58/f5a [913046,43928] 0 2024-07-03T15:44:06.176 INFO:tasks.workunit.client.0.smithi028.stdout:0/407: mkdir d2/d1c/d32/d58/d82 0 2024-07-03T15:44:06.200 INFO:tasks.workunit.client.0.smithi028.stdout:0/408: dwrite d2/f15 [0,4194304] 0 2024-07-03T15:44:06.201 INFO:tasks.workunit.client.0.smithi028.stdout:0/409: creat d2/d1c/f83 x:0 0 0 2024-07-03T15:44:06.201 INFO:tasks.workunit.client.0.smithi028.stdout:0/410: stat d2/d3/d4d/f64 0 2024-07-03T15:44:06.204 INFO:tasks.workunit.client.0.smithi028.stdout:7/209: sync 2024-07-03T15:44:06.205 INFO:tasks.workunit.client.0.smithi028.stdout:9/357: sync 2024-07-03T15:44:06.205 INFO:tasks.workunit.client.0.smithi028.stdout:3/435: sync 2024-07-03T15:44:06.205 INFO:tasks.workunit.client.0.smithi028.stdout:4/380: sync 2024-07-03T15:44:06.205 INFO:tasks.workunit.client.0.smithi028.stdout:3/436: chown d18/l3b 48283889 1 2024-07-03T15:44:06.205 INFO:tasks.workunit.client.0.smithi028.stdout:4/381: readlink d2/d5a/l29 0 2024-07-03T15:44:06.205 INFO:tasks.workunit.client.0.smithi028.stdout:4/382: chown d2/d12/d3b/f4a 1255763086 1 2024-07-03T15:44:06.205 INFO:tasks.workunit.client.0.smithi028.stdout:3/437: write d18/d43/f4d [529886,63731] 0 2024-07-03T15:44:06.205 INFO:tasks.workunit.client.0.smithi028.stdout:1/513: sync 2024-07-03T15:44:06.205 INFO:tasks.workunit.client.0.smithi028.stdout:5/400: sync 2024-07-03T15:44:06.206 INFO:tasks.workunit.client.0.smithi028.stdout:7/210: dread d9/d29/f31 [0,4194304] 0 2024-07-03T15:44:06.207 INFO:tasks.workunit.client.0.smithi028.stdout:7/211: write d9/f1a [2889978,56736] 0 2024-07-03T15:44:06.207 INFO:tasks.workunit.client.0.smithi028.stdout:7/212: creat d9/d29/f42 x:0 0 0 2024-07-03T15:44:06.208 INFO:tasks.workunit.client.0.smithi028.stdout:4/383: unlink d2/d12/d3b/f4a 0 2024-07-03T15:44:06.208 INFO:tasks.workunit.client.0.smithi028.stdout:4/384: fsync d2/d12/d42/d63/d6c/f7b 0 2024-07-03T15:44:06.208 INFO:tasks.workunit.client.0.smithi028.stdout:4/385: stat d2/d12/d2e/f6b 0 2024-07-03T15:44:06.210 INFO:tasks.workunit.client.0.smithi028.stdout:4/386: write d2/d12/d3b/f43 [1753518,61711] 0 2024-07-03T15:44:06.210 INFO:tasks.workunit.client.0.smithi028.stdout:4/387: chown d2/d12/d5b 10 1 2024-07-03T15:44:06.210 INFO:tasks.workunit.client.0.smithi028.stdout:4/388: write d2/d12/f68 [124430,99309] 0 2024-07-03T15:44:06.210 INFO:tasks.workunit.client.0.smithi028.stdout:9/358: creat d1/d16/d57/d5f/f7a x:0 0 0 2024-07-03T15:44:06.211 INFO:tasks.workunit.client.0.smithi028.stdout:9/359: readlink d1/l2c 0 2024-07-03T15:44:06.214 INFO:tasks.workunit.client.0.smithi028.stdout:3/438: getdents d18/d1a/d64 0 2024-07-03T15:44:06.218 INFO:tasks.workunit.client.0.smithi028.stdout:0/411: dwrite d2/d1c/d32/d36/d45/f7b [0,4194304] 0 2024-07-03T15:44:06.218 INFO:tasks.workunit.client.0.smithi028.stdout:0/412: fdatasync d2/d1c/d32/d36/f65 0 2024-07-03T15:44:06.219 INFO:tasks.workunit.client.0.smithi028.stdout:1/514: mkdir d4/d2c/dbe 0 2024-07-03T15:44:06.220 INFO:tasks.workunit.client.0.smithi028.stdout:0/413: dread d2/f3b [0,4194304] 0 2024-07-03T15:44:06.223 INFO:tasks.workunit.client.0.smithi028.stdout:5/401: rename d0/dd/d13/f39 to d0/dd/d13/f83 0 2024-07-03T15:44:06.232 INFO:tasks.workunit.client.0.smithi028.stdout:4/389: symlink d2/d12/d2d/l7e 0 2024-07-03T15:44:06.240 INFO:tasks.workunit.client.0.smithi028.stdout:3/439: rename d18/d1a/d5e/d60/c62 to d18/d1a/d1e/d73/c8a 0 2024-07-03T15:44:06.247 INFO:tasks.workunit.client.0.smithi028.stdout:1/515: creat d4/d2c/d3a/d5c/fbf x:0 0 0 2024-07-03T15:44:06.250 INFO:tasks.workunit.client.0.smithi028.stdout:5/402: symlink d0/dd/d13/d57/d30/d70/d7f/l84 0 2024-07-03T15:44:06.257 INFO:tasks.workunit.client.0.smithi028.stdout:5/403: read d0/dd/d1e/d3b/f4f [3215899,51606] 0 2024-07-03T15:44:06.265 INFO:tasks.workunit.client.0.smithi028.stdout:7/213: rmdir d9/d1c 39 2024-07-03T15:44:06.267 INFO:tasks.workunit.client.0.smithi028.stdout:4/390: mknod d2/d12/d42/c7f 0 2024-07-03T15:44:06.268 INFO:tasks.workunit.client.0.smithi028.stdout:9/360: getdents d1/d16 0 2024-07-03T15:44:06.271 INFO:tasks.workunit.client.0.smithi028.stdout:3/440: mknod d18/d1a/d5e/d66/c8b 0 2024-07-03T15:44:06.272 INFO:tasks.workunit.client.0.smithi028.stdout:3/441: getdents d18/d1a/d1e/d58 0 2024-07-03T15:44:06.273 INFO:tasks.workunit.client.0.smithi028.stdout:9/361: dread d1/d4a/f52 [0,4194304] 0 2024-07-03T15:44:06.276 INFO:tasks.workunit.client.0.smithi028.stdout:1/516: creat d4/de/da7/fc0 x:0 0 0 2024-07-03T15:44:06.277 INFO:tasks.workunit.client.0.smithi028.stdout:5/404: rename d0/dd/d13/d57/d30/f73 to d0/dd/d13/d57/f85 0 2024-07-03T15:44:06.277 INFO:tasks.workunit.client.0.smithi028.stdout:5/405: chown d0/dd/d13/d47/l63 80 1 2024-07-03T15:44:06.279 INFO:tasks.workunit.client.0.smithi028.stdout:4/391: truncate d2/d5a/f24 3559699 0 2024-07-03T15:44:06.280 INFO:tasks.workunit.client.0.smithi028.stdout:4/392: write f1 [8749270,38622] 0 2024-07-03T15:44:06.280 INFO:tasks.workunit.client.0.smithi028.stdout:4/393: creat d2/d12/d5b/d78/f80 x:0 0 0 2024-07-03T15:44:06.280 INFO:tasks.workunit.client.0.smithi028.stdout:4/394: chown d2/d12/d42/d63 0 1 2024-07-03T15:44:06.282 INFO:tasks.workunit.client.0.smithi028.stdout:1/517: mknod d4/de/da7/cc1 0 2024-07-03T15:44:06.283 INFO:tasks.workunit.client.0.smithi028.stdout:5/406: getdents d0/dd/d13/d57/d30/d70 0 2024-07-03T15:44:06.288 INFO:tasks.workunit.client.0.smithi028.stdout:1/518: getdents d4/de/d1c/d55 0 2024-07-03T15:44:06.291 INFO:tasks.workunit.client.0.smithi028.stdout:5/407: mkdir d0/dd/d1e/d86 0 2024-07-03T15:44:06.293 INFO:tasks.workunit.client.0.smithi028.stdout:4/395: rename d2/d12/d5b/d78/f80 to d2/d12/d5b/f81 0 2024-07-03T15:44:06.293 INFO:tasks.workunit.client.0.smithi028.stdout:4/396: write d2/d12/d5b/f81 [809390,2415] 0 2024-07-03T15:44:06.293 INFO:tasks.workunit.client.0.smithi028.stdout:4/397: readlink d2/d12/d2d/l33 0 2024-07-03T15:44:06.296 INFO:tasks.workunit.client.0.smithi028.stdout:1/519: unlink d4/d3c/d54/d79/d16/d57/d66/c95 0 2024-07-03T15:44:06.297 INFO:tasks.workunit.client.0.smithi028.stdout:5/408: mkdir d0/d87 0 2024-07-03T15:44:06.299 INFO:tasks.workunit.client.0.smithi028.stdout:7/214: dwrite f0 [4194304,4194304] 0 2024-07-03T15:44:06.302 INFO:tasks.workunit.client.0.smithi028.stdout:4/398: rename d2/f73 to d2/d61/f82 0 2024-07-03T15:44:06.305 INFO:tasks.workunit.client.0.smithi028.stdout:9/362: dwrite d1/f2 [0,4194304] 0 2024-07-03T15:44:06.305 INFO:tasks.workunit.client.0.smithi028.stdout:3/442: dwrite d18/f51 [0,4194304] 0 2024-07-03T15:44:06.307 INFO:tasks.workunit.client.0.smithi028.stdout:1/520: mkdir d4/dc2 0 2024-07-03T15:44:06.307 INFO:tasks.workunit.client.0.smithi028.stdout:1/521: chown d4/d3c/d54/d79/d16/d7c 16183 1 2024-07-03T15:44:06.307 INFO:tasks.workunit.client.0.smithi028.stdout:4/399: truncate d2/f27 2131589 0 2024-07-03T15:44:06.310 INFO:tasks.workunit.client.0.smithi028.stdout:1/522: unlink d4/d83/lad 0 2024-07-03T15:44:06.310 INFO:tasks.workunit.client.0.smithi028.stdout:1/523: write d4/d2c/d3a/d5c/d6c/d7d/d4e/d51/fa2 [4078020,85437] 0 2024-07-03T15:44:06.314 INFO:tasks.workunit.client.0.smithi028.stdout:7/215: mknod d9/d1c/c43 0 2024-07-03T15:44:06.314 INFO:tasks.workunit.client.0.smithi028.stdout:7/216: fsync f0 0 2024-07-03T15:44:06.315 INFO:tasks.workunit.client.0.smithi028.stdout:9/363: symlink d1/d10/d28/d4e/d5b/l7b 0 2024-07-03T15:44:06.317 INFO:tasks.workunit.client.0.smithi028.stdout:9/364: dread d1/d10/d28/d4e/f50 [0,4194304] 0 2024-07-03T15:44:06.317 INFO:tasks.workunit.client.0.smithi028.stdout:4/400: dread d2/d12/d2d/f32 [0,4194304] 0 2024-07-03T15:44:06.317 INFO:tasks.workunit.client.0.smithi028.stdout:4/401: chown d2/d12/d3b/c5c 17228190 1 2024-07-03T15:44:06.317 INFO:tasks.workunit.client.0.smithi028.stdout:4/402: dread - d2/d61/f67 zero size 2024-07-03T15:44:06.318 INFO:tasks.workunit.client.0.smithi028.stdout:1/524: creat d4/de/fc3 x:0 0 0 2024-07-03T15:44:06.318 INFO:tasks.workunit.client.0.smithi028.stdout:1/525: write d4/d2c/d3a/d5c/d6c/d7d/d59/d94/f96 [187252,11852] 0 2024-07-03T15:44:06.318 INFO:tasks.workunit.client.0.smithi028.stdout:1/526: chown d4/de/d1c/f3d 66397 1 2024-07-03T15:44:06.318 INFO:tasks.workunit.client.0.smithi028.stdout:1/527: chown d4/de/d1c 173 1 2024-07-03T15:44:06.331 INFO:tasks.workunit.client.0.smithi028.stdout:4/403: mknod d2/d12/d42/d63/d75/c83 0 2024-07-03T15:44:06.337 INFO:tasks.workunit.client.0.smithi028.stdout:1/528: link d4/d2c/d3a/d5c/d6c/d7d/l4a d4/d3c/d54/lc4 0 2024-07-03T15:44:06.338 INFO:tasks.workunit.client.0.smithi028.stdout:1/529: stat d4/c7 0 2024-07-03T15:44:06.342 INFO:tasks.workunit.client.0.smithi028.stdout:1/530: read d4/d3c/d54/d79/f3e [115003,47448] 0 2024-07-03T15:44:06.342 INFO:tasks.workunit.client.0.smithi028.stdout:1/531: chown d4/d3c/d54/d79/d16/fb3 89129 1 2024-07-03T15:44:06.342 INFO:tasks.workunit.client.0.smithi028.stdout:5/409: dwrite d0/dd/d13/d29/f35 [0,4194304] 0 2024-07-03T15:44:06.342 INFO:tasks.workunit.client.0.smithi028.stdout:5/410: read d0/dd/d13/d57/d26/f2e [2161163,118267] 0 2024-07-03T15:44:06.342 INFO:tasks.workunit.client.0.smithi028.stdout:5/411: write d0/dd/d13/d57/d26/d41/f43 [2902818,56909] 0 2024-07-03T15:44:06.342 INFO:tasks.workunit.client.0.smithi028.stdout:3/443: dwrite d18/d1a/f2a [0,4194304] 0 2024-07-03T15:44:06.343 INFO:tasks.workunit.client.0.smithi028.stdout:9/365: dwrite d1/dd/f3c [0,4194304] 0 2024-07-03T15:44:06.343 INFO:tasks.workunit.client.0.smithi028.stdout:4/404: mkdir d2/d5a/d45/d84 0 2024-07-03T15:44:06.343 INFO:tasks.workunit.client.0.smithi028.stdout:7/217: truncate d9/f34 3115551 0 2024-07-03T15:44:06.344 INFO:tasks.workunit.client.0.smithi028.stdout:7/218: creat d9/d29/f44 x:0 0 0 2024-07-03T15:44:06.344 INFO:tasks.workunit.client.0.smithi028.stdout:7/219: fdatasync d9/f25 0 2024-07-03T15:44:06.352 INFO:tasks.workunit.client.0.smithi028.stdout:7/220: dread f6 [0,4194304] 0 2024-07-03T15:44:06.353 INFO:tasks.workunit.client.0.smithi028.stdout:5/412: read d0/dd/d1e/f36 [1206432,63580] 0 2024-07-03T15:44:06.360 INFO:tasks.workunit.client.0.smithi028.stdout:1/532: dwrite d4/fd [0,4194304] 0 2024-07-03T15:44:06.360 INFO:tasks.workunit.client.0.smithi028.stdout:1/533: truncate d4/d3c/fa3 798725 0 2024-07-03T15:44:06.360 INFO:tasks.workunit.client.0.smithi028.stdout:1/534: creat d4/de/fc5 x:0 0 0 2024-07-03T15:44:06.360 INFO:tasks.workunit.client.0.smithi028.stdout:1/535: write d4/d2c/d3a/d5c/d6c/d7d/d59/f6a [2011166,115224] 0 2024-07-03T15:44:06.360 INFO:tasks.workunit.client.0.smithi028.stdout:1/536: fsync d4/d2c/d3a/d5c/d6c/d7d/d59/d94/f96 0 2024-07-03T15:44:06.363 INFO:tasks.workunit.client.0.smithi028.stdout:3/444: mkdir d18/d1a/d5e/d60/d8c 0 2024-07-03T15:44:06.364 INFO:tasks.workunit.client.0.smithi028.stdout:3/445: write d18/d1a/d1e/f5d [720008,114046] 0 2024-07-03T15:44:06.366 INFO:tasks.workunit.client.0.smithi028.stdout:9/366: unlink d1/dd/f35 0 2024-07-03T15:44:06.367 INFO:tasks.workunit.client.0.smithi028.stdout:7/221: dwrite d9/f25 [0,4194304] 0 2024-07-03T15:44:06.369 INFO:tasks.workunit.client.0.smithi028.stdout:5/413: creat d0/dd/d1e/d86/f88 x:0 0 0 2024-07-03T15:44:06.373 INFO:tasks.workunit.client.0.smithi028.stdout:1/537: mkdir d4/d2c/d3a/d5c/d6c/d7d/d59/dc6 0 2024-07-03T15:44:06.373 INFO:tasks.workunit.client.0.smithi028.stdout:1/538: chown d4/d2c/d3a/d5c/d6c/d7d/d4e/fbc 125971 1 2024-07-03T15:44:06.375 INFO:tasks.workunit.client.0.smithi028.stdout:3/446: mkdir d18/d1a/d5e/d8d 0 2024-07-03T15:44:06.380 INFO:tasks.workunit.client.0.smithi028.stdout:8/387: sync 2024-07-03T15:44:06.380 INFO:tasks.workunit.client.0.smithi028.stdout:8/388: write d0/d41/d50/d3/dc/d33/f7d [313656,3207] 0 2024-07-03T15:44:06.380 INFO:tasks.workunit.client.0.smithi028.stdout:9/367: rmdir d1/d2a 39 2024-07-03T15:44:06.382 INFO:tasks.workunit.client.0.smithi028.stdout:7/222: getdents d9 0 2024-07-03T15:44:06.383 INFO:tasks.workunit.client.0.smithi028.stdout:1/539: read d4/de/d1c/f3d [1874430,97825] 0 2024-07-03T15:44:06.383 INFO:tasks.workunit.client.0.smithi028.stdout:1/540: write d4/d3c/d54/d79/f1b [3085501,111399] 0 2024-07-03T15:44:06.383 INFO:tasks.workunit.client.0.smithi028.stdout:1/541: dread - d4/d2c/d3a/d5c/f98 zero size 2024-07-03T15:44:06.383 INFO:tasks.workunit.client.0.smithi028.stdout:1/542: write d4/d2c/d3a/d5c/d6c/d7d/d59/f71 [255205,81747] 0 2024-07-03T15:44:06.384 INFO:tasks.workunit.client.0.smithi028.stdout:3/447: dread d18/d1a/d1e/f57 [0,4194304] 0 2024-07-03T15:44:06.384 INFO:tasks.workunit.client.0.smithi028.stdout:3/448: stat d18/d1a/d5e/l4c 0 2024-07-03T15:44:06.387 INFO:tasks.workunit.client.0.smithi028.stdout:3/449: dread f9 [0,4194304] 0 2024-07-03T15:44:06.387 INFO:tasks.workunit.client.0.smithi028.stdout:1/543: getdents d4/d3c/d54/d79/d16/d7c 0 2024-07-03T15:44:06.390 INFO:tasks.workunit.client.0.smithi028.stdout:1/544: symlink d4/d3c/d54/d79/d16/lc7 0 2024-07-03T15:44:06.391 INFO:tasks.workunit.client.0.smithi028.stdout:1/545: write d4/d2c/d3a/d5c/d6c/d7d/d4e/d51/f5f [803724,79285] 0 2024-07-03T15:44:06.391 INFO:tasks.workunit.client.0.smithi028.stdout:1/546: chown d4/d3c/d54/d79 6496712 1 2024-07-03T15:44:06.397 INFO:tasks.workunit.client.0.smithi028.stdout:7/223: dread d9/f3b [0,4194304] 0 2024-07-03T15:44:06.402 INFO:tasks.workunit.client.0.smithi028.stdout:5/414: dwrite d0/f2c [0,4194304] 0 2024-07-03T15:44:06.402 INFO:tasks.workunit.client.0.smithi028.stdout:5/415: creat d0/dd/d1e/f89 x:0 0 0 2024-07-03T15:44:06.402 INFO:tasks.workunit.client.0.smithi028.stdout:8/389: dwrite d0/d41/d50/f27 [0,4194304] 0 2024-07-03T15:44:06.402 INFO:tasks.workunit.client.0.smithi028.stdout:9/368: dread d1/dd/f59 [0,4194304] 0 2024-07-03T15:44:06.402 INFO:tasks.workunit.client.0.smithi028.stdout:9/369: write d1/f2b [4851509,41491] 0 2024-07-03T15:44:06.402 INFO:tasks.workunit.client.0.smithi028.stdout:7/224: mkdir d9/d1c/d45 0 2024-07-03T15:44:06.402 INFO:tasks.workunit.client.0.smithi028.stdout:7/225: read - d9/d29/f42 zero size 2024-07-03T15:44:06.407 INFO:tasks.workunit.client.0.smithi028.stdout:8/390: mknod d0/d6c/c80 0 2024-07-03T15:44:06.411 INFO:tasks.workunit.client.0.smithi028.stdout:7/226: creat d9/d1c/d45/f46 x:0 0 0 2024-07-03T15:44:06.413 INFO:tasks.workunit.client.0.smithi028.stdout:9/370: rename d1/d2a/f6d to d1/d10/d76/f7c 0 2024-07-03T15:44:06.415 INFO:tasks.workunit.client.0.smithi028.stdout:9/371: dread d1/d10/f3e [0,4194304] 0 2024-07-03T15:44:06.415 INFO:tasks.workunit.client.0.smithi028.stdout:1/547: dwrite d4/fd [0,4194304] 0 2024-07-03T15:44:06.415 INFO:tasks.workunit.client.0.smithi028.stdout:1/548: fdatasync d4/de/d1c/f3d 0 2024-07-03T15:44:06.415 INFO:tasks.workunit.client.0.smithi028.stdout:1/549: read - d4/de/fc3 zero size 2024-07-03T15:44:06.416 INFO:tasks.workunit.client.0.smithi028.stdout:1/550: mknod d4/de/cc8 0 2024-07-03T15:44:06.416 INFO:tasks.workunit.client.0.smithi028.stdout:1/551: chown d4 21 1 2024-07-03T15:44:06.416 INFO:tasks.workunit.client.0.smithi028.stdout:1/552: readlink d4/de/d1c/d81/lba 0 2024-07-03T15:44:06.419 INFO:tasks.workunit.client.0.smithi028.stdout:1/553: rename d4/d2c/d3a/d43/f69 to d4/d3c/d54/d79/d16/db6/fc9 0 2024-07-03T15:44:06.422 INFO:tasks.workunit.client.0.smithi028.stdout:1/554: creat d4/d3c/d54/d79/d16/d57/d66/fca x:0 0 0 2024-07-03T15:44:06.423 INFO:tasks.workunit.client.0.smithi028.stdout:8/391: read d0/d41/d50/f27 [1170148,95251] 0 2024-07-03T15:44:06.423 INFO:tasks.workunit.client.0.smithi028.stdout:9/372: write d1/d16/d57/f71 [4024499,36048] 0 2024-07-03T15:44:06.424 INFO:tasks.workunit.client.0.smithi028.stdout:8/392: mkdir d0/d41/d50/d3/dc/d11/d81 0 2024-07-03T15:44:06.424 INFO:tasks.workunit.client.0.smithi028.stdout:5/416: dwrite d0/dd/df/f3e [0,4194304] 0 2024-07-03T15:44:06.425 INFO:tasks.workunit.client.0.smithi028.stdout:9/373: mknod d1/d10/d28/d45/d61/c7d 0 2024-07-03T15:44:06.426 INFO:tasks.workunit.client.0.smithi028.stdout:8/393: creat d0/d41/d50/d3/dc/d33/d7f/f82 x:0 0 0 2024-07-03T15:44:06.426 INFO:tasks.workunit.client.0.smithi028.stdout:8/394: chown d0/d41/d50/d3/dc/f23 2317 1 2024-07-03T15:44:06.428 INFO:tasks.workunit.client.0.smithi028.stdout:8/395: mknod d0/d41/d50/d3/dc/d11/d69/c83 0 2024-07-03T15:44:06.428 INFO:tasks.workunit.client.0.smithi028.stdout:8/396: readlink d0/d41/d50/d3/d35/d68/d22/d40/d45/l79 0 2024-07-03T15:44:06.429 INFO:tasks.workunit.client.0.smithi028.stdout:8/397: fsync d0/d41/d50/d3/dc/d3f/f61 0 2024-07-03T15:44:06.430 INFO:tasks.workunit.client.0.smithi028.stdout:8/398: unlink d0/d41/d50/c36 0 2024-07-03T15:44:06.433 INFO:tasks.workunit.client.0.smithi028.stdout:8/399: fdatasync d0/d41/d50/d3/dc/d33/f7d 0 2024-07-03T15:44:06.433 INFO:tasks.workunit.client.0.smithi028.stdout:7/227: rmdir d9/d1c/d45 39 2024-07-03T15:44:06.436 INFO:tasks.workunit.client.0.smithi028.stdout:7/228: link d9/d1c/c37 d9/d29/c47 0 2024-07-03T15:44:06.440 INFO:tasks.workunit.client.0.smithi028.stdout:9/374: dwrite d1/d2a/f72 [0,4194304] 0 2024-07-03T15:44:06.441 INFO:tasks.workunit.client.0.smithi028.stdout:9/375: fdatasync d1/d16/f2f 0 2024-07-03T15:44:06.441 INFO:tasks.workunit.client.0.smithi028.stdout:9/376: dread d1/d10/d28/d4e/f50 [0,4194304] 0 2024-07-03T15:44:06.447 INFO:tasks.workunit.client.0.smithi028.stdout:7/229: creat d9/f48 x:0 0 0 2024-07-03T15:44:06.447 INFO:tasks.workunit.client.0.smithi028.stdout:7/230: write d9/d29/f35 [4421664,86663] 0 2024-07-03T15:44:06.447 INFO:tasks.workunit.client.0.smithi028.stdout:7/231: fsync d9/f3a 0 2024-07-03T15:44:06.448 INFO:tasks.workunit.client.0.smithi028.stdout:8/400: rmdir d0 39 2024-07-03T15:44:06.461 INFO:tasks.workunit.client.0.smithi028.stdout:9/377: mkdir d1/d4a/d7e 0 2024-07-03T15:44:06.462 INFO:tasks.workunit.client.0.smithi028.stdout:9/378: fsync d1/dd/f6f 0 2024-07-03T15:44:06.465 INFO:tasks.workunit.client.0.smithi028.stdout:7/232: symlink d9/d29/l49 0 2024-07-03T15:44:06.465 INFO:tasks.workunit.client.0.smithi028.stdout:9/379: mkdir d1/d16/d57/d5f/d7f 0 2024-07-03T15:44:06.466 INFO:tasks.workunit.client.0.smithi028.stdout:9/380: creat d1/d16/d57/d5f/f80 x:0 0 0 2024-07-03T15:44:06.467 INFO:tasks.workunit.client.0.smithi028.stdout:7/233: mknod d9/d29/c4a 0 2024-07-03T15:44:06.467 INFO:tasks.workunit.client.0.smithi028.stdout:7/234: readlink d9/l32 0 2024-07-03T15:44:06.471 INFO:tasks.workunit.client.0.smithi028.stdout:7/235: dread f6 [0,4194304] 0 2024-07-03T15:44:06.471 INFO:tasks.workunit.client.0.smithi028.stdout:9/381: rename d1/dd/f20 to d1/d2a/f81 0 2024-07-03T15:44:06.473 INFO:tasks.workunit.client.0.smithi028.stdout:9/382: mknod d1/d16/d62/c82 0 2024-07-03T15:44:06.473 INFO:tasks.workunit.client.0.smithi028.stdout:7/236: link d9/d29/f2e d9/d1c/f4b 0 2024-07-03T15:44:06.475 INFO:tasks.workunit.client.0.smithi028.stdout:9/383: mknod d1/d10/d28/d45/d56/c83 0 2024-07-03T15:44:06.475 INFO:tasks.workunit.client.0.smithi028.stdout:7/237: mknod d9/c4c 0 2024-07-03T15:44:06.477 INFO:tasks.workunit.client.0.smithi028.stdout:9/384: creat d1/d10/d28/d4e/d5b/f84 x:0 0 0 2024-07-03T15:44:06.477 INFO:tasks.workunit.client.0.smithi028.stdout:9/385: chown d1/d16/d57/d5f/f80 207 1 2024-07-03T15:44:06.477 INFO:tasks.workunit.client.0.smithi028.stdout:9/386: chown d1/d4a/f65 4 1 2024-07-03T15:44:06.477 INFO:tasks.workunit.client.0.smithi028.stdout:9/387: read - d1/d16/f51 zero size 2024-07-03T15:44:06.478 INFO:tasks.workunit.client.0.smithi028.stdout:7/238: dread d9/d1c/f20 [0,4194304] 0 2024-07-03T15:44:06.478 INFO:tasks.workunit.client.0.smithi028.stdout:7/239: fdatasync d9/f41 0 2024-07-03T15:44:06.478 INFO:tasks.workunit.client.0.smithi028.stdout:9/388: creat d1/d2a/d77/f85 x:0 0 0 2024-07-03T15:44:06.480 INFO:tasks.workunit.client.0.smithi028.stdout:7/240: link d9/ld d9/l4d 0 2024-07-03T15:44:06.480 INFO:tasks.workunit.client.0.smithi028.stdout:7/241: truncate f0 10990328 0 2024-07-03T15:44:06.480 INFO:tasks.workunit.client.0.smithi028.stdout:7/242: read - d9/d29/f2e zero size 2024-07-03T15:44:06.481 INFO:tasks.workunit.client.0.smithi028.stdout:7/243: dread - d9/f22 zero size 2024-07-03T15:44:06.481 INFO:tasks.workunit.client.0.smithi028.stdout:7/244: dread - d9/f3a zero size 2024-07-03T15:44:06.481 INFO:tasks.workunit.client.0.smithi028.stdout:7/245: read f6 [1599152,40621] 0 2024-07-03T15:44:06.502 INFO:tasks.workunit.client.0.smithi028.stdout:2/412: sync 2024-07-03T15:44:06.502 INFO:tasks.workunit.client.0.smithi028.stdout:2/413: readlink l8 0 2024-07-03T15:44:06.502 INFO:tasks.workunit.client.0.smithi028.stdout:2/414: readlink da/d23/d30/l70 0 2024-07-03T15:44:06.505 INFO:tasks.workunit.client.0.smithi028.stdout:2/415: mknod da/d23/d30/c99 0 2024-07-03T15:44:06.514 INFO:tasks.workunit.client.0.smithi028.stdout:8/401: dwrite d0/f7b [0,4194304] 0 2024-07-03T15:44:06.518 INFO:tasks.workunit.client.0.smithi028.stdout:9/389: dwrite d1/d16/f51 [0,4194304] 0 2024-07-03T15:44:06.519 INFO:tasks.workunit.client.0.smithi028.stdout:7/246: dwrite d9/d29/f3e [0,4194304] 0 2024-07-03T15:44:06.520 INFO:tasks.workunit.client.0.smithi028.stdout:9/390: creat d1/d4a/d7e/f86 x:0 0 0 2024-07-03T15:44:06.520 INFO:tasks.workunit.client.0.smithi028.stdout:9/391: creat d1/d10/d28/d4e/d4f/f87 x:0 0 0 2024-07-03T15:44:06.521 INFO:tasks.workunit.client.0.smithi028.stdout:9/392: creat d1/f88 x:0 0 0 2024-07-03T15:44:06.521 INFO:tasks.workunit.client.0.smithi028.stdout:9/393: write d1/f21 [1286300,115532] 0 2024-07-03T15:44:06.524 INFO:tasks.workunit.client.0.smithi028.stdout:8/402: dread d0/d41/d50/d3/dc/f1f [0,4194304] 0 2024-07-03T15:44:06.527 INFO:tasks.workunit.client.0.smithi028.stdout:8/403: dread d0/d41/d50/d3/dc/d11/f42 [0,4194304] 0 2024-07-03T15:44:06.528 INFO:tasks.workunit.client.0.smithi028.stdout:7/247: rmdir d9/d1c 39 2024-07-03T15:44:06.532 INFO:tasks.workunit.client.0.smithi028.stdout:9/394: mknod d1/d10/d28/d45/d56/c89 0 2024-07-03T15:44:06.532 INFO:tasks.workunit.client.0.smithi028.stdout:8/404: mknod d0/d41/d50/c84 0 2024-07-03T15:44:06.532 INFO:tasks.workunit.client.0.smithi028.stdout:8/405: write d0/d41/d50/d3/d35/d68/d22/f6b [1538,10580] 0 2024-07-03T15:44:06.534 INFO:tasks.workunit.client.0.smithi028.stdout:8/406: symlink d0/l85 0 2024-07-03T15:44:06.550 INFO:tasks.workunit.client.0.smithi028.stdout:9/395: write d1/d10/f18 [3230645,112957] 0 2024-07-03T15:44:06.551 INFO:tasks.workunit.client.0.smithi028.stdout:7/248: dwrite d9/f3a [0,4194304] 0 2024-07-03T15:44:06.551 INFO:tasks.workunit.client.0.smithi028.stdout:7/249: read - d9/d29/f42 zero size 2024-07-03T15:44:06.551 INFO:tasks.workunit.client.0.smithi028.stdout:7/250: truncate d9/fa 741683 0 2024-07-03T15:44:06.551 INFO:tasks.workunit.client.0.smithi028.stdout:7/251: fsync d9/d29/f3e 0 2024-07-03T15:44:06.555 INFO:tasks.workunit.client.0.smithi028.stdout:9/396: dread d1/d2a/f6a [0,4194304] 0 2024-07-03T15:44:06.555 INFO:tasks.workunit.client.0.smithi028.stdout:9/397: read d1/d10/d76/f7c [2546577,124314] 0 2024-07-03T15:44:06.555 INFO:tasks.workunit.client.0.smithi028.stdout:8/407: dwrite d0/d41/d50/d3/dc/d33/d7a/f56 [0,4194304] 0 2024-07-03T15:44:06.559 INFO:tasks.workunit.client.0.smithi028.stdout:7/252: dread d9/d29/f35 [0,4194304] 0 2024-07-03T15:44:06.560 INFO:tasks.workunit.client.0.smithi028.stdout:7/253: truncate d9/f41 40098 0 2024-07-03T15:44:06.560 INFO:tasks.workunit.client.0.smithi028.stdout:9/398: creat d1/d10/d28/d45/d56/f8a x:0 0 0 2024-07-03T15:44:06.560 INFO:tasks.workunit.client.0.smithi028.stdout:9/399: stat d1/f43 0 2024-07-03T15:44:06.561 INFO:tasks.workunit.client.0.smithi028.stdout:8/408: rename d0/d41/d50/d3/dc/d33/d66 to d0/d41/d50/d3/d35/d86 0 2024-07-03T15:44:06.562 INFO:tasks.workunit.client.0.smithi028.stdout:8/409: rename d0/d41/d50/d3/dc/f34 to d0/d41/d50/d3/d35/d86/f87 0 2024-07-03T15:44:06.563 INFO:tasks.workunit.client.0.smithi028.stdout:9/400: dread d1/d16/d57/f71 [0,4194304] 0 2024-07-03T15:44:06.563 INFO:tasks.workunit.client.0.smithi028.stdout:9/401: chown d1/c69 0 1 2024-07-03T15:44:06.564 INFO:tasks.workunit.client.0.smithi028.stdout:7/254: mknod d9/d1c/c4e 0 2024-07-03T15:44:06.564 INFO:tasks.workunit.client.0.smithi028.stdout:7/255: truncate d9/d29/f42 756070 0 2024-07-03T15:44:06.565 INFO:tasks.workunit.client.0.smithi028.stdout:7/256: write d9/f48 [385317,125810] 0 2024-07-03T15:44:06.565 INFO:tasks.workunit.client.0.smithi028.stdout:7/257: creat d9/d29/f4f x:0 0 0 2024-07-03T15:44:06.566 INFO:tasks.workunit.client.0.smithi028.stdout:8/410: unlink d0/d6c/c80 0 2024-07-03T15:44:06.567 INFO:tasks.workunit.client.0.smithi028.stdout:8/411: dread d0/d41/d50/d3/d35/d68/d22/d40/d62/f65 [0,4194304] 0 2024-07-03T15:44:06.567 INFO:tasks.workunit.client.0.smithi028.stdout:9/402: unlink d1/cb 0 2024-07-03T15:44:06.567 INFO:tasks.workunit.client.0.smithi028.stdout:9/403: dread - d1/d16/f63 zero size 2024-07-03T15:44:06.569 INFO:tasks.workunit.client.0.smithi028.stdout:7/258: symlink d9/d1c/d45/l50 0 2024-07-03T15:44:06.569 INFO:tasks.workunit.client.0.smithi028.stdout:8/412: creat d0/d41/f88 x:0 0 0 2024-07-03T15:44:06.570 INFO:tasks.workunit.client.0.smithi028.stdout:8/413: write d0/d41/d50/d3/d35/d68/d22/d40/d62/f65 [220220,99236] 0 2024-07-03T15:44:06.570 INFO:tasks.workunit.client.0.smithi028.stdout:8/414: write d0/d41/d50/d3/dc/d33/d7a/f15 [929847,108840] 0 2024-07-03T15:44:06.571 INFO:tasks.workunit.client.0.smithi028.stdout:9/404: unlink d1/d4a/f52 0 2024-07-03T15:44:06.571 INFO:tasks.workunit.client.0.smithi028.stdout:9/405: write d1/d16/d57/d5f/d64/f75 [230361,127880] 0 2024-07-03T15:44:06.571 INFO:tasks.workunit.client.0.smithi028.stdout:7/259: mknod d9/c51 0 2024-07-03T15:44:06.573 INFO:tasks.workunit.client.0.smithi028.stdout:9/406: creat d1/d16/d62/f8b x:0 0 0 2024-07-03T15:44:06.573 INFO:tasks.workunit.client.0.smithi028.stdout:7/260: truncate d9/f3a 930883 0 2024-07-03T15:44:06.577 INFO:tasks.workunit.client.0.smithi028.stdout:8/415: fsync d0/d41/d50/d3/d35/d86/f87 0 2024-07-03T15:44:06.578 INFO:tasks.workunit.client.0.smithi028.stdout:9/407: rename d1/d2a to d1/d10/d28/d45/d61/d8c 0 2024-07-03T15:44:06.579 INFO:tasks.workunit.client.0.smithi028.stdout:7/261: dread d9/f3b [0,4194304] 0 2024-07-03T15:44:06.579 INFO:tasks.workunit.client.0.smithi028.stdout:7/262: truncate d9/d29/f4f 974538 0 2024-07-03T15:44:06.581 INFO:tasks.workunit.client.0.smithi028.stdout:8/416: mknod d0/d41/d50/d3/d35/d86/c89 0 2024-07-03T15:44:06.585 INFO:tasks.workunit.client.0.smithi028.stdout:8/417: creat d0/d41/d50/d3/dc/d33/d7f/f8a x:0 0 0 2024-07-03T15:44:06.586 INFO:tasks.workunit.client.0.smithi028.stdout:8/418: dread - d0/d41/d50/d3/dc/f7e zero size 2024-07-03T15:44:06.591 INFO:tasks.workunit.client.0.smithi028.stdout:8/419: creat d0/d41/d50/d3/dc/d33/d7a/f8b x:0 0 0 2024-07-03T15:44:06.596 INFO:tasks.workunit.client.0.smithi028.stdout:8/420: creat d0/d41/d50/d3/d35/d68/d22/d40/f8c x:0 0 0 2024-07-03T15:44:06.598 INFO:tasks.workunit.client.0.smithi028.stdout:8/421: mkdir d0/d41/d50/d3/dc/d11/d8d 0 2024-07-03T15:44:06.612 INFO:tasks.workunit.client.0.smithi028.stdout:9/408: dwrite d1/dd/f55 [0,4194304] 0 2024-07-03T15:44:06.612 INFO:tasks.workunit.client.0.smithi028.stdout:9/409: dread - d1/d10/d28/d45/d61/d8c/d77/f85 zero size 2024-07-03T15:44:06.613 INFO:tasks.workunit.client.0.smithi028.stdout:9/410: mknod d1/d16/d57/d5f/d64/c8d 0 2024-07-03T15:44:06.615 INFO:tasks.workunit.client.0.smithi028.stdout:7/263: dwrite d9/f10 [0,4194304] 0 2024-07-03T15:44:06.619 INFO:tasks.workunit.client.0.smithi028.stdout:9/411: rmdir d1/d10/d28/d45/d61/d8c 39 2024-07-03T15:44:06.619 INFO:tasks.workunit.client.0.smithi028.stdout:8/422: dwrite d0/d41/d50/d3/dc/d33/d7a/f39 [0,4194304] 0 2024-07-03T15:44:06.622 INFO:tasks.workunit.client.0.smithi028.stdout:7/264: write f0 [7095363,8360] 0 2024-07-03T15:44:06.654 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:06 smithi148 ceph-mon[36056]: [03/Jul/2024:15:44:05] ENGINE Bus STARTING 2024-07-03T15:44:06.654 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:06 smithi148 ceph-mon[36056]: mgrmap e23: smithi148.rjzsar(active, since 3s) 2024-07-03T15:44:06.654 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:06 smithi148 ceph-mon[36056]: [03/Jul/2024:15:44:05] ENGINE Serving on https://172.21.15.148:7150 2024-07-03T15:44:06.654 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:06 smithi148 ceph-mon[36056]: [03/Jul/2024:15:44:05] ENGINE Error in HTTPServer.serve 2024-07-03T15:44:06.654 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Traceback (most recent call last): 2024-07-03T15:44:06.654 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-03T15:44:06.654 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: self._connections.run(self.expiration_interval) 2024-07-03T15:44:06.654 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-03T15:44:06.654 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: self._run(expiration_interval) 2024-07-03T15:44:06.654 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-03T15:44:06.655 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-07-03T15:44:06.655 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-03T15:44:06.655 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-03T15:44:06.655 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-03T15:44:06.655 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: s = self.context.wrap_socket( 2024-07-03T15:44:06.655 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-03T15:44:06.655 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: return self.sslsocket_class._create( 2024-07-03T15:44:06.655 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-03T15:44:06.655 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: self.do_handshake() 2024-07-03T15:44:06.655 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-03T15:44:06.655 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: self._sslobj.do_handshake() 2024-07-03T15:44:06.655 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-03T15:44:06.655 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:06 smithi148 ceph-mon[36056]: [03/Jul/2024:15:44:05] ENGINE Serving on http://172.21.15.148:8765 2024-07-03T15:44:06.656 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:06 smithi148 ceph-mon[36056]: [03/Jul/2024:15:44:05] ENGINE Bus STARTED 2024-07-03T15:44:06.663 INFO:tasks.workunit.client.0.smithi028.stdout:7/265: truncate f6 475372 0 2024-07-03T15:44:06.663 INFO:tasks.workunit.client.0.smithi028.stdout:7/266: write d9/f48 [1135711,95102] 0 2024-07-03T15:44:06.688 INFO:tasks.workunit.client.0.smithi028.stdout:8/423: rename d0/d41/d50/d3/dc/d11 to d0/d41/d50/d3/dc/d3f/d8e 0 2024-07-03T15:44:06.701 INFO:tasks.workunit.client.0.smithi028.stdout:9/412: getdents d1/d10/d28/d4e 0 2024-07-03T15:44:06.701 INFO:tasks.workunit.client.0.smithi028.stdout:9/413: write d1/d10/d28/d45/d61/d8c/d77/f85 [414606,56765] 0 2024-07-03T15:44:06.703 INFO:tasks.workunit.client.0.smithi028.stdout:8/424: symlink d0/d41/d50/d3/dc/d3f/d8e/d81/l8f 0 2024-07-03T15:44:06.707 INFO:tasks.workunit.client.0.smithi028.stdout:7/267: mkdir d9/d1c/d52 0 2024-07-03T15:44:06.710 INFO:tasks.workunit.client.0.smithi028.stdout:9/414: dread d1/d10/f18 [0,4194304] 0 2024-07-03T15:44:06.710 INFO:tasks.workunit.client.0.smithi028.stdout:9/415: fsync d1/f2d 0 2024-07-03T15:44:06.710 INFO:tasks.workunit.client.0.smithi028.stdout:9/416: chown d1/d16 1563385936 1 2024-07-03T15:44:06.711 INFO:tasks.workunit.client.0.smithi028.stdout:9/417: readlink d1/l53 0 2024-07-03T15:44:06.715 INFO:tasks.workunit.client.0.smithi028.stdout:9/418: mkdir d1/d8e 0 2024-07-03T15:44:06.715 INFO:tasks.workunit.client.0.smithi028.stdout:9/419: creat d1/d16/d57/d5f/f8f x:0 0 0 2024-07-03T15:44:06.715 INFO:tasks.workunit.client.0.smithi028.stdout:9/420: truncate d1/d16/d62/f8b 974092 0 2024-07-03T15:44:06.717 INFO:tasks.workunit.client.0.smithi028.stdout:7/268: rmdir d9 39 2024-07-03T15:44:06.718 INFO:tasks.workunit.client.0.smithi028.stdout:7/269: write f6 [502710,128278] 0 2024-07-03T15:44:06.723 INFO:tasks.workunit.client.0.smithi028.stdout:9/421: link d1/f21 d1/dd/f90 0 2024-07-03T15:44:06.723 INFO:tasks.workunit.client.0.smithi028.stdout:9/422: write d1/d16/f63 [12220,14894] 0 2024-07-03T15:44:06.723 INFO:tasks.workunit.client.0.smithi028.stdout:9/423: chown d1/d16/f2e 11267 1 2024-07-03T15:44:06.726 INFO:tasks.workunit.client.0.smithi028.stdout:7/270: truncate d9/f3b 3307475 0 2024-07-03T15:44:06.727 INFO:tasks.workunit.client.0.smithi028.stdout:7/271: chown d9/cb 0 1 2024-07-03T15:44:06.727 INFO:tasks.workunit.client.0.smithi028.stdout:7/272: write d9/f34 [956484,129562] 0 2024-07-03T15:44:06.727 INFO:tasks.workunit.client.0.smithi028.stdout:7/273: chown d9/d1c/f4b 3 1 2024-07-03T15:44:06.727 INFO:tasks.workunit.client.0.smithi028.stdout:7/274: readlink d9/d29/l2b 0 2024-07-03T15:44:06.727 INFO:tasks.workunit.client.0.smithi028.stdout:7/275: readlink d9/l14 0 2024-07-03T15:44:06.727 INFO:tasks.workunit.client.0.smithi028.stdout:7/276: chown d9/d1c/c37 1 1 2024-07-03T15:44:06.733 INFO:tasks.workunit.client.0.smithi028.stdout:7/277: unlink d9/fe 0 2024-07-03T15:44:06.739 INFO:tasks.workunit.client.0.smithi028.stdout:7/278: unlink d9/l16 0 2024-07-03T15:44:06.740 INFO:tasks.workunit.client.0.smithi028.stdout:9/424: dwrite d1/dd/f6f [0,4194304] 0 2024-07-03T15:44:06.744 INFO:tasks.workunit.client.0.smithi028.stdout:7/279: dread d9/f3b [0,4194304] 0 2024-07-03T15:44:06.746 INFO:tasks.workunit.client.0.smithi028.stdout:9/425: creat d1/d10/d28/d45/d61/d8c/d77/f91 x:0 0 0 2024-07-03T15:44:06.748 INFO:tasks.workunit.client.0.smithi028.stdout:0/414: sync 2024-07-03T15:44:06.748 INFO:tasks.workunit.client.0.smithi028.stdout:6/348: sync 2024-07-03T15:44:06.748 INFO:tasks.workunit.client.0.smithi028.stdout:7/280: creat d9/d1c/d45/f53 x:0 0 0 2024-07-03T15:44:06.751 INFO:tasks.workunit.client.0.smithi028.stdout:9/426: dread d1/d16/f2f [0,4194304] 0 2024-07-03T15:44:06.754 INFO:tasks.workunit.client.0.smithi028.stdout:6/349: rename d4/da/f44 to d4/f77 0 2024-07-03T15:44:06.755 INFO:tasks.workunit.client.0.smithi028.stdout:6/350: fsync d4/f45 0 2024-07-03T15:44:06.756 INFO:tasks.workunit.client.0.smithi028.stdout:9/427: unlink d1/d10/d28/d45/d56/l42 0 2024-07-03T15:44:06.757 INFO:tasks.workunit.client.0.smithi028.stdout:0/415: dread d2/d3/f11 [0,4194304] 0 2024-07-03T15:44:06.757 INFO:tasks.workunit.client.0.smithi028.stdout:7/281: symlink d9/d1c/l54 0 2024-07-03T15:44:06.761 INFO:tasks.workunit.client.0.smithi028.stdout:0/416: unlink d2/d1c/d32/c77 0 2024-07-03T15:44:06.761 INFO:tasks.workunit.client.0.smithi028.stdout:0/417: fdatasync d2/d1c/f2b 0 2024-07-03T15:44:06.762 INFO:tasks.workunit.client.0.smithi028.stdout:0/418: stat d2/d1c/f83 0 2024-07-03T15:44:06.762 INFO:tasks.workunit.client.0.smithi028.stdout:0/419: fdatasync d2/d1c/d32/d58/d6e/f80 0 2024-07-03T15:44:06.762 INFO:tasks.workunit.client.0.smithi028.stdout:0/420: fdatasync d2/d3/f73 0 2024-07-03T15:44:06.762 INFO:tasks.workunit.client.0.smithi028.stdout:0/421: chown d2/c5 265647 1 2024-07-03T15:44:06.762 INFO:tasks.workunit.client.0.smithi028.stdout:7/282: symlink d9/d1c/d52/l55 0 2024-07-03T15:44:06.762 INFO:tasks.workunit.client.0.smithi028.stdout:9/428: dread d1/dd/f59 [0,4194304] 0 2024-07-03T15:44:06.763 INFO:tasks.workunit.client.0.smithi028.stdout:6/351: rmdir d4/da/d56 39 2024-07-03T15:44:06.763 INFO:tasks.workunit.client.0.smithi028.stdout:6/352: chown d4/da/d38/l75 19203 1 2024-07-03T15:44:06.764 INFO:tasks.workunit.client.0.smithi028.stdout:7/283: dread d9/f1f [0,4194304] 0 2024-07-03T15:44:06.765 INFO:tasks.workunit.client.0.smithi028.stdout:0/422: rename d2/c4 to d2/d1c/d32/d36/d45/c84 0 2024-07-03T15:44:06.765 INFO:tasks.workunit.client.0.smithi028.stdout:0/423: chown d2/d1c/d32/d36/f3d 102540136 1 2024-07-03T15:44:06.765 INFO:tasks.workunit.client.0.smithi028.stdout:0/424: chown d2/d3/d4d/d56 76793 1 2024-07-03T15:44:06.766 INFO:tasks.workunit.client.0.smithi028.stdout:9/429: getdents d1/d10/d28/d45/d61/d8c 0 2024-07-03T15:44:06.766 INFO:tasks.workunit.client.0.smithi028.stdout:9/430: write d1/d16/f63 [948892,9714] 0 2024-07-03T15:44:06.768 INFO:tasks.workunit.client.0.smithi028.stdout:7/284: rename d9/d1c/c43 to d9/c56 0 2024-07-03T15:44:06.768 INFO:tasks.workunit.client.0.smithi028.stdout:7/285: write d9/d1c/d45/f46 [168100,69998] 0 2024-07-03T15:44:06.768 INFO:tasks.workunit.client.0.smithi028.stdout:7/286: chown d9/d1c/c36 0 1 2024-07-03T15:44:06.768 INFO:tasks.workunit.client.0.smithi028.stdout:7/287: creat d9/d1c/d45/f57 x:0 0 0 2024-07-03T15:44:06.768 INFO:tasks.workunit.client.0.smithi028.stdout:7/288: chown d9/d29/c47 36 1 2024-07-03T15:44:06.771 INFO:tasks.workunit.client.0.smithi028.stdout:0/425: unlink d2/d1c/d6c/f71 0 2024-07-03T15:44:06.772 INFO:tasks.workunit.client.0.smithi028.stdout:9/431: rename d1/f41 to d1/d4a/f92 0 2024-07-03T15:44:06.773 INFO:tasks.workunit.client.0.smithi028.stdout:7/289: dread d9/f3a [0,4194304] 0 2024-07-03T15:44:06.774 INFO:tasks.workunit.client.0.smithi028.stdout:7/290: chown d9/l4d 1905 1 2024-07-03T15:44:06.774 INFO:tasks.workunit.client.0.smithi028.stdout:7/291: chown f0 4092084 1 2024-07-03T15:44:06.774 INFO:tasks.workunit.client.0.smithi028.stdout:7/292: dread - d9/d1c/f4b zero size 2024-07-03T15:44:06.776 INFO:tasks.workunit.client.0.smithi028.stdout:0/426: rename d2/d3/f4c to d2/d1c/d6c/f85 0 2024-07-03T15:44:06.777 INFO:tasks.workunit.client.0.smithi028.stdout:9/432: getdents d1/d16/d57/d60 0 2024-07-03T15:44:06.778 INFO:tasks.workunit.client.0.smithi028.stdout:7/293: creat d9/d1c/d52/f58 x:0 0 0 2024-07-03T15:44:06.778 INFO:tasks.workunit.client.0.smithi028.stdout:7/294: truncate d9/fa 1508138 0 2024-07-03T15:44:06.779 INFO:tasks.workunit.client.0.smithi028.stdout:0/427: symlink d2/d3/d4d/d56/l86 0 2024-07-03T15:44:06.783 INFO:tasks.workunit.client.0.smithi028.stdout:7/295: link d9/c24 d9/d1c/d52/c59 0 2024-07-03T15:44:06.783 INFO:tasks.workunit.client.0.smithi028.stdout:6/353: dwrite d4/f33 [0,4194304] 0 2024-07-03T15:44:06.784 INFO:tasks.workunit.client.0.smithi028.stdout:6/354: write d4/da/d38/f40 [1630363,89421] 0 2024-07-03T15:44:06.785 INFO:tasks.workunit.client.0.smithi028.stdout:0/428: unlink d2/d1c/d32/f39 0 2024-07-03T15:44:06.785 INFO:tasks.workunit.client.0.smithi028.stdout:0/429: readlink d2/d3/l22 0 2024-07-03T15:44:06.785 INFO:tasks.workunit.client.0.smithi028.stdout:0/430: chown d2/d1c/d32/d58/d6e/f80 64581432 1 2024-07-03T15:44:06.785 INFO:tasks.workunit.client.0.smithi028.stdout:0/431: write d2/d1c/d32/d36/d4a/f6f [54902,74972] 0 2024-07-03T15:44:06.785 INFO:tasks.workunit.client.0.smithi028.stdout:0/432: write d2/d1c/d32/f38 [1296831,76556] 0 2024-07-03T15:44:06.785 INFO:tasks.workunit.client.0.smithi028.stdout:0/433: creat d2/d1c/d32/d36/f87 x:0 0 0 2024-07-03T15:44:06.785 INFO:tasks.workunit.client.0.smithi028.stdout:0/434: creat d2/d1c/d32/d36/f88 x:0 0 0 2024-07-03T15:44:06.785 INFO:tasks.workunit.client.0.smithi028.stdout:7/296: creat d9/d1c/f5a x:0 0 0 2024-07-03T15:44:06.786 INFO:tasks.workunit.client.0.smithi028.stdout:7/297: write d9/d29/f42 [816835,96319] 0 2024-07-03T15:44:06.787 INFO:tasks.workunit.client.0.smithi028.stdout:6/355: unlink d4/da/c37 0 2024-07-03T15:44:06.791 INFO:tasks.workunit.client.0.smithi028.stdout:0/435: unlink d2/d1c/d32/f35 0 2024-07-03T15:44:06.791 INFO:tasks.workunit.client.0.smithi028.stdout:0/436: write d2/d1c/f68 [1406594,56972] 0 2024-07-03T15:44:06.792 INFO:tasks.workunit.client.0.smithi028.stdout:6/356: rmdir d4/da/d38/d3d/d69 0 2024-07-03T15:44:06.792 INFO:tasks.workunit.client.0.smithi028.stdout:7/298: write d9/f10 [3419494,11085] 0 2024-07-03T15:44:06.793 INFO:tasks.workunit.client.0.smithi028.stdout:7/299: write d9/f3a [1600383,98482] 0 2024-07-03T15:44:06.793 INFO:tasks.workunit.client.0.smithi028.stdout:0/437: symlink d2/d1c/d32/d36/d45/l89 0 2024-07-03T15:44:06.794 INFO:tasks.workunit.client.0.smithi028.stdout:6/357: getdents d4/da/d38/d59 0 2024-07-03T15:44:06.795 INFO:tasks.workunit.client.0.smithi028.stdout:7/300: mknod d9/c5b 0 2024-07-03T15:44:06.797 INFO:tasks.workunit.client.0.smithi028.stdout:7/301: creat d9/d1c/f5c x:0 0 0 2024-07-03T15:44:06.798 INFO:tasks.workunit.client.0.smithi028.stdout:6/358: mkdir d4/d78 0 2024-07-03T15:44:06.798 INFO:tasks.workunit.client.0.smithi028.stdout:9/433: dwrite d1/d17/f25 [0,4194304] 0 2024-07-03T15:44:06.799 INFO:tasks.workunit.client.0.smithi028.stdout:7/302: unlink d9/c24 0 2024-07-03T15:44:06.800 INFO:tasks.workunit.client.0.smithi028.stdout:7/303: stat d9/f3b 0 2024-07-03T15:44:06.801 INFO:tasks.workunit.client.0.smithi028.stdout:6/359: mknod d4/da/d56/c79 0 2024-07-03T15:44:06.801 INFO:tasks.workunit.client.0.smithi028.stdout:6/360: fdatasync d4/f29 0 2024-07-03T15:44:06.802 INFO:tasks.workunit.client.0.smithi028.stdout:6/361: write d4/f29 [4428225,88356] 0 2024-07-03T15:44:06.810 INFO:tasks.workunit.client.0.smithi028.stdout:6/362: dread d4/da/d1c/f61 [0,4194304] 0 2024-07-03T15:44:06.817 INFO:tasks.workunit.client.0.smithi028.stdout:0/438: dwrite d2/d1c/d32/d36/f65 [0,4194304] 0 2024-07-03T15:44:06.820 INFO:tasks.workunit.client.0.smithi028.stdout:0/439: write d2/d3/d4d/f64 [562424,130233] 0 2024-07-03T15:44:06.822 INFO:tasks.workunit.client.0.smithi028.stdout:7/304: dwrite d9/d1c/d52/f58 [0,4194304] 0 2024-07-03T15:44:06.822 INFO:tasks.workunit.client.0.smithi028.stdout:7/305: read - d9/d1c/d45/f53 zero size 2024-07-03T15:44:06.824 INFO:tasks.workunit.client.0.smithi028.stdout:0/440: mknod d2/d1c/d32/d58/c8a 0 2024-07-03T15:44:06.827 INFO:tasks.workunit.client.0.smithi028.stdout:9/434: dwrite d1/d10/d28/d45/d61/d8c/d77/f85 [0,4194304] 0 2024-07-03T15:44:06.828 INFO:tasks.workunit.client.0.smithi028.stdout:7/306: mkdir d9/d29/d5d 0 2024-07-03T15:44:06.830 INFO:tasks.workunit.client.0.smithi028.stdout:0/441: mknod d2/d1c/d32/d36/c8b 0 2024-07-03T15:44:06.832 INFO:tasks.workunit.client.0.smithi028.stdout:6/363: dwrite d4/da/d1c/f1e [0,4194304] 0 2024-07-03T15:44:06.839 INFO:tasks.workunit.client.0.smithi028.stdout:0/442: symlink d2/d1c/d32/d36/d45/l8c 0 2024-07-03T15:44:06.839 INFO:tasks.workunit.client.0.smithi028.stdout:0/443: readlink d2/d1c/d32/d36/l6b 0 2024-07-03T15:44:06.843 INFO:tasks.workunit.client.0.smithi028.stdout:9/435: dwrite d1/dd/f3c [4194304,4194304] 0 2024-07-03T15:44:06.862 INFO:tasks.workunit.client.0.smithi028.stdout:6/364: truncate d4/d15/f20 3979256 0 2024-07-03T15:44:06.862 INFO:tasks.workunit.client.0.smithi028.stdout:6/365: chown d4/d12/d2c 83534 1 2024-07-03T15:44:06.867 INFO:tasks.workunit.client.0.smithi028.stdout:0/444: truncate d2/d1c/d32/d36/f3d 2312541 0 2024-07-03T15:44:06.867 INFO:tasks.workunit.client.0.smithi028.stdout:0/445: write d2/d1c/d32/d58/f6a [528399,100136] 0 2024-07-03T15:44:06.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:06 smithi028 ceph-mon[29148]: [03/Jul/2024:15:44:05] ENGINE Bus STARTING 2024-07-03T15:44:06.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:06 smithi028 ceph-mon[29148]: mgrmap e23: smithi148.rjzsar(active, since 3s) 2024-07-03T15:44:06.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:06 smithi028 ceph-mon[29148]: [03/Jul/2024:15:44:05] ENGINE Serving on https://172.21.15.148:7150 2024-07-03T15:44:06.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:06 smithi028 ceph-mon[29148]: [03/Jul/2024:15:44:05] ENGINE Error in HTTPServer.serve 2024-07-03T15:44:06.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Traceback (most recent call last): 2024-07-03T15:44:06.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-03T15:44:06.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: self._connections.run(self.expiration_interval) 2024-07-03T15:44:06.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-03T15:44:06.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: self._run(expiration_interval) 2024-07-03T15:44:06.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-03T15:44:06.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-07-03T15:44:06.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-03T15:44:06.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-03T15:44:06.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-03T15:44:06.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: s = self.context.wrap_socket( 2024-07-03T15:44:06.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-03T15:44:06.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: return self.sslsocket_class._create( 2024-07-03T15:44:06.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-03T15:44:06.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: self.do_handshake() 2024-07-03T15:44:06.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-03T15:44:06.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: self._sslobj.do_handshake() 2024-07-03T15:44:06.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-03T15:44:06.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:06 smithi028 ceph-mon[29148]: [03/Jul/2024:15:44:05] ENGINE Serving on http://172.21.15.148:8765 2024-07-03T15:44:06.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:06 smithi028 ceph-mon[29148]: [03/Jul/2024:15:44:05] ENGINE Bus STARTED 2024-07-03T15:44:06.879 INFO:tasks.workunit.client.0.smithi028.stdout:6/366: mknod d4/da/d38/d59/c7a 0 2024-07-03T15:44:06.882 INFO:tasks.workunit.client.0.smithi028.stdout:0/446: creat d2/d3/f8d x:0 0 0 2024-07-03T15:44:06.882 INFO:tasks.workunit.client.0.smithi028.stdout:0/447: write d2/d3/d4d/d56/f7f [1191732,9531] 0 2024-07-03T15:44:06.882 INFO:tasks.workunit.client.0.smithi028.stdout:0/448: dread - d2/d1c/d32/d36/f88 zero size 2024-07-03T15:44:06.882 INFO:tasks.workunit.client.0.smithi028.stdout:0/449: write d2/d1c/d6c/f79 [1014498,36760] 0 2024-07-03T15:44:06.882 INFO:tasks.workunit.client.0.smithi028.stdout:0/450: readlink d2/l28 0 2024-07-03T15:44:06.882 INFO:tasks.workunit.client.0.smithi028.stdout:0/451: fsync d2/d1c/f2d 0 2024-07-03T15:44:06.886 INFO:tasks.workunit.client.0.smithi028.stdout:9/436: getdents d1 0 2024-07-03T15:44:06.890 INFO:tasks.workunit.client.0.smithi028.stdout:0/452: symlink d2/d1c/d32/d36/d4a/l8e 0 2024-07-03T15:44:06.894 INFO:tasks.workunit.client.0.smithi028.stdout:9/437: mknod d1/d16/d57/d5f/d7f/c93 0 2024-07-03T15:44:06.895 INFO:tasks.workunit.client.0.smithi028.stdout:9/438: symlink d1/dd/l94 0 2024-07-03T15:44:06.896 INFO:tasks.workunit.client.0.smithi028.stdout:6/367: dwrite d4/da/d38/d59/f65 [0,4194304] 0 2024-07-03T15:44:06.896 INFO:tasks.workunit.client.0.smithi028.stdout:6/368: write d4/da/d56/f73 [333680,123061] 0 2024-07-03T15:44:06.897 INFO:tasks.workunit.client.0.smithi028.stdout:9/439: truncate d1/dd/f90 1375092 0 2024-07-03T15:44:06.907 INFO:tasks.workunit.client.0.smithi028.stdout:6/369: write d4/da/f4e [2748826,3094] 0 2024-07-03T15:44:06.908 INFO:tasks.workunit.client.0.smithi028.stdout:9/440: mkdir d1/d10/d95 0 2024-07-03T15:44:06.912 INFO:tasks.workunit.client.0.smithi028.stdout:0/453: dwrite d2/d1c/d32/d36/f65 [0,4194304] 0 2024-07-03T15:44:06.920 INFO:tasks.workunit.client.0.smithi028.stdout:6/370: write d4/da/d13/f19 [3522619,63161] 0 2024-07-03T15:44:06.920 INFO:tasks.workunit.client.0.smithi028.stdout:6/371: dread - d4/da/d47/f6b zero size 2024-07-03T15:44:06.921 INFO:tasks.workunit.client.0.smithi028.stdout:0/454: getdents d2/d1c/d32/d58/d6e 0 2024-07-03T15:44:06.930 INFO:tasks.workunit.client.0.smithi028.stdout:6/372: dread d4/f10 [0,4194304] 0 2024-07-03T15:44:06.931 INFO:tasks.workunit.client.0.smithi028.stdout:6/373: creat d4/f7b x:0 0 0 2024-07-03T15:44:06.931 INFO:tasks.workunit.client.0.smithi028.stdout:6/374: read d4/d15/f5b [1021575,35781] 0 2024-07-03T15:44:06.934 INFO:tasks.workunit.client.0.smithi028.stdout:9/441: dwrite d1/f43 [0,4194304] 0 2024-07-03T15:44:06.934 INFO:tasks.workunit.client.0.smithi028.stdout:9/442: dread - d1/d10/d28/d45/d61/d8c/d77/f91 zero size 2024-07-03T15:44:06.940 INFO:tasks.workunit.client.0.smithi028.stdout:0/455: dread d2/d1c/f2b [0,4194304] 0 2024-07-03T15:44:06.941 INFO:tasks.workunit.client.0.smithi028.stdout:9/443: mkdir d1/d96 0 2024-07-03T15:44:06.944 INFO:tasks.workunit.client.0.smithi028.stdout:9/444: rename d1/d10/d28/d45/d61/d8c/c36 to d1/d10/d95/c97 0 2024-07-03T15:44:06.945 INFO:tasks.workunit.client.0.smithi028.stdout:6/375: dwrite d4/da/d1c/f1e [4194304,4194304] 0 2024-07-03T15:44:06.947 INFO:tasks.workunit.client.0.smithi028.stdout:9/445: mknod d1/d10/d28/c98 0 2024-07-03T15:44:06.951 INFO:tasks.workunit.client.0.smithi028.stdout:6/376: dread d4/da/d38/f5e [0,4194304] 0 2024-07-03T15:44:06.951 INFO:tasks.workunit.client.0.smithi028.stdout:6/377: write d4/d15/f6a [90573,57922] 0 2024-07-03T15:44:06.952 INFO:tasks.workunit.client.0.smithi028.stdout:0/456: truncate d2/d3/d4d/d56/f7f 3627296 0 2024-07-03T15:44:06.953 INFO:tasks.workunit.client.0.smithi028.stdout:0/457: dread - d2/d1c/f76 zero size 2024-07-03T15:44:06.953 INFO:tasks.workunit.client.0.smithi028.stdout:0/458: dread - d2/f3a zero size 2024-07-03T15:44:06.961 INFO:tasks.workunit.client.0.smithi028.stdout:6/378: dread d4/f33 [0,4194304] 0 2024-07-03T15:44:06.971 INFO:tasks.workunit.client.0.smithi028.stdout:9/446: dwrite d1/f2 [0,4194304] 0 2024-07-03T15:44:06.971 INFO:tasks.workunit.client.0.smithi028.stdout:9/447: stat d1/d16/d57/d5f 0 2024-07-03T15:44:06.972 INFO:tasks.workunit.client.0.smithi028.stdout:9/448: getdents d1/d10/d28/d45 0 2024-07-03T15:44:06.972 INFO:tasks.workunit.client.0.smithi028.stdout:9/449: stat d1/d16/d57/d5f/f7a 0 2024-07-03T15:44:06.973 INFO:tasks.workunit.client.0.smithi028.stdout:9/450: symlink d1/d10/d28/d45/l99 0 2024-07-03T15:44:06.973 INFO:tasks.workunit.client.0.smithi028.stdout:9/451: chown d1/f2 217 1 2024-07-03T15:44:06.974 INFO:tasks.workunit.client.0.smithi028.stdout:4/405: sync 2024-07-03T15:44:06.975 INFO:tasks.workunit.client.0.smithi028.stdout:4/406: readlink d2/d12/d3b/l3c 0 2024-07-03T15:44:06.975 INFO:tasks.workunit.client.0.smithi028.stdout:4/407: creat d2/d12/d5b/f85 x:0 0 0 2024-07-03T15:44:06.976 INFO:tasks.workunit.client.0.smithi028.stdout:9/452: link d1/f21 d1/d17/f9a 0 2024-07-03T15:44:06.979 INFO:tasks.workunit.client.0.smithi028.stdout:4/408: rename d2/d12/d2c/f76 to d2/f86 0 2024-07-03T15:44:06.979 INFO:tasks.workunit.client.0.smithi028.stdout:4/409: chown d2/d61/c6e 3 1 2024-07-03T15:44:06.980 INFO:tasks.workunit.client.0.smithi028.stdout:9/453: mknod d1/d16/d62/c9b 0 2024-07-03T15:44:06.982 INFO:tasks.workunit.client.0.smithi028.stdout:9/454: readlink d1/d10/l32 0 2024-07-03T15:44:06.982 INFO:tasks.workunit.client.0.smithi028.stdout:9/455: creat d1/d16/d57/f9c x:0 0 0 2024-07-03T15:44:06.986 INFO:tasks.workunit.client.0.smithi028.stdout:9/456: unlink d1/f2d 0 2024-07-03T15:44:06.986 INFO:tasks.workunit.client.0.smithi028.stdout:9/457: read - d1/d10/d28/d4e/d5b/f84 zero size 2024-07-03T15:44:06.991 INFO:tasks.workunit.client.0.smithi028.stdout:4/410: dwrite d2/d12/f28 [0,4194304] 0 2024-07-03T15:44:06.991 INFO:tasks.workunit.client.0.smithi028.stdout:4/411: write d2/d12/d2d/d2f/f66 [421183,96743] 0 2024-07-03T15:44:06.995 INFO:tasks.workunit.client.0.smithi028.stdout:4/412: mkdir d2/d12/d2e/d87 0 2024-07-03T15:44:06.996 INFO:tasks.workunit.client.0.smithi028.stdout:4/413: write d2/d5a/f9 [19976,16729] 0 2024-07-03T15:44:06.996 INFO:tasks.workunit.client.0.smithi028.stdout:4/414: dread - d2/d12/d42/d63/f65 zero size 2024-07-03T15:44:06.997 INFO:tasks.workunit.client.0.smithi028.stdout:4/415: symlink d2/d12/d2e/d87/l88 0 2024-07-03T15:44:06.997 INFO:tasks.workunit.client.0.smithi028.stdout:4/416: readlink d2/d5a/l29 0 2024-07-03T15:44:06.997 INFO:tasks.workunit.client.0.smithi028.stdout:4/417: chown d2/d12/d2d/f32 996 1 2024-07-03T15:44:06.998 INFO:tasks.workunit.client.0.smithi028.stdout:4/418: rmdir d2/d12/d2e/d87 39 2024-07-03T15:44:07.003 INFO:tasks.workunit.client.0.smithi028.stdout:9/458: dwrite d1/d16/d57/d5f/f7a [0,4194304] 0 2024-07-03T15:44:07.003 INFO:tasks.workunit.client.0.smithi028.stdout:9/459: readlink d1/l8 0 2024-07-03T15:44:07.007 INFO:tasks.workunit.client.0.smithi028.stdout:4/419: read d2/d12/f28 [3567933,124248] 0 2024-07-03T15:44:07.010 INFO:tasks.workunit.client.0.smithi028.stdout:4/420: link d2/d12/d2e/c30 d2/d61/c89 0 2024-07-03T15:44:07.013 INFO:tasks.workunit.client.0.smithi028.stdout:4/421: dread d2/d5a/f34 [0,4194304] 0 2024-07-03T15:44:07.013 INFO:tasks.workunit.client.0.smithi028.stdout:4/422: fsync d2/d3e/f3f 0 2024-07-03T15:44:07.019 INFO:tasks.workunit.client.0.smithi028.stdout:4/423: creat d2/d3e/f8a x:0 0 0 2024-07-03T15:44:07.019 INFO:tasks.workunit.client.0.smithi028.stdout:9/460: dwrite d1/d10/d28/d4e/d4f/f87 [0,4194304] 0 2024-07-03T15:44:07.021 INFO:tasks.workunit.client.0.smithi028.stdout:4/424: mknod d2/d12/d42/d63/d6c/c8b 0 2024-07-03T15:44:07.021 INFO:tasks.workunit.client.0.smithi028.stdout:4/425: readlink d2/d12/l5d 0 2024-07-03T15:44:07.021 INFO:tasks.workunit.client.0.smithi028.stdout:4/426: creat d2/d12/d5b/f8c x:0 0 0 2024-07-03T15:44:07.025 INFO:tasks.workunit.client.0.smithi028.stdout:9/461: creat d1/d17/f9d x:0 0 0 2024-07-03T15:44:07.028 INFO:tasks.workunit.client.0.smithi028.stdout:9/462: creat d1/d8e/f9e x:0 0 0 2024-07-03T15:44:07.029 INFO:tasks.workunit.client.0.smithi028.stdout:4/427: rmdir d2/d5a/d45 39 2024-07-03T15:44:07.032 INFO:tasks.workunit.client.0.smithi028.stdout:4/428: rename d2/d12/d2d/d2f/c57 to d2/d12/d2d/d2f/d3a/d6a/c8d 0 2024-07-03T15:44:07.032 INFO:tasks.workunit.client.0.smithi028.stdout:4/429: fsync d2/d12/d3b/f43 0 2024-07-03T15:44:07.032 INFO:tasks.workunit.client.0.smithi028.stdout:4/430: readlink d2/d12/d2d/d2f/d3a/l4d 0 2024-07-03T15:44:07.032 INFO:tasks.workunit.client.0.smithi028.stdout:4/431: readlink d2/d12/d2d/d2f/l46 0 2024-07-03T15:44:07.036 INFO:tasks.workunit.client.0.smithi028.stdout:9/463: dread d1/d10/f18 [0,4194304] 0 2024-07-03T15:44:07.043 INFO:tasks.workunit.client.0.smithi028.stdout:4/432: dwrite d2/d12/d5b/f8c [0,4194304] 0 2024-07-03T15:44:07.047 INFO:tasks.workunit.client.0.smithi028.stdout:4/433: symlink d2/d12/d2d/d2f/d3a/d6a/l8e 0 2024-07-03T15:44:07.051 INFO:tasks.workunit.client.0.smithi028.stdout:9/464: dwrite d1/d4a/f65 [0,4194304] 0 2024-07-03T15:44:07.051 INFO:tasks.workunit.client.0.smithi028.stdout:9/465: write d1/d8e/f9e [538240,47578] 0 2024-07-03T15:44:07.052 INFO:tasks.workunit.client.0.smithi028.stdout:4/434: dread d2/d12/f60 [0,4194304] 0 2024-07-03T15:44:07.055 INFO:tasks.workunit.client.0.smithi028.stdout:9/466: link d1/d10/d28/d4e/d4f/f67 d1/d10/d28/d4e/f9f 0 2024-07-03T15:44:07.060 INFO:tasks.workunit.client.0.smithi028.stdout:4/435: rename d2/d61/f82 to d2/d5a/d45/d84/f8f 0 2024-07-03T15:44:07.063 INFO:tasks.workunit.client.0.smithi028.stdout:9/467: creat d1/d10/d28/d45/d61/fa0 x:0 0 0 2024-07-03T15:44:07.063 INFO:tasks.workunit.client.0.smithi028.stdout:9/468: write d1/f88 [33232,64002] 0 2024-07-03T15:44:07.063 INFO:tasks.workunit.client.0.smithi028.stdout:9/469: chown d1/d16/f2f 0 1 2024-07-03T15:44:07.065 INFO:tasks.workunit.client.0.smithi028.stdout:4/436: dread d2/d3e/f50 [0,4194304] 0 2024-07-03T15:44:07.068 INFO:tasks.workunit.client.0.smithi028.stdout:9/470: rename d1/f2b to d1/fa1 0 2024-07-03T15:44:07.070 INFO:tasks.workunit.client.0.smithi028.stdout:4/437: truncate d2/d5a/f56 815081 0 2024-07-03T15:44:07.075 INFO:tasks.workunit.client.0.smithi028.stdout:4/438: symlink d2/d12/d2d/d2f/l90 0 2024-07-03T15:44:07.077 INFO:tasks.workunit.client.0.smithi028.stdout:9/471: mknod d1/d10/ca2 0 2024-07-03T15:44:07.077 INFO:tasks.workunit.client.0.smithi028.stdout:9/472: fdatasync d1/d16/f19 0 2024-07-03T15:44:07.080 INFO:tasks.workunit.client.0.smithi028.stdout:4/439: link d2/d12/d2d/d58/f70 d2/d61/f91 0 2024-07-03T15:44:07.085 INFO:tasks.workunit.client.0.smithi028.stdout:4/440: mkdir d2/d61/d92 0 2024-07-03T15:44:07.086 INFO:tasks.workunit.client.0.smithi028.stdout:4/441: dread - d2/d12/d5b/d78/f79 zero size 2024-07-03T15:44:07.087 INFO:tasks.workunit.client.0.smithi028.stdout:4/442: read d2/d12/f60 [215615,31816] 0 2024-07-03T15:44:07.087 INFO:tasks.workunit.client.0.smithi028.stdout:9/473: rmdir d1/d8e 39 2024-07-03T15:44:07.090 INFO:tasks.workunit.client.0.smithi028.stdout:9/474: getdents d1/d10/d76 0 2024-07-03T15:44:07.091 INFO:tasks.workunit.client.0.smithi028.stdout:9/475: write d1/d17/f25 [4348485,10538] 0 2024-07-03T15:44:07.096 INFO:tasks.workunit.client.0.smithi028.stdout:4/443: dread d2/d12/d2e/f47 [0,4194304] 0 2024-07-03T15:44:07.096 INFO:tasks.workunit.client.0.smithi028.stdout:9/476: mknod d1/d10/d28/d45/ca3 0 2024-07-03T15:44:07.102 INFO:tasks.workunit.client.0.smithi028.stdout:9/477: rename d1/d10/l32 to d1/dd/la4 0 2024-07-03T15:44:07.110 INFO:tasks.workunit.client.0.smithi028.stdout:9/478: creat d1/d10/d28/fa5 x:0 0 0 2024-07-03T15:44:07.110 INFO:tasks.workunit.client.0.smithi028.stdout:9/479: chown d1/d10/d28/d4e/d4f/f67 5186565 1 2024-07-03T15:44:07.214 INFO:tasks.workunit.client.0.smithi028.stdout:5/417: sync 2024-07-03T15:44:07.214 INFO:tasks.workunit.client.0.smithi028.stdout:1/555: sync 2024-07-03T15:44:07.214 INFO:tasks.workunit.client.0.smithi028.stdout:3/450: sync 2024-07-03T15:44:07.214 INFO:tasks.workunit.client.0.smithi028.stdout:5/418: chown d0/dd/d13/d57/d26/d41/f43 122412055 1 2024-07-03T15:44:07.214 INFO:tasks.workunit.client.0.smithi028.stdout:3/451: read - d18/d1a/d5e/f7b zero size 2024-07-03T15:44:07.214 INFO:tasks.workunit.client.0.smithi028.stdout:3/452: fsync f9 0 2024-07-03T15:44:07.214 INFO:tasks.workunit.client.0.smithi028.stdout:1/556: symlink d4/d3c/d54/d79/d16/d7c/lcb 0 2024-07-03T15:44:07.215 INFO:tasks.workunit.client.0.smithi028.stdout:5/419: mknod d0/dd/d13/d57/d30/c8a 0 2024-07-03T15:44:07.221 INFO:tasks.workunit.client.0.smithi028.stdout:5/420: dread d0/dd/d13/d57/d26/d41/f44 [0,4194304] 0 2024-07-03T15:44:07.310 INFO:tasks.workunit.client.0.smithi028.stdout:5/421: symlink d0/dd/d1e/d3b/d3f/d5a/l8b 0 2024-07-03T15:44:07.316 INFO:tasks.workunit.client.0.smithi028.stdout:5/422: rmdir d0/dd/d1e/d86 39 2024-07-03T15:44:07.322 INFO:tasks.workunit.client.0.smithi028.stdout:5/423: rmdir d0/dd/d1e/d3b/d3f/d5a 39 2024-07-03T15:44:07.324 INFO:tasks.workunit.client.0.smithi028.stdout:5/424: creat d0/dd/d13/d47/d50/f8c x:0 0 0 2024-07-03T15:44:07.324 INFO:tasks.workunit.client.0.smithi028.stdout:5/425: dread - d0/dd/d1e/d3b/d3f/d5a/f78 zero size 2024-07-03T15:44:07.332 INFO:tasks.workunit.client.0.smithi028.stdout:5/426: dread d0/dd/d13/d29/f35 [0,4194304] 0 2024-07-03T15:44:07.339 INFO:tasks.workunit.client.0.smithi028.stdout:5/427: dread d0/dd/d13/d57/d26/f42 [4194304,4194304] 0 2024-07-03T15:44:07.340 INFO:tasks.workunit.client.0.smithi028.stdout:5/428: symlink d0/dd/d1e/d3b/d3f/d5a/l8d 0 2024-07-03T15:44:07.340 INFO:tasks.workunit.client.0.smithi028.stdout:5/429: chown d0/dd/d13/d57/f21 29615 1 2024-07-03T15:44:07.341 INFO:tasks.workunit.client.0.smithi028.stdout:5/430: mkdir d0/dd/d1e/d8e 0 2024-07-03T15:44:07.361 INFO:tasks.workunit.client.0.smithi028.stdout:5/431: dwrite d0/dd/d4b/f6c [0,4194304] 0 2024-07-03T15:44:07.361 INFO:tasks.workunit.client.0.smithi028.stdout:5/432: chown d0/dd/d1e/d3b/d3f/d5a/f65 54096642 1 2024-07-03T15:44:07.361 INFO:tasks.workunit.client.0.smithi028.stdout:5/433: chown d0/dd/d13/d29 154806342 1 2024-07-03T15:44:07.362 INFO:tasks.workunit.client.0.smithi028.stdout:8/425: sync 2024-07-03T15:44:07.362 INFO:tasks.workunit.client.0.smithi028.stdout:2/416: sync 2024-07-03T15:44:07.362 INFO:tasks.workunit.client.0.smithi028.stdout:2/417: stat da/f15 0 2024-07-03T15:44:07.367 INFO:tasks.workunit.client.0.smithi028.stdout:8/426: write d0/d41/d50/d3/dc/f1f [3959754,65727] 0 2024-07-03T15:44:07.369 INFO:tasks.workunit.client.0.smithi028.stdout:5/434: dread d0/dd/df/f3e [0,4194304] 0 2024-07-03T15:44:07.369 INFO:tasks.workunit.client.0.smithi028.stdout:5/435: write d0/dd/d1e/f36 [3418237,97020] 0 2024-07-03T15:44:07.369 INFO:tasks.workunit.client.0.smithi028.stdout:5/436: truncate d0/dd/df/f52 1490898 0 2024-07-03T15:44:07.372 INFO:tasks.workunit.client.0.smithi028.stdout:8/427: link d0/d41/d50/d3/dc/d3f/d8e/d69/c83 d0/d41/d50/d3/dc/d3f/d8e/d8d/c90 0 2024-07-03T15:44:07.372 INFO:tasks.workunit.client.0.smithi028.stdout:2/418: write da/d1e/f45 [1373989,101453] 0 2024-07-03T15:44:07.372 INFO:tasks.workunit.client.0.smithi028.stdout:2/419: write da/d24/f66 [89541,35417] 0 2024-07-03T15:44:07.372 INFO:tasks.workunit.client.0.smithi028.stdout:2/420: creat da/d1e/d56/f9a x:0 0 0 2024-07-03T15:44:07.378 INFO:tasks.workunit.client.0.smithi028.stdout:5/437: unlink d0/dd/d1e/d3b/d3f/d5a/l8d 0 2024-07-03T15:44:07.381 INFO:tasks.workunit.client.0.smithi028.stdout:2/421: symlink da/l9b 0 2024-07-03T15:44:07.381 INFO:tasks.workunit.client.0.smithi028.stdout:2/422: creat da/d23/d30/f9c x:0 0 0 2024-07-03T15:44:07.381 INFO:tasks.workunit.client.0.smithi028.stdout:2/423: write da/d24/d2c/f97 [515859,39836] 0 2024-07-03T15:44:07.381 INFO:tasks.workunit.client.0.smithi028.stdout:2/424: fsync da/ff 0 2024-07-03T15:44:07.384 INFO:tasks.workunit.client.0.smithi028.stdout:5/438: mkdir d0/d8f 0 2024-07-03T15:44:07.387 INFO:tasks.workunit.client.0.smithi028.stdout:2/425: creat da/d42/d47/d78/f9d x:0 0 0 2024-07-03T15:44:07.390 INFO:tasks.workunit.client.0.smithi028.stdout:8/428: dwrite d0/d41/d50/f27 [0,4194304] 0 2024-07-03T15:44:07.390 INFO:tasks.workunit.client.0.smithi028.stdout:8/429: chown d0/d41/d50/d3/dc/f23 842 1 2024-07-03T15:44:07.391 INFO:tasks.workunit.client.0.smithi028.stdout:5/439: rename d0/dd/d13/d57/d26/f42 to d0/dd/d13/d57/d30/f90 0 2024-07-03T15:44:07.392 INFO:tasks.workunit.client.0.smithi028.stdout:2/426: mkdir da/d42/d9e 0 2024-07-03T15:44:07.402 INFO:tasks.workunit.client.0.smithi028.stdout:8/430: mknod d0/d41/d50/d3/d71/c91 0 2024-07-03T15:44:07.402 INFO:tasks.workunit.client.0.smithi028.stdout:8/431: chown d0/f7b 1 1 2024-07-03T15:44:07.404 INFO:tasks.workunit.client.0.smithi028.stdout:5/440: rename d0/dd/d13/c6e to d0/dd/d1e/c91 0 2024-07-03T15:44:07.409 INFO:tasks.workunit.client.0.smithi028.stdout:8/432: mkdir d0/d41/d92 0 2024-07-03T15:44:07.417 INFO:tasks.workunit.client.0.smithi028.stdout:6/379: sync 2024-07-03T15:44:07.418 INFO:tasks.workunit.client.0.smithi028.stdout:0/459: sync 2024-07-03T15:44:07.418 INFO:tasks.workunit.client.0.smithi028.stdout:7/307: sync 2024-07-03T15:44:07.418 INFO:tasks.workunit.client.0.smithi028.stdout:0/460: fsync d2/d1c/d32/f38 0 2024-07-03T15:44:07.418 INFO:tasks.workunit.client.0.smithi028.stdout:7/308: rename d9/f3b to d9/d29/f5e 0 2024-07-03T15:44:07.420 INFO:tasks.workunit.client.0.smithi028.stdout:0/461: symlink d2/d3/d4d/d56/l8f 0 2024-07-03T15:44:07.422 INFO:tasks.workunit.client.0.smithi028.stdout:7/309: dread d9/d1c/f20 [0,4194304] 0 2024-07-03T15:44:07.422 INFO:tasks.workunit.client.0.smithi028.stdout:7/310: read d9/f34 [2864308,121427] 0 2024-07-03T15:44:07.422 INFO:tasks.workunit.client.0.smithi028.stdout:7/311: fsync d9/d29/f4f 0 2024-07-03T15:44:07.423 INFO:tasks.workunit.client.0.smithi028.stdout:6/380: rename d4/da/d38/f5e to d4/da/f7c 0 2024-07-03T15:44:07.423 INFO:tasks.workunit.client.0.smithi028.stdout:6/381: fsync d4/da/d56/f73 0 2024-07-03T15:44:07.423 INFO:tasks.workunit.client.0.smithi028.stdout:6/382: creat d4/da/d13/f7d x:0 0 0 2024-07-03T15:44:07.424 INFO:tasks.workunit.client.0.smithi028.stdout:0/462: truncate d2/d3/f73 1835860 0 2024-07-03T15:44:07.424 INFO:tasks.workunit.client.0.smithi028.stdout:0/463: chown d2/d1c/d32/d58/c59 5138 1 2024-07-03T15:44:07.428 INFO:tasks.workunit.client.0.smithi028.stdout:7/312: unlink d9/c30 0 2024-07-03T15:44:07.432 INFO:tasks.workunit.client.0.smithi028.stdout:0/464: mknod d2/d3/d4d/d56/d78/c90 0 2024-07-03T15:44:07.433 INFO:tasks.workunit.client.0.smithi028.stdout:0/465: readlink d2/d1c/d32/d36/l50 0 2024-07-03T15:44:07.433 INFO:tasks.workunit.client.0.smithi028.stdout:0/466: chown d2/d3/l29 325626 1 2024-07-03T15:44:07.433 INFO:tasks.workunit.client.0.smithi028.stdout:5/441: dwrite d0/dd/df/f3e [0,4194304] 0 2024-07-03T15:44:07.435 INFO:tasks.workunit.client.0.smithi028.stdout:8/433: dwrite d0/f76 [0,4194304] 0 2024-07-03T15:44:07.436 INFO:tasks.workunit.client.0.smithi028.stdout:6/383: symlink d4/da/d1c/l7e 0 2024-07-03T15:44:07.436 INFO:tasks.workunit.client.0.smithi028.stdout:6/384: creat d4/d15/f7f x:0 0 0 2024-07-03T15:44:07.436 INFO:tasks.workunit.client.0.smithi028.stdout:6/385: dread - d4/f7b zero size 2024-07-03T15:44:07.438 INFO:tasks.workunit.client.0.smithi028.stdout:0/467: creat d2/d3/d4d/d56/d78/f91 x:0 0 0 2024-07-03T15:44:07.438 INFO:tasks.workunit.client.0.smithi028.stdout:0/468: write d2/d1c/f7e [561461,24662] 0 2024-07-03T15:44:07.440 INFO:tasks.workunit.client.0.smithi028.stdout:5/442: mknod d0/dd/d4b/c92 0 2024-07-03T15:44:07.448 INFO:tasks.workunit.client.0.smithi028.stdout:7/313: dwrite f6 [0,4194304] 0 2024-07-03T15:44:07.448 INFO:tasks.workunit.client.0.smithi028.stdout:7/314: chown d9/l11 0 1 2024-07-03T15:44:07.448 INFO:tasks.workunit.client.0.smithi028.stdout:7/315: truncate d9/d1c/d45/f57 285165 0 2024-07-03T15:44:07.449 INFO:tasks.workunit.client.0.smithi028.stdout:8/434: mknod d0/d41/d50/d3/dc/d33/d7a/c93 0 2024-07-03T15:44:07.449 INFO:tasks.workunit.client.0.smithi028.stdout:5/443: truncate d0/dd/d13/d57/f1f 4043010 0 2024-07-03T15:44:07.453 INFO:tasks.workunit.client.0.smithi028.stdout:6/386: mknod d4/da/d1c/c80 0 2024-07-03T15:44:07.454 INFO:tasks.workunit.client.0.smithi028.stdout:6/387: chown d4/d12/d2c/d31 1545576 1 2024-07-03T15:44:07.455 INFO:tasks.workunit.client.0.smithi028.stdout:7/316: rmdir d9/d1c/d45 39 2024-07-03T15:44:07.458 INFO:tasks.workunit.client.0.smithi028.stdout:0/469: dwrite d2/d1c/d32/d58/f5f [0,4194304] 0 2024-07-03T15:44:07.458 INFO:tasks.workunit.client.0.smithi028.stdout:6/388: stat d4/da/d38/l75 0 2024-07-03T15:44:07.459 INFO:tasks.workunit.client.0.smithi028.stdout:7/317: mknod d9/d1c/d45/c5f 0 2024-07-03T15:44:07.461 INFO:tasks.workunit.client.0.smithi028.stdout:7/318: dread d9/d1c/d45/f46 [0,4194304] 0 2024-07-03T15:44:07.461 INFO:tasks.workunit.client.0.smithi028.stdout:7/319: rename d9 to d9/d29/d5d/d60 22 2024-07-03T15:44:07.462 INFO:tasks.workunit.client.0.smithi028.stdout:0/470: truncate d2/d1c/d6c/f85 3429232 0 2024-07-03T15:44:07.463 INFO:tasks.workunit.client.0.smithi028.stdout:8/435: dread d0/d41/d50/d3/d35/d86/f87 [0,4194304] 0 2024-07-03T15:44:07.463 INFO:tasks.workunit.client.0.smithi028.stdout:8/436: chown d0/d41/d50/d3/d35/d68/d22/d40 48109 1 2024-07-03T15:44:07.463 INFO:tasks.workunit.client.0.smithi028.stdout:8/437: fdatasync d0/d41/d50/d3/dc/d3f/d8e/d5d/f72 0 2024-07-03T15:44:07.465 INFO:tasks.workunit.client.0.smithi028.stdout:6/389: link d4/da/d1c/l7e d4/da/d38/l81 0 2024-07-03T15:44:07.466 INFO:tasks.workunit.client.0.smithi028.stdout:6/390: write d4/d15/f5b [726977,1605] 0 2024-07-03T15:44:07.466 INFO:tasks.workunit.client.0.smithi028.stdout:6/391: readlink d4/d15/l25 0 2024-07-03T15:44:07.467 INFO:tasks.workunit.client.0.smithi028.stdout:7/320: truncate f0 9379962 0 2024-07-03T15:44:07.470 INFO:tasks.workunit.client.0.smithi028.stdout:5/444: dwrite d0/dd/d13/d47/f5b [0,4194304] 0 2024-07-03T15:44:07.472 INFO:tasks.workunit.client.0.smithi028.stdout:8/438: symlink d0/d41/d50/d3/dc/d33/l94 0 2024-07-03T15:44:07.483 INFO:tasks.workunit.client.0.smithi028.stdout:8/439: mknod d0/d41/d50/d3/dc/d3f/d8e/c95 0 2024-07-03T15:44:07.483 INFO:tasks.workunit.client.0.smithi028.stdout:8/440: dread - d0/d41/d50/d3/dc/f7e zero size 2024-07-03T15:44:07.483 INFO:tasks.workunit.client.0.smithi028.stdout:8/441: dread - d0/d41/d50/d3/dc/f63 zero size 2024-07-03T15:44:07.485 INFO:tasks.workunit.client.0.smithi028.stdout:0/471: rename d2/d1c/d32 to d2/d6d/d92 0 2024-07-03T15:44:07.487 INFO:tasks.workunit.client.0.smithi028.stdout:7/321: rename d9/d1c/f40 to d9/d29/f61 0 2024-07-03T15:44:07.487 INFO:tasks.workunit.client.0.smithi028.stdout:7/322: chown d9/d29/f3e 7 1 2024-07-03T15:44:07.488 INFO:tasks.workunit.client.0.smithi028.stdout:7/323: chown d9/d1c/f21 3 1 2024-07-03T15:44:07.493 INFO:tasks.workunit.client.0.smithi028.stdout:6/392: dwrite d4/da/f4e [0,4194304] 0 2024-07-03T15:44:07.496 INFO:tasks.workunit.client.0.smithi028.stdout:6/393: rename d4/d15/c2f to d4/c82 0 2024-07-03T15:44:07.502 INFO:tasks.workunit.client.0.smithi028.stdout:8/442: dread d0/d41/d50/d3/dc/d33/d7a/f2f [8388608,4194304] 0 2024-07-03T15:44:07.504 INFO:tasks.workunit.client.0.smithi028.stdout:8/443: unlink d0/f7b 0 2024-07-03T15:44:07.504 INFO:tasks.workunit.client.0.smithi028.stdout:8/444: write d0/d41/d50/d3/d35/d68/d22/d40/d62/f65 [920528,13096] 0 2024-07-03T15:44:07.504 INFO:tasks.workunit.client.0.smithi028.stdout:6/394: dread d4/f3a [0,4194304] 0 2024-07-03T15:44:07.506 INFO:tasks.workunit.client.0.smithi028.stdout:6/395: symlink d4/da/d56/l83 0 2024-07-03T15:44:07.506 INFO:tasks.workunit.client.0.smithi028.stdout:8/445: dread d0/d41/d50/d3/dc/d3f/d8e/f42 [0,4194304] 0 2024-07-03T15:44:07.507 INFO:tasks.workunit.client.0.smithi028.stdout:7/324: dwrite d9/d1c/f4b [0,4194304] 0 2024-07-03T15:44:07.507 INFO:tasks.workunit.client.0.smithi028.stdout:6/396: mknod d4/d12/d2c/d31/c84 0 2024-07-03T15:44:07.508 INFO:tasks.workunit.client.0.smithi028.stdout:7/325: dread d9/d1c/d45/f57 [0,4194304] 0 2024-07-03T15:44:07.509 INFO:tasks.workunit.client.0.smithi028.stdout:0/472: dwrite d2/d1c/f2e [0,4194304] 0 2024-07-03T15:44:07.510 INFO:tasks.workunit.client.0.smithi028.stdout:0/473: chown d2/d1c/l24 721582995 1 2024-07-03T15:44:07.510 INFO:tasks.workunit.client.0.smithi028.stdout:0/474: fsync d2/d6d/d92/d36/f87 0 2024-07-03T15:44:07.513 INFO:tasks.workunit.client.0.smithi028.stdout:0/475: dread d2/d3/d4d/f57 [0,4194304] 0 2024-07-03T15:44:07.515 INFO:tasks.workunit.client.0.smithi028.stdout:6/397: mknod d4/da/c85 0 2024-07-03T15:44:07.525 INFO:tasks.workunit.client.0.smithi028.stdout:8/446: dwrite d0/d41/d50/d3/dc/d33/d7a/f15 [0,4194304] 0 2024-07-03T15:44:07.538 INFO:tasks.workunit.client.0.smithi028.stdout:8/447: rename d0/d41/d50/d3/dc/d33/l94 to d0/d41/d50/d3/d35/d68/d22/l96 0 2024-07-03T15:44:07.538 INFO:tasks.workunit.client.0.smithi028.stdout:8/448: write d0/d41/d50/d3/dc/d3f/d8e/d4c/f7c [834260,105320] 0 2024-07-03T15:44:07.553 INFO:tasks.workunit.client.0.smithi028.stdout:8/449: write d0/d41/d50/f27 [124673,127626] 0 2024-07-03T15:44:07.559 INFO:tasks.workunit.client.0.smithi028.stdout:8/450: dread d0/d41/d50/d3/dc/d3f/d8e/d4c/f7c [0,4194304] 0 2024-07-03T15:44:07.559 INFO:tasks.workunit.client.0.smithi028.stdout:8/451: dread - d0/d41/d50/d3/dc/d3f/f61 zero size 2024-07-03T15:44:07.559 INFO:tasks.workunit.client.0.smithi028.stdout:8/452: write d0/d41/d50/d3/dc/d33/d7f/f8a [934602,105303] 0 2024-07-03T15:44:07.559 INFO:tasks.workunit.client.0.smithi028.stdout:8/453: stat d0/d41/d50/d3/d35/d86/f87 0 2024-07-03T15:44:07.559 INFO:tasks.workunit.client.0.smithi028.stdout:8/454: write d0/f2a [1798529,68623] 0 2024-07-03T15:44:07.560 INFO:tasks.workunit.client.0.smithi028.stdout:8/455: dread d0/f5b [0,4194304] 0 2024-07-03T15:44:07.569 INFO:tasks.workunit.client.0.smithi028.stdout:8/456: write d0/d41/d50/d3/dc/f2b [3506075,64999] 0 2024-07-03T15:44:07.571 INFO:tasks.workunit.client.0.smithi028.stdout:8/457: mkdir d0/d41/d50/d97 0 2024-07-03T15:44:07.573 INFO:tasks.workunit.client.0.smithi028.stdout:6/398: dwrite f0 [0,4194304] 0 2024-07-03T15:44:07.576 INFO:tasks.workunit.client.0.smithi028.stdout:8/458: rename d0/d41/d50/d3/dc/d33/d7f to d0/d41/d50/d3/d35/d98 0 2024-07-03T15:44:07.577 INFO:tasks.workunit.client.0.smithi028.stdout:0/476: dwrite d2/d3/fe [0,4194304] 0 2024-07-03T15:44:07.577 INFO:tasks.workunit.client.0.smithi028.stdout:0/477: chown d2/d6d/d92/d36/f7a 15542 1 2024-07-03T15:44:07.577 INFO:tasks.workunit.client.0.smithi028.stdout:0/478: chown d2/d1c/f68 3 1 2024-07-03T15:44:07.578 INFO:tasks.workunit.client.0.smithi028.stdout:8/459: mknod d0/d41/d50/d3/d35/d68/d22/d40/d4a/c99 0 2024-07-03T15:44:07.578 INFO:tasks.workunit.client.0.smithi028.stdout:8/460: write d0/d41/d50/d3/d35/d86/f87 [2045384,20276] 0 2024-07-03T15:44:07.581 INFO:tasks.workunit.client.0.smithi028.stdout:8/461: link d0/d41/d50/d3/dc/d33/f7d d0/d41/d50/d3/d35/d68/d22/d40/d4a/f9a 0 2024-07-03T15:44:07.581 INFO:tasks.workunit.client.0.smithi028.stdout:8/462: fdatasync d0/d41/d50/d3/dc/d33/d7a/f2f 0 2024-07-03T15:44:07.581 INFO:tasks.workunit.client.0.smithi028.stdout:8/463: stat d0/d41/d50/d3/dc/d3f/d8e/d4c 0 2024-07-03T15:44:07.581 INFO:tasks.workunit.client.0.smithi028.stdout:8/464: chown d0/f76 22 1 2024-07-03T15:44:07.581 INFO:tasks.workunit.client.0.smithi028.stdout:8/465: chown d0/f2a 1 1 2024-07-03T15:44:07.584 INFO:tasks.workunit.client.0.smithi028.stdout:0/479: write d2/d6d/d92/d36/f65 [487107,103261] 0 2024-07-03T15:44:07.589 INFO:tasks.workunit.client.0.smithi028.stdout:8/466: read d0/d41/d50/d3/dc/d33/d7a/f2f [698594,13648] 0 2024-07-03T15:44:07.591 INFO:tasks.workunit.client.0.smithi028.stdout:8/467: mknod d0/d41/d50/d3/d35/d68/d22/d40/d4a/c9b 0 2024-07-03T15:44:07.591 INFO:tasks.workunit.client.0.smithi028.stdout:8/468: chown d0/d41/d50/d3/d35/c74 487623 1 2024-07-03T15:44:07.591 INFO:tasks.workunit.client.0.smithi028.stdout:8/469: dread - d0/d41/d50/d3/dc/d33/d7a/f8b zero size 2024-07-03T15:44:07.592 INFO:tasks.workunit.client.0.smithi028.stdout:6/399: dwrite d4/da/d47/f51 [0,4194304] 0 2024-07-03T15:44:07.592 INFO:tasks.workunit.client.0.smithi028.stdout:6/400: chown d4/c3b 1772272395 1 2024-07-03T15:44:07.592 INFO:tasks.workunit.client.0.smithi028.stdout:8/470: link d0/d41/d50/d3/dc/f63 d0/d41/d50/d3/d35/d98/f9c 0 2024-07-03T15:44:07.592 INFO:tasks.workunit.client.0.smithi028.stdout:8/471: write d0/f5b [837158,126859] 0 2024-07-03T15:44:07.595 INFO:tasks.workunit.client.0.smithi028.stdout:8/472: mknod d0/d41/d50/d3/dc/d3f/d8e/d81/c9d 0 2024-07-03T15:44:07.595 INFO:tasks.workunit.client.0.smithi028.stdout:8/473: write d0/d41/d50/f78 [12877,65614] 0 2024-07-03T15:44:07.596 INFO:tasks.workunit.client.0.smithi028.stdout:6/401: link d4/d15/c53 d4/c86 0 2024-07-03T15:44:07.597 INFO:tasks.workunit.client.0.smithi028.stdout:8/474: mkdir d0/d41/d50/d3/dc/d3f/d8e/d81/d9e 0 2024-07-03T15:44:07.597 INFO:tasks.workunit.client.0.smithi028.stdout:8/475: dread - d0/d41/d50/d3/dc/f7e zero size 2024-07-03T15:44:07.597 INFO:tasks.workunit.client.0.smithi028.stdout:8/476: stat d0/d41/d50/d3/dc/c5a 0 2024-07-03T15:44:07.597 INFO:tasks.workunit.client.0.smithi028.stdout:8/477: write d0/f44 [1202220,45571] 0 2024-07-03T15:44:07.600 INFO:tasks.workunit.client.0.smithi028.stdout:8/478: getdents d0/d41 0 2024-07-03T15:44:07.601 INFO:tasks.workunit.client.0.smithi028.stdout:0/480: dwrite d2/d6d/d92/d58/f6a [0,4194304] 0 2024-07-03T15:44:07.603 INFO:tasks.workunit.client.0.smithi028.stdout:8/479: mkdir d0/d41/d50/d3/dc/d3f/d8e/d5d/d9f 0 2024-07-03T15:44:07.608 INFO:tasks.workunit.client.0.smithi028.stdout:8/480: creat d0/d41/d50/d3/dc/d3f/d8e/d4c/fa0 x:0 0 0 2024-07-03T15:44:07.608 INFO:tasks.workunit.client.0.smithi028.stdout:8/481: getdents d0/d41 0 2024-07-03T15:44:07.622 INFO:tasks.workunit.client.0.smithi028.stdout:6/402: dwrite d4/da/d47/f51 [0,4194304] 0 2024-07-03T15:44:07.623 INFO:tasks.workunit.client.0.smithi028.stdout:8/482: dwrite d0/d41/d50/d3/dc/d33/d7a/f15 [0,4194304] 0 2024-07-03T15:44:07.623 INFO:tasks.workunit.client.0.smithi028.stdout:8/483: write d0/d41/d50/d3/dc/d3f/d8e/f6a [571908,7815] 0 2024-07-03T15:44:07.625 INFO:tasks.workunit.client.0.smithi028.stdout:6/403: getdents d4/da/d47/d66 0 2024-07-03T15:44:07.626 INFO:tasks.workunit.client.0.smithi028.stdout:6/404: chown d4/d12/f4f 145 1 2024-07-03T15:44:07.626 INFO:tasks.workunit.client.0.smithi028.stdout:6/405: fsync d4/f10 0 2024-07-03T15:44:07.631 INFO:tasks.workunit.client.0.smithi028.stdout:8/484: rmdir d0/d41/d50/d3/dc/d33/d7a 39 2024-07-03T15:44:07.638 INFO:tasks.workunit.client.0.smithi028.stdout:8/485: read d0/d41/d50/d3/dc/f23 [483057,69901] 0 2024-07-03T15:44:07.638 INFO:tasks.workunit.client.0.smithi028.stdout:1/557: sync 2024-07-03T15:44:07.638 INFO:tasks.workunit.client.0.smithi028.stdout:1/558: dread - d4/de/fc3 zero size 2024-07-03T15:44:07.638 INFO:tasks.workunit.client.0.smithi028.stdout:3/453: sync 2024-07-03T15:44:07.638 INFO:tasks.workunit.client.0.smithi028.stdout:4/444: sync 2024-07-03T15:44:07.638 INFO:tasks.workunit.client.0.smithi028.stdout:9/480: sync 2024-07-03T15:44:07.642 INFO:tasks.workunit.client.0.smithi028.stdout:8/486: mknod d0/d41/d50/d97/ca1 0 2024-07-03T15:44:07.644 INFO:tasks.workunit.client.0.smithi028.stdout:1/559: creat d4/d3c/d54/d79/d16/d7c/fcc x:0 0 0 2024-07-03T15:44:07.647 INFO:tasks.workunit.client.0.smithi028.stdout:6/406: dwrite d4/f3a [0,4194304] 0 2024-07-03T15:44:07.647 INFO:tasks.workunit.client.0.smithi028.stdout:4/445: creat d2/d12/d42/d63/d6c/d74/f93 x:0 0 0 2024-07-03T15:44:07.657 INFO:tasks.workunit.client.0.smithi028.stdout:6/407: mknod d4/da/c87 0 2024-07-03T15:44:07.657 INFO:tasks.workunit.client.0.smithi028.stdout:6/408: fdatasync d4/d15/f5b 0 2024-07-03T15:44:07.658 INFO:tasks.workunit.client.0.smithi028.stdout:4/446: creat d2/d5a/f94 x:0 0 0 2024-07-03T15:44:07.658 INFO:tasks.workunit.client.0.smithi028.stdout:4/447: symlink d2/l95 0 2024-07-03T15:44:07.659 INFO:tasks.workunit.client.0.smithi028.stdout:4/448: stat d2/d5a/d45/l72 0 2024-07-03T15:44:07.659 INFO:tasks.workunit.client.0.smithi028.stdout:4/449: mknod d2/d12/d42/d63/d75/c96 0 2024-07-03T15:44:07.661 INFO:tasks.workunit.client.0.smithi028.stdout:4/450: unlink d2/l2a 0 2024-07-03T15:44:07.663 INFO:tasks.workunit.client.0.smithi028.stdout:4/451: truncate d2/d3e/f50 356362 0 2024-07-03T15:44:07.663 INFO:tasks.workunit.client.0.smithi028.stdout:4/452: readlink d2/d12/d2e/l49 0 2024-07-03T15:44:07.663 INFO:tasks.workunit.client.0.smithi028.stdout:4/453: write d2/d12/d2e/f6b [3842432,6235] 0 2024-07-03T15:44:07.663 INFO:tasks.workunit.client.0.smithi028.stdout:4/454: dread - d2/d12/d2c/f7d zero size 2024-07-03T15:44:07.664 INFO:tasks.workunit.client.0.smithi028.stdout:9/481: dwrite d1/d16/d57/f9c [0,4194304] 0 2024-07-03T15:44:07.664 INFO:tasks.workunit.client.0.smithi028.stdout:9/482: fsync d1/f31 0 2024-07-03T15:44:07.664 INFO:tasks.workunit.client.0.smithi028.stdout:4/455: mkdir d2/d12/d42/d63/d97 0 2024-07-03T15:44:07.666 INFO:tasks.workunit.client.0.smithi028.stdout:9/483: getdents d1/d10/d95 0 2024-07-03T15:44:07.667 INFO:tasks.workunit.client.0.smithi028.stdout:4/456: mkdir d2/d12/d5b/d98 0 2024-07-03T15:44:07.668 INFO:tasks.workunit.client.0.smithi028.stdout:9/484: symlink d1/dd/la6 0 2024-07-03T15:44:07.668 INFO:tasks.workunit.client.0.smithi028.stdout:9/485: write d1/f88 [870620,68684] 0 2024-07-03T15:44:07.671 INFO:tasks.workunit.client.0.smithi028.stdout:4/457: dread d2/d12/d3b/f43 [0,4194304] 0 2024-07-03T15:44:07.671 INFO:tasks.workunit.client.0.smithi028.stdout:1/560: dwrite d4/d3c/f9e [0,4194304] 0 2024-07-03T15:44:07.671 INFO:tasks.workunit.client.0.smithi028.stdout:1/561: truncate d4/d83/f8b 988196 0 2024-07-03T15:44:07.672 INFO:tasks.workunit.client.0.smithi028.stdout:8/487: dwrite d0/d41/d50/d3/dc/d3f/d8e/f6a [0,4194304] 0 2024-07-03T15:44:07.673 INFO:tasks.workunit.client.0.smithi028.stdout:8/488: dread d0/d41/d50/d3/dc/f63 [0,4194304] 0 2024-07-03T15:44:07.680 INFO:tasks.workunit.client.0.smithi028.stdout:4/458: mknod d2/d5a/c99 0 2024-07-03T15:44:07.680 INFO:tasks.workunit.client.0.smithi028.stdout:4/459: dread - d2/d5a/d45/f4c zero size 2024-07-03T15:44:07.682 INFO:tasks.workunit.client.0.smithi028.stdout:8/489: link d0/d41/d50/d3/dc/d3f/d8e/d8d/c90 d0/d41/d50/d3/dc/d3f/d8e/d8d/ca2 0 2024-07-03T15:44:07.685 INFO:tasks.workunit.client.0.smithi028.stdout:4/460: link d2/f2b d2/d12/d42/d63/d97/f9a 0 2024-07-03T15:44:07.685 INFO:tasks.workunit.client.0.smithi028.stdout:4/461: creat d2/d12/d5b/f9b x:0 0 0 2024-07-03T15:44:07.687 INFO:tasks.workunit.client.0.smithi028.stdout:1/562: write d4/d2c/d3a/d5c/d6c/d7d/d4e/fb9 [1159607,93651] 0 2024-07-03T15:44:07.687 INFO:tasks.workunit.client.0.smithi028.stdout:8/490: getdents d0/d41/d50/d3/d35/d86 0 2024-07-03T15:44:07.691 INFO:tasks.workunit.client.0.smithi028.stdout:4/462: mkdir d2/d61/d9c 0 2024-07-03T15:44:07.692 INFO:tasks.workunit.client.0.smithi028.stdout:9/486: dwrite d1/d16/f2f [0,4194304] 0 2024-07-03T15:44:07.692 INFO:tasks.workunit.client.0.smithi028.stdout:9/487: write d1/dd/f68 [173365,115593] 0 2024-07-03T15:44:07.694 INFO:tasks.workunit.client.0.smithi028.stdout:8/491: symlink d0/d41/d50/d3/d35/d68/d22/d40/d4a/la3 0 2024-07-03T15:44:07.697 INFO:tasks.workunit.client.0.smithi028.stdout:8/492: rename d0/f76 to d0/d41/d50/d97/fa4 0 2024-07-03T15:44:07.698 INFO:tasks.workunit.client.0.smithi028.stdout:9/488: symlink d1/d10/d28/d45/d56/la7 0 2024-07-03T15:44:07.699 INFO:tasks.workunit.client.0.smithi028.stdout:8/493: symlink d0/d41/d50/d3/d35/d68/d22/la5 0 2024-07-03T15:44:07.706 INFO:tasks.workunit.client.0.smithi028.stdout:4/463: read d2/d12/d2d/d2f/d3a/d6a/f71 [821064,75646] 0 2024-07-03T15:44:07.707 INFO:tasks.workunit.client.0.smithi028.stdout:4/464: getdents d2/d61/d9c 0 2024-07-03T15:44:07.708 INFO:tasks.workunit.client.0.smithi028.stdout:4/465: mkdir d2/d61/d9d 0 2024-07-03T15:44:07.708 INFO:tasks.workunit.client.0.smithi028.stdout:4/466: write d2/d12/d3b/f43 [1144638,100360] 0 2024-07-03T15:44:07.708 INFO:tasks.workunit.client.0.smithi028.stdout:4/467: dread - d2/d12/d42/d63/f65 zero size 2024-07-03T15:44:07.711 INFO:tasks.workunit.client.0.smithi028.stdout:1/563: dwrite d4/de/f85 [0,4194304] 0 2024-07-03T15:44:07.715 INFO:tasks.workunit.client.0.smithi028.stdout:1/564: link d4/d3c/f9e d4/d2c/d3a/d5c/d6c/d7d/d4e/fcd 0 2024-07-03T15:44:07.716 INFO:tasks.workunit.client.0.smithi028.stdout:1/565: write d4/d3c/d54/d79/d16/fb3 [692399,93870] 0 2024-07-03T15:44:07.725 INFO:tasks.workunit.client.0.smithi028.stdout:9/489: dwrite d1/d10/d28/d4e/f50 [0,4194304] 0 2024-07-03T15:44:07.732 INFO:tasks.workunit.client.0.smithi028.stdout:8/494: dwrite d0/f44 [0,4194304] 0 2024-07-03T15:44:07.732 INFO:tasks.workunit.client.0.smithi028.stdout:8/495: readlink d0/d41/d50/d3/dc/d33/d7a/l14 0 2024-07-03T15:44:07.732 INFO:tasks.workunit.client.0.smithi028.stdout:4/468: dwrite d2/d12/d5b/f81 [0,4194304] 0 2024-07-03T15:44:07.733 INFO:tasks.workunit.client.0.smithi028.stdout:1/566: getdents d4/d2c/d3a/d5c/d6c/d7d/d4e 0 2024-07-03T15:44:07.733 INFO:tasks.workunit.client.0.smithi028.stdout:1/567: chown d4/d2c/d3a/d5c/d6c/d7d/d59/d94/f96 31362859 1 2024-07-03T15:44:07.736 INFO:tasks.workunit.client.0.smithi028.stdout:8/496: rename d0/d41/d50/d3/d35/d98 to d0/d41/d50/d3/dc/d3f/d8e/d5d/d9f/da6 0 2024-07-03T15:44:07.737 INFO:tasks.workunit.client.0.smithi028.stdout:4/469: mkdir d2/d12/d42/d63/d9e 0 2024-07-03T15:44:07.744 INFO:tasks.workunit.client.0.smithi028.stdout:8/497: mkdir d0/d41/d50/d3/dc/d3f/d8e/d5d/d9f/da6/da7 0 2024-07-03T15:44:07.745 INFO:tasks.workunit.client.0.smithi028.stdout:8/498: read d0/f5b [558209,27850] 0 2024-07-03T15:44:07.756 INFO:tasks.workunit.client.0.smithi028.stdout:8/499: link d0/d41/d50/d3/d35/d68/d22/d40/d45/l79 d0/d41/d50/d3/dc/d3f/d8e/d81/la8 0 2024-07-03T15:44:07.756 INFO:tasks.workunit.client.0.smithi028.stdout:8/500: write d0/d41/d50/d3/dc/f63 [333176,42320] 0 2024-07-03T15:44:07.756 INFO:tasks.workunit.client.0.smithi028.stdout:8/501: fdatasync d0/d41/d50/d3/dc/d33/d7a/f8b 0 2024-07-03T15:44:07.756 INFO:tasks.workunit.client.0.smithi028.stdout:9/490: dwrite d1/d17/f25 [0,4194304] 0 2024-07-03T15:44:07.757 INFO:tasks.workunit.client.0.smithi028.stdout:1/568: dwrite d4/de/fc3 [0,4194304] 0 2024-07-03T15:44:07.757 INFO:tasks.workunit.client.0.smithi028.stdout:4/470: dwrite d2/d3e/f8a [0,4194304] 0 2024-07-03T15:44:07.757 INFO:tasks.workunit.client.0.smithi028.stdout:4/471: truncate d2/d12/d5b/f85 925669 0 2024-07-03T15:44:07.757 INFO:tasks.workunit.client.0.smithi028.stdout:8/502: rmdir d0/d41/d50/d3/d35/d68 39 2024-07-03T15:44:07.761 INFO:tasks.workunit.client.0.smithi028.stdout:9/491: creat d1/d10/d28/d45/fa8 x:0 0 0 2024-07-03T15:44:07.767 INFO:tasks.workunit.client.0.smithi028.stdout:4/472: getdents d2/d12/d2d/d2f/d3a 0 2024-07-03T15:44:07.768 INFO:tasks.workunit.client.0.smithi028.stdout:8/503: mkdir d0/d6c/da9 0 2024-07-03T15:44:07.770 INFO:tasks.workunit.client.0.smithi028.stdout:8/504: write d0/d41/d50/d3/dc/d33/f7d [282981,117577] 0 2024-07-03T15:44:07.770 INFO:tasks.workunit.client.0.smithi028.stdout:8/505: fsync d0/f2a 0 2024-07-03T15:44:07.771 INFO:tasks.workunit.client.0.smithi028.stdout:4/473: dread d2/d5a/f24 [0,4194304] 0 2024-07-03T15:44:07.772 INFO:tasks.workunit.client.0.smithi028.stdout:1/569: dread d4/d2c/f92 [0,4194304] 0 2024-07-03T15:44:07.773 INFO:tasks.workunit.client.0.smithi028.stdout:8/506: dread d0/d41/d50/d3/dc/d3f/d8e/f42 [0,4194304] 0 2024-07-03T15:44:07.776 INFO:tasks.workunit.client.0.smithi028.stdout:4/474: dread d2/d3e/f3f [0,4194304] 0 2024-07-03T15:44:07.776 INFO:tasks.workunit.client.0.smithi028.stdout:9/492: dwrite d1/dd/f90 [0,4194304] 0 2024-07-03T15:44:07.777 INFO:tasks.workunit.client.0.smithi028.stdout:8/507: rename d0/d41/d50/d3/d35/d68/d22/d40/d4a/f9a to d0/d41/d50/d3/dc/d3f/faa 0 2024-07-03T15:44:07.777 INFO:tasks.workunit.client.0.smithi028.stdout:8/508: chown d0/d41/d50/d3/dc/d33/d7a/c3e 606120491 1 2024-07-03T15:44:07.782 INFO:tasks.workunit.client.0.smithi028.stdout:8/509: dread d0/d41/d50/d3/dc/f23 [0,4194304] 0 2024-07-03T15:44:07.783 INFO:tasks.workunit.client.0.smithi028.stdout:4/475: rename d2/d12/d42/d63/d6c to d2/d3e/d9f 0 2024-07-03T15:44:07.784 INFO:tasks.workunit.client.0.smithi028.stdout:4/476: chown d2/d12/d2d/d58/f70 93 1 2024-07-03T15:44:07.785 INFO:tasks.workunit.client.0.smithi028.stdout:4/477: mknod d2/d12/d42/d63/d75/ca0 0 2024-07-03T15:44:07.785 INFO:tasks.workunit.client.0.smithi028.stdout:4/478: write d2/d5a/f94 [44110,124970] 0 2024-07-03T15:44:07.788 INFO:tasks.workunit.client.0.smithi028.stdout:4/479: dread d2/d12/f60 [0,4194304] 0 2024-07-03T15:44:07.792 INFO:tasks.workunit.client.0.smithi028.stdout:4/480: symlink d2/d12/la1 0 2024-07-03T15:44:07.794 INFO:tasks.workunit.client.0.smithi028.stdout:9/493: read d1/d10/d28/d45/d61/d8c/f72 [2385499,111092] 0 2024-07-03T15:44:07.795 INFO:tasks.workunit.client.0.smithi028.stdout:1/570: dwrite d4/d3c/d54/d79/d16/fb4 [0,4194304] 0 2024-07-03T15:44:07.795 INFO:tasks.workunit.client.0.smithi028.stdout:1/571: chown d4/d2c/d3a/d5c/d6c/d7d/d4e 317 1 2024-07-03T15:44:07.795 INFO:tasks.workunit.client.0.smithi028.stdout:1/572: write d4/d3c/d54/d79/f12 [1249033,106536] 0 2024-07-03T15:44:07.805 INFO:tasks.workunit.client.0.smithi028.stdout:9/494: read d1/d10/d28/d45/d61/d8c/f81 [2839760,42094] 0 2024-07-03T15:44:07.806 INFO:tasks.workunit.client.0.smithi028.stdout:4/481: dwrite d2/f86 [0,4194304] 0 2024-07-03T15:44:07.808 INFO:tasks.workunit.client.0.smithi028.stdout:9/495: link d1/c22 d1/d16/d57/d5f/d64/ca9 0 2024-07-03T15:44:07.808 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:07 smithi028 ceph-mon[29148]: pgmap v5: 65 pgs: 65 active+clean; 2.7 GiB data, 9.3 GiB used, 527 GiB / 536 GiB avail 2024-07-03T15:44:07.809 INFO:tasks.workunit.client.0.smithi028.stdout:4/482: dread d2/f52 [0,4194304] 0 2024-07-03T15:44:07.810 INFO:tasks.workunit.client.0.smithi028.stdout:1/573: dread d4/d3c/d54/d79/f12 [0,4194304] 0 2024-07-03T15:44:07.811 INFO:tasks.workunit.client.0.smithi028.stdout:9/496: mkdir d1/d10/d28/daa 0 2024-07-03T15:44:07.822 INFO:tasks.workunit.client.0.smithi028.stdout:1/574: getdents d4/d2c/d3a/d5c/d6c/d7d/d59/d94 0 2024-07-03T15:44:07.830 INFO:tasks.workunit.client.0.smithi028.stdout:4/483: getdents d2/d12/d2e 0 2024-07-03T15:44:07.830 INFO:tasks.workunit.client.0.smithi028.stdout:4/484: chown d2/d61/d9c 0 1 2024-07-03T15:44:07.830 INFO:tasks.workunit.client.0.smithi028.stdout:4/485: write d2/d3e/f50 [800770,95612] 0 2024-07-03T15:44:07.830 INFO:tasks.workunit.client.0.smithi028.stdout:4/486: fdatasync d2/d12/d3b/f43 0 2024-07-03T15:44:07.830 INFO:tasks.workunit.client.0.smithi028.stdout:4/487: dread - d2/d3e/d9f/d74/f93 zero size 2024-07-03T15:44:07.830 INFO:tasks.workunit.client.0.smithi028.stdout:4/488: chown d2/d5a/d45/d84 6740 1 2024-07-03T15:44:07.830 INFO:tasks.workunit.client.0.smithi028.stdout:4/489: truncate d2/d12/d2d/d58/f69 723366 0 2024-07-03T15:44:07.832 INFO:tasks.workunit.client.0.smithi028.stdout:1/575: mknod d4/de/d1c/d44/cce 0 2024-07-03T15:44:07.832 INFO:tasks.workunit.client.0.smithi028.stdout:1/576: read - d4/d3c/d54/d79/d16/db6/fc9 zero size 2024-07-03T15:44:07.835 INFO:tasks.workunit.client.0.smithi028.stdout:1/577: unlink d4/de/d1c/fb7 0 2024-07-03T15:44:07.835 INFO:tasks.workunit.client.0.smithi028.stdout:1/578: chown d4/d3c/d54/d79/l28 5 1 2024-07-03T15:44:07.835 INFO:tasks.workunit.client.0.smithi028.stdout:1/579: fdatasync d4/de/f10 0 2024-07-03T15:44:07.858 INFO:tasks.workunit.client.0.smithi028.stdout:4/490: dwrite d2/d5a/f94 [0,4194304] 0 2024-07-03T15:44:07.859 INFO:tasks.workunit.client.0.smithi028.stdout:4/491: write d2/d12/d5b/f9b [814074,88985] 0 2024-07-03T15:44:07.859 INFO:tasks.workunit.client.0.smithi028.stdout:1/580: dwrite d4/d3c/d54/d79/f12 [0,4194304] 0 2024-07-03T15:44:07.864 INFO:tasks.workunit.client.0.smithi028.stdout:4/492: rename d2/d5a/d45/l72 to d2/d12/d5b/la2 0 2024-07-03T15:44:07.865 INFO:tasks.workunit.client.0.smithi028.stdout:4/493: write d2/d12/d2e/f6b [2764173,20727] 0 2024-07-03T15:44:07.869 INFO:tasks.workunit.client.0.smithi028.stdout:1/581: creat d4/d2c/d3a/d5c/fcf x:0 0 0 2024-07-03T15:44:07.871 INFO:tasks.workunit.client.0.smithi028.stdout:4/494: creat d2/d12/d2d/d58/fa3 x:0 0 0 2024-07-03T15:44:07.872 INFO:tasks.workunit.client.0.smithi028.stdout:1/582: rename d4/d3c/d54/d79/c4d to d4/d2c/d3a/d5c/daa/cd0 0 2024-07-03T15:44:07.878 INFO:tasks.workunit.client.0.smithi028.stdout:1/583: mkdir d4/d3c/d54/d79/d16/db6/dd1 0 2024-07-03T15:44:07.878 INFO:tasks.workunit.client.0.smithi028.stdout:4/495: link d2/d12/d2d/d2f/d3a/d6a/c8d d2/d12/d42/ca4 0 2024-07-03T15:44:07.883 INFO:tasks.workunit.client.0.smithi028.stdout:4/496: mkdir d2/d5a/d45/da5 0 2024-07-03T15:44:07.883 INFO:tasks.workunit.client.0.smithi028.stdout:4/497: chown d2/c3 55925 1 2024-07-03T15:44:07.911 INFO:tasks.workunit.client.0.smithi028.stdout:1/584: dwrite d4/de/da7/fc0 [0,4194304] 0 2024-07-03T15:44:07.911 INFO:tasks.workunit.client.0.smithi028.stdout:1/585: dread - d4/de/fc5 zero size 2024-07-03T15:44:07.914 INFO:tasks.workunit.client.0.smithi028.stdout:1/586: mknod d4/d3c/d54/d79/d16/d7c/cd2 0 2024-07-03T15:44:07.930 INFO:tasks.workunit.client.0.smithi028.stdout:1/587: dread d4/d3c/d54/d79/f12 [0,4194304] 0 2024-07-03T15:44:07.930 INFO:tasks.workunit.client.0.smithi028.stdout:1/588: chown d4/d2c/f78 6 1 2024-07-03T15:44:07.932 INFO:tasks.workunit.client.0.smithi028.stdout:1/589: creat d4/de/d1c/d81/fd3 x:0 0 0 2024-07-03T15:44:07.932 INFO:tasks.workunit.client.0.smithi028.stdout:2/427: sync 2024-07-03T15:44:07.932 INFO:tasks.workunit.client.0.smithi028.stdout:2/428: fdatasync da/d1e/d56/f7e 0 2024-07-03T15:44:07.935 INFO:tasks.workunit.client.0.smithi028.stdout:2/429: symlink da/d1e/d68/l9f 0 2024-07-03T15:44:07.959 INFO:tasks.workunit.client.0.smithi028.stdout:1/590: dwrite d4/d3c/d54/f8d [0,4194304] 0 2024-07-03T15:44:07.964 INFO:tasks.workunit.client.0.smithi028.stdout:2/430: dwrite da/d24/f55 [0,4194304] 0 2024-07-03T15:44:07.964 INFO:tasks.workunit.client.0.smithi028.stdout:2/431: write da/d1e/d56/f7e [769327,85726] 0 2024-07-03T15:44:07.964 INFO:tasks.workunit.client.0.smithi028.stdout:2/432: truncate da/d23/d30/f67 279819 0 2024-07-03T15:44:07.964 INFO:tasks.workunit.client.0.smithi028.stdout:1/591: mkdir d4/d2c/d3a/d5c/dd4 0 2024-07-03T15:44:07.968 INFO:tasks.workunit.client.0.smithi028.stdout:1/592: mkdir d4/d3c/d54/d79/d16/d7c/dd5 0 2024-07-03T15:44:07.968 INFO:tasks.workunit.client.0.smithi028.stdout:1/593: write d4/f37 [4420532,75455] 0 2024-07-03T15:44:07.979 INFO:tasks.workunit.client.0.smithi028.stdout:1/594: dread d4/d3c/f9e [0,4194304] 0 2024-07-03T15:44:07.981 INFO:tasks.workunit.client.0.smithi028.stdout:1/595: truncate d4/d3c/d54/d79/d16/f27 600112 0 2024-07-03T15:44:07.981 INFO:tasks.workunit.client.0.smithi028.stdout:1/596: dread - d4/d2c/d3a/d5c/fcf zero size 2024-07-03T15:44:07.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:07 smithi148 ceph-mon[36056]: pgmap v5: 65 pgs: 65 active+clean; 2.7 GiB data, 9.3 GiB used, 527 GiB / 536 GiB avail 2024-07-03T15:44:07.982 INFO:tasks.workunit.client.0.smithi028.stdout:1/597: mkdir d4/d3c/d54/d79/d16/d57/d66/dd6 0 2024-07-03T15:44:07.982 INFO:tasks.workunit.client.0.smithi028.stdout:1/598: write d4/d3c/d54/d79/f8e [534797,115712] 0 2024-07-03T15:44:07.982 INFO:tasks.workunit.client.0.smithi028.stdout:1/599: fdatasync d4/d3c/d54/d79/d16/d7c/fcc 0 2024-07-03T15:44:07.985 INFO:tasks.workunit.client.0.smithi028.stdout:1/600: symlink d4/dc2/ld7 0 2024-07-03T15:44:07.988 INFO:tasks.workunit.client.0.smithi028.stdout:1/601: creat d4/d3c/d54/fd8 x:0 0 0 2024-07-03T15:44:07.988 INFO:tasks.workunit.client.0.smithi028.stdout:1/602: chown d4/d2c/d3a/d5c/d6c/d7d/d4e 4000 1 2024-07-03T15:44:07.989 INFO:tasks.workunit.client.0.smithi028.stdout:1/603: mknod d4/d2c/d3a/d5c/d6c/d7d/d4e/cd9 0 2024-07-03T15:44:07.989 INFO:tasks.workunit.client.0.smithi028.stdout:1/604: readlink d4/d3c/d54/d79/d16/d57/l65 0 2024-07-03T15:44:07.990 INFO:tasks.workunit.client.0.smithi028.stdout:1/605: mknod d4/d2c/d3a/d5c/daa/cda 0 2024-07-03T15:44:07.992 INFO:tasks.workunit.client.0.smithi028.stdout:1/606: mkdir d4/d3c/d54/d79/d16/db6/ddb 0 2024-07-03T15:44:08.001 INFO:tasks.workunit.client.0.smithi028.stdout:1/607: dread d4/d3c/d54/d79/d16/d7c/f9f [0,4194304] 0 2024-07-03T15:44:08.003 INFO:tasks.workunit.client.0.smithi028.stdout:1/608: dread d4/de/f31 [0,4194304] 0 2024-07-03T15:44:08.003 INFO:tasks.workunit.client.0.smithi028.stdout:1/609: write d4/de/ff [595398,19974] 0 2024-07-03T15:44:08.011 INFO:tasks.workunit.client.0.smithi028.stdout:1/610: getdents d4/d3c/d54/d79/d16 0 2024-07-03T15:44:08.014 INFO:tasks.workunit.client.0.smithi028.stdout:1/611: getdents d4/da4 0 2024-07-03T15:44:08.014 INFO:tasks.workunit.client.0.smithi028.stdout:1/612: truncate d4/f19 4869052 0 2024-07-03T15:44:08.040 INFO:tasks.workunit.client.0.smithi028.stdout:5/445: sync 2024-07-03T15:44:08.043 INFO:tasks.workunit.client.0.smithi028.stdout:5/446: rename d0/dd/d13/d57/d26/d41/l61 to d0/dd/d14/l93 0 2024-07-03T15:44:08.046 INFO:tasks.workunit.client.0.smithi028.stdout:5/447: dread d0/dd/d13/d57/d30/f90 [4194304,4194304] 0 2024-07-03T15:44:08.048 INFO:tasks.workunit.client.0.smithi028.stdout:5/448: mkdir d0/dd/d1e/d94 0 2024-07-03T15:44:08.048 INFO:tasks.workunit.client.0.smithi028.stdout:5/449: truncate d0/dd/d13/d57/f85 469157 0 2024-07-03T15:44:08.057 INFO:tasks.workunit.client.0.smithi028.stdout:5/450: write d0/dd/d13/d57/f21 [2083462,80834] 0 2024-07-03T15:44:08.057 INFO:tasks.workunit.client.0.smithi028.stdout:5/451: write d0/dd/d13/d57/f21 [4795796,27189] 0 2024-07-03T15:44:08.057 INFO:tasks.workunit.client.0.smithi028.stdout:5/452: fdatasync d0/dd/d1e/f36 0 2024-07-03T15:44:08.069 INFO:tasks.workunit.client.0.smithi028.stdout:5/453: dwrite d0/dd/d1e/f7a [0,4194304] 0 2024-07-03T15:44:08.070 INFO:tasks.workunit.client.0.smithi028.stdout:5/454: chown d0/dd/df/f52 125571285 1 2024-07-03T15:44:08.076 INFO:tasks.workunit.client.0.smithi028.stdout:5/455: write d0/dd/d1e/d3b/f80 [500361,55215] 0 2024-07-03T15:44:08.077 INFO:tasks.workunit.client.0.smithi028.stdout:5/456: read d0/dd/d1e/f36 [3514615,48596] 0 2024-07-03T15:44:08.077 INFO:tasks.workunit.client.0.smithi028.stdout:5/457: read d0/dd/d13/f17 [870437,103500] 0 2024-07-03T15:44:08.077 INFO:tasks.workunit.client.0.smithi028.stdout:5/458: fdatasync d0/dd/d1e/d86/f88 0 2024-07-03T15:44:08.078 INFO:tasks.workunit.client.0.smithi028.stdout:5/459: symlink d0/l95 0 2024-07-03T15:44:08.078 INFO:tasks.workunit.client.0.smithi028.stdout:5/460: truncate d0/dd/d13/d47/d50/f8c 44895 0 2024-07-03T15:44:08.080 INFO:tasks.workunit.client.0.smithi028.stdout:5/461: mknod d0/dd/d13/d57/d26/c96 0 2024-07-03T15:44:08.080 INFO:tasks.workunit.client.0.smithi028.stdout:5/462: fdatasync d0/dd/d1e/d3b/f4f 0 2024-07-03T15:44:08.082 INFO:tasks.workunit.client.0.smithi028.stdout:5/463: truncate d0/dd/d1e/f36 3490007 0 2024-07-03T15:44:08.082 INFO:tasks.workunit.client.0.smithi028.stdout:5/464: truncate d0/dd/d1e/d3b/d3f/d5a/f78 438748 0 2024-07-03T15:44:08.084 INFO:tasks.workunit.client.0.smithi028.stdout:5/465: mknod d0/dd/d13/d57/d30/c97 0 2024-07-03T15:44:08.084 INFO:tasks.workunit.client.0.smithi028.stdout:5/466: chown d0/dd/d1e/d3b/d3f/c5c 12491187 1 2024-07-03T15:44:08.088 INFO:tasks.workunit.client.0.smithi028.stdout:5/467: truncate d0/dd/d1e/f7a 115550 0 2024-07-03T15:44:08.091 INFO:tasks.workunit.client.0.smithi028.stdout:5/468: rename d0/dd/d4b to d0/dd/d98 0 2024-07-03T15:44:08.091 INFO:tasks.workunit.client.0.smithi028.stdout:5/469: chown d0/dd/d13/d29/f68 32602 1 2024-07-03T15:44:08.092 INFO:tasks.workunit.client.0.smithi028.stdout:5/470: write d0/f9 [9162153,125704] 0 2024-07-03T15:44:08.095 INFO:tasks.workunit.client.0.smithi028.stdout:5/471: dread d0/dd/df/f52 [0,4194304] 0 2024-07-03T15:44:08.095 INFO:tasks.workunit.client.0.smithi028.stdout:5/472: write d0/dd/d1e/f36 [3444035,74509] 0 2024-07-03T15:44:08.098 INFO:tasks.workunit.client.0.smithi028.stdout:5/473: creat d0/dd/f99 x:0 0 0 2024-07-03T15:44:08.098 INFO:tasks.workunit.client.0.smithi028.stdout:5/474: write d0/dd/d13/d47/f5b [4663464,8799] 0 2024-07-03T15:44:08.099 INFO:tasks.workunit.client.0.smithi028.stdout:5/475: rename d0/dd/d13/d47/l4d to d0/dd/d1e/l9a 0 2024-07-03T15:44:08.102 INFO:tasks.workunit.client.0.smithi028.stdout:5/476: stat d0/dd/df/c6b 0 2024-07-03T15:44:08.104 INFO:tasks.workunit.client.0.smithi028.stdout:5/477: mknod d0/c9b 0 2024-07-03T15:44:08.107 INFO:tasks.workunit.client.0.smithi028.stdout:5/478: rename d0/dd/d1e/d3b/c40 to d0/dd/d1e/d94/c9c 0 2024-07-03T15:44:08.112 INFO:tasks.workunit.client.0.smithi028.stdout:5/479: mkdir d0/dd/d1e/d3b/d3f/d5a/d9d 0 2024-07-03T15:44:08.116 INFO:tasks.workunit.client.0.smithi028.stdout:5/480: mkdir d0/dd/d13/d57/d30/d70/d7f/d9e 0 2024-07-03T15:44:08.132 INFO:tasks.workunit.client.0.smithi028.stdout:5/481: dwrite d0/dd/d1e/d3b/d3f/d5a/f65 [0,4194304] 0 2024-07-03T15:44:08.133 INFO:tasks.workunit.client.0.smithi028.stdout:5/482: getdents d0/d8f 0 2024-07-03T15:44:08.145 INFO:tasks.workunit.client.0.smithi028.stdout:5/483: write d0/dd/f7e [2479280,69802] 0 2024-07-03T15:44:08.155 INFO:tasks.workunit.client.0.smithi028.stdout:5/484: dread d0/dd/d13/d29/f68 [0,4194304] 0 2024-07-03T15:44:08.156 INFO:tasks.workunit.client.0.smithi028.stdout:5/485: rename d0/dd/d1e/d3b/d3f/d5a/d9d to d0/dd/d14/d9f 0 2024-07-03T15:44:08.158 INFO:tasks.workunit.client.0.smithi028.stdout:5/486: fdatasync d0/dd/d13/d47/f5b 0 2024-07-03T15:44:08.160 INFO:tasks.workunit.client.0.smithi028.stdout:5/487: write d0/dd/d13/d57/d26/d41/f5f [394292,108787] 0 2024-07-03T15:44:08.163 INFO:tasks.workunit.client.0.smithi028.stdout:5/488: rmdir d0/dd/d13/d47/d50 39 2024-07-03T15:44:08.166 INFO:tasks.workunit.client.0.smithi028.stdout:5/489: symlink d0/dd/d98/la0 0 2024-07-03T15:44:08.166 INFO:tasks.workunit.client.0.smithi028.stdout:5/490: chown d0/dd/df/f3e 2714279 1 2024-07-03T15:44:08.167 INFO:tasks.workunit.client.0.smithi028.stdout:5/491: truncate d0/dd/d13/d57/d77/f7d 926255 0 2024-07-03T15:44:08.167 INFO:tasks.workunit.client.0.smithi028.stdout:5/492: chown d0/l4a 446153196 1 2024-07-03T15:44:08.168 INFO:tasks.workunit.client.0.smithi028.stdout:5/493: mkdir d0/dd/df/da1 0 2024-07-03T15:44:08.168 INFO:tasks.workunit.client.0.smithi028.stdout:5/494: stat d0/dd 0 2024-07-03T15:44:08.169 INFO:tasks.workunit.client.0.smithi028.stdout:5/495: getdents d0/dd/d13/d57/d30/d70/d7f/d9e 0 2024-07-03T15:44:08.186 INFO:tasks.workunit.client.0.smithi028.stdout:5/496: dwrite d0/dd/f53 [0,4194304] 0 2024-07-03T15:44:08.199 INFO:tasks.workunit.client.0.smithi028.stdout:5/497: dwrite d0/dd/d1e/f7a [0,4194304] 0 2024-07-03T15:44:08.199 INFO:tasks.workunit.client.0.smithi028.stdout:5/498: write d0/dd/d13/f83 [4123611,106811] 0 2024-07-03T15:44:08.201 INFO:tasks.workunit.client.0.smithi028.stdout:5/499: getdents d0/dd/d1e/d3b 0 2024-07-03T15:44:08.205 INFO:tasks.workunit.client.0.smithi028.stdout:5/500: unlink d0/dd/d13/d29/f68 0 2024-07-03T15:44:08.208 INFO:tasks.workunit.client.0.smithi028.stdout:5/501: mkdir d0/dd/d13/d57/da2 0 2024-07-03T15:44:08.217 INFO:tasks.workunit.client.0.smithi028.stdout:5/502: write d0/dd/d13/d47/f5b [2442829,122946] 0 2024-07-03T15:44:08.220 INFO:tasks.workunit.client.0.smithi028.stdout:5/503: fsync d0/dd/df/f3e 0 2024-07-03T15:44:08.222 INFO:tasks.workunit.client.0.smithi028.stdout:5/504: rename d0/dd/d1e/d3b/f80 to d0/dd/d1e/d3b/fa3 0 2024-07-03T15:44:08.224 INFO:tasks.workunit.client.0.smithi028.stdout:5/505: symlink d0/dd/d1e/d3b/d3f/d5a/la4 0 2024-07-03T15:44:08.224 INFO:tasks.workunit.client.0.smithi028.stdout:5/506: stat d0/dd/d1e/d3b 0 2024-07-03T15:44:08.227 INFO:tasks.workunit.client.0.smithi028.stdout:5/507: rename d0/dd/d13/d57/d26/d41/c4c to d0/dd/d1e/d3b/d3f/d5a/ca5 0 2024-07-03T15:44:08.228 INFO:tasks.workunit.client.0.smithi028.stdout:5/508: creat d0/dd/d1e/d8e/fa6 x:0 0 0 2024-07-03T15:44:08.230 INFO:tasks.workunit.client.0.smithi028.stdout:5/509: mkdir d0/dd/df/da7 0 2024-07-03T15:44:08.231 INFO:tasks.workunit.client.0.smithi028.stdout:7/326: sync 2024-07-03T15:44:08.231 INFO:tasks.workunit.client.0.smithi028.stdout:0/481: sync 2024-07-03T15:44:08.231 INFO:tasks.workunit.client.0.smithi028.stdout:0/482: stat d2/fb 0 2024-07-03T15:44:08.231 INFO:tasks.workunit.client.0.smithi028.stdout:0/483: truncate d2/d6d/d92/d36/d4a/f6f 814099 0 2024-07-03T15:44:08.233 INFO:tasks.workunit.client.0.smithi028.stdout:7/327: creat d9/d1c/d45/f62 x:0 0 0 2024-07-03T15:44:08.235 INFO:tasks.workunit.client.0.smithi028.stdout:5/510: dread d0/dd/d13/d57/d26/d41/f5f [0,4194304] 0 2024-07-03T15:44:08.240 INFO:tasks.workunit.client.0.smithi028.stdout:7/328: dread d9/d29/f2e [0,4194304] 0 2024-07-03T15:44:08.241 INFO:tasks.workunit.client.0.smithi028.stdout:0/484: write d2/d6d/d92/d36/d45/f7b [404573,96268] 0 2024-07-03T15:44:08.243 INFO:tasks.workunit.client.0.smithi028.stdout:7/329: mknod d9/c63 0 2024-07-03T15:44:08.245 INFO:tasks.workunit.client.0.smithi028.stdout:5/511: write d0/dd/d13/d29/f35 [6943222,3624] 0 2024-07-03T15:44:08.247 INFO:tasks.workunit.client.0.smithi028.stdout:0/485: truncate d2/d3/f11 323641 0 2024-07-03T15:44:08.251 INFO:tasks.workunit.client.0.smithi028.stdout:5/512: symlink d0/dd/la8 0 2024-07-03T15:44:08.251 INFO:tasks.workunit.client.0.smithi028.stdout:5/513: creat d0/dd/d14/d9f/fa9 x:0 0 0 2024-07-03T15:44:08.256 INFO:tasks.workunit.client.0.smithi028.stdout:5/514: dread d0/dd/d13/f83 [0,4194304] 0 2024-07-03T15:44:08.257 INFO:tasks.workunit.client.0.smithi028.stdout:0/486: fsync d2/d3/f11 0 2024-07-03T15:44:08.258 INFO:tasks.workunit.client.0.smithi028.stdout:5/515: mkdir d0/dd/df/da1/daa 0 2024-07-03T15:44:08.258 INFO:tasks.workunit.client.0.smithi028.stdout:5/516: write d0/dd/d1e/d3b/d3f/d71/f81 [972000,45280] 0 2024-07-03T15:44:08.260 INFO:tasks.workunit.client.0.smithi028.stdout:0/487: stat d2/l5e 0 2024-07-03T15:44:08.261 INFO:tasks.workunit.client.0.smithi028.stdout:5/517: creat d0/dd/d13/d57/da2/fab x:0 0 0 2024-07-03T15:44:08.262 INFO:tasks.workunit.client.0.smithi028.stdout:5/518: readlink d0/l37 0 2024-07-03T15:44:08.263 INFO:tasks.workunit.client.0.smithi028.stdout:5/519: symlink d0/dd/d1e/lac 0 2024-07-03T15:44:08.264 INFO:tasks.workunit.client.0.smithi028.stdout:7/330: dwrite d9/f10 [0,4194304] 0 2024-07-03T15:44:08.270 INFO:tasks.workunit.client.0.smithi028.stdout:5/520: dread d0/dd/df/f52 [0,4194304] 0 2024-07-03T15:44:08.272 INFO:tasks.workunit.client.0.smithi028.stdout:5/521: truncate d0/dd/d98/f6c 2873213 0 2024-07-03T15:44:08.296 INFO:tasks.workunit.client.0.smithi028.stdout:0/488: dwrite d2/d3/d4d/d56/f7f [0,4194304] 0 2024-07-03T15:44:08.297 INFO:tasks.workunit.client.0.smithi028.stdout:0/489: mknod d2/d6d/d92/d58/d6e/c93 0 2024-07-03T15:44:08.298 INFO:tasks.workunit.client.0.smithi028.stdout:0/490: truncate d2/d3/d4d/d56/d78/f91 566289 0 2024-07-03T15:44:08.298 INFO:tasks.workunit.client.0.smithi028.stdout:0/491: chown d2/d6d/d92/d36/d45/f72 460201418 1 2024-07-03T15:44:08.299 INFO:tasks.workunit.client.0.smithi028.stdout:0/492: creat d2/d6d/d92/d58/d6e/f94 x:0 0 0 2024-07-03T15:44:08.300 INFO:tasks.workunit.client.0.smithi028.stdout:0/493: fdatasync d2/d3/f25 0 2024-07-03T15:44:08.300 INFO:tasks.workunit.client.0.smithi028.stdout:7/331: dwrite d9/d29/f3c [0,4194304] 0 2024-07-03T15:44:08.303 INFO:tasks.workunit.client.0.smithi028.stdout:0/494: dread d2/d1c/f2d [0,4194304] 0 2024-07-03T15:44:08.362 INFO:tasks.workunit.client.0.smithi028.stdout:7/332: rename d9/d1c/l2f to d9/d1c/d52/l64 0 2024-07-03T15:44:08.366 INFO:tasks.workunit.client.0.smithi028.stdout:7/333: symlink d9/d1c/l65 0 2024-07-03T15:44:08.370 INFO:tasks.workunit.client.0.smithi028.stdout:7/334: getdents d9/d1c/d45 0 2024-07-03T15:44:08.370 INFO:tasks.workunit.client.0.smithi028.stdout:7/335: readlink l8 0 2024-07-03T15:44:08.371 INFO:tasks.workunit.client.0.smithi028.stdout:7/336: read d9/fa [30389,92541] 0 2024-07-03T15:44:08.372 INFO:tasks.workunit.client.0.smithi028.stdout:7/337: truncate d9/d29/f2e 1272566 0 2024-07-03T15:44:08.380 INFO:tasks.workunit.client.0.smithi028.stdout:7/338: symlink d9/d1c/l66 0 2024-07-03T15:44:08.381 INFO:tasks.workunit.client.0.smithi028.stdout:7/339: creat d9/d29/f67 x:0 0 0 2024-07-03T15:44:08.398 INFO:tasks.workunit.client.0.smithi028.stdout:7/340: dwrite f0 [8388608,4194304] 0 2024-07-03T15:44:08.405 INFO:tasks.workunit.client.0.smithi028.stdout:7/341: dread f0 [4194304,4194304] 0 2024-07-03T15:44:08.420 INFO:tasks.workunit.client.0.smithi028.stdout:7/342: dwrite d9/f25 [0,4194304] 0 2024-07-03T15:44:08.460 INFO:tasks.workunit.client.0.smithi028.stdout:7/343: link d9/l4d d9/d29/d5d/l68 0 2024-07-03T15:44:08.460 INFO:tasks.workunit.client.0.smithi028.stdout:7/344: chown d9/c63 13351 1 2024-07-03T15:44:08.521 INFO:tasks.workunit.client.0.smithi028.stdout:7/345: truncate d9/d1c/f4b 140332 0 2024-07-03T15:44:08.524 INFO:tasks.workunit.client.0.smithi028.stdout:7/346: creat d9/f69 x:0 0 0 2024-07-03T15:44:08.524 INFO:tasks.workunit.client.0.smithi028.stdout:3/454: sync 2024-07-03T15:44:08.524 INFO:tasks.workunit.client.0.smithi028.stdout:3/455: chown d18/c1b 10446 1 2024-07-03T15:44:08.524 INFO:tasks.workunit.client.0.smithi028.stdout:3/456: chown d18/d1a/f4f 1556811 1 2024-07-03T15:44:08.527 INFO:tasks.workunit.client.0.smithi028.stdout:7/347: mkdir d9/d1c/d52/d6a 0 2024-07-03T15:44:08.530 INFO:tasks.workunit.client.0.smithi028.stdout:7/348: creat d9/d1c/f6b x:0 0 0 2024-07-03T15:44:08.535 INFO:tasks.workunit.client.0.smithi028.stdout:3/457: dread d18/f1d [0,4194304] 0 2024-07-03T15:44:08.535 INFO:tasks.workunit.client.0.smithi028.stdout:3/458: write d18/d3a/f74 [216008,125339] 0 2024-07-03T15:44:08.535 INFO:tasks.workunit.client.0.smithi028.stdout:3/459: write f6 [173728,1345] 0 2024-07-03T15:44:08.535 INFO:tasks.workunit.client.0.smithi028.stdout:7/349: creat d9/d1c/d52/d6a/f6c x:0 0 0 2024-07-03T15:44:08.538 INFO:tasks.workunit.client.0.smithi028.stdout:7/350: dread d9/fa [0,4194304] 0 2024-07-03T15:44:08.538 INFO:tasks.workunit.client.0.smithi028.stdout:7/351: write d9/d29/f67 [816451,26895] 0 2024-07-03T15:44:08.538 INFO:tasks.workunit.client.0.smithi028.stdout:7/352: chown d9/d29/f67 9482 1 2024-07-03T15:44:08.538 INFO:tasks.workunit.client.0.smithi028.stdout:7/353: write d9/f34 [3000514,76971] 0 2024-07-03T15:44:08.539 INFO:tasks.workunit.client.0.smithi028.stdout:7/354: mknod d9/d1c/d52/d6a/c6d 0 2024-07-03T15:44:08.542 INFO:tasks.workunit.client.0.smithi028.stdout:7/355: dread d9/d29/f42 [0,4194304] 0 2024-07-03T15:44:08.542 INFO:tasks.workunit.client.0.smithi028.stdout:7/356: write d9/fa [1708122,128945] 0 2024-07-03T15:44:08.553 INFO:tasks.workunit.client.0.smithi028.stdout:3/460: dwrite d18/d3a/f74 [0,4194304] 0 2024-07-03T15:44:08.553 INFO:tasks.workunit.client.0.smithi028.stdout:3/461: readlink l13 0 2024-07-03T15:44:08.553 INFO:tasks.workunit.client.0.smithi028.stdout:3/462: write f14 [465614,48762] 0 2024-07-03T15:44:08.557 INFO:tasks.workunit.client.0.smithi028.stdout:3/463: rename d18/d1a/f28 to d18/d1a/d5e/d60/f8e 0 2024-07-03T15:44:08.557 INFO:tasks.workunit.client.0.smithi028.stdout:3/464: readlink d18/d6f/d78/l80 0 2024-07-03T15:44:08.562 INFO:tasks.workunit.client.0.smithi028.stdout:3/465: dread d18/f55 [0,4194304] 0 2024-07-03T15:44:08.562 INFO:tasks.workunit.client.0.smithi028.stdout:3/466: read - d18/d1a/d1e/d58/f7f zero size 2024-07-03T15:44:08.564 INFO:tasks.workunit.client.0.smithi028.stdout:3/467: creat d18/d1a/d64/f8f x:0 0 0 2024-07-03T15:44:08.570 INFO:tasks.workunit.client.0.smithi028.stdout:7/357: dwrite d9/fa [0,4194304] 0 2024-07-03T15:44:08.570 INFO:tasks.workunit.client.0.smithi028.stdout:7/358: read d9/d29/f5e [107711,52226] 0 2024-07-03T15:44:08.590 INFO:tasks.workunit.client.0.smithi028.stdout:9/497: sync 2024-07-03T15:44:08.590 INFO:tasks.workunit.client.0.smithi028.stdout:9/498: chown d1/d16 4724 1 2024-07-03T15:44:08.590 INFO:tasks.workunit.client.0.smithi028.stdout:4/498: sync 2024-07-03T15:44:08.590 INFO:tasks.workunit.client.0.smithi028.stdout:6/409: sync 2024-07-03T15:44:08.590 INFO:tasks.workunit.client.0.smithi028.stdout:8/510: sync 2024-07-03T15:44:08.590 INFO:tasks.workunit.client.0.smithi028.stdout:4/499: truncate d2/f7 907314 0 2024-07-03T15:44:08.590 INFO:tasks.workunit.client.0.smithi028.stdout:6/410: dread - d4/da/d38/f5f zero size 2024-07-03T15:44:08.590 INFO:tasks.workunit.client.0.smithi028.stdout:4/500: dread - d2/d12/d2d/d58/f70 zero size 2024-07-03T15:44:08.590 INFO:tasks.workunit.client.0.smithi028.stdout:4/501: stat d2/f2b 0 2024-07-03T15:44:08.590 INFO:tasks.workunit.client.0.smithi028.stdout:4/502: fdatasync d2/d5a/f9 0 2024-07-03T15:44:08.591 INFO:tasks.workunit.client.0.smithi028.stdout:4/503: dread d2/d12/d2d/f32 [0,4194304] 0 2024-07-03T15:44:08.593 INFO:tasks.workunit.client.0.smithi028.stdout:8/511: unlink d0/l8 0 2024-07-03T15:44:08.593 INFO:tasks.workunit.client.0.smithi028.stdout:8/512: truncate d0/d41/d50/d3/dc/f63 941714 0 2024-07-03T15:44:08.594 INFO:tasks.workunit.client.0.smithi028.stdout:4/504: dread d2/d3e/f50 [0,4194304] 0 2024-07-03T15:44:08.596 INFO:tasks.workunit.client.0.smithi028.stdout:6/411: unlink d4/da/d13/f19 0 2024-07-03T15:44:08.597 INFO:tasks.workunit.client.0.smithi028.stdout:9/499: getdents d1/d10/d28 0 2024-07-03T15:44:08.598 INFO:tasks.workunit.client.0.smithi028.stdout:8/513: mknod d0/d6c/cab 0 2024-07-03T15:44:08.600 INFO:tasks.workunit.client.0.smithi028.stdout:9/500: mkdir d1/d16/d57/d60/dab 0 2024-07-03T15:44:08.600 INFO:tasks.workunit.client.0.smithi028.stdout:9/501: chown d1/d4a/d7e 2048784178 1 2024-07-03T15:44:08.602 INFO:tasks.workunit.client.0.smithi028.stdout:8/514: unlink d0/d41/d50/d3/c4f 0 2024-07-03T15:44:08.602 INFO:tasks.workunit.client.0.smithi028.stdout:8/515: dread - d0/d41/d50/d3/d35/d68/d22/d40/f8c zero size 2024-07-03T15:44:08.602 INFO:tasks.workunit.client.0.smithi028.stdout:8/516: fdatasync d0/d41/d50/d3/d35/d68/d22/d40/d62/f65 0 2024-07-03T15:44:08.603 INFO:tasks.workunit.client.0.smithi028.stdout:4/505: mkdir d2/d12/da6 0 2024-07-03T15:44:08.604 INFO:tasks.workunit.client.0.smithi028.stdout:9/502: getdents d1/d10/d28/d45/d61/d8c 0 2024-07-03T15:44:08.605 INFO:tasks.workunit.client.0.smithi028.stdout:8/517: rename d0/d41/d50/d3/d35/d68/d22/l96 to d0/d41/d92/lac 0 2024-07-03T15:44:08.606 INFO:tasks.workunit.client.0.smithi028.stdout:4/506: mknod d2/d12/d5b/ca7 0 2024-07-03T15:44:08.606 INFO:tasks.workunit.client.0.smithi028.stdout:4/507: read d2/d12/d2d/d2f/f66 [474665,18588] 0 2024-07-03T15:44:08.609 INFO:tasks.workunit.client.0.smithi028.stdout:9/503: mkdir d1/d8e/dac 0 2024-07-03T15:44:08.609 INFO:tasks.workunit.client.0.smithi028.stdout:9/504: read - d1/d10/d28/d45/d56/f8a zero size 2024-07-03T15:44:08.609 INFO:tasks.workunit.client.0.smithi028.stdout:9/505: write d1/d8e/f9e [926349,84829] 0 2024-07-03T15:44:08.612 INFO:tasks.workunit.client.0.smithi028.stdout:4/508: mkdir d2/d12/d42/d63/da8 0 2024-07-03T15:44:08.612 INFO:tasks.workunit.client.0.smithi028.stdout:4/509: read - d2/d12/d42/d63/f65 zero size 2024-07-03T15:44:08.612 INFO:tasks.workunit.client.0.smithi028.stdout:4/510: readlink d2/d12/d2d/d2f/l46 0 2024-07-03T15:44:08.612 INFO:tasks.workunit.client.0.smithi028.stdout:6/412: dwrite d4/da/d1c/f54 [4194304,4194304] 0 2024-07-03T15:44:08.612 INFO:tasks.workunit.client.0.smithi028.stdout:6/413: chown d4/d12/d2c/d31/l4b 38857 1 2024-07-03T15:44:08.612 INFO:tasks.workunit.client.0.smithi028.stdout:6/414: fdatasync d4/da/d38/f46 0 2024-07-03T15:44:08.614 INFO:tasks.workunit.client.0.smithi028.stdout:4/511: creat d2/d12/d42/d63/d75/fa9 x:0 0 0 2024-07-03T15:44:08.615 INFO:tasks.workunit.client.0.smithi028.stdout:4/512: fdatasync d2/d12/d5b/f8c 0 2024-07-03T15:44:08.615 INFO:tasks.workunit.client.0.smithi028.stdout:4/513: chown d2/d5a/d45/da5 705 1 2024-07-03T15:44:08.616 INFO:tasks.workunit.client.0.smithi028.stdout:4/514: dread d2/d5a/d45/d84/f8f [0,4194304] 0 2024-07-03T15:44:08.616 INFO:tasks.workunit.client.0.smithi028.stdout:4/515: write d2/d12/d2c/f7d [206828,75046] 0 2024-07-03T15:44:08.616 INFO:tasks.workunit.client.0.smithi028.stdout:4/516: dread - d2/d12/d2d/d58/fa3 zero size 2024-07-03T15:44:08.616 INFO:tasks.workunit.client.0.smithi028.stdout:4/517: stat d2/d5a/f9 0 2024-07-03T15:44:08.617 INFO:tasks.workunit.client.0.smithi028.stdout:4/518: fdatasync d2/d12/d5b/f7a 0 2024-07-03T15:44:08.618 INFO:tasks.workunit.client.0.smithi028.stdout:6/415: creat d4/da/d1c/f88 x:0 0 0 2024-07-03T15:44:08.618 INFO:tasks.workunit.client.0.smithi028.stdout:6/416: stat d4/da/d47/f6b 0 2024-07-03T15:44:08.619 INFO:tasks.workunit.client.0.smithi028.stdout:6/417: mkdir d4/da/d38/d3d/d89 0 2024-07-03T15:44:08.620 INFO:tasks.workunit.client.0.smithi028.stdout:8/518: dwrite d0/d41/d50/d3/d35/d68/d22/d40/d62/f65 [0,4194304] 0 2024-07-03T15:44:08.621 INFO:tasks.workunit.client.0.smithi028.stdout:8/519: write d0/d41/d50/d3/dc/d3f/d8e/d5d/d9f/da6/f9c [572466,50572] 0 2024-07-03T15:44:08.642 INFO:tasks.workunit.client.0.smithi028.stdout:6/418: write d4/da/d47/f51 [322307,26927] 0 2024-07-03T15:44:08.643 INFO:tasks.workunit.client.0.smithi028.stdout:4/519: dwrite d2/f1b [0,4194304] 0 2024-07-03T15:44:08.643 INFO:tasks.workunit.client.0.smithi028.stdout:4/520: write d2/d12/d2e/f47 [5129535,71238] 0 2024-07-03T15:44:08.644 INFO:tasks.workunit.client.0.smithi028.stdout:4/521: creat d2/d12/d42/d63/da8/faa x:0 0 0 2024-07-03T15:44:08.647 INFO:tasks.workunit.client.0.smithi028.stdout:8/520: dwrite d0/d41/d50/d3/dc/d3f/d8e/f6a [4194304,4194304] 0 2024-07-03T15:44:08.648 INFO:tasks.workunit.client.0.smithi028.stdout:8/521: write d0/d41/d50/d3/dc/f63 [122076,107745] 0 2024-07-03T15:44:08.649 INFO:tasks.workunit.client.0.smithi028.stdout:8/522: mknod d0/d41/d50/d3/d35/d68/d22/d40/d45/cad 0 2024-07-03T15:44:08.653 INFO:tasks.workunit.client.0.smithi028.stdout:6/419: dread d4/f10 [0,4194304] 0 2024-07-03T15:44:08.653 INFO:tasks.workunit.client.0.smithi028.stdout:8/523: dread d0/f2a [0,4194304] 0 2024-07-03T15:44:08.654 INFO:tasks.workunit.client.0.smithi028.stdout:6/420: mknod d4/da/d13/c8a 0 2024-07-03T15:44:08.656 INFO:tasks.workunit.client.0.smithi028.stdout:8/524: creat d0/d6c/fae x:0 0 0 2024-07-03T15:44:08.656 INFO:tasks.workunit.client.0.smithi028.stdout:8/525: write d0/d41/d50/d3/d35/d68/d22/d40/f60 [77056,99669] 0 2024-07-03T15:44:08.657 INFO:tasks.workunit.client.0.smithi028.stdout:6/421: creat d4/da/d47/d66/f8b x:0 0 0 2024-07-03T15:44:08.658 INFO:tasks.workunit.client.0.smithi028.stdout:8/526: mkdir d0/d41/d50/daf 0 2024-07-03T15:44:08.658 INFO:tasks.workunit.client.0.smithi028.stdout:8/527: dread - d0/d41/d50/d3/dc/d3f/d8e/d5d/d9f/da6/f82 zero size 2024-07-03T15:44:08.659 INFO:tasks.workunit.client.0.smithi028.stdout:8/528: mknod d0/d41/d50/daf/cb0 0 2024-07-03T15:44:08.659 INFO:tasks.workunit.client.0.smithi028.stdout:8/529: chown d0/d41 3297 1 2024-07-03T15:44:08.660 INFO:tasks.workunit.client.0.smithi028.stdout:8/530: truncate d0/d41/d50/d3/dc/d33/d7a/f56 2366165 0 2024-07-03T15:44:08.662 INFO:tasks.workunit.client.0.smithi028.stdout:8/531: creat d0/d41/d50/d3/d35/d68/d22/fb1 x:0 0 0 2024-07-03T15:44:08.663 INFO:tasks.workunit.client.0.smithi028.stdout:8/532: write d0/d41/d50/d3/dc/d3f/d8e/f42 [696204,52356] 0 2024-07-03T15:44:08.664 INFO:tasks.workunit.client.0.smithi028.stdout:4/522: dwrite d2/d12/d2d/d2f/f66 [0,4194304] 0 2024-07-03T15:44:08.664 INFO:tasks.workunit.client.0.smithi028.stdout:4/523: fdatasync d2/d5a/f56 0 2024-07-03T15:44:08.664 INFO:tasks.workunit.client.0.smithi028.stdout:8/533: creat d0/d41/d50/d3/dc/fb2 x:0 0 0 2024-07-03T15:44:08.667 INFO:tasks.workunit.client.0.smithi028.stdout:4/524: symlink d2/d12/d2d/d2f/lab 0 2024-07-03T15:44:08.669 INFO:tasks.workunit.client.0.smithi028.stdout:8/534: unlink d0/d41/d50/d3/d35/d68/d22/d40/f8c 0 2024-07-03T15:44:08.670 INFO:tasks.workunit.client.0.smithi028.stdout:6/422: write d4/da/d38/d59/f65 [1876108,31997] 0 2024-07-03T15:44:08.671 INFO:tasks.workunit.client.0.smithi028.stdout:4/525: mkdir d2/d3e/d9f/dac 0 2024-07-03T15:44:08.672 INFO:tasks.workunit.client.0.smithi028.stdout:6/423: dread d4/d15/f5b [0,4194304] 0 2024-07-03T15:44:08.673 INFO:tasks.workunit.client.0.smithi028.stdout:8/535: mkdir d0/d41/d50/d3/dc/db3 0 2024-07-03T15:44:08.675 INFO:tasks.workunit.client.0.smithi028.stdout:8/536: creat d0/d6c/da9/fb4 x:0 0 0 2024-07-03T15:44:08.683 INFO:tasks.workunit.client.0.smithi028.stdout:4/526: write f1 [2801332,81292] 0 2024-07-03T15:44:08.686 INFO:tasks.workunit.client.0.smithi028.stdout:6/424: write d4/d12/f36 [1931322,87190] 0 2024-07-03T15:44:08.686 INFO:tasks.workunit.client.0.smithi028.stdout:6/425: fdatasync d4/d15/f2a 0 2024-07-03T15:44:08.686 INFO:tasks.workunit.client.0.smithi028.stdout:6/426: creat d4/d15/f8c x:0 0 0 2024-07-03T15:44:08.686 INFO:tasks.workunit.client.0.smithi028.stdout:6/427: write d4/da/d47/d66/f8b [606236,1761] 0 2024-07-03T15:44:08.700 INFO:tasks.workunit.client.0.smithi028.stdout:6/428: rmdir d4/da/d1c 39 2024-07-03T15:44:08.700 INFO:tasks.workunit.client.0.smithi028.stdout:6/429: fsync d4/d15/f2a 0 2024-07-03T15:44:08.700 INFO:tasks.workunit.client.0.smithi028.stdout:6/430: chown d4 61 1 2024-07-03T15:44:08.701 INFO:tasks.workunit.client.0.smithi028.stdout:4/527: dwrite d2/d12/d2e/f38 [0,4194304] 0 2024-07-03T15:44:08.701 INFO:tasks.workunit.client.0.smithi028.stdout:4/528: dread - d2/d12/d5b/d78/f79 zero size 2024-07-03T15:44:08.701 INFO:tasks.workunit.client.0.smithi028.stdout:4/529: chown d2/d12/d2d/l7e 35 1 2024-07-03T15:44:08.701 INFO:tasks.workunit.client.0.smithi028.stdout:6/431: creat d4/f8d x:0 0 0 2024-07-03T15:44:08.701 INFO:tasks.workunit.client.0.smithi028.stdout:6/432: creat d4/d12/d2c/d31/f8e x:0 0 0 2024-07-03T15:44:08.702 INFO:tasks.workunit.client.0.smithi028.stdout:4/530: write d2/f13 [934565,44467] 0 2024-07-03T15:44:08.702 INFO:tasks.workunit.client.0.smithi028.stdout:4/531: fdatasync d2/d12/f68 0 2024-07-03T15:44:08.704 INFO:tasks.workunit.client.0.smithi028.stdout:4/532: dread d2/d5a/f34 [0,4194304] 0 2024-07-03T15:44:08.704 INFO:tasks.workunit.client.0.smithi028.stdout:4/533: write d2/d12/d42/d63/d75/fa9 [48515,48950] 0 2024-07-03T15:44:08.707 INFO:tasks.workunit.client.0.smithi028.stdout:6/433: rename d4/c2e to d4/da/d38/d59/c8f 0 2024-07-03T15:44:08.708 INFO:tasks.workunit.client.0.smithi028.stdout:4/534: mknod d2/d12/d42/d63/d9e/cad 0 2024-07-03T15:44:08.709 INFO:tasks.workunit.client.0.smithi028.stdout:6/434: symlink d4/da/l90 0 2024-07-03T15:44:08.710 INFO:tasks.workunit.client.0.smithi028.stdout:4/535: getdents d2/d5a/d45 0 2024-07-03T15:44:08.718 INFO:tasks.workunit.client.0.smithi028.stdout:4/536: read d2/d12/d5b/f81 [4119562,15710] 0 2024-07-03T15:44:08.719 INFO:tasks.workunit.client.0.smithi028.stdout:6/435: write d4/f3a [399861,68861] 0 2024-07-03T15:44:08.720 INFO:tasks.workunit.client.0.smithi028.stdout:4/537: mkdir d2/d3e/dae 0 2024-07-03T15:44:08.720 INFO:tasks.workunit.client.0.smithi028.stdout:4/538: write d2/d12/d2e/f6b [1171844,67063] 0 2024-07-03T15:44:08.721 INFO:tasks.workunit.client.0.smithi028.stdout:4/539: dread - d2/d12/d5b/f7a zero size 2024-07-03T15:44:08.721 INFO:tasks.workunit.client.0.smithi028.stdout:4/540: getdents d2/d61/d9c 0 2024-07-03T15:44:08.724 INFO:tasks.workunit.client.0.smithi028.stdout:4/541: getdents d2/d12/d2d 0 2024-07-03T15:44:08.725 INFO:tasks.workunit.client.0.smithi028.stdout:6/436: dread d4/f33 [0,4194304] 0 2024-07-03T15:44:08.728 INFO:tasks.workunit.client.0.smithi028.stdout:6/437: mkdir d4/da/d38/d59/d91 0 2024-07-03T15:44:08.728 INFO:tasks.workunit.client.0.smithi028.stdout:6/438: chown d4/d15/c3c 5903 1 2024-07-03T15:44:08.728 INFO:tasks.workunit.client.0.smithi028.stdout:6/439: getdents d4/d12 0 2024-07-03T15:44:08.734 INFO:tasks.workunit.client.0.smithi028.stdout:5/522: sync 2024-07-03T15:44:08.734 INFO:tasks.workunit.client.0.smithi028.stdout:2/433: sync 2024-07-03T15:44:08.734 INFO:tasks.workunit.client.0.smithi028.stdout:1/613: sync 2024-07-03T15:44:08.737 INFO:tasks.workunit.client.0.smithi028.stdout:6/440: creat d4/da/d38/d59/f92 x:0 0 0 2024-07-03T15:44:08.740 INFO:tasks.workunit.client.0.smithi028.stdout:2/434: creat da/d42/d9e/fa0 x:0 0 0 2024-07-03T15:44:08.744 INFO:tasks.workunit.client.0.smithi028.stdout:1/614: creat d4/d3c/d54/d79/d16/d7c/dd5/fdc x:0 0 0 2024-07-03T15:44:08.744 INFO:tasks.workunit.client.0.smithi028.stdout:1/615: write d4/d3c/d54/d79/d16/d7c/dd5/fdc [645144,93798] 0 2024-07-03T15:44:08.746 INFO:tasks.workunit.client.0.smithi028.stdout:6/441: mkdir d4/da/d38/d93 0 2024-07-03T15:44:08.747 INFO:tasks.workunit.client.0.smithi028.stdout:2/435: rmdir da/d23/d6f/d84 39 2024-07-03T15:44:08.749 INFO:tasks.workunit.client.0.smithi028.stdout:6/442: mknod d4/da/d56/c94 0 2024-07-03T15:44:08.749 INFO:tasks.workunit.client.0.smithi028.stdout:5/523: dwrite d0/dd/d1e/f7a [0,4194304] 0 2024-07-03T15:44:08.751 INFO:tasks.workunit.client.0.smithi028.stdout:2/436: unlink da/d42/d47/d78/f92 0 2024-07-03T15:44:08.754 INFO:tasks.workunit.client.0.smithi028.stdout:2/437: symlink da/la1 0 2024-07-03T15:44:08.755 INFO:tasks.workunit.client.0.smithi028.stdout:2/438: stat l1 0 2024-07-03T15:44:08.756 INFO:tasks.workunit.client.0.smithi028.stdout:5/524: dread d0/f2c [0,4194304] 0 2024-07-03T15:44:08.757 INFO:tasks.workunit.client.0.smithi028.stdout:6/443: mknod d4/da/d1c/c95 0 2024-07-03T15:44:08.760 INFO:tasks.workunit.client.0.smithi028.stdout:5/525: mkdir d0/dd/dad 0 2024-07-03T15:44:08.760 INFO:tasks.workunit.client.0.smithi028.stdout:5/526: creat d0/dd/fae x:0 0 0 2024-07-03T15:44:08.760 INFO:tasks.workunit.client.0.smithi028.stdout:5/527: write d0/dd/d13/d57/d26/d41/f5f [758340,18710] 0 2024-07-03T15:44:08.763 INFO:tasks.workunit.client.0.smithi028.stdout:6/444: symlink d4/da/d38/d3d/l96 0 2024-07-03T15:44:08.766 INFO:tasks.workunit.client.0.smithi028.stdout:6/445: creat d4/da/d38/d3d/d89/f97 x:0 0 0 2024-07-03T15:44:08.766 INFO:tasks.workunit.client.0.smithi028.stdout:6/446: readlink d4/d15/l30 0 2024-07-03T15:44:08.767 INFO:tasks.workunit.client.0.smithi028.stdout:6/447: creat d4/da/d1c/f98 x:0 0 0 2024-07-03T15:44:08.768 INFO:tasks.workunit.client.0.smithi028.stdout:6/448: mknod d4/da/d38/d59/c99 0 2024-07-03T15:44:08.770 INFO:tasks.workunit.client.0.smithi028.stdout:2/439: dwrite da/d23/d6f/f7c [4194304,4194304] 0 2024-07-03T15:44:08.770 INFO:tasks.workunit.client.0.smithi028.stdout:6/449: mknod d4/d15/d68/c9a 0 2024-07-03T15:44:08.770 INFO:tasks.workunit.client.0.smithi028.stdout:6/450: truncate d4/f8d 425116 0 2024-07-03T15:44:08.772 INFO:tasks.workunit.client.0.smithi028.stdout:2/440: mkdir da/d42/d47/da2 0 2024-07-03T15:44:08.772 INFO:tasks.workunit.client.0.smithi028.stdout:2/441: fsync da/d42/d47/d78/f9d 0 2024-07-03T15:44:08.772 INFO:tasks.workunit.client.0.smithi028.stdout:2/442: creat da/d42/d47/d78/fa3 x:0 0 0 2024-07-03T15:44:08.773 INFO:tasks.workunit.client.0.smithi028.stdout:6/451: creat d4/da/d38/f9b x:0 0 0 2024-07-03T15:44:08.776 INFO:tasks.workunit.client.0.smithi028.stdout:6/452: unlink d4/c82 0 2024-07-03T15:44:08.783 INFO:tasks.workunit.client.0.smithi028.stdout:6/453: dread d4/f45 [0,4194304] 0 2024-07-03T15:44:08.784 INFO:tasks.workunit.client.0.smithi028.stdout:6/454: creat d4/d15/d68/f9c x:0 0 0 2024-07-03T15:44:08.784 INFO:tasks.workunit.client.0.smithi028.stdout:6/455: chown d4/da/d38/f46 11393 1 2024-07-03T15:44:08.790 INFO:tasks.workunit.client.0.smithi028.stdout:6/456: dread d4/f10 [0,4194304] 0 2024-07-03T15:44:08.791 INFO:tasks.workunit.client.0.smithi028.stdout:6/457: mkdir d4/d15/d9d 0 2024-07-03T15:44:08.792 INFO:tasks.workunit.client.0.smithi028.stdout:6/458: rmdir d4/da/d38/d3d 39 2024-07-03T15:44:08.793 INFO:tasks.workunit.client.0.smithi028.stdout:6/459: creat d4/d15/d9d/f9e x:0 0 0 2024-07-03T15:44:08.794 INFO:tasks.workunit.client.0.smithi028.stdout:6/460: mkdir d4/da/d47/d9f 0 2024-07-03T15:44:08.794 INFO:tasks.workunit.client.0.smithi028.stdout:6/461: chown d4/f3a 369453508 1 2024-07-03T15:44:08.795 INFO:tasks.workunit.client.0.smithi028.stdout:6/462: mknod d4/d12/d2c/ca0 0 2024-07-03T15:44:08.795 INFO:tasks.workunit.client.0.smithi028.stdout:6/463: mknod d4/da/d47/ca1 0 2024-07-03T15:44:08.797 INFO:tasks.workunit.client.0.smithi028.stdout:6/464: rmdir d4/da/d38/d93 0 2024-07-03T15:44:08.798 INFO:tasks.workunit.client.0.smithi028.stdout:6/465: unlink d4/d15/l5c 0 2024-07-03T15:44:08.800 INFO:tasks.workunit.client.0.smithi028.stdout:6/466: link d4/da/f7c d4/d12/fa2 0 2024-07-03T15:44:08.928 INFO:tasks.workunit.client.0.smithi028.stdout:0/495: sync 2024-07-03T15:44:08.928 INFO:tasks.workunit.client.0.smithi028.stdout:0/496: truncate d2/d3/f8d 396962 0 2024-07-03T15:44:08.928 INFO:tasks.workunit.client.0.smithi028.stdout:0/497: stat d2/d1c/f41 0 2024-07-03T15:44:08.959 INFO:tasks.workunit.client.0.smithi028.stdout:0/498: dwrite d2/d6d/d92/d58/d6e/f80 [0,4194304] 0 2024-07-03T15:44:08.959 INFO:tasks.workunit.client.0.smithi028.stdout:0/499: write d2/f3b [664012,60590] 0 2024-07-03T15:44:08.965 INFO:tasks.workunit.client.0.smithi028.stdout:0/500: rename d2/d3/d4d/d56/d78/c90 to d2/d6d/d92/d36/d4a/c95 0 2024-07-03T15:44:08.966 INFO:tasks.workunit.client.0.smithi028.stdout:0/501: dread d2/d3/f2f [0,4194304] 0 2024-07-03T15:44:08.967 INFO:tasks.workunit.client.0.smithi028.stdout:0/502: mkdir d2/d3/d4d/d96 0 2024-07-03T15:44:08.970 INFO:tasks.workunit.client.0.smithi028.stdout:0/503: symlink d2/d6d/d92/l97 0 2024-07-03T15:44:08.991 INFO:tasks.workunit.client.0.smithi028.stdout:0/504: dwrite d2/d3/f2f [0,4194304] 0 2024-07-03T15:44:08.996 INFO:tasks.workunit.client.0.smithi028.stdout:0/505: truncate d2/d6d/d92/d36/f3d 667868 0 2024-07-03T15:44:08.999 INFO:tasks.workunit.client.0.smithi028.stdout:0/506: rename d2/d6d/d92/d36/l40 to d2/d3/d4d/d56/l98 0 2024-07-03T15:44:09.000 INFO:tasks.workunit.client.0.smithi028.stdout:3/468: sync 2024-07-03T15:44:09.000 INFO:tasks.workunit.client.0.smithi028.stdout:7/359: sync 2024-07-03T15:44:09.000 INFO:tasks.workunit.client.0.smithi028.stdout:7/360: fdatasync d9/f1a 0 2024-07-03T15:44:09.001 INFO:tasks.workunit.client.0.smithi028.stdout:0/507: rename d2/d6d/d92/d58/f5a to d2/d6d/d92/d36/d45/f99 0 2024-07-03T15:44:09.004 INFO:tasks.workunit.client.0.smithi028.stdout:7/361: dread d9/d1c/f20 [0,4194304] 0 2024-07-03T15:44:09.007 INFO:tasks.workunit.client.0.smithi028.stdout:7/362: mkdir d9/d1c/d6e 0 2024-07-03T15:44:09.007 INFO:tasks.workunit.client.0.smithi028.stdout:7/363: chown d9/d1c/l65 506062 1 2024-07-03T15:44:09.026 INFO:tasks.workunit.client.0.smithi028.stdout:3/469: dwrite d18/d3a/f3d [0,4194304] 0 2024-07-03T15:44:09.028 INFO:tasks.workunit.client.0.smithi028.stdout:3/470: mknod d18/d1a/d1e/d83/c90 0 2024-07-03T15:44:09.029 INFO:tasks.workunit.client.0.smithi028.stdout:3/471: creat d18/d1a/d5e/d66/f91 x:0 0 0 2024-07-03T15:44:09.029 INFO:tasks.workunit.client.0.smithi028.stdout:3/472: write d18/d1a/d5e/f7b [1019985,77427] 0 2024-07-03T15:44:09.029 INFO:tasks.workunit.client.0.smithi028.stdout:3/473: write d18/d1a/d1e/d58/f6c [823431,69617] 0 2024-07-03T15:44:09.029 INFO:tasks.workunit.client.0.smithi028.stdout:0/508: dwrite d2/d1c/f1e [0,4194304] 0 2024-07-03T15:44:09.032 INFO:tasks.workunit.client.0.smithi028.stdout:3/474: unlink ld 0 2024-07-03T15:44:09.034 INFO:tasks.workunit.client.0.smithi028.stdout:0/509: creat d2/d6d/d92/d36/f9a x:0 0 0 2024-07-03T15:44:09.034 INFO:tasks.workunit.client.0.smithi028.stdout:0/510: chown d2/d3/d4d 24 1 2024-07-03T15:44:09.036 INFO:tasks.workunit.client.0.smithi028.stdout:3/475: rename d18/d3a/f74 to d18/d1a/d5e/d60/d8c/f92 0 2024-07-03T15:44:09.039 INFO:tasks.workunit.client.0.smithi028.stdout:3/476: symlink d18/d6f/l93 0 2024-07-03T15:44:09.039 INFO:tasks.workunit.client.0.smithi028.stdout:3/477: readlink d18/d1a/d1e/d73/l85 0 2024-07-03T15:44:09.040 INFO:tasks.workunit.client.0.smithi028.stdout:3/478: write f6 [1901380,129591] 0 2024-07-03T15:44:09.040 INFO:tasks.workunit.client.0.smithi028.stdout:3/479: readlink d18/d1a/l49 0 2024-07-03T15:44:09.041 INFO:tasks.workunit.client.0.smithi028.stdout:0/511: dread d2/d3/f73 [0,4194304] 0 2024-07-03T15:44:09.043 INFO:tasks.workunit.client.0.smithi028.stdout:0/512: mkdir d2/d6d/d92/d58/d82/d9b 0 2024-07-03T15:44:09.048 INFO:tasks.workunit.client.0.smithi028.stdout:3/480: dread f6 [0,4194304] 0 2024-07-03T15:44:09.048 INFO:tasks.workunit.client.0.smithi028.stdout:3/481: readlink l12 0 2024-07-03T15:44:09.048 INFO:tasks.workunit.client.0.smithi028.stdout:3/482: readlink d18/d1a/l21 0 2024-07-03T15:44:09.050 INFO:tasks.workunit.client.0.smithi028.stdout:3/483: symlink d18/d6f/l94 0 2024-07-03T15:44:09.050 INFO:tasks.workunit.client.0.smithi028.stdout:3/484: creat d18/d1a/d5e/d66/f95 x:0 0 0 2024-07-03T15:44:09.052 INFO:tasks.workunit.client.0.smithi028.stdout:0/513: write d2/f15 [1863254,34183] 0 2024-07-03T15:44:09.053 INFO:tasks.workunit.client.0.smithi028.stdout:3/485: creat d18/d1a/d1e/d58/f96 x:0 0 0 2024-07-03T15:44:09.053 INFO:tasks.workunit.client.0.smithi028.stdout:3/486: dread - d18/d1a/d1e/d58/f96 zero size 2024-07-03T15:44:09.056 INFO:tasks.workunit.client.0.smithi028.stdout:0/514: symlink d2/d3/d4d/d56/l9c 0 2024-07-03T15:44:09.058 INFO:tasks.workunit.client.0.smithi028.stdout:3/487: mknod d18/d1a/d1e/d73/c97 0 2024-07-03T15:44:09.060 INFO:tasks.workunit.client.0.smithi028.stdout:0/515: truncate d2/d6d/d92/d36/d45/f99 253354 0 2024-07-03T15:44:09.062 INFO:tasks.workunit.client.0.smithi028.stdout:3/488: getdents d18/d3a 0 2024-07-03T15:44:09.071 INFO:tasks.workunit.client.0.smithi028.stdout:0/516: dread d2/d6d/d92/d58/d6e/f80 [0,4194304] 0 2024-07-03T15:44:09.071 INFO:tasks.workunit.client.0.smithi028.stdout:0/517: chown d2/d3/d4d/f57 1 1 2024-07-03T15:44:09.072 INFO:tasks.workunit.client.0.smithi028.stdout:0/518: mknod d2/d6d/d92/d58/d82/c9d 0 2024-07-03T15:44:09.074 INFO:tasks.workunit.client.0.smithi028.stdout:0/519: link d2/d6d/d92/d36/f7a d2/d3/d4d/f9e 0 2024-07-03T15:44:09.077 INFO:tasks.workunit.client.0.smithi028.stdout:0/520: rename d2/d6d/d92/d36/d4a to d2/d3/d4d/d56/d78/d9f 0 2024-07-03T15:44:09.077 INFO:tasks.workunit.client.0.smithi028.stdout:0/521: truncate d2/d6d/d92/d36/f61 1740874 0 2024-07-03T15:44:09.078 INFO:tasks.workunit.client.0.smithi028.stdout:3/489: dwrite d18/d1a/f20 [4194304,4194304] 0 2024-07-03T15:44:09.087 INFO:tasks.workunit.client.0.smithi028.stdout:3/490: rename d18/c44 to d18/d1a/d1e/d73/c98 0 2024-07-03T15:44:09.090 INFO:tasks.workunit.client.0.smithi028.stdout:0/522: getdents d2/d3/d4d/d56/d78 0 2024-07-03T15:44:09.091 INFO:tasks.workunit.client.0.smithi028.stdout:3/491: symlink d18/d6f/d78/d87/l99 0 2024-07-03T15:44:09.093 INFO:tasks.workunit.client.0.smithi028.stdout:0/523: stat d2/d3/c10 0 2024-07-03T15:44:09.097 INFO:tasks.workunit.client.0.smithi028.stdout:0/524: rename d2/d6d/d92/d36/l6b to d2/d6d/d92/d36/d45/la0 0 2024-07-03T15:44:09.112 INFO:tasks.workunit.client.0.smithi028.stdout:3/492: dwrite d18/d1a/d5e/d60/d8c/f92 [0,4194304] 0 2024-07-03T15:44:09.113 INFO:tasks.workunit.client.0.smithi028.stdout:0/525: dwrite d2/d6d/d92/d36/f9a [0,4194304] 0 2024-07-03T15:44:09.113 INFO:tasks.workunit.client.0.smithi028.stdout:0/526: write d2/d6d/d92/d36/f7a [340214,92787] 0 2024-07-03T15:44:09.113 INFO:tasks.workunit.client.0.smithi028.stdout:3/493: mkdir d18/d3a/d9a 0 2024-07-03T15:44:09.125 INFO:tasks.workunit.client.0.smithi028.stdout:0/527: dread d2/d1c/f2b [0,4194304] 0 2024-07-03T15:44:09.125 INFO:tasks.workunit.client.0.smithi028.stdout:0/528: read d2/d3/d4d/f64 [20972,120897] 0 2024-07-03T15:44:09.130 INFO:tasks.workunit.client.0.smithi028.stdout:0/529: creat d2/d3/d4d/d56/d78/fa1 x:0 0 0 2024-07-03T15:44:09.153 INFO:tasks.workunit.client.0.smithi028.stdout:0/530: dwrite d2/d1c/d6c/f79 [0,4194304] 0 2024-07-03T15:44:09.153 INFO:tasks.workunit.client.0.smithi028.stdout:0/531: chown d2/d6d/d92/f38 3 1 2024-07-03T15:44:09.153 INFO:tasks.workunit.client.0.smithi028.stdout:0/532: chown d2/d1c/f68 364647 1 2024-07-03T15:44:09.153 INFO:tasks.workunit.client.0.smithi028.stdout:0/533: chown d2/f15 159006539 1 2024-07-03T15:44:09.173 INFO:tasks.workunit.client.0.smithi028.stdout:0/534: dwrite d2/d6d/d92/d36/f61 [0,4194304] 0 2024-07-03T15:44:09.173 INFO:tasks.workunit.client.0.smithi028.stdout:0/535: chown d2/d3/c10 2053 1 2024-07-03T15:44:09.181 INFO:tasks.workunit.client.0.smithi028.stdout:0/536: getdents d2/d3/d4d/d56/d78 0 2024-07-03T15:44:09.183 INFO:tasks.workunit.client.0.smithi028.stdout:0/537: unlink d2/d6d/d92/d36/d45/la0 0 2024-07-03T15:44:09.198 INFO:tasks.workunit.client.0.smithi028.stdout:0/538: dwrite d2/d3/d4d/f57 [0,4194304] 0 2024-07-03T15:44:09.198 INFO:tasks.workunit.client.0.smithi028.stdout:0/539: write d2/d6d/d92/d36/f65 [1401976,16083] 0 2024-07-03T15:44:09.203 INFO:tasks.workunit.client.0.smithi028.stdout:0/540: dread d2/d3/f73 [0,4194304] 0 2024-07-03T15:44:09.216 INFO:tasks.workunit.client.0.smithi028.stdout:0/541: dread d2/d3/f2f [0,4194304] 0 2024-07-03T15:44:09.225 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:08 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:09.225 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:08 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:09.225 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:08 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:09.225 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:08 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:09.229 INFO:tasks.workunit.client.0.smithi028.stdout:9/506: sync 2024-07-03T15:44:09.230 INFO:tasks.workunit.client.0.smithi028.stdout:9/507: fsync d1/dd/f3c 0 2024-07-03T15:44:09.242 INFO:tasks.workunit.client.0.smithi028.stdout:9/508: dread d1/f31 [0,4194304] 0 2024-07-03T15:44:09.242 INFO:tasks.workunit.client.0.smithi028.stdout:9/509: read d1/d16/d57/d60/f6c [842360,108937] 0 2024-07-03T15:44:09.242 INFO:tasks.workunit.client.0.smithi028.stdout:9/510: fdatasync d1/f4b 0 2024-07-03T15:44:09.243 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:08 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:09.243 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:08 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:09.243 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:08 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:09.244 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:08 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:09.335 INFO:tasks.workunit.client.0.smithi028.stdout:9/511: creat d1/d16/d57/d5f/d7f/fad x:0 0 0 2024-07-03T15:44:09.338 INFO:tasks.workunit.client.0.smithi028.stdout:9/512: rename d1/d10/d28/d45/d56/f8a to d1/d10/d76/fae 0 2024-07-03T15:44:09.347 INFO:tasks.workunit.client.0.smithi028.stdout:9/513: write d1/dd/f6f [3488371,93759] 0 2024-07-03T15:44:09.386 INFO:tasks.workunit.client.0.smithi028.stdout:8/537: sync 2024-07-03T15:44:09.386 INFO:tasks.workunit.client.0.smithi028.stdout:8/538: dread - d0/d41/d50/d3/d35/d68/d22/fb1 zero size 2024-07-03T15:44:09.386 INFO:tasks.workunit.client.0.smithi028.stdout:8/539: chown d0/l49 395225111 1 2024-07-03T15:44:09.387 INFO:tasks.workunit.client.0.smithi028.stdout:8/540: rename d0/d41/d50/d3/dc/d3f/d8e/d69/c83 to d0/d41/d50/d3/dc/d3f/d8e/d69/cb5 0 2024-07-03T15:44:09.387 INFO:tasks.workunit.client.0.smithi028.stdout:8/541: fsync d0/d41/d50/d3/dc/d3f/d8e/f6d 0 2024-07-03T15:44:09.387 INFO:tasks.workunit.client.0.smithi028.stdout:8/542: fdatasync d0/d41/d50/d3/dc/d3f/d8e/d5d/d9f/da6/f82 0 2024-07-03T15:44:09.388 INFO:tasks.workunit.client.0.smithi028.stdout:8/543: write d0/d41/d50/f27 [1889288,7695] 0 2024-07-03T15:44:09.390 INFO:tasks.workunit.client.0.smithi028.stdout:8/544: rename d0/d41/d50/d3/dc/d33/d7a/c29 to d0/d41/d50/d3/d35/d68/d22/d40/d62/cb6 0 2024-07-03T15:44:09.464 INFO:tasks.workunit.client.0.smithi028.stdout:8/545: mkdir d0/d41/d50/d3/d35/d68/d22/d40/db7 0 2024-07-03T15:44:09.480 INFO:tasks.workunit.client.0.smithi028.stdout:8/546: dwrite d0/d41/d50/d3/d35/d68/d22/d40/d62/f65 [0,4194304] 0 2024-07-03T15:44:09.483 INFO:tasks.workunit.client.0.smithi028.stdout:8/547: dread d0/f5b [0,4194304] 0 2024-07-03T15:44:09.491 INFO:tasks.workunit.client.0.smithi028.stdout:8/548: dread d0/d41/d50/d3/d35/d86/f87 [0,4194304] 0 2024-07-03T15:44:09.491 INFO:tasks.workunit.client.0.smithi028.stdout:8/549: chown d0/d41/d50/d3/d35/d68/d22/d40/d45/c54 44184278 1 2024-07-03T15:44:09.491 INFO:tasks.workunit.client.0.smithi028.stdout:8/550: readlink d0/d41/d50/le 0 2024-07-03T15:44:09.491 INFO:tasks.workunit.client.0.smithi028.stdout:8/551: write d0/d41/d50/d3/dc/d3f/d8e/d5d/d9f/da6/f9c [1340752,41342] 0 2024-07-03T15:44:09.513 INFO:tasks.workunit.client.0.smithi028.stdout:8/552: creat d0/d41/d50/d3/d35/d68/fb8 x:0 0 0 2024-07-03T15:44:09.514 INFO:tasks.workunit.client.0.smithi028.stdout:8/553: fsync d0/d41/d50/d3/dc/f23 0 2024-07-03T15:44:09.516 INFO:tasks.workunit.client.0.smithi028.stdout:8/554: truncate d0/d41/d50/d3/dc/d33/d7a/f2f 7174796 0 2024-07-03T15:44:09.516 INFO:tasks.workunit.client.0.smithi028.stdout:8/555: chown d0/d41/d50/d3/dc/d33/d7a/l5e 10204483 1 2024-07-03T15:44:09.533 INFO:tasks.workunit.client.0.smithi028.stdout:8/556: dwrite d0/d41/d50/d3/f3a [0,4194304] 0 2024-07-03T15:44:09.533 INFO:tasks.workunit.client.0.smithi028.stdout:8/557: dread - d0/d41/f88 zero size 2024-07-03T15:44:09.536 INFO:tasks.workunit.client.0.smithi028.stdout:8/558: dread d0/d41/d50/d3/dc/f23 [0,4194304] 0 2024-07-03T15:44:09.540 INFO:tasks.workunit.client.0.smithi028.stdout:8/559: rename d0/d41/d50/d3/dc/d3f/d8e/d5d/f72 to d0/d41/d50/d3/dc/d3f/d8e/d8d/fb9 0 2024-07-03T15:44:09.543 INFO:tasks.workunit.client.0.smithi028.stdout:8/560: mknod d0/d6c/da9/cba 0 2024-07-03T15:44:09.543 INFO:tasks.workunit.client.0.smithi028.stdout:8/561: dread - d0/d41/d50/d3/dc/d3f/f61 zero size 2024-07-03T15:44:09.547 INFO:tasks.workunit.client.0.smithi028.stdout:8/562: creat d0/d41/d50/d3/d35/d68/d22/d40/d45/fbb x:0 0 0 2024-07-03T15:44:09.552 INFO:tasks.workunit.client.0.smithi028.stdout:4/542: sync 2024-07-03T15:44:09.553 INFO:tasks.workunit.client.0.smithi028.stdout:8/563: rename d0/d41/d50/d3/c67 to d0/d41/d50/d3/dc/d3f/d8e/d8d/cbc 0 2024-07-03T15:44:09.569 INFO:tasks.workunit.client.0.smithi028.stdout:4/543: creat d2/d61/d9d/faf x:0 0 0 2024-07-03T15:44:09.572 INFO:tasks.workunit.client.0.smithi028.stdout:8/564: creat d0/d41/d50/d3/dc/d3f/d8e/d81/fbd x:0 0 0 2024-07-03T15:44:09.573 INFO:tasks.workunit.client.0.smithi028.stdout:4/544: mknod d2/d61/cb0 0 2024-07-03T15:44:09.575 INFO:tasks.workunit.client.0.smithi028.stdout:8/565: mkdir d0/d41/d50/d3/dc/d33/dbe 0 2024-07-03T15:44:09.575 INFO:tasks.workunit.client.0.smithi028.stdout:8/566: readlink d0/l85 0 2024-07-03T15:44:09.577 INFO:tasks.workunit.client.0.smithi028.stdout:4/545: rename d2/d12/f26 to d2/d12/d2e/fb1 0 2024-07-03T15:44:09.578 INFO:tasks.workunit.client.0.smithi028.stdout:8/567: creat d0/d41/d50/fbf x:0 0 0 2024-07-03T15:44:09.582 INFO:tasks.workunit.client.0.smithi028.stdout:4/546: write d2/d5a/f1f [4459201,1670] 0 2024-07-03T15:44:09.582 INFO:tasks.workunit.client.0.smithi028.stdout:4/547: readlink d2/l1d 0 2024-07-03T15:44:09.582 INFO:tasks.workunit.client.0.smithi028.stdout:8/568: mkdir d0/d41/d50/d3/dc/d3f/d8e/d4c/dc0 0 2024-07-03T15:44:09.582 INFO:tasks.workunit.client.0.smithi028.stdout:8/569: write d0/d41/d50/f27 [2340922,37207] 0 2024-07-03T15:44:09.582 INFO:tasks.workunit.client.0.smithi028.stdout:8/570: write d0/d41/d50/f27 [4306332,102701] 0 2024-07-03T15:44:09.584 INFO:tasks.workunit.client.0.smithi028.stdout:4/548: creat d2/d12/d5b/d98/fb2 x:0 0 0 2024-07-03T15:44:09.584 INFO:tasks.workunit.client.0.smithi028.stdout:4/549: chown d2/d12/d2e/c30 5257008 1 2024-07-03T15:44:09.584 INFO:tasks.workunit.client.0.smithi028.stdout:4/550: truncate d2/d5a/f94 4504221 0 2024-07-03T15:44:09.587 INFO:tasks.workunit.client.0.smithi028.stdout:8/571: mkdir d0/d41/d50/daf/dc1 0 2024-07-03T15:44:09.587 INFO:tasks.workunit.client.0.smithi028.stdout:8/572: fsync d0/d41/d50/d3/dc/d33/d7a/f8b 0 2024-07-03T15:44:09.589 INFO:tasks.workunit.client.0.smithi028.stdout:4/551: creat d2/d5a/d45/fb3 x:0 0 0 2024-07-03T15:44:09.592 INFO:tasks.workunit.client.0.smithi028.stdout:8/573: truncate d0/f44 730605 0 2024-07-03T15:44:09.592 INFO:tasks.workunit.client.0.smithi028.stdout:8/574: chown d0/d41/d50/c10 21 1 2024-07-03T15:44:09.607 INFO:tasks.workunit.client.0.smithi028.stdout:4/552: dwrite d2/d12/d5b/f9b [0,4194304] 0 2024-07-03T15:44:09.607 INFO:tasks.workunit.client.0.smithi028.stdout:4/553: fdatasync d2/d12/d2c/f7d 0 2024-07-03T15:44:09.608 INFO:tasks.workunit.client.0.smithi028.stdout:1/616: sync 2024-07-03T15:44:09.608 INFO:tasks.workunit.client.0.smithi028.stdout:5/528: sync 2024-07-03T15:44:09.608 INFO:tasks.workunit.client.0.smithi028.stdout:2/443: sync 2024-07-03T15:44:09.612 INFO:tasks.workunit.client.0.smithi028.stdout:2/444: link da/d23/f96 da/d24/fa4 0 2024-07-03T15:44:09.620 INFO:tasks.workunit.client.0.smithi028.stdout:1/617: creat d4/d2c/d3a/d5c/d6c/d7d/d59/dc6/fdd x:0 0 0 2024-07-03T15:44:09.620 INFO:tasks.workunit.client.0.smithi028.stdout:1/618: truncate d4/d83/fac 18589 0 2024-07-03T15:44:09.621 INFO:tasks.workunit.client.0.smithi028.stdout:1/619: fsync d4/d3c/d54/d79/d16/fb3 0 2024-07-03T15:44:09.621 INFO:tasks.workunit.client.0.smithi028.stdout:1/620: dread - d4/d3c/d54/d79/d16/d7c/fcc zero size 2024-07-03T15:44:09.621 INFO:tasks.workunit.client.0.smithi028.stdout:1/621: write d4/d83/f8b [485600,116716] 0 2024-07-03T15:44:09.621 INFO:tasks.workunit.client.0.smithi028.stdout:4/554: getdents d2/d5a 0 2024-07-03T15:44:09.624 INFO:tasks.workunit.client.0.smithi028.stdout:5/529: getdents d0/dd/d1e/d8e 0 2024-07-03T15:44:09.625 INFO:tasks.workunit.client.0.smithi028.stdout:2/445: dwrite da/d42/d47/f6d [0,4194304] 0 2024-07-03T15:44:09.625 INFO:tasks.workunit.client.0.smithi028.stdout:2/446: chown da/f77 2772340 1 2024-07-03T15:44:09.625 INFO:tasks.workunit.client.0.smithi028.stdout:2/447: fdatasync da/d42/d47/d78/fa3 0 2024-07-03T15:44:09.628 INFO:tasks.workunit.client.0.smithi028.stdout:4/555: rename d2/d12/d3b/f48 to d2/d3e/d9f/fb4 0 2024-07-03T15:44:09.629 INFO:tasks.workunit.client.0.smithi028.stdout:2/448: creat da/d42/d47/fa5 x:0 0 0 2024-07-03T15:44:09.629 INFO:tasks.workunit.client.0.smithi028.stdout:1/622: write d4/d3c/d54/f8d [2272836,117619] 0 2024-07-03T15:44:09.633 INFO:tasks.workunit.client.0.smithi028.stdout:1/623: fdatasync d4/d3c/d54/f8d 0 2024-07-03T15:44:09.633 INFO:tasks.workunit.client.0.smithi028.stdout:1/624: chown d4/d2c/d3a/d5c/dd4 454 1 2024-07-03T15:44:09.637 INFO:tasks.workunit.client.0.smithi028.stdout:4/556: mknod d2/d12/d42/d63/d9e/cb5 0 2024-07-03T15:44:09.639 INFO:tasks.workunit.client.0.smithi028.stdout:2/449: creat da/d42/fa6 x:0 0 0 2024-07-03T15:44:09.641 INFO:tasks.workunit.client.0.smithi028.stdout:5/530: dwrite d0/dd/d13/d47/f5b [0,4194304] 0 2024-07-03T15:44:09.641 INFO:tasks.workunit.client.0.smithi028.stdout:1/625: link d4/de/da7/fc0 d4/de/d1c/d44/fde 0 2024-07-03T15:44:09.644 INFO:tasks.workunit.client.0.smithi028.stdout:4/557: dread d2/d12/f17 [0,4194304] 0 2024-07-03T15:44:09.644 INFO:tasks.workunit.client.0.smithi028.stdout:5/531: dread d0/dd/d13/d57/d26/d41/f5f [0,4194304] 0 2024-07-03T15:44:09.644 INFO:tasks.workunit.client.0.smithi028.stdout:5/532: readlink d0/dd/df/l66 0 2024-07-03T15:44:09.645 INFO:tasks.workunit.client.0.smithi028.stdout:5/533: write d0/dd/d13/d57/d77/f7d [1764167,9294] 0 2024-07-03T15:44:09.645 INFO:tasks.workunit.client.0.smithi028.stdout:4/558: symlink d2/d61/d9c/lb6 0 2024-07-03T15:44:09.646 INFO:tasks.workunit.client.0.smithi028.stdout:5/534: creat d0/dd/df/faf x:0 0 0 2024-07-03T15:44:09.648 INFO:tasks.workunit.client.0.smithi028.stdout:4/559: mknod d2/d12/d42/d63/d9e/cb7 0 2024-07-03T15:44:09.648 INFO:tasks.workunit.client.0.smithi028.stdout:4/560: readlink d2/l16 0 2024-07-03T15:44:09.648 INFO:tasks.workunit.client.0.smithi028.stdout:4/561: write d2/d12/d5b/f85 [1801886,1117] 0 2024-07-03T15:44:09.648 INFO:tasks.workunit.client.0.smithi028.stdout:4/562: write d2/d3e/f3f [1087643,53686] 0 2024-07-03T15:44:09.652 INFO:tasks.workunit.client.0.smithi028.stdout:5/535: mkdir d0/dd/d13/d57/d26/d41/db0 0 2024-07-03T15:44:09.654 INFO:tasks.workunit.client.0.smithi028.stdout:5/536: symlink d0/d8f/lb1 0 2024-07-03T15:44:09.654 INFO:tasks.workunit.client.0.smithi028.stdout:5/537: write d0/dd/d1e/f89 [867208,75428] 0 2024-07-03T15:44:09.654 INFO:tasks.workunit.client.0.smithi028.stdout:1/626: dread d4/d2c/d3a/f68 [0,4194304] 0 2024-07-03T15:44:09.656 INFO:tasks.workunit.client.0.smithi028.stdout:5/538: mknod d0/dd/d1e/d3b/d3f/d71/cb2 0 2024-07-03T15:44:09.656 INFO:tasks.workunit.client.0.smithi028.stdout:5/539: stat d0/dd/d1e/d3b/d3f/d5a/f65 0 2024-07-03T15:44:09.657 INFO:tasks.workunit.client.0.smithi028.stdout:5/540: creat d0/dd/d1e/d8e/fb3 x:0 0 0 2024-07-03T15:44:09.657 INFO:tasks.workunit.client.0.smithi028.stdout:5/541: chown d0/dd/d13/d47/l63 0 1 2024-07-03T15:44:09.658 INFO:tasks.workunit.client.0.smithi028.stdout:2/450: dwrite da/d1e/d56/f9a [0,4194304] 0 2024-07-03T15:44:09.658 INFO:tasks.workunit.client.0.smithi028.stdout:2/451: chown da/d23/l53 2149680 1 2024-07-03T15:44:09.662 INFO:tasks.workunit.client.0.smithi028.stdout:2/452: creat da/d23/d6f/d84/fa7 x:0 0 0 2024-07-03T15:44:09.662 INFO:tasks.workunit.client.0.smithi028.stdout:1/627: dread d4/d3c/f9e [0,4194304] 0 2024-07-03T15:44:09.668 INFO:tasks.workunit.client.0.smithi028.stdout:5/542: read d0/dd/d13/d29/f35 [1053729,88201] 0 2024-07-03T15:44:09.669 INFO:tasks.workunit.client.0.smithi028.stdout:4/563: dwrite d2/d5a/f94 [0,4194304] 0 2024-07-03T15:44:09.669 INFO:tasks.workunit.client.0.smithi028.stdout:4/564: chown d2/d5a/d45/fb3 1063949816 1 2024-07-03T15:44:09.673 INFO:tasks.workunit.client.0.smithi028.stdout:5/543: unlink d0/c9b 0 2024-07-03T15:44:09.674 INFO:tasks.workunit.client.0.smithi028.stdout:4/565: symlink d2/d3e/d9f/lb8 0 2024-07-03T15:44:09.674 INFO:tasks.workunit.client.0.smithi028.stdout:4/566: chown d2/d3e/f50 29 1 2024-07-03T15:44:09.677 INFO:tasks.workunit.client.0.smithi028.stdout:4/567: mkdir d2/d12/d2d/db9 0 2024-07-03T15:44:09.677 INFO:tasks.workunit.client.0.smithi028.stdout:4/568: write d2/f52 [5593979,27833] 0 2024-07-03T15:44:09.678 INFO:tasks.workunit.client.0.smithi028.stdout:4/569: rmdir d2/d5a 39 2024-07-03T15:44:09.679 INFO:tasks.workunit.client.0.smithi028.stdout:4/570: truncate d2/f1b 3649483 0 2024-07-03T15:44:09.681 INFO:tasks.workunit.client.0.smithi028.stdout:2/453: dwrite da/d42/d47/f6d [0,4194304] 0 2024-07-03T15:44:09.681 INFO:tasks.workunit.client.0.smithi028.stdout:2/454: fsync da/d1e/f58 0 2024-07-03T15:44:09.682 INFO:tasks.workunit.client.0.smithi028.stdout:4/571: dread d2/d12/d42/d63/d75/fa9 [0,4194304] 0 2024-07-03T15:44:09.682 INFO:tasks.workunit.client.0.smithi028.stdout:1/628: dwrite d4/d3c/d54/d79/d16/d7c/dd5/fdc [0,4194304] 0 2024-07-03T15:44:09.688 INFO:tasks.workunit.client.0.smithi028.stdout:1/629: mknod d4/d2c/cdf 0 2024-07-03T15:44:09.696 INFO:tasks.workunit.client.0.smithi028.stdout:4/572: dwrite d2/d12/d42/d63/d97/f9a [0,4194304] 0 2024-07-03T15:44:09.697 INFO:tasks.workunit.client.0.smithi028.stdout:4/573: mknod d2/d3e/dae/cba 0 2024-07-03T15:44:09.697 INFO:tasks.workunit.client.0.smithi028.stdout:1/630: dread d4/de/f1f [0,4194304] 0 2024-07-03T15:44:09.699 INFO:tasks.workunit.client.0.smithi028.stdout:1/631: creat d4/d2c/d3a/d5c/d6c/d7d/d4e/d51/fe0 x:0 0 0 2024-07-03T15:44:09.700 INFO:tasks.workunit.client.0.smithi028.stdout:1/632: symlink d4/d2c/le1 0 2024-07-03T15:44:09.718 INFO:tasks.workunit.client.0.smithi028.stdout:4/574: dwrite d2/d12/d2d/d2f/d3a/d6a/f71 [4194304,4194304] 0 2024-07-03T15:44:09.718 INFO:tasks.workunit.client.0.smithi028.stdout:4/575: chown d2/d12/d2d/d2f/l90 3343 1 2024-07-03T15:44:09.748 INFO:tasks.workunit.client.0.smithi028.stdout:6/467: sync 2024-07-03T15:44:09.748 INFO:tasks.workunit.client.0.smithi028.stdout:6/468: truncate d4/d15/f6a 660003 0 2024-07-03T15:44:09.749 INFO:tasks.workunit.client.0.smithi028.stdout:6/469: symlink d4/da/d38/la3 0 2024-07-03T15:44:09.750 INFO:tasks.workunit.client.0.smithi028.stdout:6/470: mkdir d4/da/d13/da4 0 2024-07-03T15:44:09.752 INFO:tasks.workunit.client.0.smithi028.stdout:6/471: link d4/da/d1c/f54 d4/da/d47/d9f/fa5 0 2024-07-03T15:44:09.752 INFO:tasks.workunit.client.0.smithi028.stdout:6/472: write d4/d15/d68/f9c [301576,13244] 0 2024-07-03T15:44:09.753 INFO:tasks.workunit.client.0.smithi028.stdout:6/473: unlink d4/d15/l25 0 2024-07-03T15:44:09.755 INFO:tasks.workunit.client.0.smithi028.stdout:6/474: rename d4/da/d38/f6f to d4/da/fa6 0 2024-07-03T15:44:09.771 INFO:tasks.workunit.client.0.smithi028.stdout:6/475: dwrite d4/da/d47/f6b [0,4194304] 0 2024-07-03T15:44:09.771 INFO:tasks.workunit.client.0.smithi028.stdout:6/476: stat d4/da/d38/l75 0 2024-07-03T15:44:09.771 INFO:tasks.workunit.client.0.smithi028.stdout:6/477: truncate d4/da/d38/d3d/f58 1007658 0 2024-07-03T15:44:09.771 INFO:tasks.workunit.client.0.smithi028.stdout:6/478: stat d4/d15/f2a 0 2024-07-03T15:44:09.790 INFO:tasks.workunit.client.0.smithi028.stdout:6/479: dwrite d4/da/d38/d3d/f58 [0,4194304] 0 2024-07-03T15:44:09.791 INFO:tasks.workunit.client.0.smithi028.stdout:6/480: creat d4/da/d38/fa7 x:0 0 0 2024-07-03T15:44:09.809 INFO:tasks.workunit.client.0.smithi028.stdout:6/481: dwrite d4/da/d38/f4a [0,4194304] 0 2024-07-03T15:44:09.809 INFO:tasks.workunit.client.0.smithi028.stdout:6/482: creat d4/da/fa8 x:0 0 0 2024-07-03T15:44:09.809 INFO:tasks.workunit.client.0.smithi028.stdout:6/483: readlink d4/da/l90 0 2024-07-03T15:44:09.810 INFO:tasks.workunit.client.0.smithi028.stdout:6/484: stat d4/da/d38/fa7 0 2024-07-03T15:44:09.810 INFO:tasks.workunit.client.0.smithi028.stdout:6/485: write d4/f7b [867888,45548] 0 2024-07-03T15:44:09.813 INFO:tasks.workunit.client.0.smithi028.stdout:6/486: creat d4/d12/fa9 x:0 0 0 2024-07-03T15:44:09.817 INFO:tasks.workunit.client.0.smithi028.stdout:6/487: fsync d4/da/fa8 0 2024-07-03T15:44:09.817 INFO:tasks.workunit.client.0.smithi028.stdout:6/488: chown d4/d15/l42 75207108 1 2024-07-03T15:44:09.818 INFO:tasks.workunit.client.0.smithi028.stdout:6/489: creat d4/da/d1c/faa x:0 0 0 2024-07-03T15:44:09.822 INFO:tasks.workunit.client.0.smithi028.stdout:6/490: rename d4/d15/d68/f9c to d4/da/d38/d60/fab 0 2024-07-03T15:44:09.825 INFO:tasks.workunit.client.0.smithi028.stdout:6/491: mkdir d4/d12/d2c/dac 0 2024-07-03T15:44:09.835 INFO:tasks.workunit.client.0.smithi028.stdout:6/492: getdents d4/da/d38 0 2024-07-03T15:44:09.835 INFO:tasks.workunit.client.0.smithi028.stdout:6/493: write d4/da/d38/f5f [688530,108766] 0 2024-07-03T15:44:09.843 INFO:tasks.workunit.client.0.smithi028.stdout:6/494: read d4/da/d47/d9f/fa5 [2896676,103255] 0 2024-07-03T15:44:09.844 INFO:tasks.workunit.client.0.smithi028.stdout:6/495: symlink d4/da/d47/lad 0 2024-07-03T15:44:09.934 INFO:tasks.workunit.client.0.smithi028.stdout:6/496: dwrite d4/da/d1c/f61 [4194304,4194304] 0 2024-07-03T15:44:09.934 INFO:tasks.workunit.client.0.smithi028.stdout:6/497: chown d4/d15/f7f 328614794 1 2024-07-03T15:44:09.934 INFO:tasks.workunit.client.0.smithi028.stdout:6/498: creat d4/d12/fae x:0 0 0 2024-07-03T15:44:09.942 INFO:tasks.workunit.client.0.smithi028.stdout:6/499: mkdir d4/da/d47/d66/daf 0 2024-07-03T15:44:09.942 INFO:tasks.workunit.client.0.smithi028.stdout:6/500: fsync f0 0 2024-07-03T15:44:09.945 INFO:tasks.workunit.client.0.smithi028.stdout:6/501: dread d4/f7b [0,4194304] 0 2024-07-03T15:44:09.972 INFO:tasks.workunit.client.0.smithi028.stdout:7/364: sync 2024-07-03T15:44:09.972 INFO:tasks.workunit.client.0.smithi028.stdout:3/494: sync 2024-07-03T15:44:09.972 INFO:tasks.workunit.client.0.smithi028.stdout:0/542: sync 2024-07-03T15:44:09.976 INFO:tasks.workunit.client.0.smithi028.stdout:3/495: getdents d18/d1a/d1e/d73 0 2024-07-03T15:44:09.981 INFO:tasks.workunit.client.0.smithi028.stdout:0/543: getdents d2/d1c 0 2024-07-03T15:44:09.986 INFO:tasks.workunit.client.0.smithi028.stdout:0/544: creat d2/d3/d4d/fa2 x:0 0 0 2024-07-03T15:44:09.987 INFO:tasks.workunit.client.0.smithi028.stdout:6/502: dwrite d4/d12/f4f [0,4194304] 0 2024-07-03T15:44:09.988 INFO:tasks.workunit.client.0.smithi028.stdout:0/545: mkdir d2/d6d/d92/d36/d45/da3 0 2024-07-03T15:44:09.988 INFO:tasks.workunit.client.0.smithi028.stdout:0/546: chown d2/d1c/d6c/f85 205 1 2024-07-03T15:44:09.991 INFO:tasks.workunit.client.0.smithi028.stdout:6/503: creat d4/da/d1c/fb0 x:0 0 0 2024-07-03T15:44:09.991 INFO:tasks.workunit.client.0.smithi028.stdout:6/504: dread - d4/d12/d2c/d31/f64 zero size 2024-07-03T15:44:10.003 INFO:tasks.workunit.client.0.smithi028.stdout:7/365: dwrite d9/d29/f35 [4194304,4194304] 0 2024-07-03T15:44:10.004 INFO:tasks.workunit.client.0.smithi028.stdout:3/496: dwrite d18/d1a/d1e/f41 [0,4194304] 0 2024-07-03T15:44:10.007 INFO:tasks.workunit.client.0.smithi028.stdout:7/366: truncate d9/d29/f3e 4073027 0 2024-07-03T15:44:10.008 INFO:tasks.workunit.client.0.smithi028.stdout:6/505: dread d4/da/d38/f4a [0,4194304] 0 2024-07-03T15:44:10.010 INFO:tasks.workunit.client.0.smithi028.stdout:0/547: rmdir d2/d3/d4d 39 2024-07-03T15:44:10.010 INFO:tasks.workunit.client.0.smithi028.stdout:0/548: chown d2/d6d/d92/d58/c59 14 1 2024-07-03T15:44:10.017 INFO:tasks.workunit.client.0.smithi028.stdout:0/549: rmdir d2/d3/d4d/d56/d78/d9f 39 2024-07-03T15:44:10.020 INFO:tasks.workunit.client.0.smithi028.stdout:3/497: getdents d18/d1a/d5e/d66 0 2024-07-03T15:44:10.020 INFO:tasks.workunit.client.0.smithi028.stdout:3/498: fsync d18/d1a/d5e/d60/f8e 0 2024-07-03T15:44:10.022 INFO:tasks.workunit.client.0.smithi028.stdout:6/506: link d4/d15/f2a d4/da/d1c/fb1 0 2024-07-03T15:44:10.025 INFO:tasks.workunit.client.0.smithi028.stdout:7/367: dwrite d9/d1c/d52/d6a/f6c [0,4194304] 0 2024-07-03T15:44:10.025 INFO:tasks.workunit.client.0.smithi028.stdout:7/368: write d9/d29/f61 [543752,38179] 0 2024-07-03T15:44:10.028 INFO:tasks.workunit.client.0.smithi028.stdout:0/550: rename d2/d3/f7 to d2/d6d/d92/d58/fa4 0 2024-07-03T15:44:10.029 INFO:tasks.workunit.client.0.smithi028.stdout:0/551: dread d2/d3/d4d/d56/d78/d9f/f6f [0,4194304] 0 2024-07-03T15:44:10.029 INFO:tasks.workunit.client.0.smithi028.stdout:0/552: chown d2/d1c/l30 1318708971 1 2024-07-03T15:44:10.033 INFO:tasks.workunit.client.0.smithi028.stdout:3/499: truncate d18/f2d 2442401 0 2024-07-03T15:44:10.034 INFO:tasks.workunit.client.0.smithi028.stdout:6/507: link d4/d15/f6a d4/d15/fb2 0 2024-07-03T15:44:10.035 INFO:tasks.workunit.client.0.smithi028.stdout:7/369: symlink d9/d1c/d52/d6a/l6f 0 2024-07-03T15:44:10.040 INFO:tasks.workunit.client.0.smithi028.stdout:7/370: mknod d9/d1c/d6e/c70 0 2024-07-03T15:44:10.040 INFO:tasks.workunit.client.0.smithi028.stdout:7/371: chown d9/d1c/l54 12 1 2024-07-03T15:44:10.041 INFO:tasks.workunit.client.0.smithi028.stdout:3/500: dread d18/d1a/f4f [4194304,4194304] 0 2024-07-03T15:44:10.060 INFO:tasks.workunit.client.0.smithi028.stdout:0/553: dwrite d2/d6d/d92/d36/d45/f46 [4194304,4194304] 0 2024-07-03T15:44:10.072 INFO:tasks.workunit.client.0.smithi028.stdout:6/508: dwrite f0 [0,4194304] 0 2024-07-03T15:44:10.073 INFO:tasks.workunit.client.0.smithi028.stdout:7/372: dwrite d9/f41 [0,4194304] 0 2024-07-03T15:44:10.081 INFO:tasks.workunit.client.0.smithi028.stdout:0/554: dwrite d2/d1c/f41 [0,4194304] 0 2024-07-03T15:44:10.082 INFO:tasks.workunit.client.0.smithi028.stdout:0/555: chown d2/d6d/d92/d58/d82/c9d 22 1 2024-07-03T15:44:10.131 INFO:tasks.workunit.client.0.smithi028.stdout:7/373: mkdir d9/d1c/d45/d71 0 2024-07-03T15:44:10.131 INFO:tasks.workunit.client.0.smithi028.stdout:7/374: fdatasync f0 0 2024-07-03T15:44:10.131 INFO:tasks.workunit.client.0.smithi028.stdout:7/375: creat d9/d29/f72 x:0 0 0 2024-07-03T15:44:10.229 INFO:tasks.workunit.client.0.smithi028.stdout:7/376: rmdir d9 39 2024-07-03T15:44:10.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:09 smithi148 ceph-mon[36056]: pgmap v6: 65 pgs: 65 active+clean; 2.8 GiB data, 9.5 GiB used, 527 GiB / 536 GiB avail; 29 MiB/s rd, 36 MiB/s wr, 187 op/s 2024-07-03T15:44:10.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:09 smithi148 ceph-mon[36056]: Standby manager daemon smithi028.vtqrtz started 2024-07-03T15:44:10.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:09 smithi148 ceph-mon[36056]: from='mgr.? 172.21.15.28:0/1973941006' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi028.vtqrtz/crt"}]: dispatch 2024-07-03T15:44:10.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:09 smithi148 ceph-mon[36056]: from='mgr.? 172.21.15.28:0/1973941006' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-03T15:44:10.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:09 smithi148 ceph-mon[36056]: from='mgr.? 172.21.15.28:0/1973941006' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi028.vtqrtz/key"}]: dispatch 2024-07-03T15:44:10.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:09 smithi148 ceph-mon[36056]: from='mgr.? 172.21.15.28:0/1973941006' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-03T15:44:10.236 INFO:tasks.workunit.client.0.smithi028.stdout:6/509: rmdir d4/d12/d2c/d31 39 2024-07-03T15:44:10.236 INFO:tasks.workunit.client.0.smithi028.stdout:6/510: readlink d4/d15/l5d 0 2024-07-03T15:44:10.238 INFO:tasks.workunit.client.0.smithi028.stdout:6/511: creat d4/da/d38/d60/fb3 x:0 0 0 2024-07-03T15:44:10.241 INFO:tasks.workunit.client.0.smithi028.stdout:6/512: symlink d4/da/d47/lb4 0 2024-07-03T15:44:10.245 INFO:tasks.workunit.client.0.smithi028.stdout:7/377: getdents d9/d1c/d45 0 2024-07-03T15:44:10.279 INFO:tasks.workunit.client.0.smithi028.stdout:7/378: dwrite d9/f69 [0,4194304] 0 2024-07-03T15:44:10.279 INFO:tasks.workunit.client.0.smithi028.stdout:6/513: dwrite d4/da/d38/f40 [0,4194304] 0 2024-07-03T15:44:10.279 INFO:tasks.workunit.client.0.smithi028.stdout:7/379: chown d9/d29/c47 12600 1 2024-07-03T15:44:10.280 INFO:tasks.workunit.client.0.smithi028.stdout:7/380: fsync d9/d1c/d52/f58 0 2024-07-03T15:44:10.300 INFO:tasks.workunit.client.0.smithi028.stdout:6/514: dread d4/d15/f20 [0,4194304] 0 2024-07-03T15:44:10.301 INFO:tasks.workunit.client.0.smithi028.stdout:6/515: mkdir d4/da/d38/d60/db5 0 2024-07-03T15:44:10.302 INFO:tasks.workunit.client.0.smithi028.stdout:6/516: mknod d4/da/d47/d9f/cb6 0 2024-07-03T15:44:10.302 INFO:tasks.workunit.client.0.smithi028.stdout:6/517: readlink d4/da/d47/lb4 0 2024-07-03T15:44:10.302 INFO:tasks.workunit.client.0.smithi028.stdout:6/518: fdatasync d4/da/f7c 0 2024-07-03T15:44:10.302 INFO:tasks.workunit.client.0.smithi028.stdout:6/519: creat d4/d15/d9d/fb7 x:0 0 0 2024-07-03T15:44:10.302 INFO:tasks.workunit.client.0.smithi028.stdout:6/520: readlink d4/d12/d2c/d31/l4b 0 2024-07-03T15:44:10.303 INFO:tasks.workunit.client.0.smithi028.stdout:6/521: symlink d4/da/d38/lb8 0 2024-07-03T15:44:10.303 INFO:tasks.workunit.client.0.smithi028.stdout:6/522: mkdir d4/da/d56/db9 0 2024-07-03T15:44:10.305 INFO:tasks.workunit.client.0.smithi028.stdout:6/523: dread d4/da/d38/d60/fab [0,4194304] 0 2024-07-03T15:44:10.306 INFO:tasks.workunit.client.0.smithi028.stdout:6/524: link d4/f11 d4/d15/fba 0 2024-07-03T15:44:10.306 INFO:tasks.workunit.client.0.smithi028.stdout:6/525: symlink d4/d12/d2c/dac/lbb 0 2024-07-03T15:44:10.312 INFO:tasks.workunit.client.0.smithi028.stdout:0/556: link d2/d6d/d92/l97 d2/d1c/la5 0 2024-07-03T15:44:10.315 INFO:tasks.workunit.client.0.smithi028.stdout:0/557: unlink d2/d3/l13 0 2024-07-03T15:44:10.317 INFO:tasks.workunit.client.0.smithi028.stdout:0/558: unlink d2/d3/d4d/d56/d78/f91 0 2024-07-03T15:44:10.317 INFO:tasks.workunit.client.0.smithi028.stdout:0/559: fdatasync d2/d3/f2f 0 2024-07-03T15:44:10.318 INFO:tasks.workunit.client.0.smithi028.stdout:0/560: symlink d2/d6d/la6 0 2024-07-03T15:44:10.319 INFO:tasks.workunit.client.0.smithi028.stdout:0/561: stat d2/d6d/d92/d58/d6e 0 2024-07-03T15:44:10.319 INFO:tasks.workunit.client.0.smithi028.stdout:7/381: dwrite d9/fa [0,4194304] 0 2024-07-03T15:44:10.321 INFO:tasks.workunit.client.0.smithi028.stdout:7/382: creat d9/d1c/d6e/f73 x:0 0 0 2024-07-03T15:44:10.327 INFO:tasks.workunit.client.0.smithi028.stdout:6/526: dwrite d4/f77 [0,4194304] 0 2024-07-03T15:44:10.327 INFO:tasks.workunit.client.0.smithi028.stdout:6/527: chown d4/da/fa6 50299 1 2024-07-03T15:44:10.333 INFO:tasks.workunit.client.0.smithi028.stdout:6/528: unlink d4/d12/d2c/d31/c84 0 2024-07-03T15:44:10.334 INFO:tasks.workunit.client.0.smithi028.stdout:6/529: write d4/da/d47/f6b [5091859,54299] 0 2024-07-03T15:44:10.336 INFO:tasks.workunit.client.0.smithi028.stdout:6/530: mkdir d4/da/d38/d3d/dbc 0 2024-07-03T15:44:10.338 INFO:tasks.workunit.client.0.smithi028.stdout:6/531: rename d4/da/d56/db9 to d4/da/d13/dbd 0 2024-07-03T15:44:10.338 INFO:tasks.workunit.client.0.smithi028.stdout:6/532: readlink d4/da/d1c/l7e 0 2024-07-03T15:44:10.347 INFO:tasks.workunit.client.0.smithi028.stdout:0/562: dwrite d2/d6d/d92/d36/d45/f7b [0,4194304] 0 2024-07-03T15:44:10.348 INFO:tasks.workunit.client.0.smithi028.stdout:7/383: dwrite d9/f48 [0,4194304] 0 2024-07-03T15:44:10.360 INFO:tasks.workunit.client.0.smithi028.stdout:0/563: write d2/d3/f2f [2003452,45913] 0 2024-07-03T15:44:10.360 INFO:tasks.workunit.client.0.smithi028.stdout:0/564: write d2/d1c/d6c/f79 [5019514,1076] 0 2024-07-03T15:44:10.363 INFO:tasks.workunit.client.0.smithi028.stdout:0/565: dread d2/d6d/d92/d36/f65 [0,4194304] 0 2024-07-03T15:44:10.365 INFO:tasks.workunit.client.0.smithi028.stdout:6/533: dwrite d4/da/f7c [0,4194304] 0 2024-07-03T15:44:10.365 INFO:tasks.workunit.client.0.smithi028.stdout:0/566: unlink d2/d3/d4d/d56/d78/d9f/c95 0 2024-07-03T15:44:10.365 INFO:tasks.workunit.client.0.smithi028.stdout:7/384: dwrite d9/d29/f44 [0,4194304] 0 2024-07-03T15:44:10.366 INFO:tasks.workunit.client.0.smithi028.stdout:6/534: mknod d4/da/d38/d3d/cbe 0 2024-07-03T15:44:10.367 INFO:tasks.workunit.client.0.smithi028.stdout:0/567: mknod d2/d1c/d6c/ca7 0 2024-07-03T15:44:10.368 INFO:tasks.workunit.client.0.smithi028.stdout:0/568: dread d2/d6d/d92/d36/d45/f99 [0,4194304] 0 2024-07-03T15:44:10.371 INFO:tasks.workunit.client.0.smithi028.stdout:6/535: rename d4/d15/l42 to d4/da/d38/d60/lbf 0 2024-07-03T15:44:10.371 INFO:tasks.workunit.client.0.smithi028.stdout:6/536: truncate d4/d15/f4c 597156 0 2024-07-03T15:44:10.371 INFO:tasks.workunit.client.0.smithi028.stdout:6/537: chown d4/ce 7 1 2024-07-03T15:44:10.372 INFO:tasks.workunit.client.0.smithi028.stdout:6/538: fdatasync d4/d12/f36 0 2024-07-03T15:44:10.372 INFO:tasks.workunit.client.0.smithi028.stdout:6/539: dread - d4/da/d1c/fb0 zero size 2024-07-03T15:44:10.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:09 smithi028 ceph-mon[29148]: pgmap v6: 65 pgs: 65 active+clean; 2.8 GiB data, 9.5 GiB used, 527 GiB / 536 GiB avail; 29 MiB/s rd, 36 MiB/s wr, 187 op/s 2024-07-03T15:44:10.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:09 smithi028 ceph-mon[29148]: Standby manager daemon smithi028.vtqrtz started 2024-07-03T15:44:10.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:09 smithi028 ceph-mon[29148]: from='mgr.? 172.21.15.28:0/1973941006' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi028.vtqrtz/crt"}]: dispatch 2024-07-03T15:44:10.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:09 smithi028 ceph-mon[29148]: from='mgr.? 172.21.15.28:0/1973941006' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-03T15:44:10.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:09 smithi028 ceph-mon[29148]: from='mgr.? 172.21.15.28:0/1973941006' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi028.vtqrtz/key"}]: dispatch 2024-07-03T15:44:10.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:09 smithi028 ceph-mon[29148]: from='mgr.? 172.21.15.28:0/1973941006' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-03T15:44:10.375 INFO:tasks.workunit.client.0.smithi028.stdout:6/540: dread d4/da/d1c/f50 [0,4194304] 0 2024-07-03T15:44:10.375 INFO:tasks.workunit.client.0.smithi028.stdout:6/541: fsync d4/d15/f2a 0 2024-07-03T15:44:10.380 INFO:tasks.workunit.client.0.smithi028.stdout:7/385: dread d9/d29/f35 [4194304,4194304] 0 2024-07-03T15:44:10.383 INFO:tasks.workunit.client.0.smithi028.stdout:6/542: write d4/da/d1c/f54 [5501099,84164] 0 2024-07-03T15:44:10.385 INFO:tasks.workunit.client.0.smithi028.stdout:7/386: rename d9/l14 to d9/d1c/d52/l74 0 2024-07-03T15:44:10.387 INFO:tasks.workunit.client.0.smithi028.stdout:0/569: dwrite d2/d6d/d92/d58/d6e/f94 [0,4194304] 0 2024-07-03T15:44:10.387 INFO:tasks.workunit.client.0.smithi028.stdout:0/570: write d2/d6d/d92/d36/d45/f72 [716565,42564] 0 2024-07-03T15:44:10.388 INFO:tasks.workunit.client.0.smithi028.stdout:0/571: dread d2/d3/f25 [0,4194304] 0 2024-07-03T15:44:10.393 INFO:tasks.workunit.client.0.smithi028.stdout:7/387: getdents d9/d1c/d52 0 2024-07-03T15:44:10.393 INFO:tasks.workunit.client.0.smithi028.stdout:7/388: write d9/d1c/d45/f62 [313907,122152] 0 2024-07-03T15:44:10.393 INFO:tasks.workunit.client.0.smithi028.stdout:7/389: truncate d9/d29/f2e 738250 0 2024-07-03T15:44:10.393 INFO:tasks.workunit.client.0.smithi028.stdout:7/390: readlink d9/d29/d5d/l68 0 2024-07-03T15:44:10.396 INFO:tasks.workunit.client.0.smithi028.stdout:6/543: dread d4/f3a [0,4194304] 0 2024-07-03T15:44:10.396 INFO:tasks.workunit.client.0.smithi028.stdout:6/544: chown d4/f7b 1 1 2024-07-03T15:44:10.396 INFO:tasks.workunit.client.0.smithi028.stdout:7/391: mknod d9/d29/c75 0 2024-07-03T15:44:10.396 INFO:tasks.workunit.client.0.smithi028.stdout:7/392: write d9/d1c/f3f [8551665,12006] 0 2024-07-03T15:44:10.396 INFO:tasks.workunit.client.0.smithi028.stdout:7/393: creat d9/d1c/d6e/f76 x:0 0 0 2024-07-03T15:44:10.396 INFO:tasks.workunit.client.0.smithi028.stdout:7/394: creat d9/d29/f77 x:0 0 0 2024-07-03T15:44:10.396 INFO:tasks.workunit.client.0.smithi028.stdout:7/395: chown d9/d1c/d45 126195 1 2024-07-03T15:44:10.396 INFO:tasks.workunit.client.0.smithi028.stdout:7/396: readlink d9/d1c/d52/d6a/l6f 0 2024-07-03T15:44:10.397 INFO:tasks.workunit.client.0.smithi028.stdout:9/514: sync 2024-07-03T15:44:10.397 INFO:tasks.workunit.client.0.smithi028.stdout:8/575: sync 2024-07-03T15:44:10.397 INFO:tasks.workunit.client.0.smithi028.stdout:9/515: fdatasync d1/f31 0 2024-07-03T15:44:10.397 INFO:tasks.workunit.client.0.smithi028.stdout:0/572: dread d2/d6d/d92/d58/f5f [0,4194304] 0 2024-07-03T15:44:10.402 INFO:tasks.workunit.client.0.smithi028.stdout:7/397: mkdir d9/d1c/d45/d71/d78 0 2024-07-03T15:44:10.402 INFO:tasks.workunit.client.0.smithi028.stdout:9/516: symlink d1/d10/d28/d45/d56/laf 0 2024-07-03T15:44:10.403 INFO:tasks.workunit.client.0.smithi028.stdout:0/573: mkdir d2/d3/d4d/d56/d78/da8 0 2024-07-03T15:44:10.410 INFO:tasks.workunit.client.0.smithi028.stdout:9/517: creat d1/d16/d57/d60/fb0 x:0 0 0 2024-07-03T15:44:10.411 INFO:tasks.workunit.client.0.smithi028.stdout:9/518: getdents d1/d8e/dac 0 2024-07-03T15:44:10.411 INFO:tasks.workunit.client.0.smithi028.stdout:9/519: write d1/d10/d28/d45/fa8 [206492,41886] 0 2024-07-03T15:44:10.411 INFO:tasks.workunit.client.0.smithi028.stdout:9/520: write d1/d16/d57/f66 [872999,92801] 0 2024-07-03T15:44:10.411 INFO:tasks.workunit.client.0.smithi028.stdout:9/521: truncate d1/d16/d57/d5f/f80 1009539 0 2024-07-03T15:44:10.411 INFO:tasks.workunit.client.0.smithi028.stdout:0/574: mkdir d2/d1c/d6c/da9 0 2024-07-03T15:44:10.414 INFO:tasks.workunit.client.0.smithi028.stdout:9/522: creat d1/d16/d57/fb1 x:0 0 0 2024-07-03T15:44:10.415 INFO:tasks.workunit.client.0.smithi028.stdout:6/545: dwrite d4/da/d38/fa7 [0,4194304] 0 2024-07-03T15:44:10.416 INFO:tasks.workunit.client.0.smithi028.stdout:6/546: write d4/da/d38/d59/f92 [847445,10608] 0 2024-07-03T15:44:10.416 INFO:tasks.workunit.client.0.smithi028.stdout:7/398: dwrite d9/d29/f35 [0,4194304] 0 2024-07-03T15:44:10.417 INFO:tasks.workunit.client.0.smithi028.stdout:0/575: write d2/d1c/f2b [2333272,49228] 0 2024-07-03T15:44:10.418 INFO:tasks.workunit.client.0.smithi028.stdout:0/576: write d2/d1c/d6c/f79 [5447602,36802] 0 2024-07-03T15:44:10.419 INFO:tasks.workunit.client.0.smithi028.stdout:9/523: rename d1/d10/ca2 to d1/d10/d28/d4e/d5b/cb2 0 2024-07-03T15:44:10.421 INFO:tasks.workunit.client.0.smithi028.stdout:7/399: link d9/d29/f4f d9/d29/d5d/f79 0 2024-07-03T15:44:10.421 INFO:tasks.workunit.client.0.smithi028.stdout:7/400: fdatasync d9/d1c/d45/f53 0 2024-07-03T15:44:10.428 INFO:tasks.workunit.client.0.smithi028.stdout:9/524: dread d1/d16/f19 [0,4194304] 0 2024-07-03T15:44:10.432 INFO:tasks.workunit.client.0.smithi028.stdout:9/525: getdents d1/d16/d57 0 2024-07-03T15:44:10.434 INFO:tasks.workunit.client.0.smithi028.stdout:7/401: dread d9/f1a [0,4194304] 0 2024-07-03T15:44:10.438 INFO:tasks.workunit.client.0.smithi028.stdout:0/577: write d2/d6d/d92/d36/d45/f46 [3893816,109595] 0 2024-07-03T15:44:10.439 INFO:tasks.workunit.client.0.smithi028.stdout:7/402: creat d9/d29/d5d/f7a x:0 0 0 2024-07-03T15:44:10.441 INFO:tasks.workunit.client.0.smithi028.stdout:7/403: truncate f0 8743472 0 2024-07-03T15:44:10.442 INFO:tasks.workunit.client.0.smithi028.stdout:7/404: creat d9/d1c/d52/d6a/f7b x:0 0 0 2024-07-03T15:44:10.450 INFO:tasks.workunit.client.0.smithi028.stdout:6/547: dwrite d4/da/d13/f7d [0,4194304] 0 2024-07-03T15:44:10.452 INFO:tasks.workunit.client.0.smithi028.stdout:7/405: read d9/d29/f35 [5636123,89935] 0 2024-07-03T15:44:10.452 INFO:tasks.workunit.client.0.smithi028.stdout:7/406: chown d9/d29/f44 24 1 2024-07-03T15:44:10.452 INFO:tasks.workunit.client.0.smithi028.stdout:7/407: stat d9/d1c 0 2024-07-03T15:44:10.452 INFO:tasks.workunit.client.0.smithi028.stdout:7/408: chown d9/d1c/d6e/c70 323559 1 2024-07-03T15:44:10.452 INFO:tasks.workunit.client.0.smithi028.stdout:7/409: read d9/d29/f67 [816505,22548] 0 2024-07-03T15:44:10.453 INFO:tasks.workunit.client.0.smithi028.stdout:7/410: fsync d9/f48 0 2024-07-03T15:44:10.455 INFO:tasks.workunit.client.0.smithi028.stdout:7/411: mkdir d9/d1c/d7c 0 2024-07-03T15:44:10.455 INFO:tasks.workunit.client.0.smithi028.stdout:7/412: readlink d9/d1c/d52/d6a/l6f 0 2024-07-03T15:44:10.456 INFO:tasks.workunit.client.0.smithi028.stdout:7/413: dread d9/d29/f2e [0,4194304] 0 2024-07-03T15:44:10.457 INFO:tasks.workunit.client.0.smithi028.stdout:7/414: link d9/d29/f61 d9/d1c/d52/d6a/f7d 0 2024-07-03T15:44:10.458 INFO:tasks.workunit.client.0.smithi028.stdout:7/415: truncate f6 1800004 0 2024-07-03T15:44:10.458 INFO:tasks.workunit.client.0.smithi028.stdout:7/416: chown d9/fa 1624388 1 2024-07-03T15:44:10.460 INFO:tasks.workunit.client.0.smithi028.stdout:0/578: dwrite d2/d6d/d92/d58/fa4 [4194304,4194304] 0 2024-07-03T15:44:10.465 INFO:tasks.workunit.client.0.smithi028.stdout:6/548: dread d4/f29 [0,4194304] 0 2024-07-03T15:44:10.470 INFO:tasks.workunit.client.0.smithi028.stdout:6/549: rmdir d4/da/d13 39 2024-07-03T15:44:10.474 INFO:tasks.workunit.client.0.smithi028.stdout:6/550: mknod d4/d15/cc0 0 2024-07-03T15:44:10.477 INFO:tasks.workunit.client.0.smithi028.stdout:6/551: rename d4/l41 to d4/da/d47/d66/lc1 0 2024-07-03T15:44:10.477 INFO:tasks.workunit.client.0.smithi028.stdout:6/552: fdatasync d4/f33 0 2024-07-03T15:44:10.481 INFO:tasks.workunit.client.0.smithi028.stdout:6/553: rename d4/da/d38/d60/l6e to d4/da/d13/dbd/lc2 0 2024-07-03T15:44:10.481 INFO:tasks.workunit.client.0.smithi028.stdout:6/554: write d4/da/d38/d59/f65 [1106424,112216] 0 2024-07-03T15:44:10.483 INFO:tasks.workunit.client.0.smithi028.stdout:6/555: mkdir d4/d12/dc3 0 2024-07-03T15:44:10.483 INFO:tasks.workunit.client.0.smithi028.stdout:7/417: dwrite d9/d29/f42 [0,4194304] 0 2024-07-03T15:44:10.483 INFO:tasks.workunit.client.0.smithi028.stdout:0/579: dwrite d2/d6d/d92/d36/d45/f72 [0,4194304] 0 2024-07-03T15:44:10.485 INFO:tasks.workunit.client.0.smithi028.stdout:6/556: link d4/d12/fa9 d4/da/d47/d9f/fc4 0 2024-07-03T15:44:10.485 INFO:tasks.workunit.client.0.smithi028.stdout:6/557: chown d4/f11 4052 1 2024-07-03T15:44:10.485 INFO:tasks.workunit.client.0.smithi028.stdout:6/558: readlink d4/d12/d2c/d31/l39 0 2024-07-03T15:44:10.490 INFO:tasks.workunit.client.0.smithi028.stdout:7/418: link d9/d29/c75 d9/d1c/d7c/c7e 0 2024-07-03T15:44:10.493 INFO:tasks.workunit.client.0.smithi028.stdout:0/580: symlink d2/d6d/d92/d36/d45/laa 0 2024-07-03T15:44:10.497 INFO:tasks.workunit.client.0.smithi028.stdout:0/581: creat d2/d6d/d92/d36/d45/da3/fab x:0 0 0 2024-07-03T15:44:10.498 INFO:tasks.workunit.client.0.smithi028.stdout:7/419: truncate d9/d29/f42 345189 0 2024-07-03T15:44:10.498 INFO:tasks.workunit.client.0.smithi028.stdout:7/420: fsync d9/d1c/f5a 0 2024-07-03T15:44:10.499 INFO:tasks.workunit.client.0.smithi028.stdout:7/421: fdatasync d9/f1f 0 2024-07-03T15:44:10.499 INFO:tasks.workunit.client.0.smithi028.stdout:7/422: read d9/f34 [76929,72653] 0 2024-07-03T15:44:10.508 INFO:tasks.workunit.client.0.smithi028.stdout:0/582: write d2/d6d/d92/d36/d45/f72 [2229917,76572] 0 2024-07-03T15:44:10.509 INFO:tasks.workunit.client.0.smithi028.stdout:0/583: fdatasync d2/d6d/d92/d36/d45/f7b 0 2024-07-03T15:44:10.512 INFO:tasks.workunit.client.0.smithi028.stdout:0/584: rename d2/d6d/d92/d36/f7a to d2/d3/fac 0 2024-07-03T15:44:10.513 INFO:tasks.workunit.client.0.smithi028.stdout:0/585: mknod d2/d3/d4d/d96/cad 0 2024-07-03T15:44:10.514 INFO:tasks.workunit.client.0.smithi028.stdout:0/586: unlink d2/d3/fe 0 2024-07-03T15:44:10.516 INFO:tasks.workunit.client.0.smithi028.stdout:0/587: dread d2/d3/f25 [0,4194304] 0 2024-07-03T15:44:10.518 INFO:tasks.workunit.client.0.smithi028.stdout:0/588: fsync d2/d6d/d92/d36/d45/f72 0 2024-07-03T15:44:10.519 INFO:tasks.workunit.client.0.smithi028.stdout:0/589: mkdir d2/d3/d4d/d56/d78/dae 0 2024-07-03T15:44:10.520 INFO:tasks.workunit.client.0.smithi028.stdout:0/590: creat d2/d6d/d92/d58/faf x:0 0 0 2024-07-03T15:44:10.522 INFO:tasks.workunit.client.0.smithi028.stdout:0/591: creat d2/d1c/d6c/da9/fb0 x:0 0 0 2024-07-03T15:44:10.523 INFO:tasks.workunit.client.0.smithi028.stdout:0/592: link d2/d1c/f20 d2/d3/d4d/d56/d78/da8/fb1 0 2024-07-03T15:44:10.535 INFO:tasks.workunit.client.0.smithi028.stdout:0/593: write d2/d6d/d92/d58/d6e/f94 [452086,117074] 0 2024-07-03T15:44:10.536 INFO:tasks.workunit.client.0.smithi028.stdout:0/594: link d2/d3/c43 d2/d6d/d92/d58/cb2 0 2024-07-03T15:44:10.537 INFO:tasks.workunit.client.0.smithi028.stdout:0/595: creat d2/d6d/d92/fb3 x:0 0 0 2024-07-03T15:44:10.538 INFO:tasks.workunit.client.0.smithi028.stdout:0/596: symlink d2/d6d/d92/d36/d45/lb4 0 2024-07-03T15:44:10.559 INFO:tasks.workunit.client.0.smithi028.stdout:0/597: dwrite d2/d3/fac [0,4194304] 0 2024-07-03T15:44:10.564 INFO:tasks.workunit.client.0.smithi028.stdout:0/598: truncate d2/f15 930014 0 2024-07-03T15:44:10.567 INFO:tasks.workunit.client.0.smithi028.stdout:0/599: creat d2/d3/d4d/d96/fb5 x:0 0 0 2024-07-03T15:44:10.567 INFO:tasks.workunit.client.0.smithi028.stdout:0/600: chown d2/l74 3504449 1 2024-07-03T15:44:10.570 INFO:tasks.workunit.client.0.smithi028.stdout:0/601: creat d2/d6d/d92/d58/d82/fb6 x:0 0 0 2024-07-03T15:44:10.573 INFO:tasks.workunit.client.0.smithi028.stdout:5/544: sync 2024-07-03T15:44:10.574 INFO:tasks.workunit.client.0.smithi028.stdout:5/545: mknod d0/dd/df/da1/daa/cb4 0 2024-07-03T15:44:10.576 INFO:tasks.workunit.client.0.smithi028.stdout:5/546: mkdir d0/d8f/db5 0 2024-07-03T15:44:10.576 INFO:tasks.workunit.client.0.smithi028.stdout:5/547: chown d0/dd/d13/d57/d30 32043 1 2024-07-03T15:44:10.578 INFO:tasks.workunit.client.0.smithi028.stdout:5/548: link d0/dd/d1e/d3b/d3f/c51 d0/dd/d13/d57/cb6 0 2024-07-03T15:44:10.580 INFO:tasks.workunit.client.0.smithi028.stdout:5/549: unlink d0/dd/d14/c28 0 2024-07-03T15:44:10.581 INFO:tasks.workunit.client.0.smithi028.stdout:5/550: rename d0/dd/d13/d57/d26/l69 to d0/dd/d13/d29/lb7 0 2024-07-03T15:44:10.582 INFO:tasks.workunit.client.0.smithi028.stdout:5/551: truncate d0/dd/d98/f6a 1828739 0 2024-07-03T15:44:10.583 INFO:tasks.workunit.client.0.smithi028.stdout:5/552: fdatasync d0/dd/d13/d57/d77/f7d 0 2024-07-03T15:44:10.586 INFO:tasks.workunit.client.0.smithi028.stdout:5/553: dread d0/f9 [8388608,4194304] 0 2024-07-03T15:44:10.587 INFO:tasks.workunit.client.0.smithi028.stdout:5/554: mknod d0/dd/cb8 0 2024-07-03T15:44:10.588 INFO:tasks.workunit.client.0.smithi028.stdout:5/555: creat d0/dd/d13/d47/d50/fb9 x:0 0 0 2024-07-03T15:44:10.588 INFO:tasks.workunit.client.0.smithi028.stdout:5/556: dread - d0/dd/fae zero size 2024-07-03T15:44:10.588 INFO:tasks.workunit.client.0.smithi028.stdout:5/557: chown d0/dd/d13/d57/d26/d41/l75 2059469816 1 2024-07-03T15:44:10.589 INFO:tasks.workunit.client.0.smithi028.stdout:0/602: dwrite d2/d3/d4d/fa2 [0,4194304] 0 2024-07-03T15:44:10.589 INFO:tasks.workunit.client.0.smithi028.stdout:0/603: readlink d2/d1c/l30 0 2024-07-03T15:44:10.590 INFO:tasks.workunit.client.0.smithi028.stdout:0/604: write d2/d3/f8d [1063477,12417] 0 2024-07-03T15:44:10.590 INFO:tasks.workunit.client.0.smithi028.stdout:0/605: fsync d2/d6d/d92/f38 0 2024-07-03T15:44:10.590 INFO:tasks.workunit.client.0.smithi028.stdout:5/558: creat d0/dd/d13/d57/d26/d41/db0/fba x:0 0 0 2024-07-03T15:44:10.596 INFO:tasks.workunit.client.0.smithi028.stdout:0/606: unlink d2/d3/c43 0 2024-07-03T15:44:10.597 INFO:tasks.workunit.client.0.smithi028.stdout:5/559: write d0/dd/d1e/f7a [2752885,62696] 0 2024-07-03T15:44:10.599 INFO:tasks.workunit.client.0.smithi028.stdout:0/607: mknod d2/d3/d4d/cb7 0 2024-07-03T15:44:10.602 INFO:tasks.workunit.client.0.smithi028.stdout:5/560: mkdir d0/dd/d13/d57/d30/d70/d7f/dbb 0 2024-07-03T15:44:10.602 INFO:tasks.workunit.client.0.smithi028.stdout:5/561: chown d0/dd/d1e/d8e/fb3 1498218171 1 2024-07-03T15:44:10.602 INFO:tasks.workunit.client.0.smithi028.stdout:5/562: write d0/dd/d13/d29/f4e [2585383,8049] 0 2024-07-03T15:44:10.604 INFO:tasks.workunit.client.0.smithi028.stdout:0/608: creat d2/d3/d4d/d56/fb8 x:0 0 0 2024-07-03T15:44:10.604 INFO:tasks.workunit.client.0.smithi028.stdout:0/609: creat d2/d6d/d92/d36/d45/da3/fb9 x:0 0 0 2024-07-03T15:44:10.604 INFO:tasks.workunit.client.0.smithi028.stdout:0/610: write d2/d1c/f7e [325113,46500] 0 2024-07-03T15:44:10.606 INFO:tasks.workunit.client.0.smithi028.stdout:0/611: write d2/d3/f25 [9701,55173] 0 2024-07-03T15:44:10.608 INFO:tasks.workunit.client.0.smithi028.stdout:5/563: rename d0/dd/dad to d0/dd/d13/d47/dbc 0 2024-07-03T15:44:10.608 INFO:tasks.workunit.client.0.smithi028.stdout:5/564: write d0/dd/d1e/d3b/f4f [1243367,89611] 0 2024-07-03T15:44:10.608 INFO:tasks.workunit.client.0.smithi028.stdout:5/565: write d0/dd/f7e [925408,18198] 0 2024-07-03T15:44:10.619 INFO:tasks.workunit.client.0.smithi028.stdout:5/566: creat d0/dd/d13/d57/d30/d70/d7f/fbd x:0 0 0 2024-07-03T15:44:10.628 INFO:tasks.workunit.client.0.smithi028.stdout:0/612: dwrite d2/d3/f25 [0,4194304] 0 2024-07-03T15:44:10.641 INFO:tasks.workunit.client.0.smithi028.stdout:5/567: dwrite d0/dd/d13/d57/f1f [0,4194304] 0 2024-07-03T15:44:10.641 INFO:tasks.workunit.client.0.smithi028.stdout:0/613: dwrite d2/d6d/d92/d36/d45/f99 [0,4194304] 0 2024-07-03T15:44:10.650 INFO:tasks.workunit.client.0.smithi028.stdout:5/568: creat d0/dd/d14/d9f/fbe x:0 0 0 2024-07-03T15:44:10.650 INFO:tasks.workunit.client.0.smithi028.stdout:5/569: dread - d0/dd/f99 zero size 2024-07-03T15:44:10.650 INFO:tasks.workunit.client.0.smithi028.stdout:5/570: creat d0/dd/d13/d57/d30/d70/d7f/fbf x:0 0 0 2024-07-03T15:44:10.661 INFO:tasks.workunit.client.0.smithi028.stdout:0/614: dwrite d2/d3/d4d/d56/d78/da8/fb1 [4194304,4194304] 0 2024-07-03T15:44:10.664 INFO:tasks.workunit.client.0.smithi028.stdout:5/571: unlink d0/dd/d1e/d3b/c45 0 2024-07-03T15:44:10.668 INFO:tasks.workunit.client.0.smithi028.stdout:0/615: dread d2/d6d/d92/d36/d45/f99 [0,4194304] 0 2024-07-03T15:44:10.706 INFO:tasks.workunit.client.0.smithi028.stdout:5/572: mkdir d0/dd/d14/dc0 0 2024-07-03T15:44:10.707 INFO:tasks.workunit.client.0.smithi028.stdout:5/573: write d0/dd/d1e/d3b/fa3 [2140622,56322] 0 2024-07-03T15:44:10.708 INFO:tasks.workunit.client.0.smithi028.stdout:5/574: creat d0/dd/d13/d57/fc1 x:0 0 0 2024-07-03T15:44:10.710 INFO:tasks.workunit.client.0.smithi028.stdout:0/616: symlink d2/d6d/d92/d58/d82/d9b/lba 0 2024-07-03T15:44:10.711 INFO:tasks.workunit.client.0.smithi028.stdout:5/575: rename d0/dd/d13/d57/d30/d70/d7f/fbf to d0/dd/df/fc2 0 2024-07-03T15:44:10.713 INFO:tasks.workunit.client.0.smithi028.stdout:5/576: rename d0/dd/d1e/c7b to d0/dd/d13/cc3 0 2024-07-03T15:44:10.713 INFO:tasks.workunit.client.0.smithi028.stdout:5/577: chown d0/dd/d13/d47/d50/l7c 140916 1 2024-07-03T15:44:10.713 INFO:tasks.workunit.client.0.smithi028.stdout:5/578: truncate d0/dd/d1e/d3b/d3f/d5a/f78 463622 0 2024-07-03T15:44:10.713 INFO:tasks.workunit.client.0.smithi028.stdout:5/579: chown d0/dd/d1e/lac 252347 1 2024-07-03T15:44:10.715 INFO:tasks.workunit.client.0.smithi028.stdout:5/580: rename d0/dd/d14/d9f to d0/dd/d13/d29/dc4 0 2024-07-03T15:44:10.718 INFO:tasks.workunit.client.0.smithi028.stdout:5/581: creat d0/dd/d13/d57/d30/fc5 x:0 0 0 2024-07-03T15:44:10.720 INFO:tasks.workunit.client.0.smithi028.stdout:0/617: dread d2/d1c/f2e [0,4194304] 0 2024-07-03T15:44:10.722 INFO:tasks.workunit.client.0.smithi028.stdout:0/618: symlink d2/d3/d4d/d56/d78/da8/lbb 0 2024-07-03T15:44:10.725 INFO:tasks.workunit.client.0.smithi028.stdout:0/619: truncate d2/d1c/d6c/f79 3260636 0 2024-07-03T15:44:10.729 INFO:tasks.workunit.client.0.smithi028.stdout:0/620: truncate d2/d6d/d92/d58/d6e/f80 3702728 0 2024-07-03T15:44:10.731 INFO:tasks.workunit.client.0.smithi028.stdout:0/621: mknod d2/d6d/d92/cbc 0 2024-07-03T15:44:10.732 INFO:tasks.workunit.client.0.smithi028.stdout:0/622: symlink d2/d3/d4d/d56/d78/lbd 0 2024-07-03T15:44:10.732 INFO:tasks.workunit.client.0.smithi028.stdout:0/623: fsync d2/f3a 0 2024-07-03T15:44:10.735 INFO:tasks.workunit.client.0.smithi028.stdout:0/624: mkdir d2/d3/d4d/d56/d78/dbe 0 2024-07-03T15:44:10.737 INFO:tasks.workunit.client.0.smithi028.stdout:0/625: fdatasync d2/d6d/d92/d58/fa4 0 2024-07-03T15:44:10.737 INFO:tasks.workunit.client.0.smithi028.stdout:0/626: creat d2/d1c/d6c/da9/fbf x:0 0 0 2024-07-03T15:44:10.739 INFO:tasks.workunit.client.0.smithi028.stdout:5/582: dwrite d0/dd/d1e/d3b/d3f/d5a/f65 [0,4194304] 0 2024-07-03T15:44:10.743 INFO:tasks.workunit.client.0.smithi028.stdout:0/627: mknod d2/d6d/d92/d58/d82/d9b/cc0 0 2024-07-03T15:44:10.743 INFO:tasks.workunit.client.0.smithi028.stdout:0/628: creat d2/d6d/d92/fc1 x:0 0 0 2024-07-03T15:44:10.746 INFO:tasks.workunit.client.0.smithi028.stdout:0/629: dread - d2/d6d/d92/d58/faf zero size 2024-07-03T15:44:10.747 INFO:tasks.workunit.client.0.smithi028.stdout:0/630: chown d2/d6d/d92/l97 157 1 2024-07-03T15:44:10.749 INFO:tasks.workunit.client.0.smithi028.stdout:0/631: mkdir d2/d3/d4d/d96/dc2 0 2024-07-03T15:44:10.755 INFO:tasks.workunit.client.0.smithi028.stdout:5/583: dwrite d0/dd/d13/d57/d26/d41/db0/fba [0,4194304] 0 2024-07-03T15:44:10.758 INFO:tasks.workunit.client.0.smithi028.stdout:5/584: truncate d0/dd/d98/f6c 1442847 0 2024-07-03T15:44:10.759 INFO:tasks.workunit.client.0.smithi028.stdout:5/585: getdents d0/dd/d1e 0 2024-07-03T15:44:10.759 INFO:tasks.workunit.client.0.smithi028.stdout:5/586: write d0/dd/fae [389903,100045] 0 2024-07-03T15:44:10.761 INFO:tasks.workunit.client.0.smithi028.stdout:0/632: dread d2/d3/fac [0,4194304] 0 2024-07-03T15:44:10.761 INFO:tasks.workunit.client.0.smithi028.stdout:5/587: fsync d0/dd/df/f3e 0 2024-07-03T15:44:10.761 INFO:tasks.workunit.client.0.smithi028.stdout:5/588: creat d0/dd/d1e/d8e/fc6 x:0 0 0 2024-07-03T15:44:10.762 INFO:tasks.workunit.client.0.smithi028.stdout:0/633: symlink d2/d6d/d92/d58/d82/lc3 0 2024-07-03T15:44:10.768 INFO:tasks.workunit.client.0.smithi028.stdout:5/589: truncate d0/dd/d13/d57/d26/d41/f44 3224643 0 2024-07-03T15:44:10.769 INFO:tasks.workunit.client.0.smithi028.stdout:0/634: dread d2/d6d/d92/d58/d6e/f80 [0,4194304] 0 2024-07-03T15:44:10.782 INFO:tasks.workunit.client.0.smithi028.stdout:0/635: rename d2/d6d/d92 to d2/d3/dc4 0 2024-07-03T15:44:10.788 INFO:tasks.workunit.client.0.smithi028.stdout:5/590: dwrite d0/dd/d1e/d3b/f4f [4194304,4194304] 0 2024-07-03T15:44:10.788 INFO:tasks.workunit.client.0.smithi028.stdout:5/591: fsync d0/dd/d13/d57/d30/d70/d7f/fbd 0 2024-07-03T15:44:10.788 INFO:tasks.workunit.client.0.smithi028.stdout:5/592: dread - d0/dd/d13/d57/d30/d70/d7f/fbd zero size 2024-07-03T15:44:10.789 INFO:tasks.workunit.client.0.smithi028.stdout:5/593: rename d0/dd/d13/d57/d26/d41/f5f to d0/dd/d13/d47/dbc/fc7 0 2024-07-03T15:44:10.792 INFO:tasks.workunit.client.0.smithi028.stdout:0/636: dread d2/d1c/d6c/f85 [0,4194304] 0 2024-07-03T15:44:10.794 INFO:tasks.workunit.client.0.smithi028.stdout:5/594: dread d0/dd/d98/f6a [0,4194304] 0 2024-07-03T15:44:10.796 INFO:tasks.workunit.client.0.smithi028.stdout:1/633: sync 2024-07-03T15:44:10.796 INFO:tasks.workunit.client.0.smithi028.stdout:4/576: sync 2024-07-03T15:44:10.796 INFO:tasks.workunit.client.0.smithi028.stdout:2/455: sync 2024-07-03T15:44:10.796 INFO:tasks.workunit.client.0.smithi028.stdout:2/456: fdatasync da/d24/fa4 0 2024-07-03T15:44:10.797 INFO:tasks.workunit.client.0.smithi028.stdout:3/501: sync 2024-07-03T15:44:10.797 INFO:tasks.workunit.client.0.smithi028.stdout:5/595: creat d0/dd/df/da7/fc8 x:0 0 0 2024-07-03T15:44:10.797 INFO:tasks.workunit.client.0.smithi028.stdout:5/596: dread - d0/dd/d1e/d8e/fb3 zero size 2024-07-03T15:44:10.802 INFO:tasks.workunit.client.0.smithi028.stdout:1/634: creat d4/d3c/d54/d79/d16/fe2 x:0 0 0 2024-07-03T15:44:10.802 INFO:tasks.workunit.client.0.smithi028.stdout:4/577: mknod d2/d12/d3b/cbb 0 2024-07-03T15:44:10.805 INFO:tasks.workunit.client.0.smithi028.stdout:3/502: truncate fe 1674992 0 2024-07-03T15:44:10.805 INFO:tasks.workunit.client.0.smithi028.stdout:4/578: truncate d2/d5a/f9 113483 0 2024-07-03T15:44:10.807 INFO:tasks.workunit.client.0.smithi028.stdout:2/457: symlink da/d1e/la8 0 2024-07-03T15:44:10.810 INFO:tasks.workunit.client.0.smithi028.stdout:4/579: mknod d2/d61/d9c/cbc 0 2024-07-03T15:44:10.811 INFO:tasks.workunit.client.0.smithi028.stdout:2/458: truncate da/d24/fa4 2357985 0 2024-07-03T15:44:10.813 INFO:tasks.workunit.client.0.smithi028.stdout:4/580: link d2/d12/d42/d63/d9e/cb7 d2/d3e/d9f/dac/cbd 0 2024-07-03T15:44:10.813 INFO:tasks.workunit.client.0.smithi028.stdout:1/635: write d4/de/da7/fc0 [609303,36616] 0 2024-07-03T15:44:10.813 INFO:tasks.workunit.client.0.smithi028.stdout:4/581: chown d2/d12/d42/d63/da8 20775 1 2024-07-03T15:44:10.813 INFO:tasks.workunit.client.0.smithi028.stdout:1/636: creat d4/d83/fe3 x:0 0 0 2024-07-03T15:44:10.813 INFO:tasks.workunit.client.0.smithi028.stdout:4/582: write d2/d12/d2d/d58/fa3 [951193,32181] 0 2024-07-03T15:44:10.814 INFO:tasks.workunit.client.0.smithi028.stdout:4/583: write d2/d12/d2d/d58/f5e [1088395,104080] 0 2024-07-03T15:44:10.816 INFO:tasks.workunit.client.0.smithi028.stdout:2/459: mknod da/d23/d6f/d84/ca9 0 2024-07-03T15:44:10.816 INFO:tasks.workunit.client.0.smithi028.stdout:2/460: truncate da/d1e/d68/f6c 1218403 0 2024-07-03T15:44:10.818 INFO:tasks.workunit.client.0.smithi028.stdout:4/584: mkdir d2/d12/d42/d63/d97/dbe 0 2024-07-03T15:44:10.819 INFO:tasks.workunit.client.0.smithi028.stdout:2/461: mkdir da/d42/d47/daa 0 2024-07-03T15:44:10.819 INFO:tasks.workunit.client.0.smithi028.stdout:4/585: creat d2/d12/da6/fbf x:0 0 0 2024-07-03T15:44:10.820 INFO:tasks.workunit.client.0.smithi028.stdout:2/462: creat da/d42/d47/da2/fab x:0 0 0 2024-07-03T15:44:10.820 INFO:tasks.workunit.client.0.smithi028.stdout:2/463: chown da/d23/d30/f3e 7226765 1 2024-07-03T15:44:10.821 INFO:tasks.workunit.client.0.smithi028.stdout:4/586: rename d2/d12/d42/d63/d75/c96 to d2/d5a/d45/cc0 0 2024-07-03T15:44:10.823 INFO:tasks.workunit.client.0.smithi028.stdout:0/637: dwrite d2/d1c/f68 [0,4194304] 0 2024-07-03T15:44:10.823 INFO:tasks.workunit.client.0.smithi028.stdout:4/587: rename d2/d12/d42/d63/d97/f9a to d2/d12/d42/d63/da8/fc1 0 2024-07-03T15:44:10.826 INFO:tasks.workunit.client.0.smithi028.stdout:0/638: truncate d2/d1c/f2d 2797871 0 2024-07-03T15:44:10.827 INFO:tasks.workunit.client.0.smithi028.stdout:4/588: mknod d2/d5a/cc2 0 2024-07-03T15:44:10.827 INFO:tasks.workunit.client.0.smithi028.stdout:4/589: getdents d2/d12 0 2024-07-03T15:44:10.829 INFO:tasks.workunit.client.0.smithi028.stdout:4/590: link d2/d3e/d9f/d74/f93 d2/d12/d2e/d87/fc3 0 2024-07-03T15:44:10.830 INFO:tasks.workunit.client.0.smithi028.stdout:0/639: dread d2/d3/dc4/d36/f3d [0,4194304] 0 2024-07-03T15:44:10.830 INFO:tasks.workunit.client.0.smithi028.stdout:4/591: link d2/f52 d2/d61/d9d/fc4 0 2024-07-03T15:44:10.831 INFO:tasks.workunit.client.0.smithi028.stdout:5/597: dwrite d0/dd/d1e/d3b/f5e [4194304,4194304] 0 2024-07-03T15:44:10.832 INFO:tasks.workunit.client.0.smithi028.stdout:1/637: dwrite d4/d3c/d54/d79/d16/d7c/dd5/fdc [0,4194304] 0 2024-07-03T15:44:10.832 INFO:tasks.workunit.client.0.smithi028.stdout:1/638: dread - d4/d3c/d54/d84/f9b zero size 2024-07-03T15:44:10.833 INFO:tasks.workunit.client.0.smithi028.stdout:0/640: mkdir d2/d1c/d6c/dc5 0 2024-07-03T15:44:10.837 INFO:tasks.workunit.client.0.smithi028.stdout:1/639: unlink d4/d2c/d3a/d5c/d6c/d7d/d59/c56 0 2024-07-03T15:44:10.839 INFO:tasks.workunit.client.0.smithi028.stdout:0/641: creat d2/d3/fc6 x:0 0 0 2024-07-03T15:44:10.840 INFO:tasks.workunit.client.0.smithi028.stdout:4/592: write d2/f2b [3127228,128972] 0 2024-07-03T15:44:10.840 INFO:tasks.workunit.client.0.smithi028.stdout:4/593: creat d2/d61/fc5 x:0 0 0 2024-07-03T15:44:10.840 INFO:tasks.workunit.client.0.smithi028.stdout:3/503: dwrite d18/d1a/d1e/d58/f6c [0,4194304] 0 2024-07-03T15:44:10.840 INFO:tasks.workunit.client.0.smithi028.stdout:4/594: fdatasync f1 0 2024-07-03T15:44:10.841 INFO:tasks.workunit.client.0.smithi028.stdout:1/640: unlink d4/d3c/f7b 0 2024-07-03T15:44:10.841 INFO:tasks.workunit.client.0.smithi028.stdout:1/641: write d4/d2c/d3a/d5c/daa/fb5 [325846,69251] 0 2024-07-03T15:44:10.842 INFO:tasks.workunit.client.0.smithi028.stdout:0/642: symlink d2/lc7 0 2024-07-03T15:44:10.843 INFO:tasks.workunit.client.0.smithi028.stdout:0/643: fsync d2/fb 0 2024-07-03T15:44:10.849 INFO:tasks.workunit.client.0.smithi028.stdout:3/504: rename d18/d3a/f77 to d18/d1a/d64/f9b 0 2024-07-03T15:44:10.849 INFO:tasks.workunit.client.0.smithi028.stdout:3/505: write d18/d6f/f7d [1014307,47476] 0 2024-07-03T15:44:10.851 INFO:tasks.workunit.client.0.smithi028.stdout:2/464: dwrite da/d74/f8d [0,4194304] 0 2024-07-03T15:44:10.852 INFO:tasks.workunit.client.0.smithi028.stdout:4/595: mknod d2/d12/d3b/cc6 0 2024-07-03T15:44:10.854 INFO:tasks.workunit.client.0.smithi028.stdout:5/598: dwrite d0/dd/d1e/f89 [0,4194304] 0 2024-07-03T15:44:10.854 INFO:tasks.workunit.client.0.smithi028.stdout:0/644: dread d2/d1c/f68 [0,4194304] 0 2024-07-03T15:44:10.854 INFO:tasks.workunit.client.0.smithi028.stdout:0/645: read d2/d3/dc4/d36/f3d [553638,107489] 0 2024-07-03T15:44:10.856 INFO:tasks.workunit.client.0.smithi028.stdout:2/465: dread da/d24/d2c/f32 [0,4194304] 0 2024-07-03T15:44:10.857 INFO:tasks.workunit.client.0.smithi028.stdout:1/642: mknod d4/d2c/d3a/d5c/ce4 0 2024-07-03T15:44:10.861 INFO:tasks.workunit.client.0.smithi028.stdout:3/506: mkdir d18/d6f/d78/d87/d9c 0 2024-07-03T15:44:10.864 INFO:tasks.workunit.client.0.smithi028.stdout:4/596: mknod d2/d12/d2d/d2f/d3a/cc7 0 2024-07-03T15:44:10.878 INFO:tasks.workunit.client.0.smithi028.stdout:5/599: creat d0/d8f/fc9 x:0 0 0 2024-07-03T15:44:10.879 INFO:tasks.workunit.client.0.smithi028.stdout:2/466: creat da/d42/fac x:0 0 0 2024-07-03T15:44:10.879 INFO:tasks.workunit.client.0.smithi028.stdout:2/467: chown da/d23/d30/l70 4272487 1 2024-07-03T15:44:10.880 INFO:tasks.workunit.client.0.smithi028.stdout:4/597: dwrite d2/f2b [0,4194304] 0 2024-07-03T15:44:10.880 INFO:tasks.workunit.client.0.smithi028.stdout:1/643: rmdir d4/d3c/d54/d79 39 2024-07-03T15:44:10.880 INFO:tasks.workunit.client.0.smithi028.stdout:1/644: write d4/d3c/d54/d79/f3e [754601,49495] 0 2024-07-03T15:44:10.881 INFO:tasks.workunit.client.0.smithi028.stdout:1/645: chown d4/d3c/d54/d79/l9c 13 1 2024-07-03T15:44:10.881 INFO:tasks.workunit.client.0.smithi028.stdout:2/468: dread da/d1e/d56/f7e [0,4194304] 0 2024-07-03T15:44:10.882 INFO:tasks.workunit.client.0.smithi028.stdout:5/600: symlink d0/dd/d14/dc0/lca 0 2024-07-03T15:44:10.883 INFO:tasks.workunit.client.0.smithi028.stdout:0/646: getdents d2/d3/d4d/d56 0 2024-07-03T15:44:10.884 INFO:tasks.workunit.client.0.smithi028.stdout:4/598: rmdir d2/d12 39 2024-07-03T15:44:10.886 INFO:tasks.workunit.client.0.smithi028.stdout:1/646: creat d4/de/da7/fe5 x:0 0 0 2024-07-03T15:44:10.888 INFO:tasks.workunit.client.0.smithi028.stdout:2/469: rmdir da/d23 39 2024-07-03T15:44:10.890 INFO:tasks.workunit.client.0.smithi028.stdout:5/601: creat d0/dd/d14/fcb x:0 0 0 2024-07-03T15:44:10.890 INFO:tasks.workunit.client.0.smithi028.stdout:1/647: write d4/d3c/f9e [4104139,79698] 0 2024-07-03T15:44:10.890 INFO:tasks.workunit.client.0.smithi028.stdout:1/648: creat d4/d2c/d3a/d43/fe6 x:0 0 0 2024-07-03T15:44:10.890 INFO:tasks.workunit.client.0.smithi028.stdout:1/649: read - d4/d3c/d54/d84/f9b zero size 2024-07-03T15:44:10.892 INFO:tasks.workunit.client.0.smithi028.stdout:2/470: dread da/d42/d47/f81 [0,4194304] 0 2024-07-03T15:44:10.892 INFO:tasks.workunit.client.0.smithi028.stdout:0/647: dread d2/d3/d4d/f57 [0,4194304] 0 2024-07-03T15:44:10.896 INFO:tasks.workunit.client.0.smithi028.stdout:4/599: unlink d2/d12/d2d/d2f/d3a/d6a/f71 0 2024-07-03T15:44:10.896 INFO:tasks.workunit.client.0.smithi028.stdout:4/600: write d2/d5a/f34 [1786741,112027] 0 2024-07-03T15:44:10.898 INFO:tasks.workunit.client.0.smithi028.stdout:3/507: rename d18/d1a/d5e/d60 to d18/d3a/d9d 0 2024-07-03T15:44:10.898 INFO:tasks.workunit.client.0.smithi028.stdout:3/508: chown l12 13 1 2024-07-03T15:44:10.898 INFO:tasks.workunit.client.0.smithi028.stdout:3/509: chown d18/f52 1 1 2024-07-03T15:44:10.898 INFO:tasks.workunit.client.0.smithi028.stdout:3/510: creat d18/d6f/f9e x:0 0 0 2024-07-03T15:44:10.900 INFO:tasks.workunit.client.0.smithi028.stdout:5/602: creat d0/dd/d1e/d94/fcc x:0 0 0 2024-07-03T15:44:10.900 INFO:tasks.workunit.client.0.smithi028.stdout:5/603: chown d0/dd/d13/d57/d26/f2e 51 1 2024-07-03T15:44:10.900 INFO:tasks.workunit.client.0.smithi028.stdout:5/604: fsync d0/dd/d13/d57/d26/d41/f44 0 2024-07-03T15:44:10.900 INFO:tasks.workunit.client.0.smithi028.stdout:5/605: truncate d0/dd/d14/fcb 876350 0 2024-07-03T15:44:10.902 INFO:tasks.workunit.client.0.smithi028.stdout:1/650: creat d4/d3c/d54/fe7 x:0 0 0 2024-07-03T15:44:10.905 INFO:tasks.workunit.client.0.smithi028.stdout:2/471: mknod da/d24/d2c/cad 0 2024-07-03T15:44:10.905 INFO:tasks.workunit.client.0.smithi028.stdout:2/472: truncate da/f12 1792753 0 2024-07-03T15:44:10.909 INFO:tasks.workunit.client.0.smithi028.stdout:0/648: mkdir d2/d3/dc4/d36/d45/dc8 0 2024-07-03T15:44:10.909 INFO:tasks.workunit.client.0.smithi028.stdout:0/649: write d2/f49 [38,91580] 0 2024-07-03T15:44:10.910 INFO:tasks.workunit.client.0.smithi028.stdout:0/650: chown d2/d1c/f83 10431 1 2024-07-03T15:44:10.910 INFO:tasks.workunit.client.0.smithi028.stdout:4/601: mkdir d2/d12/d42/d63/d6d/dc8 0 2024-07-03T15:44:10.910 INFO:tasks.workunit.client.0.smithi028.stdout:4/602: dread - d2/d12/d2e/d87/fc3 zero size 2024-07-03T15:44:10.911 INFO:tasks.workunit.client.0.smithi028.stdout:4/603: read d2/d12/f17 [358189,42888] 0 2024-07-03T15:44:10.912 INFO:tasks.workunit.client.0.smithi028.stdout:3/511: truncate d18/d1a/d1e/f57 158837 0 2024-07-03T15:44:10.914 INFO:tasks.workunit.client.0.smithi028.stdout:5/606: creat d0/dd/fcd x:0 0 0 2024-07-03T15:44:10.917 INFO:tasks.workunit.client.0.smithi028.stdout:2/473: creat da/d23/d6f/fae x:0 0 0 2024-07-03T15:44:10.919 INFO:tasks.workunit.client.0.smithi028.stdout:0/651: creat d2/d3/dc4/d36/d45/fc9 x:0 0 0 2024-07-03T15:44:10.921 INFO:tasks.workunit.client.0.smithi028.stdout:4/604: mkdir d2/d61/d92/dc9 0 2024-07-03T15:44:10.923 INFO:tasks.workunit.client.0.smithi028.stdout:3/512: rmdir d18/d1a/d5e/d8d 0 2024-07-03T15:44:10.925 INFO:tasks.workunit.client.0.smithi028.stdout:4/605: mknod d2/d12/d5b/d78/cca 0 2024-07-03T15:44:10.925 INFO:tasks.workunit.client.0.smithi028.stdout:4/606: truncate d2/d5a/f36 88359 0 2024-07-03T15:44:10.933 INFO:tasks.workunit.client.0.smithi028.stdout:2/474: dwrite da/d24/d2c/f97 [0,4194304] 0 2024-07-03T15:44:10.937 INFO:tasks.workunit.client.0.smithi028.stdout:1/651: rename d4/d3c/d54/d79/d16/d7c/dd5 to d4/d83/de8 0 2024-07-03T15:44:10.937 INFO:tasks.workunit.client.0.smithi028.stdout:5/607: rename d0/dd to d0/dd/d13/d57/d26/d41/db0/dce 22 2024-07-03T15:44:10.939 INFO:tasks.workunit.client.0.smithi028.stdout:0/652: rename d2/d3/dc4/d58/d82/d9b/lba to d2/d3/d4d/d56/d78/d9f/lca 0 2024-07-03T15:44:10.939 INFO:tasks.workunit.client.0.smithi028.stdout:5/608: mknod d0/dd/d13/d57/d26/d41/ccf 0 2024-07-03T15:44:10.939 INFO:tasks.workunit.client.0.smithi028.stdout:5/609: creat d0/dd/d13/d57/d30/fd0 x:0 0 0 2024-07-03T15:44:10.940 INFO:tasks.workunit.client.0.smithi028.stdout:1/652: creat d4/fe9 x:0 0 0 2024-07-03T15:44:10.942 INFO:tasks.workunit.client.0.smithi028.stdout:1/653: dread d4/d2c/d3a/d5c/daa/fb5 [0,4194304] 0 2024-07-03T15:44:10.942 INFO:tasks.workunit.client.0.smithi028.stdout:3/513: rename d18/d1a/d1e/d83 to d18/d1a/d1e/d58/d9f 0 2024-07-03T15:44:10.943 INFO:tasks.workunit.client.0.smithi028.stdout:0/653: symlink d2/d3/d4d/d56/d78/dbe/lcb 0 2024-07-03T15:44:10.945 INFO:tasks.workunit.client.0.smithi028.stdout:5/610: link d0/dd/f7e d0/dd/d13/d57/d77/fd1 0 2024-07-03T15:44:10.945 INFO:tasks.workunit.client.0.smithi028.stdout:5/611: dread - d0/dd/d13/d29/dc4/fa9 zero size 2024-07-03T15:44:10.946 INFO:tasks.workunit.client.0.smithi028.stdout:1/654: link d4/d83/de8/fdc d4/d83/de8/fea 0 2024-07-03T15:44:10.946 INFO:tasks.workunit.client.0.smithi028.stdout:1/655: fsync d4/d2c/d3a/d5c/d6c/d7d/d4e/d51/fa2 0 2024-07-03T15:44:10.947 INFO:tasks.workunit.client.0.smithi028.stdout:3/514: creat d18/d3a/d9a/fa0 x:0 0 0 2024-07-03T15:44:10.948 INFO:tasks.workunit.client.0.smithi028.stdout:1/656: rmdir d4/d2c/d3a/d5c/d6c/d7d/d59/d7f 39 2024-07-03T15:44:10.948 INFO:tasks.workunit.client.0.smithi028.stdout:1/657: fdatasync d4/d2c/d3a/d5c/d6c/d7d/d4e/fcd 0 2024-07-03T15:44:10.949 INFO:tasks.workunit.client.0.smithi028.stdout:1/658: write d4/d2c/d3a/d5c/daa/fb5 [166663,60012] 0 2024-07-03T15:44:10.951 INFO:tasks.workunit.client.0.smithi028.stdout:1/659: mknod d4/ceb 0 2024-07-03T15:44:10.955 INFO:tasks.workunit.client.0.smithi028.stdout:3/515: dread d18/d1a/f2a [0,4194304] 0 2024-07-03T15:44:10.955 INFO:tasks.workunit.client.0.smithi028.stdout:1/660: unlink d4/d2c/d3a/d5c/d6c/d7d/d4e/fcd 0 2024-07-03T15:44:10.958 INFO:tasks.workunit.client.0.smithi028.stdout:3/516: unlink l10 0 2024-07-03T15:44:10.962 INFO:tasks.workunit.client.0.smithi028.stdout:1/661: rmdir d4/de 39 2024-07-03T15:44:10.962 INFO:tasks.workunit.client.0.smithi028.stdout:1/662: write d4/f19 [1094463,38402] 0 2024-07-03T15:44:10.963 INFO:tasks.workunit.client.0.smithi028.stdout:0/654: dwrite d2/d3/dc4/d36/d45/f72 [0,4194304] 0 2024-07-03T15:44:10.965 INFO:tasks.workunit.client.0.smithi028.stdout:1/663: mknod d4/d3c/d54/d79/d16/d57/d66/cec 0 2024-07-03T15:44:10.965 INFO:tasks.workunit.client.0.smithi028.stdout:1/664: fsync d4/d3c/d54/d79/d16/f27 0 2024-07-03T15:44:10.966 INFO:tasks.workunit.client.0.smithi028.stdout:0/655: unlink d2/d3/c37 0 2024-07-03T15:44:10.969 INFO:tasks.workunit.client.0.smithi028.stdout:0/656: symlink d2/d3/d4d/d56/d78/da8/lcc 0 2024-07-03T15:44:10.970 INFO:tasks.workunit.client.0.smithi028.stdout:1/665: dread d4/d2c/d3a/d5c/daa/fb5 [0,4194304] 0 2024-07-03T15:44:10.973 INFO:tasks.workunit.client.0.smithi028.stdout:3/517: dwrite d18/d3a/f70 [0,4194304] 0 2024-07-03T15:44:10.973 INFO:tasks.workunit.client.0.smithi028.stdout:3/518: readlink d18/d1a/d1e/d58/l59 0 2024-07-03T15:44:10.977 INFO:tasks.workunit.client.0.smithi028.stdout:3/519: mknod d18/d6f/ca1 0 2024-07-03T15:44:10.980 INFO:tasks.workunit.client.0.smithi028.stdout:3/520: dread d18/d43/f4d [0,4194304] 0 2024-07-03T15:44:10.983 INFO:tasks.workunit.client.0.smithi028.stdout:3/521: rename d18/d3a/f3f to d18/d1a/d1e/d58/d9f/fa2 0 2024-07-03T15:44:10.983 INFO:tasks.workunit.client.0.smithi028.stdout:8/576: sync 2024-07-03T15:44:10.984 INFO:tasks.workunit.client.0.smithi028.stdout:0/657: dwrite d2/d1c/d6c/da9/fbf [0,4194304] 0 2024-07-03T15:44:10.984 INFO:tasks.workunit.client.0.smithi028.stdout:0/658: fdatasync d2/d1c/f2e 0 2024-07-03T15:44:10.988 INFO:tasks.workunit.client.0.smithi028.stdout:8/577: symlink d0/d41/d50/d3/d35/d68/d22/d40/db7/lc2 0 2024-07-03T15:44:10.988 INFO:tasks.workunit.client.0.smithi028.stdout:8/578: chown d0/d41/d50/d3/d35/d86/c77 109537437 1 2024-07-03T15:44:10.989 INFO:tasks.workunit.client.0.smithi028.stdout:8/579: write d0/d41/d50/d3/dc/d3f/d8e/f6d [291842,35479] 0 2024-07-03T15:44:10.989 INFO:tasks.workunit.client.0.smithi028.stdout:8/580: fdatasync d0/d41/d50/d3/dc/d3f/d8e/d5d/d9f/da6/f82 0 2024-07-03T15:44:10.989 INFO:tasks.workunit.client.0.smithi028.stdout:1/666: dwrite d4/d3c/d54/d79/f12 [0,4194304] 0 2024-07-03T15:44:11.003 INFO:tasks.workunit.client.0.smithi028.stdout:0/659: write d2/d3/dc4/d36/d45/f72 [3173384,108003] 0 2024-07-03T15:44:11.003 INFO:tasks.workunit.client.0.smithi028.stdout:3/522: dwrite d18/d1a/d1e/d58/f7e [0,4194304] 0 2024-07-03T15:44:11.003 INFO:tasks.workunit.client.0.smithi028.stdout:0/660: dread - d2/d3/fc6 zero size 2024-07-03T15:44:11.004 INFO:tasks.workunit.client.0.smithi028.stdout:8/581: write d0/d41/d50/d3/f3a [3037131,118139] 0 2024-07-03T15:44:11.006 INFO:tasks.workunit.client.0.smithi028.stdout:3/523: stat d18/d1a/f20 0 2024-07-03T15:44:11.008 INFO:tasks.workunit.client.0.smithi028.stdout:0/661: mkdir d2/d3/d4d/d56/dcd 0 2024-07-03T15:44:11.009 INFO:tasks.workunit.client.0.smithi028.stdout:8/582: stat d0/d41/d50/d3/dc/d33/d7a/c3e 0 2024-07-03T15:44:11.009 INFO:tasks.workunit.client.0.smithi028.stdout:8/583: fsync d0/d41/d50/d3/dc/d3f/d8e/d81/fbd 0 2024-07-03T15:44:11.010 INFO:tasks.workunit.client.0.smithi028.stdout:8/584: stat d0/d41/d50/d3/dc/d3f/d8e/d81/c9d 0 2024-07-03T15:44:11.014 INFO:tasks.workunit.client.0.smithi028.stdout:1/667: dwrite d4/d83/fe3 [0,4194304] 0 2024-07-03T15:44:11.014 INFO:tasks.workunit.client.0.smithi028.stdout:8/585: dread d0/d41/d50/d3/dc/d3f/d8e/f42 [0,4194304] 0 2024-07-03T15:44:11.014 INFO:tasks.workunit.client.0.smithi028.stdout:8/586: creat d0/d41/d50/d3/dc/d3f/d8e/d81/fc3 x:0 0 0 2024-07-03T15:44:11.014 INFO:tasks.workunit.client.0.smithi028.stdout:8/587: chown d0/f44 5 1 2024-07-03T15:44:11.015 INFO:tasks.workunit.client.0.smithi028.stdout:1/668: creat d4/d2c/d3a/d5c/fed x:0 0 0 2024-07-03T15:44:11.042 INFO:tasks.workunit.client.0.smithi028.stdout:3/524: dwrite d18/d1a/d1e/d58/f6c [0,4194304] 0 2024-07-03T15:44:11.089 INFO:tasks.workunit.client.0.smithi028.stdout:0/662: dwrite d2/d3/dc4/d36/d45/f7b [0,4194304] 0 2024-07-03T15:44:11.090 INFO:tasks.workunit.client.0.smithi028.stdout:8/588: dwrite d0/d41/d50/d3/d35/d68/d22/d40/f60 [0,4194304] 0 2024-07-03T15:44:11.090 INFO:tasks.workunit.client.0.smithi028.stdout:3/525: write d18/d1a/d1e/f41 [595716,66875] 0 2024-07-03T15:44:11.090 INFO:tasks.workunit.client.0.smithi028.stdout:3/526: write f6 [383346,67620] 0 2024-07-03T15:44:11.090 INFO:tasks.workunit.client.0.smithi028.stdout:3/527: stat d18/d1a/l56 0 2024-07-03T15:44:11.090 INFO:tasks.workunit.client.0.smithi028.stdout:8/589: dread d0/d41/d50/f27 [0,4194304] 0 2024-07-03T15:44:11.090 INFO:tasks.workunit.client.0.smithi028.stdout:8/590: write d0/d6c/fae [851359,56216] 0 2024-07-03T15:44:11.123 INFO:tasks.workunit.client.0.smithi028.stdout:0/663: symlink d2/d3/d4d/d56/d78/dbe/lce 0 2024-07-03T15:44:11.178 INFO:tasks.workunit.client.0.smithi028.stdout:3/528: mkdir d18/d1a/d5e/d66/da3 0 2024-07-03T15:44:11.179 INFO:tasks.workunit.client.0.smithi028.stdout:8/591: rename d0/d41/d50/c19 to d0/d41/d50/d3/dc/d3f/d8e/d4c/dc0/cc4 0 2024-07-03T15:44:11.179 INFO:tasks.workunit.client.0.smithi028.stdout:8/592: creat d0/d41/d50/d3/d35/d68/fc5 x:0 0 0 2024-07-03T15:44:11.197 INFO:tasks.workunit.client.0.smithi028.stdout:0/664: mkdir d2/d3/d4d/d56/d78/dcf 0 2024-07-03T15:44:11.200 INFO:tasks.workunit.client.0.smithi028.stdout:8/593: link d0/d6c/fae d0/d41/d50/d3/dc/d4d/fc6 0 2024-07-03T15:44:11.200 INFO:tasks.workunit.client.0.smithi028.stdout:8/594: stat d0/d41/d50/fbf 0 2024-07-03T15:44:11.200 INFO:tasks.workunit.client.0.smithi028.stdout:8/595: write d0/d41/d50/d3/dc/d4d/fc6 [1164109,66609] 0 2024-07-03T15:44:11.201 INFO:tasks.workunit.client.0.smithi028.stdout:8/596: write d0/d6c/fae [271721,46363] 0 2024-07-03T15:44:11.202 INFO:tasks.workunit.client.0.smithi028.stdout:3/529: creat d18/d6f/d78/d87/fa4 x:0 0 0 2024-07-03T15:44:11.202 INFO:tasks.workunit.client.0.smithi028.stdout:3/530: stat d18/d1a/f2a 0 2024-07-03T15:44:11.206 INFO:tasks.workunit.client.0.smithi028.stdout:8/597: rename d0/d41/d50/d3/dc/d33/d7a/l5e to d0/d41/d50/d3/dc/d33/d7a/lc7 0 2024-07-03T15:44:11.207 INFO:tasks.workunit.client.0.smithi028.stdout:3/531: creat d18/d1a/d1e/d58/d9f/fa5 x:0 0 0 2024-07-03T15:44:11.207 INFO:tasks.workunit.client.0.smithi028.stdout:3/532: dread - d18/d1a/d1e/d58/d9f/fa5 zero size 2024-07-03T15:44:11.208 INFO:tasks.workunit.client.0.smithi028.stdout:0/665: write d2/d3/d4d/fa2 [3872819,4185] 0 2024-07-03T15:44:11.211 INFO:tasks.workunit.client.0.smithi028.stdout:8/598: rename d0/d41/d50/d3/dc/d3f/d8e/d5d/d9f to d0/d41/d50/daf/dc8 0 2024-07-03T15:44:11.214 INFO:tasks.workunit.client.0.smithi028.stdout:3/533: symlink d18/d1a/d1e/d58/d9f/la6 0 2024-07-03T15:44:11.218 INFO:tasks.workunit.client.0.smithi028.stdout:0/666: mkdir d2/d3/d4d/d56/d78/da8/dd0 0 2024-07-03T15:44:11.218 INFO:tasks.workunit.client.0.smithi028.stdout:0/667: chown d2/d3/d4d/d56/l98 629593 1 2024-07-03T15:44:11.220 INFO:tasks.workunit.client.0.smithi028.stdout:8/599: mknod d0/d6c/da9/cc9 0 2024-07-03T15:44:11.221 INFO:tasks.workunit.client.0.smithi028.stdout:3/534: link d18/d1a/d5e/d66/c8b d18/d1a/d1e/d58/d9f/ca7 0 2024-07-03T15:44:11.223 INFO:tasks.workunit.client.0.smithi028.stdout:8/600: truncate d0/d41/d50/d3/dc/d33/d7a/f2f 4412770 0 2024-07-03T15:44:11.224 INFO:tasks.workunit.client.0.smithi028.stdout:8/601: read - d0/d41/d50/d3/dc/d3f/f61 zero size 2024-07-03T15:44:11.224 INFO:tasks.workunit.client.0.smithi028.stdout:3/535: mknod d18/d1a/d1e/d58/d9f/ca8 0 2024-07-03T15:44:11.226 INFO:tasks.workunit.client.0.smithi028.stdout:3/536: mkdir d18/d1a/d1e/d58/d9f/da9 0 2024-07-03T15:44:11.231 INFO:tasks.workunit.client.0.smithi028.stdout:0/668: dwrite d2/f49 [0,4194304] 0 2024-07-03T15:44:11.232 INFO:tasks.workunit.client.0.smithi028.stdout:0/669: truncate d2/d3/dc4/d36/d45/fc9 640105 0 2024-07-03T15:44:11.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:10 smithi148 ceph-mon[36056]: mgrmap e24: smithi148.rjzsar(active, since 7s), standbys: smithi028.vtqrtz 2024-07-03T15:44:11.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:10 smithi148 ceph-mon[36056]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "mgr metadata", "who": "smithi028.vtqrtz", "id": "smithi028.vtqrtz"}]: dispatch 2024-07-03T15:44:11.233 INFO:tasks.workunit.client.0.smithi028.stdout:3/537: symlink d18/d3a/d9d/laa 0 2024-07-03T15:44:11.234 INFO:tasks.workunit.client.0.smithi028.stdout:3/538: fsync d18/d3a/f39 0 2024-07-03T15:44:11.239 INFO:tasks.workunit.client.0.smithi028.stdout:9/526: sync 2024-07-03T15:44:11.239 INFO:tasks.workunit.client.0.smithi028.stdout:6/559: sync 2024-07-03T15:44:11.239 INFO:tasks.workunit.client.0.smithi028.stdout:9/527: write d1/d16/d57/f66 [1338567,62052] 0 2024-07-03T15:44:11.240 INFO:tasks.workunit.client.0.smithi028.stdout:3/539: symlink d18/d1a/d1e/lab 0 2024-07-03T15:44:11.241 INFO:tasks.workunit.client.0.smithi028.stdout:3/540: write d18/d3a/d9a/fa0 [401673,38604] 0 2024-07-03T15:44:11.241 INFO:tasks.workunit.client.0.smithi028.stdout:3/541: chown d18/d1a/d1e/d58/c65 100 1 2024-07-03T15:44:11.246 INFO:tasks.workunit.client.0.smithi028.stdout:3/542: creat d18/d1a/d5e/d66/da3/fac x:0 0 0 2024-07-03T15:44:11.253 INFO:tasks.workunit.client.0.smithi028.stdout:6/560: dread d4/f11 [8388608,4194304] 0 2024-07-03T15:44:11.257 INFO:tasks.workunit.client.0.smithi028.stdout:8/602: dwrite d0/d41/d50/d3/dc/d3f/d8e/d81/fbd [0,4194304] 0 2024-07-03T15:44:11.258 INFO:tasks.workunit.client.0.smithi028.stdout:0/670: dwrite d2/d3/d4d/d56/fb8 [0,4194304] 0 2024-07-03T15:44:11.258 INFO:tasks.workunit.client.0.smithi028.stdout:8/603: write d0/d41/d50/d3/dc/d3f/d8e/d4c/f7c [1865998,121564] 0 2024-07-03T15:44:11.258 INFO:tasks.workunit.client.0.smithi028.stdout:0/671: chown d2/d3/d4d/d56/d78/d9f/l8e 0 1 2024-07-03T15:44:11.260 INFO:tasks.workunit.client.0.smithi028.stdout:6/561: dread d4/f29 [0,4194304] 0 2024-07-03T15:44:11.268 INFO:tasks.workunit.client.0.smithi028.stdout:0/672: symlink d2/d1c/d6c/ld1 0 2024-07-03T15:44:11.271 INFO:tasks.workunit.client.0.smithi028.stdout:3/543: dwrite d18/d1a/d1e/f38 [0,4194304] 0 2024-07-03T15:44:11.273 INFO:tasks.workunit.client.0.smithi028.stdout:8/604: dwrite d0/d41/f88 [0,4194304] 0 2024-07-03T15:44:11.273 INFO:tasks.workunit.client.0.smithi028.stdout:8/605: chown d0/d6c/cab 3578257 1 2024-07-03T15:44:11.273 INFO:tasks.workunit.client.0.smithi028.stdout:8/606: stat d0/d41/d50/le 0 2024-07-03T15:44:11.280 INFO:tasks.workunit.client.0.smithi028.stdout:6/562: mkdir d4/da/d38/d60/dc5 0 2024-07-03T15:44:11.288 INFO:tasks.workunit.client.0.smithi028.stdout:8/607: rename d0/d41/d50/f78 to d0/d41/d50/d3/dc/d3f/d8e/fca 0 2024-07-03T15:44:11.289 INFO:tasks.workunit.client.0.smithi028.stdout:0/673: dwrite d2/d3/d4d/f64 [0,4194304] 0 2024-07-03T15:44:11.289 INFO:tasks.workunit.client.0.smithi028.stdout:6/563: truncate d4/f7b 138428 0 2024-07-03T15:44:11.290 INFO:tasks.workunit.client.0.smithi028.stdout:8/608: creat d0/d6c/fcb x:0 0 0 2024-07-03T15:44:11.292 INFO:tasks.workunit.client.0.smithi028.stdout:3/544: dwrite f14 [0,4194304] 0 2024-07-03T15:44:11.293 INFO:tasks.workunit.client.0.smithi028.stdout:3/545: dread f9 [0,4194304] 0 2024-07-03T15:44:11.294 INFO:tasks.workunit.client.0.smithi028.stdout:0/674: stat d2/d3/dc4/d58/f6a 0 2024-07-03T15:44:11.296 INFO:tasks.workunit.client.0.smithi028.stdout:6/564: creat d4/da/d56/fc6 x:0 0 0 2024-07-03T15:44:11.301 INFO:tasks.workunit.client.0.smithi028.stdout:8/609: write d0/d41/d50/d3/dc/d33/d7a/f39 [2151406,110329] 0 2024-07-03T15:44:11.304 INFO:tasks.workunit.client.0.smithi028.stdout:3/546: truncate f17 293497 0 2024-07-03T15:44:11.308 INFO:tasks.workunit.client.0.smithi028.stdout:0/675: link d2/d3/d4d/d56/l9c d2/d6d/ld2 0 2024-07-03T15:44:11.309 INFO:tasks.workunit.client.0.smithi028.stdout:6/565: rename d4/d15/d9d/fb7 to d4/d15/fc7 0 2024-07-03T15:44:11.313 INFO:tasks.workunit.client.0.smithi028.stdout:0/676: mknod d2/d3/dc4/d58/d6e/cd3 0 2024-07-03T15:44:11.315 INFO:tasks.workunit.client.0.smithi028.stdout:8/610: dwrite d0/d41/d50/d3/dc/d3f/d8e/fca [0,4194304] 0 2024-07-03T15:44:11.316 INFO:tasks.workunit.client.0.smithi028.stdout:6/566: link d4/da/d47/d9f/cb6 d4/da/d38/d59/d91/cc8 0 2024-07-03T15:44:11.318 INFO:tasks.workunit.client.0.smithi028.stdout:3/547: truncate fe 1047492 0 2024-07-03T15:44:11.321 INFO:tasks.workunit.client.0.smithi028.stdout:0/677: creat d2/d3/dc4/fd4 x:0 0 0 2024-07-03T15:44:11.321 INFO:tasks.workunit.client.0.smithi028.stdout:0/678: write d2/d3/dc4/f38 [2766625,117902] 0 2024-07-03T15:44:11.327 INFO:tasks.workunit.client.0.smithi028.stdout:8/611: rename d0/d41/d50/d3/dc/f63 to d0/d41/d50/d3/dc/fcc 0 2024-07-03T15:44:11.328 INFO:tasks.workunit.client.0.smithi028.stdout:6/567: creat d4/fc9 x:0 0 0 2024-07-03T15:44:11.332 INFO:tasks.workunit.client.0.smithi028.stdout:3/548: mknod d18/d6f/cad 0 2024-07-03T15:44:11.337 INFO:tasks.workunit.client.0.smithi028.stdout:0/679: dwrite d2/d3/dc4/d36/d45/fc9 [0,4194304] 0 2024-07-03T15:44:11.342 INFO:tasks.workunit.client.0.smithi028.stdout:8/612: truncate d0/d41/f88 423561 0 2024-07-03T15:44:11.345 INFO:tasks.workunit.client.0.smithi028.stdout:3/549: symlink d18/d1a/d1e/d58/d9f/lae 0 2024-07-03T15:44:11.345 INFO:tasks.workunit.client.0.smithi028.stdout:3/550: truncate d18/d1a/d64/f6e 400831 0 2024-07-03T15:44:11.346 INFO:tasks.workunit.client.0.smithi028.stdout:7/423: sync 2024-07-03T15:44:11.351 INFO:tasks.workunit.client.0.smithi028.stdout:0/680: creat d2/d3/d4d/d96/dc2/fd5 x:0 0 0 2024-07-03T15:44:11.351 INFO:tasks.workunit.client.0.smithi028.stdout:6/568: dwrite d4/d12/f21 [0,4194304] 0 2024-07-03T15:44:11.351 INFO:tasks.workunit.client.0.smithi028.stdout:0/681: chown d2/d3/c3c 7704 1 2024-07-03T15:44:11.351 INFO:tasks.workunit.client.0.smithi028.stdout:6/569: readlink d4/d12/d2c/l35 0 2024-07-03T15:44:11.352 INFO:tasks.workunit.client.0.smithi028.stdout:0/682: write d2/d3/dc4/d36/f88 [744703,126950] 0 2024-07-03T15:44:11.361 INFO:tasks.workunit.client.0.smithi028.stdout:6/570: rmdir d4/d12/d2c/d31 39 2024-07-03T15:44:11.361 INFO:tasks.workunit.client.0.smithi028.stdout:6/571: chown d4/da/c14 33396002 1 2024-07-03T15:44:11.363 INFO:tasks.workunit.client.0.smithi028.stdout:0/683: dread d2/d3/dc4/f66 [0,4194304] 0 2024-07-03T15:44:11.364 INFO:tasks.workunit.client.0.smithi028.stdout:8/613: dwrite d0/d41/d50/d3/dc/d3f/d8e/f6d [0,4194304] 0 2024-07-03T15:44:11.366 INFO:tasks.workunit.client.0.smithi028.stdout:7/424: dwrite d9/d29/f42 [0,4194304] 0 2024-07-03T15:44:11.366 INFO:tasks.workunit.client.0.smithi028.stdout:0/684: unlink d2/d3/dc4/d58/c59 0 2024-07-03T15:44:11.366 INFO:tasks.workunit.client.0.smithi028.stdout:0/685: fdatasync d2/d3/dc4/d36/d45/da3/fab 0 2024-07-03T15:44:11.370 INFO:tasks.workunit.client.0.smithi028.stdout:7/425: creat d9/d1c/d45/d71/f7f x:0 0 0 2024-07-03T15:44:11.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:10 smithi028 ceph-mon[29148]: mgrmap e24: smithi148.rjzsar(active, since 7s), standbys: smithi028.vtqrtz 2024-07-03T15:44:11.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:10 smithi028 ceph-mon[29148]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "mgr metadata", "who": "smithi028.vtqrtz", "id": "smithi028.vtqrtz"}]: dispatch 2024-07-03T15:44:11.372 INFO:tasks.workunit.client.0.smithi028.stdout:0/686: write d2/d3/dc4/f66 [2589048,73559] 0 2024-07-03T15:44:11.372 INFO:tasks.workunit.client.0.smithi028.stdout:0/687: write d2/d3/f8d [1503533,75410] 0 2024-07-03T15:44:11.376 INFO:tasks.workunit.client.0.smithi028.stdout:7/426: creat d9/d1c/d7c/f80 x:0 0 0 2024-07-03T15:44:11.377 INFO:tasks.workunit.client.0.smithi028.stdout:6/572: dwrite d4/da/d56/f73 [0,4194304] 0 2024-07-03T15:44:11.381 INFO:tasks.workunit.client.0.smithi028.stdout:0/688: mknod d2/d3/d4d/d96/dc2/cd6 0 2024-07-03T15:44:11.385 INFO:tasks.workunit.client.0.smithi028.stdout:7/427: dread d9/d1c/d52/d6a/f6c [0,4194304] 0 2024-07-03T15:44:11.391 INFO:tasks.workunit.client.0.smithi028.stdout:0/689: mkdir d2/d3/d4d/d56/d78/da8/dd7 0 2024-07-03T15:44:11.392 INFO:tasks.workunit.client.0.smithi028.stdout:3/551: dwrite f17 [0,4194304] 0 2024-07-03T15:44:11.392 INFO:tasks.workunit.client.0.smithi028.stdout:3/552: stat d18/d3a/d9d/c84 0 2024-07-03T15:44:11.392 INFO:tasks.workunit.client.0.smithi028.stdout:0/690: symlink d2/d3/d4d/d96/ld8 0 2024-07-03T15:44:11.395 INFO:tasks.workunit.client.0.smithi028.stdout:3/553: link d18/d1a/d64/f9b d18/d3a/d9d/d8c/faf 0 2024-07-03T15:44:11.395 INFO:tasks.workunit.client.0.smithi028.stdout:7/428: dread d9/d1c/f3f [4194304,4194304] 0 2024-07-03T15:44:11.395 INFO:tasks.workunit.client.0.smithi028.stdout:7/429: creat d9/d29/f81 x:0 0 0 2024-07-03T15:44:11.395 INFO:tasks.workunit.client.0.smithi028.stdout:7/430: fsync d9/d1c/d52/d6a/f7b 0 2024-07-03T15:44:11.396 INFO:tasks.workunit.client.0.smithi028.stdout:7/431: dread d9/d29/f4f [0,4194304] 0 2024-07-03T15:44:11.396 INFO:tasks.workunit.client.0.smithi028.stdout:7/432: chown d9/d1c/f20 7 1 2024-07-03T15:44:11.397 INFO:tasks.workunit.client.0.smithi028.stdout:3/554: truncate d18/d6f/f71 2684529 0 2024-07-03T15:44:11.402 INFO:tasks.workunit.client.0.smithi028.stdout:1/669: sync 2024-07-03T15:44:11.402 INFO:tasks.workunit.client.0.smithi028.stdout:2/475: sync 2024-07-03T15:44:11.402 INFO:tasks.workunit.client.0.smithi028.stdout:4/607: sync 2024-07-03T15:44:11.402 INFO:tasks.workunit.client.0.smithi028.stdout:1/670: chown d4/d3c/d54/d79/d16/d57/d66/dd6 22223717 1 2024-07-03T15:44:11.402 INFO:tasks.workunit.client.0.smithi028.stdout:2/476: creat da/d24/d2c/faf x:0 0 0 2024-07-03T15:44:11.402 INFO:tasks.workunit.client.0.smithi028.stdout:5/612: sync 2024-07-03T15:44:11.402 INFO:tasks.workunit.client.0.smithi028.stdout:2/477: creat da/d42/fb0 x:0 0 0 2024-07-03T15:44:11.405 INFO:tasks.workunit.client.0.smithi028.stdout:4/608: mknod d2/d12/d2d/ccb 0 2024-07-03T15:44:11.410 INFO:tasks.workunit.client.0.smithi028.stdout:1/671: mknod d4/d2c/d3a/d43/cee 0 2024-07-03T15:44:11.412 INFO:tasks.workunit.client.0.smithi028.stdout:7/433: dread d9/fa [0,4194304] 0 2024-07-03T15:44:11.412 INFO:tasks.workunit.client.0.smithi028.stdout:0/691: dwrite d2/d1c/d6c/f85 [0,4194304] 0 2024-07-03T15:44:11.414 INFO:tasks.workunit.client.0.smithi028.stdout:2/478: rename da/d23/d6f/f7c to da/d42/d47/d78/fb1 0 2024-07-03T15:44:11.414 INFO:tasks.workunit.client.0.smithi028.stdout:2/479: fdatasync da/f12 0 2024-07-03T15:44:11.415 INFO:tasks.workunit.client.0.smithi028.stdout:1/672: mknod d4/d2c/d3a/d5c/cef 0 2024-07-03T15:44:11.417 INFO:tasks.workunit.client.0.smithi028.stdout:7/434: symlink d9/d1c/d52/l82 0 2024-07-03T15:44:11.418 INFO:tasks.workunit.client.0.smithi028.stdout:0/692: dread d2/d3/dc4/d36/f88 [0,4194304] 0 2024-07-03T15:44:11.418 INFO:tasks.workunit.client.0.smithi028.stdout:0/693: truncate d2/d3/dc4/fc1 92128 0 2024-07-03T15:44:11.419 INFO:tasks.workunit.client.0.smithi028.stdout:1/673: mkdir d4/d2c/d3a/d5c/daa/df0 0 2024-07-03T15:44:11.422 INFO:tasks.workunit.client.0.smithi028.stdout:1/674: unlink d4/de/d1c/d55/l8a 0 2024-07-03T15:44:11.422 INFO:tasks.workunit.client.0.smithi028.stdout:1/675: write d4/de/d1c/f3d [1937192,108178] 0 2024-07-03T15:44:11.424 INFO:tasks.workunit.client.0.smithi028.stdout:1/676: rename d4/d2c/d3a/d5c/d6c/db2 to d4/de/d1c/df1 0 2024-07-03T15:44:11.424 INFO:tasks.workunit.client.0.smithi028.stdout:1/677: write d4/d3c/d54/d84/f9b [536669,7816] 0 2024-07-03T15:44:11.432 INFO:tasks.workunit.client.0.smithi028.stdout:2/480: write da/d42/f46 [3710561,81659] 0 2024-07-03T15:44:11.433 INFO:tasks.workunit.client.0.smithi028.stdout:0/694: dread d2/d3/dc4/d58/d6e/f94 [0,4194304] 0 2024-07-03T15:44:11.434 INFO:tasks.workunit.client.0.smithi028.stdout:0/695: write d2/d3/f2f [3161096,8384] 0 2024-07-03T15:44:11.434 INFO:tasks.workunit.client.0.smithi028.stdout:0/696: write d2/d3/f8d [2023869,116892] 0 2024-07-03T15:44:11.434 INFO:tasks.workunit.client.0.smithi028.stdout:2/481: link da/la1 da/d42/d47/d78/lb2 0 2024-07-03T15:44:11.435 INFO:tasks.workunit.client.0.smithi028.stdout:0/697: symlink d2/d3/dc4/d58/d82/d9b/ld9 0 2024-07-03T15:44:11.438 INFO:tasks.workunit.client.0.smithi028.stdout:3/555: dwrite d18/d1a/f4f [4194304,4194304] 0 2024-07-03T15:44:11.438 INFO:tasks.workunit.client.0.smithi028.stdout:3/556: chown d18/d1a/d1e/lab 26 1 2024-07-03T15:44:11.438 INFO:tasks.workunit.client.0.smithi028.stdout:0/698: rename d2/d1c/d6c/ld1 to d2/d3/d4d/d56/d78/lda 0 2024-07-03T15:44:11.438 INFO:tasks.workunit.client.0.smithi028.stdout:4/609: dwrite d2/d5a/d45/f4c [0,4194304] 0 2024-07-03T15:44:11.439 INFO:tasks.workunit.client.0.smithi028.stdout:1/678: write d4/d3c/d54/d79/f12 [4166088,41335] 0 2024-07-03T15:44:11.439 INFO:tasks.workunit.client.0.smithi028.stdout:3/557: mknod d18/d6f/d78/cb0 0 2024-07-03T15:44:11.440 INFO:tasks.workunit.client.0.smithi028.stdout:3/558: write d18/d1a/d1e/f41 [2019326,19076] 0 2024-07-03T15:44:11.440 INFO:tasks.workunit.client.0.smithi028.stdout:3/559: creat d18/d1a/d1e/d58/d9f/fb1 x:0 0 0 2024-07-03T15:44:11.440 INFO:tasks.workunit.client.0.smithi028.stdout:5/613: dwrite d0/dd/df/fc2 [0,4194304] 0 2024-07-03T15:44:11.440 INFO:tasks.workunit.client.0.smithi028.stdout:5/614: read - d0/dd/d13/d29/dc4/fa9 zero size 2024-07-03T15:44:11.445 INFO:tasks.workunit.client.0.smithi028.stdout:7/435: rmdir d9/d29 39 2024-07-03T15:44:11.448 INFO:tasks.workunit.client.0.smithi028.stdout:3/560: write d18/d43/f4d [1573763,63524] 0 2024-07-03T15:44:11.453 INFO:tasks.workunit.client.0.smithi028.stdout:0/699: write d2/d3/dc4/d36/d45/f46 [8246455,82592] 0 2024-07-03T15:44:11.453 INFO:tasks.workunit.client.0.smithi028.stdout:0/700: fdatasync d2/d1c/f83 0 2024-07-03T15:44:11.456 INFO:tasks.workunit.client.0.smithi028.stdout:3/561: fsync d18/d43/f4d 0 2024-07-03T15:44:11.456 INFO:tasks.workunit.client.0.smithi028.stdout:3/562: write d18/d1a/d1e/d58/f96 [1045229,20264] 0 2024-07-03T15:44:11.458 INFO:tasks.workunit.client.0.smithi028.stdout:4/610: getdents d2/d3e 0 2024-07-03T15:44:11.465 INFO:tasks.workunit.client.0.smithi028.stdout:4/611: truncate d2/d3e/d9f/fb4 784309 0 2024-07-03T15:44:11.465 INFO:tasks.workunit.client.0.smithi028.stdout:4/612: dread - d2/d12/d2d/d58/f70 zero size 2024-07-03T15:44:11.465 INFO:tasks.workunit.client.0.smithi028.stdout:4/613: fdatasync d2/d12/f17 0 2024-07-03T15:44:11.466 INFO:tasks.workunit.client.0.smithi028.stdout:4/614: creat d2/d12/da6/fcc x:0 0 0 2024-07-03T15:44:11.466 INFO:tasks.workunit.client.0.smithi028.stdout:4/615: read d2/d5a/f9 [86399,87061] 0 2024-07-03T15:44:11.466 INFO:tasks.workunit.client.0.smithi028.stdout:4/616: read d2/d12/d2d/d58/f5e [1106817,129223] 0 2024-07-03T15:44:11.469 INFO:tasks.workunit.client.0.smithi028.stdout:4/617: mkdir d2/d61/d9c/dcd 0 2024-07-03T15:44:11.469 INFO:tasks.workunit.client.0.smithi028.stdout:1/679: dwrite d4/d3c/f9e [0,4194304] 0 2024-07-03T15:44:11.471 INFO:tasks.workunit.client.0.smithi028.stdout:7/436: dwrite d9/d29/f81 [0,4194304] 0 2024-07-03T15:44:11.471 INFO:tasks.workunit.client.0.smithi028.stdout:7/437: chown d9/f1a 12873 1 2024-07-03T15:44:11.471 INFO:tasks.workunit.client.0.smithi028.stdout:1/680: symlink d4/d2c/d3a/d5c/lf2 0 2024-07-03T15:44:11.471 INFO:tasks.workunit.client.0.smithi028.stdout:1/681: write d4/d2c/d3a/d5c/d6c/d7d/d59/f2d [701282,16027] 0 2024-07-03T15:44:11.471 INFO:tasks.workunit.client.0.smithi028.stdout:1/682: chown d4/d83/de8 159 1 2024-07-03T15:44:11.471 INFO:tasks.workunit.client.0.smithi028.stdout:1/683: chown d4/de/fc5 38 1 2024-07-03T15:44:11.476 INFO:tasks.workunit.client.0.smithi028.stdout:4/618: dread d2/f52 [0,4194304] 0 2024-07-03T15:44:11.476 INFO:tasks.workunit.client.0.smithi028.stdout:4/619: write d2/d12/d5b/d78/f79 [1003130,39429] 0 2024-07-03T15:44:11.477 INFO:tasks.workunit.client.0.smithi028.stdout:1/684: creat d4/de/d1c/df1/ff3 x:0 0 0 2024-07-03T15:44:11.477 INFO:tasks.workunit.client.0.smithi028.stdout:1/685: write d4/fd [367039,82177] 0 2024-07-03T15:44:11.477 INFO:tasks.workunit.client.0.smithi028.stdout:1/686: stat d4/d2c/d3a/d5c/d6c/d7d/d4e/fb9 0 2024-07-03T15:44:11.477 INFO:tasks.workunit.client.0.smithi028.stdout:3/563: read d18/f42 [530022,7743] 0 2024-07-03T15:44:11.482 INFO:tasks.workunit.client.0.smithi028.stdout:9/528: sync 2024-07-03T15:44:11.482 INFO:tasks.workunit.client.0.smithi028.stdout:9/529: creat d1/d10/d28/d4e/d4f/fb3 x:0 0 0 2024-07-03T15:44:11.482 INFO:tasks.workunit.client.0.smithi028.stdout:9/530: write d1/d17/f9d [1020624,35628] 0 2024-07-03T15:44:11.482 INFO:tasks.workunit.client.0.smithi028.stdout:4/620: getdents d2/d3e/dae 0 2024-07-03T15:44:11.486 INFO:tasks.workunit.client.0.smithi028.stdout:1/687: dread d4/de/fc3 [0,4194304] 0 2024-07-03T15:44:11.486 INFO:tasks.workunit.client.0.smithi028.stdout:1/688: truncate d4/d2c/d3a/d5c/fcf 374250 0 2024-07-03T15:44:11.488 INFO:tasks.workunit.client.0.smithi028.stdout:9/531: unlink d1/f2 0 2024-07-03T15:44:11.489 INFO:tasks.workunit.client.0.smithi028.stdout:9/532: write d1/d10/d76/fae [998883,67227] 0 2024-07-03T15:44:11.489 INFO:tasks.workunit.client.0.smithi028.stdout:9/533: chown d1/d17 66 1 2024-07-03T15:44:11.489 INFO:tasks.workunit.client.0.smithi028.stdout:9/534: chown d1/d10/d28/d4e/d4f/fb3 4320657 1 2024-07-03T15:44:11.489 INFO:tasks.workunit.client.0.smithi028.stdout:4/621: rename d2/d12/d2c/f7d to d2/d3e/d9f/d74/fce 0 2024-07-03T15:44:11.490 INFO:tasks.workunit.client.0.smithi028.stdout:4/622: creat d2/d12/d2d/d2f/d3a/d6a/fcf x:0 0 0 2024-07-03T15:44:11.490 INFO:tasks.workunit.client.0.smithi028.stdout:3/564: dwrite d18/d6f/f9e [0,4194304] 0 2024-07-03T15:44:11.491 INFO:tasks.workunit.client.0.smithi028.stdout:1/689: truncate d4/d83/f8b 416659 0 2024-07-03T15:44:11.493 INFO:tasks.workunit.client.0.smithi028.stdout:9/535: creat d1/d8e/dac/fb4 x:0 0 0 2024-07-03T15:44:11.494 INFO:tasks.workunit.client.0.smithi028.stdout:9/536: dread - d1/d16/d57/d60/fb0 zero size 2024-07-03T15:44:11.494 INFO:tasks.workunit.client.0.smithi028.stdout:9/537: chown d1/d16/f1c 10005177 1 2024-07-03T15:44:11.497 INFO:tasks.workunit.client.0.smithi028.stdout:4/623: symlink d2/d12/d42/d63/d6d/dc8/ld0 0 2024-07-03T15:44:11.497 INFO:tasks.workunit.client.0.smithi028.stdout:4/624: chown d2/d12/d42/c51 92 1 2024-07-03T15:44:11.499 INFO:tasks.workunit.client.0.smithi028.stdout:4/625: dread d2/d12/f68 [0,4194304] 0 2024-07-03T15:44:11.499 INFO:tasks.workunit.client.0.smithi028.stdout:4/626: stat d2/d5a/cc2 0 2024-07-03T15:44:11.505 INFO:tasks.workunit.client.0.smithi028.stdout:9/538: dread d1/dd/f59 [0,4194304] 0 2024-07-03T15:44:11.508 INFO:tasks.workunit.client.0.smithi028.stdout:1/690: symlink d4/d3c/d54/d79/d16/db6/ddb/lf4 0 2024-07-03T15:44:11.509 INFO:tasks.workunit.client.0.smithi028.stdout:9/539: truncate d1/d4a/f92 4333001 0 2024-07-03T15:44:11.509 INFO:tasks.workunit.client.0.smithi028.stdout:9/540: creat d1/d10/d28/d45/d61/d8c/d77/fb5 x:0 0 0 2024-07-03T15:44:11.510 INFO:tasks.workunit.client.0.smithi028.stdout:3/565: dread d18/d6f/f71 [0,4194304] 0 2024-07-03T15:44:11.510 INFO:tasks.workunit.client.0.smithi028.stdout:1/691: mknod d4/d2c/cf5 0 2024-07-03T15:44:11.510 INFO:tasks.workunit.client.0.smithi028.stdout:1/692: write d4/de/f6f [4960143,44702] 0 2024-07-03T15:44:11.511 INFO:tasks.workunit.client.0.smithi028.stdout:1/693: write d4/d2c/d3a/f68 [1915939,41624] 0 2024-07-03T15:44:11.511 INFO:tasks.workunit.client.0.smithi028.stdout:1/694: chown d4/d2c/d3a/d5c/daa/cda 1088802335 1 2024-07-03T15:44:11.511 INFO:tasks.workunit.client.0.smithi028.stdout:1/695: read d4/d2c/d3a/d5c/daa/fb5 [267408,41466] 0 2024-07-03T15:44:11.512 INFO:tasks.workunit.client.0.smithi028.stdout:1/696: dread d4/de/f10 [0,4194304] 0 2024-07-03T15:44:11.512 INFO:tasks.workunit.client.0.smithi028.stdout:1/697: dread - d4/d2c/d3a/d5c/fbf zero size 2024-07-03T15:44:11.512 INFO:tasks.workunit.client.0.smithi028.stdout:1/698: readlink d4/d3c/d54/d79/d16/d57/d66/la0 0 2024-07-03T15:44:11.515 INFO:tasks.workunit.client.0.smithi028.stdout:9/541: getdents d1/d10/d28/d45/d56 0 2024-07-03T15:44:11.516 INFO:tasks.workunit.client.0.smithi028.stdout:4/627: dwrite d2/d12/d5b/f7a [0,4194304] 0 2024-07-03T15:44:11.516 INFO:tasks.workunit.client.0.smithi028.stdout:4/628: chown d2/l95 179269 1 2024-07-03T15:44:11.517 INFO:tasks.workunit.client.0.smithi028.stdout:1/699: rename d4/d3c/d54/d79/d16/d7c/cd2 to d4/cf6 0 2024-07-03T15:44:11.517 INFO:tasks.workunit.client.0.smithi028.stdout:1/700: readlink d4/d2c/d3a/d5c/d6c/d7d/l4a 0 2024-07-03T15:44:11.518 INFO:tasks.workunit.client.0.smithi028.stdout:9/542: link d1/d10/d28/d45/d61/fa0 d1/dd/fb6 0 2024-07-03T15:44:11.521 INFO:tasks.workunit.client.0.smithi028.stdout:3/566: dread d18/f51 [0,4194304] 0 2024-07-03T15:44:11.521 INFO:tasks.workunit.client.0.smithi028.stdout:3/567: dread - d18/d1a/d1e/d58/d9f/fb1 zero size 2024-07-03T15:44:11.522 INFO:tasks.workunit.client.0.smithi028.stdout:4/629: rename d2/d5a/f9 to d2/d61/d9d/fd1 0 2024-07-03T15:44:11.522 INFO:tasks.workunit.client.0.smithi028.stdout:1/701: symlink d4/d2c/d3a/lf7 0 2024-07-03T15:44:11.523 INFO:tasks.workunit.client.0.smithi028.stdout:3/568: dread f9 [0,4194304] 0 2024-07-03T15:44:11.527 INFO:tasks.workunit.client.0.smithi028.stdout:1/702: symlink d4/d2c/d3a/d5c/d6c/d7d/d59/lf8 0 2024-07-03T15:44:11.531 INFO:tasks.workunit.client.0.smithi028.stdout:3/569: dread d18/d43/f61 [0,4194304] 0 2024-07-03T15:44:11.532 INFO:tasks.workunit.client.0.smithi028.stdout:1/703: mknod d4/d83/cf9 0 2024-07-03T15:44:11.534 INFO:tasks.workunit.client.0.smithi028.stdout:9/543: rename d1/d16/d57/d60 to d1/d10/d28/d45/d61/d8c/d77/db7 0 2024-07-03T15:44:11.535 INFO:tasks.workunit.client.0.smithi028.stdout:4/630: dwrite d2/d12/da6/fcc [0,4194304] 0 2024-07-03T15:44:11.535 INFO:tasks.workunit.client.0.smithi028.stdout:4/631: fsync d2/d12/d2d/d2f/d3a/d6a/fcf 0 2024-07-03T15:44:11.535 INFO:tasks.workunit.client.0.smithi028.stdout:1/704: dread d4/d83/fac [0,4194304] 0 2024-07-03T15:44:11.535 INFO:tasks.workunit.client.0.smithi028.stdout:1/705: truncate d4/d3c/d54/f8d 4685288 0 2024-07-03T15:44:11.539 INFO:tasks.workunit.client.0.smithi028.stdout:3/570: dread d18/d1a/f20 [4194304,4194304] 0 2024-07-03T15:44:11.541 INFO:tasks.workunit.client.0.smithi028.stdout:4/632: creat d2/d12/d42/d63/d6d/fd2 x:0 0 0 2024-07-03T15:44:11.543 INFO:tasks.workunit.client.0.smithi028.stdout:1/706: symlink d4/de/lfa 0 2024-07-03T15:44:11.545 INFO:tasks.workunit.client.0.smithi028.stdout:9/544: getdents d1/d16/d57/d5f/d64 0 2024-07-03T15:44:11.547 INFO:tasks.workunit.client.0.smithi028.stdout:3/571: creat d18/d1a/d5e/d66/fb2 x:0 0 0 2024-07-03T15:44:11.549 INFO:tasks.workunit.client.0.smithi028.stdout:4/633: rmdir d2/d61/d92/dc9 0 2024-07-03T15:44:11.551 INFO:tasks.workunit.client.0.smithi028.stdout:1/707: read d4/d2c/f75 [2893474,16793] 0 2024-07-03T15:44:11.554 INFO:tasks.workunit.client.0.smithi028.stdout:9/545: dread d1/dd/f90 [0,4194304] 0 2024-07-03T15:44:11.562 INFO:tasks.workunit.client.0.smithi028.stdout:3/572: link d18/d1a/d1e/f3e d18/fb3 0 2024-07-03T15:44:11.562 INFO:tasks.workunit.client.0.smithi028.stdout:3/573: write d18/fb3 [582360,21314] 0 2024-07-03T15:44:11.563 INFO:tasks.workunit.client.0.smithi028.stdout:1/708: read d4/d3c/d54/d79/d16/d57/d66/f67 [1403714,52764] 0 2024-07-03T15:44:11.563 INFO:tasks.workunit.client.0.smithi028.stdout:3/574: mkdir d18/d3a/d9d/d8c/db4 0 2024-07-03T15:44:11.575 INFO:tasks.workunit.client.0.smithi028.stdout:9/546: dwrite d1/d16/d57/d5f/d64/f75 [0,4194304] 0 2024-07-03T15:44:11.577 INFO:tasks.workunit.client.0.smithi028.stdout:4/634: dwrite d2/f7 [0,4194304] 0 2024-07-03T15:44:11.578 INFO:tasks.workunit.client.0.smithi028.stdout:4/635: write d2/d12/d2d/d58/f69 [298347,99413] 0 2024-07-03T15:44:11.581 INFO:tasks.workunit.client.0.smithi028.stdout:9/547: symlink d1/d10/lb8 0 2024-07-03T15:44:11.583 INFO:tasks.workunit.client.0.smithi028.stdout:9/548: rename d1/f43 to d1/d16/d57/d5f/fb9 0 2024-07-03T15:44:11.585 INFO:tasks.workunit.client.0.smithi028.stdout:9/549: rename d1/dd/l23 to d1/d10/d28/d45/d61/d8c/lba 0 2024-07-03T15:44:11.587 INFO:tasks.workunit.client.0.smithi028.stdout:9/550: symlink d1/d4a/d7e/lbb 0 2024-07-03T15:44:11.591 INFO:tasks.workunit.client.0.smithi028.stdout:3/575: dwrite d18/d1a/d1e/d58/d9f/fa2 [0,4194304] 0 2024-07-03T15:44:11.591 INFO:tasks.workunit.client.0.smithi028.stdout:1/709: dwrite d4/d2c/d3a/d5c/fbf [0,4194304] 0 2024-07-03T15:44:11.592 INFO:tasks.workunit.client.0.smithi028.stdout:3/576: symlink d18/d1a/d1e/d73/lb5 0 2024-07-03T15:44:11.592 INFO:tasks.workunit.client.0.smithi028.stdout:3/577: truncate d18/d1a/d5e/d66/fb2 309691 0 2024-07-03T15:44:11.600 INFO:tasks.workunit.client.0.smithi028.stdout:3/578: mkdir d18/d3a/db6 0 2024-07-03T15:44:11.600 INFO:tasks.workunit.client.0.smithi028.stdout:1/710: getdents d4/d3c/d54 0 2024-07-03T15:44:11.603 INFO:tasks.workunit.client.0.smithi028.stdout:1/711: getdents d4/d3c/d54/d79/d16 0 2024-07-03T15:44:11.605 INFO:tasks.workunit.client.0.smithi028.stdout:1/712: mkdir d4/d3c/d54/d84/dfb 0 2024-07-03T15:44:11.605 INFO:tasks.workunit.client.0.smithi028.stdout:1/713: creat d4/d2c/d3a/d5c/d6c/d7d/d59/dc6/ffc x:0 0 0 2024-07-03T15:44:11.608 INFO:tasks.workunit.client.0.smithi028.stdout:3/579: dread d18/d1a/f24 [0,4194304] 0 2024-07-03T15:44:11.608 INFO:tasks.workunit.client.0.smithi028.stdout:9/551: dwrite d1/dd/f6f [0,4194304] 0 2024-07-03T15:44:11.610 INFO:tasks.workunit.client.0.smithi028.stdout:1/714: write d4/d3c/d54/d79/d16/d7c/fcc [90826,2506] 0 2024-07-03T15:44:11.610 INFO:tasks.workunit.client.0.smithi028.stdout:1/715: creat d4/d2c/d3a/d5c/d6c/d7d/d59/dc6/ffd x:0 0 0 2024-07-03T15:44:11.610 INFO:tasks.workunit.client.0.smithi028.stdout:1/716: chown d4/d3c/d54/d79/d16/fb4 505976565 1 2024-07-03T15:44:11.610 INFO:tasks.workunit.client.0.smithi028.stdout:1/717: truncate d4/de/d1c/d81/fd3 360900 0 2024-07-03T15:44:11.613 INFO:tasks.workunit.client.0.smithi028.stdout:3/580: chown d18/f2d 819472953 1 2024-07-03T15:44:11.615 INFO:tasks.workunit.client.0.smithi028.stdout:9/552: truncate d1/d16/d57/f66 145000 0 2024-07-03T15:44:11.617 INFO:tasks.workunit.client.0.smithi028.stdout:3/581: dread d18/d1a/f24 [0,4194304] 0 2024-07-03T15:44:11.617 INFO:tasks.workunit.client.0.smithi028.stdout:1/718: rmdir d4/d3c/d54/d79/d16/d7c 39 2024-07-03T15:44:11.617 INFO:tasks.workunit.client.0.smithi028.stdout:1/719: fsync d4/d2c/d3a/d5c/fcf 0 2024-07-03T15:44:11.617 INFO:tasks.workunit.client.0.smithi028.stdout:1/720: write d4/d2c/f78 [1857316,64497] 0 2024-07-03T15:44:11.618 INFO:tasks.workunit.client.0.smithi028.stdout:9/553: truncate d1/d16/f3a 381919 0 2024-07-03T15:44:11.618 INFO:tasks.workunit.client.0.smithi028.stdout:1/721: dread d4/d3c/d54/d79/d16/fb3 [0,4194304] 0 2024-07-03T15:44:11.619 INFO:tasks.workunit.client.0.smithi028.stdout:1/722: write d4/d3c/d54/d79/f1b [2573781,92480] 0 2024-07-03T15:44:11.626 INFO:tasks.workunit.client.0.smithi028.stdout:9/554: truncate d1/d10/f5d 289517 0 2024-07-03T15:44:11.626 INFO:tasks.workunit.client.0.smithi028.stdout:9/555: creat d1/d10/d28/d4e/d5b/fbc x:0 0 0 2024-07-03T15:44:11.629 INFO:tasks.workunit.client.0.smithi028.stdout:3/582: write d18/d1a/d64/f6e [1360027,3029] 0 2024-07-03T15:44:11.629 INFO:tasks.workunit.client.0.smithi028.stdout:3/583: creat d18/d1a/d5e/d66/fb7 x:0 0 0 2024-07-03T15:44:11.631 INFO:tasks.workunit.client.0.smithi028.stdout:9/556: creat d1/d10/d28/d45/d61/d8c/d77/db7/fbd x:0 0 0 2024-07-03T15:44:11.632 INFO:tasks.workunit.client.0.smithi028.stdout:1/723: dwrite d4/d3c/d54/d79/f12 [4194304,4194304] 0 2024-07-03T15:44:11.634 INFO:tasks.workunit.client.0.smithi028.stdout:9/557: truncate d1/dd/f59 1593543 0 2024-07-03T15:44:11.634 INFO:tasks.workunit.client.0.smithi028.stdout:3/584: read d18/d1a/d1e/f57 [145752,5418] 0 2024-07-03T15:44:11.635 INFO:tasks.workunit.client.0.smithi028.stdout:3/585: chown d18/d3a/f39 170053348 1 2024-07-03T15:44:11.639 INFO:tasks.workunit.client.0.smithi028.stdout:9/558: dread d1/d17/f9d [0,4194304] 0 2024-07-03T15:44:11.639 INFO:tasks.workunit.client.0.smithi028.stdout:9/559: readlink d1/dd/la6 0 2024-07-03T15:44:11.652 INFO:tasks.workunit.client.0.smithi028.stdout:1/724: dread d4/d2c/d3a/d5c/d6c/d7d/d59/f46 [0,4194304] 0 2024-07-03T15:44:11.657 INFO:tasks.workunit.client.0.smithi028.stdout:1/725: dread d4/d2c/f92 [0,4194304] 0 2024-07-03T15:44:11.657 INFO:tasks.workunit.client.0.smithi028.stdout:1/726: readlink d4/de/d1c/d44/l4c 0 2024-07-03T15:44:11.659 INFO:tasks.workunit.client.0.smithi028.stdout:3/586: dwrite d18/d1a/d1e/f53 [0,4194304] 0 2024-07-03T15:44:11.660 INFO:tasks.workunit.client.0.smithi028.stdout:3/587: dread - d18/d1a/d1e/d58/f7f zero size 2024-07-03T15:44:11.660 INFO:tasks.workunit.client.0.smithi028.stdout:3/588: chown d18/d1a/d1e/d58/d9f 0 1 2024-07-03T15:44:11.666 INFO:tasks.workunit.client.0.smithi028.stdout:3/589: unlink d18/d1a/d1e/d58/d9f/fb1 0 2024-07-03T15:44:11.666 INFO:tasks.workunit.client.0.smithi028.stdout:3/590: write f9 [1399732,7165] 0 2024-07-03T15:44:11.666 INFO:tasks.workunit.client.0.smithi028.stdout:3/591: readlink d18/d1a/l7c 0 2024-07-03T15:44:11.666 INFO:tasks.workunit.client.0.smithi028.stdout:3/592: fsync d18/d1a/d1e/d58/f96 0 2024-07-03T15:44:11.667 INFO:tasks.workunit.client.0.smithi028.stdout:3/593: creat d18/d1a/d1e/d58/d9f/fb8 x:0 0 0 2024-07-03T15:44:11.674 INFO:tasks.workunit.client.0.smithi028.stdout:3/594: dread d18/d1a/d1e/d58/f6c [0,4194304] 0 2024-07-03T15:44:11.674 INFO:tasks.workunit.client.0.smithi028.stdout:3/595: readlink d18/d3a/d9d/laa 0 2024-07-03T15:44:11.674 INFO:tasks.workunit.client.0.smithi028.stdout:1/727: dwrite d4/de/fc5 [0,4194304] 0 2024-07-03T15:44:11.675 INFO:tasks.workunit.client.0.smithi028.stdout:3/596: mknod d18/d3a/d9d/d8c/db4/cb9 0 2024-07-03T15:44:11.676 INFO:tasks.workunit.client.0.smithi028.stdout:1/728: dread d4/d3c/d54/d79/f3e [0,4194304] 0 2024-07-03T15:44:11.681 INFO:tasks.workunit.client.0.smithi028.stdout:3/597: mknod d18/d1a/d1e/cba 0 2024-07-03T15:44:11.683 INFO:tasks.workunit.client.0.smithi028.stdout:1/729: rmdir d4/d2c/d3a/d5c/d6c/d7d/d59/d94 39 2024-07-03T15:44:11.683 INFO:tasks.workunit.client.0.smithi028.stdout:1/730: truncate d4/d83/de8/fea 4393520 0 2024-07-03T15:44:11.685 INFO:tasks.workunit.client.0.smithi028.stdout:1/731: dread d4/d3c/d54/d79/f8e [0,4194304] 0 2024-07-03T15:44:11.685 INFO:tasks.workunit.client.0.smithi028.stdout:1/732: readlink d4/d3c/d54/d79/l6d 0 2024-07-03T15:44:11.685 INFO:tasks.workunit.client.0.smithi028.stdout:3/598: rmdir d18/d1a/d1e/d73 39 2024-07-03T15:44:11.686 INFO:tasks.workunit.client.0.smithi028.stdout:1/733: truncate d4/d3c/fa3 99579 0 2024-07-03T15:44:11.687 INFO:tasks.workunit.client.0.smithi028.stdout:1/734: read d4/f19 [4461031,13600] 0 2024-07-03T15:44:11.688 INFO:tasks.workunit.client.0.smithi028.stdout:1/735: creat d4/d3c/d54/d79/d16/db6/dd1/ffe x:0 0 0 2024-07-03T15:44:11.690 INFO:tasks.workunit.client.0.smithi028.stdout:3/599: dread d18/d1a/f2a [0,4194304] 0 2024-07-03T15:44:11.692 INFO:tasks.workunit.client.0.smithi028.stdout:3/600: dread d18/d1a/d1e/f57 [0,4194304] 0 2024-07-03T15:44:11.694 INFO:tasks.workunit.client.0.smithi028.stdout:3/601: mkdir d18/d3a/dbb 0 2024-07-03T15:44:11.696 INFO:tasks.workunit.client.0.smithi028.stdout:3/602: creat d18/d6f/d78/fbc x:0 0 0 2024-07-03T15:44:11.698 INFO:tasks.workunit.client.0.smithi028.stdout:3/603: write d18/d43/f4d [1505948,57125] 0 2024-07-03T15:44:11.698 INFO:tasks.workunit.client.0.smithi028.stdout:3/604: readlink d18/d1a/d1e/d58/l59 0 2024-07-03T15:44:11.703 INFO:tasks.workunit.client.0.smithi028.stdout:1/736: dwrite d4/de/d1c/f3d [0,4194304] 0 2024-07-03T15:44:11.707 INFO:tasks.workunit.client.0.smithi028.stdout:3/605: getdents d18/d3a 0 2024-07-03T15:44:11.709 INFO:tasks.workunit.client.0.smithi028.stdout:1/737: rmdir d4/de 39 2024-07-03T15:44:11.712 INFO:tasks.workunit.client.0.smithi028.stdout:1/738: rename d4/d2c/d3a/d5c/d6c/d7d/d59/d7f to d4/d2c/d3a/d5c/dff 0 2024-07-03T15:44:11.712 INFO:tasks.workunit.client.0.smithi028.stdout:1/739: fdatasync d4/de/d1c/d44/fbd 0 2024-07-03T15:44:11.723 INFO:tasks.workunit.client.0.smithi028.stdout:1/740: read d4/d2c/d3a/d5c/d6c/d7d/f5d [2129415,84977] 0 2024-07-03T15:44:11.723 INFO:tasks.workunit.client.0.smithi028.stdout:1/741: chown d4/d3c/d54/d79/f12 34 1 2024-07-03T15:44:11.723 INFO:tasks.workunit.client.0.smithi028.stdout:1/742: chown d4/de 22084 1 2024-07-03T15:44:11.723 INFO:tasks.workunit.client.0.smithi028.stdout:1/743: write d4/d2c/d3a/d5c/fcf [970872,16501] 0 2024-07-03T15:44:11.724 INFO:tasks.workunit.client.0.smithi028.stdout:1/744: write d4/fe9 [912648,24029] 0 2024-07-03T15:44:11.724 INFO:tasks.workunit.client.0.smithi028.stdout:1/745: mknod d4/d83/c100 0 2024-07-03T15:44:11.725 INFO:tasks.workunit.client.0.smithi028.stdout:3/606: dwrite ff [0,4194304] 0 2024-07-03T15:44:11.725 INFO:tasks.workunit.client.0.smithi028.stdout:1/746: symlink d4/d2c/dbe/l101 0 2024-07-03T15:44:11.727 INFO:tasks.workunit.client.0.smithi028.stdout:3/607: write d18/d1a/d5e/f7b [1005506,114838] 0 2024-07-03T15:44:11.727 INFO:tasks.workunit.client.0.smithi028.stdout:3/608: fdatasync d18/d1a/d1e/d58/d9f/fb8 0 2024-07-03T15:44:11.728 INFO:tasks.workunit.client.0.smithi028.stdout:1/747: mknod d4/d2c/d3a/d5c/dff/c102 0 2024-07-03T15:44:11.729 INFO:tasks.workunit.client.0.smithi028.stdout:1/748: mknod d4/d2c/c103 0 2024-07-03T15:44:11.731 INFO:tasks.workunit.client.0.smithi028.stdout:1/749: rename d4/de/d1c/l36 to d4/d2c/d3a/d5c/daa/l104 0 2024-07-03T15:44:11.733 INFO:tasks.workunit.client.0.smithi028.stdout:1/750: dread d4/d3c/d54/d84/f9b [0,4194304] 0 2024-07-03T15:44:11.738 INFO:tasks.workunit.client.0.smithi028.stdout:1/751: link d4/de/f85 d4/d2c/d3a/d5c/f105 0 2024-07-03T15:44:11.740 INFO:tasks.workunit.client.0.smithi028.stdout:1/752: truncate d4/d83/de8/fdc 468037 0 2024-07-03T15:44:11.742 INFO:tasks.workunit.client.0.smithi028.stdout:1/753: rmdir d4/d3c/d54/d79/d16/db6/ddb 39 2024-07-03T15:44:11.745 INFO:tasks.workunit.client.0.smithi028.stdout:1/754: link d4/d2c/d3a/d5c/d6c/d7d/d4e/d51/fa2 d4/d2c/d3a/d5c/f106 0 2024-07-03T15:44:11.747 INFO:tasks.workunit.client.0.smithi028.stdout:1/755: readlink d4/d3c/d54/d79/d16/d7c/lcb 0 2024-07-03T15:44:11.772 INFO:tasks.workunit.client.0.smithi028.stdout:1/756: dwrite d4/d3c/fa3 [0,4194304] 0 2024-07-03T15:44:11.780 INFO:tasks.workunit.client.0.smithi028.stdout:1/757: truncate d4/de/f85 2968826 0 2024-07-03T15:44:11.783 INFO:tasks.workunit.client.0.smithi028.stdout:1/758: symlink d4/d2c/d3a/d43/l107 0 2024-07-03T15:44:11.911 INFO:tasks.workunit.client.0.smithi028.stdout:1/759: rename d4/de/f5a to d4/d2c/d3a/d5c/dff/f108 0 2024-07-03T15:44:11.916 INFO:tasks.workunit.client.0.smithi028.stdout:1/760: creat d4/d3c/d54/d79/f109 x:0 0 0 2024-07-03T15:44:11.917 INFO:tasks.workunit.client.0.smithi028.stdout:8/614: sync 2024-07-03T15:44:11.919 INFO:tasks.workunit.client.0.smithi028.stdout:1/761: mkdir d4/d3c/d54/d79/d16/d7c/d10a 0 2024-07-03T15:44:11.919 INFO:tasks.workunit.client.0.smithi028.stdout:1/762: write d4/d3c/d54/fe7 [963817,678] 0 2024-07-03T15:44:11.920 INFO:tasks.workunit.client.0.smithi028.stdout:1/763: creat d4/d2c/d3a/d5c/d6c/d7d/f10b x:0 0 0 2024-07-03T15:44:11.920 INFO:tasks.workunit.client.0.smithi028.stdout:1/764: creat d4/d83/f10c x:0 0 0 2024-07-03T15:44:11.924 INFO:tasks.workunit.client.0.smithi028.stdout:8/615: dread d0/d41/d50/d3/dc/d33/d7a/f39 [0,4194304] 0 2024-07-03T15:44:11.925 INFO:tasks.workunit.client.0.smithi028.stdout:8/616: mknod d0/d41/d50/d3/dc/db3/ccd 0 2024-07-03T15:44:11.927 INFO:tasks.workunit.client.0.smithi028.stdout:8/617: rename d0/f44 to d0/d6c/fce 0 2024-07-03T15:44:11.927 INFO:tasks.workunit.client.0.smithi028.stdout:8/618: write d0/d41/d50/d3/d35/d68/d22/f25 [338116,84216] 0 2024-07-03T15:44:11.929 INFO:tasks.workunit.client.0.smithi028.stdout:6/573: sync 2024-07-03T15:44:11.934 INFO:tasks.workunit.client.0.smithi028.stdout:6/574: creat d4/da/d38/d60/dc5/fca x:0 0 0 2024-07-03T15:44:11.934 INFO:tasks.workunit.client.0.smithi028.stdout:6/575: write d4/da/d38/f5f [1621784,47970] 0 2024-07-03T15:44:11.934 INFO:tasks.workunit.client.0.smithi028.stdout:6/576: fsync d4/f29 0 2024-07-03T15:44:11.936 INFO:tasks.workunit.client.0.smithi028.stdout:6/577: creat d4/d78/fcb x:0 0 0 2024-07-03T15:44:11.936 INFO:tasks.workunit.client.0.smithi028.stdout:6/578: write d4/d15/fc7 [27804,6791] 0 2024-07-03T15:44:11.938 INFO:tasks.workunit.client.0.smithi028.stdout:6/579: rename d4/da/d1c/f1e to d4/da/d13/fcc 0 2024-07-03T15:44:11.939 INFO:tasks.workunit.client.0.smithi028.stdout:6/580: symlink d4/d12/d2c/lcd 0 2024-07-03T15:44:11.939 INFO:tasks.workunit.client.0.smithi028.stdout:6/581: creat d4/da/d38/d60/dc5/fce x:0 0 0 2024-07-03T15:44:11.941 INFO:tasks.workunit.client.0.smithi028.stdout:6/582: mkdir d4/da/d38/d60/dc5/dcf 0 2024-07-03T15:44:11.943 INFO:tasks.workunit.client.0.smithi028.stdout:8/619: dwrite d0/d41/d50/d3/dc/d3f/d8e/f42 [0,4194304] 0 2024-07-03T15:44:11.943 INFO:tasks.workunit.client.0.smithi028.stdout:1/765: dwrite d4/d3c/d54/fd8 [0,4194304] 0 2024-07-03T15:44:11.944 INFO:tasks.workunit.client.0.smithi028.stdout:6/583: rename d4/d15/f7f to d4/da/d38/d60/db5/fd0 0 2024-07-03T15:44:11.944 INFO:tasks.workunit.client.0.smithi028.stdout:6/584: getdents d4/da/d13/da4 0 2024-07-03T15:44:11.948 INFO:tasks.workunit.client.0.smithi028.stdout:6/585: rmdir d4/da/d38 39 2024-07-03T15:44:11.949 INFO:tasks.workunit.client.0.smithi028.stdout:6/586: read - d4/da/fa6 zero size 2024-07-03T15:44:11.950 INFO:tasks.workunit.client.0.smithi028.stdout:6/587: mknod d4/d12/cd1 0 2024-07-03T15:44:11.950 INFO:tasks.workunit.client.0.smithi028.stdout:6/588: fsync d4/f33 0 2024-07-03T15:44:11.952 INFO:tasks.workunit.client.0.smithi028.stdout:6/589: rmdir d4/da/d13/da4 0 2024-07-03T15:44:11.952 INFO:tasks.workunit.client.0.smithi028.stdout:6/590: write d4/d15/f5b [5872,63561] 0 2024-07-03T15:44:11.953 INFO:tasks.workunit.client.0.smithi028.stdout:6/591: write d4/da/d1c/f88 [42201,104741] 0 2024-07-03T15:44:11.954 INFO:tasks.workunit.client.0.smithi028.stdout:6/592: chown d4/l1f 51856 1 2024-07-03T15:44:11.954 INFO:tasks.workunit.client.0.smithi028.stdout:6/593: fdatasync d4/da/d38/d60/db5/fd0 0 2024-07-03T15:44:11.954 INFO:tasks.workunit.client.0.smithi028.stdout:6/594: fsync d4/f7b 0 2024-07-03T15:44:11.954 INFO:tasks.workunit.client.0.smithi028.stdout:1/766: dread d4/d2c/d3a/d5c/d6c/d7d/f89 [0,4194304] 0 2024-07-03T15:44:11.955 INFO:tasks.workunit.client.0.smithi028.stdout:1/767: rmdir d4/de 39 2024-07-03T15:44:11.957 INFO:tasks.workunit.client.0.smithi028.stdout:1/768: mkdir d4/de/d1c/d81/d10d 0 2024-07-03T15:44:11.964 INFO:tasks.workunit.client.0.smithi028.stdout:8/620: dwrite d0/d41/d50/d3/d35/d68/fc5 [0,4194304] 0 2024-07-03T15:44:11.969 INFO:tasks.workunit.client.0.smithi028.stdout:8/621: read d0/d41/d50/d3/dc/d33/d7a/f56 [653430,130044] 0 2024-07-03T15:44:11.969 INFO:tasks.workunit.client.0.smithi028.stdout:6/595: fsync d4/da/d38/d60/db5/fd0 0 2024-07-03T15:44:11.969 INFO:tasks.workunit.client.0.smithi028.stdout:8/622: write d0/d41/d50/d3/d35/d68/d22/fb1 [955535,111691] 0 2024-07-03T15:44:11.970 INFO:tasks.workunit.client.0.smithi028.stdout:8/623: write d0/d41/d50/d3/dc/d3f/d8e/d4c/fa0 [319930,70027] 0 2024-07-03T15:44:11.970 INFO:tasks.workunit.client.0.smithi028.stdout:8/624: readlink d0/l85 0 2024-07-03T15:44:11.970 INFO:tasks.workunit.client.0.smithi028.stdout:8/625: dread - d0/d41/d50/d3/dc/d3f/f61 zero size 2024-07-03T15:44:11.970 INFO:tasks.workunit.client.0.smithi028.stdout:6/596: unlink d4/da/d13/c76 0 2024-07-03T15:44:11.974 INFO:tasks.workunit.client.0.smithi028.stdout:8/626: rename d0/d41/d50/d3/dc/d3f/d8e/d4c to d0/d41/d92/dcf 0 2024-07-03T15:44:11.974 INFO:tasks.workunit.client.0.smithi028.stdout:8/627: write d0/d41/d50/d3/d35/d68/d22/fb1 [947423,90819] 0 2024-07-03T15:44:11.975 INFO:tasks.workunit.client.0.smithi028.stdout:6/597: rename d4/da/d56/f73 to d4/d12/fd2 0 2024-07-03T15:44:11.980 INFO:tasks.workunit.client.0.smithi028.stdout:6/598: dread d4/da/f7c [0,4194304] 0 2024-07-03T15:44:11.982 INFO:tasks.workunit.client.0.smithi028.stdout:6/599: rename d4/da/d56/c94 to d4/da/d38/d3d/cd3 0 2024-07-03T15:44:11.983 INFO:tasks.workunit.client.0.smithi028.stdout:6/600: creat d4/da/d38/d59/d91/fd4 x:0 0 0 2024-07-03T15:44:11.983 INFO:tasks.workunit.client.0.smithi028.stdout:6/601: fdatasync d4/d15/fc7 0 2024-07-03T15:44:11.984 INFO:tasks.workunit.client.0.smithi028.stdout:6/602: symlink d4/d12/dc3/ld5 0 2024-07-03T15:44:11.985 INFO:tasks.workunit.client.0.smithi028.stdout:6/603: read d4/da/d47/f6b [4832195,46864] 0 2024-07-03T15:44:11.987 INFO:tasks.workunit.client.0.smithi028.stdout:6/604: symlink d4/da/d13/dbd/ld6 0 2024-07-03T15:44:11.988 INFO:tasks.workunit.client.0.smithi028.stdout:6/605: symlink d4/da/d47/d66/daf/ld7 0 2024-07-03T15:44:12.001 INFO:tasks.workunit.client.0.smithi028.stdout:6/606: dwrite d4/f45 [0,4194304] 0 2024-07-03T15:44:12.008 INFO:tasks.workunit.client.0.smithi028.stdout:6/607: write d4/da/d38/f40 [1559074,100362] 0 2024-07-03T15:44:12.010 INFO:tasks.workunit.client.0.smithi028.stdout:6/608: write d4/da/d38/d59/f92 [116382,17218] 0 2024-07-03T15:44:12.017 INFO:tasks.workunit.client.0.smithi028.stdout:6/609: mkdir d4/da/d38/d3d/d89/dd8 0 2024-07-03T15:44:12.022 INFO:tasks.workunit.client.0.smithi028.stdout:6/610: read d4/f3a [4050450,6997] 0 2024-07-03T15:44:12.024 INFO:tasks.workunit.client.0.smithi028.stdout:6/611: dread d4/da/d47/d66/f8b [0,4194304] 0 2024-07-03T15:44:12.048 INFO:tasks.workunit.client.0.smithi028.stdout:6/612: dwrite d4/da/d1c/f61 [8388608,4194304] 0 2024-07-03T15:44:12.054 INFO:tasks.workunit.client.0.smithi028.stdout:6/613: dread d4/f3a [0,4194304] 0 2024-07-03T15:44:12.079 INFO:tasks.workunit.client.0.smithi028.stdout:6/614: dwrite d4/da/d38/d60/dc5/fce [0,4194304] 0 2024-07-03T15:44:12.080 INFO:tasks.workunit.client.0.smithi028.stdout:6/615: stat d4/da/d38/d60/dc5/dcf 0 2024-07-03T15:44:12.102 INFO:tasks.workunit.client.0.smithi028.stdout:6/616: dwrite d4/da/d1c/fb1 [0,4194304] 0 2024-07-03T15:44:12.102 INFO:tasks.workunit.client.0.smithi028.stdout:6/617: chown d4/d12/fa9 8 1 2024-07-03T15:44:12.103 INFO:tasks.workunit.client.0.smithi028.stdout:6/618: dread d4/da/d38/d60/fab [0,4194304] 0 2024-07-03T15:44:12.120 INFO:tasks.workunit.client.0.smithi028.stdout:6/619: dwrite d4/da/d38/d60/fab [0,4194304] 0 2024-07-03T15:44:12.123 INFO:tasks.workunit.client.0.smithi028.stdout:6/620: creat d4/da/d38/d60/dc5/dcf/fd9 x:0 0 0 2024-07-03T15:44:12.123 INFO:tasks.workunit.client.0.smithi028.stdout:6/621: chown d4/da/d38/la3 20390 1 2024-07-03T15:44:12.126 INFO:tasks.workunit.client.0.smithi028.stdout:6/622: symlink d4/d12/d2c/dac/lda 0 2024-07-03T15:44:12.126 INFO:tasks.workunit.client.0.smithi028.stdout:6/623: dread - d4/fc9 zero size 2024-07-03T15:44:12.127 INFO:tasks.workunit.client.0.smithi028.stdout:6/624: creat d4/da/d38/d60/dc5/dcf/fdb x:0 0 0 2024-07-03T15:44:12.127 INFO:tasks.workunit.client.0.smithi028.stdout:6/625: dread d4/f3a [4194304,4194304] 0 2024-07-03T15:44:12.128 INFO:tasks.workunit.client.0.smithi028.stdout:4/636: sync 2024-07-03T15:44:12.128 INFO:tasks.workunit.client.0.smithi028.stdout:2/482: sync 2024-07-03T15:44:12.128 INFO:tasks.workunit.client.0.smithi028.stdout:5/615: sync 2024-07-03T15:44:12.128 INFO:tasks.workunit.client.0.smithi028.stdout:0/701: sync 2024-07-03T15:44:12.128 INFO:tasks.workunit.client.0.smithi028.stdout:5/616: fdatasync d0/dd/d13/d57/d30/fc5 0 2024-07-03T15:44:12.128 INFO:tasks.workunit.client.0.smithi028.stdout:4/637: read d2/d12/d2e/fb1 [223788,128893] 0 2024-07-03T15:44:12.129 INFO:tasks.workunit.client.0.smithi028.stdout:7/438: sync 2024-07-03T15:44:12.129 INFO:tasks.workunit.client.0.smithi028.stdout:6/626: creat d4/d12/dc3/fdc x:0 0 0 2024-07-03T15:44:12.131 INFO:tasks.workunit.client.0.smithi028.stdout:6/627: mkdir d4/d12/d2c/ddd 0 2024-07-03T15:44:12.131 INFO:tasks.workunit.client.0.smithi028.stdout:6/628: mknod d4/da/d47/d9f/cde 0 2024-07-03T15:44:12.139 INFO:tasks.workunit.client.0.smithi028.stdout:6/629: write d4/da/d1c/f50 [3839559,79301] 0 2024-07-03T15:44:12.141 INFO:tasks.workunit.client.0.smithi028.stdout:6/630: unlink d4/da/d13/f7d 0 2024-07-03T15:44:12.142 INFO:tasks.workunit.client.0.smithi028.stdout:0/702: write d2/d3/dc4/d58/f6a [2888827,69045] 0 2024-07-03T15:44:12.148 INFO:tasks.workunit.client.0.smithi028.stdout:7/439: dread d9/d29/f3c [0,4194304] 0 2024-07-03T15:44:12.157 INFO:tasks.workunit.client.0.smithi028.stdout:7/440: dread d9/f25 [0,4194304] 0 2024-07-03T15:44:12.158 INFO:tasks.workunit.client.0.smithi028.stdout:6/631: read d4/d12/f21 [3549312,8245] 0 2024-07-03T15:44:12.158 INFO:tasks.workunit.client.0.smithi028.stdout:6/632: chown d4/d15/c48 129784 1 2024-07-03T15:44:12.158 INFO:tasks.workunit.client.0.smithi028.stdout:7/441: mknod d9/d1c/d52/d6a/c83 0 2024-07-03T15:44:12.158 INFO:tasks.workunit.client.0.smithi028.stdout:6/633: dread - d4/da/d38/d60/dc5/dcf/fd9 zero size 2024-07-03T15:44:12.158 INFO:tasks.workunit.client.0.smithi028.stdout:7/442: write d9/d1c/d7c/f80 [148370,118873] 0 2024-07-03T15:44:12.159 INFO:tasks.workunit.client.0.smithi028.stdout:6/634: creat d4/da/d47/d9f/fdf x:0 0 0 2024-07-03T15:44:12.159 INFO:tasks.workunit.client.0.smithi028.stdout:6/635: chown d4/da/d1c/c95 11 1 2024-07-03T15:44:12.159 INFO:tasks.workunit.client.0.smithi028.stdout:6/636: truncate d4/d78/fcb 1020131 0 2024-07-03T15:44:12.159 INFO:tasks.workunit.client.0.smithi028.stdout:5/617: dwrite d0/dd/d98/f6c [0,4194304] 0 2024-07-03T15:44:12.159 INFO:tasks.workunit.client.0.smithi028.stdout:5/618: write d0/dd/d1e/d94/fcc [547900,37757] 0 2024-07-03T15:44:12.162 INFO:tasks.workunit.client.0.smithi028.stdout:6/637: creat d4/d12/d2c/fe0 x:0 0 0 2024-07-03T15:44:12.162 INFO:tasks.workunit.client.0.smithi028.stdout:7/443: getdents d9/d1c/d6e 0 2024-07-03T15:44:12.163 INFO:tasks.workunit.client.0.smithi028.stdout:2/483: dwrite da/d74/f90 [0,4194304] 0 2024-07-03T15:44:12.164 INFO:tasks.workunit.client.0.smithi028.stdout:2/484: chown da/d23/d30/l44 30548 1 2024-07-03T15:44:12.164 INFO:tasks.workunit.client.0.smithi028.stdout:0/703: dwrite d2/d3/dc4/d36/d45/f46 [4194304,4194304] 0 2024-07-03T15:44:12.164 INFO:tasks.workunit.client.0.smithi028.stdout:0/704: readlink d2/l28 0 2024-07-03T15:44:12.164 INFO:tasks.workunit.client.0.smithi028.stdout:6/638: link d4/da/d38/f46 d4/da/d38/d60/dc5/fe1 0 2024-07-03T15:44:12.164 INFO:tasks.workunit.client.0.smithi028.stdout:6/639: readlink d4/d12/d2c/l35 0 2024-07-03T15:44:12.164 INFO:tasks.workunit.client.0.smithi028.stdout:6/640: fdatasync d4/da/d38/f46 0 2024-07-03T15:44:12.167 INFO:tasks.workunit.client.0.smithi028.stdout:5/619: getdents d0/dd/d13/d47/d50 0 2024-07-03T15:44:12.176 INFO:tasks.workunit.client.0.smithi028.stdout:7/444: dwrite d9/f25 [0,4194304] 0 2024-07-03T15:44:12.178 INFO:tasks.workunit.client.0.smithi028.stdout:6/641: dread d4/d12/f36 [0,4194304] 0 2024-07-03T15:44:12.187 INFO:tasks.workunit.client.0.smithi028.stdout:2/485: creat da/d1e/d68/fb3 x:0 0 0 2024-07-03T15:44:12.189 INFO:tasks.workunit.client.0.smithi028.stdout:5/620: dwrite d0/dd/d1e/d3b/fa3 [0,4194304] 0 2024-07-03T15:44:12.189 INFO:tasks.workunit.client.0.smithi028.stdout:0/705: unlink d2/d3/d4d/f64 0 2024-07-03T15:44:12.189 INFO:tasks.workunit.client.0.smithi028.stdout:0/706: write d2/d3/dc4/d36/d45/da3/fab [881872,63799] 0 2024-07-03T15:44:12.201 INFO:tasks.workunit.client.0.smithi028.stdout:6/642: dwrite d4/da/d38/fa7 [4194304,4194304] 0 2024-07-03T15:44:12.201 INFO:tasks.workunit.client.0.smithi028.stdout:6/643: creat d4/da/d38/d60/dc5/dcf/fe2 x:0 0 0 2024-07-03T15:44:12.205 INFO:tasks.workunit.client.0.smithi028.stdout:6/644: mknod d4/da/d38/d59/d91/ce3 0 2024-07-03T15:44:12.223 INFO:tasks.workunit.client.0.smithi028.stdout:6/645: dwrite d4/da/d1c/f61 [8388608,4194304] 0 2024-07-03T15:44:12.224 INFO:tasks.workunit.client.0.smithi028.stdout:6/646: symlink d4/da/d38/d59/le4 0 2024-07-03T15:44:12.224 INFO:tasks.workunit.client.0.smithi028.stdout:6/647: readlink d4/da/d1c/l7e 0 2024-07-03T15:44:12.224 INFO:tasks.workunit.client.0.smithi028.stdout:6/648: chown d4/l1f 54320350 1 2024-07-03T15:44:12.226 INFO:tasks.workunit.client.0.smithi028.stdout:6/649: rmdir d4/da/d38/d3d/dbc 0 2024-07-03T15:44:12.226 INFO:tasks.workunit.client.0.smithi028.stdout:6/650: read - d4/da/d38/d60/dc5/fca zero size 2024-07-03T15:44:12.226 INFO:tasks.workunit.client.0.smithi028.stdout:6/651: chown d4/da/d56/fc6 214935812 1 2024-07-03T15:44:12.226 INFO:tasks.workunit.client.0.smithi028.stdout:6/652: write d4/d15/f20 [3100588,75129] 0 2024-07-03T15:44:12.226 INFO:tasks.workunit.client.0.smithi028.stdout:6/653: write d4/da/d38/d60/db5/fd0 [29517,64883] 0 2024-07-03T15:44:12.227 INFO:tasks.workunit.client.0.smithi028.stdout:6/654: mkdir d4/d15/de5 0 2024-07-03T15:44:12.227 INFO:tasks.workunit.client.0.smithi028.stdout:6/655: mknod d4/da/ce6 0 2024-07-03T15:44:12.230 INFO:tasks.workunit.client.0.smithi028.stdout:6/656: rename d4/f77 to d4/da/d13/dbd/fe7 0 2024-07-03T15:44:12.230 INFO:tasks.workunit.client.0.smithi028.stdout:6/657: chown d4/da/d38/d59/d91/cc8 840093 1 2024-07-03T15:44:12.230 INFO:tasks.workunit.client.0.smithi028.stdout:6/658: chown d4/da/cf 14629380 1 2024-07-03T15:44:12.230 INFO:tasks.workunit.client.0.smithi028.stdout:6/659: creat d4/d12/d2c/fe8 x:0 0 0 2024-07-03T15:44:12.230 INFO:tasks.workunit.client.0.smithi028.stdout:6/660: creat d4/da/d47/d9f/fe9 x:0 0 0 2024-07-03T15:44:12.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:11 smithi148 ceph-mon[36056]: pgmap v7: 65 pgs: 65 active+clean; 2.9 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 113 MiB/s rd, 144 MiB/s wr, 278 op/s 2024-07-03T15:44:12.233 INFO:tasks.workunit.client.0.smithi028.stdout:6/661: mknod d4/da/d38/d60/db5/cea 0 2024-07-03T15:44:12.234 INFO:tasks.workunit.client.0.smithi028.stdout:6/662: getdents d4/da/d47/d66/daf 0 2024-07-03T15:44:12.238 INFO:tasks.workunit.client.0.smithi028.stdout:6/663: symlink d4/da/d1c/leb 0 2024-07-03T15:44:12.253 INFO:tasks.workunit.client.0.smithi028.stdout:3/609: sync 2024-07-03T15:44:12.253 INFO:tasks.workunit.client.0.smithi028.stdout:9/560: sync 2024-07-03T15:44:12.253 INFO:tasks.workunit.client.0.smithi028.stdout:6/664: rmdir d4/da/d47/d9f 39 2024-07-03T15:44:12.253 INFO:tasks.workunit.client.0.smithi028.stdout:6/665: stat d4/f45 0 2024-07-03T15:44:12.253 INFO:tasks.workunit.client.0.smithi028.stdout:6/666: fdatasync d4/d15/f5b 0 2024-07-03T15:44:12.254 INFO:tasks.workunit.client.0.smithi028.stdout:6/667: chown d4/d15/fba 49181498 1 2024-07-03T15:44:12.256 INFO:tasks.workunit.client.0.smithi028.stdout:6/668: creat d4/d12/d2c/ddd/fec x:0 0 0 2024-07-03T15:44:12.260 INFO:tasks.workunit.client.0.smithi028.stdout:6/669: link d4/da/d38/d3d/d89/f97 d4/d15/d68/fed 0 2024-07-03T15:44:12.261 INFO:tasks.workunit.client.0.smithi028.stdout:3/610: read d18/d3a/f70 [2034813,106984] 0 2024-07-03T15:44:12.262 INFO:tasks.workunit.client.0.smithi028.stdout:6/670: rename d4/da/d1c/f54 to d4/da/d38/d3d/d89/dd8/fee 0 2024-07-03T15:44:12.269 INFO:tasks.workunit.client.0.smithi028.stdout:9/561: dwrite d1/d16/d57/f9c [4194304,4194304] 0 2024-07-03T15:44:12.270 INFO:tasks.workunit.client.0.smithi028.stdout:6/671: dread d4/da/d38/f4a [0,4194304] 0 2024-07-03T15:44:12.274 INFO:tasks.workunit.client.0.smithi028.stdout:3/611: read d18/d1a/d1e/f53 [3694863,48123] 0 2024-07-03T15:44:12.275 INFO:tasks.workunit.client.0.smithi028.stdout:3/612: mknod d18/d3a/d9d/d8c/cbd 0 2024-07-03T15:44:12.277 INFO:tasks.workunit.client.0.smithi028.stdout:3/613: link d18/d1a/d1e/lab d18/d1a/d64/lbe 0 2024-07-03T15:44:12.277 INFO:tasks.workunit.client.0.smithi028.stdout:3/614: write d18/d3a/d9d/f88 [500357,34466] 0 2024-07-03T15:44:12.278 INFO:tasks.workunit.client.0.smithi028.stdout:3/615: creat d18/d3a/fbf x:0 0 0 2024-07-03T15:44:12.279 INFO:tasks.workunit.client.0.smithi028.stdout:3/616: rmdir d18/d3a/d9d 39 2024-07-03T15:44:12.286 INFO:tasks.workunit.client.0.smithi028.stdout:9/562: write d1/dd/f6f [4043759,81725] 0 2024-07-03T15:44:12.290 INFO:tasks.workunit.client.0.smithi028.stdout:9/563: creat d1/d10/d28/d45/d61/d8c/fbe x:0 0 0 2024-07-03T15:44:12.295 INFO:tasks.workunit.client.0.smithi028.stdout:6/672: dwrite d4/da/d56/fc6 [0,4194304] 0 2024-07-03T15:44:12.295 INFO:tasks.workunit.client.0.smithi028.stdout:6/673: dread - d4/da/d1c/f98 zero size 2024-07-03T15:44:12.296 INFO:tasks.workunit.client.0.smithi028.stdout:9/564: dread d1/d10/f18 [0,4194304] 0 2024-07-03T15:44:12.296 INFO:tasks.workunit.client.0.smithi028.stdout:9/565: creat d1/d8e/dac/fbf x:0 0 0 2024-07-03T15:44:12.300 INFO:tasks.workunit.client.0.smithi028.stdout:6/674: creat d4/d12/d2c/dac/fef x:0 0 0 2024-07-03T15:44:12.300 INFO:tasks.workunit.client.0.smithi028.stdout:6/675: readlink d4/l1b 0 2024-07-03T15:44:12.302 INFO:tasks.workunit.client.0.smithi028.stdout:9/566: mknod d1/d10/d76/cc0 0 2024-07-03T15:44:12.303 INFO:tasks.workunit.client.0.smithi028.stdout:9/567: write d1/d16/d62/f8b [10079,106635] 0 2024-07-03T15:44:12.303 INFO:tasks.workunit.client.0.smithi028.stdout:6/676: mkdir d4/d78/df0 0 2024-07-03T15:44:12.304 INFO:tasks.workunit.client.0.smithi028.stdout:9/568: symlink d1/d16/d62/lc1 0 2024-07-03T15:44:12.306 INFO:tasks.workunit.client.0.smithi028.stdout:6/677: link d4/da/fa8 d4/da/d47/d9f/ff1 0 2024-07-03T15:44:12.307 INFO:tasks.workunit.client.0.smithi028.stdout:6/678: unlink d4/da/c14 0 2024-07-03T15:44:12.309 INFO:tasks.workunit.client.0.smithi028.stdout:6/679: link d4/da/d1c/f50 d4/d12/dc3/ff2 0 2024-07-03T15:44:12.309 INFO:tasks.workunit.client.0.smithi028.stdout:6/680: chown d4/d15/d68/c9a 161 1 2024-07-03T15:44:12.309 INFO:tasks.workunit.client.0.smithi028.stdout:6/681: dread - d4/d12/d2c/d31/f64 zero size 2024-07-03T15:44:12.309 INFO:tasks.workunit.client.0.smithi028.stdout:6/682: chown d4/d15/f8c 3502 1 2024-07-03T15:44:12.310 INFO:tasks.workunit.client.0.smithi028.stdout:6/683: symlink d4/da/d38/d60/lf3 0 2024-07-03T15:44:12.313 INFO:tasks.workunit.client.0.smithi028.stdout:9/569: dread d1/d10/d28/d45/d61/d8c/f6a [0,4194304] 0 2024-07-03T15:44:12.313 INFO:tasks.workunit.client.0.smithi028.stdout:9/570: truncate d1/d17/f25 5331034 0 2024-07-03T15:44:12.314 INFO:tasks.workunit.client.0.smithi028.stdout:9/571: symlink d1/d10/d28/d45/d61/d8c/d77/db7/lc2 0 2024-07-03T15:44:12.314 INFO:tasks.workunit.client.0.smithi028.stdout:9/572: write d1/d10/d28/f79 [434112,47343] 0 2024-07-03T15:44:12.319 INFO:tasks.workunit.client.0.smithi028.stdout:6/684: dread d4/da/d47/f6b [0,4194304] 0 2024-07-03T15:44:12.324 INFO:tasks.workunit.client.0.smithi028.stdout:9/573: getdents d1/d10/d28/d45/d61/d8c/d77/db7 0 2024-07-03T15:44:12.325 INFO:tasks.workunit.client.0.smithi028.stdout:9/574: chown d1/d16/d57/f9c 236 1 2024-07-03T15:44:12.325 INFO:tasks.workunit.client.0.smithi028.stdout:9/575: read d1/d10/f18 [1769177,131060] 0 2024-07-03T15:44:12.325 INFO:tasks.workunit.client.0.smithi028.stdout:9/576: stat d1/d16/f63 0 2024-07-03T15:44:12.325 INFO:tasks.workunit.client.0.smithi028.stdout:9/577: readlink d1/l8 0 2024-07-03T15:44:12.325 INFO:tasks.workunit.client.0.smithi028.stdout:9/578: mkdir d1/d16/d57/d5f/d64/dc3 0 2024-07-03T15:44:12.337 INFO:tasks.workunit.client.0.smithi028.stdout:9/579: write d1/d10/d28/d4e/f50 [2546128,61735] 0 2024-07-03T15:44:12.338 INFO:tasks.workunit.client.0.smithi028.stdout:9/580: creat d1/d10/d28/d45/d61/fc4 x:0 0 0 2024-07-03T15:44:12.338 INFO:tasks.workunit.client.0.smithi028.stdout:9/581: chown d1/l7 792 1 2024-07-03T15:44:12.339 INFO:tasks.workunit.client.0.smithi028.stdout:6/685: dwrite d4/da/d56/fc6 [4194304,4194304] 0 2024-07-03T15:44:12.348 INFO:tasks.workunit.client.0.smithi028.stdout:9/582: rename d1/d16/d57/d5f/f8f to d1/d10/d28/d45/d56/fc5 0 2024-07-03T15:44:12.354 INFO:tasks.workunit.client.0.smithi028.stdout:6/686: mkdir d4/d12/d2c/df4 0 2024-07-03T15:44:12.359 INFO:tasks.workunit.client.0.smithi028.stdout:9/583: creat d1/d8e/fc6 x:0 0 0 2024-07-03T15:44:12.359 INFO:tasks.workunit.client.0.smithi028.stdout:9/584: fsync d1/d16/d57/f9c 0 2024-07-03T15:44:12.359 INFO:tasks.workunit.client.0.smithi028.stdout:6/687: mkdir d4/d12/d2c/d31/df5 0 2024-07-03T15:44:12.363 INFO:tasks.workunit.client.0.smithi028.stdout:9/585: symlink d1/d10/d28/daa/lc7 0 2024-07-03T15:44:12.363 INFO:tasks.workunit.client.0.smithi028.stdout:9/586: chown d1/c22 343 1 2024-07-03T15:44:12.363 INFO:tasks.workunit.client.0.smithi028.stdout:9/587: chown d1/d16/f3a 82442755 1 2024-07-03T15:44:12.363 INFO:tasks.workunit.client.0.smithi028.stdout:9/588: fdatasync d1/d4a/f65 0 2024-07-03T15:44:12.363 INFO:tasks.workunit.client.0.smithi028.stdout:9/589: stat d1/d10/d28/d45/d61/d8c/d77/fb5 0 2024-07-03T15:44:12.365 INFO:tasks.workunit.client.0.smithi028.stdout:9/590: symlink d1/lc8 0 2024-07-03T15:44:12.365 INFO:tasks.workunit.client.0.smithi028.stdout:9/591: write d1/dd/f4c [2309099,125787] 0 2024-07-03T15:44:12.365 INFO:tasks.workunit.client.0.smithi028.stdout:9/592: chown d1/dd/f90 538639922 1 2024-07-03T15:44:12.366 INFO:tasks.workunit.client.0.smithi028.stdout:8/628: sync 2024-07-03T15:44:12.366 INFO:tasks.workunit.client.0.smithi028.stdout:1/769: sync 2024-07-03T15:44:12.366 INFO:tasks.workunit.client.0.smithi028.stdout:8/629: readlink d0/d41/d50/d3/dc/d3f/d8e/d81/l8f 0 2024-07-03T15:44:12.366 INFO:tasks.workunit.client.0.smithi028.stdout:1/770: chown d4/d3c/d54/c61 54783 1 2024-07-03T15:44:12.367 INFO:tasks.workunit.client.0.smithi028.stdout:9/593: creat d1/d96/fc9 x:0 0 0 2024-07-03T15:44:12.368 INFO:tasks.workunit.client.0.smithi028.stdout:8/630: mknod d0/d41/d50/daf/dc8/da6/cd0 0 2024-07-03T15:44:12.370 INFO:tasks.workunit.client.0.smithi028.stdout:9/594: truncate d1/d16/d57/f71 3352782 0 2024-07-03T15:44:12.371 INFO:tasks.workunit.client.0.smithi028.stdout:9/595: chown d1/dd/f59 21 1 2024-07-03T15:44:12.371 INFO:tasks.workunit.client.0.smithi028.stdout:9/596: chown d1/d10/d76 11 1 2024-07-03T15:44:12.371 INFO:tasks.workunit.client.0.smithi028.stdout:8/631: rmdir d0/d41/d50/d3/d35/d68/d22 39 2024-07-03T15:44:12.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:11 smithi028 ceph-mon[29148]: pgmap v7: 65 pgs: 65 active+clean; 2.9 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 113 MiB/s rd, 144 MiB/s wr, 278 op/s 2024-07-03T15:44:12.373 INFO:tasks.workunit.client.0.smithi028.stdout:6/688: dwrite d4/da/d47/d9f/fc4 [0,4194304] 0 2024-07-03T15:44:12.373 INFO:tasks.workunit.client.0.smithi028.stdout:9/597: mkdir d1/d10/d28/d45/d61/d8c/d77/db7/dab/dca 0 2024-07-03T15:44:12.379 INFO:tasks.workunit.client.0.smithi028.stdout:9/598: mkdir d1/d10/d28/d45/d61/d8c/d77/db7/dab/dcb 0 2024-07-03T15:44:12.382 INFO:tasks.workunit.client.0.smithi028.stdout:8/632: rename d0/d41/d50/d3/d35/d68/d22/d40/d45 to d0/d41/d50/daf/dc1/dd1 0 2024-07-03T15:44:12.383 INFO:tasks.workunit.client.0.smithi028.stdout:8/633: unlink d0/d41/d50/daf/dc1/dd1/fbb 0 2024-07-03T15:44:12.383 INFO:tasks.workunit.client.0.smithi028.stdout:8/634: readlink d0/d41/d92/lac 0 2024-07-03T15:44:12.388 INFO:tasks.workunit.client.0.smithi028.stdout:1/771: dwrite d4/de/d1c/d81/fd3 [0,4194304] 0 2024-07-03T15:44:12.388 INFO:tasks.workunit.client.0.smithi028.stdout:1/772: fdatasync d4/d2c/d3a/d5c/fed 0 2024-07-03T15:44:12.388 INFO:tasks.workunit.client.0.smithi028.stdout:1/773: stat d4/d3c/d54/d79/d16/d57/d66/fca 0 2024-07-03T15:44:12.388 INFO:tasks.workunit.client.0.smithi028.stdout:1/774: dread - d4/d3c/d54/d79/d16/fe2 zero size 2024-07-03T15:44:12.389 INFO:tasks.workunit.client.0.smithi028.stdout:6/689: write d4/da/d47/d9f/fc4 [4220201,10585] 0 2024-07-03T15:44:12.389 INFO:tasks.workunit.client.0.smithi028.stdout:6/690: fsync d4/da/d47/d9f/fa5 0 2024-07-03T15:44:12.392 INFO:tasks.workunit.client.0.smithi028.stdout:6/691: link d4/da/d47/d9f/fdf d4/da/d47/ff6 0 2024-07-03T15:44:12.394 INFO:tasks.workunit.client.0.smithi028.stdout:9/599: dread d1/d10/d28/d45/d61/d8c/f81 [0,4194304] 0 2024-07-03T15:44:12.405 INFO:tasks.workunit.client.0.smithi028.stdout:6/692: dread d4/d12/fa9 [0,4194304] 0 2024-07-03T15:44:12.405 INFO:tasks.workunit.client.0.smithi028.stdout:6/693: truncate d4/da/d47/ff6 1021909 0 2024-07-03T15:44:12.407 INFO:tasks.workunit.client.0.smithi028.stdout:8/635: dwrite d0/d41/d50/d3/d35/d68/d22/fb1 [0,4194304] 0 2024-07-03T15:44:12.408 INFO:tasks.workunit.client.0.smithi028.stdout:6/694: mkdir d4/da/d1c/df7 0 2024-07-03T15:44:12.410 INFO:tasks.workunit.client.0.smithi028.stdout:9/600: dread d1/d10/d28/d45/d61/d8c/f72 [0,4194304] 0 2024-07-03T15:44:12.410 INFO:tasks.workunit.client.0.smithi028.stdout:6/695: read d4/da/d47/d9f/fdf [594485,64032] 0 2024-07-03T15:44:12.410 INFO:tasks.workunit.client.0.smithi028.stdout:1/775: dwrite d4/f19 [0,4194304] 0 2024-07-03T15:44:12.411 INFO:tasks.workunit.client.0.smithi028.stdout:8/636: mkdir d0/d41/d50/d3/d35/d68/d22/d40/dd2 0 2024-07-03T15:44:12.411 INFO:tasks.workunit.client.0.smithi028.stdout:8/637: write d0/f2a [255476,14476] 0 2024-07-03T15:44:12.411 INFO:tasks.workunit.client.0.smithi028.stdout:8/638: creat d0/d6c/da9/fd3 x:0 0 0 2024-07-03T15:44:12.412 INFO:tasks.workunit.client.0.smithi028.stdout:9/601: rename d1/d10/d28/d45/d56/c89 to d1/d10/d28/d45/d61/d8c/d77/db7/dab/dcb/ccc 0 2024-07-03T15:44:12.414 INFO:tasks.workunit.client.0.smithi028.stdout:1/776: truncate d4/fd 1350120 0 2024-07-03T15:44:12.414 INFO:tasks.workunit.client.0.smithi028.stdout:1/777: write d4/d2c/d3a/d5c/dff/f87 [1707870,18124] 0 2024-07-03T15:44:12.420 INFO:tasks.workunit.client.0.smithi028.stdout:9/602: mkdir d1/d10/d28/d45/d56/dcd 0 2024-07-03T15:44:12.420 INFO:tasks.workunit.client.0.smithi028.stdout:9/603: creat d1/d10/d28/d45/d61/d8c/fce x:0 0 0 2024-07-03T15:44:12.420 INFO:tasks.workunit.client.0.smithi028.stdout:8/639: dread d0/d41/d50/d3/d35/d68/d22/d40/d62/f65 [0,4194304] 0 2024-07-03T15:44:12.420 INFO:tasks.workunit.client.0.smithi028.stdout:8/640: fsync d0/d41/d50/d3/dc/fcc 0 2024-07-03T15:44:12.420 INFO:tasks.workunit.client.0.smithi028.stdout:8/641: chown d0/d41/d50/d3/dc/d33/l59 191 1 2024-07-03T15:44:12.420 INFO:tasks.workunit.client.0.smithi028.stdout:6/696: dread d4/d15/fba [8388608,4194304] 0 2024-07-03T15:44:12.424 INFO:tasks.workunit.client.0.smithi028.stdout:8/642: creat d0/d41/d50/d3/d35/d68/d22/d40/fd4 x:0 0 0 2024-07-03T15:44:12.424 INFO:tasks.workunit.client.0.smithi028.stdout:8/643: write d0/d41/d50/daf/dc8/da6/f9c [4693512,125593] 0 2024-07-03T15:44:12.427 INFO:tasks.workunit.client.0.smithi028.stdout:6/697: creat d4/da/d38/ff8 x:0 0 0 2024-07-03T15:44:12.429 INFO:tasks.workunit.client.0.smithi028.stdout:9/604: dread d1/d16/f51 [0,4194304] 0 2024-07-03T15:44:12.429 INFO:tasks.workunit.client.0.smithi028.stdout:1/778: dwrite d4/de/fc5 [4194304,4194304] 0 2024-07-03T15:44:12.429 INFO:tasks.workunit.client.0.smithi028.stdout:1/779: chown d4/la 236 1 2024-07-03T15:44:12.429 INFO:tasks.workunit.client.0.smithi028.stdout:1/780: write d4/d2c/d3a/d5c/d6c/d7d/d59/f82 [4352736,1516] 0 2024-07-03T15:44:12.432 INFO:tasks.workunit.client.0.smithi028.stdout:9/605: creat d1/fcf x:0 0 0 2024-07-03T15:44:12.432 INFO:tasks.workunit.client.0.smithi028.stdout:9/606: fdatasync d1/d10/d28/d4e/d4f/f87 0 2024-07-03T15:44:12.432 INFO:tasks.workunit.client.0.smithi028.stdout:9/607: stat d1/d4a/f65 0 2024-07-03T15:44:12.439 INFO:tasks.workunit.client.0.smithi028.stdout:1/781: mkdir d4/d3c/d54/d79/d16/d57/d66/d10e 0 2024-07-03T15:44:12.445 INFO:tasks.workunit.client.0.smithi028.stdout:1/782: mknod d4/d3c/d54/d79/d16/d57/d66/dd6/c10f 0 2024-07-03T15:44:12.449 INFO:tasks.workunit.client.0.smithi028.stdout:8/644: dwrite d0/d41/d50/f27 [0,4194304] 0 2024-07-03T15:44:12.449 INFO:tasks.workunit.client.0.smithi028.stdout:8/645: chown d0/d41/d50/d97/ca1 2 1 2024-07-03T15:44:12.449 INFO:tasks.workunit.client.0.smithi028.stdout:9/608: dwrite d1/d10/d28/d45/d61/d8c/f6a [0,4194304] 0 2024-07-03T15:44:12.451 INFO:tasks.workunit.client.0.smithi028.stdout:8/646: creat d0/d41/d50/d3/d35/d68/d22/fd5 x:0 0 0 2024-07-03T15:44:12.452 INFO:tasks.workunit.client.0.smithi028.stdout:9/609: creat d1/d10/d28/fd0 x:0 0 0 2024-07-03T15:44:12.454 INFO:tasks.workunit.client.0.smithi028.stdout:8/647: link d0/d41/d50/le d0/d41/d50/d3/d71/ld6 0 2024-07-03T15:44:12.454 INFO:tasks.workunit.client.0.smithi028.stdout:8/648: write d0/d41/d50/d3/dc/fb2 [237860,1987] 0 2024-07-03T15:44:12.454 INFO:tasks.workunit.client.0.smithi028.stdout:8/649: write d0/d41/d92/dcf/fa0 [1053633,126286] 0 2024-07-03T15:44:12.454 INFO:tasks.workunit.client.0.smithi028.stdout:8/650: creat d0/d6c/da9/fd7 x:0 0 0 2024-07-03T15:44:12.458 INFO:tasks.workunit.client.0.smithi028.stdout:8/651: link d0/d41/d50/d3/dc/d33/d7a/f39 d0/d41/d50/d3/d35/d68/d22/d40/d4a/fd8 0 2024-07-03T15:44:12.458 INFO:tasks.workunit.client.0.smithi028.stdout:8/652: write d0/d41/d50/d3/dc/d3f/f61 [108833,93050] 0 2024-07-03T15:44:12.459 INFO:tasks.workunit.client.0.smithi028.stdout:1/783: dwrite d4/d2c/d3a/d5c/d6c/d7d/f5d [4194304,4194304] 0 2024-07-03T15:44:12.461 INFO:tasks.workunit.client.0.smithi028.stdout:9/610: read d1/d10/d76/f7c [3417956,46836] 0 2024-07-03T15:44:12.461 INFO:tasks.workunit.client.0.smithi028.stdout:9/611: write d1/d10/d28/d45/d61/d8c/d77/f91 [1011029,42244] 0 2024-07-03T15:44:12.461 INFO:tasks.workunit.client.0.smithi028.stdout:9/612: write d1/d10/d28/d45/d61/d8c/d77/db7/fbd [427010,76456] 0 2024-07-03T15:44:12.465 INFO:tasks.workunit.client.0.smithi028.stdout:9/613: creat d1/d16/d57/d5f/d64/fd1 x:0 0 0 2024-07-03T15:44:12.465 INFO:tasks.workunit.client.0.smithi028.stdout:8/653: dread d0/d41/d50/d3/dc/d33/d7a/f2f [0,4194304] 0 2024-07-03T15:44:12.467 INFO:tasks.workunit.client.0.smithi028.stdout:9/614: truncate d1/d16/d62/f8b 614527 0 2024-07-03T15:44:12.467 INFO:tasks.workunit.client.0.smithi028.stdout:9/615: chown d1/d10/d28/d45/d61/d8c/d77 35170 1 2024-07-03T15:44:12.477 INFO:tasks.workunit.client.0.smithi028.stdout:1/784: dwrite d4/d83/f8b [0,4194304] 0 2024-07-03T15:44:12.485 INFO:tasks.workunit.client.0.smithi028.stdout:8/654: unlink d0/d41/d50/d3/d35/d68/fb8 0 2024-07-03T15:44:12.490 INFO:tasks.workunit.client.0.smithi028.stdout:9/616: link d1/l53 d1/d17/ld2 0 2024-07-03T15:44:12.490 INFO:tasks.workunit.client.0.smithi028.stdout:9/617: creat d1/d10/d28/fd3 x:0 0 0 2024-07-03T15:44:12.490 INFO:tasks.workunit.client.0.smithi028.stdout:9/618: creat d1/d16/d57/d5f/fd4 x:0 0 0 2024-07-03T15:44:12.491 INFO:tasks.workunit.client.0.smithi028.stdout:1/785: dwrite d4/d3c/d54/fe7 [0,4194304] 0 2024-07-03T15:44:12.491 INFO:tasks.workunit.client.0.smithi028.stdout:1/786: write d4/d2c/d3a/d5c/d6c/d7d/f10b [166963,51268] 0 2024-07-03T15:44:12.506 INFO:tasks.workunit.client.0.smithi028.stdout:8/655: truncate d0/d41/d50/d3/dc/d33/d7a/f39 1112078 0 2024-07-03T15:44:12.506 INFO:tasks.workunit.client.0.smithi028.stdout:8/656: chown d0/d41/d50/d3/dc/c18 41 1 2024-07-03T15:44:12.509 INFO:tasks.workunit.client.0.smithi028.stdout:9/619: mkdir d1/d10/d28/d45/d56/dcd/dd5 0 2024-07-03T15:44:12.509 INFO:tasks.workunit.client.0.smithi028.stdout:9/620: fdatasync d1/d16/f2f 0 2024-07-03T15:44:12.511 INFO:tasks.workunit.client.0.smithi028.stdout:4/638: sync 2024-07-03T15:44:12.515 INFO:tasks.workunit.client.0.smithi028.stdout:8/657: write d0/d41/d50/d3/dc/d3f/d8e/f6a [1367679,17131] 0 2024-07-03T15:44:12.515 INFO:tasks.workunit.client.0.smithi028.stdout:1/787: creat d4/dc2/f110 x:0 0 0 2024-07-03T15:44:12.518 INFO:tasks.workunit.client.0.smithi028.stdout:9/621: mknod d1/d10/d28/d45/d61/d8c/cd6 0 2024-07-03T15:44:12.520 INFO:tasks.workunit.client.0.smithi028.stdout:4/639: creat d2/d12/d2d/d2f/fd3 x:0 0 0 2024-07-03T15:44:12.522 INFO:tasks.workunit.client.0.smithi028.stdout:8/658: symlink d0/d41/d92/ld9 0 2024-07-03T15:44:12.522 INFO:tasks.workunit.client.0.smithi028.stdout:1/788: getdents d4/d2c/d3a/d5c/d6c/d7d/d4e 0 2024-07-03T15:44:12.525 INFO:tasks.workunit.client.0.smithi028.stdout:1/789: write d4/d2c/d3a/d5c/d6c/d7d/d4e/d51/f5e [101119,38437] 0 2024-07-03T15:44:12.525 INFO:tasks.workunit.client.0.smithi028.stdout:9/622: mkdir d1/d10/d76/dd7 0 2024-07-03T15:44:12.526 INFO:tasks.workunit.client.0.smithi028.stdout:8/659: dread d0/d41/d50/daf/dc8/da6/f8a [0,4194304] 0 2024-07-03T15:44:12.528 INFO:tasks.workunit.client.0.smithi028.stdout:9/623: mkdir d1/d16/d57/d5f/dd8 0 2024-07-03T15:44:12.529 INFO:tasks.workunit.client.0.smithi028.stdout:4/640: rmdir d2/d12/d5b/d78 39 2024-07-03T15:44:12.530 INFO:tasks.workunit.client.0.smithi028.stdout:1/790: getdents d4/d2c/d3a/d5c/d6c/d7d 0 2024-07-03T15:44:12.530 INFO:tasks.workunit.client.0.smithi028.stdout:1/791: creat d4/f111 x:0 0 0 2024-07-03T15:44:12.532 INFO:tasks.workunit.client.0.smithi028.stdout:9/624: unlink d1/d16/d57/d5f/d64/f75 0 2024-07-03T15:44:12.532 INFO:tasks.workunit.client.0.smithi028.stdout:9/625: readlink d1/d10/d28/d45/d61/d8c/d77/db7/lc2 0 2024-07-03T15:44:12.534 INFO:tasks.workunit.client.0.smithi028.stdout:1/792: creat d4/d3c/d54/d84/f112 x:0 0 0 2024-07-03T15:44:12.535 INFO:tasks.workunit.client.0.smithi028.stdout:1/793: write d4/d2c/d3a/d5c/d6c/d7d/d4e/fb9 [1847170,116363] 0 2024-07-03T15:44:12.536 INFO:tasks.workunit.client.0.smithi028.stdout:9/626: rename d1/d10/lb8 to d1/d10/d28/ld9 0 2024-07-03T15:44:12.540 INFO:tasks.workunit.client.0.smithi028.stdout:9/627: rename d1/d17/f9d to d1/d16/d57/fda 0 2024-07-03T15:44:12.541 INFO:tasks.workunit.client.0.smithi028.stdout:9/628: rename d1/d4a/d7e/lbb to d1/d16/ldb 0 2024-07-03T15:44:12.544 INFO:tasks.workunit.client.0.smithi028.stdout:1/794: read d4/d2c/d3a/d5c/d6c/d7d/f5d [4290484,9754] 0 2024-07-03T15:44:12.545 INFO:tasks.workunit.client.0.smithi028.stdout:1/795: link d4/dc2/f110 d4/d2c/d3a/d5c/d6c/d7d/d4e/f113 0 2024-07-03T15:44:12.546 INFO:tasks.workunit.client.0.smithi028.stdout:1/796: creat d4/d2c/d3a/d5c/d6c/d7d/f114 x:0 0 0 2024-07-03T15:44:12.546 INFO:tasks.workunit.client.0.smithi028.stdout:1/797: mkdir d4/d2c/d3a/d5c/d6c/d7d/d4e/d51/d115 0 2024-07-03T15:44:12.547 INFO:tasks.workunit.client.0.smithi028.stdout:1/798: getdents d4/d3c/d54/d79/d16/d57/d66/dd6 0 2024-07-03T15:44:12.548 INFO:tasks.workunit.client.0.smithi028.stdout:1/799: fdatasync d4/de/f85 0 2024-07-03T15:44:12.548 INFO:tasks.workunit.client.0.smithi028.stdout:1/800: truncate d4/d3c/d54/d79/d16/d7c/f9f 3321958 0 2024-07-03T15:44:12.549 INFO:tasks.workunit.client.0.smithi028.stdout:1/801: symlink d4/de/d1c/d81/l116 0 2024-07-03T15:44:12.550 INFO:tasks.workunit.client.0.smithi028.stdout:1/802: truncate d4/d3c/d54/d79/f8e 241647 0 2024-07-03T15:44:12.555 INFO:tasks.workunit.client.0.smithi028.stdout:8/660: dwrite d0/d41/d50/d3/dc/d3f/d8e/f6a [4194304,4194304] 0 2024-07-03T15:44:12.555 INFO:tasks.workunit.client.0.smithi028.stdout:8/661: getdents d0/d41/d50/d3/dc/d33/dbe 0 2024-07-03T15:44:12.555 INFO:tasks.workunit.client.0.smithi028.stdout:8/662: chown d0/d41/d50/d3/dc/d3f/d8e/fca 1974 1 2024-07-03T15:44:12.555 INFO:tasks.workunit.client.0.smithi028.stdout:8/663: fdatasync d0/d41/d50/d3/dc/d3f/d8e/f6a 0 2024-07-03T15:44:12.555 INFO:tasks.workunit.client.0.smithi028.stdout:8/664: readlink d0/d41/d50/d3/dc/d33/d7a/l1b 0 2024-07-03T15:44:12.555 INFO:tasks.workunit.client.0.smithi028.stdout:4/641: dwrite d2/f86 [0,4194304] 0 2024-07-03T15:44:12.555 INFO:tasks.workunit.client.0.smithi028.stdout:9/629: write d1/d10/d28/d4e/d4f/f87 [2467479,55457] 0 2024-07-03T15:44:12.559 INFO:tasks.workunit.client.0.smithi028.stdout:8/665: creat d0/d41/d50/daf/dc8/da6/da7/fda x:0 0 0 2024-07-03T15:44:12.560 INFO:tasks.workunit.client.0.smithi028.stdout:9/630: creat d1/d16/d57/d5f/d64/dc3/fdc x:0 0 0 2024-07-03T15:44:12.562 INFO:tasks.workunit.client.0.smithi028.stdout:9/631: creat d1/d10/d28/d45/fdd x:0 0 0 2024-07-03T15:44:12.562 INFO:tasks.workunit.client.0.smithi028.stdout:9/632: chown d1/dd 386580 1 2024-07-03T15:44:12.562 INFO:tasks.workunit.client.0.smithi028.stdout:9/633: dread - d1/d4a/d7e/f86 zero size 2024-07-03T15:44:12.563 INFO:tasks.workunit.client.0.smithi028.stdout:9/634: truncate d1/d10/d28/d45/d61/d8c/d77/fb5 69566 0 2024-07-03T15:44:12.563 INFO:tasks.workunit.client.0.smithi028.stdout:9/635: write d1/d16/d62/f8b [653149,6602] 0 2024-07-03T15:44:12.563 INFO:tasks.workunit.client.0.smithi028.stdout:9/636: write d1/d10/d28/d45/fdd [568936,113616] 0 2024-07-03T15:44:12.565 INFO:tasks.workunit.client.0.smithi028.stdout:1/803: dwrite d4/d2c/d3a/d5c/dff/f108 [4194304,4194304] 0 2024-07-03T15:44:12.567 INFO:tasks.workunit.client.0.smithi028.stdout:9/637: creat d1/d10/d76/fde x:0 0 0 2024-07-03T15:44:12.571 INFO:tasks.workunit.client.0.smithi028.stdout:1/804: creat d4/d3c/d54/f117 x:0 0 0 2024-07-03T15:44:12.575 INFO:tasks.workunit.client.0.smithi028.stdout:1/805: rmdir d4/d2c/d3a/d5c/d6c/d7d/d59/d94 39 2024-07-03T15:44:12.575 INFO:tasks.workunit.client.0.smithi028.stdout:1/806: write d4/d2c/d3a/d5c/d6c/d7d/f5d [6612768,10558] 0 2024-07-03T15:44:12.578 INFO:tasks.workunit.client.0.smithi028.stdout:8/666: dwrite d0/d6c/fae [0,4194304] 0 2024-07-03T15:44:12.579 INFO:tasks.workunit.client.0.smithi028.stdout:1/807: mkdir d4/d2c/d118 0 2024-07-03T15:44:12.582 INFO:tasks.workunit.client.0.smithi028.stdout:8/667: getdents d0/d41/d50/d3/dc 0 2024-07-03T15:44:12.582 INFO:tasks.workunit.client.0.smithi028.stdout:8/668: chown d0/d41/d50/d3/d35 6299638 1 2024-07-03T15:44:12.582 INFO:tasks.workunit.client.0.smithi028.stdout:8/669: fdatasync d0/d41/d50/d3/dc/f7e 0 2024-07-03T15:44:12.583 INFO:tasks.workunit.client.0.smithi028.stdout:8/670: fdatasync d0/d41/d50/d3/dc/fcc 0 2024-07-03T15:44:12.583 INFO:tasks.workunit.client.0.smithi028.stdout:8/671: write d0/d6c/da9/fd3 [568667,6194] 0 2024-07-03T15:44:12.583 INFO:tasks.workunit.client.0.smithi028.stdout:8/672: write d0/d41/d50/d3/d35/d86/f87 [3383527,28627] 0 2024-07-03T15:44:12.583 INFO:tasks.workunit.client.0.smithi028.stdout:8/673: creat d0/d41/d50/daf/dc8/da6/da7/fdb x:0 0 0 2024-07-03T15:44:12.583 INFO:tasks.workunit.client.0.smithi028.stdout:8/674: chown d0/d41/d50/d3/dc/d3f/d8e/d69/cb5 24921 1 2024-07-03T15:44:12.583 INFO:tasks.workunit.client.0.smithi028.stdout:1/808: mknod d4/d2c/d3a/d5c/dd4/c119 0 2024-07-03T15:44:12.583 INFO:tasks.workunit.client.0.smithi028.stdout:9/638: dwrite d1/d17/f9a [0,4194304] 0 2024-07-03T15:44:12.584 INFO:tasks.workunit.client.0.smithi028.stdout:1/809: mkdir d4/d3c/d54/d79/d11a 0 2024-07-03T15:44:12.584 INFO:tasks.workunit.client.0.smithi028.stdout:1/810: stat d4/d83 0 2024-07-03T15:44:12.591 INFO:tasks.workunit.client.0.smithi028.stdout:9/639: dread d1/d16/f63 [0,4194304] 0 2024-07-03T15:44:12.593 INFO:tasks.workunit.client.0.smithi028.stdout:9/640: link d1/d10/f3e d1/d10/d28/d45/d56/dcd/dd5/fdf 0 2024-07-03T15:44:12.596 INFO:tasks.workunit.client.0.smithi028.stdout:1/811: dread d4/de/f6f [0,4194304] 0 2024-07-03T15:44:12.598 INFO:tasks.workunit.client.0.smithi028.stdout:1/812: creat d4/d2c/d118/f11b x:0 0 0 2024-07-03T15:44:12.600 INFO:tasks.workunit.client.0.smithi028.stdout:8/675: dwrite d0/d41/d50/d3/dc/d33/d7a/f56 [0,4194304] 0 2024-07-03T15:44:12.600 INFO:tasks.workunit.client.0.smithi028.stdout:1/813: creat d4/d3c/d54/d79/d16/db6/ddb/f11c x:0 0 0 2024-07-03T15:44:12.602 INFO:tasks.workunit.client.0.smithi028.stdout:8/676: symlink d0/d41/d50/d3/dc/d3f/d8e/d81/d9e/ldc 0 2024-07-03T15:44:12.602 INFO:tasks.workunit.client.0.smithi028.stdout:8/677: read d0/d41/d50/d3/dc/d33/d7a/f39 [733398,75196] 0 2024-07-03T15:44:12.603 INFO:tasks.workunit.client.0.smithi028.stdout:8/678: dread - d0/d41/d50/d3/dc/d3f/d8e/d81/fc3 zero size 2024-07-03T15:44:12.604 INFO:tasks.workunit.client.0.smithi028.stdout:1/814: symlink d4/de/d1c/d55/l11d 0 2024-07-03T15:44:12.604 INFO:tasks.workunit.client.0.smithi028.stdout:1/815: readlink d4/d3c/d54/d79/d16/d57/d66/la0 0 2024-07-03T15:44:12.604 INFO:tasks.workunit.client.0.smithi028.stdout:1/816: stat d4/d3c/d54/d79/f109 0 2024-07-03T15:44:12.605 INFO:tasks.workunit.client.0.smithi028.stdout:8/679: rename d0/d41/d50/d3/dc/c70 to d0/d41/d50/d3/d71/cdd 0 2024-07-03T15:44:12.607 INFO:tasks.workunit.client.0.smithi028.stdout:1/817: unlink d4/d83/fac 0 2024-07-03T15:44:12.607 INFO:tasks.workunit.client.0.smithi028.stdout:1/818: truncate d4/d3c/d54/d79/f1b 4636687 0 2024-07-03T15:44:12.607 INFO:tasks.workunit.client.0.smithi028.stdout:1/819: creat d4/d2c/d118/f11e x:0 0 0 2024-07-03T15:44:12.607 INFO:tasks.workunit.client.0.smithi028.stdout:1/820: creat d4/de/d1c/d81/f11f x:0 0 0 2024-07-03T15:44:12.610 INFO:tasks.workunit.client.0.smithi028.stdout:9/641: dwrite d1/fa1 [0,4194304] 0 2024-07-03T15:44:12.610 INFO:tasks.workunit.client.0.smithi028.stdout:8/680: rmdir d0/d41/d50/d3/d71 39 2024-07-03T15:44:12.612 INFO:tasks.workunit.client.0.smithi028.stdout:9/642: link d1/d16/d57/fb1 d1/d16/d57/d5f/fe0 0 2024-07-03T15:44:12.612 INFO:tasks.workunit.client.0.smithi028.stdout:9/643: dread - d1/d4a/d7e/f86 zero size 2024-07-03T15:44:12.613 INFO:tasks.workunit.client.0.smithi028.stdout:8/681: creat d0/d41/d50/d3/dc/d3f/d8e/d5d/fde x:0 0 0 2024-07-03T15:44:12.613 INFO:tasks.workunit.client.0.smithi028.stdout:8/682: write d0/d41/d50/d3/dc/f23 [478512,11810] 0 2024-07-03T15:44:12.613 INFO:tasks.workunit.client.0.smithi028.stdout:9/644: stat d1/d10/f5d 0 2024-07-03T15:44:12.614 INFO:tasks.workunit.client.0.smithi028.stdout:9/645: chown d1/d10/d28/d4e/d5b 909 1 2024-07-03T15:44:12.614 INFO:tasks.workunit.client.0.smithi028.stdout:8/683: truncate d0/d41/d50/d3/d35/d86/f87 1165796 0 2024-07-03T15:44:12.615 INFO:tasks.workunit.client.0.smithi028.stdout:8/684: truncate d0/f2a 1044019 0 2024-07-03T15:44:12.615 INFO:tasks.workunit.client.0.smithi028.stdout:1/821: dread d4/d2c/f75 [0,4194304] 0 2024-07-03T15:44:12.617 INFO:tasks.workunit.client.0.smithi028.stdout:8/685: mknod d0/d41/d50/daf/dc1/cdf 0 2024-07-03T15:44:12.620 INFO:tasks.workunit.client.0.smithi028.stdout:8/686: rename d0/d41/d92 to d0/de0 0 2024-07-03T15:44:12.626 INFO:tasks.workunit.client.0.smithi028.stdout:8/687: getdents d0/d41/d50/d3/dc 0 2024-07-03T15:44:12.626 INFO:tasks.workunit.client.0.smithi028.stdout:8/688: stat d0/d41/d50/d3/dc/d3f/f61 0 2024-07-03T15:44:12.632 INFO:tasks.workunit.client.0.smithi028.stdout:9/646: dwrite d1/d10/d28/fd3 [0,4194304] 0 2024-07-03T15:44:12.632 INFO:tasks.workunit.client.0.smithi028.stdout:9/647: fsync d1/d10/d28/fd0 0 2024-07-03T15:44:12.637 INFO:tasks.workunit.client.0.smithi028.stdout:1/822: dwrite d4/de/f10 [0,4194304] 0 2024-07-03T15:44:12.640 INFO:tasks.workunit.client.0.smithi028.stdout:1/823: dread d4/d83/de8/fdc [0,4194304] 0 2024-07-03T15:44:12.641 INFO:tasks.workunit.client.0.smithi028.stdout:1/824: getdents d4/d2c/d3a/d5c/dd4 0 2024-07-03T15:44:12.641 INFO:tasks.workunit.client.0.smithi028.stdout:1/825: chown d4/d3c/d54/d79/d16/db6/dd1/ffe 26457 1 2024-07-03T15:44:12.649 INFO:tasks.workunit.client.0.smithi028.stdout:1/826: dread d4/f37 [0,4194304] 0 2024-07-03T15:44:12.649 INFO:tasks.workunit.client.0.smithi028.stdout:1/827: fsync d4/d3c/d54/d79/d16/fe2 0 2024-07-03T15:44:12.650 INFO:tasks.workunit.client.0.smithi028.stdout:1/828: symlink d4/de/da7/l120 0 2024-07-03T15:44:12.653 INFO:tasks.workunit.client.0.smithi028.stdout:1/829: rename d4/d2c to d4/d83/de8/d121 0 2024-07-03T15:44:12.653 INFO:tasks.workunit.client.0.smithi028.stdout:1/830: chown d4/d3c/d54/d79/d16/d57/d66/d10e 334691 1 2024-07-03T15:44:12.654 INFO:tasks.workunit.client.0.smithi028.stdout:1/831: mknod d4/d83/c122 0 2024-07-03T15:44:12.659 INFO:tasks.workunit.client.0.smithi028.stdout:8/689: dwrite d0/d41/d50/d3/dc/d3f/d8e/d81/fbd [0,4194304] 0 2024-07-03T15:44:12.660 INFO:tasks.workunit.client.0.smithi028.stdout:8/690: chown d0/d41/d50/d3/d35/d86 77692920 1 2024-07-03T15:44:12.661 INFO:tasks.workunit.client.0.smithi028.stdout:8/691: dread d0/d41/d50/d3/d35/d68/d22/d40/d4a/fd8 [0,4194304] 0 2024-07-03T15:44:12.662 INFO:tasks.workunit.client.0.smithi028.stdout:8/692: read d0/d41/d50/d3/d35/d86/f87 [478886,109682] 0 2024-07-03T15:44:12.663 INFO:tasks.workunit.client.0.smithi028.stdout:7/445: sync 2024-07-03T15:44:12.666 INFO:tasks.workunit.client.0.smithi028.stdout:1/832: read d4/d83/de8/d121/d3a/d5c/d6c/d7d/d4e/d51/fa2 [96633,686] 0 2024-07-03T15:44:12.666 INFO:tasks.workunit.client.0.smithi028.stdout:9/648: dwrite d1/d10/f18 [0,4194304] 0 2024-07-03T15:44:12.667 INFO:tasks.workunit.client.0.smithi028.stdout:9/649: getdents d1/d16/d57/d5f/dd8 0 2024-07-03T15:44:12.668 INFO:tasks.workunit.client.0.smithi028.stdout:1/833: mkdir d4/d83/de8/d121/d3a/d5c/daa/df0/d123 0 2024-07-03T15:44:12.669 INFO:tasks.workunit.client.0.smithi028.stdout:9/650: mknod d1/d10/d28/ce1 0 2024-07-03T15:44:12.669 INFO:tasks.workunit.client.0.smithi028.stdout:9/651: stat d1/l7 0 2024-07-03T15:44:12.673 INFO:tasks.workunit.client.0.smithi028.stdout:8/693: write d0/d41/d50/d3/dc/d4d/fc6 [670587,103573] 0 2024-07-03T15:44:12.674 INFO:tasks.workunit.client.0.smithi028.stdout:8/694: stat d0/d41/d50/daf/dc1/dd1/c54 0 2024-07-03T15:44:12.677 INFO:tasks.workunit.client.0.smithi028.stdout:7/446: dwrite d9/d1c/f5c [0,4194304] 0 2024-07-03T15:44:12.678 INFO:tasks.workunit.client.0.smithi028.stdout:7/447: truncate d9/d29/f77 377218 0 2024-07-03T15:44:12.681 INFO:tasks.workunit.client.0.smithi028.stdout:7/448: creat d9/f84 x:0 0 0 2024-07-03T15:44:12.681 INFO:tasks.workunit.client.0.smithi028.stdout:7/449: chown d9/d1c/d52/d6a/f7d 1273 1 2024-07-03T15:44:12.681 INFO:tasks.workunit.client.0.smithi028.stdout:7/450: creat d9/d1c/d52/d6a/f85 x:0 0 0 2024-07-03T15:44:12.684 INFO:tasks.workunit.client.0.smithi028.stdout:7/451: rmdir d9/d1c/d45 39 2024-07-03T15:44:12.684 INFO:tasks.workunit.client.0.smithi028.stdout:7/452: truncate d9/d29/f44 4800825 0 2024-07-03T15:44:12.687 INFO:tasks.workunit.client.0.smithi028.stdout:7/453: dread d9/d29/f61 [0,4194304] 0 2024-07-03T15:44:12.687 INFO:tasks.workunit.client.0.smithi028.stdout:7/454: write d9/fa [4460874,83950] 0 2024-07-03T15:44:12.687 INFO:tasks.workunit.client.0.smithi028.stdout:7/455: read - d9/d1c/d52/d6a/f85 zero size 2024-07-03T15:44:12.688 INFO:tasks.workunit.client.0.smithi028.stdout:7/456: readlink d9/d1c/d52/l64 0 2024-07-03T15:44:12.688 INFO:tasks.workunit.client.0.smithi028.stdout:7/457: write d9/d1c/d45/d71/f7f [136687,31606] 0 2024-07-03T15:44:12.688 INFO:tasks.workunit.client.0.smithi028.stdout:7/458: chown d9/d29/c75 24686 1 2024-07-03T15:44:12.689 INFO:tasks.workunit.client.0.smithi028.stdout:7/459: unlink d9/d1c/d7c/f80 0 2024-07-03T15:44:12.690 INFO:tasks.workunit.client.0.smithi028.stdout:9/652: dwrite d1/dd/f59 [0,4194304] 0 2024-07-03T15:44:12.690 INFO:tasks.workunit.client.0.smithi028.stdout:9/653: stat d1/d10/d28/fd0 0 2024-07-03T15:44:12.691 INFO:tasks.workunit.client.0.smithi028.stdout:9/654: creat d1/d10/d28/d45/d61/d8c/d77/db7/dab/dcb/fe2 x:0 0 0 2024-07-03T15:44:12.692 INFO:tasks.workunit.client.0.smithi028.stdout:7/460: mkdir d9/d1c/d86 0 2024-07-03T15:44:12.693 INFO:tasks.workunit.client.0.smithi028.stdout:9/655: mkdir d1/d16/d57/d5f/de3 0 2024-07-03T15:44:12.694 INFO:tasks.workunit.client.0.smithi028.stdout:7/461: symlink d9/d1c/d86/l87 0 2024-07-03T15:44:12.695 INFO:tasks.workunit.client.0.smithi028.stdout:9/656: truncate d1/d10/f5d 1154521 0 2024-07-03T15:44:12.695 INFO:tasks.workunit.client.0.smithi028.stdout:9/657: readlink d1/d10/d28/d45/d61/d8c/d77/db7/lc2 0 2024-07-03T15:44:12.696 INFO:tasks.workunit.client.0.smithi028.stdout:7/462: symlink d9/d1c/d45/d71/l88 0 2024-07-03T15:44:12.696 INFO:tasks.workunit.client.0.smithi028.stdout:7/463: read d9/d1c/d45/f57 [125729,11834] 0 2024-07-03T15:44:12.700 INFO:tasks.workunit.client.0.smithi028.stdout:8/695: dwrite d0/d41/d50/d3/dc/d3f/f61 [0,4194304] 0 2024-07-03T15:44:12.701 INFO:tasks.workunit.client.0.smithi028.stdout:7/464: link d9/f41 d9/d1c/d86/f89 0 2024-07-03T15:44:12.701 INFO:tasks.workunit.client.0.smithi028.stdout:7/465: chown d9/d1c/d52/d6a/l6f 15069359 1 2024-07-03T15:44:12.701 INFO:tasks.workunit.client.0.smithi028.stdout:1/834: dwrite d4/d83/de8/d121/f78 [0,4194304] 0 2024-07-03T15:44:12.708 INFO:tasks.workunit.client.0.smithi028.stdout:7/466: creat d9/d1c/d6e/f8a x:0 0 0 2024-07-03T15:44:12.708 INFO:tasks.workunit.client.0.smithi028.stdout:7/467: write d9/d1c/f6b [15917,11480] 0 2024-07-03T15:44:12.710 INFO:tasks.workunit.client.0.smithi028.stdout:9/658: dwrite d1/d10/d28/d45/d61/d8c/f6a [0,4194304] 0 2024-07-03T15:44:12.721 INFO:tasks.workunit.client.0.smithi028.stdout:7/468: mkdir d9/d1c/d7c/d8b 0 2024-07-03T15:44:12.721 INFO:tasks.workunit.client.0.smithi028.stdout:7/469: truncate d9/d29/d5d/f7a 655059 0 2024-07-03T15:44:12.721 INFO:tasks.workunit.client.0.smithi028.stdout:7/470: write d9/d1c/d6e/f73 [977088,12968] 0 2024-07-03T15:44:12.721 INFO:tasks.workunit.client.0.smithi028.stdout:7/471: chown d9/c56 43164 1 2024-07-03T15:44:12.728 INFO:tasks.workunit.client.0.smithi028.stdout:7/472: dread d9/d29/f5e [0,4194304] 0 2024-07-03T15:44:12.728 INFO:tasks.workunit.client.0.smithi028.stdout:7/473: chown d9/d1c/d45/d71/f7f 11 1 2024-07-03T15:44:12.728 INFO:tasks.workunit.client.0.smithi028.stdout:7/474: write d9/d1c/d6e/f73 [681812,110273] 0 2024-07-03T15:44:12.728 INFO:tasks.workunit.client.0.smithi028.stdout:7/475: chown d9/d1c/d45 2229619 1 2024-07-03T15:44:12.735 INFO:tasks.workunit.client.0.smithi028.stdout:8/696: dwrite d0/d41/d50/d3/f3a [0,4194304] 0 2024-07-03T15:44:12.736 INFO:tasks.workunit.client.0.smithi028.stdout:1/835: dwrite d4/d83/de8/d121/d3a/d5c/d6c/d7d/d4e/d51/f5e [0,4194304] 0 2024-07-03T15:44:12.736 INFO:tasks.workunit.client.0.smithi028.stdout:1/836: write d4/d83/de8/d121/d3a/d5c/d6c/d7d/d4e/f113 [460981,53775] 0 2024-07-03T15:44:12.736 INFO:tasks.workunit.client.0.smithi028.stdout:1/837: chown d4/d83/de8/d121/d3a/d5c/lf2 600 1 2024-07-03T15:44:12.736 INFO:tasks.workunit.client.0.smithi028.stdout:1/838: read - d4/d83/de8/d121/d3a/d5c/d6c/d7d/d4e/d51/fe0 zero size 2024-07-03T15:44:12.736 INFO:tasks.workunit.client.0.smithi028.stdout:9/659: dwrite d1/dd/fb6 [0,4194304] 0 2024-07-03T15:44:12.736 INFO:tasks.workunit.client.0.smithi028.stdout:9/660: dread - d1/d10/d76/fde zero size 2024-07-03T15:44:12.739 INFO:tasks.workunit.client.0.smithi028.stdout:7/476: dread d9/f3a [0,4194304] 0 2024-07-03T15:44:12.759 INFO:tasks.workunit.client.0.smithi028.stdout:7/477: dwrite d9/d1c/f6b [0,4194304] 0 2024-07-03T15:44:12.759 INFO:tasks.workunit.client.0.smithi028.stdout:7/478: fdatasync d9/f41 0 2024-07-03T15:44:12.759 INFO:tasks.workunit.client.0.smithi028.stdout:7/479: chown d9/d29/f3c 483139 1 2024-07-03T15:44:12.759 INFO:tasks.workunit.client.0.smithi028.stdout:7/480: creat d9/d1c/d6e/f8c x:0 0 0 2024-07-03T15:44:12.763 INFO:tasks.workunit.client.0.smithi028.stdout:1/839: mknod d4/d3c/d54/d79/d16/c124 0 2024-07-03T15:44:12.764 INFO:tasks.workunit.client.0.smithi028.stdout:1/840: read d4/d83/de8/d121/d3a/d5c/d6c/d7d/d4e/d51/f5f [1480280,76346] 0 2024-07-03T15:44:12.768 INFO:tasks.workunit.client.0.smithi028.stdout:9/661: truncate d1/d16/f2e 2800517 0 2024-07-03T15:44:12.768 INFO:tasks.workunit.client.0.smithi028.stdout:9/662: fdatasync d1/d10/d28/d45/d61/fa0 0 2024-07-03T15:44:12.772 INFO:tasks.workunit.client.0.smithi028.stdout:1/841: mkdir d4/d83/de8/d121/d3a/d5c/daa/df0/d123/d125 0 2024-07-03T15:44:12.773 INFO:tasks.workunit.client.0.smithi028.stdout:3/617: sync 2024-07-03T15:44:12.773 INFO:tasks.workunit.client.0.smithi028.stdout:0/707: sync 2024-07-03T15:44:12.773 INFO:tasks.workunit.client.0.smithi028.stdout:6/698: sync 2024-07-03T15:44:12.773 INFO:tasks.workunit.client.0.smithi028.stdout:2/486: sync 2024-07-03T15:44:12.773 INFO:tasks.workunit.client.0.smithi028.stdout:5/621: sync 2024-07-03T15:44:12.775 INFO:tasks.workunit.client.0.smithi028.stdout:4/642: sync 2024-07-03T15:44:12.776 INFO:tasks.workunit.client.0.smithi028.stdout:1/842: creat d4/de/d1c/d81/d10d/f126 x:0 0 0 2024-07-03T15:44:12.780 INFO:tasks.workunit.client.0.smithi028.stdout:3/618: truncate d18/d3a/f5a 4045374 0 2024-07-03T15:44:12.785 INFO:tasks.workunit.client.0.smithi028.stdout:1/843: dread d4/d3c/d54/fe7 [0,4194304] 0 2024-07-03T15:44:12.785 INFO:tasks.workunit.client.0.smithi028.stdout:6/699: truncate d4/d12/fd2 1163177 0 2024-07-03T15:44:12.785 INFO:tasks.workunit.client.0.smithi028.stdout:6/700: stat d4/d12/d2c/fe0 0 2024-07-03T15:44:12.787 INFO:tasks.workunit.client.0.smithi028.stdout:5/622: creat d0/dd/d13/d57/d30/d70/d7f/d9e/fd2 x:0 0 0 2024-07-03T15:44:12.790 INFO:tasks.workunit.client.0.smithi028.stdout:9/663: truncate d1/d10/d28/d45/d61/fa0 1935694 0 2024-07-03T15:44:12.792 INFO:tasks.workunit.client.0.smithi028.stdout:1/844: dread d4/da4/fb0 [0,4194304] 0 2024-07-03T15:44:12.793 INFO:tasks.workunit.client.0.smithi028.stdout:2/487: write da/d1e/d68/f6c [1204401,34245] 0 2024-07-03T15:44:12.793 INFO:tasks.workunit.client.0.smithi028.stdout:2/488: chown da/d24/f55 3919 1 2024-07-03T15:44:12.794 INFO:tasks.workunit.client.0.smithi028.stdout:3/619: dwrite d18/d1a/d1e/d58/f96 [0,4194304] 0 2024-07-03T15:44:12.795 INFO:tasks.workunit.client.0.smithi028.stdout:0/708: mknod d2/d3/cdb 0 2024-07-03T15:44:12.795 INFO:tasks.workunit.client.0.smithi028.stdout:0/709: chown d2/d3/dc4/l62 0 1 2024-07-03T15:44:12.803 INFO:tasks.workunit.client.0.smithi028.stdout:5/623: creat d0/dd/d14/dc0/fd3 x:0 0 0 2024-07-03T15:44:12.803 INFO:tasks.workunit.client.0.smithi028.stdout:5/624: chown d0/dd/df/da7 0 1 2024-07-03T15:44:12.806 INFO:tasks.workunit.client.0.smithi028.stdout:9/664: link d1/d17/f9a d1/d10/d28/d45/d61/d8c/d77/db7/dab/dcb/fe4 0 2024-07-03T15:44:12.807 INFO:tasks.workunit.client.0.smithi028.stdout:9/665: chown d1/d10/d28/daa/lc7 394708349 1 2024-07-03T15:44:12.808 INFO:tasks.workunit.client.0.smithi028.stdout:1/845: unlink d4/d83/de8/d121/d3a/d5c/d6c/d7d/f89 0 2024-07-03T15:44:12.811 INFO:tasks.workunit.client.0.smithi028.stdout:2/489: unlink da/d24/d2c/c29 0 2024-07-03T15:44:12.812 INFO:tasks.workunit.client.0.smithi028.stdout:4/643: write d2/d3e/d9f/fb4 [171043,42096] 0 2024-07-03T15:44:12.812 INFO:tasks.workunit.client.0.smithi028.stdout:6/701: dwrite d4/d12/f4f [0,4194304] 0 2024-07-03T15:44:12.816 INFO:tasks.workunit.client.0.smithi028.stdout:0/710: mkdir d2/d3/d4d/ddc 0 2024-07-03T15:44:12.818 INFO:tasks.workunit.client.0.smithi028.stdout:5/625: mknod d0/dd/d13/d57/d30/d70/d7f/dbb/cd4 0 2024-07-03T15:44:12.820 INFO:tasks.workunit.client.0.smithi028.stdout:1/846: creat d4/d83/de8/d121/d3a/d5c/daa/df0/f127 x:0 0 0 2024-07-03T15:44:12.821 INFO:tasks.workunit.client.0.smithi028.stdout:4/644: symlink d2/d12/d42/d63/da8/ld4 0 2024-07-03T15:44:12.821 INFO:tasks.workunit.client.0.smithi028.stdout:6/702: mkdir d4/da/d38/d3d/df9 0 2024-07-03T15:44:12.823 INFO:tasks.workunit.client.0.smithi028.stdout:3/620: rename d18/c35 to d18/d43/cc0 0 2024-07-03T15:44:12.823 INFO:tasks.workunit.client.0.smithi028.stdout:3/621: chown f9 127656577 1 2024-07-03T15:44:12.824 INFO:tasks.workunit.client.0.smithi028.stdout:0/711: symlink d2/d6d/ldd 0 2024-07-03T15:44:12.825 INFO:tasks.workunit.client.0.smithi028.stdout:4/645: link d2/d12/d2e/l49 d2/ld5 0 2024-07-03T15:44:12.826 INFO:tasks.workunit.client.0.smithi028.stdout:4/646: chown d2/d5a/f36 75297 1 2024-07-03T15:44:12.830 INFO:tasks.workunit.client.0.smithi028.stdout:2/490: rename da/d23/c8f to da/d23/d6f/cb4 0 2024-07-03T15:44:12.830 INFO:tasks.workunit.client.0.smithi028.stdout:2/491: stat da/d42/d47/da2 0 2024-07-03T15:44:12.831 INFO:tasks.workunit.client.0.smithi028.stdout:6/703: read d4/da/d1c/fb1 [2149192,107324] 0 2024-07-03T15:44:12.831 INFO:tasks.workunit.client.0.smithi028.stdout:6/704: readlink d4/d12/d2c/l35 0 2024-07-03T15:44:12.831 INFO:tasks.workunit.client.0.smithi028.stdout:3/622: mkdir d18/d3a/dc1 0 2024-07-03T15:44:12.832 INFO:tasks.workunit.client.0.smithi028.stdout:0/712: mknod d2/d3/d4d/d56/d78/da8/cde 0 2024-07-03T15:44:12.833 INFO:tasks.workunit.client.0.smithi028.stdout:0/713: write d2/d1c/f1e [4822108,41754] 0 2024-07-03T15:44:12.835 INFO:tasks.workunit.client.0.smithi028.stdout:9/666: dwrite d1/d17/f9a [4194304,4194304] 0 2024-07-03T15:44:12.837 INFO:tasks.workunit.client.0.smithi028.stdout:1/847: dread d4/d83/de8/d121/d3a/d5c/fbf [0,4194304] 0 2024-07-03T15:44:12.837 INFO:tasks.workunit.client.0.smithi028.stdout:4/647: rename d2/d61/c77 to d2/d61/d9d/cd6 0 2024-07-03T15:44:12.837 INFO:tasks.workunit.client.0.smithi028.stdout:4/648: dread - d2/d5a/d45/f7c zero size 2024-07-03T15:44:12.837 INFO:tasks.workunit.client.0.smithi028.stdout:4/649: write d2/d12/d2e/fb1 [867330,32045] 0 2024-07-03T15:44:12.837 INFO:tasks.workunit.client.0.smithi028.stdout:4/650: chown d2/d61/cb0 1453160 1 2024-07-03T15:44:12.837 INFO:tasks.workunit.client.0.smithi028.stdout:4/651: readlink d2/d12/d2d/l33 0 2024-07-03T15:44:12.837 INFO:tasks.workunit.client.0.smithi028.stdout:4/652: readlink d2/l1d 0 2024-07-03T15:44:12.837 INFO:tasks.workunit.client.0.smithi028.stdout:4/653: fsync d2/d12/d2d/d58/f69 0 2024-07-03T15:44:12.837 INFO:tasks.workunit.client.0.smithi028.stdout:6/705: unlink d4/da/d47/ca1 0 2024-07-03T15:44:12.838 INFO:tasks.workunit.client.0.smithi028.stdout:6/706: truncate d4/f10 4665755 0 2024-07-03T15:44:12.838 INFO:tasks.workunit.client.0.smithi028.stdout:6/707: creat d4/da/d38/d60/dc5/ffa x:0 0 0 2024-07-03T15:44:12.838 INFO:tasks.workunit.client.0.smithi028.stdout:6/708: fsync d4/d12/d2c/ddd/fec 0 2024-07-03T15:44:12.838 INFO:tasks.workunit.client.0.smithi028.stdout:3/623: link la d18/d1a/d1e/d73/lc2 0 2024-07-03T15:44:12.838 INFO:tasks.workunit.client.0.smithi028.stdout:5/626: dwrite d0/dd/d13/d57/f1f [0,4194304] 0 2024-07-03T15:44:12.840 INFO:tasks.workunit.client.0.smithi028.stdout:0/714: creat d2/d3/d4d/d56/d78/dcf/fdf x:0 0 0 2024-07-03T15:44:12.840 INFO:tasks.workunit.client.0.smithi028.stdout:0/715: fsync d2/d1c/d6c/da9/fb0 0 2024-07-03T15:44:12.847 INFO:tasks.workunit.client.0.smithi028.stdout:1/848: symlink d4/d83/de8/d121/d3a/d5c/d6c/d7d/d4e/d51/d115/l128 0 2024-07-03T15:44:12.853 INFO:tasks.workunit.client.0.smithi028.stdout:3/624: symlink d18/d1a/d1e/d58/d9f/da9/lc3 0 2024-07-03T15:44:12.853 INFO:tasks.workunit.client.0.smithi028.stdout:2/492: dwrite da/d42/d47/d78/fb1 [8388608,4194304] 0 2024-07-03T15:44:12.856 INFO:tasks.workunit.client.0.smithi028.stdout:0/716: creat d2/d3/dc4/d58/d82/d9b/fe0 x:0 0 0 2024-07-03T15:44:12.856 INFO:tasks.workunit.client.0.smithi028.stdout:0/717: dread - d2/d3/d4d/d56/d78/dcf/fdf zero size 2024-07-03T15:44:12.856 INFO:tasks.workunit.client.0.smithi028.stdout:0/718: readlink d2/d3/l22 0 2024-07-03T15:44:12.861 INFO:tasks.workunit.client.0.smithi028.stdout:1/849: creat d4/d3c/d54/d79/d16/db6/f129 x:0 0 0 2024-07-03T15:44:12.861 INFO:tasks.workunit.client.0.smithi028.stdout:1/850: truncate d4/d3c/d54/d79/d16/fe2 715569 0 2024-07-03T15:44:12.863 INFO:tasks.workunit.client.0.smithi028.stdout:3/625: dread f17 [0,4194304] 0 2024-07-03T15:44:12.863 INFO:tasks.workunit.client.0.smithi028.stdout:2/493: mknod da/d42/d47/d78/cb5 0 2024-07-03T15:44:12.863 INFO:tasks.workunit.client.0.smithi028.stdout:2/494: stat da/d42/f50 0 2024-07-03T15:44:12.863 INFO:tasks.workunit.client.0.smithi028.stdout:2/495: read - da/d42/d47/d78/fa3 zero size 2024-07-03T15:44:12.864 INFO:tasks.workunit.client.0.smithi028.stdout:1/851: mkdir d4/d83/de8/d121/d3a/d5c/dd4/d12a 0 2024-07-03T15:44:12.864 INFO:tasks.workunit.client.0.smithi028.stdout:1/852: write d4/d83/de8/d121/d3a/d5c/d6c/d7d/d59/dc6/ffd [64261,120077] 0 2024-07-03T15:44:12.865 INFO:tasks.workunit.client.0.smithi028.stdout:2/496: creat da/d24/fb6 x:0 0 0 2024-07-03T15:44:12.866 INFO:tasks.workunit.client.0.smithi028.stdout:0/719: mknod d2/d3/ce1 0 2024-07-03T15:44:12.866 INFO:tasks.workunit.client.0.smithi028.stdout:0/720: chown d2/d3/dc4/d36/d45/f7b 1510348433 1 2024-07-03T15:44:12.867 INFO:tasks.workunit.client.0.smithi028.stdout:1/853: rmdir d4/d3c/d54/d79/d16/d7c 39 2024-07-03T15:44:12.869 INFO:tasks.workunit.client.0.smithi028.stdout:0/721: rename d2/d1c/f7e to d2/d3/d4d/d56/d78/da8/dd7/fe2 0 2024-07-03T15:44:12.869 INFO:tasks.workunit.client.0.smithi028.stdout:0/722: write d2/d3/f11 [77831,54927] 0 2024-07-03T15:44:12.871 INFO:tasks.workunit.client.0.smithi028.stdout:0/723: creat d2/d1c/d6c/dc5/fe3 x:0 0 0 2024-07-03T15:44:12.871 INFO:tasks.workunit.client.0.smithi028.stdout:0/724: chown d2/d1c/la5 5089 1 2024-07-03T15:44:12.883 INFO:tasks.workunit.client.0.smithi028.stdout:2/497: write da/d23/d30/f51 [1914013,45200] 0 2024-07-03T15:44:12.883 INFO:tasks.workunit.client.0.smithi028.stdout:2/498: stat da/d74/f8d 0 2024-07-03T15:44:12.890 INFO:tasks.workunit.client.0.smithi028.stdout:4/654: dwrite d2/d12/d2e/f38 [0,4194304] 0 2024-07-03T15:44:12.891 INFO:tasks.workunit.client.0.smithi028.stdout:2/499: dread da/d23/f3b [0,4194304] 0 2024-07-03T15:44:12.893 INFO:tasks.workunit.client.0.smithi028.stdout:5/627: dwrite d0/dd/d1e/f36 [0,4194304] 0 2024-07-03T15:44:12.894 INFO:tasks.workunit.client.0.smithi028.stdout:1/854: dwrite d4/d83/de8/d121/d3a/d5c/d6c/d7d/d4e/fb9 [0,4194304] 0 2024-07-03T15:44:12.894 INFO:tasks.workunit.client.0.smithi028.stdout:6/709: dwrite d4/f3a [4194304,4194304] 0 2024-07-03T15:44:12.894 INFO:tasks.workunit.client.0.smithi028.stdout:9/667: dwrite d1/d10/d28/d45/d61/d8c/d77/db7/fb0 [0,4194304] 0 2024-07-03T15:44:12.894 INFO:tasks.workunit.client.0.smithi028.stdout:9/668: fdatasync d1/dd/fb6 0 2024-07-03T15:44:12.894 INFO:tasks.workunit.client.0.smithi028.stdout:9/669: fdatasync d1/d16/d57/d5f/fd4 0 2024-07-03T15:44:12.895 INFO:tasks.workunit.client.0.smithi028.stdout:9/670: chown d1/d16/d62/c9b 8026853 1 2024-07-03T15:44:12.896 INFO:tasks.workunit.client.0.smithi028.stdout:5/628: stat d0/dd/d1e/d3b/d3f/c51 0 2024-07-03T15:44:12.896 INFO:tasks.workunit.client.0.smithi028.stdout:0/725: dwrite d2/f49 [0,4194304] 0 2024-07-03T15:44:12.899 INFO:tasks.workunit.client.0.smithi028.stdout:6/710: dread d4/da/d38/d3d/d89/dd8/fee [4194304,4194304] 0 2024-07-03T15:44:12.899 INFO:tasks.workunit.client.0.smithi028.stdout:6/711: chown d4/da/d38/d60/dc5/dcf/fe2 0 1 2024-07-03T15:44:12.899 INFO:tasks.workunit.client.0.smithi028.stdout:1/855: link d4/cae d4/de/da7/c12b 0 2024-07-03T15:44:12.905 INFO:tasks.workunit.client.0.smithi028.stdout:5/629: write d0/dd/d1e/d3b/fa3 [3714804,42218] 0 2024-07-03T15:44:12.906 INFO:tasks.workunit.client.0.smithi028.stdout:5/630: stat d0/dd/d13/d57/d26/c46 0 2024-07-03T15:44:12.909 INFO:tasks.workunit.client.0.smithi028.stdout:6/712: symlink d4/da/d38/d3d/df9/lfb 0 2024-07-03T15:44:12.921 INFO:tasks.workunit.client.0.smithi028.stdout:1/856: rename d4/d83/de8/d121/d3a/d5c/lf2 to d4/d3c/d54/l12c 0 2024-07-03T15:44:12.925 INFO:tasks.workunit.client.0.smithi028.stdout:9/671: rmdir d1/d16/d57/d5f/d64 39 2024-07-03T15:44:12.929 INFO:tasks.workunit.client.0.smithi028.stdout:1/857: rename d4/d3c/d54/f8d to d4/d3c/d54/d79/d16/db6/f12d 0 2024-07-03T15:44:12.930 INFO:tasks.workunit.client.0.smithi028.stdout:9/672: symlink d1/d8e/dac/le5 0 2024-07-03T15:44:12.933 INFO:tasks.workunit.client.0.smithi028.stdout:6/713: getdents d4/da/d47 0 2024-07-03T15:44:12.935 INFO:tasks.workunit.client.0.smithi028.stdout:1/858: unlink d4/d83/de8/d121/d3a/d5c/d6c/d7d/d59/f82 0 2024-07-03T15:44:12.935 INFO:tasks.workunit.client.0.smithi028.stdout:1/859: write d4/d83/de8/d121/d3a/d5c/d6c/d7d/d4e/d51/f5f [792210,122130] 0 2024-07-03T15:44:12.938 INFO:tasks.workunit.client.0.smithi028.stdout:9/673: creat d1/d10/d28/d45/d61/d8c/d77/db7/fe6 x:0 0 0 2024-07-03T15:44:12.938 INFO:tasks.workunit.client.0.smithi028.stdout:9/674: fsync d1/d17/f25 0 2024-07-03T15:44:12.939 INFO:tasks.workunit.client.0.smithi028.stdout:6/714: creat d4/d15/de5/ffc x:0 0 0 2024-07-03T15:44:12.940 INFO:tasks.workunit.client.0.smithi028.stdout:9/675: symlink d1/d10/d28/d45/d56/dcd/dd5/le7 0 2024-07-03T15:44:12.940 INFO:tasks.workunit.client.0.smithi028.stdout:9/676: stat d1/l15 0 2024-07-03T15:44:12.941 INFO:tasks.workunit.client.0.smithi028.stdout:1/860: dread d4/de/fc3 [0,4194304] 0 2024-07-03T15:44:12.941 INFO:tasks.workunit.client.0.smithi028.stdout:1/861: fsync d4/d3c/d54/d79/d16/d57/d66/f67 0 2024-07-03T15:44:12.941 INFO:tasks.workunit.client.0.smithi028.stdout:1/862: write d4/d3c/d54/d79/d16/fe2 [295615,38279] 0 2024-07-03T15:44:12.942 INFO:tasks.workunit.client.0.smithi028.stdout:4/655: dwrite d2/d12/d5b/d78/f79 [0,4194304] 0 2024-07-03T15:44:12.943 INFO:tasks.workunit.client.0.smithi028.stdout:6/715: mkdir d4/da/d1c/dfd 0 2024-07-03T15:44:12.943 INFO:tasks.workunit.client.0.smithi028.stdout:6/716: truncate d4/d15/fc7 1008441 0 2024-07-03T15:44:12.943 INFO:tasks.workunit.client.0.smithi028.stdout:6/717: write d4/da/d47/d9f/fa5 [1390688,32769] 0 2024-07-03T15:44:12.943 INFO:tasks.workunit.client.0.smithi028.stdout:6/718: dread - d4/fc9 zero size 2024-07-03T15:44:12.943 INFO:tasks.workunit.client.0.smithi028.stdout:0/726: dwrite d2/d3/dc4/d58/d6e/f94 [0,4194304] 0 2024-07-03T15:44:12.944 INFO:tasks.workunit.client.0.smithi028.stdout:2/500: dwrite da/ff [0,4194304] 0 2024-07-03T15:44:12.945 INFO:tasks.workunit.client.0.smithi028.stdout:5/631: dwrite d0/dd/d1e/d94/fcc [0,4194304] 0 2024-07-03T15:44:12.949 INFO:tasks.workunit.client.0.smithi028.stdout:4/656: rename d2/d12/d2d/d2f/lab to d2/d12/d42/d63/d6d/ld7 0 2024-07-03T15:44:12.949 INFO:tasks.workunit.client.0.smithi028.stdout:9/677: dread d1/d4a/f65 [0,4194304] 0 2024-07-03T15:44:12.949 INFO:tasks.workunit.client.0.smithi028.stdout:9/678: truncate d1/d8e/f9e 1994401 0 2024-07-03T15:44:12.950 INFO:tasks.workunit.client.0.smithi028.stdout:6/719: mkdir d4/da/d38/d60/dc5/dfe 0 2024-07-03T15:44:12.950 INFO:tasks.workunit.client.0.smithi028.stdout:6/720: creat d4/d12/d2c/dac/fff x:0 0 0 2024-07-03T15:44:12.953 INFO:tasks.workunit.client.0.smithi028.stdout:0/727: truncate d2/d3/dc4/d58/f6a 2591077 0 2024-07-03T15:44:12.953 INFO:tasks.workunit.client.0.smithi028.stdout:0/728: readlink d2/d3/dc4/d58/d82/lc3 0 2024-07-03T15:44:12.959 INFO:tasks.workunit.client.0.smithi028.stdout:2/501: dread f4 [4194304,4194304] 0 2024-07-03T15:44:12.959 INFO:tasks.workunit.client.0.smithi028.stdout:2/502: chown da/f76 7700746 1 2024-07-03T15:44:12.959 INFO:tasks.workunit.client.0.smithi028.stdout:2/503: readlink da/d24/l5a 0 2024-07-03T15:44:12.959 INFO:tasks.workunit.client.0.smithi028.stdout:2/504: readlink da/d23/d6f/l88 0 2024-07-03T15:44:12.961 INFO:tasks.workunit.client.0.smithi028.stdout:5/632: rename d0/dd/d13/f17 to d0/dd/d13/d57/d30/d70/d7f/d9e/fd5 0 2024-07-03T15:44:12.969 INFO:tasks.workunit.client.0.smithi028.stdout:2/505: mknod da/cb7 0 2024-07-03T15:44:12.971 INFO:tasks.workunit.client.0.smithi028.stdout:4/657: dwrite d2/d5a/d45/d84/f8f [0,4194304] 0 2024-07-03T15:44:12.971 INFO:tasks.workunit.client.0.smithi028.stdout:4/658: chown d2/d12/d5b/f8c 1800070897 1 2024-07-03T15:44:12.971 INFO:tasks.workunit.client.0.smithi028.stdout:4/659: write d2/d12/f17 [1465058,21759] 0 2024-07-03T15:44:12.971 INFO:tasks.workunit.client.0.smithi028.stdout:4/660: chown d2/d12/d42 1352364 1 2024-07-03T15:44:12.971 INFO:tasks.workunit.client.0.smithi028.stdout:4/661: read - d2/d61/fc5 zero size 2024-07-03T15:44:12.974 INFO:tasks.workunit.client.0.smithi028.stdout:2/506: mkdir da/d23/d6f/d84/db8 0 2024-07-03T15:44:12.982 INFO:tasks.workunit.client.0.smithi028.stdout:4/662: dread d2/d5a/f36 [0,4194304] 0 2024-07-03T15:44:12.982 INFO:tasks.workunit.client.0.smithi028.stdout:2/507: rename da/d23/l53 to da/d24/lb9 0 2024-07-03T15:44:12.982 INFO:tasks.workunit.client.0.smithi028.stdout:2/508: readlink da/l25 0 2024-07-03T15:44:12.984 INFO:tasks.workunit.client.0.smithi028.stdout:6/721: dwrite d4/da/fa8 [0,4194304] 0 2024-07-03T15:44:12.984 INFO:tasks.workunit.client.0.smithi028.stdout:5/633: dwrite d0/dd/d1e/f56 [0,4194304] 0 2024-07-03T15:44:12.984 INFO:tasks.workunit.client.0.smithi028.stdout:5/634: fsync d0/dd/d1e/f89 0 2024-07-03T15:44:12.984 INFO:tasks.workunit.client.0.smithi028.stdout:5/635: write d0/dd/d13/d47/d50/f8c [524416,125134] 0 2024-07-03T15:44:12.984 INFO:tasks.workunit.client.0.smithi028.stdout:5/636: chown d0/dd/d13/d29 845 1 2024-07-03T15:44:12.984 INFO:tasks.workunit.client.0.smithi028.stdout:5/637: readlink d0/dd/d1e/l9a 0 2024-07-03T15:44:12.984 INFO:tasks.workunit.client.0.smithi028.stdout:0/729: dwrite d2/d3/dc4/d58/d6e/f94 [0,4194304] 0 2024-07-03T15:44:12.985 INFO:tasks.workunit.client.0.smithi028.stdout:4/663: rmdir d2/d61/d9c/dcd 0 2024-07-03T15:44:12.985 INFO:tasks.workunit.client.0.smithi028.stdout:2/509: mkdir da/d24/d2c/dba 0 2024-07-03T15:44:12.986 INFO:tasks.workunit.client.0.smithi028.stdout:2/510: dread da/d23/d30/f67 [0,4194304] 0 2024-07-03T15:44:12.988 INFO:tasks.workunit.client.0.smithi028.stdout:9/679: rmdir d1/d10/d28/d45/d61/d8c 39 2024-07-03T15:44:12.991 INFO:tasks.workunit.client.0.smithi028.stdout:9/680: dread d1/d16/d62/f8b [0,4194304] 0 2024-07-03T15:44:12.992 INFO:tasks.workunit.client.0.smithi028.stdout:9/681: write d1/d10/d28/d45/d56/dcd/dd5/fdf [353846,118117] 0 2024-07-03T15:44:12.993 INFO:tasks.workunit.client.0.smithi028.stdout:6/722: unlink d4/d12/fd2 0 2024-07-03T15:44:12.995 INFO:tasks.workunit.client.0.smithi028.stdout:5/638: mkdir d0/dd/d13/d57/d30/d70/d7f/d9e/dd6 0 2024-07-03T15:44:12.999 INFO:tasks.workunit.client.0.smithi028.stdout:0/730: mkdir d2/d3/d4d/d56/dcd/de4 0 2024-07-03T15:44:13.006 INFO:tasks.workunit.client.0.smithi028.stdout:9/682: dwrite d1/d17/f9a [0,4194304] 0 2024-07-03T15:44:13.007 INFO:tasks.workunit.client.0.smithi028.stdout:9/683: read d1/dd/f4c [1903209,115311] 0 2024-07-03T15:44:13.009 INFO:tasks.workunit.client.0.smithi028.stdout:9/684: write d1/d10/d28/d45/d61/d8c/d77/f91 [849409,118891] 0 2024-07-03T15:44:13.017 INFO:tasks.workunit.client.0.smithi028.stdout:1/863: truncate d4/d83/de8/d121/d3a/d5c/d6c/d7d/d4e/fb9 4127478 0 2024-07-03T15:44:13.020 INFO:tasks.workunit.client.0.smithi028.stdout:2/511: mknod da/d24/d2c/cbb 0 2024-07-03T15:44:13.029 INFO:tasks.workunit.client.0.smithi028.stdout:6/723: creat d4/da/d38/d60/db5/f100 x:0 0 0 2024-07-03T15:44:13.029 INFO:tasks.workunit.client.0.smithi028.stdout:6/724: chown d4/d12/d2c/dac/fff 38000 1 2024-07-03T15:44:13.034 INFO:tasks.workunit.client.0.smithi028.stdout:9/685: symlink d1/d17/le8 0 2024-07-03T15:44:13.036 INFO:tasks.workunit.client.0.smithi028.stdout:1/864: creat d4/d3c/d54/f12e x:0 0 0 2024-07-03T15:44:13.039 INFO:tasks.workunit.client.0.smithi028.stdout:6/725: mknod d4/d12/dc3/c101 0 2024-07-03T15:44:13.039 INFO:tasks.workunit.client.0.smithi028.stdout:6/726: readlink d4/da/d47/lb4 0 2024-07-03T15:44:13.040 INFO:tasks.workunit.client.0.smithi028.stdout:6/727: write d4/d12/dc3/fdc [335640,31144] 0 2024-07-03T15:44:13.040 INFO:tasks.workunit.client.0.smithi028.stdout:6/728: write d4/d12/d2c/dac/fff [487267,46308] 0 2024-07-03T15:44:13.040 INFO:tasks.workunit.client.0.smithi028.stdout:6/729: write d4/da/d47/d9f/fdf [849412,83834] 0 2024-07-03T15:44:13.040 INFO:tasks.workunit.client.0.smithi028.stdout:5/639: rmdir d0/dd/d13/d57/d77 39 2024-07-03T15:44:13.040 INFO:tasks.workunit.client.0.smithi028.stdout:5/640: readlink d0/lb 0 2024-07-03T15:44:13.042 INFO:tasks.workunit.client.0.smithi028.stdout:0/731: rename d2/d3/dc4/l97 to d2/d3/le5 0 2024-07-03T15:44:13.045 INFO:tasks.workunit.client.0.smithi028.stdout:9/686: truncate d1/d10/d28/d45/d61/d8c/d77/db7/fb0 2749774 0 2024-07-03T15:44:13.048 INFO:tasks.workunit.client.0.smithi028.stdout:2/512: read da/d1e/d56/f9a [2612088,32389] 0 2024-07-03T15:44:13.055 INFO:tasks.workunit.client.0.smithi028.stdout:1/865: rmdir d4/d3c/d54/d79/d16/d57/d66/dd6 39 2024-07-03T15:44:13.055 INFO:tasks.workunit.client.0.smithi028.stdout:1/866: dread - d4/d83/f10c zero size 2024-07-03T15:44:13.055 INFO:tasks.workunit.client.0.smithi028.stdout:6/730: dwrite d4/da/d38/d60/dc5/dcf/fe2 [0,4194304] 0 2024-07-03T15:44:13.056 INFO:tasks.workunit.client.0.smithi028.stdout:5/641: creat d0/d87/fd7 x:0 0 0 2024-07-03T15:44:13.057 INFO:tasks.workunit.client.0.smithi028.stdout:0/732: link d2/d3/d4d/d56/d78/d9f/lca d2/d3/d4d/d56/d78/da8/le6 0 2024-07-03T15:44:13.059 INFO:tasks.workunit.client.0.smithi028.stdout:9/687: mkdir d1/d10/d28/d4e/d5b/de9 0 2024-07-03T15:44:13.059 INFO:tasks.workunit.client.0.smithi028.stdout:2/513: chown da/d1e/f7b 76939150 1 2024-07-03T15:44:13.064 INFO:tasks.workunit.client.0.smithi028.stdout:9/688: dread d1/f4b [0,4194304] 0 2024-07-03T15:44:13.066 INFO:tasks.workunit.client.0.smithi028.stdout:5/642: creat d0/dd/d1e/d3b/d3f/fd8 x:0 0 0 2024-07-03T15:44:13.066 INFO:tasks.workunit.client.0.smithi028.stdout:6/731: write d4/f3a [5865124,88705] 0 2024-07-03T15:44:13.067 INFO:tasks.workunit.client.0.smithi028.stdout:5/643: mknod d0/d8f/cd9 0 2024-07-03T15:44:13.067 INFO:tasks.workunit.client.0.smithi028.stdout:5/644: chown d0/dd/d13/d57/d26/f2e 640452158 1 2024-07-03T15:44:13.067 INFO:tasks.workunit.client.0.smithi028.stdout:5/645: dread - d0/dd/d1e/d8e/fa6 zero size 2024-07-03T15:44:13.067 INFO:tasks.workunit.client.0.smithi028.stdout:5/646: write d0/dd/d13/d57/d77/f7d [2629147,123857] 0 2024-07-03T15:44:13.069 INFO:tasks.workunit.client.0.smithi028.stdout:6/732: creat d4/da/d38/d3d/df9/f102 x:0 0 0 2024-07-03T15:44:13.071 INFO:tasks.workunit.client.0.smithi028.stdout:5/647: symlink d0/d8f/db5/lda 0 2024-07-03T15:44:13.071 INFO:tasks.workunit.client.0.smithi028.stdout:5/648: stat d0/dd/d13/d57/d30/f90 0 2024-07-03T15:44:13.086 INFO:tasks.workunit.client.0.smithi028.stdout:1/867: dwrite d4/d3c/d54/d79/f12 [0,4194304] 0 2024-07-03T15:44:13.088 INFO:tasks.workunit.client.0.smithi028.stdout:2/514: dwrite da/f77 [0,4194304] 0 2024-07-03T15:44:13.088 INFO:tasks.workunit.client.0.smithi028.stdout:2/515: fsync da/f76 0 2024-07-03T15:44:13.095 INFO:tasks.workunit.client.0.smithi028.stdout:2/516: getdents da/d42/d9e 0 2024-07-03T15:44:13.096 INFO:tasks.workunit.client.0.smithi028.stdout:2/517: rename da/d24/fb6 to da/d42/d47/fbc 0 2024-07-03T15:44:13.097 INFO:tasks.workunit.client.0.smithi028.stdout:2/518: write da/d23/d30/f3e [829876,60652] 0 2024-07-03T15:44:13.098 INFO:tasks.workunit.client.0.smithi028.stdout:2/519: symlink da/d1e/d68/lbd 0 2024-07-03T15:44:13.098 INFO:tasks.workunit.client.0.smithi028.stdout:2/520: chown da/d42/c94 6047629 1 2024-07-03T15:44:13.099 INFO:tasks.workunit.client.0.smithi028.stdout:9/689: dwrite d1/d16/d57/f9c [0,4194304] 0 2024-07-03T15:44:13.102 INFO:tasks.workunit.client.0.smithi028.stdout:6/733: dwrite d4/da/d1c/fb0 [0,4194304] 0 2024-07-03T15:44:13.102 INFO:tasks.workunit.client.0.smithi028.stdout:6/734: creat d4/d12/d2c/ddd/f103 x:0 0 0 2024-07-03T15:44:13.103 INFO:tasks.workunit.client.0.smithi028.stdout:0/733: dwrite d2/d1c/f20 [4194304,4194304] 0 2024-07-03T15:44:13.103 INFO:tasks.workunit.client.0.smithi028.stdout:0/734: readlink d2/d3/d4d/d56/l86 0 2024-07-03T15:44:13.103 INFO:tasks.workunit.client.0.smithi028.stdout:9/690: mkdir d1/d10/dea 0 2024-07-03T15:44:13.109 INFO:tasks.workunit.client.0.smithi028.stdout:9/691: symlink d1/d10/d28/d45/d61/d8c/d77/db7/dab/dca/leb 0 2024-07-03T15:44:13.119 INFO:tasks.workunit.client.0.smithi028.stdout:1/868: dwrite d4/de/f10 [0,4194304] 0 2024-07-03T15:44:13.124 INFO:tasks.workunit.client.0.smithi028.stdout:1/869: creat d4/d83/de8/d121/d3a/d5c/d6c/d7d/d59/d94/f12f x:0 0 0 2024-07-03T15:44:13.125 INFO:tasks.workunit.client.0.smithi028.stdout:1/870: write d4/d83/de8/d121/d3a/d5c/f98 [792307,91335] 0 2024-07-03T15:44:13.125 INFO:tasks.workunit.client.0.smithi028.stdout:1/871: dread - d4/d83/de8/d121/d3a/d5c/d6c/d7d/d59/dc6/fdd zero size 2024-07-03T15:44:13.126 INFO:tasks.workunit.client.0.smithi028.stdout:1/872: dread d4/d3c/d54/d84/f9b [0,4194304] 0 2024-07-03T15:44:13.127 INFO:tasks.workunit.client.0.smithi028.stdout:0/735: dwrite d2/d1c/f2e [0,4194304] 0 2024-07-03T15:44:13.132 INFO:tasks.workunit.client.0.smithi028.stdout:2/521: dwrite da/d23/d30/f67 [0,4194304] 0 2024-07-03T15:44:13.132 INFO:tasks.workunit.client.0.smithi028.stdout:1/873: link d4/d3c/d54/d79/d16/db6/ddb/lf4 d4/de/d1c/df1/l130 0 2024-07-03T15:44:13.134 INFO:tasks.workunit.client.0.smithi028.stdout:1/874: creat d4/d83/de8/d121/d3a/d5c/daa/df0/d123/d125/f131 x:0 0 0 2024-07-03T15:44:13.148 INFO:tasks.workunit.client.0.smithi028.stdout:1/875: write d4/d83/de8/d121/d3a/d5c/d6c/d7d/d59/f2a [1772898,50805] 0 2024-07-03T15:44:13.154 INFO:tasks.workunit.client.0.smithi028.stdout:1/876: creat d4/d83/de8/d121/d3a/d5c/daa/f132 x:0 0 0 2024-07-03T15:44:13.154 INFO:tasks.workunit.client.0.smithi028.stdout:1/877: stat d4/d3c/d54/d79/d16/d57/l65 0 2024-07-03T15:44:13.156 INFO:tasks.workunit.client.0.smithi028.stdout:2/522: dwrite da/d74/f8d [0,4194304] 0 2024-07-03T15:44:13.156 INFO:tasks.workunit.client.0.smithi028.stdout:2/523: chown da/d23/d30/c31 653078 1 2024-07-03T15:44:13.156 INFO:tasks.workunit.client.0.smithi028.stdout:2/524: creat da/d24/fbe x:0 0 0 2024-07-03T15:44:13.157 INFO:tasks.workunit.client.0.smithi028.stdout:1/878: dread d4/d83/de8/d121/d3a/d5c/fcf [0,4194304] 0 2024-07-03T15:44:13.161 INFO:tasks.workunit.client.0.smithi028.stdout:1/879: mkdir d4/d3c/d54/d79/d16/d133 0 2024-07-03T15:44:13.164 INFO:tasks.workunit.client.0.smithi028.stdout:2/525: rmdir da/d74 39 2024-07-03T15:44:13.165 INFO:tasks.workunit.client.0.smithi028.stdout:2/526: chown da/d42/d47/fa5 0 1 2024-07-03T15:44:13.167 INFO:tasks.workunit.client.0.smithi028.stdout:1/880: rename d4/de/cc8 to d4/d3c/d54/d79/d11a/c134 0 2024-07-03T15:44:13.171 INFO:tasks.workunit.client.0.smithi028.stdout:2/527: mknod da/d1e/cbf 0 2024-07-03T15:44:13.174 INFO:tasks.workunit.client.0.smithi028.stdout:1/881: truncate d4/d3c/f9e 3280312 0 2024-07-03T15:44:13.184 INFO:tasks.workunit.client.0.smithi028.stdout:5/649: write d0/dd/d98/f6a [729979,16389] 0 2024-07-03T15:44:13.184 INFO:tasks.workunit.client.0.smithi028.stdout:5/650: truncate d0/dd/d13/d57/d30/d70/d7f/d9e/fd2 842558 0 2024-07-03T15:44:13.187 INFO:tasks.workunit.client.0.smithi028.stdout:5/651: mknod d0/dd/d13/d57/d30/d70/d7f/cdb 0 2024-07-03T15:44:13.189 INFO:tasks.workunit.client.0.smithi028.stdout:5/652: getdents d0/dd/d13/d57/d77 0 2024-07-03T15:44:13.189 INFO:tasks.workunit.client.0.smithi028.stdout:5/653: chown d0/dd/d1e/d3b/d3f/d71 116284803 1 2024-07-03T15:44:13.192 INFO:tasks.workunit.client.0.smithi028.stdout:5/654: rmdir d0/d87 39 2024-07-03T15:44:13.194 INFO:tasks.workunit.client.0.smithi028.stdout:5/655: truncate d0/dd/d98/f6c 2907845 0 2024-07-03T15:44:13.196 INFO:tasks.workunit.client.0.smithi028.stdout:5/656: mkdir d0/d8f/db5/ddc 0 2024-07-03T15:44:13.197 INFO:tasks.workunit.client.0.smithi028.stdout:2/528: dwrite da/d24/d2c/f61 [0,4194304] 0 2024-07-03T15:44:13.198 INFO:tasks.workunit.client.0.smithi028.stdout:5/657: mkdir d0/dd/d1e/d3b/ddd 0 2024-07-03T15:44:13.198 INFO:tasks.workunit.client.0.smithi028.stdout:5/658: fdatasync d0/dd/d14/dc0/fd3 0 2024-07-03T15:44:13.198 INFO:tasks.workunit.client.0.smithi028.stdout:5/659: stat d0/dd/d1e/d3b/d3f 0 2024-07-03T15:44:13.198 INFO:tasks.workunit.client.0.smithi028.stdout:5/660: write d0/dd/d13/d57/d26/f2e [4886654,74841] 0 2024-07-03T15:44:13.201 INFO:tasks.workunit.client.0.smithi028.stdout:1/882: dwrite d4/d83/de8/d121/d3a/d5c/d6c/d7d/f114 [0,4194304] 0 2024-07-03T15:44:13.207 INFO:tasks.workunit.client.0.smithi028.stdout:1/883: rmdir d4/d3c/d54/d79/d16/d7c 39 2024-07-03T15:44:13.222 INFO:tasks.workunit.client.0.smithi028.stdout:5/661: dwrite d0/dd/d1e/d3b/fa3 [4194304,4194304] 0 2024-07-03T15:44:13.223 INFO:tasks.workunit.client.0.smithi028.stdout:5/662: creat d0/dd/d13/d57/d30/fde x:0 0 0 2024-07-03T15:44:13.224 INFO:tasks.workunit.client.0.smithi028.stdout:5/663: rmdir d0/dd/d13/d57/d30/d70/d7f/dbb 39 2024-07-03T15:44:13.225 INFO:tasks.workunit.client.0.smithi028.stdout:5/664: mkdir d0/dd/d13/d57/d26/ddf 0 2024-07-03T15:44:13.231 INFO:tasks.workunit.client.0.smithi028.stdout:2/529: dwrite da/d42/d47/fbc [0,4194304] 0 2024-07-03T15:44:13.231 INFO:tasks.workunit.client.0.smithi028.stdout:1/884: dwrite d4/de/f31 [4194304,4194304] 0 2024-07-03T15:44:13.242 INFO:tasks.workunit.client.0.smithi028.stdout:1/885: write d4/d83/de8/d121/d3a/d5c/d6c/d7d/d4e/fb9 [2966798,122865] 0 2024-07-03T15:44:13.244 INFO:tasks.workunit.client.0.smithi028.stdout:1/886: link d4/d83/de8/d121/l45 d4/d83/de8/d121/d3a/d5c/daa/df0/l135 0 2024-07-03T15:44:13.244 INFO:tasks.workunit.client.0.smithi028.stdout:1/887: read d4/d3c/d54/d79/d16/fe2 [615111,79402] 0 2024-07-03T15:44:13.245 INFO:tasks.workunit.client.0.smithi028.stdout:1/888: mknod d4/d83/de8/c136 0 2024-07-03T15:44:13.258 INFO:tasks.workunit.client.0.smithi028.stdout:1/889: dwrite d4/d83/de8/d121/d3a/d5c/d6c/d7d/d4e/d51/fe0 [0,4194304] 0 2024-07-03T15:44:13.258 INFO:tasks.workunit.client.0.smithi028.stdout:1/890: stat d4/d3c/d54/d79/d16/fb4 0 2024-07-03T15:44:13.258 INFO:tasks.workunit.client.0.smithi028.stdout:1/891: write d4/d83/de8/d121/f92 [4847364,62952] 0 2024-07-03T15:44:13.258 INFO:tasks.workunit.client.0.smithi028.stdout:1/892: truncate d4/d83/de8/d121/d3a/d5c/d6c/d7d/d59/dc6/fdd 185206 0 2024-07-03T15:44:13.266 INFO:tasks.workunit.client.0.smithi028.stdout:1/893: write d4/de/fc5 [82300,99489] 0 2024-07-03T15:44:13.266 INFO:tasks.workunit.client.0.smithi028.stdout:1/894: write d4/d3c/d54/d79/d16/db6/ddb/f11c [554716,37037] 0 2024-07-03T15:44:13.342 INFO:tasks.workunit.client.0.smithi028.stdout:1/895: mkdir d4/d83/de8/d121/d3a/d5c/d6c/d7d/d137 0 2024-07-03T15:44:13.357 INFO:tasks.workunit.client.0.smithi028.stdout:1/896: mkdir d4/d3c/d54/d79/d16/d57/d138 0 2024-07-03T15:44:13.357 INFO:tasks.workunit.client.0.smithi028.stdout:1/897: chown d4/d83/de8/d121/d3a/d5c/d6c/d7d 3622 1 2024-07-03T15:44:13.374 INFO:tasks.workunit.client.0.smithi028.stdout:1/898: dwrite d4/d83/de8/d121/d3a/d5c/f106 [0,4194304] 0 2024-07-03T15:44:13.375 INFO:tasks.workunit.client.0.smithi028.stdout:1/899: dread - d4/de/da7/fe5 zero size 2024-07-03T15:44:13.389 INFO:tasks.workunit.client.0.smithi028.stdout:8/697: sync 2024-07-03T15:44:13.389 INFO:tasks.workunit.client.0.smithi028.stdout:7/481: sync 2024-07-03T15:44:13.389 INFO:tasks.workunit.client.0.smithi028.stdout:8/698: stat d0/d41/d50/daf/dc8/da6 0 2024-07-03T15:44:13.389 INFO:tasks.workunit.client.0.smithi028.stdout:8/699: chown d0/d41/d50/d3/dc/c6e 2010830 1 2024-07-03T15:44:13.390 INFO:tasks.workunit.client.0.smithi028.stdout:8/700: creat d0/d41/d50/d3/d35/d86/fe1 x:0 0 0 2024-07-03T15:44:13.391 INFO:tasks.workunit.client.0.smithi028.stdout:8/701: symlink d0/d41/d50/d3/d35/le2 0 2024-07-03T15:44:13.393 INFO:tasks.workunit.client.0.smithi028.stdout:8/702: mknod d0/d41/d50/d3/dc/d33/ce3 0 2024-07-03T15:44:13.393 INFO:tasks.workunit.client.0.smithi028.stdout:8/703: getdents d0/de0/dcf 0 2024-07-03T15:44:13.395 INFO:tasks.workunit.client.0.smithi028.stdout:8/704: creat d0/d41/d50/daf/dc8/fe4 x:0 0 0 2024-07-03T15:44:13.395 INFO:tasks.workunit.client.0.smithi028.stdout:8/705: chown d0/d41/c52 2 1 2024-07-03T15:44:13.398 INFO:tasks.workunit.client.0.smithi028.stdout:7/482: read d9/f41 [1811046,10896] 0 2024-07-03T15:44:13.398 INFO:tasks.workunit.client.0.smithi028.stdout:7/483: chown d9/d1c 1225 1 2024-07-03T15:44:13.398 INFO:tasks.workunit.client.0.smithi028.stdout:7/484: fsync d9/f41 0 2024-07-03T15:44:13.398 INFO:tasks.workunit.client.0.smithi028.stdout:8/706: rmdir d0/d41/d50/d3/d35/d68/d22 39 2024-07-03T15:44:13.401 INFO:tasks.workunit.client.0.smithi028.stdout:8/707: getdents d0/d41/d50/daf/dc8/da6 0 2024-07-03T15:44:13.402 INFO:tasks.workunit.client.0.smithi028.stdout:8/708: mknod d0/d41/d50/d3/d35/d68/ce5 0 2024-07-03T15:44:13.403 INFO:tasks.workunit.client.0.smithi028.stdout:8/709: mkdir d0/d41/d50/d3/d35/d68/d22/d40/d4a/de6 0 2024-07-03T15:44:13.420 INFO:tasks.workunit.client.0.smithi028.stdout:7/485: dwrite d9/d29/f72 [0,4194304] 0 2024-07-03T15:44:13.421 INFO:tasks.workunit.client.0.smithi028.stdout:8/710: dwrite d0/d41/d50/d3/dc/d3f/d8e/d5d/fde [0,4194304] 0 2024-07-03T15:44:13.425 INFO:tasks.workunit.client.0.smithi028.stdout:7/486: dread d9/f41 [0,4194304] 0 2024-07-03T15:44:13.426 INFO:tasks.workunit.client.0.smithi028.stdout:8/711: rename d0/d41/d50/d3/dc/d3f/d8e/c95 to d0/de0/dcf/ce7 0 2024-07-03T15:44:13.428 INFO:tasks.workunit.client.0.smithi028.stdout:7/487: symlink d9/d1c/d52/d6a/l8d 0 2024-07-03T15:44:13.428 INFO:tasks.workunit.client.0.smithi028.stdout:7/488: fsync d9/d29/f77 0 2024-07-03T15:44:13.428 INFO:tasks.workunit.client.0.smithi028.stdout:7/489: fdatasync f4 0 2024-07-03T15:44:13.428 INFO:tasks.workunit.client.0.smithi028.stdout:8/712: symlink d0/d41/d50/d3/d35/d68/d22/d40/dd2/le8 0 2024-07-03T15:44:13.430 INFO:tasks.workunit.client.0.smithi028.stdout:7/490: mknod d9/d1c/d45/d71/d78/c8e 0 2024-07-03T15:44:13.430 INFO:tasks.workunit.client.0.smithi028.stdout:7/491: chown d9/fa 17233196 1 2024-07-03T15:44:13.430 INFO:tasks.workunit.client.0.smithi028.stdout:8/713: rmdir d0/d41/d50/d3/d35 39 2024-07-03T15:44:13.438 INFO:tasks.workunit.client.0.smithi028.stdout:7/492: dread d9/d29/f42 [0,4194304] 0 2024-07-03T15:44:13.438 INFO:tasks.workunit.client.0.smithi028.stdout:7/493: rename d9/d1c/d7c to d9/d1c/d7c/d8b/d8f 22 2024-07-03T15:44:13.438 INFO:tasks.workunit.client.0.smithi028.stdout:7/494: write d9/d1c/f3f [1107505,5892] 0 2024-07-03T15:44:13.438 INFO:tasks.workunit.client.0.smithi028.stdout:7/495: dread - d9/f22 zero size 2024-07-03T15:44:13.438 INFO:tasks.workunit.client.0.smithi028.stdout:7/496: chown d9/d1c/d7c/d8b 189 1 2024-07-03T15:44:13.438 INFO:tasks.workunit.client.0.smithi028.stdout:7/497: readlink d9/d1c/l54 0 2024-07-03T15:44:13.438 INFO:tasks.workunit.client.0.smithi028.stdout:7/498: write d9/d1c/d86/f89 [1953778,7132] 0 2024-07-03T15:44:13.440 INFO:tasks.workunit.client.0.smithi028.stdout:7/499: creat d9/d1c/d7c/d8b/f90 x:0 0 0 2024-07-03T15:44:13.441 INFO:tasks.workunit.client.0.smithi028.stdout:7/500: fsync d9/d29/f44 0 2024-07-03T15:44:13.441 INFO:tasks.workunit.client.0.smithi028.stdout:7/501: dread - d9/d1c/d45/f53 zero size 2024-07-03T15:44:13.441 INFO:tasks.workunit.client.0.smithi028.stdout:7/502: write d9/d29/f61 [1221507,6361] 0 2024-07-03T15:44:13.441 INFO:tasks.workunit.client.0.smithi028.stdout:7/503: symlink d9/d1c/d45/l91 0 2024-07-03T15:44:13.443 INFO:tasks.workunit.client.0.smithi028.stdout:7/504: mkdir d9/d29/d92 0 2024-07-03T15:44:13.446 INFO:tasks.workunit.client.0.smithi028.stdout:8/714: dwrite d0/d41/d50/d3/d35/d68/d22/d40/f60 [0,4194304] 0 2024-07-03T15:44:13.454 INFO:tasks.workunit.client.0.smithi028.stdout:8/715: dread d0/d41/d50/d3/d35/d68/d22/d40/d62/f65 [0,4194304] 0 2024-07-03T15:44:13.455 INFO:tasks.workunit.client.0.smithi028.stdout:8/716: mknod d0/d6c/ce9 0 2024-07-03T15:44:13.464 INFO:tasks.workunit.client.0.smithi028.stdout:7/505: dwrite d9/d1c/d7c/d8b/f90 [0,4194304] 0 2024-07-03T15:44:13.466 INFO:tasks.workunit.client.0.smithi028.stdout:7/506: creat d9/d1c/f93 x:0 0 0 2024-07-03T15:44:13.471 INFO:tasks.workunit.client.0.smithi028.stdout:8/717: dwrite d0/d41/d50/d3/dc/d3f/d8e/d81/fc3 [0,4194304] 0 2024-07-03T15:44:13.486 INFO:tasks.workunit.client.0.smithi028.stdout:8/718: dwrite d0/d41/d50/d3/d35/d86/fe1 [0,4194304] 0 2024-07-03T15:44:13.486 INFO:tasks.workunit.client.0.smithi028.stdout:8/719: write d0/d6c/fae [3657040,47430] 0 2024-07-03T15:44:13.491 INFO:tasks.workunit.client.0.smithi028.stdout:8/720: link d0/d41/d50/d3/d35/d68/d22/f25 d0/d41/d50/d3/dc/d3f/d8e/d81/d9e/fea 0 2024-07-03T15:44:13.497 INFO:tasks.workunit.client.0.smithi028.stdout:8/721: creat d0/d41/d50/daf/feb x:0 0 0 2024-07-03T15:44:13.497 INFO:tasks.workunit.client.0.smithi028.stdout:8/722: chown d0/d41/d50/d3/dc/d33/f7d 1298 1 2024-07-03T15:44:13.502 INFO:tasks.workunit.client.0.smithi028.stdout:8/723: rmdir d0/d41/d50/daf/dc8 39 2024-07-03T15:44:13.504 INFO:tasks.workunit.client.0.smithi028.stdout:8/724: rename d0/d41/d50/d3/dc/f7e to d0/d41/d50/d3/dc/d3f/d8e/fec 0 2024-07-03T15:44:13.504 INFO:tasks.workunit.client.0.smithi028.stdout:8/725: write d0/d41/d50/daf/dc8/da6/f8a [946251,104118] 0 2024-07-03T15:44:13.520 INFO:tasks.workunit.client.0.smithi028.stdout:8/726: dwrite d0/d41/d50/fbf [0,4194304] 0 2024-07-03T15:44:13.523 INFO:tasks.workunit.client.0.smithi028.stdout:8/727: creat d0/d41/d50/d3/d35/d68/d22/d40/d62/fed x:0 0 0 2024-07-03T15:44:13.527 INFO:tasks.workunit.client.0.smithi028.stdout:4/664: sync 2024-07-03T15:44:13.527 INFO:tasks.workunit.client.0.smithi028.stdout:6/735: sync 2024-07-03T15:44:13.527 INFO:tasks.workunit.client.0.smithi028.stdout:3/626: sync 2024-07-03T15:44:13.533 INFO:tasks.workunit.client.0.smithi028.stdout:6/736: symlink d4/da/d38/d3d/d89/l104 0 2024-07-03T15:44:13.533 INFO:tasks.workunit.client.0.smithi028.stdout:6/737: stat d4/f8d 0 2024-07-03T15:44:13.534 INFO:tasks.workunit.client.0.smithi028.stdout:6/738: creat d4/da/d38/d3d/df9/f105 x:0 0 0 2024-07-03T15:44:13.536 INFO:tasks.workunit.client.0.smithi028.stdout:6/739: write d4/d12/fa9 [1982504,51480] 0 2024-07-03T15:44:13.539 INFO:tasks.workunit.client.0.smithi028.stdout:6/740: symlink d4/d15/d68/l106 0 2024-07-03T15:44:13.539 INFO:tasks.workunit.client.0.smithi028.stdout:6/741: stat d4/da/d38/d59 0 2024-07-03T15:44:13.541 INFO:tasks.workunit.client.0.smithi028.stdout:6/742: symlink d4/da/d1c/l107 0 2024-07-03T15:44:13.541 INFO:tasks.workunit.client.0.smithi028.stdout:6/743: write d4/da/d38/f40 [2387613,45755] 0 2024-07-03T15:44:13.555 INFO:tasks.workunit.client.0.smithi028.stdout:3/627: dwrite d18/d3a/f39 [4194304,4194304] 0 2024-07-03T15:44:13.555 INFO:tasks.workunit.client.0.smithi028.stdout:4/665: dwrite d2/d61/f67 [0,4194304] 0 2024-07-03T15:44:13.556 INFO:tasks.workunit.client.0.smithi028.stdout:3/628: creat d18/d1a/d5e/d66/da3/fc4 x:0 0 0 2024-07-03T15:44:13.556 INFO:tasks.workunit.client.0.smithi028.stdout:4/666: fsync d2/d61/f91 0 2024-07-03T15:44:13.556 INFO:tasks.workunit.client.0.smithi028.stdout:4/667: chown d2/d12/d2e 18288782 1 2024-07-03T15:44:13.562 INFO:tasks.workunit.client.0.smithi028.stdout:3/629: truncate d18/d6f/f9e 2136201 0 2024-07-03T15:44:13.564 INFO:tasks.workunit.client.0.smithi028.stdout:4/668: dread d2/d5a/f94 [0,4194304] 0 2024-07-03T15:44:13.566 INFO:tasks.workunit.client.0.smithi028.stdout:3/630: rename d18/d1a/d64/f9b to d18/d3a/d9d/fc5 0 2024-07-03T15:44:13.571 INFO:tasks.workunit.client.0.smithi028.stdout:3/631: dread d18/d3a/f70 [0,4194304] 0 2024-07-03T15:44:13.579 INFO:tasks.workunit.client.0.smithi028.stdout:4/669: dwrite d2/d12/d5b/f85 [0,4194304] 0 2024-07-03T15:44:13.579 INFO:tasks.workunit.client.0.smithi028.stdout:4/670: readlink d2/d12/d2d/l31 0 2024-07-03T15:44:13.610 INFO:tasks.workunit.client.0.smithi028.stdout:3/632: symlink d18/d1a/d1e/d58/d9f/lc6 0 2024-07-03T15:44:13.619 INFO:tasks.workunit.client.0.smithi028.stdout:4/671: rename d2/d12/d5b/f8c to d2/d12/d3b/fd8 0 2024-07-03T15:44:13.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:13 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:13.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:13 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:13.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:13 smithi028 ceph-mon[29148]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:44:13.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:13 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:44:13.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:13 smithi028 ceph-mon[29148]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:44:13.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:13 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:44:13.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:13 smithi028 ceph-mon[29148]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:44:13.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:13 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:44:13.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:13 smithi028 ceph-mon[29148]: Adjusting osd_memory_target on smithi148 to 2581M 2024-07-03T15:44:13.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:13 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:13.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:13 smithi028 ceph-mon[29148]: pgmap v8: 65 pgs: 65 active+clean; 2.9 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 88 MiB/s rd, 112 MiB/s wr, 216 op/s 2024-07-03T15:44:13.623 INFO:tasks.workunit.client.0.smithi028.stdout:3/633: mknod d18/d1a/d1e/d58/d9f/cc7 0 2024-07-03T15:44:13.624 INFO:tasks.workunit.client.0.smithi028.stdout:4/672: rename d2/d3e/d9f/d74/f93 to d2/d61/d9c/fd9 0 2024-07-03T15:44:13.627 INFO:tasks.workunit.client.0.smithi028.stdout:3/634: creat d18/d1a/fc8 x:0 0 0 2024-07-03T15:44:13.627 INFO:tasks.workunit.client.0.smithi028.stdout:3/635: readlink d18/d6f/l93 0 2024-07-03T15:44:13.630 INFO:tasks.workunit.client.0.smithi028.stdout:4/673: mkdir d2/d12/d2d/db9/dda 0 2024-07-03T15:44:13.633 INFO:tasks.workunit.client.0.smithi028.stdout:4/674: mkdir d2/d3e/d9f/ddb 0 2024-07-03T15:44:13.633 INFO:tasks.workunit.client.0.smithi028.stdout:4/675: truncate d2/d3e/d9f/fb4 1474259 0 2024-07-03T15:44:13.633 INFO:tasks.workunit.client.0.smithi028.stdout:4/676: write d2/d5a/d45/fb3 [968200,359] 0 2024-07-03T15:44:13.635 INFO:tasks.workunit.client.0.smithi028.stdout:3/636: getdents d18/d3a 0 2024-07-03T15:44:13.640 INFO:tasks.workunit.client.0.smithi028.stdout:4/677: getdents d2/d12/d3b 0 2024-07-03T15:44:13.657 INFO:tasks.workunit.client.0.smithi028.stdout:3/637: dwrite d18/d1a/d1e/f41 [0,4194304] 0 2024-07-03T15:44:13.657 INFO:tasks.workunit.client.0.smithi028.stdout:3/638: read d18/d1a/d1e/f57 [152122,81190] 0 2024-07-03T15:44:13.661 INFO:tasks.workunit.client.0.smithi028.stdout:4/678: dwrite d2/d12/f68 [0,4194304] 0 2024-07-03T15:44:13.672 INFO:tasks.workunit.client.0.smithi028.stdout:3/639: dwrite d18/d1a/f2a [0,4194304] 0 2024-07-03T15:44:13.679 INFO:tasks.workunit.client.0.smithi028.stdout:3/640: write d18/d1a/f24 [1606211,130155] 0 2024-07-03T15:44:13.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:13 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:13.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:13 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:13.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:13 smithi148 ceph-mon[36056]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:44:13.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:13 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:44:13.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:13 smithi148 ceph-mon[36056]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:44:13.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:13 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:44:13.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:13 smithi148 ceph-mon[36056]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:44:13.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:13 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:44:13.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:13 smithi148 ceph-mon[36056]: Adjusting osd_memory_target on smithi148 to 2581M 2024-07-03T15:44:13.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:13 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:13.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:13 smithi148 ceph-mon[36056]: pgmap v8: 65 pgs: 65 active+clean; 2.9 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 88 MiB/s rd, 112 MiB/s wr, 216 op/s 2024-07-03T15:44:13.743 INFO:tasks.workunit.client.0.smithi028.stdout:9/692: sync 2024-07-03T15:44:13.743 INFO:tasks.workunit.client.0.smithi028.stdout:9/693: stat d1/dd/f3c 0 2024-07-03T15:44:13.745 INFO:tasks.workunit.client.0.smithi028.stdout:9/694: unlink d1/d10/d28/d4e/f50 0 2024-07-03T15:44:13.745 INFO:tasks.workunit.client.0.smithi028.stdout:9/695: fsync d1/d10/d76/fae 0 2024-07-03T15:44:13.745 INFO:tasks.workunit.client.0.smithi028.stdout:9/696: chown d1/fa1 2 1 2024-07-03T15:44:13.748 INFO:tasks.workunit.client.0.smithi028.stdout:9/697: rename d1/d10/d28/d45/d61/d8c/d77/db7/dab/dca/leb to d1/d16/d57/d5f/dd8/lec 0 2024-07-03T15:44:13.749 INFO:tasks.workunit.client.0.smithi028.stdout:9/698: mkdir d1/d10/d28/daa/ded 0 2024-07-03T15:44:13.751 INFO:tasks.workunit.client.0.smithi028.stdout:9/699: mkdir d1/d10/d28/d4e/d4f/dee 0 2024-07-03T15:44:13.752 INFO:tasks.workunit.client.0.smithi028.stdout:9/700: mknod d1/d10/d28/d45/d56/dcd/cef 0 2024-07-03T15:44:13.754 INFO:tasks.workunit.client.0.smithi028.stdout:9/701: rmdir d1/d10/d28/d4e/d4f/dee 0 2024-07-03T15:44:13.755 INFO:tasks.workunit.client.0.smithi028.stdout:9/702: truncate d1/dd/f59 3271992 0 2024-07-03T15:44:13.755 INFO:tasks.workunit.client.0.smithi028.stdout:0/736: sync 2024-07-03T15:44:13.756 INFO:tasks.workunit.client.0.smithi028.stdout:9/703: mkdir d1/d10/d28/d4e/d5b/df0 0 2024-07-03T15:44:13.758 INFO:tasks.workunit.client.0.smithi028.stdout:0/737: mkdir d2/d3/dc4/d58/d82/de7 0 2024-07-03T15:44:13.758 INFO:tasks.workunit.client.0.smithi028.stdout:0/738: dread - d2/d3/dc4/d58/d82/fb6 zero size 2024-07-03T15:44:13.758 INFO:tasks.workunit.client.0.smithi028.stdout:0/739: fsync d2/d3/dc4/d36/d45/da3/fab 0 2024-07-03T15:44:13.761 INFO:tasks.workunit.client.0.smithi028.stdout:9/704: truncate d1/d10/d28/d45/d61/d8c/f72 3556177 0 2024-07-03T15:44:13.761 INFO:tasks.workunit.client.0.smithi028.stdout:9/705: chown d1/d16/d62/c82 18700005 1 2024-07-03T15:44:13.761 INFO:tasks.workunit.client.0.smithi028.stdout:9/706: chown d1/d10/d28/d45/d61/d8c/d77/db7/dab/dcb/fe4 78 1 2024-07-03T15:44:13.762 INFO:tasks.workunit.client.0.smithi028.stdout:0/740: symlink d2/d3/dc4/d58/d82/de7/le8 0 2024-07-03T15:44:13.762 INFO:tasks.workunit.client.0.smithi028.stdout:0/741: chown d2/d3/dc4/d36/f88 255129 1 2024-07-03T15:44:13.765 INFO:tasks.workunit.client.0.smithi028.stdout:9/707: getdents d1/d10/d28/d45/d61/d8c/d77/db7/dab/dca 0 2024-07-03T15:44:13.765 INFO:tasks.workunit.client.0.smithi028.stdout:9/708: mknod d1/d16/d57/d5f/cf1 0 2024-07-03T15:44:13.768 INFO:tasks.workunit.client.0.smithi028.stdout:9/709: link d1/d10/d28/d45/d56/l4d d1/d10/lf2 0 2024-07-03T15:44:13.768 INFO:tasks.workunit.client.0.smithi028.stdout:9/710: write d1/d10/d28/d4e/d4f/f87 [4615711,31930] 0 2024-07-03T15:44:13.768 INFO:tasks.workunit.client.0.smithi028.stdout:9/711: write d1/d10/d28/d4e/d5b/fbc [30344,777] 0 2024-07-03T15:44:13.770 INFO:tasks.workunit.client.0.smithi028.stdout:9/712: creat d1/d16/d57/d5f/d64/ff3 x:0 0 0 2024-07-03T15:44:13.770 INFO:tasks.workunit.client.0.smithi028.stdout:9/713: write d1/d10/d28/d45/fa8 [1167587,123873] 0 2024-07-03T15:44:13.770 INFO:tasks.workunit.client.0.smithi028.stdout:9/714: read - d1/d10/d28/fa5 zero size 2024-07-03T15:44:13.770 INFO:tasks.workunit.client.0.smithi028.stdout:9/715: readlink d1/d10/d28/d45/d56/la7 0 2024-07-03T15:44:13.773 INFO:tasks.workunit.client.0.smithi028.stdout:9/716: truncate d1/f31 2575934 0 2024-07-03T15:44:13.773 INFO:tasks.workunit.client.0.smithi028.stdout:9/717: chown d1/d8e/f9e 2241667 1 2024-07-03T15:44:13.773 INFO:tasks.workunit.client.0.smithi028.stdout:9/718: dread - d1/d16/d57/d5f/d64/fd1 zero size 2024-07-03T15:44:13.774 INFO:tasks.workunit.client.0.smithi028.stdout:9/719: dread d1/d16/d57/f66 [0,4194304] 0 2024-07-03T15:44:13.776 INFO:tasks.workunit.client.0.smithi028.stdout:9/720: read d1/d16/d57/fda [308593,107614] 0 2024-07-03T15:44:13.781 INFO:tasks.workunit.client.0.smithi028.stdout:0/742: dwrite d2/d3/dc4/d36/f61 [4194304,4194304] 0 2024-07-03T15:44:13.781 INFO:tasks.workunit.client.0.smithi028.stdout:0/743: fsync d2/d3/f73 0 2024-07-03T15:44:13.785 INFO:tasks.workunit.client.0.smithi028.stdout:0/744: symlink d2/d3/d4d/d56/d78/d9f/le9 0 2024-07-03T15:44:13.785 INFO:tasks.workunit.client.0.smithi028.stdout:9/721: write d1/d16/d57/f71 [300224,48714] 0 2024-07-03T15:44:13.793 INFO:tasks.workunit.client.0.smithi028.stdout:9/722: dread d1/d10/d28/d4e/d4f/f67 [0,4194304] 0 2024-07-03T15:44:13.795 INFO:tasks.workunit.client.0.smithi028.stdout:9/723: symlink d1/d8e/lf4 0 2024-07-03T15:44:13.796 INFO:tasks.workunit.client.0.smithi028.stdout:9/724: write d1/d10/d28/d45/d61/d8c/d77/db7/dab/dcb/fe2 [333726,100399] 0 2024-07-03T15:44:13.796 INFO:tasks.workunit.client.0.smithi028.stdout:9/725: chown d1/d10/d76/fae 31328118 1 2024-07-03T15:44:13.797 INFO:tasks.workunit.client.0.smithi028.stdout:9/726: mknod d1/d10/d28/d45/d61/d8c/d77/db7/dab/dca/cf5 0 2024-07-03T15:44:13.801 INFO:tasks.workunit.client.0.smithi028.stdout:0/745: dwrite d2/d3/d4d/d96/dc2/fd5 [0,4194304] 0 2024-07-03T15:44:13.801 INFO:tasks.workunit.client.0.smithi028.stdout:0/746: creat d2/d3/d4d/d96/dc2/fea x:0 0 0 2024-07-03T15:44:13.811 INFO:tasks.workunit.client.0.smithi028.stdout:9/727: dwrite d1/d16/f51 [0,4194304] 0 2024-07-03T15:44:13.822 INFO:tasks.workunit.client.0.smithi028.stdout:9/728: write d1/f21 [7823370,57190] 0 2024-07-03T15:44:13.823 INFO:tasks.workunit.client.0.smithi028.stdout:9/729: truncate d1/d10/d28/d45/d61/d8c/d77/f91 1145665 0 2024-07-03T15:44:13.824 INFO:tasks.workunit.client.0.smithi028.stdout:9/730: rename d1/ca to d1/d10/d28/d45/d61/d8c/d77/db7/dab/cf6 0 2024-07-03T15:44:13.824 INFO:tasks.workunit.client.0.smithi028.stdout:9/731: fdatasync d1/d8e/fc6 0 2024-07-03T15:44:13.827 INFO:tasks.workunit.client.0.smithi028.stdout:9/732: mknod d1/d10/d28/cf7 0 2024-07-03T15:44:13.827 INFO:tasks.workunit.client.0.smithi028.stdout:9/733: chown d1/d10/dea 8234 1 2024-07-03T15:44:14.000 INFO:tasks.workunit.client.0.smithi028.stdout:2/530: sync 2024-07-03T15:44:14.001 INFO:tasks.workunit.client.0.smithi028.stdout:5/665: sync 2024-07-03T15:44:14.003 INFO:tasks.workunit.client.0.smithi028.stdout:5/666: mkdir d0/dd/d1e/d3b/de0 0 2024-07-03T15:44:14.003 INFO:tasks.workunit.client.0.smithi028.stdout:5/667: chown d0/dd/d14 4341541 1 2024-07-03T15:44:14.004 INFO:tasks.workunit.client.0.smithi028.stdout:2/531: mkdir da/d1e/d56/dc0 0 2024-07-03T15:44:14.005 INFO:tasks.workunit.client.0.smithi028.stdout:2/532: stat da/d23/d30/f3e 0 2024-07-03T15:44:14.005 INFO:tasks.workunit.client.0.smithi028.stdout:2/533: stat da/d1e/d56/dc0 0 2024-07-03T15:44:14.023 INFO:tasks.workunit.client.0.smithi028.stdout:5/668: dwrite d0/dd/df/faf [0,4194304] 0 2024-07-03T15:44:14.023 INFO:tasks.workunit.client.0.smithi028.stdout:5/669: write d0/dd/d1e/d3b/d3f/d5a/f78 [316470,104755] 0 2024-07-03T15:44:14.023 INFO:tasks.workunit.client.0.smithi028.stdout:5/670: chown d0/dd/d13/d57/d30/d70/d7f/fbd 3 1 2024-07-03T15:44:14.174 INFO:tasks.workunit.client.0.smithi028.stdout:7/507: sync 2024-07-03T15:44:14.175 INFO:tasks.workunit.client.0.smithi028.stdout:1/900: sync 2024-07-03T15:44:14.175 INFO:tasks.workunit.client.0.smithi028.stdout:8/728: sync 2024-07-03T15:44:14.175 INFO:tasks.workunit.client.0.smithi028.stdout:7/508: fsync d9/d1c/f6b 0 2024-07-03T15:44:14.175 INFO:tasks.workunit.client.0.smithi028.stdout:8/729: truncate d0/d41/d50/f27 411933 0 2024-07-03T15:44:14.177 INFO:tasks.workunit.client.0.smithi028.stdout:1/901: truncate d4/de/fc5 4464378 0 2024-07-03T15:44:14.180 INFO:tasks.workunit.client.0.smithi028.stdout:7/509: symlink d9/d1c/d6e/l94 0 2024-07-03T15:44:14.181 INFO:tasks.workunit.client.0.smithi028.stdout:1/902: symlink d4/d3c/d54/d79/d16/d57/d138/l139 0 2024-07-03T15:44:14.197 INFO:tasks.workunit.client.0.smithi028.stdout:8/730: dwrite d0/d41/d50/d3/dc/f2b [0,4194304] 0 2024-07-03T15:44:14.198 INFO:tasks.workunit.client.0.smithi028.stdout:1/903: dread d4/f29 [0,4194304] 0 2024-07-03T15:44:14.198 INFO:tasks.workunit.client.0.smithi028.stdout:8/731: readlink d0/d41/d50/l51 0 2024-07-03T15:44:14.198 INFO:tasks.workunit.client.0.smithi028.stdout:8/732: read d0/d41/d50/d3/dc/d33/d7a/f2f [3892830,75598] 0 2024-07-03T15:44:14.198 INFO:tasks.workunit.client.0.smithi028.stdout:8/733: dread - d0/d6c/da9/fd7 zero size 2024-07-03T15:44:14.199 INFO:tasks.workunit.client.0.smithi028.stdout:1/904: mkdir d4/d83/de8/d121/d3a/d5c/d6c/d7d/d59/dc6/d13a 0 2024-07-03T15:44:14.200 INFO:tasks.workunit.client.0.smithi028.stdout:8/734: link d0/de0/lac d0/d41/d50/daf/lee 0 2024-07-03T15:44:14.200 INFO:tasks.workunit.client.0.smithi028.stdout:8/735: fsync d0/de0/dcf/f7c 0 2024-07-03T15:44:14.204 INFO:tasks.workunit.client.0.smithi028.stdout:7/510: dwrite d9/d29/f77 [0,4194304] 0 2024-07-03T15:44:14.204 INFO:tasks.workunit.client.0.smithi028.stdout:1/905: symlink d4/d3c/d54/d79/d16/d7c/d10a/l13b 0 2024-07-03T15:44:14.204 INFO:tasks.workunit.client.0.smithi028.stdout:7/511: read d9/d29/d5d/f79 [702450,100157] 0 2024-07-03T15:44:14.204 INFO:tasks.workunit.client.0.smithi028.stdout:7/512: chown d9/f1f 2 1 2024-07-03T15:44:14.205 INFO:tasks.workunit.client.0.smithi028.stdout:8/736: dread d0/d41/d50/d3/d35/d68/d22/d40/d62/f65 [0,4194304] 0 2024-07-03T15:44:14.209 INFO:tasks.workunit.client.0.smithi028.stdout:1/906: mknod d4/d3c/d54/d79/d16/d57/d66/d10e/c13c 0 2024-07-03T15:44:14.212 INFO:tasks.workunit.client.0.smithi028.stdout:8/737: creat d0/d6c/fef x:0 0 0 2024-07-03T15:44:14.212 INFO:tasks.workunit.client.0.smithi028.stdout:7/513: dread d9/d1c/d52/f58 [0,4194304] 0 2024-07-03T15:44:14.212 INFO:tasks.workunit.client.0.smithi028.stdout:7/514: read f0 [6774657,116585] 0 2024-07-03T15:44:14.213 INFO:tasks.workunit.client.0.smithi028.stdout:1/907: unlink d4/d83/de8/d121/d3a/d5c/daa/fb5 0 2024-07-03T15:44:14.213 INFO:tasks.workunit.client.0.smithi028.stdout:1/908: creat d4/d83/de8/d121/d3a/d5c/daa/df0/d123/d125/f13d x:0 0 0 2024-07-03T15:44:14.213 INFO:tasks.workunit.client.0.smithi028.stdout:1/909: write d4/d83/de8/d121/d3a/d5c/d6c/d7d/d59/f2d [68127,130378] 0 2024-07-03T15:44:14.214 INFO:tasks.workunit.client.0.smithi028.stdout:8/738: mknod d0/d41/d50/d97/cf0 0 2024-07-03T15:44:14.214 INFO:tasks.workunit.client.0.smithi028.stdout:7/515: rmdir d9/d1c/d45 39 2024-07-03T15:44:14.214 INFO:tasks.workunit.client.0.smithi028.stdout:7/516: dread - d9/d1c/d6e/f8c zero size 2024-07-03T15:44:14.216 INFO:tasks.workunit.client.0.smithi028.stdout:7/517: dread d9/d29/f67 [0,4194304] 0 2024-07-03T15:44:14.216 INFO:tasks.workunit.client.0.smithi028.stdout:7/518: stat d9/d29/f3e 0 2024-07-03T15:44:14.217 INFO:tasks.workunit.client.0.smithi028.stdout:1/910: link d4/d83/de8/d121/d3a/f68 d4/de/d1c/d55/f13e 0 2024-07-03T15:44:14.217 INFO:tasks.workunit.client.0.smithi028.stdout:1/911: chown d4/d3c/d54/d79 4592357 1 2024-07-03T15:44:14.217 INFO:tasks.workunit.client.0.smithi028.stdout:1/912: dread - d4/d83/de8/d121/d3a/d43/fe6 zero size 2024-07-03T15:44:14.217 INFO:tasks.workunit.client.0.smithi028.stdout:1/913: write d4/d3c/d54/d84/f112 [669671,118787] 0 2024-07-03T15:44:14.217 INFO:tasks.workunit.client.0.smithi028.stdout:1/914: fsync d4/de/d1c/d81/d10d/f126 0 2024-07-03T15:44:14.217 INFO:tasks.workunit.client.0.smithi028.stdout:1/915: chown d4/d3c/d54/d84/lbb 19695 1 2024-07-03T15:44:14.218 INFO:tasks.workunit.client.0.smithi028.stdout:8/739: rename d0/d41/d50/d3/d35/d68/d22/d40/fd4 to d0/d41/d50/d3/dc/d3f/ff1 0 2024-07-03T15:44:14.219 INFO:tasks.workunit.client.0.smithi028.stdout:7/519: unlink d9/d1c/d52/d6a/f6c 0 2024-07-03T15:44:14.221 INFO:tasks.workunit.client.0.smithi028.stdout:1/916: link d4/d3c/f9e d4/d83/de8/d121/dbe/f13f 0 2024-07-03T15:44:14.221 INFO:tasks.workunit.client.0.smithi028.stdout:1/917: write d4/de/f85 [3921261,97101] 0 2024-07-03T15:44:14.221 INFO:tasks.workunit.client.0.smithi028.stdout:1/918: truncate d4/d83/de8/d121/d3a/d5c/d6c/d7d/d59/f2d 1628774 0 2024-07-03T15:44:14.222 INFO:tasks.workunit.client.0.smithi028.stdout:8/740: rename d0/d41/d50/d3/d35/d68/c5f to d0/d41/d50/daf/dc1/dd1/cf2 0 2024-07-03T15:44:14.224 INFO:tasks.workunit.client.0.smithi028.stdout:8/741: dread d0/d41/d50/daf/dc8/da6/f9c [4194304,4194304] 0 2024-07-03T15:44:14.224 INFO:tasks.workunit.client.0.smithi028.stdout:8/742: chown d0/d6c/cab 77188691 1 2024-07-03T15:44:14.224 INFO:tasks.workunit.client.0.smithi028.stdout:7/520: symlink d9/d1c/d45/d71/d78/l95 0 2024-07-03T15:44:14.226 INFO:tasks.workunit.client.0.smithi028.stdout:8/743: rename d0/d41/d50/d3/dc/d3f/d8e/d8d to d0/d41/d50/d3/dc/d3f/df3 0 2024-07-03T15:44:14.227 INFO:tasks.workunit.client.0.smithi028.stdout:7/521: symlink d9/d1c/d86/l96 0 2024-07-03T15:44:14.229 INFO:tasks.workunit.client.0.smithi028.stdout:8/744: rmdir d0/de0 39 2024-07-03T15:44:14.230 INFO:tasks.workunit.client.0.smithi028.stdout:7/522: dread d9/d1c/d52/d6a/f7d [0,4194304] 0 2024-07-03T15:44:14.231 INFO:tasks.workunit.client.0.smithi028.stdout:8/745: creat d0/d41/d50/d3/dc/d3f/d8e/ff4 x:0 0 0 2024-07-03T15:44:14.233 INFO:tasks.workunit.client.0.smithi028.stdout:8/746: creat d0/d41/d50/d3/ff5 x:0 0 0 2024-07-03T15:44:14.235 INFO:tasks.workunit.client.0.smithi028.stdout:7/523: dread d9/d1c/d52/f58 [0,4194304] 0 2024-07-03T15:44:14.236 INFO:tasks.workunit.client.0.smithi028.stdout:8/747: creat d0/d41/ff6 x:0 0 0 2024-07-03T15:44:14.238 INFO:tasks.workunit.client.0.smithi028.stdout:1/919: dwrite d4/de/d1c/d81/d10d/f126 [0,4194304] 0 2024-07-03T15:44:14.238 INFO:tasks.workunit.client.0.smithi028.stdout:1/920: fsync d4/d3c/d54/d79/d16/fb4 0 2024-07-03T15:44:14.245 INFO:tasks.workunit.client.0.smithi028.stdout:1/921: dread d4/d3c/d54/d79/f12 [4194304,4194304] 0 2024-07-03T15:44:14.245 INFO:tasks.workunit.client.0.smithi028.stdout:1/922: write d4/de/d1c/f3d [4214789,22035] 0 2024-07-03T15:44:14.259 INFO:tasks.workunit.client.0.smithi028.stdout:1/923: write d4/d83/fe3 [2312441,6399] 0 2024-07-03T15:44:14.259 INFO:tasks.workunit.client.0.smithi028.stdout:8/748: dwrite d0/d41/d50/d3/dc/d3f/df3/fb9 [0,4194304] 0 2024-07-03T15:44:14.264 INFO:tasks.workunit.client.0.smithi028.stdout:1/924: truncate d4/f29 4471835 0 2024-07-03T15:44:14.267 INFO:tasks.workunit.client.0.smithi028.stdout:8/749: creat d0/d41/d50/d3/dc/ff7 x:0 0 0 2024-07-03T15:44:14.269 INFO:tasks.workunit.client.0.smithi028.stdout:8/750: dread d0/d41/d50/f27 [0,4194304] 0 2024-07-03T15:44:14.269 INFO:tasks.workunit.client.0.smithi028.stdout:1/925: rename d4/d83/de8/c136 to d4/d3c/d54/d79/d16/d57/d66/c140 0 2024-07-03T15:44:14.272 INFO:tasks.workunit.client.0.smithi028.stdout:8/751: unlink d0/d41/d50/daf/dc8/da6/da7/fdb 0 2024-07-03T15:44:14.274 INFO:tasks.workunit.client.0.smithi028.stdout:8/752: mknod d0/d41/d50/daf/dc1/cf8 0 2024-07-03T15:44:14.274 INFO:tasks.workunit.client.0.smithi028.stdout:8/753: fsync d0/d41/d50/daf/dc8/da6/da7/fda 0 2024-07-03T15:44:14.292 INFO:tasks.workunit.client.0.smithi028.stdout:1/926: dwrite d4/d83/de8/d121/d3a/d5c/d6c/d7d/d59/dc6/ffc [0,4194304] 0 2024-07-03T15:44:14.292 INFO:tasks.workunit.client.0.smithi028.stdout:8/754: dwrite d0/d41/d50/d3/dc/d3f/d8e/fec [0,4194304] 0 2024-07-03T15:44:14.292 INFO:tasks.workunit.client.0.smithi028.stdout:8/755: creat d0/d41/d50/d3/dc/d3f/d8e/d5d/ff9 x:0 0 0 2024-07-03T15:44:14.295 INFO:tasks.workunit.client.0.smithi028.stdout:1/927: symlink d4/d83/de8/d121/d3a/d5c/daa/df0/d123/d125/l141 0 2024-07-03T15:44:14.295 INFO:tasks.workunit.client.0.smithi028.stdout:1/928: fdatasync d4/d83/de8/d121/d3a/d5c/fcf 0 2024-07-03T15:44:14.300 INFO:tasks.workunit.client.0.smithi028.stdout:1/929: link d4/d83/de8/d121/d3a/d5c/dff/c102 d4/d3c/d54/d79/d16/d57/d66/d10e/c142 0 2024-07-03T15:44:14.302 INFO:tasks.workunit.client.0.smithi028.stdout:1/930: creat d4/d3c/d54/d79/d16/d7c/f143 x:0 0 0 2024-07-03T15:44:14.302 INFO:tasks.workunit.client.0.smithi028.stdout:1/931: fdatasync d4/d83/de8/d121/d3a/d5c/fbf 0 2024-07-03T15:44:14.303 INFO:tasks.workunit.client.0.smithi028.stdout:1/932: symlink d4/d83/de8/d121/d3a/d5c/d6c/d7d/d137/l144 0 2024-07-03T15:44:14.303 INFO:tasks.workunit.client.0.smithi028.stdout:1/933: readlink d4/d3c/d54/d79/d16/d7c/d10a/l13b 0 2024-07-03T15:44:14.308 INFO:tasks.workunit.client.0.smithi028.stdout:8/756: dwrite d0/d41/d50/d3/d35/d68/d22/fd5 [0,4194304] 0 2024-07-03T15:44:14.311 INFO:tasks.workunit.client.0.smithi028.stdout:8/757: symlink d0/d41/d50/daf/dc8/da6/da7/lfa 0 2024-07-03T15:44:14.312 INFO:tasks.workunit.client.0.smithi028.stdout:8/758: dread - d0/d6c/fef zero size 2024-07-03T15:44:14.312 INFO:tasks.workunit.client.0.smithi028.stdout:8/759: chown d0/d41/d50/daf/dc8/da6/da7 17677 1 2024-07-03T15:44:14.317 INFO:tasks.workunit.client.0.smithi028.stdout:8/760: dread d0/d6c/fae [0,4194304] 0 2024-07-03T15:44:14.318 INFO:tasks.workunit.client.0.smithi028.stdout:8/761: getdents d0/d41/d50/d3/d35/d68/d22 0 2024-07-03T15:44:14.335 INFO:tasks.workunit.client.0.smithi028.stdout:4/679: sync 2024-07-03T15:44:14.336 INFO:tasks.workunit.client.0.smithi028.stdout:6/744: sync 2024-07-03T15:44:14.336 INFO:tasks.workunit.client.0.smithi028.stdout:8/762: dwrite d0/d41/d50/f27 [0,4194304] 0 2024-07-03T15:44:14.338 INFO:tasks.workunit.client.0.smithi028.stdout:4/680: symlink d2/d61/d92/ldc 0 2024-07-03T15:44:14.338 INFO:tasks.workunit.client.0.smithi028.stdout:4/681: creat d2/d12/d5b/d98/fdd x:0 0 0 2024-07-03T15:44:14.338 INFO:tasks.workunit.client.0.smithi028.stdout:8/763: symlink d0/d6c/lfb 0 2024-07-03T15:44:14.338 INFO:tasks.workunit.client.0.smithi028.stdout:8/764: read d0/d41/d50/d3/dc/d3f/faa [35719,55315] 0 2024-07-03T15:44:14.339 INFO:tasks.workunit.client.0.smithi028.stdout:6/745: link d4/da/d1c/leb d4/da/d1c/l108 0 2024-07-03T15:44:14.419 INFO:tasks.workunit.client.0.smithi028.stdout:6/746: truncate d4/f11 489808 0 2024-07-03T15:44:14.419 INFO:tasks.workunit.client.0.smithi028.stdout:8/765: dwrite d0/d6c/fae [0,4194304] 0 2024-07-03T15:44:14.421 INFO:tasks.workunit.client.0.smithi028.stdout:4/682: getdents d2 0 2024-07-03T15:44:14.423 INFO:tasks.workunit.client.0.smithi028.stdout:6/747: creat d4/da/d38/d60/dc5/dfe/f109 x:0 0 0 2024-07-03T15:44:14.423 INFO:tasks.workunit.client.0.smithi028.stdout:6/748: chown d4/d12/d2c/d31 4095 1 2024-07-03T15:44:14.426 INFO:tasks.workunit.client.0.smithi028.stdout:8/766: truncate d0/d41/d50/d3/dc/d3f/d8e/f6a 6835880 0 2024-07-03T15:44:14.428 INFO:tasks.workunit.client.0.smithi028.stdout:4/683: rmdir d2/d12/d2d/d2f/d3a 39 2024-07-03T15:44:14.428 INFO:tasks.workunit.client.0.smithi028.stdout:4/684: chown d2/cd 5 1 2024-07-03T15:44:14.432 INFO:tasks.workunit.client.0.smithi028.stdout:6/749: mknod d4/da/d47/d66/c10a 0 2024-07-03T15:44:14.433 INFO:tasks.workunit.client.0.smithi028.stdout:6/750: write d4/d12/d2c/dac/fff [775326,66958] 0 2024-07-03T15:44:14.433 INFO:tasks.workunit.client.0.smithi028.stdout:8/767: unlink d0/d6c/fcb 0 2024-07-03T15:44:14.437 INFO:tasks.workunit.client.0.smithi028.stdout:4/685: rename d2/d61/d92/ldc to d2/d5a/d45/da5/lde 0 2024-07-03T15:44:14.437 INFO:tasks.workunit.client.0.smithi028.stdout:4/686: read d2/d12/d2d/d58/f69 [504235,88591] 0 2024-07-03T15:44:14.438 INFO:tasks.workunit.client.0.smithi028.stdout:8/768: mkdir d0/d41/d50/d3/dc/d3f/d8e/d81/dfc 0 2024-07-03T15:44:14.440 INFO:tasks.workunit.client.0.smithi028.stdout:4/687: mknod d2/d5a/d45/cdf 0 2024-07-03T15:44:14.440 INFO:tasks.workunit.client.0.smithi028.stdout:4/688: fsync d2/d5a/f56 0 2024-07-03T15:44:14.441 INFO:tasks.workunit.client.0.smithi028.stdout:8/769: creat d0/de0/dcf/dc0/ffd x:0 0 0 2024-07-03T15:44:14.441 INFO:tasks.workunit.client.0.smithi028.stdout:8/770: fsync d0/d41/f88 0 2024-07-03T15:44:14.442 INFO:tasks.workunit.client.0.smithi028.stdout:4/689: write d2/d3e/f3f [762318,9528] 0 2024-07-03T15:44:14.443 INFO:tasks.workunit.client.0.smithi028.stdout:6/751: dread d4/f29 [0,4194304] 0 2024-07-03T15:44:14.443 INFO:tasks.workunit.client.0.smithi028.stdout:6/752: truncate d4/da/d38/d59/f92 1749239 0 2024-07-03T15:44:14.443 INFO:tasks.workunit.client.0.smithi028.stdout:6/753: write d4/f7b [866624,89774] 0 2024-07-03T15:44:14.443 INFO:tasks.workunit.client.0.smithi028.stdout:6/754: dread - d4/d12/d2c/d31/f64 zero size 2024-07-03T15:44:14.443 INFO:tasks.workunit.client.0.smithi028.stdout:8/771: link d0/d41/d50/d3/dc/c5a d0/d41/d50/d3/d35/d68/d22/d40/d4a/cfe 0 2024-07-03T15:44:14.445 INFO:tasks.workunit.client.0.smithi028.stdout:4/690: mknod d2/d12/d2d/db9/dda/ce0 0 2024-07-03T15:44:14.446 INFO:tasks.workunit.client.0.smithi028.stdout:6/755: unlink d4/d12/d2c/dac/fef 0 2024-07-03T15:44:14.448 INFO:tasks.workunit.client.0.smithi028.stdout:8/772: mkdir d0/d41/d50/d3/dc/d33/dff 0 2024-07-03T15:44:14.448 INFO:tasks.workunit.client.0.smithi028.stdout:8/773: readlink d0/d41/d50/daf/dc8/da6/da7/lfa 0 2024-07-03T15:44:14.449 INFO:tasks.workunit.client.0.smithi028.stdout:4/691: creat d2/d5a/d45/d84/fe1 x:0 0 0 2024-07-03T15:44:14.449 INFO:tasks.workunit.client.0.smithi028.stdout:4/692: chown d2/d12/d42/d63/da8/fc1 97481907 1 2024-07-03T15:44:14.450 INFO:tasks.workunit.client.0.smithi028.stdout:6/756: truncate d4/d15/f20 965620 0 2024-07-03T15:44:14.450 INFO:tasks.workunit.client.0.smithi028.stdout:6/757: write d4/d12/fa2 [7571987,75395] 0 2024-07-03T15:44:14.450 INFO:tasks.workunit.client.0.smithi028.stdout:6/758: readlink d4/da/l90 0 2024-07-03T15:44:14.451 INFO:tasks.workunit.client.0.smithi028.stdout:4/693: mknod d2/d3e/d9f/d74/ce2 0 2024-07-03T15:44:14.452 INFO:tasks.workunit.client.0.smithi028.stdout:6/759: rename d4/da/f4e to d4/d12/d2c/d31/f10b 0 2024-07-03T15:44:14.454 INFO:tasks.workunit.client.0.smithi028.stdout:4/694: mknod d2/d5a/ce3 0 2024-07-03T15:44:14.454 INFO:tasks.workunit.client.0.smithi028.stdout:4/695: truncate d2/d61/d9d/faf 470163 0 2024-07-03T15:44:14.454 INFO:tasks.workunit.client.0.smithi028.stdout:4/696: chown d2/d12/c21 49305 1 2024-07-03T15:44:14.454 INFO:tasks.workunit.client.0.smithi028.stdout:4/697: getdents d2/d3e/d9f/ddb 0 2024-07-03T15:44:14.454 INFO:tasks.workunit.client.0.smithi028.stdout:4/698: chown d2 351908 1 2024-07-03T15:44:14.454 INFO:tasks.workunit.client.0.smithi028.stdout:6/760: dread d4/da/d38/d60/db5/fd0 [0,4194304] 0 2024-07-03T15:44:14.470 INFO:tasks.workunit.client.0.smithi028.stdout:8/774: dwrite d0/d41/d50/d3/d35/d68/d22/f25 [4194304,4194304] 0 2024-07-03T15:44:14.477 INFO:tasks.workunit.client.0.smithi028.stdout:4/699: rename d2/d12/d2d/d2f to d2/d12/d5b/d78/de4 0 2024-07-03T15:44:14.477 INFO:tasks.workunit.client.0.smithi028.stdout:4/700: chown d2 967159103 1 2024-07-03T15:44:14.478 INFO:tasks.workunit.client.0.smithi028.stdout:8/775: getdents d0/d41/d50/d3/d35/d68/d22/d40/dd2 0 2024-07-03T15:44:14.482 INFO:tasks.workunit.client.0.smithi028.stdout:4/701: link d2/d3e/dae/cba d2/d5a/d45/ce5 0 2024-07-03T15:44:14.496 INFO:tasks.workunit.client.0.smithi028.stdout:6/761: rename d4/da to d4/d78/d10c 0 2024-07-03T15:44:14.497 INFO:tasks.workunit.client.0.smithi028.stdout:4/702: dwrite d2/d12/d5b/d78/de4/fd3 [0,4194304] 0 2024-07-03T15:44:14.497 INFO:tasks.workunit.client.0.smithi028.stdout:4/703: write d2/d61/fc5 [563003,101904] 0 2024-07-03T15:44:14.498 INFO:tasks.workunit.client.0.smithi028.stdout:8/776: dwrite d0/d41/d50/d3/dc/d33/f7d [0,4194304] 0 2024-07-03T15:44:14.498 INFO:tasks.workunit.client.0.smithi028.stdout:4/704: write d2/d61/d9d/faf [25333,101081] 0 2024-07-03T15:44:14.501 INFO:tasks.workunit.client.0.smithi028.stdout:4/705: creat d2/d12/d3b/fe6 x:0 0 0 2024-07-03T15:44:14.503 INFO:tasks.workunit.client.0.smithi028.stdout:4/706: rename d2/d12/f68 to d2/d5a/fe7 0 2024-07-03T15:44:14.508 INFO:tasks.workunit.client.0.smithi028.stdout:8/777: write d0/d41/d50/d3/d35/d68/d22/fd5 [4068886,111937] 0 2024-07-03T15:44:14.513 INFO:tasks.workunit.client.0.smithi028.stdout:4/707: dread d2/f1b [0,4194304] 0 2024-07-03T15:44:14.513 INFO:tasks.workunit.client.0.smithi028.stdout:4/708: chown d2/d12/d2d/l33 418392439 1 2024-07-03T15:44:14.518 INFO:tasks.workunit.client.0.smithi028.stdout:8/778: dread d0/d41/d50/d3/dc/d4d/fc6 [0,4194304] 0 2024-07-03T15:44:14.518 INFO:tasks.workunit.client.0.smithi028.stdout:8/779: read - d0/de0/dcf/dc0/ffd zero size 2024-07-03T15:44:14.519 INFO:tasks.workunit.client.0.smithi028.stdout:8/780: truncate d0/d41/d50/d3/dc/d3f/d8e/d81/d9e/fea 9391318 0 2024-07-03T15:44:14.519 INFO:tasks.workunit.client.0.smithi028.stdout:8/781: mkdir d0/d41/d50/d3/dc/d3f/d8e/d81/d100 0 2024-07-03T15:44:14.521 INFO:tasks.workunit.client.0.smithi028.stdout:6/762: dwrite f0 [0,4194304] 0 2024-07-03T15:44:14.529 INFO:tasks.workunit.client.0.smithi028.stdout:6/763: write d4/d78/d10c/d13/dbd/fe7 [1240294,45126] 0 2024-07-03T15:44:14.529 INFO:tasks.workunit.client.0.smithi028.stdout:6/764: truncate d4/d78/d10c/d38/d60/dc5/dcf/fe2 4158936 0 2024-07-03T15:44:14.530 INFO:tasks.workunit.client.0.smithi028.stdout:6/765: dread - d4/d12/d2c/d31/f64 zero size 2024-07-03T15:44:14.535 INFO:tasks.workunit.client.0.smithi028.stdout:8/782: dread d0/d6c/fce [0,4194304] 0 2024-07-03T15:44:14.535 INFO:tasks.workunit.client.0.smithi028.stdout:8/783: fdatasync d0/d41/d50/d3/f3a 0 2024-07-03T15:44:14.536 INFO:tasks.workunit.client.0.smithi028.stdout:4/709: dwrite d2/d5a/f1f [4194304,4194304] 0 2024-07-03T15:44:14.536 INFO:tasks.workunit.client.0.smithi028.stdout:8/784: symlink d0/d41/d50/d3/dc/d3f/d8e/d81/l101 0 2024-07-03T15:44:14.536 INFO:tasks.workunit.client.0.smithi028.stdout:8/785: write d0/d6c/fef [922205,33308] 0 2024-07-03T15:44:14.537 INFO:tasks.workunit.client.0.smithi028.stdout:8/786: chown d0/d41/d50/d3/dc/d3f/l57 1062747 1 2024-07-03T15:44:14.537 INFO:tasks.workunit.client.0.smithi028.stdout:8/787: readlink d0/d41/d50/d3/d35/d68/d22/d40/dd2/le8 0 2024-07-03T15:44:14.537 INFO:tasks.workunit.client.0.smithi028.stdout:4/710: getdents d2/d12/d42/d63/d6d/dc8 0 2024-07-03T15:44:14.537 INFO:tasks.workunit.client.0.smithi028.stdout:4/711: write d2/d12/d5b/d98/fb2 [901338,48092] 0 2024-07-03T15:44:14.541 INFO:tasks.workunit.client.0.smithi028.stdout:6/766: dread d4/d78/d10c/fa8 [0,4194304] 0 2024-07-03T15:44:14.545 INFO:tasks.workunit.client.0.smithi028.stdout:4/712: write d2/d5a/d45/f4c [1544457,122236] 0 2024-07-03T15:44:14.547 INFO:tasks.workunit.client.0.smithi028.stdout:4/713: symlink d2/d12/d42/d63/d97/le8 0 2024-07-03T15:44:14.571 INFO:tasks.workunit.client.0.smithi028.stdout:3/641: sync 2024-07-03T15:44:14.572 INFO:tasks.workunit.client.0.smithi028.stdout:3/642: link d18/d1a/d1e/cba d18/d1a/d1e/d58/d9f/cc9 0 2024-07-03T15:44:14.577 INFO:tasks.workunit.client.0.smithi028.stdout:6/767: dwrite d4/d78/d10c/d47/ff6 [0,4194304] 0 2024-07-03T15:44:14.580 INFO:tasks.workunit.client.0.smithi028.stdout:4/714: dwrite d2/d12/d42/d63/d75/fa9 [0,4194304] 0 2024-07-03T15:44:14.580 INFO:tasks.workunit.client.0.smithi028.stdout:4/715: stat d2/d12/d2d/l7e 0 2024-07-03T15:44:14.581 INFO:tasks.workunit.client.0.smithi028.stdout:6/768: mkdir d4/d78/d10c/d38/d3d/d89/dd8/d10d 0 2024-07-03T15:44:14.581 INFO:tasks.workunit.client.0.smithi028.stdout:6/769: fsync d4/d78/d10c/d38/f4a 0 2024-07-03T15:44:14.581 INFO:tasks.workunit.client.0.smithi028.stdout:6/770: chown d4/d12/d2c/lcd 30498945 1 2024-07-03T15:44:14.581 INFO:tasks.workunit.client.0.smithi028.stdout:6/771: creat d4/d78/f10e x:0 0 0 2024-07-03T15:44:14.584 INFO:tasks.workunit.client.0.smithi028.stdout:4/716: mkdir d2/d12/d2d/de9 0 2024-07-03T15:44:14.585 INFO:tasks.workunit.client.0.smithi028.stdout:8/788: dwrite d0/d6c/fef [0,4194304] 0 2024-07-03T15:44:14.586 INFO:tasks.workunit.client.0.smithi028.stdout:3/643: dread d18/f52 [0,4194304] 0 2024-07-03T15:44:14.590 INFO:tasks.workunit.client.0.smithi028.stdout:6/772: getdents d4/d78/d10c/d38/d3d/d89 0 2024-07-03T15:44:14.591 INFO:tasks.workunit.client.0.smithi028.stdout:4/717: truncate d2/d12/d5b/f7a 2347077 0 2024-07-03T15:44:14.591 INFO:tasks.workunit.client.0.smithi028.stdout:4/718: dread - d2/d12/d42/d63/f65 zero size 2024-07-03T15:44:14.591 INFO:tasks.workunit.client.0.smithi028.stdout:4/719: stat d2/d12/d2e/d87 0 2024-07-03T15:44:14.592 INFO:tasks.workunit.client.0.smithi028.stdout:4/720: creat d2/d12/d5b/fea x:0 0 0 2024-07-03T15:44:14.594 INFO:tasks.workunit.client.0.smithi028.stdout:3/644: mkdir d18/d6f/d78/d87/dca 0 2024-07-03T15:44:14.594 INFO:tasks.workunit.client.0.smithi028.stdout:3/645: write d18/d43/f4d [984218,58877] 0 2024-07-03T15:44:14.599 INFO:tasks.workunit.client.0.smithi028.stdout:4/721: mknod d2/d3e/d9f/d74/ceb 0 2024-07-03T15:44:14.601 INFO:tasks.workunit.client.0.smithi028.stdout:3/646: unlink d18/d1a/f20 0 2024-07-03T15:44:14.603 INFO:tasks.workunit.client.0.smithi028.stdout:3/647: mknod d18/d6f/ccb 0 2024-07-03T15:44:14.606 INFO:tasks.workunit.client.0.smithi028.stdout:8/789: dwrite d0/d6c/fae [0,4194304] 0 2024-07-03T15:44:14.606 INFO:tasks.workunit.client.0.smithi028.stdout:6/773: dwrite d4/d78/d10c/d38/f46 [0,4194304] 0 2024-07-03T15:44:14.606 INFO:tasks.workunit.client.0.smithi028.stdout:3/648: link d18/d1a/d1e/d58/f96 d18/d6f/d78/d87/dca/fcc 0 2024-07-03T15:44:14.609 INFO:tasks.workunit.client.0.smithi028.stdout:8/790: truncate d0/d41/d50/d3/d35/d68/d22/d40/d4a/fd8 1075141 0 2024-07-03T15:44:14.611 INFO:tasks.workunit.client.0.smithi028.stdout:6/774: dread d4/f29 [0,4194304] 0 2024-07-03T15:44:14.615 INFO:tasks.workunit.client.0.smithi028.stdout:3/649: creat d18/d1a/d1e/fcd x:0 0 0 2024-07-03T15:44:14.622 INFO:tasks.workunit.client.0.smithi028.stdout:0/747: sync 2024-07-03T15:44:14.623 INFO:tasks.workunit.client.0.smithi028.stdout:8/791: rename d0/d41/d50/d3/d35/d68/d22/d40/d4a/fd8 to d0/d41/d50/d3/dc/d33/d7a/f102 0 2024-07-03T15:44:14.625 INFO:tasks.workunit.client.0.smithi028.stdout:0/748: write d2/d1c/f68 [4081153,85019] 0 2024-07-03T15:44:14.626 INFO:tasks.workunit.client.0.smithi028.stdout:6/775: rename d4/fc9 to d4/d78/d10c/d47/d9f/f10f 0 2024-07-03T15:44:14.629 INFO:tasks.workunit.client.0.smithi028.stdout:8/792: getdents d0/d41/d50/d3/dc/d3f/d8e/d69 0 2024-07-03T15:44:14.629 INFO:tasks.workunit.client.0.smithi028.stdout:8/793: chown d0/d41/d50/d3/d35/c74 1651037 1 2024-07-03T15:44:14.636 INFO:tasks.workunit.client.0.smithi028.stdout:3/650: rename fe to d18/d43/fce 0 2024-07-03T15:44:14.636 INFO:tasks.workunit.client.0.smithi028.stdout:8/794: dread d0/d41/d50/d3/dc/d3f/d8e/f42 [0,4194304] 0 2024-07-03T15:44:14.639 INFO:tasks.workunit.client.0.smithi028.stdout:0/749: rename d2/d3/d4d/d96/dc2/fd5 to d2/d3/d4d/d56/d78/da8/dd0/feb 0 2024-07-03T15:44:14.641 INFO:tasks.workunit.client.0.smithi028.stdout:0/750: rename d2/d3/d4d/d96/fb5 to d2/d3/d4d/d56/d78/dae/fec 0 2024-07-03T15:44:14.641 INFO:tasks.workunit.client.0.smithi028.stdout:0/751: chown d2/d6d/ld2 1 1 2024-07-03T15:44:14.642 INFO:tasks.workunit.client.0.smithi028.stdout:0/752: mknod d2/d1c/d6c/da9/ced 0 2024-07-03T15:44:14.644 INFO:tasks.workunit.client.0.smithi028.stdout:0/753: creat d2/d1c/fee x:0 0 0 2024-07-03T15:44:14.645 INFO:tasks.workunit.client.0.smithi028.stdout:0/754: symlink d2/d3/d4d/d96/dc2/lef 0 2024-07-03T15:44:14.647 INFO:tasks.workunit.client.0.smithi028.stdout:0/755: creat d2/d3/d4d/d56/d78/ff0 x:0 0 0 2024-07-03T15:44:14.648 INFO:tasks.workunit.client.0.smithi028.stdout:0/756: getdents d2/d3/d4d/d56/d78/dbe 0 2024-07-03T15:44:14.652 INFO:tasks.workunit.client.0.smithi028.stdout:8/795: dread d0/d41/d50/d3/dc/d3f/df3/fb9 [0,4194304] 0 2024-07-03T15:44:14.654 INFO:tasks.workunit.client.0.smithi028.stdout:8/796: mkdir d0/d41/d50/d3/d35/d68/d22/d40/db7/d103 0 2024-07-03T15:44:14.654 INFO:tasks.workunit.client.0.smithi028.stdout:8/797: write d0/d6c/da9/fb4 [773184,63125] 0 2024-07-03T15:44:14.655 INFO:tasks.workunit.client.0.smithi028.stdout:3/651: dwrite d18/d1a/d5e/d66/f95 [0,4194304] 0 2024-07-03T15:44:14.655 INFO:tasks.workunit.client.0.smithi028.stdout:8/798: mkdir d0/d41/d50/d3/dc/d33/d7a/d104 0 2024-07-03T15:44:14.661 INFO:tasks.workunit.client.0.smithi028.stdout:0/757: dread d2/d3/d4d/fa2 [0,4194304] 0 2024-07-03T15:44:14.666 INFO:tasks.workunit.client.0.smithi028.stdout:3/652: link f9 d18/d3a/d9a/fcf 0 2024-07-03T15:44:14.669 INFO:tasks.workunit.client.0.smithi028.stdout:8/799: symlink d0/d41/d50/d3/d35/d68/d22/d40/l105 0 2024-07-03T15:44:14.670 INFO:tasks.workunit.client.0.smithi028.stdout:0/758: unlink d2/d3/d4d/d56/d78/lda 0 2024-07-03T15:44:14.671 INFO:tasks.workunit.client.0.smithi028.stdout:3/653: mkdir d18/d3a/dc1/dd0 0 2024-07-03T15:44:14.672 INFO:tasks.workunit.client.0.smithi028.stdout:3/654: write d18/d1a/d5e/f5f [1163013,1921] 0 2024-07-03T15:44:14.672 INFO:tasks.workunit.client.0.smithi028.stdout:3/655: chown f14 5 1 2024-07-03T15:44:14.672 INFO:tasks.workunit.client.0.smithi028.stdout:8/800: creat d0/d41/d50/d3/d35/d68/d22/d40/db7/f106 x:0 0 0 2024-07-03T15:44:14.676 INFO:tasks.workunit.client.0.smithi028.stdout:3/656: rename d18/d1a/d1e/d58/f7e to d18/d1a/fd1 0 2024-07-03T15:44:14.679 INFO:tasks.workunit.client.0.smithi028.stdout:3/657: unlink d18/d6f/f71 0 2024-07-03T15:44:14.684 INFO:tasks.workunit.client.0.smithi028.stdout:3/658: dread f6 [0,4194304] 0 2024-07-03T15:44:14.685 INFO:tasks.workunit.client.0.smithi028.stdout:3/659: creat d18/d1a/d1e/fd2 x:0 0 0 2024-07-03T15:44:14.694 INFO:tasks.workunit.client.0.smithi028.stdout:8/801: dwrite d0/d41/d50/d3/d35/d86/f87 [0,4194304] 0 2024-07-03T15:44:14.695 INFO:tasks.workunit.client.0.smithi028.stdout:8/802: symlink d0/d41/d50/l107 0 2024-07-03T15:44:14.695 INFO:tasks.workunit.client.0.smithi028.stdout:8/803: mkdir d0/d41/d50/daf/dc8/d108 0 2024-07-03T15:44:14.697 INFO:tasks.workunit.client.0.smithi028.stdout:8/804: symlink d0/d41/d50/d3/dc/l109 0 2024-07-03T15:44:14.704 INFO:tasks.workunit.client.0.smithi028.stdout:3/660: dwrite d18/f42 [0,4194304] 0 2024-07-03T15:44:14.711 INFO:tasks.workunit.client.0.smithi028.stdout:8/805: dread d0/d41/d50/d3/d35/d68/d22/f25 [4194304,4194304] 0 2024-07-03T15:44:14.711 INFO:tasks.workunit.client.0.smithi028.stdout:8/806: truncate d0/d41/d50/d3/dc/fcc 3188754 0 2024-07-03T15:44:14.712 INFO:tasks.workunit.client.0.smithi028.stdout:8/807: fdatasync d0/f2a 0 2024-07-03T15:44:14.713 INFO:tasks.workunit.client.0.smithi028.stdout:8/808: getdents d0/d41/d50/d97 0 2024-07-03T15:44:14.717 INFO:tasks.workunit.client.0.smithi028.stdout:3/661: write d18/d1a/f2a [3934620,70248] 0 2024-07-03T15:44:14.719 INFO:tasks.workunit.client.0.smithi028.stdout:3/662: creat d18/d3a/dc1/fd3 x:0 0 0 2024-07-03T15:44:14.721 INFO:tasks.workunit.client.0.smithi028.stdout:3/663: creat d18/d6f/d78/d87/d9c/fd4 x:0 0 0 2024-07-03T15:44:14.723 INFO:tasks.workunit.client.0.smithi028.stdout:3/664: write d18/d6f/f7d [59746,32391] 0 2024-07-03T15:44:14.726 INFO:tasks.workunit.client.0.smithi028.stdout:8/809: write d0/d41/d50/d3/dc/d3f/d8e/fec [3209824,76927] 0 2024-07-03T15:44:14.726 INFO:tasks.workunit.client.0.smithi028.stdout:8/810: chown d0/d41/d50/d3/dc/fb2 61 1 2024-07-03T15:44:14.726 INFO:tasks.workunit.client.0.smithi028.stdout:8/811: chown d0/d41/d50/d3/dc 160894 1 2024-07-03T15:44:14.730 INFO:tasks.workunit.client.0.smithi028.stdout:3/665: read d18/f51 [1474915,66737] 0 2024-07-03T15:44:14.733 INFO:tasks.workunit.client.0.smithi028.stdout:3/666: creat d18/d1a/d1e/d58/d9f/da9/fd5 x:0 0 0 2024-07-03T15:44:14.733 INFO:tasks.workunit.client.0.smithi028.stdout:3/667: dread - d18/d3a/f76 zero size 2024-07-03T15:44:14.735 INFO:tasks.workunit.client.0.smithi028.stdout:3/668: getdents d18/d1a/d1e/d73 0 2024-07-03T15:44:14.736 INFO:tasks.workunit.client.0.smithi028.stdout:3/669: chown d18/d3a/f5a 51963594 1 2024-07-03T15:44:14.736 INFO:tasks.workunit.client.0.smithi028.stdout:3/670: write d18/d3a/f76 [856461,66915] 0 2024-07-03T15:44:14.736 INFO:tasks.workunit.client.0.smithi028.stdout:3/671: creat d18/d1a/d5e/d66/da3/fd6 x:0 0 0 2024-07-03T15:44:14.745 INFO:tasks.workunit.client.0.smithi028.stdout:8/812: dwrite d0/d41/d50/d3/d35/d68/d22/f25 [0,4194304] 0 2024-07-03T15:44:14.746 INFO:tasks.workunit.client.0.smithi028.stdout:8/813: getdents d0/de0/dcf 0 2024-07-03T15:44:14.746 INFO:tasks.workunit.client.0.smithi028.stdout:8/814: creat d0/d41/d50/d3/d35/d68/d22/d40/db7/f10a x:0 0 0 2024-07-03T15:44:14.748 INFO:tasks.workunit.client.0.smithi028.stdout:9/734: sync 2024-07-03T15:44:14.748 INFO:tasks.workunit.client.0.smithi028.stdout:7/524: sync 2024-07-03T15:44:14.749 INFO:tasks.workunit.client.0.smithi028.stdout:2/534: sync 2024-07-03T15:44:14.749 INFO:tasks.workunit.client.0.smithi028.stdout:1/934: sync 2024-07-03T15:44:14.749 INFO:tasks.workunit.client.0.smithi028.stdout:7/525: write d9/d1c/d45/f62 [1059720,107378] 0 2024-07-03T15:44:14.749 INFO:tasks.workunit.client.0.smithi028.stdout:5/671: sync 2024-07-03T15:44:14.750 INFO:tasks.workunit.client.0.smithi028.stdout:1/935: truncate d4/de/f6f 68850 0 2024-07-03T15:44:14.751 INFO:tasks.workunit.client.0.smithi028.stdout:9/735: symlink d1/d10/d28/d4e/d5b/de9/lf8 0 2024-07-03T15:44:14.752 INFO:tasks.workunit.client.0.smithi028.stdout:5/672: mkdir d0/dd/d1e/d3b/d3f/d71/de1 0 2024-07-03T15:44:14.752 INFO:tasks.workunit.client.0.smithi028.stdout:5/673: chown d0/dd/d13/d29/dc4/fbe 279 1 2024-07-03T15:44:14.754 INFO:tasks.workunit.client.0.smithi028.stdout:9/736: unlink d1/d10/d28/ld9 0 2024-07-03T15:44:14.754 INFO:tasks.workunit.client.0.smithi028.stdout:9/737: readlink d1/d10/d28/d45/d61/d8c/l44 0 2024-07-03T15:44:14.757 INFO:tasks.workunit.client.0.smithi028.stdout:3/672: dwrite ff [0,4194304] 0 2024-07-03T15:44:14.757 INFO:tasks.workunit.client.0.smithi028.stdout:3/673: readlink d18/d1a/d1e/d58/d9f/lc6 0 2024-07-03T15:44:14.761 INFO:tasks.workunit.client.0.smithi028.stdout:2/535: dread da/d1e/d68/f6c [0,4194304] 0 2024-07-03T15:44:14.762 INFO:tasks.workunit.client.0.smithi028.stdout:1/936: dread d4/d3c/d54/fe7 [0,4194304] 0 2024-07-03T15:44:14.765 INFO:tasks.workunit.client.0.smithi028.stdout:7/526: dwrite d9/d1c/f3f [4194304,4194304] 0 2024-07-03T15:44:14.766 INFO:tasks.workunit.client.0.smithi028.stdout:8/815: dwrite d0/d41/d50/d3/dc/d4d/fc6 [0,4194304] 0 2024-07-03T15:44:14.767 INFO:tasks.workunit.client.0.smithi028.stdout:7/527: dread d9/d29/f2e [0,4194304] 0 2024-07-03T15:44:14.767 INFO:tasks.workunit.client.0.smithi028.stdout:7/528: chown d9/d29/f4f 0 1 2024-07-03T15:44:14.829 INFO:tasks.workunit.client.0.smithi028.stdout:5/674: mknod d0/dd/d13/d47/d50/ce2 0 2024-07-03T15:44:14.829 INFO:tasks.workunit.client.0.smithi028.stdout:5/675: getdents d0/dd/d1e/d3b/ddd 0 2024-07-03T15:44:14.831 INFO:tasks.workunit.client.0.smithi028.stdout:9/738: rename d1/dd/f68 to d1/d10/d28/d45/d61/d8c/ff9 0 2024-07-03T15:44:14.832 INFO:tasks.workunit.client.0.smithi028.stdout:9/739: truncate d1/d10/d28/d4e/d5b/f84 297209 0 2024-07-03T15:44:14.832 INFO:tasks.workunit.client.0.smithi028.stdout:9/740: write d1/d16/d57/d5f/d7f/fad [751294,43271] 0 2024-07-03T15:44:14.832 INFO:tasks.workunit.client.0.smithi028.stdout:9/741: chown d1/dd/f3c 1276569 1 2024-07-03T15:44:14.833 INFO:tasks.workunit.client.0.smithi028.stdout:3/674: creat d18/d3a/d9d/d8c/fd7 x:0 0 0 2024-07-03T15:44:14.833 INFO:tasks.workunit.client.0.smithi028.stdout:3/675: creat d18/d1a/d5e/d66/fd8 x:0 0 0 2024-07-03T15:44:14.858 INFO:tasks.workunit.client.0.smithi028.stdout:2/536: unlink da/d23/d30/f51 0 2024-07-03T15:44:14.860 INFO:tasks.workunit.client.0.smithi028.stdout:5/676: dwrite d0/dd/d1e/f89 [0,4194304] 0 2024-07-03T15:44:14.860 INFO:tasks.workunit.client.0.smithi028.stdout:5/677: fdatasync d0/dd/d14/dc0/fd3 0 2024-07-03T15:44:14.860 INFO:tasks.workunit.client.0.smithi028.stdout:5/678: fdatasync d0/dd/d13/d57/d77/f7d 0 2024-07-03T15:44:14.861 INFO:tasks.workunit.client.0.smithi028.stdout:7/529: symlink d9/d1c/d7c/d8b/l97 0 2024-07-03T15:44:14.875 INFO:tasks.workunit.client.0.smithi028.stdout:3/676: unlink d18/d1a/d1e/d58/d9f/cc9 0 2024-07-03T15:44:14.879 INFO:tasks.workunit.client.0.smithi028.stdout:2/537: mkdir da/d24/dc1 0 2024-07-03T15:44:14.882 INFO:tasks.workunit.client.0.smithi028.stdout:5/679: rename d0/dd/d13/d57/d30/d70/d7f/d9e/fd5 to d0/dd/d1e/d3b/de0/fe3 0 2024-07-03T15:44:14.884 INFO:tasks.workunit.client.0.smithi028.stdout:1/937: rmdir d4/d83/de8/d121/d3a/d5c/daa/df0 39 2024-07-03T15:44:14.884 INFO:tasks.workunit.client.0.smithi028.stdout:2/538: read da/d24/fa4 [512589,48100] 0 2024-07-03T15:44:14.885 INFO:tasks.workunit.client.0.smithi028.stdout:7/530: rmdir d9/d1c/d52 39 2024-07-03T15:44:14.885 INFO:tasks.workunit.client.0.smithi028.stdout:7/531: readlink d9/d1c/d6e/l94 0 2024-07-03T15:44:14.886 INFO:tasks.workunit.client.0.smithi028.stdout:3/677: getdents d18 0 2024-07-03T15:44:14.886 INFO:tasks.workunit.client.0.smithi028.stdout:3/678: dread - d18/d3a/d9d/fc5 zero size 2024-07-03T15:44:14.888 INFO:tasks.workunit.client.0.smithi028.stdout:5/680: mkdir d0/dd/d1e/d8e/de4 0 2024-07-03T15:44:14.888 INFO:tasks.workunit.client.0.smithi028.stdout:5/681: write d0/f2c [7373890,64923] 0 2024-07-03T15:44:14.888 INFO:tasks.workunit.client.0.smithi028.stdout:1/938: creat d4/d3c/d54/d84/dfb/f145 x:0 0 0 2024-07-03T15:44:14.889 INFO:tasks.workunit.client.0.smithi028.stdout:2/539: mknod da/d23/d6f/cc2 0 2024-07-03T15:44:14.889 INFO:tasks.workunit.client.0.smithi028.stdout:2/540: read da/d1e/d56/f9a [3304936,22278] 0 2024-07-03T15:44:14.890 INFO:tasks.workunit.client.0.smithi028.stdout:2/541: readlink da/d1e/d68/l7f 0 2024-07-03T15:44:14.890 INFO:tasks.workunit.client.0.smithi028.stdout:3/679: rename d18/d3a/d9d/d8c/cbd to d18/d1a/cd9 0 2024-07-03T15:44:14.893 INFO:tasks.workunit.client.0.smithi028.stdout:1/939: creat d4/d3c/d54/d79/d16/d57/d66/dd6/f146 x:0 0 0 2024-07-03T15:44:14.893 INFO:tasks.workunit.client.0.smithi028.stdout:1/940: dread - d4/d3c/d54/d79/d16/db6/f129 zero size 2024-07-03T15:44:14.893 INFO:tasks.workunit.client.0.smithi028.stdout:2/542: truncate da/d24/fa4 1548504 0 2024-07-03T15:44:14.893 INFO:tasks.workunit.client.0.smithi028.stdout:2/543: fsync da/d24/f55 0 2024-07-03T15:44:14.893 INFO:tasks.workunit.client.0.smithi028.stdout:7/532: write d9/d29/f3e [2733877,70971] 0 2024-07-03T15:44:14.895 INFO:tasks.workunit.client.0.smithi028.stdout:3/680: link d18/d3a/d9d/laa d18/d6f/d78/d87/d9c/lda 0 2024-07-03T15:44:14.895 INFO:tasks.workunit.client.0.smithi028.stdout:3/681: getdents d18/d3a/db6 0 2024-07-03T15:44:14.895 INFO:tasks.workunit.client.0.smithi028.stdout:3/682: creat d18/d3a/d9d/d8c/fdb x:0 0 0 2024-07-03T15:44:14.898 INFO:tasks.workunit.client.0.smithi028.stdout:2/544: stat da/d1e/c4e 0 2024-07-03T15:44:14.898 INFO:tasks.workunit.client.0.smithi028.stdout:2/545: chown da/d42/d47/d78/fb1 25366 1 2024-07-03T15:44:14.898 INFO:tasks.workunit.client.0.smithi028.stdout:7/533: symlink d9/d29/d5d/l98 0 2024-07-03T15:44:14.899 INFO:tasks.workunit.client.0.smithi028.stdout:3/683: mkdir d18/d1a/d64/ddc 0 2024-07-03T15:44:14.900 INFO:tasks.workunit.client.0.smithi028.stdout:1/941: getdents d4/de/d1c 0 2024-07-03T15:44:14.901 INFO:tasks.workunit.client.0.smithi028.stdout:2/546: symlink da/d1e/d56/dc0/lc3 0 2024-07-03T15:44:14.903 INFO:tasks.workunit.client.0.smithi028.stdout:3/684: mkdir d18/d1a/d1e/d58/ddd 0 2024-07-03T15:44:14.903 INFO:tasks.workunit.client.0.smithi028.stdout:3/685: dread - d18/d3a/d9d/d8c/faf zero size 2024-07-03T15:44:14.903 INFO:tasks.workunit.client.0.smithi028.stdout:3/686: stat d18/d3a/c4b 0 2024-07-03T15:44:14.903 INFO:tasks.workunit.client.0.smithi028.stdout:3/687: stat d18/d1a/d5e/d66/f91 0 2024-07-03T15:44:14.904 INFO:tasks.workunit.client.0.smithi028.stdout:1/942: stat d4/d83/de8/d121/l45 0 2024-07-03T15:44:14.905 INFO:tasks.workunit.client.0.smithi028.stdout:7/534: rename d9/d1c/d52 to d9/d29/d92/d99 0 2024-07-03T15:44:14.907 INFO:tasks.workunit.client.0.smithi028.stdout:1/943: creat d4/d3c/d54/d79/d16/d7c/f147 x:0 0 0 2024-07-03T15:44:14.907 INFO:tasks.workunit.client.0.smithi028.stdout:1/944: readlink d4/d83/de8/d121/d3a/d5c/d6c/d7d/l34 0 2024-07-03T15:44:14.908 INFO:tasks.workunit.client.0.smithi028.stdout:3/688: rename d18/d1a/d1e/fcd to d18/d3a/d9d/fde 0 2024-07-03T15:44:14.911 INFO:tasks.workunit.client.0.smithi028.stdout:3/689: mkdir d18/d3a/ddf 0 2024-07-03T15:44:14.911 INFO:tasks.workunit.client.0.smithi028.stdout:3/690: fdatasync d18/d3a/f39 0 2024-07-03T15:44:14.911 INFO:tasks.workunit.client.0.smithi028.stdout:3/691: dread - d18/d1a/d1e/f82 zero size 2024-07-03T15:44:14.913 INFO:tasks.workunit.client.0.smithi028.stdout:3/692: mknod d18/d3a/dc1/ce0 0 2024-07-03T15:44:14.923 INFO:tasks.workunit.client.0.smithi028.stdout:5/682: write d0/f9 [3386514,26603] 0 2024-07-03T15:44:14.927 INFO:tasks.workunit.client.0.smithi028.stdout:1/945: dwrite d4/d83/de8/d121/d3a/d43/f77 [4194304,4194304] 0 2024-07-03T15:44:14.928 INFO:tasks.workunit.client.0.smithi028.stdout:1/946: dread - d4/de/d1c/d81/f11f zero size 2024-07-03T15:44:14.938 INFO:tasks.workunit.client.0.smithi028.stdout:1/947: rmdir d4/d83/de8/d121/d3a/d5c/d6c/d7d/d59 39 2024-07-03T15:44:14.939 INFO:tasks.workunit.client.0.smithi028.stdout:2/547: dwrite da/d74/f8d [0,4194304] 0 2024-07-03T15:44:14.939 INFO:tasks.workunit.client.0.smithi028.stdout:1/948: dread - d4/d83/de8/d121/d3a/d5c/daa/df0/d123/d125/f131 zero size 2024-07-03T15:44:14.945 INFO:tasks.workunit.client.0.smithi028.stdout:1/949: link d4/d83/de8/d121/d3a/d43/f77 d4/d3c/d54/d79/d16/d7c/f148 0 2024-07-03T15:44:14.947 INFO:tasks.workunit.client.0.smithi028.stdout:5/683: dwrite d0/dd/d1e/d3b/d3f/d5a/f78 [0,4194304] 0 2024-07-03T15:44:14.947 INFO:tasks.workunit.client.0.smithi028.stdout:5/684: chown d0/dd/d13/d47/d50/ce2 0 1 2024-07-03T15:44:14.947 INFO:tasks.workunit.client.0.smithi028.stdout:2/548: read da/f10 [252408,49315] 0 2024-07-03T15:44:14.948 INFO:tasks.workunit.client.0.smithi028.stdout:1/950: rename d4/d3c/d54/d79/d16/d57/d138/l139 to d4/de/d1c/d44/l149 0 2024-07-03T15:44:14.958 INFO:tasks.workunit.client.0.smithi028.stdout:5/685: symlink d0/dd/d1e/d8e/de4/le5 0 2024-07-03T15:44:14.959 INFO:tasks.workunit.client.0.smithi028.stdout:2/549: link da/d42/fac da/fc4 0 2024-07-03T15:44:14.959 INFO:tasks.workunit.client.0.smithi028.stdout:2/550: creat da/d23/d30/fc5 x:0 0 0 2024-07-03T15:44:14.961 INFO:tasks.workunit.client.0.smithi028.stdout:1/951: truncate d4/d83/de8/d121/dbe/f13f 268495 0 2024-07-03T15:44:14.962 INFO:tasks.workunit.client.0.smithi028.stdout:5/686: truncate d0/dd/d1e/d3b/fa3 3489897 0 2024-07-03T15:44:14.962 INFO:tasks.workunit.client.0.smithi028.stdout:5/687: write d0/dd/d1e/d86/f88 [85630,100526] 0 2024-07-03T15:44:14.965 INFO:tasks.workunit.client.0.smithi028.stdout:1/952: getdents d4/d83/de8/d121/d3a/d5c/d6c/d7d/d137 0 2024-07-03T15:44:14.966 INFO:tasks.workunit.client.0.smithi028.stdout:5/688: truncate d0/dd/df/faf 742814 0 2024-07-03T15:44:14.967 INFO:tasks.workunit.client.0.smithi028.stdout:5/689: write d0/dd/d13/d29/f35 [5455334,66501] 0 2024-07-03T15:44:14.989 INFO:tasks.workunit.client.0.smithi028.stdout:2/551: dwrite da/d23/f75 [0,4194304] 0 2024-07-03T15:44:14.989 INFO:tasks.workunit.client.0.smithi028.stdout:5/690: dwrite d0/dd/fae [0,4194304] 0 2024-07-03T15:44:14.991 INFO:tasks.workunit.client.0.smithi028.stdout:1/953: dwrite d4/d3c/d54/d79/d16/d7c/fcc [0,4194304] 0 2024-07-03T15:44:14.994 INFO:tasks.workunit.client.0.smithi028.stdout:2/552: mkdir da/d24/d2c/dc6 0 2024-07-03T15:44:14.995 INFO:tasks.workunit.client.0.smithi028.stdout:2/553: chown da/d42/d47/fa5 16257952 1 2024-07-03T15:44:14.995 INFO:tasks.workunit.client.0.smithi028.stdout:2/554: creat da/d24/fc7 x:0 0 0 2024-07-03T15:44:14.995 INFO:tasks.workunit.client.0.smithi028.stdout:2/555: write da/d24/f8c [1405956,91964] 0 2024-07-03T15:44:14.999 INFO:tasks.workunit.client.0.smithi028.stdout:5/691: mkdir d0/dd/d1e/d3b/d3f/d71/de1/de6 0 2024-07-03T15:44:15.001 INFO:tasks.workunit.client.0.smithi028.stdout:1/954: write d4/de/f85 [287304,16721] 0 2024-07-03T15:44:15.006 INFO:tasks.workunit.client.0.smithi028.stdout:2/556: creat da/d74/fc8 x:0 0 0 2024-07-03T15:44:15.006 INFO:tasks.workunit.client.0.smithi028.stdout:2/557: write da/d23/d6f/d84/fa7 [707879,130368] 0 2024-07-03T15:44:15.008 INFO:tasks.workunit.client.0.smithi028.stdout:5/692: creat d0/dd/d13/d57/d26/fe7 x:0 0 0 2024-07-03T15:44:15.008 INFO:tasks.workunit.client.0.smithi028.stdout:5/693: write d0/dd/d13/d57/f85 [91972,130411] 0 2024-07-03T15:44:15.008 INFO:tasks.workunit.client.0.smithi028.stdout:5/694: chown d0/dd/d13/d57/d30/d70/d7f 15828 1 2024-07-03T15:44:15.009 INFO:tasks.workunit.client.0.smithi028.stdout:2/558: creat da/d1e/fc9 x:0 0 0 2024-07-03T15:44:15.009 INFO:tasks.workunit.client.0.smithi028.stdout:2/559: fsync da/d42/d47/fbc 0 2024-07-03T15:44:15.014 INFO:tasks.workunit.client.0.smithi028.stdout:2/560: dread da/d42/d47/f81 [0,4194304] 0 2024-07-03T15:44:15.014 INFO:tasks.workunit.client.0.smithi028.stdout:2/561: write da/d42/d47/d78/fa3 [989210,32590] 0 2024-07-03T15:44:15.016 INFO:tasks.workunit.client.0.smithi028.stdout:2/562: dread da/d24/fa4 [0,4194304] 0 2024-07-03T15:44:15.018 INFO:tasks.workunit.client.0.smithi028.stdout:1/955: dwrite d4/d83/de8/d121/d3a/d5c/d6c/d7d/f114 [4194304,4194304] 0 2024-07-03T15:44:15.027 INFO:tasks.workunit.client.0.smithi028.stdout:2/563: getdents da/d42/d47/d78 0 2024-07-03T15:44:15.027 INFO:tasks.workunit.client.0.smithi028.stdout:2/564: write da/d24/f8c [1785207,58792] 0 2024-07-03T15:44:15.037 INFO:tasks.workunit.client.0.smithi028.stdout:2/565: write da/d1e/f58 [579981,36799] 0 2024-07-03T15:44:15.041 INFO:tasks.workunit.client.0.smithi028.stdout:1/956: dwrite d4/d83/f8b [0,4194304] 0 2024-07-03T15:44:15.041 INFO:tasks.workunit.client.0.smithi028.stdout:1/957: chown d4/d83/de8/d121/d3a/d5c/d6c/d7d/f114 222724 1 2024-07-03T15:44:15.058 INFO:tasks.workunit.client.0.smithi028.stdout:5/695: rmdir d0/dd/d1e/d3b 39 2024-07-03T15:44:15.064 INFO:tasks.workunit.client.0.smithi028.stdout:2/566: dwrite da/d42/d9e/fa0 [0,4194304] 0 2024-07-03T15:44:15.064 INFO:tasks.workunit.client.0.smithi028.stdout:2/567: chown da/ff 898965660 1 2024-07-03T15:44:15.064 INFO:tasks.workunit.client.0.smithi028.stdout:1/958: dwrite d4/de/fc5 [0,4194304] 0 2024-07-03T15:44:15.069 INFO:tasks.workunit.client.0.smithi028.stdout:5/696: write d0/dd/d1e/d3b/f4f [5975424,126504] 0 2024-07-03T15:44:15.076 INFO:tasks.workunit.client.0.smithi028.stdout:2/568: dread da/ff [0,4194304] 0 2024-07-03T15:44:15.079 INFO:tasks.workunit.client.0.smithi028.stdout:4/722: sync 2024-07-03T15:44:15.079 INFO:tasks.workunit.client.0.smithi028.stdout:4/723: creat d2/d5a/d45/d84/fec x:0 0 0 2024-07-03T15:44:15.079 INFO:tasks.workunit.client.0.smithi028.stdout:4/724: stat d2/d12/d42/d63/f65 0 2024-07-03T15:44:15.084 INFO:tasks.workunit.client.0.smithi028.stdout:5/697: dread d0/dd/d98/f6c [0,4194304] 0 2024-07-03T15:44:15.098 INFO:tasks.workunit.client.0.smithi028.stdout:4/725: dwrite d2/d12/d5b/fea [0,4194304] 0 2024-07-03T15:44:15.098 INFO:tasks.workunit.client.0.smithi028.stdout:4/726: creat d2/d5a/d45/fed x:0 0 0 2024-07-03T15:44:15.118 INFO:tasks.workunit.client.0.smithi028.stdout:2/569: unlink da/d1e/d68/f6c 0 2024-07-03T15:44:15.118 INFO:tasks.workunit.client.0.smithi028.stdout:2/570: chown da/d1e/d56/f9a 374 1 2024-07-03T15:44:15.118 INFO:tasks.workunit.client.0.smithi028.stdout:2/571: chown da/d23/d6f/fae 466362137 1 2024-07-03T15:44:15.150 INFO:tasks.workunit.client.0.smithi028.stdout:4/727: symlink d2/d5a/lee 0 2024-07-03T15:44:15.159 INFO:tasks.workunit.client.0.smithi028.stdout:2/572: dwrite da/d24/d2c/f97 [0,4194304] 0 2024-07-03T15:44:15.163 INFO:tasks.workunit.client.0.smithi028.stdout:2/573: read da/d42/f4d [153690,42067] 0 2024-07-03T15:44:15.183 INFO:tasks.workunit.client.0.smithi028.stdout:4/728: dwrite d2/d3e/f50 [0,4194304] 0 2024-07-03T15:44:15.217 INFO:tasks.workunit.client.0.smithi028.stdout:1/959: getdents d4 0 2024-07-03T15:44:15.225 INFO:tasks.workunit.client.0.smithi028.stdout:2/574: link da/f12 da/d23/d6f/d84/fca 0 2024-07-03T15:44:15.225 INFO:tasks.workunit.client.0.smithi028.stdout:2/575: chown da/l59 129406 1 2024-07-03T15:44:15.225 INFO:tasks.workunit.client.0.smithi028.stdout:2/576: write da/d42/f57 [1229262,98638] 0 2024-07-03T15:44:15.225 INFO:tasks.workunit.client.0.smithi028.stdout:2/577: dread - da/d24/d2c/faf zero size 2024-07-03T15:44:15.225 INFO:tasks.workunit.client.0.smithi028.stdout:2/578: readlink da/d23/d30/l3a 0 2024-07-03T15:44:15.226 INFO:tasks.workunit.client.0.smithi028.stdout:1/960: dread d4/d3c/d54/d79/d16/d7c/f148 [4194304,4194304] 0 2024-07-03T15:44:15.228 INFO:tasks.workunit.client.0.smithi028.stdout:4/729: creat d2/d12/d2d/de9/fef x:0 0 0 2024-07-03T15:44:15.229 INFO:tasks.workunit.client.0.smithi028.stdout:4/730: creat d2/d12/d2c/ff0 x:0 0 0 2024-07-03T15:44:15.232 INFO:tasks.workunit.client.0.smithi028.stdout:2/579: dread da/d24/d2c/f61 [0,4194304] 0 2024-07-03T15:44:15.233 INFO:tasks.workunit.client.0.smithi028.stdout:1/961: creat d4/d83/de8/f14a x:0 0 0 2024-07-03T15:44:15.240 INFO:tasks.workunit.client.0.smithi028.stdout:1/962: rename d4/de/d1c/d81/fd3 to d4/d3c/d54/d79/f14b 0 2024-07-03T15:44:15.242 INFO:tasks.workunit.client.0.smithi028.stdout:1/963: rename d4/d3c/d54/d79/d16/d57/d66/dd6/f146 to d4/d3c/d54/d79/d16/f14c 0 2024-07-03T15:44:15.243 INFO:tasks.workunit.client.0.smithi028.stdout:1/964: read - d4/d3c/d54/d79/d16/f14c zero size 2024-07-03T15:44:15.243 INFO:tasks.workunit.client.0.smithi028.stdout:1/965: creat d4/d3c/d54/d79/f14d x:0 0 0 2024-07-03T15:44:15.246 INFO:tasks.workunit.client.0.smithi028.stdout:1/966: mknod d4/d3c/d54/d79/d16/db6/ddb/c14e 0 2024-07-03T15:44:15.248 INFO:tasks.workunit.client.0.smithi028.stdout:1/967: symlink d4/d83/de8/d121/d3a/d5c/d6c/d7d/d59/d94/l14f 0 2024-07-03T15:44:15.248 INFO:tasks.workunit.client.0.smithi028.stdout:4/731: dwrite d2/d12/f60 [0,4194304] 0 2024-07-03T15:44:15.257 INFO:tasks.workunit.client.0.smithi028.stdout:2/580: dwrite da/d74/f8d [0,4194304] 0 2024-07-03T15:44:15.262 INFO:tasks.workunit.client.0.smithi028.stdout:1/968: dread d4/d83/de8/d121/d3a/d5c/d6c/d7d/d59/f46 [0,4194304] 0 2024-07-03T15:44:15.263 INFO:tasks.workunit.client.0.smithi028.stdout:2/581: rename da/d23/d30/f9c to da/d42/d47/da2/fcb 0 2024-07-03T15:44:15.263 INFO:tasks.workunit.client.0.smithi028.stdout:2/582: chown da/f15 35 1 2024-07-03T15:44:15.264 INFO:tasks.workunit.client.0.smithi028.stdout:2/583: readlink da/d24/l5a 0 2024-07-03T15:44:15.265 INFO:tasks.workunit.client.0.smithi028.stdout:1/969: mkdir d4/d3c/d54/d79/d16/db6/ddb/d150 0 2024-07-03T15:44:15.273 INFO:tasks.workunit.client.0.smithi028.stdout:4/732: rmdir d2/d61/d9d 39 2024-07-03T15:44:15.277 INFO:tasks.workunit.client.0.smithi028.stdout:4/733: dread d2/f13 [0,4194304] 0 2024-07-03T15:44:15.279 INFO:tasks.workunit.client.0.smithi028.stdout:2/584: dwrite da/d24/f66 [0,4194304] 0 2024-07-03T15:44:15.279 INFO:tasks.workunit.client.0.smithi028.stdout:2/585: readlink da/d23/d6f/l88 0 2024-07-03T15:44:15.289 INFO:tasks.workunit.client.0.smithi028.stdout:1/970: dwrite d4/d83/de8/d121/d118/f11e [0,4194304] 0 2024-07-03T15:44:15.289 INFO:tasks.workunit.client.0.smithi028.stdout:4/734: write d2/d12/d2e/f47 [2481262,80257] 0 2024-07-03T15:44:15.293 INFO:tasks.workunit.client.0.smithi028.stdout:1/971: rename d4/c35 to d4/d3c/d54/d79/d16/db6/ddb/d150/c151 0 2024-07-03T15:44:15.293 INFO:tasks.workunit.client.0.smithi028.stdout:4/735: symlink d2/d61/lf1 0 2024-07-03T15:44:15.369 INFO:tasks.workunit.client.0.smithi028.stdout:1/972: creat d4/d3c/d54/d79/d16/d133/f152 x:0 0 0 2024-07-03T15:44:15.369 INFO:tasks.workunit.client.0.smithi028.stdout:1/973: chown d4/d83/de8/d121/d3a 3851 1 2024-07-03T15:44:15.371 INFO:tasks.workunit.client.0.smithi028.stdout:4/736: creat d2/d12/d5b/d78/de4/d3a/ff2 x:0 0 0 2024-07-03T15:44:15.372 INFO:tasks.workunit.client.0.smithi028.stdout:6/776: sync 2024-07-03T15:44:15.372 INFO:tasks.workunit.client.0.smithi028.stdout:7/535: sync 2024-07-03T15:44:15.372 INFO:tasks.workunit.client.0.smithi028.stdout:3/693: sync 2024-07-03T15:44:15.372 INFO:tasks.workunit.client.0.smithi028.stdout:5/698: sync 2024-07-03T15:44:15.372 INFO:tasks.workunit.client.0.smithi028.stdout:8/816: sync 2024-07-03T15:44:15.372 INFO:tasks.workunit.client.0.smithi028.stdout:8/817: chown d0/d41/d50/d3/d35/d68/d22/d40/db7/lc2 2887 1 2024-07-03T15:44:15.372 INFO:tasks.workunit.client.0.smithi028.stdout:6/777: creat d4/d12/d2c/ddd/f110 x:0 0 0 2024-07-03T15:44:15.373 INFO:tasks.workunit.client.0.smithi028.stdout:9/742: sync 2024-07-03T15:44:15.373 INFO:tasks.workunit.client.0.smithi028.stdout:0/759: sync 2024-07-03T15:44:15.373 INFO:tasks.workunit.client.0.smithi028.stdout:1/974: creat d4/d83/de8/d121/d3a/d5c/d6c/d7d/d137/f153 x:0 0 0 2024-07-03T15:44:15.379 INFO:tasks.workunit.client.0.smithi028.stdout:5/699: stat d0/dd/d1e/d3b/c3d 0 2024-07-03T15:44:15.380 INFO:tasks.workunit.client.0.smithi028.stdout:6/778: truncate d4/d78/d10c/d13/fcc 5581037 0 2024-07-03T15:44:15.380 INFO:tasks.workunit.client.0.smithi028.stdout:6/779: chown d4/d78/d10c/d38/d3d/d89/dd8 2505638 1 2024-07-03T15:44:15.380 INFO:tasks.workunit.client.0.smithi028.stdout:7/536: dread d9/d29/f81 [0,4194304] 0 2024-07-03T15:44:15.383 INFO:tasks.workunit.client.0.smithi028.stdout:1/975: mkdir d4/d3c/d54/d79/d16/d57/d66/d10e/d154 0 2024-07-03T15:44:15.383 INFO:tasks.workunit.client.0.smithi028.stdout:1/976: chown d4/d3c/d54/d79/d16/d7c/f148 3 1 2024-07-03T15:44:15.386 INFO:tasks.workunit.client.0.smithi028.stdout:3/694: rmdir d18/d1a/d5e/d66/da3 39 2024-07-03T15:44:15.388 INFO:tasks.workunit.client.0.smithi028.stdout:5/700: mknod d0/d87/ce8 0 2024-07-03T15:44:15.388 INFO:tasks.workunit.client.0.smithi028.stdout:5/701: chown d0/dd/d13/d57/fc1 31 1 2024-07-03T15:44:15.390 INFO:tasks.workunit.client.0.smithi028.stdout:6/780: creat d4/d78/d10c/d13/f111 x:0 0 0 2024-07-03T15:44:15.390 INFO:tasks.workunit.client.0.smithi028.stdout:7/537: getdents d9/d1c/d86 0 2024-07-03T15:44:15.392 INFO:tasks.workunit.client.0.smithi028.stdout:1/977: rename d4/d83/de8/d121/c41 to d4/d83/de8/d121/d3a/d5c/dff/c155 0 2024-07-03T15:44:15.399 INFO:tasks.workunit.client.0.smithi028.stdout:3/695: dread d18/d1a/d1e/d58/f6c [0,4194304] 0 2024-07-03T15:44:15.400 INFO:tasks.workunit.client.0.smithi028.stdout:0/760: dwrite d2/d3/dc4/f66 [0,4194304] 0 2024-07-03T15:44:15.400 INFO:tasks.workunit.client.0.smithi028.stdout:0/761: stat d2/d1c/d6c/f79 0 2024-07-03T15:44:15.400 INFO:tasks.workunit.client.0.smithi028.stdout:0/762: truncate d2/d3/dc4/d58/d82/fb6 685616 0 2024-07-03T15:44:15.412 INFO:tasks.workunit.client.0.smithi028.stdout:5/702: dwrite d0/d8f/fc9 [0,4194304] 0 2024-07-03T15:44:15.412 INFO:tasks.workunit.client.0.smithi028.stdout:5/703: fsync d0/f2c 0 2024-07-03T15:44:15.422 INFO:tasks.workunit.client.0.smithi028.stdout:6/781: dwrite d4/d78/d10c/d47/f51 [4194304,4194304] 0 2024-07-03T15:44:15.445 INFO:tasks.workunit.client.0.smithi028.stdout:7/538: dwrite d9/d29/f3c [0,4194304] 0 2024-07-03T15:44:15.445 INFO:tasks.workunit.client.0.smithi028.stdout:6/782: write d4/d78/d10c/d38/d60/fab [3650116,288] 0 2024-07-03T15:44:15.445 INFO:tasks.workunit.client.0.smithi028.stdout:0/763: dwrite d2/d3/dc4/d36/f88 [0,4194304] 0 2024-07-03T15:44:15.445 INFO:tasks.workunit.client.0.smithi028.stdout:0/764: write d2/f3a [337130,36567] 0 2024-07-03T15:44:15.445 INFO:tasks.workunit.client.0.smithi028.stdout:0/765: getdents d2/d3/dc4/d36/d45/dc8 0 2024-07-03T15:44:15.445 INFO:tasks.workunit.client.0.smithi028.stdout:1/978: dwrite d4/d83/de8/d121/d3a/d5c/d6c/d7d/d59/f46 [0,4194304] 0 2024-07-03T15:44:15.445 INFO:tasks.workunit.client.0.smithi028.stdout:1/979: fdatasync d4/d3c/d54/d79/d16/d7c/f9f 0 2024-07-03T15:44:15.445 INFO:tasks.workunit.client.0.smithi028.stdout:5/704: dwrite d0/dd/d1e/d3b/d3f/fd8 [0,4194304] 0 2024-07-03T15:44:15.445 INFO:tasks.workunit.client.0.smithi028.stdout:5/705: write d0/dd/d1e/d86/f88 [230990,34659] 0 2024-07-03T15:44:15.446 INFO:tasks.workunit.client.0.smithi028.stdout:5/706: write d0/dd/d13/d57/d30/fd0 [868857,123331] 0 2024-07-03T15:44:15.446 INFO:tasks.workunit.client.0.smithi028.stdout:5/707: fdatasync d0/dd/d1e/d3b/de0/fe3 0 2024-07-03T15:44:15.446 INFO:tasks.workunit.client.0.smithi028.stdout:7/539: creat d9/d1c/d45/d71/d78/f9a x:0 0 0 2024-07-03T15:44:15.449 INFO:tasks.workunit.client.0.smithi028.stdout:6/783: dread d4/d12/dc3/ff2 [0,4194304] 0 2024-07-03T15:44:15.449 INFO:tasks.workunit.client.0.smithi028.stdout:6/784: getdents d4/d78/d10c/d38/d3d/d89/dd8/d10d 0 2024-07-03T15:44:15.451 INFO:tasks.workunit.client.0.smithi028.stdout:0/766: creat d2/d3/d4d/d56/d78/da8/ff1 x:0 0 0 2024-07-03T15:44:15.451 INFO:tasks.workunit.client.0.smithi028.stdout:0/767: write d2/d1c/f83 [629272,112558] 0 2024-07-03T15:44:15.457 INFO:tasks.workunit.client.0.smithi028.stdout:5/708: mkdir d0/dd/d13/d57/da2/de9 0 2024-07-03T15:44:15.458 INFO:tasks.workunit.client.0.smithi028.stdout:6/785: dread d4/d78/d10c/d47/f51 [0,4194304] 0 2024-07-03T15:44:15.458 INFO:tasks.workunit.client.0.smithi028.stdout:7/540: truncate d9/f25 1879468 0 2024-07-03T15:44:15.460 INFO:tasks.workunit.client.0.smithi028.stdout:0/768: rename d2/d6d/ld2 to d2/d1c/lf2 0 2024-07-03T15:44:15.462 INFO:tasks.workunit.client.0.smithi028.stdout:1/980: mkdir d4/d3c/d156 0 2024-07-03T15:44:15.462 INFO:tasks.workunit.client.0.smithi028.stdout:1/981: fsync d4/d83/de8/d121/d3a/d5c/f105 0 2024-07-03T15:44:15.463 INFO:tasks.workunit.client.0.smithi028.stdout:5/709: creat d0/dd/d14/fea x:0 0 0 2024-07-03T15:44:15.464 INFO:tasks.workunit.client.0.smithi028.stdout:7/541: symlink d9/l9b 0 2024-07-03T15:44:15.466 INFO:tasks.workunit.client.0.smithi028.stdout:6/786: rename d4/d15/c3c to d4/d78/d10c/d13/dbd/c112 0 2024-07-03T15:44:15.466 INFO:tasks.workunit.client.0.smithi028.stdout:6/787: fdatasync d4/d12/d2c/fe8 0 2024-07-03T15:44:15.467 INFO:tasks.workunit.client.0.smithi028.stdout:0/769: creat d2/d3/d4d/d56/d78/da8/dd0/ff3 x:0 0 0 2024-07-03T15:44:15.469 INFO:tasks.workunit.client.0.smithi028.stdout:1/982: unlink d4/d3c/d54/d79/d16/fb3 0 2024-07-03T15:44:15.474 INFO:tasks.workunit.client.0.smithi028.stdout:7/542: mknod d9/d29/c9c 0 2024-07-03T15:44:15.474 INFO:tasks.workunit.client.0.smithi028.stdout:7/543: dread - d9/d1c/f5a zero size 2024-07-03T15:44:15.474 INFO:tasks.workunit.client.0.smithi028.stdout:7/544: readlink d9/d29/d92/d99/l82 0 2024-07-03T15:44:15.476 INFO:tasks.workunit.client.0.smithi028.stdout:6/788: rename d4/d78/d10c/d38/d59/c99 to d4/d78/d10c/d38/d3d/d89/c113 0 2024-07-03T15:44:15.476 INFO:tasks.workunit.client.0.smithi028.stdout:0/770: dread d2/d3/f2f [0,4194304] 0 2024-07-03T15:44:15.477 INFO:tasks.workunit.client.0.smithi028.stdout:5/710: rmdir d0/dd/d13/d57/d30/d70/d7f/d9e 39 2024-07-03T15:44:15.478 INFO:tasks.workunit.client.0.smithi028.stdout:5/711: write d0/dd/d13/d57/d26/fe7 [8185,25993] 0 2024-07-03T15:44:15.478 INFO:tasks.workunit.client.0.smithi028.stdout:5/712: chown d0/dd/df/f16 296771 1 2024-07-03T15:44:15.478 INFO:tasks.workunit.client.0.smithi028.stdout:5/713: read - d0/dd/d14/fea zero size 2024-07-03T15:44:15.486 INFO:tasks.workunit.client.0.smithi028.stdout:1/983: rename d4/d83/de8/d121/d3a/d5c to d4/d3c/d54/d84/d157 0 2024-07-03T15:44:15.488 INFO:tasks.workunit.client.0.smithi028.stdout:6/789: link d4/d78/d10c/d1c/l108 d4/l114 0 2024-07-03T15:44:15.489 INFO:tasks.workunit.client.0.smithi028.stdout:5/714: rename d0/dd/d13/d57/d77/f7d to d0/d8f/feb 0 2024-07-03T15:44:15.489 INFO:tasks.workunit.client.0.smithi028.stdout:0/771: write d2/d3/dc4/d36/d45/f46 [6140649,55750] 0 2024-07-03T15:44:15.490 INFO:tasks.workunit.client.0.smithi028.stdout:0/772: chown d2/d3/d4d/d56/d78/dbe/lcb 1 1 2024-07-03T15:44:15.490 INFO:tasks.workunit.client.0.smithi028.stdout:0/773: dread - d2/d3/dc4/d58/faf zero size 2024-07-03T15:44:15.490 INFO:tasks.workunit.client.0.smithi028.stdout:0/774: chown d2/d3/c26 164 1 2024-07-03T15:44:15.490 INFO:tasks.workunit.client.0.smithi028.stdout:7/545: dwrite d9/d1c/f5a [0,4194304] 0 2024-07-03T15:44:15.491 INFO:tasks.workunit.client.0.smithi028.stdout:5/715: mkdir d0/dd/d1e/d86/dec 0 2024-07-03T15:44:15.493 INFO:tasks.workunit.client.0.smithi028.stdout:0/775: mknod d2/d1c/d6c/da9/cf4 0 2024-07-03T15:44:15.496 INFO:tasks.workunit.client.0.smithi028.stdout:7/546: truncate f0 7992193 0 2024-07-03T15:44:15.500 INFO:tasks.workunit.client.0.smithi028.stdout:6/790: rmdir d4/d12/d2c/dac 39 2024-07-03T15:44:15.506 INFO:tasks.workunit.client.0.smithi028.stdout:1/984: dwrite d4/d3c/d54/d84/d157/d6c/d7d/d59/dc6/ffd [0,4194304] 0 2024-07-03T15:44:15.506 INFO:tasks.workunit.client.0.smithi028.stdout:1/985: chown d4/d3c/d54/d79/f1b 7 1 2024-07-03T15:44:15.506 INFO:tasks.workunit.client.0.smithi028.stdout:1/986: chown d4/d3c/d54/l12c 3174 1 2024-07-03T15:44:15.506 INFO:tasks.workunit.client.0.smithi028.stdout:1/987: write d4/d3c/d54/d84/f9b [571351,94961] 0 2024-07-03T15:44:15.507 INFO:tasks.workunit.client.0.smithi028.stdout:7/547: rename d9/l4d to d9/d1c/l9d 0 2024-07-03T15:44:15.512 INFO:tasks.workunit.client.0.smithi028.stdout:1/988: dread d4/f37 [0,4194304] 0 2024-07-03T15:44:15.512 INFO:tasks.workunit.client.0.smithi028.stdout:1/989: write d4/de/d1c/d81/d10d/f126 [4704823,52547] 0 2024-07-03T15:44:15.516 INFO:tasks.workunit.client.0.smithi028.stdout:5/716: dwrite d0/dd/d13/d29/f35 [8388608,4194304] 0 2024-07-03T15:44:15.517 INFO:tasks.workunit.client.0.smithi028.stdout:6/791: dwrite d4/f7b [0,4194304] 0 2024-07-03T15:44:15.518 INFO:tasks.workunit.client.0.smithi028.stdout:6/792: write d4/d78/d10c/d38/d59/d91/fd4 [186201,3822] 0 2024-07-03T15:44:15.521 INFO:tasks.workunit.client.0.smithi028.stdout:1/990: creat d4/d3c/d54/d79/d16/d57/d66/d10e/d154/f158 x:0 0 0 2024-07-03T15:44:15.525 INFO:tasks.workunit.client.0.smithi028.stdout:0/776: dwrite d2/d3/dc4/d58/f5f [0,4194304] 0 2024-07-03T15:44:15.531 INFO:tasks.workunit.client.0.smithi028.stdout:7/548: dwrite d9/d29/f35 [4194304,4194304] 0 2024-07-03T15:44:15.532 INFO:tasks.workunit.client.0.smithi028.stdout:7/549: truncate d9/d1c/d45/d71/d78/f9a 1011068 0 2024-07-03T15:44:15.532 INFO:tasks.workunit.client.0.smithi028.stdout:7/550: write d9/d1c/d6e/f76 [58171,49225] 0 2024-07-03T15:44:15.533 INFO:tasks.workunit.client.0.smithi028.stdout:1/991: rmdir d4/de/d1c/d44 39 2024-07-03T15:44:15.533 INFO:tasks.workunit.client.0.smithi028.stdout:1/992: fdatasync d4/d3c/d54/d79/d16/db6/f12d 0 2024-07-03T15:44:15.533 INFO:tasks.workunit.client.0.smithi028.stdout:1/993: write d4/d3c/d54/d84/d157/daa/df0/d123/d125/f131 [438754,110495] 0 2024-07-03T15:44:15.537 INFO:tasks.workunit.client.0.smithi028.stdout:0/777: dread d2/d3/dc4/d36/d45/f72 [0,4194304] 0 2024-07-03T15:44:15.537 INFO:tasks.workunit.client.0.smithi028.stdout:0/778: readlink d2/d3/d4d/d56/d78/d9f/lca 0 2024-07-03T15:44:15.537 INFO:tasks.workunit.client.0.smithi028.stdout:0/779: write d2/d3/d4d/d96/dc2/fea [342947,47402] 0 2024-07-03T15:44:15.538 INFO:tasks.workunit.client.0.smithi028.stdout:0/780: write d2/d1c/f68 [2619607,26986] 0 2024-07-03T15:44:15.538 INFO:tasks.workunit.client.0.smithi028.stdout:0/781: chown d2/d3/dc4/d36/d45/c84 123477 1 2024-07-03T15:44:15.538 INFO:tasks.workunit.client.0.smithi028.stdout:7/551: mkdir d9/d29/d92/d9e 0 2024-07-03T15:44:15.538 INFO:tasks.workunit.client.0.smithi028.stdout:7/552: dread - d9/d1c/f93 zero size 2024-07-03T15:44:15.540 INFO:tasks.workunit.client.0.smithi028.stdout:0/782: dread d2/d3/d4d/d56/d78/d9f/f6f [0,4194304] 0 2024-07-03T15:44:15.540 INFO:tasks.workunit.client.0.smithi028.stdout:0/783: dread - d2/d3/dc4/d58/faf zero size 2024-07-03T15:44:15.540 INFO:tasks.workunit.client.0.smithi028.stdout:0/784: readlink d2/d1c/l48 0 2024-07-03T15:44:15.541 INFO:tasks.workunit.client.0.smithi028.stdout:1/994: link d4/d3c/d54/d79/d11a/c134 d4/d3c/d54/d79/d16/d57/d138/c159 0 2024-07-03T15:44:15.541 INFO:tasks.workunit.client.0.smithi028.stdout:1/995: creat d4/d3c/d54/d79/d16/db6/f15a x:0 0 0 2024-07-03T15:44:15.544 INFO:tasks.workunit.client.0.smithi028.stdout:1/996: mkdir d4/d3c/d54/d84/d157/daa/d15b 0 2024-07-03T15:44:15.545 INFO:tasks.workunit.client.0.smithi028.stdout:1/997: creat d4/d3c/d54/d84/d157/daa/df0/f15c x:0 0 0 2024-07-03T15:44:15.546 INFO:tasks.workunit.client.0.smithi028.stdout:1/998: mkdir d4/d3c/d54/d79/d16/d57/d138/d15d 0 2024-07-03T15:44:15.546 INFO:tasks.workunit.client.0.smithi028.stdout:1/999: fsync d4/de/f6f 0 2024-07-03T15:44:15.547 INFO:tasks.workunit.client.0.smithi028.stdout:5/717: dwrite d0/dd/d13/d57/f21 [0,4194304] 0 2024-07-03T15:44:15.549 INFO:tasks.workunit.client.0.smithi028.stdout:6/793: dwrite d4/d12/d2c/fe0 [0,4194304] 0 2024-07-03T15:44:15.549 INFO:tasks.workunit.client.0.smithi028.stdout:5/718: mknod d0/dd/d13/d57/ced 0 2024-07-03T15:44:15.549 INFO:tasks.workunit.client.0.smithi028.stdout:5/719: write d0/dd/d13/d57/d30/d70/d7f/d9e/fd2 [1443153,9144] 0 2024-07-03T15:44:15.554 INFO:tasks.workunit.client.0.smithi028.stdout:6/794: mknod d4/d78/d10c/d47/d9f/c115 0 2024-07-03T15:44:15.555 INFO:tasks.workunit.client.0.smithi028.stdout:6/795: truncate d4/d78/d10c/fa6 265931 0 2024-07-03T15:44:15.557 INFO:tasks.workunit.client.0.smithi028.stdout:5/720: mkdir d0/dd/d14/dee 0 2024-07-03T15:44:15.557 INFO:tasks.workunit.client.0.smithi028.stdout:5/721: truncate d0/dd/d14/fcb 1300426 0 2024-07-03T15:44:15.557 INFO:tasks.workunit.client.0.smithi028.stdout:5/722: dread - d0/dd/d13/d47/d50/fb9 zero size 2024-07-03T15:44:15.559 INFO:tasks.workunit.client.0.smithi028.stdout:5/723: getdents d0/d8f/db5 0 2024-07-03T15:44:15.568 INFO:tasks.workunit.client.0.smithi028.stdout:7/553: dwrite f6 [0,4194304] 0 2024-07-03T15:44:15.569 INFO:tasks.workunit.client.0.smithi028.stdout:7/554: link d9/c3d d9/d1c/d6e/c9f 0 2024-07-03T15:44:15.572 INFO:tasks.workunit.client.0.smithi028.stdout:7/555: truncate d9/d29/f35 1758774 0 2024-07-03T15:44:15.572 INFO:tasks.workunit.client.0.smithi028.stdout:7/556: write d9/d29/d5d/f79 [10435,46966] 0 2024-07-03T15:44:15.577 INFO:tasks.workunit.client.0.smithi028.stdout:0/785: dwrite d2/d3/dc4/d58/f6a [0,4194304] 0 2024-07-03T15:44:15.584 INFO:tasks.workunit.client.0.smithi028.stdout:6/796: dwrite d4/d15/d9d/f9e [0,4194304] 0 2024-07-03T15:44:15.584 INFO:tasks.workunit.client.0.smithi028.stdout:7/557: dread d9/f1f [0,4194304] 0 2024-07-03T15:44:15.592 INFO:tasks.workunit.client.0.smithi028.stdout:0/786: dwrite d2/d1c/fee [0,4194304] 0 2024-07-03T15:44:15.602 INFO:tasks.workunit.client.0.smithi028.stdout:0/787: read d2/f3b [2324835,45988] 0 2024-07-03T15:44:15.602 INFO:tasks.workunit.client.0.smithi028.stdout:0/788: fdatasync d2/d3/dc4/d36/f61 0 2024-07-03T15:44:15.605 INFO:tasks.workunit.client.0.smithi028.stdout:0/789: getdents d2/d1c/d6c/da9 0 2024-07-03T15:44:15.605 INFO:tasks.workunit.client.0.smithi028.stdout:0/790: write d2/d3/dc4/d58/d82/d9b/fe0 [167169,58971] 0 2024-07-03T15:44:15.605 INFO:tasks.workunit.client.0.smithi028.stdout:0/791: truncate d2/d3/dc4/d36/f88 748082 0 2024-07-03T15:44:15.611 INFO:tasks.workunit.client.0.smithi028.stdout:6/797: dwrite d4/d12/d2c/d31/f64 [0,4194304] 0 2024-07-03T15:44:15.611 INFO:tasks.workunit.client.0.smithi028.stdout:6/798: chown d4/f10 4148095 1 2024-07-03T15:44:15.611 INFO:tasks.workunit.client.0.smithi028.stdout:6/799: chown d4/d78/d10c/d1c/dfd 20691 1 2024-07-03T15:44:15.613 INFO:tasks.workunit.client.0.smithi028.stdout:0/792: write d2/d3/d4d/d56/d78/da8/fb1 [2744813,30622] 0 2024-07-03T15:44:15.613 INFO:tasks.workunit.client.0.smithi028.stdout:7/558: dwrite d9/d29/d92/d99/d6a/f7b [0,4194304] 0 2024-07-03T15:44:15.613 INFO:tasks.workunit.client.0.smithi028.stdout:7/559: stat d9/c5b 0 2024-07-03T15:44:15.613 INFO:tasks.workunit.client.0.smithi028.stdout:7/560: fdatasync d9/d1c/f93 0 2024-07-03T15:44:15.613 INFO:tasks.workunit.client.0.smithi028.stdout:6/800: mknod d4/d12/dc3/c116 0 2024-07-03T15:44:15.616 INFO:tasks.workunit.client.0.smithi028.stdout:7/561: mkdir d9/d1c/d7c/d8b/da0 0 2024-07-03T15:44:15.616 INFO:tasks.workunit.client.0.smithi028.stdout:7/562: getdents d9/d29/d92/d9e 0 2024-07-03T15:44:15.617 INFO:tasks.workunit.client.0.smithi028.stdout:6/801: symlink d4/d78/d10c/d13/dbd/l117 0 2024-07-03T15:44:15.617 INFO:tasks.workunit.client.0.smithi028.stdout:6/802: fdatasync d4/d78/d10c/d38/ff8 0 2024-07-03T15:44:15.617 INFO:tasks.workunit.client.0.smithi028.stdout:6/803: fsync d4/d78/d10c/d38/fa7 0 2024-07-03T15:44:15.617 INFO:tasks.workunit.client.0.smithi028.stdout:6/804: write d4/d78/d10c/d38/d60/fab [1047104,11886] 0 2024-07-03T15:44:15.618 INFO:tasks.workunit.client.0.smithi028.stdout:6/805: truncate d4/d78/d10c/d38/ff8 39740 0 2024-07-03T15:44:15.620 INFO:tasks.workunit.client.0.smithi028.stdout:7/563: rename d9/d29/f3c to d9/d29/d92/d9e/fa1 0 2024-07-03T15:44:15.620 INFO:tasks.workunit.client.0.smithi028.stdout:7/564: truncate d9/d29/f61 1914943 0 2024-07-03T15:44:15.621 INFO:tasks.workunit.client.0.smithi028.stdout:6/806: creat d4/d78/d10c/f118 x:0 0 0 2024-07-03T15:44:15.622 INFO:tasks.workunit.client.0.smithi028.stdout:7/565: symlink d9/d1c/d6e/la2 0 2024-07-03T15:44:15.622 INFO:tasks.workunit.client.0.smithi028.stdout:7/566: creat d9/fa3 x:0 0 0 2024-07-03T15:44:15.622 INFO:tasks.workunit.client.0.smithi028.stdout:7/567: write d9/d1c/f20 [1306024,23429] 0 2024-07-03T15:44:15.622 INFO:tasks.workunit.client.0.smithi028.stdout:6/807: dread d4/d78/d10c/d1c/f88 [0,4194304] 0 2024-07-03T15:44:15.624 INFO:tasks.workunit.client.0.smithi028.stdout:7/568: creat d9/d29/fa4 x:0 0 0 2024-07-03T15:44:15.625 INFO:tasks.workunit.client.0.smithi028.stdout:7/569: write d9/d1c/d45/d71/d78/f9a [451299,61818] 0 2024-07-03T15:44:15.626 INFO:tasks.workunit.client.0.smithi028.stdout:6/808: rename d4/d78/d10c/d38/d3d/d89/dd8 to d4/d12/d2c/df4/d119 0 2024-07-03T15:44:15.629 INFO:tasks.workunit.client.0.smithi028.stdout:7/570: getdents d9/d29/d92 0 2024-07-03T15:44:15.629 INFO:tasks.workunit.client.0.smithi028.stdout:7/571: readlink d9/d1c/d45/l91 0 2024-07-03T15:44:15.632 INFO:tasks.workunit.client.0.smithi028.stdout:7/572: mknod d9/d1c/d45/d71/ca5 0 2024-07-03T15:44:15.634 INFO:tasks.workunit.client.0.smithi028.stdout:6/809: dread d4/d78/d10c/d38/d60/dc5/dcf/fe2 [0,4194304] 0 2024-07-03T15:44:15.634 INFO:tasks.workunit.client.0.smithi028.stdout:6/810: write d4/d12/fae [301433,45134] 0 2024-07-03T15:44:15.635 INFO:tasks.workunit.client.0.smithi028.stdout:6/811: link d4/d78/d10c/d38/d60/db5/fd0 d4/d12/d2c/df4/f11a 0 2024-07-03T15:44:15.643 INFO:tasks.workunit.client.0.smithi028.stdout:7/573: dread f0 [0,4194304] 0 2024-07-03T15:44:15.646 INFO:tasks.workunit.client.0.smithi028.stdout:7/574: unlink d9/d1c/f21 0 2024-07-03T15:44:15.648 INFO:tasks.workunit.client.0.smithi028.stdout:7/575: dread d9/d1c/d45/d71/d78/f9a [0,4194304] 0 2024-07-03T15:44:15.649 INFO:tasks.workunit.client.0.smithi028.stdout:7/576: mknod d9/d1c/d6e/ca6 0 2024-07-03T15:44:15.661 INFO:tasks.workunit.client.0.smithi028.stdout:7/577: dwrite d9/d29/f42 [0,4194304] 0 2024-07-03T15:44:15.701 INFO:tasks.workunit.client.0.smithi028.stdout:7/578: symlink d9/d29/d92/d9e/la7 0 2024-07-03T15:44:15.701 INFO:tasks.workunit.client.0.smithi028.stdout:7/579: write d9/f84 [166308,113982] 0 2024-07-03T15:44:15.709 INFO:tasks.workunit.client.0.smithi028.stdout:7/580: write d9/d29/f44 [1428609,40955] 0 2024-07-03T15:44:15.714 INFO:tasks.workunit.client.0.smithi028.stdout:7/581: mknod d9/d1c/d6e/ca8 0 2024-07-03T15:44:15.715 INFO:tasks.workunit.client.0.smithi028.stdout:7/582: truncate d9/d29/f81 1460841 0 2024-07-03T15:44:15.715 INFO:tasks.workunit.client.0.smithi028.stdout:7/583: write d9/d1c/d6e/f73 [1935671,79923] 0 2024-07-03T15:44:15.715 INFO:tasks.workunit.client.0.smithi028.stdout:7/584: write d9/d29/d92/d99/d6a/f85 [319732,92833] 0 2024-07-03T15:44:15.716 INFO:tasks.workunit.client.0.smithi028.stdout:7/585: mkdir d9/d1c/da9 0 2024-07-03T15:44:15.722 INFO:tasks.workunit.client.0.smithi028.stdout:7/586: rename d9/c56 to d9/d1c/caa 0 2024-07-03T15:44:15.743 INFO:tasks.workunit.client.0.smithi028.stdout:7/587: dwrite d9/f25 [0,4194304] 0 2024-07-03T15:44:15.763 INFO:tasks.workunit.client.0.smithi028.stdout:7/588: dwrite d9/d29/d5d/f79 [0,4194304] 0 2024-07-03T15:44:15.768 INFO:tasks.workunit.client.0.smithi028.stdout:7/589: unlink d9/d29/f61 0 2024-07-03T15:44:15.808 INFO:tasks.workunit.client.0.smithi028.stdout:7/590: mknod d9/d1c/da9/cab 0 2024-07-03T15:44:15.808 INFO:tasks.workunit.client.0.smithi028.stdout:7/591: creat d9/d1c/d45/d71/fac x:0 0 0 2024-07-03T15:44:15.808 INFO:tasks.workunit.client.0.smithi028.stdout:7/592: read d9/f41 [1807260,11329] 0 2024-07-03T15:44:15.814 INFO:tasks.workunit.client.0.smithi028.stdout:7/593: rmdir d9/d1c/d6e 39 2024-07-03T15:44:15.817 INFO:tasks.workunit.client.0.smithi028.stdout:7/594: rename d9/d29/f35 to d9/d29/d92/d99/d6a/fad 0 2024-07-03T15:44:15.819 INFO:tasks.workunit.client.0.smithi028.stdout:7/595: creat d9/d1c/d45/fae x:0 0 0 2024-07-03T15:44:15.820 INFO:tasks.workunit.client.0.smithi028.stdout:7/596: mkdir d9/d29/d92/d99/daf 0 2024-07-03T15:44:15.820 INFO:tasks.workunit.client.0.smithi028.stdout:7/597: chown d9/d1c/d45/f46 122773331 1 2024-07-03T15:44:15.843 INFO:tasks.workunit.client.0.smithi028.stdout:7/598: dwrite d9/d1c/d6e/f76 [0,4194304] 0 2024-07-03T15:44:15.845 INFO:tasks.workunit.client.0.smithi028.stdout:7/599: rmdir d9/d29/d92/d99/d6a 39 2024-07-03T15:44:15.845 INFO:tasks.workunit.client.0.smithi028.stdout:7/600: creat d9/d29/fb0 x:0 0 0 2024-07-03T15:44:15.845 INFO:tasks.workunit.client.0.smithi028.stdout:7/601: write d9/d1c/f3f [2139667,75282] 0 2024-07-03T15:44:15.846 INFO:tasks.workunit.client.0.smithi028.stdout:7/602: creat d9/d1c/d45/d71/fb1 x:0 0 0 2024-07-03T15:44:15.855 INFO:tasks.workunit.client.0.smithi028.stdout:7/603: write d9/d29/d92/d99/d6a/f7b [1479880,97830] 0 2024-07-03T15:44:15.857 INFO:tasks.workunit.client.0.smithi028.stdout:7/604: rename d9/c51 to d9/d29/cb2 0 2024-07-03T15:44:15.857 INFO:tasks.workunit.client.0.smithi028.stdout:7/605: readlink d9/d29/d92/d99/l82 0 2024-07-03T15:44:15.857 INFO:tasks.workunit.client.0.smithi028.stdout:7/606: mknod d9/d1c/d86/cb3 0 2024-07-03T15:44:15.858 INFO:tasks.workunit.client.0.smithi028.stdout:7/607: write d9/d1c/d45/fae [391971,28579] 0 2024-07-03T15:44:15.861 INFO:tasks.workunit.client.0.smithi028.stdout:7/608: dread d9/d29/d92/d99/f58 [0,4194304] 0 2024-07-03T15:44:15.863 INFO:tasks.workunit.client.0.smithi028.stdout:7/609: truncate f6 2067415 0 2024-07-03T15:44:15.866 INFO:tasks.workunit.client.0.smithi028.stdout:7/610: creat d9/d1c/d7c/d8b/da0/fb4 x:0 0 0 2024-07-03T15:44:15.866 INFO:tasks.workunit.client.0.smithi028.stdout:7/611: dread - d9/d29/fa4 zero size 2024-07-03T15:44:15.868 INFO:tasks.workunit.client.0.smithi028.stdout:7/612: rmdir d9/d1c/d6e 39 2024-07-03T15:44:15.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:15 smithi028 ceph-mon[29148]: pgmap v9: 65 pgs: 65 active+clean; 2.9 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 131 MiB/s rd, 170 MiB/s wr, 334 op/s 2024-07-03T15:44:15.901 INFO:tasks.workunit.client.0.smithi028.stdout:2/586: sync 2024-07-03T15:44:15.902 INFO:tasks.workunit.client.0.smithi028.stdout:2/587: chown da/d23/d30/l44 0 1 2024-07-03T15:44:15.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:15 smithi148 ceph-mon[36056]: pgmap v9: 65 pgs: 65 active+clean; 2.9 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 131 MiB/s rd, 170 MiB/s wr, 334 op/s 2024-07-03T15:44:16.059 INFO:tasks.workunit.client.0.smithi028.stdout:9/743: sync 2024-07-03T15:44:16.059 INFO:tasks.workunit.client.0.smithi028.stdout:4/737: sync 2024-07-03T15:44:16.059 INFO:tasks.workunit.client.0.smithi028.stdout:4/738: write d2/d12/d2e/f6b [4887856,124701] 0 2024-07-03T15:44:16.059 INFO:tasks.workunit.client.0.smithi028.stdout:8/818: sync 2024-07-03T15:44:16.059 INFO:tasks.workunit.client.0.smithi028.stdout:8/819: creat d0/d41/d50/d3/d35/d68/d22/d40/d62/f10b x:0 0 0 2024-07-03T15:44:16.063 INFO:tasks.workunit.client.0.smithi028.stdout:9/744: symlink d1/d16/d57/d5f/d7f/lfa 0 2024-07-03T15:44:16.063 INFO:tasks.workunit.client.0.smithi028.stdout:9/745: fdatasync d1/d10/d28/d45/fdd 0 2024-07-03T15:44:16.065 INFO:tasks.workunit.client.0.smithi028.stdout:8/820: mkdir d0/d41/d50/d3/d35/d68/d22/d40/db7/d10c 0 2024-07-03T15:44:16.071 INFO:tasks.workunit.client.0.smithi028.stdout:9/746: creat d1/d10/d28/d45/d61/d8c/ffb x:0 0 0 2024-07-03T15:44:16.073 INFO:tasks.workunit.client.0.smithi028.stdout:9/747: symlink d1/d10/d28/d4e/d5b/de9/lfc 0 2024-07-03T15:44:16.096 INFO:tasks.workunit.client.0.smithi028.stdout:4/739: dwrite d2/d61/fc5 [0,4194304] 0 2024-07-03T15:44:16.096 INFO:tasks.workunit.client.0.smithi028.stdout:4/740: chown d2/d61/d9d/faf 2 1 2024-07-03T15:44:16.096 INFO:tasks.workunit.client.0.smithi028.stdout:4/741: write d2/d12/d5b/d78/de4/d3a/ff2 [582778,126610] 0 2024-07-03T15:44:16.099 INFO:tasks.workunit.client.0.smithi028.stdout:8/821: dwrite d0/d41/d50/d3/f3a [0,4194304] 0 2024-07-03T15:44:16.104 INFO:tasks.workunit.client.0.smithi028.stdout:8/822: mknod d0/d41/d50/d3/d35/d86/c10d 0 2024-07-03T15:44:16.107 INFO:tasks.workunit.client.0.smithi028.stdout:8/823: link d0/d41/d50/d3/d35/d68/d22/d40/d4a/cfe d0/d41/d50/d3/dc/d3f/df3/c10e 0 2024-07-03T15:44:16.108 INFO:tasks.workunit.client.0.smithi028.stdout:8/824: readlink d0/d41/d50/d3/dc/d3f/d8e/d81/l8f 0 2024-07-03T15:44:16.108 INFO:tasks.workunit.client.0.smithi028.stdout:8/825: readlink d0/d41/d50/l51 0 2024-07-03T15:44:16.111 INFO:tasks.workunit.client.0.smithi028.stdout:8/826: rename d0/de0/dcf/ce7 to d0/d41/d50/d3/d35/d68/d22/d40/d4a/de6/c10f 0 2024-07-03T15:44:16.111 INFO:tasks.workunit.client.0.smithi028.stdout:8/827: chown d0/d41/d50/d3/l3d 7 1 2024-07-03T15:44:16.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.107+0000 7ff96a127640 1 -- 172.21.15.28:0/2533002854 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff9640fc330 msgr2=0x7ff9640fc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:16.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.107+0000 7ff96a127640 1 --2- 172.21.15.28:0/2533002854 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff9640fc330 0x7ff9640fc740 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff9540048d0 tx=0x7ff954009da0 comp rx=0 tx=0).stop 2024-07-03T15:44:16.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.108+0000 7ff96a127640 1 -- 172.21.15.28:0/2533002854 shutdown_connections 2024-07-03T15:44:16.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.108+0000 7ff96a127640 1 --2- 172.21.15.28:0/2533002854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9640fd600 0x7ff9640fda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:16.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.108+0000 7ff96a127640 1 --2- 172.21.15.28:0/2533002854 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff9640fc330 0x7ff9640fc740 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:16.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.108+0000 7ff96a127640 1 -- 172.21.15.28:0/2533002854 >> 172.21.15.28:0/2533002854 conn(0x7ff9640f7af0 msgr2=0x7ff9640f9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:44:16.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.108+0000 7ff96a127640 1 -- 172.21.15.28:0/2533002854 shutdown_connections 2024-07-03T15:44:16.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.108+0000 7ff96a127640 1 -- 172.21.15.28:0/2533002854 wait complete. 2024-07-03T15:44:16.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.109+0000 7ff96a127640 1 Processor -- start 2024-07-03T15:44:16.113 INFO:tasks.workunit.client.0.smithi028.stdout:8/828: unlink d0/d41/d50/daf/lee 0 2024-07-03T15:44:16.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.109+0000 7ff96a127640 1 -- start start 2024-07-03T15:44:16.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.109+0000 7ff96a127640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff9640fc330 0x7ff96410cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:44:16.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.109+0000 7ff96a127640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9640fd600 0x7ff96410d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:44:16.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.109+0000 7ff96a127640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96410daf0 con 0x7ff9640fc330 2024-07-03T15:44:16.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.109+0000 7ff96a127640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96410e310 con 0x7ff9640fd600 2024-07-03T15:44:16.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.110+0000 7ff9637fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff9640fc330 0x7ff96410cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:44:16.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.110+0000 7ff9637fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff9640fc330 0x7ff96410cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50426/0 (socket says 172.21.15.28:50426) 2024-07-03T15:44:16.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.110+0000 7ff9637fe640 1 -- 172.21.15.28:0/4061346093 learned_addr learned my addr 172.21.15.28:0/4061346093 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:44:16.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.110+0000 7ff962ffd640 1 --2- 172.21.15.28:0/4061346093 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9640fd600 0x7ff96410d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:44:16.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.110+0000 7ff962ffd640 1 -- 172.21.15.28:0/4061346093 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff9640fc330 msgr2=0x7ff96410cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:16.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.110+0000 7ff962ffd640 1 --2- 172.21.15.28:0/4061346093 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff9640fc330 0x7ff96410cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:16.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.110+0000 7ff962ffd640 1 -- 172.21.15.28:0/4061346093 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff954005040 con 0x7ff9640fd600 2024-07-03T15:44:16.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.110+0000 7ff9637fe640 1 --2- 172.21.15.28:0/4061346093 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff9640fc330 0x7ff96410cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-03T15:44:16.115 INFO:tasks.workunit.client.0.smithi028.stdout:8/829: mknod d0/d41/d50/d3/d35/d68/d22/d40/dd2/c110 0 2024-07-03T15:44:16.115 INFO:tasks.workunit.client.0.smithi028.stdout:8/830: fsync d0/d41/d50/d3/dc/d3f/f61 0 2024-07-03T15:44:16.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.110+0000 7ff962ffd640 1 --2- 172.21.15.28:0/4061346093 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9640fd600 0x7ff96410d4a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff95800eb40 tx=0x7ff95800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:44:16.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.111+0000 7ff94bfff640 1 -- 172.21.15.28:0/4061346093 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff958007c30 con 0x7ff9640fd600 2024-07-03T15:44:16.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.111+0000 7ff94bfff640 1 -- 172.21.15.28:0/4061346093 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff95800f040 con 0x7ff9640fd600 2024-07-03T15:44:16.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.111+0000 7ff94bfff640 1 -- 172.21.15.28:0/4061346093 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9580095d0 con 0x7ff9640fd600 2024-07-03T15:44:16.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.111+0000 7ff96a127640 1 -- 172.21.15.28:0/4061346093 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff96410e540 con 0x7ff9640fd600 2024-07-03T15:44:16.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.111+0000 7ff96a127640 1 -- 172.21.15.28:0/4061346093 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff96410ea60 con 0x7ff9640fd600 2024-07-03T15:44:16.117 INFO:tasks.workunit.client.0.smithi028.stdout:8/831: rename d0/d41/d50/d3/dc/c6e to d0/d41/d50/d3/d71/c111 0 2024-07-03T15:44:16.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.112+0000 7ff96a127640 1 -- 172.21.15.28:0/4061346093 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9640fb790 con 0x7ff9640fd600 2024-07-03T15:44:16.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.112+0000 7ff94bfff640 1 -- 172.21.15.28:0/4061346093 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 24) v1 ==== 88617+0+0 (secure 0 0 0) 0x7ff958009730 con 0x7ff9640fd600 2024-07-03T15:44:16.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.113+0000 7ff94bfff640 1 --2- 172.21.15.28:0/4061346093 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7ff94c06a6d0 0x7ff94c06cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:44:16.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.113+0000 7ff94bfff640 1 -- 172.21.15.28:0/4061346093 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7ff958088d30 con 0x7ff9640fd600 2024-07-03T15:44:16.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.113+0000 7ff9637fe640 1 --2- 172.21.15.28:0/4061346093 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7ff94c06a6d0 0x7ff94c06cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:44:16.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.114+0000 7ff9637fe640 1 --2- 172.21.15.28:0/4061346093 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7ff94c06a6d0 0x7ff94c06cb80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff954000c00 tx=0x7ff954035040 comp rx=0 tx=0).ready entity=mgr.24403 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:44:16.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.118+0000 7ff94bfff640 1 -- 172.21.15.28:0/4061346093 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7ff958052910 con 0x7ff9640fd600 2024-07-03T15:44:16.124 INFO:tasks.workunit.client.0.smithi028.stdout:8/832: dread d0/d41/d50/d3/dc/d33/d7a/f56 [0,4194304] 0 2024-07-03T15:44:16.127 INFO:tasks.workunit.client.0.smithi028.stdout:8/833: rmdir d0/d41/d50/d3/d35/d68/d22/d40/db7/d10c 0 2024-07-03T15:44:16.130 INFO:tasks.workunit.client.0.smithi028.stdout:8/834: unlink d0/d41/d50/d3/d35/d68/d22/d40/dd2/c110 0 2024-07-03T15:44:16.134 INFO:tasks.workunit.client.0.smithi028.stdout:8/835: rename d0/d41/d50/d3/d35/d68/d22/d40/db7/f10a to d0/d6c/da9/f112 0 2024-07-03T15:44:16.137 INFO:tasks.workunit.client.0.smithi028.stdout:8/836: symlink d0/d41/d50/d3/dc/d3f/d8e/d81/dfc/l113 0 2024-07-03T15:44:16.137 INFO:tasks.workunit.client.0.smithi028.stdout:8/837: fsync d0/d41/d50/d3/d35/d68/d22/d40/db7/f106 0 2024-07-03T15:44:16.137 INFO:tasks.workunit.client.0.smithi028.stdout:8/838: readlink d0/de0/ld9 0 2024-07-03T15:44:16.138 INFO:tasks.workunit.client.0.smithi028.stdout:8/839: mknod d0/d41/d50/d3/dc/d3f/d8e/c114 0 2024-07-03T15:44:16.154 INFO:tasks.workunit.client.0.smithi028.stdout:8/840: dwrite d0/d41/d50/daf/dc8/da6/da7/fda [0,4194304] 0 2024-07-03T15:44:16.157 INFO:tasks.workunit.client.0.smithi028.stdout:8/841: rmdir d0/d41/d50/d3/d35/d68/d22/d40/d4a/de6 39 2024-07-03T15:44:16.159 INFO:tasks.workunit.client.0.smithi028.stdout:8/842: creat d0/d41/d50/d3/d35/d68/d22/f115 x:0 0 0 2024-07-03T15:44:16.159 INFO:tasks.workunit.client.0.smithi028.stdout:8/843: fdatasync d0/d41/d50/d3/dc/d3f/d8e/d5d/fde 0 2024-07-03T15:44:16.179 INFO:tasks.workunit.client.0.smithi028.stdout:8/844: dwrite d0/f2a [0,4194304] 0 2024-07-03T15:44:16.182 INFO:tasks.workunit.client.0.smithi028.stdout:8/845: symlink d0/d41/d50/d3/dc/db3/l116 0 2024-07-03T15:44:16.182 INFO:tasks.workunit.client.0.smithi028.stdout:8/846: fsync d0/d41/d50/d3/dc/d3f/d8e/d5d/ff9 0 2024-07-03T15:44:16.203 INFO:tasks.workunit.client.0.smithi028.stdout:8/847: dwrite d0/d41/d50/d3/dc/ff7 [0,4194304] 0 2024-07-03T15:44:16.273 INFO:tasks.workunit.client.0.smithi028.stdout:3/696: sync 2024-07-03T15:44:16.273 INFO:tasks.workunit.client.0.smithi028.stdout:5/724: sync 2024-07-03T15:44:16.275 INFO:tasks.workunit.client.0.smithi028.stdout:5/725: unlink d0/dd/d1e/d3b/d3f/d71/cb2 0 2024-07-03T15:44:16.275 INFO:tasks.workunit.client.0.smithi028.stdout:5/726: fsync d0/dd/d14/dc0/fd3 0 2024-07-03T15:44:16.278 INFO:tasks.workunit.client.0.smithi028.stdout:5/727: truncate d0/dd/d13/d57/d26/d41/db0/fba 2315973 0 2024-07-03T15:44:16.279 INFO:tasks.workunit.client.0.smithi028.stdout:5/728: symlink d0/dd/d13/d57/d30/d70/d7f/lef 0 2024-07-03T15:44:16.279 INFO:tasks.workunit.client.0.smithi028.stdout:5/729: getdents d0/dd/d13/d57/d30/d70/d7f/d9e/dd6 0 2024-07-03T15:44:16.283 INFO:tasks.workunit.client.0.smithi028.stdout:3/697: write d18/d1a/d1e/f41 [2756113,80892] 0 2024-07-03T15:44:16.288 INFO:tasks.workunit.client.0.smithi028.stdout:3/698: link d18/d6f/d78/cb0 d18/d1a/d5e/d66/ce1 0 2024-07-03T15:44:16.293 INFO:tasks.workunit.client.0.smithi028.stdout:3/699: mkdir d18/d1a/d1e/d58/d9f/de2 0 2024-07-03T15:44:16.298 INFO:tasks.workunit.client.0.smithi028.stdout:5/730: dwrite d0/d8f/feb [0,4194304] 0 2024-07-03T15:44:16.300 INFO:tasks.workunit.client.0.smithi028.stdout:5/731: unlink d0/dd/d13/d47/dbc/fc7 0 2024-07-03T15:44:16.312 INFO:tasks.workunit.client.0.smithi028.stdout:3/700: dwrite d18/d1a/d1e/f3e [0,4194304] 0 2024-07-03T15:44:16.317 INFO:tasks.workunit.client.0.smithi028.stdout:3/701: dread d18/d3a/f5a [0,4194304] 0 2024-07-03T15:44:16.317 INFO:tasks.workunit.client.0.smithi028.stdout:3/702: chown d18/d3a/d9a/fa0 27840 1 2024-07-03T15:44:16.320 INFO:tasks.workunit.client.0.smithi028.stdout:5/732: dwrite d0/dd/d1e/d3b/d3f/fd8 [0,4194304] 0 2024-07-03T15:44:16.321 INFO:tasks.workunit.client.0.smithi028.stdout:3/703: rename d18/f51 to d18/d3a/d9d/d8c/fe3 0 2024-07-03T15:44:16.327 INFO:tasks.workunit.client.0.smithi028.stdout:5/733: mkdir d0/dd/d1e/d3b/ddd/df0 0 2024-07-03T15:44:16.327 INFO:tasks.workunit.client.0.smithi028.stdout:3/704: getdents d18/d1a 0 2024-07-03T15:44:16.330 INFO:tasks.workunit.client.0.smithi028.stdout:3/705: rename d18/d1a/d5e/d66/da3/fc4 to d18/d1a/d1e/d58/d9f/da9/fe4 0 2024-07-03T15:44:16.333 INFO:tasks.workunit.client.0.smithi028.stdout:3/706: getdents d18/d6f 0 2024-07-03T15:44:16.333 INFO:tasks.workunit.client.0.smithi028.stdout:5/734: read d0/d8f/fc9 [487188,44269] 0 2024-07-03T15:44:16.336 INFO:tasks.workunit.client.0.smithi028.stdout:5/735: rename d0/dd/df/f3e to d0/dd/d13/d57/d26/ddf/ff1 0 2024-07-03T15:44:16.336 INFO:tasks.workunit.client.0.smithi028.stdout:5/736: dread - d0/dd/d13/d29/dc4/fa9 zero size 2024-07-03T15:44:16.348 INFO:tasks.workunit.client.0.smithi028.stdout:3/707: dwrite d18/d1a/d64/f8f [0,4194304] 0 2024-07-03T15:44:16.348 INFO:tasks.workunit.client.0.smithi028.stdout:3/708: dread - d18/d3a/d9d/d8c/fdb zero size 2024-07-03T15:44:16.363 INFO:tasks.workunit.client.0.smithi028.stdout:5/737: dwrite d0/dd/d1e/d3b/fa3 [0,4194304] 0 2024-07-03T15:44:16.363 INFO:tasks.workunit.client.0.smithi028.stdout:5/738: creat d0/dd/df/da7/ff2 x:0 0 0 2024-07-03T15:44:16.366 INFO:tasks.workunit.client.0.smithi028.stdout:3/709: dwrite d18/d1a/d1e/f38 [0,4194304] 0 2024-07-03T15:44:16.368 INFO:tasks.workunit.client.0.smithi028.stdout:2/588: sync 2024-07-03T15:44:16.369 INFO:tasks.workunit.client.0.smithi028.stdout:7/613: sync 2024-07-03T15:44:16.369 INFO:tasks.workunit.client.0.smithi028.stdout:0/793: sync 2024-07-03T15:44:16.369 INFO:tasks.workunit.client.0.smithi028.stdout:6/812: sync 2024-07-03T15:44:16.370 INFO:tasks.workunit.client.0.smithi028.stdout:0/794: readlink d2/d3/d4d/d56/d78/d9f/lca 0 2024-07-03T15:44:16.370 INFO:tasks.workunit.client.0.smithi028.stdout:3/710: dread d18/d1a/d64/f6e [0,4194304] 0 2024-07-03T15:44:16.370 INFO:tasks.workunit.client.0.smithi028.stdout:3/711: write d18/d3a/d9d/d8c/faf [510254,103635] 0 2024-07-03T15:44:16.370 INFO:tasks.workunit.client.0.smithi028.stdout:3/712: stat d18/l3b 0 2024-07-03T15:44:16.370 INFO:tasks.workunit.client.0.smithi028.stdout:2/589: truncate da/d24/f8c 209581 0 2024-07-03T15:44:16.377 INFO:tasks.workunit.client.0.smithi028.stdout:7/614: rename d9/d29 to d9/d1c/d45/db5 0 2024-07-03T15:44:16.377 INFO:tasks.workunit.client.0.smithi028.stdout:7/615: chown d9/d1c/d45/db5/d92 719695 1 2024-07-03T15:44:16.377 INFO:tasks.workunit.client.0.smithi028.stdout:7/616: chown l8 7302 1 2024-07-03T15:44:16.381 INFO:tasks.workunit.client.0.smithi028.stdout:6/813: rename d4/d78/d10c/fa6 to d4/d78/df0/f11b 0 2024-07-03T15:44:16.381 INFO:tasks.workunit.client.0.smithi028.stdout:6/814: fsync d4/d15/f8c 0 2024-07-03T15:44:16.384 INFO:tasks.workunit.client.0.smithi028.stdout:2/590: dread da/d74/f8d [0,4194304] 0 2024-07-03T15:44:16.384 INFO:tasks.workunit.client.0.smithi028.stdout:2/591: creat da/d23/d30/fcc x:0 0 0 2024-07-03T15:44:16.387 INFO:tasks.workunit.client.0.smithi028.stdout:5/739: dwrite d0/dd/d13/d57/d30/f90 [0,4194304] 0 2024-07-03T15:44:16.387 INFO:tasks.workunit.client.0.smithi028.stdout:5/740: chown d0/dd/d1e/l9a 28511 1 2024-07-03T15:44:16.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.399+0000 7ff96a127640 1 -- 172.21.15.28:0/4061346093 --> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff96410ee40 con 0x7ff94c06a6d0 2024-07-03T15:44:16.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.400+0000 7ff94bfff640 1 -- 172.21.15.28:0/4061346093 <== mgr.24403 v2:172.21.15.148:6828/4285739175 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7ff96410ee40 con 0x7ff94c06a6d0 2024-07-03T15:44:16.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.402+0000 7ff96a127640 1 -- 172.21.15.28:0/4061346093 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7ff94c06a6d0 msgr2=0x7ff94c06cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:16.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.402+0000 7ff96a127640 1 --2- 172.21.15.28:0/4061346093 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7ff94c06a6d0 0x7ff94c06cb80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff954000c00 tx=0x7ff954035040 comp rx=0 tx=0).stop 2024-07-03T15:44:16.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.402+0000 7ff96a127640 1 -- 172.21.15.28:0/4061346093 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9640fd600 msgr2=0x7ff96410d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:16.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.402+0000 7ff96a127640 1 --2- 172.21.15.28:0/4061346093 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9640fd600 0x7ff96410d4a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff95800eb40 tx=0x7ff95800b040 comp rx=0 tx=0).stop 2024-07-03T15:44:16.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.403+0000 7ff96a127640 1 -- 172.21.15.28:0/4061346093 shutdown_connections 2024-07-03T15:44:16.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.403+0000 7ff96a127640 1 --2- 172.21.15.28:0/4061346093 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9640fd600 0x7ff96410d4a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:16.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.403+0000 7ff96a127640 1 --2- 172.21.15.28:0/4061346093 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7ff94c06a6d0 0x7ff94c06cb80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:16.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.403+0000 7ff96a127640 1 --2- 172.21.15.28:0/4061346093 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff9640fc330 0x7ff96410cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:16.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.403+0000 7ff96a127640 1 -- 172.21.15.28:0/4061346093 >> 172.21.15.28:0/4061346093 conn(0x7ff9640f7af0 msgr2=0x7ff9640f91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:44:16.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.403+0000 7ff96a127640 1 -- 172.21.15.28:0/4061346093 shutdown_connections 2024-07-03T15:44:16.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.403+0000 7ff96a127640 1 -- 172.21.15.28:0/4061346093 wait complete. 2024-07-03T15:44:16.416 INFO:teuthology.orchestra.run.smithi028.stdout:true 2024-07-03T15:44:16.422 INFO:tasks.workunit.client.0.smithi028.stdout:3/713: dwrite d18/d6f/d78/fbc [0,4194304] 0 2024-07-03T15:44:16.422 INFO:tasks.workunit.client.0.smithi028.stdout:0/795: dwrite d2/d1c/f20 [4194304,4194304] 0 2024-07-03T15:44:16.423 INFO:tasks.workunit.client.0.smithi028.stdout:6/815: dwrite d4/d78/d10c/d38/d60/dc5/ffa [0,4194304] 0 2024-07-03T15:44:16.423 INFO:tasks.workunit.client.0.smithi028.stdout:6/816: stat d4/d78/d10c/d38/d60/dc5/fce 0 2024-07-03T15:44:16.423 INFO:tasks.workunit.client.0.smithi028.stdout:5/741: dwrite d0/dd/d13/d57/d30/d70/d7f/fbd [0,4194304] 0 2024-07-03T15:44:16.428 INFO:tasks.workunit.client.0.smithi028.stdout:3/714: rename d18/l4e to d18/d6f/d78/d87/le5 0 2024-07-03T15:44:16.428 INFO:tasks.workunit.client.0.smithi028.stdout:3/715: creat d18/d3a/d9d/d8c/fe6 x:0 0 0 2024-07-03T15:44:16.432 INFO:tasks.workunit.client.0.smithi028.stdout:0/796: dread d2/d3/d4d/d56/f7f [0,4194304] 0 2024-07-03T15:44:16.432 INFO:tasks.workunit.client.0.smithi028.stdout:3/716: dread d18/d3a/f70 [0,4194304] 0 2024-07-03T15:44:16.437 INFO:tasks.workunit.client.0.smithi028.stdout:5/742: dwrite d0/dd/d13/f83 [0,4194304] 0 2024-07-03T15:44:16.442 INFO:tasks.workunit.client.0.smithi028.stdout:6/817: rename d4/d78/d10c/d38/d59/c7a to d4/d78/d10c/d38/d60/dc5/c11c 0 2024-07-03T15:44:16.444 INFO:tasks.workunit.client.0.smithi028.stdout:0/797: rmdir d2/d3/dc4/d36 39 2024-07-03T15:44:16.448 INFO:tasks.workunit.client.0.smithi028.stdout:3/717: truncate d18/fb3 3668909 0 2024-07-03T15:44:16.448 INFO:tasks.workunit.client.0.smithi028.stdout:3/718: readlink d18/d1a/d1e/d58/d9f/da9/lc3 0 2024-07-03T15:44:16.451 INFO:tasks.workunit.client.0.smithi028.stdout:5/743: rename d0/dd/df/da7/fc8 to d0/dd/d13/d57/da2/de9/ff3 0 2024-07-03T15:44:16.451 INFO:tasks.workunit.client.0.smithi028.stdout:5/744: write d0/dd/d13/d57/d30/fd0 [1713337,105682] 0 2024-07-03T15:44:16.455 INFO:tasks.workunit.client.0.smithi028.stdout:0/798: mkdir d2/d3/dc4/d58/d6e/df5 0 2024-07-03T15:44:16.455 INFO:tasks.workunit.client.0.smithi028.stdout:0/799: fdatasync d2/f15 0 2024-07-03T15:44:16.457 INFO:tasks.workunit.client.0.smithi028.stdout:3/719: creat d18/d3a/d9d/d8c/db4/fe7 x:0 0 0 2024-07-03T15:44:16.460 INFO:tasks.workunit.client.0.smithi028.stdout:5/745: truncate d0/dd/d13/d57/f1f 272284 0 2024-07-03T15:44:16.463 INFO:tasks.workunit.client.0.smithi028.stdout:6/818: mkdir d4/d11d 0 2024-07-03T15:44:16.464 INFO:tasks.workunit.client.0.smithi028.stdout:6/819: write d4/d12/d2c/fe0 [5068516,116031] 0 2024-07-03T15:44:16.464 INFO:tasks.workunit.client.0.smithi028.stdout:3/720: symlink d18/d1a/d1e/d73/le8 0 2024-07-03T15:44:16.470 INFO:tasks.workunit.client.0.smithi028.stdout:5/746: dread d0/dd/d1e/d3b/f4f [4194304,4194304] 0 2024-07-03T15:44:16.472 INFO:tasks.workunit.client.0.smithi028.stdout:5/747: mkdir d0/dd/d1e/d3b/d3f/d71/df4 0 2024-07-03T15:44:16.474 INFO:tasks.workunit.client.0.smithi028.stdout:5/748: creat d0/d8f/db5/ddc/ff5 x:0 0 0 2024-07-03T15:44:16.474 INFO:tasks.workunit.client.0.smithi028.stdout:5/749: readlink d0/dd/d13/d57/d30/d70/d7f/l84 0 2024-07-03T15:44:16.477 INFO:tasks.workunit.client.0.smithi028.stdout:3/721: dread d18/d1a/d1e/f57 [0,4194304] 0 2024-07-03T15:44:16.478 INFO:tasks.workunit.client.0.smithi028.stdout:3/722: write d18/d1a/d1e/f53 [3739995,95995] 0 2024-07-03T15:44:16.486 INFO:tasks.workunit.client.0.smithi028.stdout:0/800: dwrite d2/d3/dc4/f38 [0,4194304] 0 2024-07-03T15:44:16.486 INFO:tasks.workunit.client.0.smithi028.stdout:6/820: dwrite d4/d78/d10c/d38/d3d/df9/f105 [0,4194304] 0 2024-07-03T15:44:16.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.508+0000 7fd1f37fe640 1 -- 172.21.15.28:0/936917085 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd1ec0fd600 msgr2=0x7fd1ec0fda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:16.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.508+0000 7fd1f37fe640 1 --2- 172.21.15.28:0/936917085 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd1ec0fd600 0x7fd1ec0fda70 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc0048d0 tx=0x7fd1dc009da0 comp rx=0 tx=0).stop 2024-07-03T15:44:16.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.508+0000 7fd1f37fe640 1 -- 172.21.15.28:0/936917085 shutdown_connections 2024-07-03T15:44:16.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.508+0000 7fd1f37fe640 1 --2- 172.21.15.28:0/936917085 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd1ec0fd600 0x7fd1ec0fda70 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:16.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.508+0000 7fd1f37fe640 1 --2- 172.21.15.28:0/936917085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1ec0fc330 0x7fd1ec0fc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:16.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.508+0000 7fd1f37fe640 1 -- 172.21.15.28:0/936917085 >> 172.21.15.28:0/936917085 conn(0x7fd1ec0f7af0 msgr2=0x7fd1ec0f9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:44:16.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.509+0000 7fd1f37fe640 1 -- 172.21.15.28:0/936917085 shutdown_connections 2024-07-03T15:44:16.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.509+0000 7fd1f37fe640 1 -- 172.21.15.28:0/936917085 wait complete. 2024-07-03T15:44:16.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.509+0000 7fd1f37fe640 1 Processor -- start 2024-07-03T15:44:16.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.509+0000 7fd1f37fe640 1 -- start start 2024-07-03T15:44:16.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.509+0000 7fd1f37fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd1ec0fc330 0x7fd1ec071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:44:16.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.509+0000 7fd1f37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1ec0fd600 0x7fd1ec06f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:44:16.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.509+0000 7fd1f37fe640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec06fe00 con 0x7fd1ec0fc330 2024-07-03T15:44:16.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.509+0000 7fd1f37fe640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec06ff40 con 0x7fd1ec0fd600 2024-07-03T15:44:16.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.510+0000 7fd1f159a640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd1ec0fc330 0x7fd1ec071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:44:16.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.510+0000 7fd1f159a640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd1ec0fc330 0x7fd1ec071380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50436/0 (socket says 172.21.15.28:50436) 2024-07-03T15:44:16.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.510+0000 7fd1f159a640 1 -- 172.21.15.28:0/2798132163 learned_addr learned my addr 172.21.15.28:0/2798132163 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:44:16.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.510+0000 7fd1f0d99640 1 --2- 172.21.15.28:0/2798132163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1ec0fd600 0x7fd1ec06f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:44:16.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.510+0000 7fd1f159a640 1 -- 172.21.15.28:0/2798132163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1ec0fd600 msgr2=0x7fd1ec06f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:16.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.510+0000 7fd1f159a640 1 --2- 172.21.15.28:0/2798132163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1ec0fd600 0x7fd1ec06f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:16.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.510+0000 7fd1f159a640 1 -- 172.21.15.28:0/2798132163 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1dc005040 con 0x7fd1ec0fc330 2024-07-03T15:44:16.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.510+0000 7fd1f159a640 1 --2- 172.21.15.28:0/2798132163 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd1ec0fc330 0x7fd1ec071380 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd1e800eb70 tx=0x7fd1e800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:44:16.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.510+0000 7fd1e1ffb640 1 -- 172.21.15.28:0/2798132163 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1e8007c30 con 0x7fd1ec0fc330 2024-07-03T15:44:16.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.510+0000 7fd1e1ffb640 1 -- 172.21.15.28:0/2798132163 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd1e800f040 con 0x7fd1ec0fc330 2024-07-03T15:44:16.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.510+0000 7fd1f37fe640 1 -- 172.21.15.28:0/2798132163 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1ec0701c0 con 0x7fd1ec0fc330 2024-07-03T15:44:16.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.510+0000 7fd1e1ffb640 1 -- 172.21.15.28:0/2798132163 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1e800cad0 con 0x7fd1ec0fc330 2024-07-03T15:44:16.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.511+0000 7fd1f37fe640 1 -- 172.21.15.28:0/2798132163 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1ec114fa0 con 0x7fd1ec0fc330 2024-07-03T15:44:16.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.511+0000 7fd1f37fe640 1 -- 172.21.15.28:0/2798132163 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1ec0fb790 con 0x7fd1ec0fc330 2024-07-03T15:44:16.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.512+0000 7fd1e1ffb640 1 -- 172.21.15.28:0/2798132163 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 24) v1 ==== 88617+0+0 (secure 0 0 0) 0x7fd1e800cc30 con 0x7fd1ec0fc330 2024-07-03T15:44:16.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.512+0000 7fd1e1ffb640 1 --2- 172.21.15.28:0/2798132163 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fd1d806a6d0 0x7fd1d806cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:44:16.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.512+0000 7fd1e1ffb640 1 -- 172.21.15.28:0/2798132163 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fd1e8088c10 con 0x7fd1ec0fc330 2024-07-03T15:44:16.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.513+0000 7fd1f0d99640 1 --2- 172.21.15.28:0/2798132163 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fd1d806a6d0 0x7fd1d806cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:44:16.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.513+0000 7fd1f0d99640 1 --2- 172.21.15.28:0/2798132163 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fd1d806a6d0 0x7fd1d806cb80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc009f80 tx=0x7fd1dc035040 comp rx=0 tx=0).ready entity=mgr.24403 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:44:16.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.517+0000 7fd1e1ffb640 1 -- 172.21.15.28:0/2798132163 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7fd1e8052850 con 0x7fd1ec0fc330 2024-07-03T15:44:16.544 INFO:tasks.workunit.client.0.smithi028.stdout:8/848: sync 2024-07-03T15:44:16.544 INFO:tasks.workunit.client.0.smithi028.stdout:4/742: sync 2024-07-03T15:44:16.544 INFO:tasks.workunit.client.0.smithi028.stdout:9/748: sync 2024-07-03T15:44:16.544 INFO:tasks.workunit.client.0.smithi028.stdout:4/743: fdatasync d2/d12/d3b/fe6 0 2024-07-03T15:44:16.544 INFO:tasks.workunit.client.0.smithi028.stdout:4/744: chown d2/d12/d2e/l49 41 1 2024-07-03T15:44:16.544 INFO:tasks.workunit.client.0.smithi028.stdout:3/723: mkdir d18/d1a/d5e/de9 0 2024-07-03T15:44:16.547 INFO:tasks.workunit.client.0.smithi028.stdout:6/821: mkdir d4/d12/dc3/d11e 0 2024-07-03T15:44:16.582 INFO:tasks.workunit.client.0.smithi028.stdout:9/749: link d1/d10/d28/d45/d61/d8c/cd6 d1/d10/d28/daa/ded/cfd 0 2024-07-03T15:44:16.584 INFO:tasks.workunit.client.0.smithi028.stdout:8/849: creat d0/d41/d50/d3/d35/d68/d22/d40/f117 x:0 0 0 2024-07-03T15:44:16.588 INFO:tasks.workunit.client.0.smithi028.stdout:4/745: rename d2/d12/d5b/d78/de4/d3a/d6a to d2/d12/d2e/df3 0 2024-07-03T15:44:16.589 INFO:tasks.workunit.client.0.smithi028.stdout:4/746: write d2/d5a/d45/fed [70638,44037] 0 2024-07-03T15:44:16.590 INFO:tasks.workunit.client.0.smithi028.stdout:6/822: mkdir d4/d12/d2c/d11f 0 2024-07-03T15:44:16.594 INFO:tasks.workunit.client.0.smithi028.stdout:9/750: mknod d1/dd/cfe 0 2024-07-03T15:44:16.596 INFO:tasks.workunit.client.0.smithi028.stdout:4/747: dread f1 [4194304,4194304] 0 2024-07-03T15:44:16.598 INFO:tasks.workunit.client.0.smithi028.stdout:8/850: link d0/d41/d50/d3/d35/c74 d0/d41/d50/d3/dc/d3f/d8e/d81/d100/c118 0 2024-07-03T15:44:16.598 INFO:tasks.workunit.client.0.smithi028.stdout:8/851: chown d0 1 1 2024-07-03T15:44:16.602 INFO:tasks.workunit.client.0.smithi028.stdout:9/751: write d1/dd/f59 [2915992,70787] 0 2024-07-03T15:44:16.602 INFO:tasks.workunit.client.0.smithi028.stdout:9/752: chown d1/d10/d28/d4e/f9f 3 1 2024-07-03T15:44:16.602 INFO:tasks.workunit.client.0.smithi028.stdout:9/753: creat d1/d10/d28/d45/d61/d8c/fff x:0 0 0 2024-07-03T15:44:16.603 INFO:tasks.workunit.client.0.smithi028.stdout:3/724: rename d18/d1a/f2a to d18/d6f/d78/fea 0 2024-07-03T15:44:16.606 INFO:tasks.workunit.client.0.smithi028.stdout:6/823: link d4/d78/d10c/d1c/f50 d4/d78/d10c/d38/d3d/d89/f120 0 2024-07-03T15:44:16.609 INFO:tasks.workunit.client.0.smithi028.stdout:8/852: mkdir d0/d41/d50/d3/dc/d3f/d8e/d81/d9e/d119 0 2024-07-03T15:44:16.612 INFO:tasks.workunit.client.0.smithi028.stdout:9/754: link d1/d16/f19 d1/d10/d28/d45/d61/d8c/f100 0 2024-07-03T15:44:16.613 INFO:tasks.workunit.client.0.smithi028.stdout:9/755: dread d1/d10/d28/d45/d61/d8c/d77/fb5 [0,4194304] 0 2024-07-03T15:44:16.613 INFO:tasks.workunit.client.0.smithi028.stdout:9/756: dread - d1/d10/d28/d45/d56/fc5 zero size 2024-07-03T15:44:16.613 INFO:tasks.workunit.client.0.smithi028.stdout:9/757: getdents d1/d10/d28/d4e/d5b/df0 0 2024-07-03T15:44:16.614 INFO:tasks.workunit.client.0.smithi028.stdout:4/748: rename d2/d61 to d2/d5a/d45/d84/df4 0 2024-07-03T15:44:16.614 INFO:tasks.workunit.client.0.smithi028.stdout:4/749: dread - d2/d12/d42/d63/da8/faa zero size 2024-07-03T15:44:16.615 INFO:tasks.workunit.client.0.smithi028.stdout:3/725: truncate d18/d43/f4d 1469987 0 2024-07-03T15:44:16.615 INFO:tasks.workunit.client.0.smithi028.stdout:3/726: chown d18/d43 31 1 2024-07-03T15:44:16.616 INFO:tasks.workunit.client.0.smithi028.stdout:3/727: read - d18/d1a/d5e/d66/da3/fac zero size 2024-07-03T15:44:16.617 INFO:tasks.workunit.client.0.smithi028.stdout:6/824: unlink d4/d78/d10c/d13/c8a 0 2024-07-03T15:44:16.618 INFO:tasks.workunit.client.0.smithi028.stdout:9/758: write d1/d16/f1c [1259124,117064] 0 2024-07-03T15:44:16.618 INFO:tasks.workunit.client.0.smithi028.stdout:9/759: chown d1/d16/d57/f9c 937017385 1 2024-07-03T15:44:16.618 INFO:tasks.workunit.client.0.smithi028.stdout:9/760: truncate d1/d17/f9a 8580955 0 2024-07-03T15:44:16.618 INFO:tasks.workunit.client.0.smithi028.stdout:9/761: write d1/dd/f90 [7336595,43074] 0 2024-07-03T15:44:16.621 INFO:tasks.workunit.client.0.smithi028.stdout:4/750: creat d2/d5a/d45/d84/df4/d9d/ff5 x:0 0 0 2024-07-03T15:44:16.622 INFO:tasks.workunit.client.0.smithi028.stdout:9/762: creat d1/d10/d28/d45/d61/f101 x:0 0 0 2024-07-03T15:44:16.622 INFO:tasks.workunit.client.0.smithi028.stdout:9/763: write d1/d10/d28/fd0 [322802,77751] 0 2024-07-03T15:44:16.622 INFO:tasks.workunit.client.0.smithi028.stdout:9/764: readlink d1/d10/d28/d45/d61/d8c/lba 0 2024-07-03T15:44:16.625 INFO:tasks.workunit.client.0.smithi028.stdout:4/751: rename d2/d12/c21 to d2/d12/d42/d63/d97/dbe/cf6 0 2024-07-03T15:44:16.626 INFO:tasks.workunit.client.0.smithi028.stdout:9/765: symlink d1/d10/d28/d45/d61/d8c/d77/db7/dab/dca/l102 0 2024-07-03T15:44:16.628 INFO:tasks.workunit.client.0.smithi028.stdout:4/752: getdents d2/d12/d2d/db9/dda 0 2024-07-03T15:44:16.629 INFO:tasks.workunit.client.0.smithi028.stdout:3/728: read d18/d3a/f3d [1749120,95650] 0 2024-07-03T15:44:16.631 INFO:tasks.workunit.client.0.smithi028.stdout:9/766: rename d1/d16/d62/c9b to d1/d10/d76/c103 0 2024-07-03T15:44:16.632 INFO:tasks.workunit.client.0.smithi028.stdout:3/729: mknod d18/d1a/d1e/d73/ceb 0 2024-07-03T15:44:16.635 INFO:tasks.workunit.client.0.smithi028.stdout:9/767: link d1/d8e/dac/fbf d1/f104 0 2024-07-03T15:44:16.638 INFO:tasks.workunit.client.0.smithi028.stdout:6/825: dwrite d4/d78/d10c/d1c/f61 [4194304,4194304] 0 2024-07-03T15:44:16.638 INFO:tasks.workunit.client.0.smithi028.stdout:6/826: write d4/d78/f10e [113717,120586] 0 2024-07-03T15:44:16.638 INFO:tasks.workunit.client.0.smithi028.stdout:9/768: link d1/d10/d28/d45/d56/l4d d1/d8e/l105 0 2024-07-03T15:44:16.641 INFO:tasks.workunit.client.0.smithi028.stdout:3/730: write d18/d1a/f4f [4588469,6521] 0 2024-07-03T15:44:16.641 INFO:tasks.workunit.client.0.smithi028.stdout:6/827: rename d4/d78/d10c/d47/d66/c6d to d4/d12/dc3/c121 0 2024-07-03T15:44:16.642 INFO:tasks.workunit.client.0.smithi028.stdout:9/769: creat d1/d10/d28/d4e/f106 x:0 0 0 2024-07-03T15:44:16.644 INFO:tasks.workunit.client.0.smithi028.stdout:4/753: write d2/d5a/fe7 [385732,44851] 0 2024-07-03T15:44:16.645 INFO:tasks.workunit.client.0.smithi028.stdout:3/731: symlink d18/d3a/dbb/lec 0 2024-07-03T15:44:16.648 INFO:tasks.workunit.client.0.smithi028.stdout:4/754: rmdir d2 39 2024-07-03T15:44:16.650 INFO:tasks.workunit.client.0.smithi028.stdout:3/732: mknod d18/d3a/ddf/ced 0 2024-07-03T15:44:16.650 INFO:tasks.workunit.client.0.smithi028.stdout:3/733: readlink d18/d1a/d1e/d73/le8 0 2024-07-03T15:44:16.650 INFO:tasks.workunit.client.0.smithi028.stdout:3/734: stat d18/d1a/d5e 0 2024-07-03T15:44:16.651 INFO:tasks.workunit.client.0.smithi028.stdout:4/755: stat d2/d5a/d45/cc0 0 2024-07-03T15:44:16.656 INFO:tasks.workunit.client.0.smithi028.stdout:4/756: creat d2/d5a/d45/ff7 x:0 0 0 2024-07-03T15:44:16.666 INFO:tasks.workunit.client.0.smithi028.stdout:6/828: dwrite d4/f7b [4194304,4194304] 0 2024-07-03T15:44:16.666 INFO:tasks.workunit.client.0.smithi028.stdout:9/770: dwrite d1/d10/d28/d45/d61/d8c/f81 [0,4194304] 0 2024-07-03T15:44:16.667 INFO:tasks.workunit.client.0.smithi028.stdout:6/829: symlink d4/d78/l122 0 2024-07-03T15:44:16.669 INFO:tasks.workunit.client.0.smithi028.stdout:9/771: mknod d1/d8e/c107 0 2024-07-03T15:44:16.670 INFO:tasks.workunit.client.0.smithi028.stdout:6/830: symlink d4/d78/d10c/d47/d66/daf/l123 0 2024-07-03T15:44:16.670 INFO:tasks.workunit.client.0.smithi028.stdout:6/831: write d4/d12/dc3/fdc [1271606,41329] 0 2024-07-03T15:44:16.670 INFO:tasks.workunit.client.0.smithi028.stdout:6/832: write d4/d78/d10c/d47/d9f/f10f [329653,53371] 0 2024-07-03T15:44:16.671 INFO:tasks.workunit.client.0.smithi028.stdout:9/772: symlink d1/d10/l108 0 2024-07-03T15:44:16.671 INFO:tasks.workunit.client.0.smithi028.stdout:9/773: symlink d1/d10/d28/d45/d61/d8c/d77/l109 0 2024-07-03T15:44:16.672 INFO:tasks.workunit.client.0.smithi028.stdout:9/774: mknod d1/d10/d28/d45/d61/d8c/d77/db7/dab/dcb/c10a 0 2024-07-03T15:44:16.673 INFO:tasks.workunit.client.0.smithi028.stdout:4/757: dwrite d2/d12/d42/d63/d6d/fd2 [0,4194304] 0 2024-07-03T15:44:16.673 INFO:tasks.workunit.client.0.smithi028.stdout:9/775: creat d1/d16/d57/d5f/dd8/f10b x:0 0 0 2024-07-03T15:44:16.677 INFO:tasks.workunit.client.0.smithi028.stdout:4/758: mknod d2/d5a/d45/d84/df4/d9d/cf8 0 2024-07-03T15:44:16.677 INFO:tasks.workunit.client.0.smithi028.stdout:4/759: fsync d2/d5a/d45/d84/df4/d9d/faf 0 2024-07-03T15:44:16.678 INFO:tasks.workunit.client.0.smithi028.stdout:9/776: creat d1/d10/d28/d45/d61/d8c/d77/db7/dab/f10c x:0 0 0 2024-07-03T15:44:16.679 INFO:tasks.workunit.client.0.smithi028.stdout:4/760: creat d2/d12/d2d/db9/dda/ff9 x:0 0 0 2024-07-03T15:44:16.681 INFO:tasks.workunit.client.0.smithi028.stdout:4/761: unlink d2/d12/d42/d63/d6d/ld7 0 2024-07-03T15:44:16.684 INFO:tasks.workunit.client.0.smithi028.stdout:4/762: getdents d2/d5a/d45 0 2024-07-03T15:44:16.684 INFO:tasks.workunit.client.0.smithi028.stdout:4/763: dread - d2/d12/d2d/de9/fef zero size 2024-07-03T15:44:16.686 INFO:tasks.workunit.client.0.smithi028.stdout:9/777: dread d1/d16/f51 [0,4194304] 0 2024-07-03T15:44:16.686 INFO:tasks.workunit.client.0.smithi028.stdout:9/778: stat d1/d16/d57 0 2024-07-03T15:44:16.686 INFO:tasks.workunit.client.0.smithi028.stdout:9/779: getdents d1/d10/d28/d45/d56/dcd 0 2024-07-03T15:44:16.692 INFO:tasks.workunit.client.0.smithi028.stdout:4/764: write d2/d12/d5b/d78/de4/f66 [163612,110922] 0 2024-07-03T15:44:16.694 INFO:tasks.workunit.client.0.smithi028.stdout:9/780: getdents d1/d10/d28/d45/d61/d8c/d77 0 2024-07-03T15:44:16.696 INFO:tasks.workunit.client.0.smithi028.stdout:4/765: mkdir d2/d12/d42/d63/dfa 0 2024-07-03T15:44:16.696 INFO:tasks.workunit.client.0.smithi028.stdout:4/766: truncate d2/d5a/d45/d84/df4/fc5 4312636 0 2024-07-03T15:44:16.697 INFO:tasks.workunit.client.0.smithi028.stdout:9/781: link d1/d16/d57/d5f/d64/fd1 d1/d10/d28/d45/d56/dcd/dd5/f10d 0 2024-07-03T15:44:16.700 INFO:tasks.workunit.client.0.smithi028.stdout:9/782: rename d1/d10/d28/d45/d61/d8c/fbe to d1/d4a/d7e/f10e 0 2024-07-03T15:44:16.706 INFO:tasks.workunit.client.0.smithi028.stdout:9/783: dread d1/d16/f19 [0,4194304] 0 2024-07-03T15:44:16.706 INFO:tasks.workunit.client.0.smithi028.stdout:9/784: fsync d1/d10/f5d 0 2024-07-03T15:44:16.709 INFO:tasks.workunit.client.0.smithi028.stdout:9/785: unlink d1/d8e/dac/le5 0 2024-07-03T15:44:16.709 INFO:tasks.workunit.client.0.smithi028.stdout:9/786: creat d1/dd/f10f x:0 0 0 2024-07-03T15:44:16.709 INFO:tasks.workunit.client.0.smithi028.stdout:9/787: write d1/d10/d28/d45/d61/d8c/d77/f91 [184016,77822] 0 2024-07-03T15:44:16.710 INFO:tasks.workunit.client.0.smithi028.stdout:9/788: fdatasync d1/d10/d28/fd0 0 2024-07-03T15:44:16.710 INFO:tasks.workunit.client.0.smithi028.stdout:9/789: chown d1/d10/d28/fa5 32 1 2024-07-03T15:44:16.714 INFO:tasks.workunit.client.0.smithi028.stdout:4/767: dwrite d2/d12/d2e/f38 [0,4194304] 0 2024-07-03T15:44:16.715 INFO:tasks.workunit.client.0.smithi028.stdout:4/768: write d2/d5a/d45/fed [972712,96857] 0 2024-07-03T15:44:16.715 INFO:tasks.workunit.client.0.smithi028.stdout:4/769: truncate d2/d12/d42/d63/f65 840000 0 2024-07-03T15:44:16.715 INFO:tasks.workunit.client.0.smithi028.stdout:4/770: chown d2/d5a/d45/d84/df4/d9d/cd6 129666766 1 2024-07-03T15:44:16.738 INFO:tasks.workunit.client.0.smithi028.stdout:9/790: write d1/d4a/f92 [2575371,124111] 0 2024-07-03T15:44:16.739 INFO:tasks.workunit.client.0.smithi028.stdout:9/791: fsync d1/d10/d28/d45/d61/d8c/fff 0 2024-07-03T15:44:16.748 INFO:tasks.workunit.client.0.smithi028.stdout:9/792: mknod d1/d10/d28/d4e/d5b/de9/c110 0 2024-07-03T15:44:16.748 INFO:tasks.workunit.client.0.smithi028.stdout:9/793: chown d1/d8e/f9e 4 1 2024-07-03T15:44:16.749 INFO:tasks.workunit.client.0.smithi028.stdout:4/771: dwrite d2/d5a/d45/fb3 [0,4194304] 0 2024-07-03T15:44:16.754 INFO:tasks.workunit.client.0.smithi028.stdout:4/772: mkdir d2/d12/d2e/df3/dfb 0 2024-07-03T15:44:16.756 INFO:tasks.workunit.client.0.smithi028.stdout:9/794: dread d1/d10/d28/d45/d61/d8c/f72 [0,4194304] 0 2024-07-03T15:44:16.756 INFO:tasks.workunit.client.0.smithi028.stdout:9/795: truncate d1/d16/d57/d5f/d7f/fad 1583910 0 2024-07-03T15:44:16.756 INFO:tasks.workunit.client.0.smithi028.stdout:9/796: dread - d1/d16/d57/d5f/d64/ff3 zero size 2024-07-03T15:44:16.758 INFO:tasks.workunit.client.0.smithi028.stdout:4/773: link d2/cd d2/cfc 0 2024-07-03T15:44:16.758 INFO:tasks.workunit.client.0.smithi028.stdout:4/774: fsync d2/d5a/d45/d84/fec 0 2024-07-03T15:44:16.758 INFO:tasks.workunit.client.0.smithi028.stdout:4/775: readlink d2/d12/d5b/la2 0 2024-07-03T15:44:16.759 INFO:tasks.workunit.client.0.smithi028.stdout:9/797: rename d1/d10/d28/d45/d61/d8c/f72 to d1/d4a/d7e/f111 0 2024-07-03T15:44:16.760 INFO:tasks.workunit.client.0.smithi028.stdout:4/776: write d2/d5a/f56 [438177,28787] 0 2024-07-03T15:44:16.760 INFO:tasks.workunit.client.0.smithi028.stdout:9/798: rmdir d1/d10/d28/d45/d61/d8c/d77/db7/dab/dcb 39 2024-07-03T15:44:16.763 INFO:tasks.workunit.client.0.smithi028.stdout:4/777: rename d2/d12/d5b/d78/f79 to d2/d12/d42/ffd 0 2024-07-03T15:44:16.763 INFO:tasks.workunit.client.0.smithi028.stdout:9/799: mknod d1/d10/d28/d45/d61/d8c/d77/db7/dab/c112 0 2024-07-03T15:44:16.766 INFO:tasks.workunit.client.0.smithi028.stdout:9/800: creat d1/d10/d28/d4e/d5b/df0/f113 x:0 0 0 2024-07-03T15:44:16.766 INFO:tasks.workunit.client.0.smithi028.stdout:9/801: fdatasync d1/d8e/f9e 0 2024-07-03T15:44:16.766 INFO:tasks.workunit.client.0.smithi028.stdout:9/802: fsync d1/d16/d57/d5f/d64/dc3/fdc 0 2024-07-03T15:44:16.766 INFO:tasks.workunit.client.0.smithi028.stdout:4/778: rmdir d2/d12/d2d/db9/dda 39 2024-07-03T15:44:16.776 INFO:tasks.workunit.client.0.smithi028.stdout:4/779: write d2/d12/da6/fcc [1069906,54254] 0 2024-07-03T15:44:16.776 INFO:tasks.workunit.client.0.smithi028.stdout:4/780: chown d2/d12/d42/d63/d6d/dc8 1 1 2024-07-03T15:44:16.782 INFO:tasks.workunit.client.0.smithi028.stdout:9/803: dwrite d1/d10/d28/d45/d61/d8c/d77/f91 [0,4194304] 0 2024-07-03T15:44:16.795 INFO:tasks.workunit.client.0.smithi028.stdout:4/781: dwrite d2/d12/d2d/d58/f5e [0,4194304] 0 2024-07-03T15:44:16.795 INFO:tasks.workunit.client.0.smithi028.stdout:4/782: chown d2/f86 5161480 1 2024-07-03T15:44:16.801 INFO:tasks.workunit.client.0.smithi028.stdout:9/804: dwrite d1/d16/f63 [0,4194304] 0 2024-07-03T15:44:16.802 INFO:tasks.workunit.client.0.smithi028.stdout:9/805: creat d1/d10/d28/d45/d61/d8c/d77/db7/dab/f114 x:0 0 0 2024-07-03T15:44:16.802 INFO:tasks.workunit.client.0.smithi028.stdout:9/806: write d1/d10/d28/d45/d61/d8c/f6a [580083,16990] 0 2024-07-03T15:44:16.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.801+0000 7fd1f37fe640 1 -- 172.21.15.28:0/2798132163 --> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1ec101920 con 0x7fd1d806a6d0 2024-07-03T15:44:16.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.802+0000 7fd1e1ffb640 1 -- 172.21.15.28:0/2798132163 <== mgr.24403 v2:172.21.15.148:6828/4285739175 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7fd1ec101920 con 0x7fd1d806a6d0 2024-07-03T15:44:16.806 INFO:tasks.workunit.client.0.smithi028.stdout:9/807: mkdir d1/d16/d57/d5f/de3/d115 0 2024-07-03T15:44:16.808 INFO:tasks.workunit.client.0.smithi028.stdout:7/617: sync 2024-07-03T15:44:16.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.806+0000 7fd1f37fe640 1 -- 172.21.15.28:0/2798132163 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fd1d806a6d0 msgr2=0x7fd1d806cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:16.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.806+0000 7fd1f37fe640 1 --2- 172.21.15.28:0/2798132163 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fd1d806a6d0 0x7fd1d806cb80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc009f80 tx=0x7fd1dc035040 comp rx=0 tx=0).stop 2024-07-03T15:44:16.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.806+0000 7fd1f37fe640 1 -- 172.21.15.28:0/2798132163 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd1ec0fc330 msgr2=0x7fd1ec071380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:16.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.806+0000 7fd1f37fe640 1 --2- 172.21.15.28:0/2798132163 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd1ec0fc330 0x7fd1ec071380 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd1e800eb70 tx=0x7fd1e800b040 comp rx=0 tx=0).stop 2024-07-03T15:44:16.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.806+0000 7fd1f37fe640 1 -- 172.21.15.28:0/2798132163 shutdown_connections 2024-07-03T15:44:16.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.806+0000 7fd1f37fe640 1 --2- 172.21.15.28:0/2798132163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1ec0fd600 0x7fd1ec06f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:16.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.806+0000 7fd1f37fe640 1 --2- 172.21.15.28:0/2798132163 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fd1d806a6d0 0x7fd1d806cb80 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:16.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.806+0000 7fd1f37fe640 1 --2- 172.21.15.28:0/2798132163 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd1ec0fc330 0x7fd1ec071380 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:16.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.806+0000 7fd1f37fe640 1 -- 172.21.15.28:0/2798132163 >> 172.21.15.28:0/2798132163 conn(0x7fd1ec0f7af0 msgr2=0x7fd1ec0f91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:44:16.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.806+0000 7fd1f37fe640 1 -- 172.21.15.28:0/2798132163 shutdown_connections 2024-07-03T15:44:16.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.806+0000 7fd1f37fe640 1 -- 172.21.15.28:0/2798132163 wait complete. 2024-07-03T15:44:16.812 INFO:tasks.workunit.client.0.smithi028.stdout:4/783: dwrite d2/f27 [0,4194304] 0 2024-07-03T15:44:16.812 INFO:tasks.workunit.client.0.smithi028.stdout:4/784: truncate d2/d12/d2d/db9/dda/ff9 1046009 0 2024-07-03T15:44:16.814 INFO:tasks.workunit.client.0.smithi028.stdout:4/785: rename f1 to d2/d12/ffe 0 2024-07-03T15:44:16.814 INFO:tasks.workunit.client.0.smithi028.stdout:4/786: fsync d2/d5a/d45/d84/df4/d9d/fc4 0 2024-07-03T15:44:16.815 INFO:tasks.workunit.client.0.smithi028.stdout:4/787: symlink d2/d12/lff 0 2024-07-03T15:44:16.816 INFO:tasks.workunit.client.0.smithi028.stdout:7/618: dread d9/f10 [0,4194304] 0 2024-07-03T15:44:16.817 INFO:tasks.workunit.client.0.smithi028.stdout:4/788: creat d2/d12/d2e/f100 x:0 0 0 2024-07-03T15:44:16.817 INFO:tasks.workunit.client.0.smithi028.stdout:4/789: write d2/d5a/d45/f4c [2363646,16732] 0 2024-07-03T15:44:16.817 INFO:tasks.workunit.client.0.smithi028.stdout:4/790: chown d2/d5a/d45/d84/df4/d9c/cbc 5996 1 2024-07-03T15:44:16.818 INFO:tasks.workunit.client.0.smithi028.stdout:7/619: rename d9/d1c/c37 to d9/d1c/d7c/d8b/da0/cb6 0 2024-07-03T15:44:16.818 INFO:tasks.workunit.client.0.smithi028.stdout:7/620: chown d9/d1c/d45/d71/d78/c8e 3 1 2024-07-03T15:44:16.820 INFO:tasks.workunit.client.0.smithi028.stdout:7/621: chown d9/d1c/d45/db5/d92/d99/d6a/c6d 5324853 1 2024-07-03T15:44:16.821 INFO:tasks.workunit.client.0.smithi028.stdout:7/622: creat d9/d1c/d86/fb7 x:0 0 0 2024-07-03T15:44:16.826 INFO:tasks.workunit.client.0.smithi028.stdout:9/808: dwrite d1/d10/d28/d45/fdd [0,4194304] 0 2024-07-03T15:44:16.827 INFO:tasks.workunit.client.0.smithi028.stdout:7/623: dread d9/f84 [0,4194304] 0 2024-07-03T15:44:16.828 INFO:tasks.workunit.client.0.smithi028.stdout:4/791: write d2/d12/d5b/f9b [3439689,21938] 0 2024-07-03T15:44:16.832 INFO:tasks.workunit.client.0.smithi028.stdout:4/792: creat d2/d12/d5b/f101 x:0 0 0 2024-07-03T15:44:16.832 INFO:tasks.workunit.client.0.smithi028.stdout:9/809: dread d1/d16/f1c [0,4194304] 0 2024-07-03T15:44:16.832 INFO:tasks.workunit.client.0.smithi028.stdout:9/810: chown d1/d10/d28/fd0 257814 1 2024-07-03T15:44:16.832 INFO:tasks.workunit.client.0.smithi028.stdout:7/624: write d9/f10 [4049584,65583] 0 2024-07-03T15:44:16.833 INFO:tasks.workunit.client.0.smithi028.stdout:4/793: creat d2/d12/d42/d63/dfa/f102 x:0 0 0 2024-07-03T15:44:16.834 INFO:tasks.workunit.client.0.smithi028.stdout:9/811: mkdir d1/d10/d28/d116 0 2024-07-03T15:44:16.835 INFO:tasks.workunit.client.0.smithi028.stdout:4/794: rmdir d2/d5a/d45 39 2024-07-03T15:44:16.839 INFO:tasks.workunit.client.0.smithi028.stdout:7/625: dread d9/f1a [0,4194304] 0 2024-07-03T15:44:16.840 INFO:tasks.workunit.client.0.smithi028.stdout:9/812: mknod d1/d10/d28/d45/d61/d8c/d77/db7/dab/dca/c117 0 2024-07-03T15:44:16.841 INFO:tasks.workunit.client.0.smithi028.stdout:9/813: write d1/d8e/f9e [588278,3893] 0 2024-07-03T15:44:16.841 INFO:tasks.workunit.client.0.smithi028.stdout:9/814: chown d1/d16/f3a 89 1 2024-07-03T15:44:16.842 INFO:tasks.workunit.client.0.smithi028.stdout:7/626: link d9/l9b d9/d1c/d7c/d8b/lb8 0 2024-07-03T15:44:16.842 INFO:tasks.workunit.client.0.smithi028.stdout:7/627: creat d9/d1c/d7c/d8b/fb9 x:0 0 0 2024-07-03T15:44:16.842 INFO:tasks.workunit.client.0.smithi028.stdout:7/628: fsync d9/fa3 0 2024-07-03T15:44:16.843 INFO:tasks.workunit.client.0.smithi028.stdout:9/815: creat d1/d10/d28/d45/d56/dcd/dd5/f118 x:0 0 0 2024-07-03T15:44:16.844 INFO:tasks.workunit.client.0.smithi028.stdout:4/795: dread d2/f7 [0,4194304] 0 2024-07-03T15:44:16.845 INFO:tasks.workunit.client.0.smithi028.stdout:7/629: mknod d9/d1c/d45/db5/cba 0 2024-07-03T15:44:16.845 INFO:tasks.workunit.client.0.smithi028.stdout:7/630: fdatasync d9/f1a 0 2024-07-03T15:44:16.845 INFO:tasks.workunit.client.0.smithi028.stdout:7/631: creat d9/d1c/d45/d71/d78/fbb x:0 0 0 2024-07-03T15:44:16.847 INFO:tasks.workunit.client.0.smithi028.stdout:9/816: rename d1/d10/d28/d45/d61/d8c/ff9 to d1/d8e/f119 0 2024-07-03T15:44:16.847 INFO:tasks.workunit.client.0.smithi028.stdout:9/817: readlink d1/d8e/lf4 0 2024-07-03T15:44:16.851 INFO:tasks.workunit.client.0.smithi028.stdout:7/632: creat d9/d1c/d6e/fbc x:0 0 0 2024-07-03T15:44:16.851 INFO:tasks.workunit.client.0.smithi028.stdout:7/633: read d9/d1c/d45/f62 [335410,38934] 0 2024-07-03T15:44:16.853 INFO:tasks.workunit.client.0.smithi028.stdout:9/818: mkdir d1/d10/d28/d45/d56/d11a 0 2024-07-03T15:44:16.853 INFO:tasks.workunit.client.0.smithi028.stdout:9/819: getdents d1/d10/d28/d45/d61/d8c/d77 0 2024-07-03T15:44:16.853 INFO:tasks.workunit.client.0.smithi028.stdout:9/820: getdents d1/d10/d28/d45/d56/d11a 0 2024-07-03T15:44:16.854 INFO:tasks.workunit.client.0.smithi028.stdout:4/796: rmdir d2/d12/d42/d63/da8 39 2024-07-03T15:44:16.856 INFO:tasks.workunit.client.0.smithi028.stdout:9/821: mkdir d1/d10/d28/d45/d56/d11a/d11b 0 2024-07-03T15:44:16.858 INFO:tasks.workunit.client.0.smithi028.stdout:4/797: mknod d2/c103 0 2024-07-03T15:44:16.858 INFO:tasks.workunit.client.0.smithi028.stdout:4/798: chown d2/d12/d5b/d78/de4/d3a/l4f 0 1 2024-07-03T15:44:16.859 INFO:tasks.workunit.client.0.smithi028.stdout:9/822: truncate d1/dd/f6f 815646 0 2024-07-03T15:44:16.859 INFO:tasks.workunit.client.0.smithi028.stdout:9/823: write d1/d10/d28/d45/d61/f101 [293676,28213] 0 2024-07-03T15:44:16.862 INFO:tasks.workunit.client.0.smithi028.stdout:4/799: link d2/d12/d3b/fd8 d2/d5a/d45/d84/df4/d92/f104 0 2024-07-03T15:44:16.865 INFO:tasks.workunit.client.0.smithi028.stdout:4/800: link d2/d5a/d45/d84/df4/f91 d2/d12/d5b/d98/f105 0 2024-07-03T15:44:16.868 INFO:tasks.workunit.client.0.smithi028.stdout:7/634: dwrite d9/d1c/d45/db5/f44 [0,4194304] 0 2024-07-03T15:44:16.868 INFO:tasks.workunit.client.0.smithi028.stdout:9/824: rename d1/d10/l38 to d1/d10/d28/d4e/d5b/l11c 0 2024-07-03T15:44:16.868 INFO:tasks.workunit.client.0.smithi028.stdout:9/825: fdatasync d1/d10/d28/d45/d61/d8c/f100 0 2024-07-03T15:44:16.868 INFO:tasks.workunit.client.0.smithi028.stdout:9/826: chown d1/dd/f90 7062013 1 2024-07-03T15:44:16.869 INFO:tasks.workunit.client.0.smithi028.stdout:4/801: mkdir d2/d12/d3b/d106 0 2024-07-03T15:44:16.869 INFO:tasks.workunit.client.0.smithi028.stdout:4/802: chown d2/d5a/d45/d84/fe1 114653 1 2024-07-03T15:44:16.870 INFO:tasks.workunit.client.0.smithi028.stdout:4/803: write d2/d12/d5b/d98/fb2 [445884,118781] 0 2024-07-03T15:44:16.872 INFO:tasks.workunit.client.0.smithi028.stdout:7/635: creat d9/d1c/d45/db5/d92/d99/daf/fbd x:0 0 0 2024-07-03T15:44:16.872 INFO:tasks.workunit.client.0.smithi028.stdout:7/636: chown d9/d1c/d45/db5/d92/d9e 122852888 1 2024-07-03T15:44:16.875 INFO:tasks.workunit.client.0.smithi028.stdout:9/827: mkdir d1/d10/d28/d45/d61/d8c/d77/db7/dab/dca/d11d 0 2024-07-03T15:44:16.877 INFO:tasks.workunit.client.0.smithi028.stdout:4/804: mkdir d2/d5a/d45/d84/df4/d107 0 2024-07-03T15:44:16.878 INFO:tasks.workunit.client.0.smithi028.stdout:7/637: truncate d9/d1c/d45/db5/f4f 3527800 0 2024-07-03T15:44:16.878 INFO:tasks.workunit.client.0.smithi028.stdout:7/638: write d9/f69 [5175901,127744] 0 2024-07-03T15:44:16.878 INFO:tasks.workunit.client.0.smithi028.stdout:7/639: write d9/d1c/d45/fae [745325,117569] 0 2024-07-03T15:44:16.879 INFO:tasks.workunit.client.0.smithi028.stdout:9/828: getdents d1/d10/d28/d45/d61/d8c/d77/db7/dab/dcb 0 2024-07-03T15:44:16.880 INFO:tasks.workunit.client.0.smithi028.stdout:4/805: mknod d2/d12/d42/d63/d6d/dc8/c108 0 2024-07-03T15:44:16.882 INFO:tasks.workunit.client.0.smithi028.stdout:7/640: write d9/f1f [1949850,100087] 0 2024-07-03T15:44:16.882 INFO:tasks.workunit.client.0.smithi028.stdout:7/641: creat d9/d1c/d86/fbe x:0 0 0 2024-07-03T15:44:16.882 INFO:tasks.workunit.client.0.smithi028.stdout:7/642: creat d9/d1c/d6e/fbf x:0 0 0 2024-07-03T15:44:16.882 INFO:tasks.workunit.client.0.smithi028.stdout:9/829: mknod d1/d10/d28/d4e/d5b/c11e 0 2024-07-03T15:44:16.882 INFO:tasks.workunit.client.0.smithi028.stdout:9/830: stat d1/d10/d28/d45/d61 0 2024-07-03T15:44:16.884 INFO:tasks.workunit.client.0.smithi028.stdout:4/806: creat d2/d3e/dae/f109 x:0 0 0 2024-07-03T15:44:16.886 INFO:tasks.workunit.client.0.smithi028.stdout:7/643: creat d9/d1c/d45/db5/d92/d99/d6a/fc0 x:0 0 0 2024-07-03T15:44:16.889 INFO:tasks.workunit.client.0.smithi028.stdout:4/807: symlink d2/d12/d42/d63/dfa/l10a 0 2024-07-03T15:44:16.890 INFO:tasks.workunit.client.0.smithi028.stdout:7/644: mkdir d9/d1c/da9/dc1 0 2024-07-03T15:44:16.895 INFO:tasks.workunit.client.0.smithi028.stdout:9/831: rmdir d1/d10/d28/d45/d61/d8c 39 2024-07-03T15:44:16.896 INFO:tasks.workunit.client.0.smithi028.stdout:4/808: symlink d2/d12/d5b/d78/l10b 0 2024-07-03T15:44:16.897 INFO:tasks.workunit.client.0.smithi028.stdout:7/645: symlink d9/d1c/da9/dc1/lc2 0 2024-07-03T15:44:16.900 INFO:tasks.workunit.client.0.smithi028.stdout:9/832: mkdir d1/d10/d28/daa/d11f 0 2024-07-03T15:44:16.900 INFO:tasks.workunit.client.0.smithi028.stdout:9/833: chown d1/d10/d28/d45/d61/d8c/d77/db7/fb0 42 1 2024-07-03T15:44:16.904 INFO:tasks.workunit.client.0.smithi028.stdout:7/646: write f0 [1149203,76329] 0 2024-07-03T15:44:16.907 INFO:tasks.workunit.client.0.smithi028.stdout:4/809: symlink d2/d3e/d9f/d74/l10c 0 2024-07-03T15:44:16.910 INFO:tasks.workunit.client.0.smithi028.stdout:9/834: creat d1/d10/d28/d45/f120 x:0 0 0 2024-07-03T15:44:16.912 INFO:tasks.workunit.client.0.smithi028.stdout:7/647: symlink d9/lc3 0 2024-07-03T15:44:16.912 INFO:tasks.workunit.client.0.smithi028.stdout:7/648: write d9/d1c/d45/db5/d92/d99/f58 [4584247,39046] 0 2024-07-03T15:44:16.914 INFO:tasks.workunit.client.0.smithi028.stdout:9/835: symlink d1/d10/d28/d45/d61/d8c/l121 0 2024-07-03T15:44:16.916 INFO:tasks.workunit.client.0.smithi028.stdout:4/810: write d2/d12/d42/ffd [334203,108050] 0 2024-07-03T15:44:16.918 INFO:tasks.workunit.client.0.smithi028.stdout:7/649: mkdir d9/d1c/d45/db5/d92/d99/daf/dc4 0 2024-07-03T15:44:16.918 INFO:tasks.workunit.client.0.smithi028.stdout:7/650: truncate d9/f34 4096223 0 2024-07-03T15:44:16.920 INFO:tasks.workunit.client.0.smithi028.stdout:9/836: link d1/d10/d28/d4e/f9f d1/d10/d28/d45/d56/dcd/f122 0 2024-07-03T15:44:16.922 INFO:tasks.workunit.client.0.smithi028.stdout:4/811: mkdir d2/d10d 0 2024-07-03T15:44:16.922 INFO:tasks.workunit.client.0.smithi028.stdout:4/812: chown d2/d5a/d45/d84 1 1 2024-07-03T15:44:16.923 INFO:tasks.workunit.client.0.smithi028.stdout:4/813: write d2/d12/d5b/d98/f105 [608785,82608] 0 2024-07-03T15:44:16.924 INFO:tasks.workunit.client.0.smithi028.stdout:4/814: rename d2/d12/d42/c7f to d2/d3e/d9f/d74/c10e 0 2024-07-03T15:44:16.924 INFO:tasks.workunit.client.0.smithi028.stdout:4/815: chown d2/d12/d3b/fe6 0 1 2024-07-03T15:44:16.924 INFO:tasks.workunit.client.0.smithi028.stdout:4/816: stat d2/d3e/d9f/d74/fce 0 2024-07-03T15:44:16.925 INFO:tasks.workunit.client.0.smithi028.stdout:4/817: mknod d2/d3e/d9f/c10f 0 2024-07-03T15:44:16.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.922+0000 7f45918ab640 1 -- 172.21.15.28:0/1179849466 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f458c0fb250 msgr2=0x7f458c0fb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:16.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.922+0000 7f45918ab640 1 --2- 172.21.15.28:0/1179849466 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f458c0fb250 0x7f458c0fb660 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f457c0048d0 tx=0x7f457c009da0 comp rx=0 tx=0).stop 2024-07-03T15:44:16.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.923+0000 7f45918ab640 1 -- 172.21.15.28:0/1179849466 shutdown_connections 2024-07-03T15:44:16.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.923+0000 7f45918ab640 1 --2- 172.21.15.28:0/1179849466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f458c0fc520 0x7f458c0fc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:16.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.923+0000 7f45918ab640 1 --2- 172.21.15.28:0/1179849466 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f458c0fb250 0x7f458c0fb660 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:16.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.923+0000 7f45918ab640 1 -- 172.21.15.28:0/1179849466 >> 172.21.15.28:0/1179849466 conn(0x7f458c0f6a30 msgr2=0x7f458c0f8e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:44:16.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.923+0000 7f45918ab640 1 -- 172.21.15.28:0/1179849466 shutdown_connections 2024-07-03T15:44:16.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.923+0000 7f45918ab640 1 -- 172.21.15.28:0/1179849466 wait complete. 2024-07-03T15:44:16.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.924+0000 7f45918ab640 1 Processor -- start 2024-07-03T15:44:16.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.924+0000 7f45918ab640 1 -- start start 2024-07-03T15:44:16.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.924+0000 7f45918ab640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f458c0fb250 0x7f458c10bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:44:16.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.924+0000 7f45918ab640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f458c0fc520 0x7f458c10c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:44:16.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.925+0000 7f45918ab640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458c10d2e0 con 0x7f458c0fc520 2024-07-03T15:44:16.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.925+0000 7f45918ab640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458c10d450 con 0x7f458c0fb250 2024-07-03T15:44:16.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.925+0000 7f458a7fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f458c0fc520 0x7f458c10c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:44:16.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.925+0000 7f458a7fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f458c0fc520 0x7f458c10c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50458/0 (socket says 172.21.15.28:50458) 2024-07-03T15:44:16.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.925+0000 7f458a7fc640 1 -- 172.21.15.28:0/2834878974 learned_addr learned my addr 172.21.15.28:0/2834878974 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:44:16.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.925+0000 7f458affd640 1 --2- 172.21.15.28:0/2834878974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f458c0fb250 0x7f458c10bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:44:16.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.925+0000 7f458a7fc640 1 -- 172.21.15.28:0/2834878974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f458c0fb250 msgr2=0x7f458c10bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:16.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.925+0000 7f458a7fc640 1 --2- 172.21.15.28:0/2834878974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f458c0fb250 0x7f458c10bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:16.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.925+0000 7f458a7fc640 1 -- 172.21.15.28:0/2834878974 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f457c005040 con 0x7f458c0fc520 2024-07-03T15:44:16.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.925+0000 7f458affd640 1 --2- 172.21.15.28:0/2834878974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f458c0fb250 0x7f458c10bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-03T15:44:16.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.926+0000 7f458a7fc640 1 --2- 172.21.15.28:0/2834878974 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f458c0fc520 0x7f458c10c470 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f458000b560 tx=0x7f45800090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:44:16.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.926+0000 7f45737fe640 1 -- 172.21.15.28:0/2834878974 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f458000f6f0 con 0x7f458c0fc520 2024-07-03T15:44:16.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.926+0000 7f45737fe640 1 -- 172.21.15.28:0/2834878974 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f458000fd20 con 0x7f458c0fc520 2024-07-03T15:44:16.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.926+0000 7f45918ab640 1 -- 172.21.15.28:0/2834878974 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f458c10d5f0 con 0x7f458c0fc520 2024-07-03T15:44:16.930 INFO:tasks.workunit.client.0.smithi028.stdout:9/837: write d1/dd/f55 [2629788,52121] 0 2024-07-03T15:44:16.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.926+0000 7f45737fe640 1 -- 172.21.15.28:0/2834878974 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45800103a0 con 0x7f458c0fc520 2024-07-03T15:44:16.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.926+0000 7f45918ab640 1 -- 172.21.15.28:0/2834878974 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f458c10db90 con 0x7f458c0fc520 2024-07-03T15:44:16.931 INFO:tasks.workunit.client.0.smithi028.stdout:4/818: dread d2/d12/da6/fcc [0,4194304] 0 2024-07-03T15:44:16.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.928+0000 7f45918ab640 1 -- 172.21.15.28:0/2834878974 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f458c0fa6b0 con 0x7f458c0fc520 2024-07-03T15:44:16.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.929+0000 7f45737fe640 1 -- 172.21.15.28:0/2834878974 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 24) v1 ==== 88617+0+0 (secure 0 0 0) 0x7f4580011400 con 0x7f458c0fc520 2024-07-03T15:44:16.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.929+0000 7f45737fe640 1 --2- 172.21.15.28:0/2834878974 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f457406a6d0 0x7f457406cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:44:16.933 INFO:tasks.workunit.client.0.smithi028.stdout:4/819: symlink d2/d12/d3b/l110 0 2024-07-03T15:44:16.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.930+0000 7f45737fe640 1 -- 172.21.15.28:0/2834878974 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f4580088cd0 con 0x7f458c0fc520 2024-07-03T15:44:16.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.930+0000 7f458affd640 1 --2- 172.21.15.28:0/2834878974 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f457406a6d0 0x7f457406cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:44:16.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.930+0000 7f458affd640 1 --2- 172.21.15.28:0/2834878974 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f457406a6d0 0x7f457406cb80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f457c003080 tx=0x7f457c009da0 comp rx=0 tx=0).ready entity=mgr.24403 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:44:16.935 INFO:tasks.workunit.client.0.smithi028.stdout:4/820: getdents d2/d12/d42/d63/d97 0 2024-07-03T15:44:16.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:16.935+0000 7f45737fe640 1 -- 172.21.15.28:0/2834878974 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7f4580056540 con 0x7f458c0fc520 2024-07-03T15:44:16.943 INFO:tasks.workunit.client.0.smithi028.stdout:7/651: dwrite f0 [0,4194304] 0 2024-07-03T15:44:16.944 INFO:tasks.workunit.client.0.smithi028.stdout:9/838: dread d1/fa1 [0,4194304] 0 2024-07-03T15:44:16.948 INFO:tasks.workunit.client.0.smithi028.stdout:4/821: symlink d2/d12/d3b/l111 0 2024-07-03T15:44:16.950 INFO:tasks.workunit.client.0.smithi028.stdout:9/839: mkdir d1/d10/d28/d4e/d5b/df0/d123 0 2024-07-03T15:44:16.955 INFO:tasks.workunit.client.0.smithi028.stdout:4/822: symlink d2/d12/d3b/l112 0 2024-07-03T15:44:16.958 INFO:tasks.workunit.client.0.smithi028.stdout:4/823: mknod d2/d12/d2d/de9/c113 0 2024-07-03T15:44:16.960 INFO:tasks.workunit.client.0.smithi028.stdout:7/652: dwrite d9/d1c/d45/d71/d78/f9a [0,4194304] 0 2024-07-03T15:44:16.962 INFO:tasks.workunit.client.0.smithi028.stdout:4/824: mknod d2/d12/d5b/d78/c114 0 2024-07-03T15:44:16.971 INFO:tasks.workunit.client.0.smithi028.stdout:4/825: mkdir d2/d12/d2e/df3/dfb/d115 0 2024-07-03T15:44:16.973 INFO:tasks.workunit.client.0.smithi028.stdout:4/826: creat d2/d3e/d9f/dac/f116 x:0 0 0 2024-07-03T15:44:16.973 INFO:tasks.workunit.client.0.smithi028.stdout:4/827: rename d2/d12 to d2/d12/d42/d117 22 2024-07-03T15:44:16.973 INFO:tasks.workunit.client.0.smithi028.stdout:4/828: creat d2/d12/d2d/d58/f118 x:0 0 0 2024-07-03T15:44:16.975 INFO:tasks.workunit.client.0.smithi028.stdout:7/653: dwrite d9/d1c/d6e/fbc [0,4194304] 0 2024-07-03T15:44:16.975 INFO:tasks.workunit.client.0.smithi028.stdout:7/654: fsync d9/d1c/d7c/d8b/fb9 0 2024-07-03T15:44:16.980 INFO:tasks.workunit.client.0.smithi028.stdout:7/655: mkdir d9/d1c/da9/dc1/dc5 0 2024-07-03T15:44:16.981 INFO:tasks.workunit.client.0.smithi028.stdout:7/656: creat d9/d1c/d45/db5/d92/fc6 x:0 0 0 2024-07-03T15:44:16.982 INFO:tasks.workunit.client.0.smithi028.stdout:7/657: creat d9/d1c/d45/db5/d92/d99/daf/dc4/fc7 x:0 0 0 2024-07-03T15:44:16.987 INFO:tasks.workunit.client.0.smithi028.stdout:4/829: dwrite d2/d12/da6/fbf [0,4194304] 0 2024-07-03T15:44:16.987 INFO:tasks.workunit.client.0.smithi028.stdout:4/830: truncate d2/d12/d5b/d78/de4/f66 4546484 0 2024-07-03T15:44:16.988 INFO:tasks.workunit.client.0.smithi028.stdout:4/831: read d2/d3e/f3f [104696,130624] 0 2024-07-03T15:44:17.004 INFO:tasks.workunit.client.0.smithi028.stdout:7/658: dwrite d9/d1c/d86/fb7 [0,4194304] 0 2024-07-03T15:44:17.004 INFO:tasks.workunit.client.0.smithi028.stdout:4/832: dwrite d2/d12/d5b/f81 [0,4194304] 0 2024-07-03T15:44:17.012 INFO:tasks.workunit.client.0.smithi028.stdout:4/833: symlink d2/d12/d2c/l119 0 2024-07-03T15:44:17.012 INFO:tasks.workunit.client.0.smithi028.stdout:4/834: chown d2/d12/d5b/d78/de4 15 1 2024-07-03T15:44:17.021 INFO:tasks.workunit.client.0.smithi028.stdout:4/835: dread d2/f52 [4194304,4194304] 0 2024-07-03T15:44:17.021 INFO:tasks.workunit.client.0.smithi028.stdout:4/836: readlink d2/d12/d2c/l119 0 2024-07-03T15:44:17.023 INFO:tasks.workunit.client.0.smithi028.stdout:4/837: symlink d2/d12/d2d/de9/l11a 0 2024-07-03T15:44:17.026 INFO:tasks.workunit.client.0.smithi028.stdout:7/659: dwrite d9/d1c/d45/db5/d92/d99/d6a/f7b [0,4194304] 0 2024-07-03T15:44:17.026 INFO:tasks.workunit.client.0.smithi028.stdout:7/660: dread - d9/d1c/d7c/d8b/fb9 zero size 2024-07-03T15:44:17.026 INFO:tasks.workunit.client.0.smithi028.stdout:7/661: dread - d9/d1c/f93 zero size 2024-07-03T15:44:17.042 INFO:tasks.workunit.client.0.smithi028.stdout:4/838: dwrite d2/d3e/d9f/dac/f116 [0,4194304] 0 2024-07-03T15:44:17.045 INFO:tasks.workunit.client.0.smithi028.stdout:4/839: dread d2/d12/d2d/d58/fa3 [0,4194304] 0 2024-07-03T15:44:17.052 INFO:tasks.workunit.client.0.smithi028.stdout:4/840: unlink d2/f27 0 2024-07-03T15:44:17.052 INFO:tasks.workunit.client.0.smithi028.stdout:4/841: write d2/d5a/d45/d84/df4/d9d/faf [968970,8851] 0 2024-07-03T15:44:17.063 INFO:tasks.workunit.client.0.smithi028.stdout:4/842: dread d2/d12/ffe [0,4194304] 0 2024-07-03T15:44:17.069 INFO:tasks.workunit.client.0.smithi028.stdout:4/843: dread d2/d12/d2d/d58/f70 [0,4194304] 0 2024-07-03T15:44:17.069 INFO:tasks.workunit.client.0.smithi028.stdout:4/844: creat d2/d3e/dae/f11b x:0 0 0 2024-07-03T15:44:17.070 INFO:tasks.workunit.client.0.smithi028.stdout:4/845: symlink d2/d12/d5b/d98/l11c 0 2024-07-03T15:44:17.073 INFO:tasks.workunit.client.0.smithi028.stdout:4/846: link d2/d5a/d45/fb3 d2/d3e/dae/f11d 0 2024-07-03T15:44:17.076 INFO:tasks.workunit.client.0.smithi028.stdout:4/847: link d2/l95 d2/d12/d42/d63/da8/l11e 0 2024-07-03T15:44:17.079 INFO:tasks.workunit.client.0.smithi028.stdout:4/848: mknod d2/d12/d2e/df3/c11f 0 2024-07-03T15:44:17.085 INFO:tasks.workunit.client.0.smithi028.stdout:4/849: unlink d2/d12/d2d/l31 0 2024-07-03T15:44:17.087 INFO:tasks.workunit.client.0.smithi028.stdout:4/850: mkdir d2/d3e/d9f/d120 0 2024-07-03T15:44:17.089 INFO:tasks.workunit.client.0.smithi028.stdout:4/851: creat d2/d12/f121 x:0 0 0 2024-07-03T15:44:17.093 INFO:tasks.workunit.client.0.smithi028.stdout:4/852: getdents d2/d5a 0 2024-07-03T15:44:17.093 INFO:tasks.workunit.client.0.smithi028.stdout:4/853: readlink d2/d12/d42/d63/da8/ld4 0 2024-07-03T15:44:17.093 INFO:tasks.workunit.client.0.smithi028.stdout:4/854: stat d2/d5a/d45/d84/df4/d92 0 2024-07-03T15:44:17.095 INFO:tasks.workunit.client.0.smithi028.stdout:4/855: mknod d2/d12/d3b/d106/c122 0 2024-07-03T15:44:17.095 INFO:tasks.workunit.client.0.smithi028.stdout:4/856: dread - d2/d12/f121 zero size 2024-07-03T15:44:17.116 INFO:tasks.workunit.client.0.smithi028.stdout:4/857: dwrite d2/f52 [0,4194304] 0 2024-07-03T15:44:17.116 INFO:tasks.workunit.client.0.smithi028.stdout:4/858: chown d2/d3e/d9f/c6f 336338 1 2024-07-03T15:44:17.116 INFO:tasks.workunit.client.0.smithi028.stdout:4/859: chown d2/d5a/d45/d84/df4/f91 24202 1 2024-07-03T15:44:17.120 INFO:tasks.workunit.client.0.smithi028.stdout:4/860: rmdir d2/d12/d42/d63/d6d 39 2024-07-03T15:44:17.124 INFO:tasks.workunit.client.0.smithi028.stdout:4/861: creat d2/d12/d2d/db9/f123 x:0 0 0 2024-07-03T15:44:17.128 INFO:tasks.workunit.client.0.smithi028.stdout:2/592: sync 2024-07-03T15:44:17.129 INFO:tasks.workunit.client.0.smithi028.stdout:5/750: sync 2024-07-03T15:44:17.129 INFO:tasks.workunit.client.0.smithi028.stdout:5/751: write d0/dd/d13/d29/dc4/fbe [652658,79521] 0 2024-07-03T15:44:17.135 INFO:tasks.workunit.client.0.smithi028.stdout:4/862: creat d2/d3e/f124 x:0 0 0 2024-07-03T15:44:17.135 INFO:tasks.workunit.client.0.smithi028.stdout:4/863: write d2/d5a/d45/d84/fe1 [231657,75978] 0 2024-07-03T15:44:17.139 INFO:tasks.workunit.client.0.smithi028.stdout:2/593: creat da/d42/d47/d78/fcd x:0 0 0 2024-07-03T15:44:17.141 INFO:tasks.workunit.client.0.smithi028.stdout:5/752: mknod d0/dd/d13/d57/d30/d70/cf6 0 2024-07-03T15:44:17.141 INFO:tasks.workunit.client.0.smithi028.stdout:2/594: dread da/d24/fa4 [0,4194304] 0 2024-07-03T15:44:17.152 INFO:tasks.workunit.client.0.smithi028.stdout:5/753: creat d0/dd/d1e/d3b/de0/ff7 x:0 0 0 2024-07-03T15:44:17.153 INFO:tasks.workunit.client.0.smithi028.stdout:2/595: creat da/d23/d6f/d84/db8/fce x:0 0 0 2024-07-03T15:44:17.153 INFO:tasks.workunit.client.0.smithi028.stdout:2/596: write da/d42/f50 [645540,54434] 0 2024-07-03T15:44:17.155 INFO:tasks.workunit.client.0.smithi028.stdout:5/754: mkdir d0/dd/d1e/d86/dec/df8 0 2024-07-03T15:44:17.155 INFO:tasks.workunit.client.0.smithi028.stdout:5/755: truncate d0/dd/d1e/d3b/de0/ff7 338620 0 2024-07-03T15:44:17.157 INFO:tasks.workunit.client.0.smithi028.stdout:2/597: rename da/d24/d2c/c4c to da/d1e/d68/ccf 0 2024-07-03T15:44:17.157 INFO:tasks.workunit.client.0.smithi028.stdout:2/598: stat da/d23/d6f/d84/c87 0 2024-07-03T15:44:17.159 INFO:tasks.workunit.client.0.smithi028.stdout:5/756: creat d0/dd/d13/d57/d26/ddf/ff9 x:0 0 0 2024-07-03T15:44:17.159 INFO:tasks.workunit.client.0.smithi028.stdout:5/757: stat d0/dd/d13/d47/c60 0 2024-07-03T15:44:17.173 INFO:tasks.workunit.client.0.smithi028.stdout:5/758: dwrite d0/dd/d1e/d8e/fc6 [0,4194304] 0 2024-07-03T15:44:17.177 INFO:tasks.workunit.client.0.smithi028.stdout:2/599: mkdir da/d1e/dd0 0 2024-07-03T15:44:17.179 INFO:tasks.workunit.client.0.smithi028.stdout:2/600: unlink da/d23/d30/f5c 0 2024-07-03T15:44:17.182 INFO:tasks.workunit.client.0.smithi028.stdout:5/759: dread d0/dd/d13/d57/d30/f90 [4194304,4194304] 0 2024-07-03T15:44:17.183 INFO:tasks.workunit.client.0.smithi028.stdout:2/601: rmdir da/d24 39 2024-07-03T15:44:17.183 INFO:tasks.workunit.client.0.smithi028.stdout:5/760: stat d0/dd/d1e/d3b/d3f/c5c 0 2024-07-03T15:44:17.185 INFO:tasks.workunit.client.0.smithi028.stdout:2/602: creat da/d42/d47/fd1 x:0 0 0 2024-07-03T15:44:17.186 INFO:tasks.workunit.client.0.smithi028.stdout:5/761: mkdir d0/dd/d1e/d3b/de0/dfa 0 2024-07-03T15:44:17.187 INFO:tasks.workunit.client.0.smithi028.stdout:2/603: dread da/d1e/f58 [0,4194304] 0 2024-07-03T15:44:17.189 INFO:tasks.workunit.client.0.smithi028.stdout:5/762: creat d0/dd/d1e/d3b/d3f/d5a/ffb x:0 0 0 2024-07-03T15:44:17.189 INFO:tasks.workunit.client.0.smithi028.stdout:5/763: chown d0/dd/df/da1/daa 16373305 1 2024-07-03T15:44:17.189 INFO:tasks.workunit.client.0.smithi028.stdout:5/764: chown d0/dd/d1e/l9a 373655139 1 2024-07-03T15:44:17.190 INFO:tasks.workunit.client.0.smithi028.stdout:5/765: dread - d0/dd/d14/fea zero size 2024-07-03T15:44:17.190 INFO:tasks.workunit.client.0.smithi028.stdout:5/766: write d0/dd/d98/f6c [683825,92568] 0 2024-07-03T15:44:17.190 INFO:tasks.workunit.client.0.smithi028.stdout:5/767: creat d0/dd/d13/d57/d26/d41/ffc x:0 0 0 2024-07-03T15:44:17.190 INFO:tasks.workunit.client.0.smithi028.stdout:5/768: dread - d0/dd/d13/d57/da2/fab zero size 2024-07-03T15:44:17.196 INFO:tasks.workunit.client.0.smithi028.stdout:5/769: creat d0/dd/d13/ffd x:0 0 0 2024-07-03T15:44:17.206 INFO:tasks.workunit.client.0.smithi028.stdout:5/770: dread d0/dd/d13/f83 [0,4194304] 0 2024-07-03T15:44:17.206 INFO:tasks.workunit.client.0.smithi028.stdout:5/771: write d0/dd/d1e/d3b/d3f/d71/f81 [1901530,31228] 0 2024-07-03T15:44:17.207 INFO:tasks.workunit.client.0.smithi028.stdout:5/772: truncate d0/dd/d1e/f89 849118 0 2024-07-03T15:44:17.208 INFO:tasks.workunit.client.0.smithi028.stdout:2/604: dwrite da/d42/fa6 [0,4194304] 0 2024-07-03T15:44:17.209 INFO:tasks.workunit.client.0.smithi028.stdout:5/773: creat d0/ffe x:0 0 0 2024-07-03T15:44:17.209 INFO:tasks.workunit.client.0.smithi028.stdout:5/774: fsync d0/dd/d13/d29/dc4/fa9 0 2024-07-03T15:44:17.214 INFO:tasks.workunit.client.0.smithi028.stdout:2/605: read - da/d42/d47/d78/f9d zero size 2024-07-03T15:44:17.216 INFO:tasks.workunit.client.0.smithi028.stdout:5/775: unlink d0/dd/d1e/d3b/d3f/c5c 0 2024-07-03T15:44:17.217 INFO:tasks.workunit.client.0.smithi028.stdout:2/606: getdents da/d42/d47 0 2024-07-03T15:44:17.217 INFO:tasks.workunit.client.0.smithi028.stdout:2/607: chown da/d42/c5d 24088332 1 2024-07-03T15:44:17.221 INFO:tasks.workunit.client.0.smithi028.stdout:5/776: rmdir d0/dd/d1e/d3b/ddd/df0 0 2024-07-03T15:44:17.236 INFO:tasks.workunit.client.0.smithi028.stdout:2/608: dwrite da/f10 [0,4194304] 0 2024-07-03T15:44:17.236 INFO:tasks.workunit.client.0.smithi028.stdout:2/609: creat da/d23/d30/fd2 x:0 0 0 2024-07-03T15:44:17.238 INFO:tasks.workunit.client.0.smithi028.stdout:2/610: getdents da/d24/d2c 0 2024-07-03T15:44:17.239 INFO:tasks.workunit.client.0.smithi028.stdout:5/777: dwrite d0/dd/d13/d29/f4e [4194304,4194304] 0 2024-07-03T15:44:17.239 INFO:tasks.workunit.client.0.smithi028.stdout:5/778: fdatasync d0/dd/d13/d57/d30/fc5 0 2024-07-03T15:44:17.240 INFO:tasks.workunit.client.0.smithi028.stdout:2/611: mknod da/d42/d47/da2/cd3 0 2024-07-03T15:44:17.243 INFO:tasks.workunit.client.0.smithi028.stdout:2/612: link da/d42/d47/fbc da/d1e/dd0/fd4 0 2024-07-03T15:44:17.243 INFO:tasks.workunit.client.0.smithi028.stdout:2/613: write da/d23/f96 [647788,48071] 0 2024-07-03T15:44:17.245 INFO:tasks.workunit.client.0.smithi028.stdout:2/614: creat da/d24/d2c/dc6/fd5 x:0 0 0 2024-07-03T15:44:17.249 INFO:tasks.workunit.client.0.smithi028.stdout:5/779: read d0/dd/d1e/d3b/f5e [40235,127064] 0 2024-07-03T15:44:17.249 INFO:tasks.workunit.client.0.smithi028.stdout:2/615: link da/d1e/f7b da/d24/d2c/dba/fd6 0 2024-07-03T15:44:17.253 INFO:tasks.workunit.client.0.smithi028.stdout:5/780: mkdir d0/d8f/db5/ddc/dff 0 2024-07-03T15:44:17.255 INFO:tasks.workunit.client.0.smithi028.stdout:2/616: mknod da/d42/d9e/cd7 0 2024-07-03T15:44:17.256 INFO:tasks.workunit.client.0.smithi028.stdout:5/781: creat d0/dd/d13/d57/d30/d70/d7f/dbb/f100 x:0 0 0 2024-07-03T15:44:17.256 INFO:tasks.workunit.client.0.smithi028.stdout:5/782: readlink d0/l37 0 2024-07-03T15:44:17.260 INFO:tasks.workunit.client.0.smithi028.stdout:2/617: rmdir da/d24/dc1 0 2024-07-03T15:44:17.260 INFO:tasks.workunit.client.0.smithi028.stdout:2/618: stat da/d23/d6f/cc2 0 2024-07-03T15:44:17.271 INFO:tasks.workunit.client.0.smithi028.stdout:6/833: sync 2024-07-03T15:44:17.271 INFO:tasks.workunit.client.0.smithi028.stdout:8/853: sync 2024-07-03T15:44:17.271 INFO:tasks.workunit.client.0.smithi028.stdout:0/801: sync 2024-07-03T15:44:17.272 INFO:tasks.workunit.client.0.smithi028.stdout:9/840: sync 2024-07-03T15:44:17.273 INFO:tasks.workunit.client.0.smithi028.stdout:3/735: sync 2024-07-03T15:44:17.273 INFO:tasks.workunit.client.0.smithi028.stdout:3/736: chown d18/d43/fce 288744 1 2024-07-03T15:44:17.273 INFO:tasks.workunit.client.0.smithi028.stdout:3/737: stat d18/d1a/d64/ddc 0 2024-07-03T15:44:17.273 INFO:tasks.workunit.client.0.smithi028.stdout:3/738: write d18/d3a/d9d/d8c/f92 [4976568,21419] 0 2024-07-03T15:44:17.273 INFO:tasks.workunit.client.0.smithi028.stdout:6/834: truncate d4/d78/d10c/d1c/fb1 2302445 0 2024-07-03T15:44:17.275 INFO:tasks.workunit.client.0.smithi028.stdout:8/854: creat d0/d41/d50/d3/d35/f11a x:0 0 0 2024-07-03T15:44:17.276 INFO:tasks.workunit.client.0.smithi028.stdout:0/802: rename d2/d3/dc4/d58/f5f to d2/d3/dc4/d58/d6e/ff6 0 2024-07-03T15:44:17.278 INFO:tasks.workunit.client.0.smithi028.stdout:6/835: mkdir d4/d78/d10c/d47/d9f/d124 0 2024-07-03T15:44:17.279 INFO:tasks.workunit.client.0.smithi028.stdout:8/855: symlink d0/d41/d50/d3/dc/d33/dbe/l11b 0 2024-07-03T15:44:17.279 INFO:tasks.workunit.client.0.smithi028.stdout:8/856: chown d0/d41/d50/d3/d35/d86 1745 1 2024-07-03T15:44:17.279 INFO:tasks.workunit.client.0.smithi028.stdout:8/857: chown d0/d41/c52 24927041 1 2024-07-03T15:44:17.279 INFO:tasks.workunit.client.0.smithi028.stdout:6/836: dread d4/d78/d10c/d38/d60/db5/fd0 [0,4194304] 0 2024-07-03T15:44:17.280 INFO:tasks.workunit.client.0.smithi028.stdout:0/803: unlink d2/d3/dc4/fd4 0 2024-07-03T15:44:17.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.276+0000 7f45918ab640 1 -- 172.21.15.28:0/2834878974 --> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f458c10dfe0 con 0x7f457406a6d0 2024-07-03T15:44:17.281 INFO:tasks.workunit.client.0.smithi028.stdout:5/783: dwrite d0/dd/d13/d47/d50/f8c [0,4194304] 0 2024-07-03T15:44:17.281 INFO:tasks.workunit.client.0.smithi028.stdout:8/858: creat d0/d41/d50/daf/dc8/da6/da7/f11c x:0 0 0 2024-07-03T15:44:17.282 INFO:tasks.workunit.client.0.smithi028.stdout:6/837: rename d4/d78/d10c/cd to d4/d12/dc3/c125 0 2024-07-03T15:44:17.283 INFO:tasks.workunit.client.0.smithi028.stdout:2/619: dwrite da/d23/d6f/d84/fa7 [0,4194304] 0 2024-07-03T15:44:17.283 INFO:tasks.workunit.client.0.smithi028.stdout:2/620: dread - da/d1e/fc9 zero size 2024-07-03T15:44:17.285 INFO:tasks.workunit.client.0.smithi028.stdout:0/804: truncate d2/d3/d4d/d56/d78/da8/dd7/fe2 66839 0 2024-07-03T15:44:17.288 INFO:tasks.workunit.client.0.smithi028.stdout:5/784: creat d0/dd/d13/d57/da2/de9/f101 x:0 0 0 2024-07-03T15:44:17.288 INFO:tasks.workunit.client.0.smithi028.stdout:5/785: write d0/dd/d1e/d8e/fb3 [551827,119530] 0 2024-07-03T15:44:17.291 INFO:tasks.workunit.client.0.smithi028.stdout:0/805: dread d2/d3/dc4/d36/f88 [0,4194304] 0 2024-07-03T15:44:17.291 INFO:tasks.workunit.client.0.smithi028.stdout:0/806: write d2/d3/f2f [4207004,77459] 0 2024-07-03T15:44:17.294 INFO:tasks.workunit.client.0.smithi028.stdout:8/859: write d0/d41/d50/d3/dc/f2b [4054014,65623] 0 2024-07-03T15:44:17.295 INFO:tasks.workunit.client.0.smithi028.stdout:5/786: rename d0/dd/d1e/d86/dec/df8 to d0/dd/d1e/d8e/de4/d102 0 2024-07-03T15:44:17.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.291+0000 7f45737fe640 1 -- 172.21.15.28:0/2834878974 <== mgr.24403 v2:172.21.15.148:6828/4285739175 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f458c10dfe0 con 0x7f457406a6d0 2024-07-03T15:44:17.296 INFO:teuthology.orchestra.run.smithi028.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-03T15:44:17.296 INFO:teuthology.orchestra.run.smithi028.stdout:alertmanager.smithi028 smithi028 *:9093,9094 running (6m) 2m ago 7m 24.9M - 0.25.0 c8568f914cd2 b9a1a484a0a2 2024-07-03T15:44:17.296 INFO:teuthology.orchestra.run.smithi028.stdout:crash.smithi028 smithi028 running (8m) 2m ago 8m 6899k - 17.2.7-1047-gd4b063ab 4ca399d9e65d 535660e448f9 2024-07-03T15:44:17.297 INFO:teuthology.orchestra.run.smithi028.stdout:crash.smithi148 smithi148 running (7m) 9s ago 7m 6899k - 17.2.7-1047-gd4b063ab 4ca399d9e65d 4bb7675895e7 2024-07-03T15:44:17.297 INFO:teuthology.orchestra.run.smithi028.stdout:grafana.smithi028 smithi028 *:3000 running (6m) 2m ago 7m 88.5M - 9.4.7 954c08fa6188 07e0ae3c15aa 2024-07-03T15:44:17.297 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi028.rhpvve smithi028 running (2m) 2m ago 2m 15.6M - 17.2.7-1047-gd4b063ab 4ca399d9e65d 3ed5abd02cfb 2024-07-03T15:44:17.297 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi028.wmbxew smithi028 running (2m) 2m ago 2m 14.9M - 17.2.7-1047-gd4b063ab 4ca399d9e65d b3bf736998f9 2024-07-03T15:44:17.297 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi148.szomfl smithi148 running (2m) 9s ago 2m 13.9M - 17.2.7-1047-gd4b063ab 4ca399d9e65d d94f5c95fa8d 2024-07-03T15:44:17.297 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi148.xgpuld smithi148 running (2m) 9s ago 2m 11.9M - 17.2.7-1047-gd4b063ab 4ca399d9e65d fc70360a8de0 2024-07-03T15:44:17.297 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.smithi028.vtqrtz smithi028 *:9283 running (9m) 2m ago 9m 447M - 17.2.7-1047-gd4b063ab 4ca399d9e65d 0d26e74cb154 2024-07-03T15:44:17.297 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.smithi148.rjzsar smithi148 *:8443,9283 running (29s) 9s ago 6m 477M - 19.0.0-2550-gc00506dc 9d49405603e8 1fe24bbddd1a 2024-07-03T15:44:17.297 INFO:teuthology.orchestra.run.smithi028.stdout:mon.smithi028 smithi028 running (9m) 2m ago 9m 43.2M 2048M 17.2.7-1047-gd4b063ab 4ca399d9e65d 7461f9bef4e0 2024-07-03T15:44:17.297 INFO:teuthology.orchestra.run.smithi028.stdout:mon.smithi148 smithi148 running (6m) 9s ago 6m 37.6M 2048M 17.2.7-1047-gd4b063ab 4ca399d9e65d 4c6c8ed2ab85 2024-07-03T15:44:17.297 INFO:teuthology.orchestra.run.smithi028.stdout:node-exporter.smithi028 smithi028 *:9100 running (8m) 2m ago 8m 16.0M - 1.5.0 0da6a335fe13 68318a0ca520 2024-07-03T15:44:17.297 INFO:teuthology.orchestra.run.smithi028.stdout:node-exporter.smithi148 smithi148 *:9100 running (7m) 9s ago 7m 17.1M - 1.5.0 0da6a335fe13 4994e60bd57c 2024-07-03T15:44:17.297 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0 smithi028 running (6m) 2m ago 6m 48.5M 4289M 17.2.7-1047-gd4b063ab 4ca399d9e65d b18089a439ba 2024-07-03T15:44:17.298 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1 smithi028 running (5m) 2m ago 5m 45.5M 4289M 17.2.7-1047-gd4b063ab 4ca399d9e65d 87d415dde231 2024-07-03T15:44:17.298 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2 smithi028 running (5m) 2m ago 5m 46.3M 4289M 17.2.7-1047-gd4b063ab 4ca399d9e65d b2cbb312212d 2024-07-03T15:44:17.298 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3 smithi148 running (4m) 9s ago 4m 274M 2581M 17.2.7-1047-gd4b063ab 4ca399d9e65d 5be69a27df31 2024-07-03T15:44:17.298 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4 smithi148 running (4m) 9s ago 4m 219M 2581M 17.2.7-1047-gd4b063ab 4ca399d9e65d 54f7c26c77ca 2024-07-03T15:44:17.298 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5 smithi148 running (4m) 9s ago 4m 279M 2581M 17.2.7-1047-gd4b063ab 4ca399d9e65d d185c6905384 2024-07-03T15:44:17.298 INFO:teuthology.orchestra.run.smithi028.stdout:prometheus.smithi028 smithi028 *:9095 running (6m) 2m ago 7m 54.2M - 2.43.0 a07b618ecd1d 97f838e118a3 2024-07-03T15:44:17.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.295+0000 7f45918ab640 1 -- 172.21.15.28:0/2834878974 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f457406a6d0 msgr2=0x7f457406cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:17.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.295+0000 7f45918ab640 1 --2- 172.21.15.28:0/2834878974 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f457406a6d0 0x7f457406cb80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f457c003080 tx=0x7f457c009da0 comp rx=0 tx=0).stop 2024-07-03T15:44:17.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.295+0000 7f45918ab640 1 -- 172.21.15.28:0/2834878974 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f458c0fc520 msgr2=0x7f458c10c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:17.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.295+0000 7f45918ab640 1 --2- 172.21.15.28:0/2834878974 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f458c0fc520 0x7f458c10c470 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f458000b560 tx=0x7f45800090f0 comp rx=0 tx=0).stop 2024-07-03T15:44:17.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.296+0000 7f45918ab640 1 -- 172.21.15.28:0/2834878974 shutdown_connections 2024-07-03T15:44:17.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.296+0000 7f45918ab640 1 --2- 172.21.15.28:0/2834878974 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f457406a6d0 0x7f457406cb80 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:17.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.296+0000 7f45918ab640 1 --2- 172.21.15.28:0/2834878974 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f458c0fc520 0x7f458c10c470 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:17.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.296+0000 7f45918ab640 1 --2- 172.21.15.28:0/2834878974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f458c0fb250 0x7f458c10bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:17.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.296+0000 7f45918ab640 1 -- 172.21.15.28:0/2834878974 >> 172.21.15.28:0/2834878974 conn(0x7f458c0f6a30 msgr2=0x7f458c0f8160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:44:17.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.296+0000 7f45918ab640 1 -- 172.21.15.28:0/2834878974 shutdown_connections 2024-07-03T15:44:17.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.296+0000 7f45918ab640 1 -- 172.21.15.28:0/2834878974 wait complete. 2024-07-03T15:44:17.302 INFO:tasks.workunit.client.0.smithi028.stdout:6/838: dread d4/f33 [0,4194304] 0 2024-07-03T15:44:17.302 INFO:tasks.workunit.client.0.smithi028.stdout:5/787: truncate d0/dd/fae 2055693 0 2024-07-03T15:44:17.304 INFO:tasks.workunit.client.0.smithi028.stdout:9/841: dwrite d1/d10/d28/d45/d61/d8c/f81 [4194304,4194304] 0 2024-07-03T15:44:17.305 INFO:tasks.workunit.client.0.smithi028.stdout:3/739: dwrite d18/d1a/d1e/f3e [0,4194304] 0 2024-07-03T15:44:17.305 INFO:tasks.workunit.client.0.smithi028.stdout:3/740: fsync d18/d1a/fc8 0 2024-07-03T15:44:17.305 INFO:tasks.workunit.client.0.smithi028.stdout:6/839: creat d4/d78/d10c/d47/d9f/f126 x:0 0 0 2024-07-03T15:44:17.307 INFO:tasks.workunit.client.0.smithi028.stdout:5/788: rename d0/d8f/feb to d0/dd/d13/d57/d30/d70/d7f/dbb/f103 0 2024-07-03T15:44:17.314 INFO:tasks.workunit.client.0.smithi028.stdout:0/807: write d2/d3/dc4/d36/d45/f7b [3709216,89874] 0 2024-07-03T15:44:17.315 INFO:tasks.workunit.client.0.smithi028.stdout:2/621: dwrite da/d1e/f45 [0,4194304] 0 2024-07-03T15:44:17.318 INFO:tasks.workunit.client.0.smithi028.stdout:9/842: write d1/d10/d28/d45/d61/d8c/d77/f91 [1780392,1437] 0 2024-07-03T15:44:17.322 INFO:tasks.workunit.client.0.smithi028.stdout:6/840: mknod d4/d78/d10c/d47/d9f/c127 0 2024-07-03T15:44:17.323 INFO:tasks.workunit.client.0.smithi028.stdout:6/841: creat d4/d78/d10c/d47/d9f/f128 x:0 0 0 2024-07-03T15:44:17.330 INFO:tasks.workunit.client.0.smithi028.stdout:3/741: dwrite d18/d6f/d78/fea [0,4194304] 0 2024-07-03T15:44:17.330 INFO:tasks.workunit.client.0.smithi028.stdout:3/742: creat d18/d3a/d9a/fee x:0 0 0 2024-07-03T15:44:17.330 INFO:tasks.workunit.client.0.smithi028.stdout:3/743: write d18/d1a/d5e/d66/da3/fac [493336,91720] 0 2024-07-03T15:44:17.340 INFO:tasks.workunit.client.0.smithi028.stdout:2/622: symlink da/d42/d47/da2/ld8 0 2024-07-03T15:44:17.341 INFO:tasks.workunit.client.0.smithi028.stdout:2/623: fdatasync da/d42/d47/fa5 0 2024-07-03T15:44:17.341 INFO:tasks.workunit.client.0.smithi028.stdout:3/744: write d18/d1a/fd1 [1729874,93692] 0 2024-07-03T15:44:17.342 INFO:tasks.workunit.client.0.smithi028.stdout:0/808: dwrite d2/d3/dc4/d36/f65 [0,4194304] 0 2024-07-03T15:44:17.342 INFO:tasks.workunit.client.0.smithi028.stdout:0/809: stat d2/d1c/d6c/dc5/fe3 0 2024-07-03T15:44:17.342 INFO:tasks.workunit.client.0.smithi028.stdout:0/810: fdatasync d2/d3/dc4/d58/d82/fb6 0 2024-07-03T15:44:17.343 INFO:tasks.workunit.client.0.smithi028.stdout:0/811: stat d2/d3/dc4/d36/d45 0 2024-07-03T15:44:17.343 INFO:tasks.workunit.client.0.smithi028.stdout:0/812: fsync d2/d3/dc4/d36/d45/f72 0 2024-07-03T15:44:17.343 INFO:tasks.workunit.client.0.smithi028.stdout:5/789: dwrite d0/dd/df/f52 [0,4194304] 0 2024-07-03T15:44:17.343 INFO:tasks.workunit.client.0.smithi028.stdout:5/790: truncate d0/dd/fcd 93988 0 2024-07-03T15:44:17.346 INFO:tasks.workunit.client.0.smithi028.stdout:6/842: unlink d4/d78/d10c/d1c/c95 0 2024-07-03T15:44:17.346 INFO:tasks.workunit.client.0.smithi028.stdout:6/843: creat d4/d15/de5/f129 x:0 0 0 2024-07-03T15:44:17.353 INFO:tasks.workunit.client.0.smithi028.stdout:9/843: getdents d1/d8e 0 2024-07-03T15:44:17.355 INFO:tasks.workunit.client.0.smithi028.stdout:2/624: dwrite da/d24/f55 [0,4194304] 0 2024-07-03T15:44:17.357 INFO:tasks.workunit.client.0.smithi028.stdout:3/745: symlink d18/d43/lef 0 2024-07-03T15:44:17.357 INFO:tasks.workunit.client.0.smithi028.stdout:3/746: creat d18/d6f/d78/ff0 x:0 0 0 2024-07-03T15:44:17.363 INFO:tasks.workunit.client.0.smithi028.stdout:0/813: mkdir d2/d3/d4d/d56/d78/da8/dd0/df7 0 2024-07-03T15:44:17.372 INFO:tasks.workunit.client.0.smithi028.stdout:5/791: rename d0/dd/d13/d57/d30/d70/d7f/dbb/f100 to d0/dd/df/da7/f104 0 2024-07-03T15:44:17.373 INFO:tasks.workunit.client.0.smithi028.stdout:3/747: dwrite d18/d1a/d1e/f53 [4194304,4194304] 0 2024-07-03T15:44:17.375 INFO:tasks.workunit.client.0.smithi028.stdout:6/844: link d4/c86 d4/d78/d10c/d47/d9f/c12a 0 2024-07-03T15:44:17.378 INFO:tasks.workunit.client.0.smithi028.stdout:2/625: link da/d23/d6f/cc2 da/d1e/d56/cd9 0 2024-07-03T15:44:17.384 INFO:tasks.workunit.client.0.smithi028.stdout:3/748: write d18/d1a/d1e/f3e [2992872,121258] 0 2024-07-03T15:44:17.384 INFO:tasks.workunit.client.0.smithi028.stdout:3/749: chown d18/d43/l69 314707 1 2024-07-03T15:44:17.384 INFO:tasks.workunit.client.0.smithi028.stdout:3/750: chown d18/d1a/d1e/l6a 365607149 1 2024-07-03T15:44:17.385 INFO:tasks.workunit.client.0.smithi028.stdout:3/751: readlink d18/l34 0 2024-07-03T15:44:17.385 INFO:tasks.workunit.client.0.smithi028.stdout:3/752: getdents d18/d3a/dc1/dd0 0 2024-07-03T15:44:17.385 INFO:tasks.workunit.client.0.smithi028.stdout:3/753: write d18/d3a/d9a/fcf [1896985,76092] 0 2024-07-03T15:44:17.385 INFO:tasks.workunit.client.0.smithi028.stdout:0/814: symlink d2/d3/d4d/d56/d78/dcf/lf8 0 2024-07-03T15:44:17.385 INFO:tasks.workunit.client.0.smithi028.stdout:0/815: dread - d2/d3/d4d/d56/d78/fa1 zero size 2024-07-03T15:44:17.385 INFO:tasks.workunit.client.0.smithi028.stdout:0/816: truncate d2/d1c/f76 906580 0 2024-07-03T15:44:17.394 INFO:tasks.workunit.client.0.smithi028.stdout:5/792: unlink d0/dd/d1e/d3b/fa3 0 2024-07-03T15:44:17.394 INFO:tasks.workunit.client.0.smithi028.stdout:5/793: read - d0/dd/d13/d57/da2/de9/f101 zero size 2024-07-03T15:44:17.398 INFO:tasks.workunit.client.0.smithi028.stdout:6/845: creat d4/d78/d10c/d47/f12b x:0 0 0 2024-07-03T15:44:17.400 INFO:tasks.workunit.client.0.smithi028.stdout:9/844: mknod d1/d10/d28/d45/c124 0 2024-07-03T15:44:17.402 INFO:tasks.workunit.client.0.smithi028.stdout:9/845: dread d1/d10/f3e [0,4194304] 0 2024-07-03T15:44:17.405 INFO:tasks.workunit.client.0.smithi028.stdout:3/754: creat d18/d1a/d1e/d58/d9f/de2/ff1 x:0 0 0 2024-07-03T15:44:17.407 INFO:tasks.workunit.client.0.smithi028.stdout:0/817: creat d2/d3/d4d/d56/d78/da8/dd0/df7/ff9 x:0 0 0 2024-07-03T15:44:17.407 INFO:tasks.workunit.client.0.smithi028.stdout:0/818: fdatasync d2/d1c/f68 0 2024-07-03T15:44:17.407 INFO:tasks.workunit.client.0.smithi028.stdout:0/819: chown d2/d1c/c33 182 1 2024-07-03T15:44:17.407 INFO:tasks.workunit.client.0.smithi028.stdout:0/820: chown d2/d3/dc4/l62 3023826 1 2024-07-03T15:44:17.409 INFO:tasks.workunit.client.0.smithi028.stdout:5/794: creat d0/dd/d14/dee/f105 x:0 0 0 2024-07-03T15:44:17.410 INFO:tasks.workunit.client.0.smithi028.stdout:6/846: dread d4/d12/fa9 [0,4194304] 0 2024-07-03T15:44:17.411 INFO:tasks.workunit.client.0.smithi028.stdout:6/847: readlink d4/d78/d10c/d13/dbd/lc2 0 2024-07-03T15:44:17.411 INFO:tasks.workunit.client.0.smithi028.stdout:2/626: symlink da/d1e/lda 0 2024-07-03T15:44:17.414 INFO:tasks.workunit.client.0.smithi028.stdout:9/846: rename d1/dd/f4c to d1/d10/d28/d45/d56/d11a/f125 0 2024-07-03T15:44:17.417 INFO:tasks.workunit.client.0.smithi028.stdout:0/821: dread d2/f3b [0,4194304] 0 2024-07-03T15:44:17.417 INFO:tasks.workunit.client.0.smithi028.stdout:0/822: truncate d2/d3/d4d/d56/d78/da8/dd0/feb 4720512 0 2024-07-03T15:44:17.417 INFO:tasks.workunit.client.0.smithi028.stdout:0/823: chown d2/d3/d4d/d56/d78/dbe 146626920 1 2024-07-03T15:44:17.418 INFO:tasks.workunit.client.0.smithi028.stdout:5/795: mkdir d0/dd/d13/d57/d26/d41/d106 0 2024-07-03T15:44:17.420 INFO:tasks.workunit.client.0.smithi028.stdout:6/848: mkdir d4/d12/d2c/d31/df5/d12c 0 2024-07-03T15:44:17.422 INFO:tasks.workunit.client.0.smithi028.stdout:9/847: unlink d1/d10/d28/d45/fdd 0 2024-07-03T15:44:17.425 INFO:tasks.workunit.client.0.smithi028.stdout:9/848: unlink d1/d16/d57/d5f/d64/ff3 0 2024-07-03T15:44:17.425 INFO:tasks.workunit.client.0.smithi028.stdout:3/755: dwrite d18/d1a/d1e/f38 [4194304,4194304] 0 2024-07-03T15:44:17.425 INFO:tasks.workunit.client.0.smithi028.stdout:3/756: chown d18/d3a/c4a 233283995 1 2024-07-03T15:44:17.425 INFO:tasks.workunit.client.0.smithi028.stdout:7/662: sync 2024-07-03T15:44:17.431 INFO:tasks.workunit.client.0.smithi028.stdout:9/849: rmdir d1/d10/d28/d4e/d5b/df0/d123 0 2024-07-03T15:44:17.433 INFO:tasks.workunit.client.0.smithi028.stdout:3/757: creat d18/d6f/ff2 x:0 0 0 2024-07-03T15:44:17.433 INFO:tasks.workunit.client.0.smithi028.stdout:3/758: fsync d18/d1a/d1e/d58/d9f/da9/fe4 0 2024-07-03T15:44:17.433 INFO:tasks.workunit.client.0.smithi028.stdout:3/759: fdatasync d18/d1a/d1e/f41 0 2024-07-03T15:44:17.433 INFO:tasks.workunit.client.0.smithi028.stdout:3/760: truncate d18/d3a/dc1/fd3 486546 0 2024-07-03T15:44:17.434 INFO:tasks.workunit.client.0.smithi028.stdout:7/663: creat d9/d1c/d45/db5/d5d/fc8 x:0 0 0 2024-07-03T15:44:17.434 INFO:tasks.workunit.client.0.smithi028.stdout:7/664: write d9/fa3 [785440,52452] 0 2024-07-03T15:44:17.437 INFO:tasks.workunit.client.0.smithi028.stdout:9/850: rename d1/d10/d28/d45/d56/laf to d1/d10/d28/l126 0 2024-07-03T15:44:17.440 INFO:tasks.workunit.client.0.smithi028.stdout:7/665: dread d9/d1c/d45/db5/f5e [0,4194304] 0 2024-07-03T15:44:17.440 INFO:tasks.workunit.client.0.smithi028.stdout:5/796: dread d0/f9 [4194304,4194304] 0 2024-07-03T15:44:17.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.437+0000 7f598999d640 1 -- 172.21.15.28:0/102439850 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59840ff850 msgr2=0x7f59840ffcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:17.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.437+0000 7f598999d640 1 --2- 172.21.15.28:0/102439850 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59840ff850 0x7f59840ffcc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f59780048d0 tx=0x7f5978009da0 comp rx=0 tx=0).stop 2024-07-03T15:44:17.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.437+0000 7f598999d640 1 -- 172.21.15.28:0/102439850 shutdown_connections 2024-07-03T15:44:17.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.437+0000 7f598999d640 1 --2- 172.21.15.28:0/102439850 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59840ff850 0x7f59840ffcc0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:17.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.437+0000 7f598999d640 1 --2- 172.21.15.28:0/102439850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59840fe580 0x7f59840fe990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:17.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.437+0000 7f598999d640 1 -- 172.21.15.28:0/102439850 >> 172.21.15.28:0/102439850 conn(0x7f59840f9e70 msgr2=0x7f59840fc250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:44:17.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.437+0000 7f598999d640 1 -- 172.21.15.28:0/102439850 shutdown_connections 2024-07-03T15:44:17.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.437+0000 7f598999d640 1 -- 172.21.15.28:0/102439850 wait complete. 2024-07-03T15:44:17.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.438+0000 7f598999d640 1 Processor -- start 2024-07-03T15:44:17.442 INFO:tasks.workunit.client.0.smithi028.stdout:7/666: truncate d9/d1c/d7c/d8b/f90 1969430 0 2024-07-03T15:44:17.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.438+0000 7f598999d640 1 -- start start 2024-07-03T15:44:17.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.439+0000 7f598999d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59840fe580 0x7f5984071cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:44:17.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.439+0000 7f598999d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59840ff850 0x7f5984072230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:44:17.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.439+0000 7f598999d640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59840728c0 con 0x7f59840ff850 2024-07-03T15:44:17.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.439+0000 7f598999d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59840730f0 con 0x7f59840fe580 2024-07-03T15:44:17.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.439+0000 7f5982ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59840fe580 0x7f5984071cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:44:17.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.439+0000 7f5982ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59840fe580 0x7f5984071cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.28:51220/0 (socket says 172.21.15.28:51220) 2024-07-03T15:44:17.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.439+0000 7f5982ffd640 1 -- 172.21.15.28:0/1760684325 learned_addr learned my addr 172.21.15.28:0/1760684325 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:44:17.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.439+0000 7f59827fc640 1 --2- 172.21.15.28:0/1760684325 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59840ff850 0x7f5984072230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:44:17.444 INFO:tasks.workunit.client.0.smithi028.stdout:5/797: getdents d0/dd/d1e/d3b/d3f 0 2024-07-03T15:44:17.444 INFO:tasks.workunit.client.0.smithi028.stdout:5/798: fdatasync d0/dd/d1e/d86/f88 0 2024-07-03T15:44:17.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.440+0000 7f5982ffd640 1 -- 172.21.15.28:0/1760684325 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59840ff850 msgr2=0x7f5984072230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:17.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.440+0000 7f5982ffd640 1 --2- 172.21.15.28:0/1760684325 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59840ff850 0x7f5984072230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:17.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.440+0000 7f5982ffd640 1 -- 172.21.15.28:0/1760684325 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5978005040 con 0x7f59840fe580 2024-07-03T15:44:17.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.440+0000 7f5982ffd640 1 --2- 172.21.15.28:0/1760684325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59840fe580 0x7f5984071cf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f597400b790 tx=0x7f59740091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:44:17.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.440+0000 7f596b7fe640 1 -- 172.21.15.28:0/1760684325 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f597400c700 con 0x7f59840fe580 2024-07-03T15:44:17.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.440+0000 7f596b7fe640 1 -- 172.21.15.28:0/1760684325 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f597400cd30 con 0x7f59840fe580 2024-07-03T15:44:17.445 INFO:tasks.workunit.client.0.smithi028.stdout:2/627: dwrite da/d1e/dd0/fd4 [0,4194304] 0 2024-07-03T15:44:17.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.440+0000 7f596b7fe640 1 -- 172.21.15.28:0/1760684325 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5974013370 con 0x7f59840fe580 2024-07-03T15:44:17.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.440+0000 7f598999d640 1 -- 172.21.15.28:0/1760684325 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5984073370 con 0x7f59840fe580 2024-07-03T15:44:17.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.441+0000 7f598999d640 1 -- 172.21.15.28:0/1760684325 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5984073830 con 0x7f59840fe580 2024-07-03T15:44:17.446 INFO:tasks.workunit.client.0.smithi028.stdout:6/849: dwrite d4/d78/d10c/d1c/f88 [0,4194304] 0 2024-07-03T15:44:17.446 INFO:tasks.workunit.client.0.smithi028.stdout:5/799: mkdir d0/dd/d1e/d3b/de0/dfa/d107 0 2024-07-03T15:44:17.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.442+0000 7f596b7fe640 1 -- 172.21.15.28:0/1760684325 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 24) v1 ==== 88617+0+0 (secure 0 0 0) 0x7f5974012070 con 0x7f59840fe580 2024-07-03T15:44:17.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.443+0000 7f598999d640 1 -- 172.21.15.28:0/1760684325 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5970005240 con 0x7f59840fe580 2024-07-03T15:44:17.447 INFO:tasks.workunit.client.0.smithi028.stdout:7/667: creat d9/d1c/fc9 x:0 0 0 2024-07-03T15:44:17.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.443+0000 7f596b7fe640 1 --2- 172.21.15.28:0/1760684325 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f596c06a780 0x7f596c06cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:44:17.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.443+0000 7f596b7fe640 1 -- 172.21.15.28:0/1760684325 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f59740886e0 con 0x7f59840fe580 2024-07-03T15:44:17.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.444+0000 7f59827fc640 1 --2- 172.21.15.28:0/1760684325 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f596c06a780 0x7f596c06cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:44:17.448 INFO:tasks.workunit.client.0.smithi028.stdout:0/824: dwrite d2/d3/fc6 [0,4194304] 0 2024-07-03T15:44:17.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.444+0000 7f59827fc640 1 --2- 172.21.15.28:0/1760684325 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f596c06a780 0x7f596c06cc30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5978000c00 tx=0x7f5978035040 comp rx=0 tx=0).ready entity=mgr.24403 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:44:17.449 INFO:tasks.workunit.client.0.smithi028.stdout:9/851: write d1/d10/d28/fd3 [319457,43769] 0 2024-07-03T15:44:17.456 INFO:tasks.workunit.client.0.smithi028.stdout:3/761: dread f9 [0,4194304] 0 2024-07-03T15:44:17.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.453+0000 7f596b7fe640 1 -- 172.21.15.28:0/1760684325 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7f5974052420 con 0x7f59840fe580 2024-07-03T15:44:17.459 INFO:tasks.workunit.client.0.smithi028.stdout:0/825: write d2/d3/fc6 [2297246,41468] 0 2024-07-03T15:44:17.459 INFO:tasks.workunit.client.0.smithi028.stdout:0/826: write d2/d3/d4d/fa2 [970027,127455] 0 2024-07-03T15:44:17.461 INFO:tasks.workunit.client.0.smithi028.stdout:6/850: mkdir d4/d12/d2c/d12d 0 2024-07-03T15:44:17.466 INFO:tasks.workunit.client.0.smithi028.stdout:7/668: dwrite d9/d1c/d45/d71/fb1 [0,4194304] 0 2024-07-03T15:44:17.466 INFO:tasks.workunit.client.0.smithi028.stdout:7/669: write d9/f3a [1656416,41173] 0 2024-07-03T15:44:17.466 INFO:tasks.workunit.client.0.smithi028.stdout:7/670: rename d9 to d9/d1c/d45/db5/d92/d99/daf/dc4/dca 22 2024-07-03T15:44:17.468 INFO:tasks.workunit.client.0.smithi028.stdout:3/762: mkdir d18/d1a/d1e/d58/df3 0 2024-07-03T15:44:17.471 INFO:tasks.workunit.client.0.smithi028.stdout:6/851: creat d4/d78/d10c/d38/d60/dc5/dcf/f12e x:0 0 0 2024-07-03T15:44:17.471 INFO:tasks.workunit.client.0.smithi028.stdout:6/852: write d4/d78/d10c/d38/f9b [708613,84921] 0 2024-07-03T15:44:17.473 INFO:tasks.workunit.client.0.smithi028.stdout:7/671: mknod d9/d1c/d45/ccb 0 2024-07-03T15:44:17.474 INFO:tasks.workunit.client.0.smithi028.stdout:3/763: rename d18/d1a/d1e/d73/l85 to d18/d1a/d1e/d58/lf4 0 2024-07-03T15:44:17.475 INFO:tasks.workunit.client.0.smithi028.stdout:3/764: stat d18/d6f/l94 0 2024-07-03T15:44:17.475 INFO:tasks.workunit.client.0.smithi028.stdout:0/827: rmdir d2/d3/dc4/d36/d45 39 2024-07-03T15:44:17.476 INFO:tasks.workunit.client.0.smithi028.stdout:6/853: mknod d4/d12/dc3/d11e/c12f 0 2024-07-03T15:44:17.477 INFO:tasks.workunit.client.0.smithi028.stdout:2/628: dwrite da/d24/f55 [0,4194304] 0 2024-07-03T15:44:17.477 INFO:tasks.workunit.client.0.smithi028.stdout:9/852: dwrite d1/d10/d28/d45/d61/d8c/d77/db7/fb0 [0,4194304] 0 2024-07-03T15:44:17.478 INFO:tasks.workunit.client.0.smithi028.stdout:9/853: write d1/d10/d28/d45/d56/fc5 [881658,20850] 0 2024-07-03T15:44:17.478 INFO:tasks.workunit.client.0.smithi028.stdout:7/672: truncate d9/f34 4334456 0 2024-07-03T15:44:17.481 INFO:tasks.workunit.client.0.smithi028.stdout:3/765: link d18/d1a/d1e/d58/f7f d18/d43/ff5 0 2024-07-03T15:44:17.481 INFO:tasks.workunit.client.0.smithi028.stdout:3/766: write d18/d3a/dc1/fd3 [676805,22466] 0 2024-07-03T15:44:17.483 INFO:tasks.workunit.client.0.smithi028.stdout:5/800: write d0/dd/fcd [652609,99634] 0 2024-07-03T15:44:17.491 INFO:tasks.workunit.client.0.smithi028.stdout:6/854: write d4/f7b [6476404,31694] 0 2024-07-03T15:44:17.492 INFO:tasks.workunit.client.0.smithi028.stdout:6/855: stat d4/d15/l16 0 2024-07-03T15:44:17.494 INFO:tasks.workunit.client.0.smithi028.stdout:0/828: dwrite d2/fb [0,4194304] 0 2024-07-03T15:44:17.494 INFO:tasks.workunit.client.0.smithi028.stdout:0/829: stat d2/f49 0 2024-07-03T15:44:17.497 INFO:tasks.workunit.client.0.smithi028.stdout:9/854: symlink d1/d10/d28/d45/d56/dcd/dd5/l127 0 2024-07-03T15:44:17.497 INFO:tasks.workunit.client.0.smithi028.stdout:9/855: truncate d1/d10/d76/fde 719442 0 2024-07-03T15:44:17.499 INFO:tasks.workunit.client.0.smithi028.stdout:7/673: creat d9/d1c/da9/dc1/fcc x:0 0 0 2024-07-03T15:44:17.501 INFO:tasks.workunit.client.0.smithi028.stdout:0/830: dread d2/d3/dc4/d36/d45/f7b [0,4194304] 0 2024-07-03T15:44:17.501 INFO:tasks.workunit.client.0.smithi028.stdout:0/831: chown d2/d3/c26 863 1 2024-07-03T15:44:17.503 INFO:tasks.workunit.client.0.smithi028.stdout:3/767: unlink d18/d1a/f24 0 2024-07-03T15:44:17.504 INFO:tasks.workunit.client.0.smithi028.stdout:5/801: truncate d0/dd/d1e/d3b/d3f/d5a/f65 3834104 0 2024-07-03T15:44:17.505 INFO:tasks.workunit.client.0.smithi028.stdout:6/856: creat d4/d12/d2c/d31/f130 x:0 0 0 2024-07-03T15:44:17.512 INFO:tasks.workunit.client.0.smithi028.stdout:9/856: mknod d1/d8e/c128 0 2024-07-03T15:44:17.512 INFO:tasks.workunit.client.0.smithi028.stdout:9/857: write d1/d16/d62/f8b [1429174,77235] 0 2024-07-03T15:44:17.512 INFO:tasks.workunit.client.0.smithi028.stdout:4/864: sync 2024-07-03T15:44:17.512 INFO:tasks.workunit.client.0.smithi028.stdout:4/865: readlink d2/d12/d3b/l110 0 2024-07-03T15:44:17.513 INFO:tasks.workunit.client.0.smithi028.stdout:5/802: dread d0/dd/d13/d29/f4e [0,4194304] 0 2024-07-03T15:44:17.515 INFO:tasks.workunit.client.0.smithi028.stdout:7/674: truncate d9/d1c/d7c/d8b/f90 1224553 0 2024-07-03T15:44:17.515 INFO:tasks.workunit.client.0.smithi028.stdout:7/675: dread - d9/d1c/fc9 zero size 2024-07-03T15:44:17.517 INFO:tasks.workunit.client.0.smithi028.stdout:0/832: link d2/d3/d4d/d56/d78/ff0 d2/d3/d4d/d56/d78/d9f/ffa 0 2024-07-03T15:44:17.519 INFO:tasks.workunit.client.0.smithi028.stdout:6/857: dwrite d4/d78/d10c/d47/d9f/f128 [0,4194304] 0 2024-07-03T15:44:17.522 INFO:tasks.workunit.client.0.smithi028.stdout:3/768: rename d18/f52 to d18/d3a/d9d/d8c/ff6 0 2024-07-03T15:44:17.522 INFO:tasks.workunit.client.0.smithi028.stdout:3/769: chown d18/d1a/d64 0 1 2024-07-03T15:44:17.525 INFO:tasks.workunit.client.0.smithi028.stdout:9/858: symlink d1/d16/d57/d5f/d64/dc3/l129 0 2024-07-03T15:44:17.528 INFO:tasks.workunit.client.0.smithi028.stdout:4/866: link d2/d3e/d9f/d74/ceb d2/d12/d2e/c125 0 2024-07-03T15:44:17.530 INFO:tasks.workunit.client.0.smithi028.stdout:7/676: mknod d9/ccd 0 2024-07-03T15:44:17.532 INFO:tasks.workunit.client.0.smithi028.stdout:0/833: link d2/l28 d2/d3/dc4/d36/d45/lfb 0 2024-07-03T15:44:17.533 INFO:tasks.workunit.client.0.smithi028.stdout:9/859: write d1/d10/d28/d4e/d4f/f67 [1260730,23004] 0 2024-07-03T15:44:17.535 INFO:tasks.workunit.client.0.smithi028.stdout:5/803: dwrite d0/dd/df/f52 [0,4194304] 0 2024-07-03T15:44:17.535 INFO:tasks.workunit.client.0.smithi028.stdout:5/804: write d0/dd/d13/d57/d30/fd0 [2793196,86004] 0 2024-07-03T15:44:17.535 INFO:tasks.workunit.client.0.smithi028.stdout:5/805: readlink d0/dd/d1e/l9a 0 2024-07-03T15:44:17.539 INFO:tasks.workunit.client.0.smithi028.stdout:6/858: truncate d4/d12/f36 2593393 0 2024-07-03T15:44:17.543 INFO:tasks.workunit.client.0.smithi028.stdout:6/859: write d4/d78/d10c/d38/d60/dc5/dcf/fe2 [2078575,60719] 0 2024-07-03T15:44:17.545 INFO:tasks.workunit.client.0.smithi028.stdout:3/770: rename d18/d3a/d9d/c84 to d18/d3a/dbb/cf7 0 2024-07-03T15:44:17.549 INFO:tasks.workunit.client.0.smithi028.stdout:4/867: link d2/d12/lff d2/l126 0 2024-07-03T15:44:17.554 INFO:tasks.workunit.client.0.smithi028.stdout:0/834: mknod d2/cfc 0 2024-07-03T15:44:17.554 INFO:tasks.workunit.client.0.smithi028.stdout:9/860: mknod d1/d4a/d7e/c12a 0 2024-07-03T15:44:17.555 INFO:tasks.workunit.client.0.smithi028.stdout:5/806: unlink d0/dd/d13/d57/c27 0 2024-07-03T15:44:17.556 INFO:tasks.workunit.client.0.smithi028.stdout:5/807: creat d0/dd/d1e/d3b/d3f/d5a/f108 x:0 0 0 2024-07-03T15:44:17.559 INFO:tasks.workunit.client.0.smithi028.stdout:3/771: write d18/f1d [2431838,46467] 0 2024-07-03T15:44:17.563 INFO:tasks.workunit.client.0.smithi028.stdout:4/868: mknod d2/d5a/d45/d84/df4/d107/c127 0 2024-07-03T15:44:17.564 INFO:tasks.workunit.client.0.smithi028.stdout:9/861: mknod d1/d16/d57/c12b 0 2024-07-03T15:44:17.566 INFO:tasks.workunit.client.0.smithi028.stdout:7/677: dwrite d9/d1c/da9/dc1/fcc [0,4194304] 0 2024-07-03T15:44:17.569 INFO:tasks.workunit.client.0.smithi028.stdout:6/860: dwrite d4/d78/d10c/d38/d3d/df9/f105 [0,4194304] 0 2024-07-03T15:44:17.569 INFO:tasks.workunit.client.0.smithi028.stdout:0/835: dread d2/d1c/f2e [0,4194304] 0 2024-07-03T15:44:17.570 INFO:tasks.workunit.client.0.smithi028.stdout:6/861: write d4/d78/d10c/d38/d60/db5/fd0 [94929,82979] 0 2024-07-03T15:44:17.571 INFO:tasks.workunit.client.0.smithi028.stdout:4/869: unlink d2/d3e/d9f/c10f 0 2024-07-03T15:44:17.571 INFO:tasks.workunit.client.0.smithi028.stdout:0/836: dread d2/d3/dc4/d36/f88 [0,4194304] 0 2024-07-03T15:44:17.574 INFO:tasks.workunit.client.0.smithi028.stdout:7/678: symlink d9/d1c/d45/db5/d92/d9e/lce 0 2024-07-03T15:44:17.590 INFO:tasks.workunit.client.0.smithi028.stdout:5/808: dwrite d0/dd/d1e/f56 [4194304,4194304] 0 2024-07-03T15:44:17.590 INFO:tasks.workunit.client.0.smithi028.stdout:3/772: dwrite d18/d1a/d1e/f5d [0,4194304] 0 2024-07-03T15:44:17.591 INFO:tasks.workunit.client.0.smithi028.stdout:9/862: dwrite d1/d16/d57/d5f/fd4 [0,4194304] 0 2024-07-03T15:44:17.601 INFO:tasks.workunit.client.0.smithi028.stdout:4/870: truncate d2/d12/d2e/f38 2459995 0 2024-07-03T15:44:17.601 INFO:tasks.workunit.client.0.smithi028.stdout:4/871: chown d2/d12/d5b/d78/de4/d3a 229158648 1 2024-07-03T15:44:17.601 INFO:tasks.workunit.client.0.smithi028.stdout:5/809: write d0/dd/d13/d47/d50/f8c [4098157,66677] 0 2024-07-03T15:44:17.602 INFO:tasks.workunit.client.0.smithi028.stdout:6/862: dread d4/d15/f4c [0,4194304] 0 2024-07-03T15:44:17.604 INFO:tasks.workunit.client.0.smithi028.stdout:0/837: rename d2/d1c/f76 to d2/d1c/d6c/dc5/ffd 0 2024-07-03T15:44:17.605 INFO:tasks.workunit.client.0.smithi028.stdout:7/679: symlink d9/d1c/d7c/lcf 0 2024-07-03T15:44:17.605 INFO:tasks.workunit.client.0.smithi028.stdout:7/680: write d9/d1c/f4b [1737305,85753] 0 2024-07-03T15:44:17.612 INFO:tasks.workunit.client.0.smithi028.stdout:3/773: symlink d18/d43/lf8 0 2024-07-03T15:44:17.616 INFO:tasks.workunit.client.0.smithi028.stdout:4/872: dwrite d2/d5a/d45/d84/df4/d9d/fd1 [0,4194304] 0 2024-07-03T15:44:17.616 INFO:tasks.workunit.client.0.smithi028.stdout:4/873: creat d2/d12/d2d/db9/f128 x:0 0 0 2024-07-03T15:44:17.617 INFO:tasks.workunit.client.0.smithi028.stdout:4/874: write d2/d5a/d45/d84/df4/d9d/faf [1055654,34759] 0 2024-07-03T15:44:17.622 INFO:tasks.workunit.client.0.smithi028.stdout:5/810: mkdir d0/dd/df/da1/daa/d109 0 2024-07-03T15:44:17.626 INFO:tasks.workunit.client.0.smithi028.stdout:6/863: rename d4/d78/d10c/d38/d60/db5/cea to d4/d78/d10c/d38/d60/dc5/dcf/c131 0 2024-07-03T15:44:17.628 INFO:tasks.workunit.client.0.smithi028.stdout:0/838: link d2/d3/d4d/d56/l63 d2/d3/dc4/d36/d45/lfe 0 2024-07-03T15:44:17.628 INFO:tasks.workunit.client.0.smithi028.stdout:0/839: chown d2/d3/d4d/d56/l63 11 1 2024-07-03T15:44:17.628 INFO:tasks.workunit.client.0.smithi028.stdout:0/840: readlink d2/d6d/la6 0 2024-07-03T15:44:17.630 INFO:tasks.workunit.client.0.smithi028.stdout:3/774: mkdir d18/d1a/d1e/d58/df3/df9 0 2024-07-03T15:44:17.634 INFO:tasks.workunit.client.0.smithi028.stdout:4/875: mknod d2/d12/d5b/d98/c129 0 2024-07-03T15:44:17.635 INFO:tasks.workunit.client.0.smithi028.stdout:4/876: write d2/d3e/f3f [1876979,100180] 0 2024-07-03T15:44:17.635 INFO:tasks.workunit.client.0.smithi028.stdout:4/877: fsync d2/d12/d42/d63/da8/fc1 0 2024-07-03T15:44:17.635 INFO:tasks.workunit.client.0.smithi028.stdout:4/878: write d2/d3e/f124 [735267,3289] 0 2024-07-03T15:44:17.636 INFO:tasks.workunit.client.0.smithi028.stdout:5/811: creat d0/dd/d1e/d8e/de4/f10a x:0 0 0 2024-07-03T15:44:17.636 INFO:tasks.workunit.client.0.smithi028.stdout:5/812: creat d0/dd/d13/d57/d26/ddf/f10b x:0 0 0 2024-07-03T15:44:17.637 INFO:tasks.workunit.client.0.smithi028.stdout:3/775: write d18/d1a/d1e/f5d [1134846,111547] 0 2024-07-03T15:44:17.638 INFO:tasks.workunit.client.0.smithi028.stdout:3/776: write d18/d1a/d1e/d58/d9f/de2/ff1 [672644,106638] 0 2024-07-03T15:44:17.639 INFO:tasks.workunit.client.0.smithi028.stdout:7/681: rename d9/d1c/d45/db5/d92/d99/l55 to d9/d1c/d45/ld0 0 2024-07-03T15:44:17.639 INFO:tasks.workunit.client.0.smithi028.stdout:7/682: chown d9/d1c/d86/l87 1897840 1 2024-07-03T15:44:17.639 INFO:tasks.workunit.client.0.smithi028.stdout:7/683: write d9/d1c/d45/db5/f3e [4813380,32449] 0 2024-07-03T15:44:17.642 INFO:tasks.workunit.client.0.smithi028.stdout:6/864: creat d4/d78/d10c/d38/f132 x:0 0 0 2024-07-03T15:44:17.642 INFO:tasks.workunit.client.0.smithi028.stdout:6/865: chown d4/d78/d10c/d1c/l27 58 1 2024-07-03T15:44:17.645 INFO:tasks.workunit.client.0.smithi028.stdout:0/841: mkdir d2/d3/dc4/d58/d6e/df5/dff 0 2024-07-03T15:44:17.645 INFO:tasks.workunit.client.0.smithi028.stdout:0/842: dread - d2/d3/d4d/d56/d78/dae/fec zero size 2024-07-03T15:44:17.646 INFO:tasks.workunit.client.0.smithi028.stdout:4/879: creat d2/d12/d2e/df3/f12a x:0 0 0 2024-07-03T15:44:17.647 INFO:tasks.workunit.client.0.smithi028.stdout:4/880: read d2/d12/d2d/d58/fa3 [580536,59894] 0 2024-07-03T15:44:17.648 INFO:tasks.workunit.client.0.smithi028.stdout:4/881: read d2/d5a/d45/d84/df4/d9d/faf [498215,13423] 0 2024-07-03T15:44:17.648 INFO:tasks.workunit.client.0.smithi028.stdout:5/813: getdents d0/dd/d1e/d3b/ddd 0 2024-07-03T15:44:17.660 INFO:tasks.workunit.client.0.smithi028.stdout:3/777: dwrite d18/d3a/d9d/d8c/fdb [0,4194304] 0 2024-07-03T15:44:17.664 INFO:tasks.workunit.client.0.smithi028.stdout:3/778: creat d18/d3a/dbb/ffa x:0 0 0 2024-07-03T15:44:17.665 INFO:tasks.workunit.client.0.smithi028.stdout:6/866: dwrite d4/d78/d10c/d38/d60/dc5/fce [4194304,4194304] 0 2024-07-03T15:44:17.665 INFO:tasks.workunit.client.0.smithi028.stdout:6/867: write d4/d78/d10c/d38/f9b [1098340,35002] 0 2024-07-03T15:44:17.665 INFO:tasks.workunit.client.0.smithi028.stdout:6/868: chown d4/d78/d10c/d13/dbd/ld6 3408 1 2024-07-03T15:44:17.665 INFO:tasks.workunit.client.0.smithi028.stdout:5/814: dwrite d0/dd/d98/f6a [0,4194304] 0 2024-07-03T15:44:17.667 INFO:tasks.workunit.client.0.smithi028.stdout:6/869: mkdir d4/d12/d2c/d31/df5/d12c/d133 0 2024-07-03T15:44:17.670 INFO:tasks.workunit.client.0.smithi028.stdout:5/815: mknod d0/dd/d1e/d3b/c10c 0 2024-07-03T15:44:17.670 INFO:tasks.workunit.client.0.smithi028.stdout:5/816: chown d0/dd/d13/d57/d30/d70/d7f/d9e/fd2 3058 1 2024-07-03T15:44:17.670 INFO:tasks.workunit.client.0.smithi028.stdout:5/817: chown d0/dd/d13/cc3 6 1 2024-07-03T15:44:17.671 INFO:tasks.workunit.client.0.smithi028.stdout:3/779: rmdir d18/d3a/d9a 39 2024-07-03T15:44:17.672 INFO:tasks.workunit.client.0.smithi028.stdout:5/818: write d0/dd/d98/f6c [368460,55371] 0 2024-07-03T15:44:17.676 INFO:tasks.workunit.client.0.smithi028.stdout:6/870: dread d4/d78/d10c/d47/d9f/fa5 [0,4194304] 0 2024-07-03T15:44:17.676 INFO:tasks.workunit.client.0.smithi028.stdout:6/871: truncate d4/d78/d10c/d47/d9f/fa5 8644342 0 2024-07-03T15:44:17.677 INFO:tasks.workunit.client.0.smithi028.stdout:4/882: truncate d2/d3e/f124 713235 0 2024-07-03T15:44:17.680 INFO:tasks.workunit.client.0.smithi028.stdout:5/819: rename d0/dd/d14/dee to d0/dd/d1e/d94/d10d 0 2024-07-03T15:44:17.681 INFO:tasks.workunit.client.0.smithi028.stdout:4/883: mkdir d2/d12/da6/d12b 0 2024-07-03T15:44:17.683 INFO:tasks.workunit.client.0.smithi028.stdout:6/872: write d4/d78/d10c/d38/fa7 [7290719,30768] 0 2024-07-03T15:44:17.684 INFO:tasks.workunit.client.0.smithi028.stdout:5/820: unlink d0/dd/d1e/d94/fcc 0 2024-07-03T15:44:17.684 INFO:tasks.workunit.client.0.smithi028.stdout:5/821: truncate d0/dd/d13/d57/d30/fc5 216589 0 2024-07-03T15:44:17.686 INFO:tasks.workunit.client.0.smithi028.stdout:6/873: symlink d4/d78/d10c/d38/d59/l134 0 2024-07-03T15:44:17.688 INFO:tasks.workunit.client.0.smithi028.stdout:5/822: rename d0/dd/d13/d57/d26/c96 to d0/dd/d1e/d3b/de0/dfa/d107/c10e 0 2024-07-03T15:44:17.688 INFO:tasks.workunit.client.0.smithi028.stdout:5/823: creat d0/dd/d13/d57/d30/d70/d7f/dbb/f10f x:0 0 0 2024-07-03T15:44:17.688 INFO:tasks.workunit.client.0.smithi028.stdout:5/824: write d0/dd/d1e/d94/d10d/f105 [284053,62655] 0 2024-07-03T15:44:17.688 INFO:tasks.workunit.client.0.smithi028.stdout:5/825: chown d0/dd/la8 2635 1 2024-07-03T15:44:17.688 INFO:tasks.workunit.client.0.smithi028.stdout:5/826: write d0/dd/d1e/d8e/de4/f10a [207220,8769] 0 2024-07-03T15:44:17.688 INFO:tasks.workunit.client.0.smithi028.stdout:5/827: chown d0/dd/d13/d47/d50/l7c 850570 1 2024-07-03T15:44:17.689 INFO:tasks.workunit.client.0.smithi028.stdout:0/843: dwrite d2/d1c/d6c/dc5/ffd [0,4194304] 0 2024-07-03T15:44:17.691 INFO:tasks.workunit.client.0.smithi028.stdout:6/874: symlink d4/d78/d10c/d47/d66/l135 0 2024-07-03T15:44:17.694 INFO:tasks.workunit.client.0.smithi028.stdout:5/828: rmdir d0/dd/df/da1/daa/d109 0 2024-07-03T15:44:17.699 INFO:tasks.workunit.client.0.smithi028.stdout:5/829: dread d0/dd/d13/d57/d30/fd0 [0,4194304] 0 2024-07-03T15:44:17.699 INFO:tasks.workunit.client.0.smithi028.stdout:4/884: rmdir d2/d3e/d9f/ddb 0 2024-07-03T15:44:17.702 INFO:tasks.workunit.client.0.smithi028.stdout:8/860: sync 2024-07-03T15:44:17.707 INFO:tasks.workunit.client.0.smithi028.stdout:6/875: getdents d4/d78/d10c/d56 0 2024-07-03T15:44:17.707 INFO:tasks.workunit.client.0.smithi028.stdout:6/876: dread - d4/d15/f8c zero size 2024-07-03T15:44:17.710 INFO:tasks.workunit.client.0.smithi028.stdout:8/861: dread d0/f2a [0,4194304] 0 2024-07-03T15:44:17.710 INFO:tasks.workunit.client.0.smithi028.stdout:5/830: mkdir d0/dd/d13/d29/d110 0 2024-07-03T15:44:17.711 INFO:tasks.workunit.client.0.smithi028.stdout:4/885: symlink d2/d12/d42/d63/d9e/l12c 0 2024-07-03T15:44:17.715 INFO:tasks.workunit.client.0.smithi028.stdout:6/877: mkdir d4/d78/d10c/d38/d60/dc5/dcf/d136 0 2024-07-03T15:44:17.718 INFO:tasks.workunit.client.0.smithi028.stdout:0/844: dwrite d2/d1c/f68 [0,4194304] 0 2024-07-03T15:44:17.720 INFO:tasks.workunit.client.0.smithi028.stdout:4/886: mknod d2/c12d 0 2024-07-03T15:44:17.721 INFO:tasks.workunit.client.0.smithi028.stdout:6/878: creat d4/d78/d10c/d47/d66/daf/f137 x:0 0 0 2024-07-03T15:44:17.722 INFO:tasks.workunit.client.0.smithi028.stdout:6/879: dread - d4/d78/d10c/d1c/f98 zero size 2024-07-03T15:44:17.723 INFO:tasks.workunit.client.0.smithi028.stdout:5/831: dread d0/dd/df/fc2 [0,4194304] 0 2024-07-03T15:44:17.724 INFO:tasks.workunit.client.0.smithi028.stdout:4/887: dread d2/d5a/d45/d84/df4/d9d/faf [0,4194304] 0 2024-07-03T15:44:17.724 INFO:tasks.workunit.client.0.smithi028.stdout:0/845: truncate d2/d3/dc4/d36/d45/f46 7289642 0 2024-07-03T15:44:17.726 INFO:tasks.workunit.client.0.smithi028.stdout:8/862: symlink d0/d41/d50/d3/l11d 0 2024-07-03T15:44:17.726 INFO:tasks.workunit.client.0.smithi028.stdout:8/863: fsync d0/d41/d50/d3/d35/d68/d22/fb1 0 2024-07-03T15:44:17.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:17 smithi148 ceph-mon[36056]: from='client.24429 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:44:17.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:17 smithi148 ceph-mon[36056]: pgmap v10: 65 pgs: 65 active+clean; 2.9 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 120 MiB/s rd, 155 MiB/s wr, 306 op/s 2024-07-03T15:44:17.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:17 smithi148 ceph-mon[36056]: from='client.14650 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:44:17.735 INFO:tasks.workunit.client.0.smithi028.stdout:5/832: mkdir d0/dd/df/da1/daa/d111 0 2024-07-03T15:44:17.736 INFO:tasks.workunit.client.0.smithi028.stdout:6/880: rename d4/d12/dc3/c125 to d4/c138 0 2024-07-03T15:44:17.736 INFO:tasks.workunit.client.0.smithi028.stdout:6/881: creat d4/d12/dc3/f139 x:0 0 0 2024-07-03T15:44:17.738 INFO:tasks.workunit.client.0.smithi028.stdout:5/833: symlink d0/dd/d13/d29/d110/l112 0 2024-07-03T15:44:17.738 INFO:tasks.workunit.client.0.smithi028.stdout:5/834: chown d0/dd/d13/d57/d26/d41/ccf 25765 1 2024-07-03T15:44:17.740 INFO:tasks.workunit.client.0.smithi028.stdout:6/882: symlink d4/d12/d2c/ddd/l13a 0 2024-07-03T15:44:17.740 INFO:tasks.workunit.client.0.smithi028.stdout:6/883: read - d4/d15/de5/f129 zero size 2024-07-03T15:44:17.741 INFO:tasks.workunit.client.0.smithi028.stdout:6/884: read d4/d78/d10c/d47/d66/f8b [199890,79925] 0 2024-07-03T15:44:17.784 INFO:tasks.workunit.client.0.smithi028.stdout:5/835: symlink d0/dd/d1e/d3b/ddd/l113 0 2024-07-03T15:44:17.787 INFO:tasks.workunit.client.0.smithi028.stdout:6/885: link d4/d12/d2c/d31/f130 d4/d12/f13b 0 2024-07-03T15:44:17.788 INFO:tasks.workunit.client.0.smithi028.stdout:6/886: write d4/d78/d10c/d38/fa7 [5452581,10615] 0 2024-07-03T15:44:17.788 INFO:tasks.workunit.client.0.smithi028.stdout:4/888: rename d2/d12/d3b to d2/d5a/d45/d84/df4/d92/d12e 0 2024-07-03T15:44:17.788 INFO:tasks.workunit.client.0.smithi028.stdout:4/889: write d2/d5a/d45/fed [1984987,76273] 0 2024-07-03T15:44:17.789 INFO:tasks.workunit.client.0.smithi028.stdout:8/864: dwrite d0/d41/f88 [0,4194304] 0 2024-07-03T15:44:17.791 INFO:tasks.workunit.client.0.smithi028.stdout:6/887: getdents d4/d12/d2c/dac 0 2024-07-03T15:44:17.792 INFO:tasks.workunit.client.0.smithi028.stdout:4/890: mkdir d2/d12/d2e/df3/d12f 0 2024-07-03T15:44:17.793 INFO:tasks.workunit.client.0.smithi028.stdout:5/836: write d0/dd/f53 [2435981,5811] 0 2024-07-03T15:44:17.795 INFO:tasks.workunit.client.0.smithi028.stdout:8/865: creat d0/d41/d50/d3/dc/d3f/d8e/d81/d9e/f11e x:0 0 0 2024-07-03T15:44:17.795 INFO:tasks.workunit.client.0.smithi028.stdout:4/891: symlink d2/d12/da6/d12b/l130 0 2024-07-03T15:44:17.797 INFO:tasks.workunit.client.0.smithi028.stdout:5/837: mknod d0/d8f/db5/ddc/c114 0 2024-07-03T15:44:17.798 INFO:tasks.workunit.client.0.smithi028.stdout:8/866: stat d0/d41/d50/d3/d71/ld6 0 2024-07-03T15:44:17.798 INFO:tasks.workunit.client.0.smithi028.stdout:8/867: write d0/d6c/fae [4310113,91044] 0 2024-07-03T15:44:17.801 INFO:tasks.workunit.client.0.smithi028.stdout:5/838: unlink d0/dd/d13/d29/lb7 0 2024-07-03T15:44:17.802 INFO:tasks.workunit.client.0.smithi028.stdout:8/868: mkdir d0/d41/d50/d97/d11f 0 2024-07-03T15:44:17.806 INFO:tasks.workunit.client.0.smithi028.stdout:8/869: link d0/d41/d50/d3/d35/d68/d22/d40/d62/cb6 d0/d41/d50/d3/d35/d68/d22/d40/c120 0 2024-07-03T15:44:17.818 INFO:tasks.workunit.client.0.smithi028.stdout:4/892: dwrite d2/d12/d2d/d58/f118 [0,4194304] 0 2024-07-03T15:44:17.818 INFO:tasks.workunit.client.0.smithi028.stdout:4/893: dread - d2/d12/d5b/f101 zero size 2024-07-03T15:44:17.821 INFO:tasks.workunit.client.0.smithi028.stdout:6/888: dwrite d4/d78/d10c/d47/d9f/f126 [0,4194304] 0 2024-07-03T15:44:17.821 INFO:tasks.workunit.client.0.smithi028.stdout:8/870: write d0/d41/d50/d3/dc/d3f/df3/fb9 [904511,100725] 0 2024-07-03T15:44:17.821 INFO:tasks.workunit.client.0.smithi028.stdout:8/871: chown d0/d41/d50/d3/dc/d3f/d8e/d81/la8 0 1 2024-07-03T15:44:17.821 INFO:tasks.workunit.client.0.smithi028.stdout:8/872: readlink d0/d41/d50/d3/d35/d68/d22/d40/d4a/la3 0 2024-07-03T15:44:17.823 INFO:tasks.workunit.client.0.smithi028.stdout:4/894: mkdir d2/d12/d131 0 2024-07-03T15:44:17.823 INFO:tasks.workunit.client.0.smithi028.stdout:4/895: truncate d2/d12/d42/d63/dfa/f102 448310 0 2024-07-03T15:44:17.824 INFO:tasks.workunit.client.0.smithi028.stdout:4/896: creat d2/d12/d42/d63/dfa/f132 x:0 0 0 2024-07-03T15:44:17.824 INFO:tasks.workunit.client.0.smithi028.stdout:4/897: chown d2/d5a/cc2 1589 1 2024-07-03T15:44:17.824 INFO:tasks.workunit.client.0.smithi028.stdout:4/898: read - d2/d12/d2e/f100 zero size 2024-07-03T15:44:17.831 INFO:tasks.workunit.client.0.smithi028.stdout:6/889: dread d4/d78/d10c/d47/d9f/fc4 [0,4194304] 0 2024-07-03T15:44:17.837 INFO:tasks.workunit.client.0.smithi028.stdout:6/890: creat d4/d78/d10c/d13/dbd/f13c x:0 0 0 2024-07-03T15:44:17.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.834+0000 7f598999d640 1 -- 172.21.15.28:0/1760684325 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f59700057c0 con 0x7f59840fe580 2024-07-03T15:44:17.838 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:17 smithi028 ceph-mon[29148]: from='client.24429 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:44:17.838 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:17 smithi028 ceph-mon[29148]: pgmap v10: 65 pgs: 65 active+clean; 2.9 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 120 MiB/s rd, 155 MiB/s wr, 306 op/s 2024-07-03T15:44:17.838 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:17 smithi028 ceph-mon[29148]: from='client.14650 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:44:17.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.835+0000 7f596b7fe640 1 -- 172.21.15.28:0/1760684325 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+849 (secure 0 0 0) 0x7f5974055ae0 con 0x7f59840fe580 2024-07-03T15:44:17.839 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-07-03T15:44:17.839 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": { 2024-07-03T15:44:17.839 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 2 2024-07-03T15:44:17.839 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:44:17.839 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": { 2024-07-03T15:44:17.839 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 1, 2024-07-03T15:44:17.839 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 1 2024-07-03T15:44:17.839 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:44:17.839 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": { 2024-07-03T15:44:17.840 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 6 2024-07-03T15:44:17.840 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:44:17.840 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": { 2024-07-03T15:44:17.840 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 4 2024-07-03T15:44:17.840 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:44:17.840 INFO:teuthology.orchestra.run.smithi028.stdout: "overall": { 2024-07-03T15:44:17.840 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 13, 2024-07-03T15:44:17.840 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 1 2024-07-03T15:44:17.840 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-07-03T15:44:17.840 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-07-03T15:44:17.841 INFO:tasks.workunit.client.0.smithi028.stdout:6/891: creat d4/d78/d10c/d47/d9f/d124/f13d x:0 0 0 2024-07-03T15:44:17.841 INFO:tasks.workunit.client.0.smithi028.stdout:4/899: dwrite d2/d5a/d45/d84/df4/d9d/fc4 [0,4194304] 0 2024-07-03T15:44:17.841 INFO:tasks.workunit.client.0.smithi028.stdout:8/873: dwrite d0/d41/d50/d3/dc/d3f/d8e/f6d [0,4194304] 0 2024-07-03T15:44:17.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.838+0000 7f598999d640 1 -- 172.21.15.28:0/1760684325 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f596c06a780 msgr2=0x7f596c06cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:17.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.838+0000 7f598999d640 1 --2- 172.21.15.28:0/1760684325 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f596c06a780 0x7f596c06cc30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5978000c00 tx=0x7f5978035040 comp rx=0 tx=0).stop 2024-07-03T15:44:17.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.838+0000 7f598999d640 1 -- 172.21.15.28:0/1760684325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59840fe580 msgr2=0x7f5984071cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:17.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.838+0000 7f598999d640 1 --2- 172.21.15.28:0/1760684325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59840fe580 0x7f5984071cf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f597400b790 tx=0x7f59740091d0 comp rx=0 tx=0).stop 2024-07-03T15:44:17.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.838+0000 7f598999d640 1 -- 172.21.15.28:0/1760684325 shutdown_connections 2024-07-03T15:44:17.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.838+0000 7f598999d640 1 --2- 172.21.15.28:0/1760684325 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f596c06a780 0x7f596c06cc30 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:17.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.838+0000 7f598999d640 1 --2- 172.21.15.28:0/1760684325 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59840ff850 0x7f5984072230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:17.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.838+0000 7f598999d640 1 --2- 172.21.15.28:0/1760684325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59840fe580 0x7f5984071cf0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:17.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.838+0000 7f598999d640 1 -- 172.21.15.28:0/1760684325 >> 172.21.15.28:0/1760684325 conn(0x7f59840f9e70 msgr2=0x7f59840fc520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:44:17.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.838+0000 7f598999d640 1 -- 172.21.15.28:0/1760684325 shutdown_connections 2024-07-03T15:44:17.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.838+0000 7f598999d640 1 -- 172.21.15.28:0/1760684325 wait complete. 2024-07-03T15:44:17.843 INFO:tasks.workunit.client.0.smithi028.stdout:4/900: write d2/d3e/f3f [1669523,72228] 0 2024-07-03T15:44:17.852 INFO:tasks.workunit.client.0.smithi028.stdout:6/892: dread d4/d78/d10c/d56/fc6 [0,4194304] 0 2024-07-03T15:44:17.853 INFO:tasks.workunit.client.0.smithi028.stdout:6/893: truncate d4/d78/d10c/d47/f12b 368719 0 2024-07-03T15:44:17.853 INFO:tasks.workunit.client.0.smithi028.stdout:6/894: readlink d4/d78/d10c/d1c/l108 0 2024-07-03T15:44:17.853 INFO:tasks.workunit.client.0.smithi028.stdout:6/895: chown d4/d78/d10c/d1c/f98 59 1 2024-07-03T15:44:17.853 INFO:tasks.workunit.client.0.smithi028.stdout:8/874: dread d0/d41/d50/d3/f3a [0,4194304] 0 2024-07-03T15:44:17.854 INFO:tasks.workunit.client.0.smithi028.stdout:8/875: write d0/d41/d50/d3/d35/d68/d22/fb1 [4388161,121992] 0 2024-07-03T15:44:17.855 INFO:tasks.workunit.client.0.smithi028.stdout:8/876: rmdir d0/d6c 39 2024-07-03T15:44:17.856 INFO:tasks.workunit.client.0.smithi028.stdout:8/877: unlink d0/d41/d50/d3/d35/d86/c10d 0 2024-07-03T15:44:17.857 INFO:tasks.workunit.client.0.smithi028.stdout:4/901: dwrite d2/d3e/dae/f11d [0,4194304] 0 2024-07-03T15:44:17.857 INFO:tasks.workunit.client.0.smithi028.stdout:4/902: fdatasync d2/d12/d5b/f81 0 2024-07-03T15:44:17.857 INFO:tasks.workunit.client.0.smithi028.stdout:4/903: truncate d2/d5a/f24 3632235 0 2024-07-03T15:44:17.859 INFO:tasks.workunit.client.0.smithi028.stdout:4/904: rename d2/d5a/d45/d84/df4/d9d/ff5 to d2/d5a/d45/d84/df4/d107/f133 0 2024-07-03T15:44:17.859 INFO:tasks.workunit.client.0.smithi028.stdout:4/905: fsync d2/d12/d5b/f7a 0 2024-07-03T15:44:17.859 INFO:tasks.workunit.client.0.smithi028.stdout:4/906: chown d2/d12/d42/d63/d75/fa9 1699981 1 2024-07-03T15:44:17.860 INFO:tasks.workunit.client.0.smithi028.stdout:6/896: dread d4/d78/d10c/d38/f4a [0,4194304] 0 2024-07-03T15:44:17.862 INFO:tasks.workunit.client.0.smithi028.stdout:6/897: rename d4/l1b to d4/d12/d2c/d31/df5/d12c/d133/l13e 0 2024-07-03T15:44:17.864 INFO:tasks.workunit.client.0.smithi028.stdout:8/878: read d0/d41/d50/d3/dc/fcc [2550644,108642] 0 2024-07-03T15:44:17.864 INFO:tasks.workunit.client.0.smithi028.stdout:8/879: truncate d0/de0/dcf/fa0 1341780 0 2024-07-03T15:44:17.864 INFO:tasks.workunit.client.0.smithi028.stdout:6/898: mknod d4/d78/d10c/d38/d3d/c13f 0 2024-07-03T15:44:17.864 INFO:tasks.workunit.client.0.smithi028.stdout:8/880: creat d0/d41/d50/d3/dc/d3f/d8e/d81/d100/f121 x:0 0 0 2024-07-03T15:44:17.866 INFO:tasks.workunit.client.0.smithi028.stdout:6/899: link d4/d78/d10c/d38/d60/dc5/dcf/fe2 d4/d12/d2c/d31/df5/f140 0 2024-07-03T15:44:17.868 INFO:tasks.workunit.client.0.smithi028.stdout:6/900: creat d4/d78/d10c/d1c/dfd/f141 x:0 0 0 2024-07-03T15:44:17.873 INFO:tasks.workunit.client.0.smithi028.stdout:6/901: dread d4/d12/fa9 [0,4194304] 0 2024-07-03T15:44:17.873 INFO:tasks.workunit.client.0.smithi028.stdout:6/902: chown d4/d12/dc3/ff2 118920568 1 2024-07-03T15:44:17.873 INFO:tasks.workunit.client.0.smithi028.stdout:6/903: stat d4/d12/cd1 0 2024-07-03T15:44:17.875 INFO:tasks.workunit.client.0.smithi028.stdout:8/881: truncate d0/d41/d50/d3/d35/d68/d22/fb1 1457647 0 2024-07-03T15:44:17.875 INFO:tasks.workunit.client.0.smithi028.stdout:8/882: chown d0/d41/d50/d3/dc/d3f/d8e/d81/fbd 164 1 2024-07-03T15:44:17.875 INFO:tasks.workunit.client.0.smithi028.stdout:8/883: readlink d0/d41/d50/d3/d71/ld6 0 2024-07-03T15:44:17.875 INFO:tasks.workunit.client.0.smithi028.stdout:8/884: fsync d0/d41/d50/d3/dc/d3f/d8e/d81/d9e/fea 0 2024-07-03T15:44:17.879 INFO:tasks.workunit.client.0.smithi028.stdout:8/885: symlink d0/d41/d50/d3/dc/d3f/df3/l122 0 2024-07-03T15:44:17.879 INFO:tasks.workunit.client.0.smithi028.stdout:4/907: dwrite d2/d5a/d45/d84/f8f [0,4194304] 0 2024-07-03T15:44:17.882 INFO:tasks.workunit.client.0.smithi028.stdout:8/886: creat d0/d41/d50/daf/dc8/d108/f123 x:0 0 0 2024-07-03T15:44:17.886 INFO:tasks.workunit.client.0.smithi028.stdout:4/908: mknod d2/d12/da6/d12b/c134 0 2024-07-03T15:44:17.886 INFO:tasks.workunit.client.0.smithi028.stdout:6/904: write d4/d78/d10c/d38/d60/dc5/fce [4122348,7013] 0 2024-07-03T15:44:17.887 INFO:tasks.workunit.client.0.smithi028.stdout:6/905: stat d4/d78/d10c/d38/d59/l134 0 2024-07-03T15:44:17.888 INFO:tasks.workunit.client.0.smithi028.stdout:4/909: symlink d2/d5a/d45/d84/l135 0 2024-07-03T15:44:17.889 INFO:tasks.workunit.client.0.smithi028.stdout:6/906: creat d4/d78/d10c/d1c/df7/f142 x:0 0 0 2024-07-03T15:44:17.893 INFO:tasks.workunit.client.0.smithi028.stdout:4/910: unlink d2/d12/d2e/f47 0 2024-07-03T15:44:17.895 INFO:tasks.workunit.client.0.smithi028.stdout:8/887: dread d0/d41/d50/f27 [0,4194304] 0 2024-07-03T15:44:17.895 INFO:tasks.workunit.client.0.smithi028.stdout:8/888: chown d0/d41/d50/d3/d71 0 1 2024-07-03T15:44:17.895 INFO:tasks.workunit.client.0.smithi028.stdout:4/911: write d2/d12/d2d/db9/dda/ff9 [1021659,68678] 0 2024-07-03T15:44:17.895 INFO:tasks.workunit.client.0.smithi028.stdout:6/907: dread d4/d12/fae [0,4194304] 0 2024-07-03T15:44:17.895 INFO:tasks.workunit.client.0.smithi028.stdout:6/908: write d4/d78/d10c/d38/d3d/f58 [5085851,55503] 0 2024-07-03T15:44:17.895 INFO:tasks.workunit.client.0.smithi028.stdout:6/909: dread d4/d78/d10c/d47/f12b [0,4194304] 0 2024-07-03T15:44:17.900 INFO:tasks.workunit.client.0.smithi028.stdout:8/889: write d0/d41/d50/d3/dc/d3f/d8e/f42 [3027126,56794] 0 2024-07-03T15:44:17.901 INFO:tasks.workunit.client.0.smithi028.stdout:6/910: dread d4/d12/d2c/fe0 [4194304,4194304] 0 2024-07-03T15:44:17.901 INFO:tasks.workunit.client.0.smithi028.stdout:6/911: fsync d4/d15/f6a 0 2024-07-03T15:44:17.904 INFO:tasks.workunit.client.0.smithi028.stdout:4/912: write d2/d3e/f8a [4146420,16936] 0 2024-07-03T15:44:17.904 INFO:tasks.workunit.client.0.smithi028.stdout:4/913: write d2/d3e/dae/f11b [36073,50833] 0 2024-07-03T15:44:17.905 INFO:tasks.workunit.client.0.smithi028.stdout:6/912: unlink d4/d78/d10c/d38/d60/dc5/fca 0 2024-07-03T15:44:17.905 INFO:tasks.workunit.client.0.smithi028.stdout:6/913: write d4/d15/d68/fed [1002225,74857] 0 2024-07-03T15:44:17.915 INFO:tasks.workunit.client.0.smithi028.stdout:6/914: truncate d4/d12/d2c/d31/f64 3419407 0 2024-07-03T15:44:17.917 INFO:tasks.workunit.client.0.smithi028.stdout:6/915: mkdir d4/d78/d10c/d38/d59/d143 0 2024-07-03T15:44:17.917 INFO:tasks.workunit.client.0.smithi028.stdout:4/914: write d2/d5a/f1f [6641054,51425] 0 2024-07-03T15:44:17.919 INFO:tasks.workunit.client.0.smithi028.stdout:4/915: getdents d2/d3e/d9f/d74 0 2024-07-03T15:44:17.921 INFO:tasks.workunit.client.0.smithi028.stdout:4/916: truncate d2/d3e/d9f/dac/f116 2289520 0 2024-07-03T15:44:17.921 INFO:tasks.workunit.client.0.smithi028.stdout:4/917: chown d2/d5a/d45/f7c 0 1 2024-07-03T15:44:17.923 INFO:tasks.workunit.client.0.smithi028.stdout:4/918: link d2/d12/d2e/df3/fcf d2/d5a/d45/d84/df4/d92/f136 0 2024-07-03T15:44:17.925 INFO:tasks.workunit.client.0.smithi028.stdout:8/890: dwrite d0/f5b [0,4194304] 0 2024-07-03T15:44:17.928 INFO:tasks.workunit.client.0.smithi028.stdout:6/916: write d4/d78/d10c/d1c/f88 [1967086,83936] 0 2024-07-03T15:44:17.941 INFO:tasks.workunit.client.0.smithi028.stdout:6/917: dread d4/d78/d10c/d38/d3d/df9/f105 [0,4194304] 0 2024-07-03T15:44:17.942 INFO:tasks.workunit.client.0.smithi028.stdout:6/918: symlink d4/d12/d2c/l144 0 2024-07-03T15:44:17.943 INFO:tasks.workunit.client.0.smithi028.stdout:8/891: dwrite d0/d41/d50/d3/dc/d3f/faa [0,4194304] 0 2024-07-03T15:44:17.943 INFO:tasks.workunit.client.0.smithi028.stdout:8/892: creat d0/d41/d50/d3/d35/d86/f124 x:0 0 0 2024-07-03T15:44:17.945 INFO:tasks.workunit.client.0.smithi028.stdout:6/919: link d4/d12/d2c/ddd/f103 d4/d78/d10c/d38/d59/f145 0 2024-07-03T15:44:17.945 INFO:tasks.workunit.client.0.smithi028.stdout:6/920: readlink d4/d78/d10c/d38/l75 0 2024-07-03T15:44:17.951 INFO:tasks.workunit.client.0.smithi028.stdout:8/893: symlink d0/d41/d50/d97/d11f/l125 0 2024-07-03T15:44:17.951 INFO:tasks.workunit.client.0.smithi028.stdout:8/894: write d0/d41/d50/d3/d35/f11a [956996,118194] 0 2024-07-03T15:44:17.955 INFO:tasks.workunit.client.0.smithi028.stdout:8/895: mkdir d0/d41/d50/daf/dc8/d108/d126 0 2024-07-03T15:44:17.956 INFO:tasks.workunit.client.0.smithi028.stdout:8/896: mknod d0/d41/d50/d3/dc/d3f/d8e/d81/d100/c127 0 2024-07-03T15:44:17.962 INFO:tasks.workunit.client.0.smithi028.stdout:6/921: dwrite d4/d78/d10c/d38/d60/dc5/dcf/fe2 [0,4194304] 0 2024-07-03T15:44:17.964 INFO:tasks.workunit.client.0.smithi028.stdout:6/922: rename d4/d78/d10c/d1c/l108 to d4/d78/d10c/l146 0 2024-07-03T15:44:17.965 INFO:tasks.workunit.client.0.smithi028.stdout:6/923: mknod d4/d78/d10c/d38/d3d/d89/c147 0 2024-07-03T15:44:17.965 INFO:tasks.workunit.client.0.smithi028.stdout:6/924: creat d4/d15/d9d/f148 x:0 0 0 2024-07-03T15:44:17.970 INFO:tasks.workunit.client.0.smithi028.stdout:8/897: write d0/d6c/fae [668304,94182] 0 2024-07-03T15:44:17.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.967+0000 7fd64ca7c640 1 -- 172.21.15.28:0/3092458178 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd648066440 msgr2=0x7fd6480fefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:17.971 INFO:tasks.workunit.client.0.smithi028.stdout:8/898: write d0/d41/d50/d3/d35/d86/f124 [415824,109558] 0 2024-07-03T15:44:17.971 INFO:tasks.workunit.client.0.smithi028.stdout:8/899: truncate d0/de0/dcf/dc0/ffd 77028 0 2024-07-03T15:44:17.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.967+0000 7fd64ca7c640 1 --2- 172.21.15.28:0/3092458178 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd648066440 0x7fd6480fefe0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd638009c50 tx=0x7fd638004d10 comp rx=0 tx=0).stop 2024-07-03T15:44:17.971 INFO:tasks.workunit.client.0.smithi028.stdout:8/900: dread - d0/d41/d50/daf/dc8/d108/f123 zero size 2024-07-03T15:44:17.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.967+0000 7fd64ca7c640 1 -- 172.21.15.28:0/3092458178 shutdown_connections 2024-07-03T15:44:17.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.967+0000 7fd64ca7c640 1 --2- 172.21.15.28:0/3092458178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6480ff520 0x7fd648101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:17.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.967+0000 7fd64ca7c640 1 --2- 172.21.15.28:0/3092458178 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd648066440 0x7fd6480fefe0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:17.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.967+0000 7fd64ca7c640 1 -- 172.21.15.28:0/3092458178 >> 172.21.15.28:0/3092458178 conn(0x7fd6480f6a30 msgr2=0x7fd6480f8e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:44:17.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.967+0000 7fd64ca7c640 1 -- 172.21.15.28:0/3092458178 shutdown_connections 2024-07-03T15:44:17.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.967+0000 7fd64ca7c640 1 -- 172.21.15.28:0/3092458178 wait complete. 2024-07-03T15:44:17.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.968+0000 7fd64ca7c640 1 Processor -- start 2024-07-03T15:44:17.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.968+0000 7fd64ca7c640 1 -- start start 2024-07-03T15:44:17.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.968+0000 7fd64ca7c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd648066440 0x7fd6480fc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:44:17.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.968+0000 7fd64ca7c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6480ff520 0x7fd6480faee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:44:17.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.968+0000 7fd64ca7c640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6480fb500 con 0x7fd6480ff520 2024-07-03T15:44:17.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.968+0000 7fd64ca7c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6480fb640 con 0x7fd648066440 2024-07-03T15:44:17.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.968+0000 7fd63ffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6480ff520 0x7fd6480faee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:44:17.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.968+0000 7fd63ffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6480ff520 0x7fd6480faee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50484/0 (socket says 172.21.15.28:50484) 2024-07-03T15:44:17.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.968+0000 7fd63ffff640 1 -- 172.21.15.28:0/3544844550 learned_addr learned my addr 172.21.15.28:0/3544844550 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:44:17.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.969+0000 7fd63ffff640 1 -- 172.21.15.28:0/3544844550 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd648066440 msgr2=0x7fd6480fc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:17.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.969+0000 7fd63ffff640 1 --2- 172.21.15.28:0/3544844550 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd648066440 0x7fd6480fc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:17.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.969+0000 7fd63ffff640 1 -- 172.21.15.28:0/3544844550 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6380098b0 con 0x7fd6480ff520 2024-07-03T15:44:17.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.969+0000 7fd63ffff640 1 --2- 172.21.15.28:0/3544844550 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6480ff520 0x7fd6480faee0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd63000b580 tx=0x7fd6300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:44:17.974 INFO:tasks.workunit.client.0.smithi028.stdout:8/901: truncate d0/d41/d50/d3/d35/d86/f87 2192816 0 2024-07-03T15:44:17.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.969+0000 7fd63effd640 1 -- 172.21.15.28:0/3544844550 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd63000f570 con 0x7fd6480ff520 2024-07-03T15:44:17.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.969+0000 7fd63effd640 1 -- 172.21.15.28:0/3544844550 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd63000fba0 con 0x7fd6480ff520 2024-07-03T15:44:17.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.969+0000 7fd63effd640 1 -- 172.21.15.28:0/3544844550 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd630010dd0 con 0x7fd6480ff520 2024-07-03T15:44:17.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.969+0000 7fd64ca7c640 1 -- 172.21.15.28:0/3544844550 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6480fb920 con 0x7fd6480ff520 2024-07-03T15:44:17.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.970+0000 7fd64ca7c640 1 -- 172.21.15.28:0/3544844550 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6481150c0 con 0x7fd6480ff520 2024-07-03T15:44:17.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.971+0000 7fd64ca7c640 1 -- 172.21.15.28:0/3544844550 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6480fa6b0 con 0x7fd6480ff520 2024-07-03T15:44:17.976 INFO:tasks.workunit.client.0.smithi028.stdout:8/902: mknod d0/d41/d50/d3/d71/c128 0 2024-07-03T15:44:17.976 INFO:tasks.workunit.client.0.smithi028.stdout:8/903: chown d0/d41/d50/d3/dc/d4d/fc6 125335 1 2024-07-03T15:44:17.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.972+0000 7fd63effd640 1 -- 172.21.15.28:0/3544844550 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 24) v1 ==== 88617+0+0 (secure 0 0 0) 0x7fd63000f6d0 con 0x7fd6480ff520 2024-07-03T15:44:17.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.973+0000 7fd63effd640 1 --2- 172.21.15.28:0/3544844550 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fd63406a6d0 0x7fd63406cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:44:17.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.973+0000 7fd63effd640 1 -- 172.21.15.28:0/3544844550 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fd6300895c0 con 0x7fd6480ff520 2024-07-03T15:44:17.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.974+0000 7fd64659c640 1 --2- 172.21.15.28:0/3544844550 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fd63406a6d0 0x7fd63406cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:44:17.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.974+0000 7fd64659c640 1 --2- 172.21.15.28:0/3544844550 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fd63406a6d0 0x7fd63406cb80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd638000f80 tx=0x7fd638004d10 comp rx=0 tx=0).ready entity=mgr.24403 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:44:17.978 INFO:tasks.workunit.client.0.smithi028.stdout:8/904: mkdir d0/de0/dcf/d129 0 2024-07-03T15:44:17.978 INFO:tasks.workunit.client.0.smithi028.stdout:8/905: chown d0/d41/d50/d3/d35/d86/fe1 802674 1 2024-07-03T15:44:17.980 INFO:tasks.workunit.client.0.smithi028.stdout:8/906: rename d0/d41/d50/d3/dc/d3f/d8e/d81/d100/c127 to d0/de0/c12a 0 2024-07-03T15:44:17.980 INFO:tasks.workunit.client.0.smithi028.stdout:8/907: chown d0/d41/d50/d3/dc/d3f/d8e/d81/d100 15132 1 2024-07-03T15:44:17.980 INFO:tasks.workunit.client.0.smithi028.stdout:8/908: chown d0/d41/d50/d3/dc/d33/d7a/f2f 427024960 1 2024-07-03T15:44:17.981 INFO:tasks.workunit.client.0.smithi028.stdout:6/925: dwrite d4/d78/fcb [0,4194304] 0 2024-07-03T15:44:17.982 INFO:tasks.workunit.client.0.smithi028.stdout:6/926: write d4/d78/d10c/d1c/faa [899094,123988] 0 2024-07-03T15:44:17.982 INFO:tasks.workunit.client.0.smithi028.stdout:8/909: getdents d0/d41/d50/d3/d35/d68/d22/d40/db7 0 2024-07-03T15:44:17.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:17.980+0000 7fd63effd640 1 -- 172.21.15.28:0/3544844550 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7fd630053300 con 0x7fd6480ff520 2024-07-03T15:44:17.984 INFO:tasks.workunit.client.0.smithi028.stdout:6/927: mknod d4/d12/d2c/ddd/c149 0 2024-07-03T15:44:17.987 INFO:tasks.workunit.client.0.smithi028.stdout:8/910: truncate d0/f5b 1982448 0 2024-07-03T15:44:17.988 INFO:tasks.workunit.client.0.smithi028.stdout:6/928: truncate d4/d12/d2c/d31/f64 997750 0 2024-07-03T15:44:18.040 INFO:tasks.workunit.client.0.smithi028.stdout:2/629: sync 2024-07-03T15:44:18.043 INFO:tasks.workunit.client.0.smithi028.stdout:6/929: link d4/d12/d2c/ddd/fec d4/d78/d10c/d1c/f14a 0 2024-07-03T15:44:18.049 INFO:tasks.workunit.client.0.smithi028.stdout:6/930: link d4/d15/d9d/f148 d4/d78/d10c/d47/d66/daf/f14b 0 2024-07-03T15:44:18.049 INFO:tasks.workunit.client.0.smithi028.stdout:6/931: write d4/d78/d10c/d47/f51 [8676513,34028] 0 2024-07-03T15:44:18.049 INFO:tasks.workunit.client.0.smithi028.stdout:6/932: dread - d4/d15/de5/ffc zero size 2024-07-03T15:44:18.052 INFO:tasks.workunit.client.0.smithi028.stdout:6/933: dread d4/d12/d2c/d31/f64 [0,4194304] 0 2024-07-03T15:44:18.054 INFO:tasks.workunit.client.0.smithi028.stdout:6/934: rmdir d4/d11d 0 2024-07-03T15:44:18.054 INFO:tasks.workunit.client.0.smithi028.stdout:6/935: write d4/d78/d10c/d47/f51 [1948902,8074] 0 2024-07-03T15:44:18.055 INFO:tasks.workunit.client.0.smithi028.stdout:6/936: stat d4/d12/c2b 0 2024-07-03T15:44:18.055 INFO:tasks.workunit.client.0.smithi028.stdout:6/937: fsync d4/d78/d10c/d47/d9f/fc4 0 2024-07-03T15:44:18.061 INFO:tasks.workunit.client.0.smithi028.stdout:8/911: dwrite d0/de0/dcf/fa0 [0,4194304] 0 2024-07-03T15:44:18.066 INFO:tasks.workunit.client.0.smithi028.stdout:2/630: dwrite da/d1e/d56/f7a [0,4194304] 0 2024-07-03T15:44:18.071 INFO:tasks.workunit.client.0.smithi028.stdout:8/912: write d0/d41/d50/d3/dc/d3f/f61 [3415368,101804] 0 2024-07-03T15:44:18.071 INFO:tasks.workunit.client.0.smithi028.stdout:2/631: symlink da/d24/d2c/dc6/ldb 0 2024-07-03T15:44:18.072 INFO:tasks.workunit.client.0.smithi028.stdout:2/632: creat da/d1e/dd0/fdc x:0 0 0 2024-07-03T15:44:18.079 INFO:tasks.workunit.client.0.smithi028.stdout:2/633: dread da/d1e/f7b [0,4194304] 0 2024-07-03T15:44:18.079 INFO:tasks.workunit.client.0.smithi028.stdout:6/938: dwrite d4/d78/d10c/d38/f9b [0,4194304] 0 2024-07-03T15:44:18.079 INFO:tasks.workunit.client.0.smithi028.stdout:6/939: fsync d4/d78/d10c/d38/f40 0 2024-07-03T15:44:18.079 INFO:tasks.workunit.client.0.smithi028.stdout:6/940: chown d4/d12/d2c/dac/fff 1 1 2024-07-03T15:44:18.080 INFO:tasks.workunit.client.0.smithi028.stdout:2/634: creat da/d23/fdd x:0 0 0 2024-07-03T15:44:18.083 INFO:tasks.workunit.client.0.smithi028.stdout:6/941: symlink d4/d78/df0/l14c 0 2024-07-03T15:44:18.087 INFO:tasks.workunit.client.0.smithi028.stdout:8/913: dread d0/d41/d50/d3/dc/f1f [0,4194304] 0 2024-07-03T15:44:18.091 INFO:tasks.workunit.client.0.smithi028.stdout:2/635: dread da/d42/d47/d78/fb1 [8388608,4194304] 0 2024-07-03T15:44:18.091 INFO:tasks.workunit.client.0.smithi028.stdout:2/636: truncate da/d1e/fc9 973580 0 2024-07-03T15:44:18.099 INFO:tasks.workunit.client.0.smithi028.stdout:2/637: dread da/d24/f66 [0,4194304] 0 2024-07-03T15:44:18.102 INFO:tasks.workunit.client.0.smithi028.stdout:2/638: link da/f12 da/d42/d47/fde 0 2024-07-03T15:44:18.105 INFO:tasks.workunit.client.0.smithi028.stdout:6/942: dwrite d4/d15/de5/f129 [0,4194304] 0 2024-07-03T15:44:18.106 INFO:tasks.workunit.client.0.smithi028.stdout:6/943: dread d4/d12/d2c/df4/f11a [0,4194304] 0 2024-07-03T15:44:18.107 INFO:tasks.workunit.client.0.smithi028.stdout:8/914: dwrite d0/d41/d50/daf/feb [0,4194304] 0 2024-07-03T15:44:18.107 INFO:tasks.workunit.client.0.smithi028.stdout:8/915: write d0/d6c/da9/fd7 [168291,15929] 0 2024-07-03T15:44:18.110 INFO:tasks.workunit.client.0.smithi028.stdout:6/944: link d4/d78/d10c/d47/d66/l135 d4/d78/d10c/d56/l14d 0 2024-07-03T15:44:18.110 INFO:tasks.workunit.client.0.smithi028.stdout:6/945: dread - d4/d78/d10c/d47/d9f/d124/f13d zero size 2024-07-03T15:44:18.110 INFO:tasks.workunit.client.0.smithi028.stdout:6/946: creat d4/d78/d10c/d38/d60/dc5/dcf/f14e x:0 0 0 2024-07-03T15:44:18.110 INFO:tasks.workunit.client.0.smithi028.stdout:6/947: creat d4/d78/d10c/d1c/df7/f14f x:0 0 0 2024-07-03T15:44:18.116 INFO:tasks.workunit.client.0.smithi028.stdout:8/916: rmdir d0/d41/d50/d3/d35/d68 39 2024-07-03T15:44:18.119 INFO:tasks.workunit.client.0.smithi028.stdout:2/639: dwrite da/d23/f3b [0,4194304] 0 2024-07-03T15:44:18.119 INFO:tasks.workunit.client.0.smithi028.stdout:2/640: chown da/d1e/d56/dc0 127 1 2024-07-03T15:44:18.126 INFO:tasks.workunit.client.0.smithi028.stdout:2/641: rename da/d1e/d56/f7e to da/d42/d47/daa/fdf 0 2024-07-03T15:44:18.127 INFO:tasks.workunit.client.0.smithi028.stdout:6/948: dwrite d4/d78/d10c/d38/d60/dc5/dcf/fdb [0,4194304] 0 2024-07-03T15:44:18.128 INFO:tasks.workunit.client.0.smithi028.stdout:2/642: symlink da/d42/d9e/le0 0 2024-07-03T15:44:18.128 INFO:tasks.workunit.client.0.smithi028.stdout:2/643: dread - da/d42/d47/d78/fcd zero size 2024-07-03T15:44:18.137 INFO:tasks.workunit.client.0.smithi028.stdout:8/917: dwrite d0/d6c/fce [0,4194304] 0 2024-07-03T15:44:18.137 INFO:tasks.workunit.client.0.smithi028.stdout:6/949: rename d4/d78/d10c/d38/d59/d91/ce3 to d4/d78/d10c/d56/c150 0 2024-07-03T15:44:18.139 INFO:tasks.workunit.client.0.smithi028.stdout:6/950: creat d4/d78/d10c/d38/d60/dc5/dcf/f151 x:0 0 0 2024-07-03T15:44:18.139 INFO:tasks.workunit.client.0.smithi028.stdout:6/951: chown d4/d78/d10c/c87 3 1 2024-07-03T15:44:18.139 INFO:tasks.workunit.client.0.smithi028.stdout:6/952: chown d4/d78/d10c/d13/dbd/fe7 120 1 2024-07-03T15:44:18.152 INFO:tasks.workunit.client.0.smithi028.stdout:2/644: dwrite da/d24/d2c/faf [0,4194304] 0 2024-07-03T15:44:18.156 INFO:tasks.workunit.client.0.smithi028.stdout:6/953: dwrite f0 [4194304,4194304] 0 2024-07-03T15:44:18.156 INFO:tasks.workunit.client.0.smithi028.stdout:6/954: write d4/d12/d2c/df4/f11a [1203426,45621] 0 2024-07-03T15:44:18.160 INFO:tasks.workunit.client.0.smithi028.stdout:8/918: symlink d0/d41/d50/d97/l12b 0 2024-07-03T15:44:18.169 INFO:tasks.workunit.client.0.smithi028.stdout:8/919: dread d0/d41/d50/d3/dc/d3f/f61 [0,4194304] 0 2024-07-03T15:44:18.169 INFO:tasks.workunit.client.0.smithi028.stdout:8/920: creat d0/d41/d50/d3/dc/f12c x:0 0 0 2024-07-03T15:44:18.171 INFO:tasks.workunit.client.0.smithi028.stdout:2/645: mknod da/d24/d2c/ce1 0 2024-07-03T15:44:18.171 INFO:tasks.workunit.client.0.smithi028.stdout:2/646: write da/d42/f4d [1637904,38481] 0 2024-07-03T15:44:18.177 INFO:tasks.workunit.client.0.smithi028.stdout:6/955: symlink d4/d78/d10c/d47/l152 0 2024-07-03T15:44:18.177 INFO:tasks.workunit.client.0.smithi028.stdout:9/863: sync 2024-07-03T15:44:18.180 INFO:tasks.workunit.client.0.smithi028.stdout:8/921: mknod d0/d41/d50/d3/dc/d33/d7a/c12d 0 2024-07-03T15:44:18.180 INFO:tasks.workunit.client.0.smithi028.stdout:8/922: chown d0/d41/d50/d3/d35/d68/d22 7463 1 2024-07-03T15:44:18.180 INFO:tasks.workunit.client.0.smithi028.stdout:8/923: stat d0/d6c/da9 0 2024-07-03T15:44:18.186 INFO:tasks.workunit.client.0.smithi028.stdout:2/647: truncate da/d1e/f58 272022 0 2024-07-03T15:44:18.186 INFO:tasks.workunit.client.0.smithi028.stdout:2/648: chown da/d24/d2c/f97 2720 1 2024-07-03T15:44:18.189 INFO:tasks.workunit.client.0.smithi028.stdout:6/956: creat d4/d15/f153 x:0 0 0 2024-07-03T15:44:18.190 INFO:tasks.workunit.client.0.smithi028.stdout:8/924: mknod d0/d41/d50/d3/dc/d33/d7a/c12e 0 2024-07-03T15:44:18.193 INFO:tasks.workunit.client.0.smithi028.stdout:6/957: symlink d4/d78/d10c/d38/d59/d143/l154 0 2024-07-03T15:44:18.193 INFO:tasks.workunit.client.0.smithi028.stdout:6/958: fdatasync d4/d78/d10c/d47/d9f/f10f 0 2024-07-03T15:44:18.195 INFO:tasks.workunit.client.0.smithi028.stdout:9/864: symlink d1/d10/l12c 0 2024-07-03T15:44:18.200 INFO:tasks.workunit.client.0.smithi028.stdout:8/925: symlink d0/d41/d50/d3/dc/d33/d7a/l12f 0 2024-07-03T15:44:18.200 INFO:tasks.workunit.client.0.smithi028.stdout:8/926: dread - d0/d41/d50/daf/dc8/fe4 zero size 2024-07-03T15:44:18.200 INFO:tasks.workunit.client.0.smithi028.stdout:8/927: stat d0/de0/dcf/dc0/ffd 0 2024-07-03T15:44:18.200 INFO:tasks.workunit.client.0.smithi028.stdout:8/928: write d0/d41/d50/d3/dc/f1f [1665720,72879] 0 2024-07-03T15:44:18.202 INFO:tasks.workunit.client.0.smithi028.stdout:6/959: rename d4/d12/d2c/d31/l4b to d4/d12/d2c/d31/l155 0 2024-07-03T15:44:18.205 INFO:tasks.workunit.client.0.smithi028.stdout:9/865: link d1/d16/d57/f66 d1/d10/dea/f12d 0 2024-07-03T15:44:18.210 INFO:tasks.workunit.client.0.smithi028.stdout:8/929: mknod d0/d41/d50/d3/dc/d3f/d8e/d81/c130 0 2024-07-03T15:44:18.212 INFO:tasks.workunit.client.0.smithi028.stdout:6/960: symlink d4/d78/d10c/d1c/dfd/l156 0 2024-07-03T15:44:18.214 INFO:tasks.workunit.client.0.smithi028.stdout:8/930: mkdir d0/d41/d50/daf/dc1/d131 0 2024-07-03T15:44:18.217 INFO:tasks.workunit.client.0.smithi028.stdout:6/961: rename d4/d78/df0/f11b to d4/d78/d10c/d47/d66/daf/f157 0 2024-07-03T15:44:18.221 INFO:tasks.workunit.client.0.smithi028.stdout:8/931: rename d0/d41/d50/d3/dc/fcc to d0/d41/d50/d3/d35/d68/d22/d40/db7/f132 0 2024-07-03T15:44:18.224 INFO:tasks.workunit.client.0.smithi028.stdout:9/866: dwrite d1/d10/d28/d45/d61/d8c/fce [0,4194304] 0 2024-07-03T15:44:18.225 INFO:tasks.workunit.client.0.smithi028.stdout:6/962: rename d4/d12/d2c/df4/d119/fee to d4/d78/d10c/d38/d3d/f158 0 2024-07-03T15:44:18.225 INFO:tasks.workunit.client.0.smithi028.stdout:6/963: readlink d4/d15/l30 0 2024-07-03T15:44:18.231 INFO:tasks.workunit.client.0.smithi028.stdout:9/867: mknod d1/d10/d28/d45/d61/d8c/d77/db7/dab/dca/c12e 0 2024-07-03T15:44:18.231 INFO:tasks.workunit.client.0.smithi028.stdout:6/964: mkdir d4/d78/d10c/d1c/dfd/d159 0 2024-07-03T15:44:18.233 INFO:tasks.workunit.client.0.smithi028.stdout:9/868: dread d1/d10/d28/f79 [0,4194304] 0 2024-07-03T15:44:18.233 INFO:tasks.workunit.client.0.smithi028.stdout:9/869: write d1/d16/f1c [2648384,6167] 0 2024-07-03T15:44:18.234 INFO:tasks.workunit.client.0.smithi028.stdout:9/870: unlink d1/d10/d28/d45/d61/d8c/d77/l109 0 2024-07-03T15:44:18.241 INFO:tasks.workunit.client.0.smithi028.stdout:6/965: dread d4/d12/d2c/fe0 [0,4194304] 0 2024-07-03T15:44:18.254 INFO:tasks.workunit.client.0.smithi028.stdout:6/966: write d4/d78/d10c/d47/f6b [1671848,60464] 0 2024-07-03T15:44:18.254 INFO:tasks.workunit.client.0.smithi028.stdout:6/967: creat d4/d15/f15a x:0 0 0 2024-07-03T15:44:18.255 INFO:tasks.workunit.client.0.smithi028.stdout:9/871: dwrite d1/d10/d28/d45/d61/d8c/f81 [8388608,4194304] 0 2024-07-03T15:44:18.255 INFO:tasks.workunit.client.0.smithi028.stdout:9/872: stat d1/d8e/f119 0 2024-07-03T15:44:18.256 INFO:tasks.workunit.client.0.smithi028.stdout:6/968: symlink d4/d78/d10c/d38/d59/d143/l15b 0 2024-07-03T15:44:18.257 INFO:tasks.workunit.client.0.smithi028.stdout:6/969: creat d4/d12/dc3/f15c x:0 0 0 2024-07-03T15:44:18.257 INFO:tasks.workunit.client.0.smithi028.stdout:6/970: truncate d4/d78/d10c/d1c/f98 88858 0 2024-07-03T15:44:18.260 INFO:tasks.workunit.client.0.smithi028.stdout:9/873: creat d1/d16/d57/f12f x:0 0 0 2024-07-03T15:44:18.260 INFO:tasks.workunit.client.0.smithi028.stdout:9/874: write d1/d10/d28/d45/d61/d8c/d77/db7/f6c [1886416,40972] 0 2024-07-03T15:44:18.261 INFO:tasks.workunit.client.0.smithi028.stdout:6/971: dread d4/d78/d10c/d47/d9f/f10f [0,4194304] 0 2024-07-03T15:44:18.261 INFO:tasks.workunit.client.0.smithi028.stdout:6/972: truncate d4/d78/d10c/d1c/dfd/f141 201255 0 2024-07-03T15:44:18.282 INFO:tasks.workunit.client.0.smithi028.stdout:6/973: dwrite d4/d12/f13b [0,4194304] 0 2024-07-03T15:44:18.285 INFO:tasks.workunit.client.0.smithi028.stdout:9/875: dwrite d1/d10/d28/d45/d61/f101 [0,4194304] 0 2024-07-03T15:44:18.289 INFO:tasks.workunit.client.0.smithi028.stdout:9/876: mkdir d1/d16/d57/d5f/dd8/d130 0 2024-07-03T15:44:18.290 INFO:tasks.workunit.client.0.smithi028.stdout:9/877: creat d1/d10/d28/d45/d61/d8c/f131 x:0 0 0 2024-07-03T15:44:18.290 INFO:tasks.workunit.client.0.smithi028.stdout:9/878: getdents d1/d16/d57/d5f/de3/d115 0 2024-07-03T15:44:18.291 INFO:tasks.workunit.client.0.smithi028.stdout:9/879: getdents d1/d10/d28/d4e 0 2024-07-03T15:44:18.302 INFO:tasks.workunit.client.0.smithi028.stdout:6/974: dwrite d4/d78/d10c/fa8 [0,4194304] 0 2024-07-03T15:44:18.302 INFO:tasks.workunit.client.0.smithi028.stdout:6/975: creat d4/d78/d10c/d1c/df7/f15d x:0 0 0 2024-07-03T15:44:18.307 INFO:tasks.workunit.client.0.smithi028.stdout:9/880: dread d1/d16/d57/f9c [0,4194304] 0 2024-07-03T15:44:18.307 INFO:tasks.workunit.client.0.smithi028.stdout:9/881: chown d1/dd/f6f 0 1 2024-07-03T15:44:18.307 INFO:tasks.workunit.client.0.smithi028.stdout:9/882: readlink d1/d16/d57/d5f/d64/dc3/l129 0 2024-07-03T15:44:18.310 INFO:tasks.workunit.client.0.smithi028.stdout:6/976: unlink d4/d78/d10c/d38/d3d/d89/l104 0 2024-07-03T15:44:18.310 INFO:tasks.workunit.client.0.smithi028.stdout:9/883: read d1/d16/f19 [558285,112256] 0 2024-07-03T15:44:18.312 INFO:tasks.workunit.client.0.smithi028.stdout:6/977: symlink d4/d12/dc3/d11e/l15e 0 2024-07-03T15:44:18.314 INFO:tasks.workunit.client.0.smithi028.stdout:9/884: rename d1/d17/f25 to d1/d10/d76/dd7/f132 0 2024-07-03T15:44:18.314 INFO:tasks.workunit.client.0.smithi028.stdout:9/885: write d1/d10/d28/d45/d61/d8c/ffb [421906,26004] 0 2024-07-03T15:44:18.316 INFO:tasks.workunit.client.0.smithi028.stdout:6/978: dread d4/d12/fa9 [0,4194304] 0 2024-07-03T15:44:18.319 INFO:tasks.workunit.client.0.smithi028.stdout:9/886: mkdir d1/d133 0 2024-07-03T15:44:18.319 INFO:tasks.workunit.client.0.smithi028.stdout:9/887: creat d1/d16/d57/d5f/d64/f134 x:0 0 0 2024-07-03T15:44:18.322 INFO:tasks.workunit.client.0.smithi028.stdout:9/888: link d1/d10/d28/d4e/d5b/de9/c110 d1/d10/d28/d45/d61/d8c/d77/db7/c135 0 2024-07-03T15:44:18.326 INFO:tasks.workunit.client.0.smithi028.stdout:9/889: rename d1/d8e/fc6 to d1/d16/d57/d5f/d7f/f136 0 2024-07-03T15:44:18.327 INFO:tasks.workunit.client.0.smithi028.stdout:9/890: mknod d1/d10/d28/c137 0 2024-07-03T15:44:18.331 INFO:tasks.workunit.client.0.smithi028.stdout:9/891: mknod d1/d10/d28/d45/c138 0 2024-07-03T15:44:18.332 INFO:tasks.workunit.client.0.smithi028.stdout:9/892: write d1/d10/f3e [411356,125137] 0 2024-07-03T15:44:18.333 INFO:tasks.workunit.client.0.smithi028.stdout:9/893: write d1/d10/d28/d45/d61/d8c/d77/db7/fbd [346337,5982] 0 2024-07-03T15:44:18.333 INFO:tasks.workunit.client.0.smithi028.stdout:9/894: truncate d1/d10/d28/d45/d56/dcd/dd5/f10d 118051 0 2024-07-03T15:44:18.333 INFO:tasks.workunit.client.0.smithi028.stdout:9/895: chown d1/d16/d57/f71 12982 1 2024-07-03T15:44:18.333 INFO:tasks.workunit.client.0.smithi028.stdout:6/979: dwrite d4/d78/d10c/d38/d60/fab [0,4194304] 0 2024-07-03T15:44:18.335 INFO:tasks.workunit.client.0.smithi028.stdout:9/896: mknod d1/d16/c139 0 2024-07-03T15:44:18.335 INFO:tasks.workunit.client.0.smithi028.stdout:9/897: dread - d1/d8e/dac/fbf zero size 2024-07-03T15:44:18.337 INFO:tasks.workunit.client.0.smithi028.stdout:6/980: mknod d4/d12/dc3/d11e/c15f 0 2024-07-03T15:44:18.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.334+0000 7fd64ca7c640 1 -- 172.21.15.28:0/3544844550 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd648115350 con 0x7fd6480ff520 2024-07-03T15:44:18.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.334+0000 7fd63effd640 1 -- 172.21.15.28:0/3544844550 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1838 (secure 0 0 0) 0x7fd6300569c0 con 0x7fd6480ff520 2024-07-03T15:44:18.338 INFO:teuthology.orchestra.run.smithi028.stdout:e17 2024-07-03T15:44:18.338 INFO:teuthology.orchestra.run.smithi028.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-03T15:44:18.339 INFO:teuthology.orchestra.run.smithi028.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:44:18.339 INFO:teuthology.orchestra.run.smithi028.stdout:legacy client fscid: 1 2024-07-03T15:44:18.339 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:44:18.339 INFO:teuthology.orchestra.run.smithi028.stdout:Filesystem 'cephfs' (1) 2024-07-03T15:44:18.339 INFO:teuthology.orchestra.run.smithi028.stdout:fs_name cephfs 2024-07-03T15:44:18.339 INFO:teuthology.orchestra.run.smithi028.stdout:epoch 16 2024-07-03T15:44:18.339 INFO:teuthology.orchestra.run.smithi028.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-03T15:44:18.339 INFO:teuthology.orchestra.run.smithi028.stdout:created 2024-07-03T15:41:40.656298+0000 2024-07-03T15:44:18.339 INFO:teuthology.orchestra.run.smithi028.stdout:modified 2024-07-03T15:41:56.454984+0000 2024-07-03T15:44:18.340 INFO:teuthology.orchestra.run.smithi028.stdout:tableserver 0 2024-07-03T15:44:18.340 INFO:teuthology.orchestra.run.smithi028.stdout:root 0 2024-07-03T15:44:18.340 INFO:teuthology.orchestra.run.smithi028.stdout:session_timeout 60 2024-07-03T15:44:18.340 INFO:teuthology.orchestra.run.smithi028.stdout:session_autoclose 300 2024-07-03T15:44:18.340 INFO:teuthology.orchestra.run.smithi028.stdout:max_file_size 1099511627776 2024-07-03T15:44:18.340 INFO:teuthology.orchestra.run.smithi028.stdout:required_client_features {} 2024-07-03T15:44:18.340 INFO:teuthology.orchestra.run.smithi028.stdout:last_failure 0 2024-07-03T15:44:18.340 INFO:teuthology.orchestra.run.smithi028.stdout:last_failure_osd_epoch 0 2024-07-03T15:44:18.340 INFO:teuthology.orchestra.run.smithi028.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:44:18.340 INFO:teuthology.orchestra.run.smithi028.stdout:max_mds 1 2024-07-03T15:44:18.341 INFO:teuthology.orchestra.run.smithi028.stdout:in 0 2024-07-03T15:44:18.341 INFO:teuthology.orchestra.run.smithi028.stdout:up {0=14450} 2024-07-03T15:44:18.341 INFO:teuthology.orchestra.run.smithi028.stdout:failed 2024-07-03T15:44:18.341 INFO:teuthology.orchestra.run.smithi028.stdout:damaged 2024-07-03T15:44:18.341 INFO:teuthology.orchestra.run.smithi028.stdout:stopped 2024-07-03T15:44:18.341 INFO:teuthology.orchestra.run.smithi028.stdout:data_pools [3] 2024-07-03T15:44:18.341 INFO:teuthology.orchestra.run.smithi028.stdout:metadata_pool 2 2024-07-03T15:44:18.341 INFO:teuthology.orchestra.run.smithi028.stdout:inline_data disabled 2024-07-03T15:44:18.341 INFO:teuthology.orchestra.run.smithi028.stdout:balancer 2024-07-03T15:44:18.341 INFO:teuthology.orchestra.run.smithi028.stdout:standby_count_wanted 1 2024-07-03T15:44:18.342 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi028.rhpvve{0:14450} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.28:6826/1613936042,v1:172.21.15.28:6827/1613936042] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:44:18.342 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi028.wmbxew{0:14460} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.28:6828/1730336821,v1:172.21.15.28:6829/1730336821] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:44:18.342 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:44:18.342 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:44:18.342 INFO:teuthology.orchestra.run.smithi028.stdout:Standby daemons: 2024-07-03T15:44:18.342 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:44:18.342 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi148.szomfl{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/4171231858,v1:172.21.15.148:6827/4171231858] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:44:18.342 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi148.xgpuld{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/2347536398,v1:172.21.15.148:6825/2347536398] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:44:18.342 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 17 2024-07-03T15:44:18.343 INFO:tasks.workunit.client.0.smithi028.stdout:9/898: symlink d1/d16/d57/d5f/d64/l13a 0 2024-07-03T15:44:18.343 INFO:tasks.workunit.client.0.smithi028.stdout:9/899: truncate d1/d10/d28/d4e/f106 608186 0 2024-07-03T15:44:18.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.336+0000 7fd64ca7c640 1 -- 172.21.15.28:0/3544844550 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fd63406a6d0 msgr2=0x7fd63406cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:18.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.336+0000 7fd64ca7c640 1 --2- 172.21.15.28:0/3544844550 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fd63406a6d0 0x7fd63406cb80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd638000f80 tx=0x7fd638004d10 comp rx=0 tx=0).stop 2024-07-03T15:44:18.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.337+0000 7fd64ca7c640 1 -- 172.21.15.28:0/3544844550 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6480ff520 msgr2=0x7fd6480faee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:18.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.337+0000 7fd64ca7c640 1 --2- 172.21.15.28:0/3544844550 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6480ff520 0x7fd6480faee0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd63000b580 tx=0x7fd6300091a0 comp rx=0 tx=0).stop 2024-07-03T15:44:18.344 INFO:tasks.workunit.client.0.smithi028.stdout:6/981: mknod d4/d12/d2c/d12d/c160 0 2024-07-03T15:44:18.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.337+0000 7fd64ca7c640 1 -- 172.21.15.28:0/3544844550 shutdown_connections 2024-07-03T15:44:18.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.337+0000 7fd64ca7c640 1 --2- 172.21.15.28:0/3544844550 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fd63406a6d0 0x7fd63406cb80 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:18.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.337+0000 7fd64ca7c640 1 --2- 172.21.15.28:0/3544844550 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6480ff520 0x7fd6480faee0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:18.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.337+0000 7fd64ca7c640 1 --2- 172.21.15.28:0/3544844550 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd648066440 0x7fd6480fc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:18.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.337+0000 7fd64ca7c640 1 -- 172.21.15.28:0/3544844550 >> 172.21.15.28:0/3544844550 conn(0x7fd6480f6a30 msgr2=0x7fd6480f6e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:44:18.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.337+0000 7fd64ca7c640 1 -- 172.21.15.28:0/3544844550 shutdown_connections 2024-07-03T15:44:18.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.337+0000 7fd64ca7c640 1 -- 172.21.15.28:0/3544844550 wait complete. 2024-07-03T15:44:18.345 INFO:tasks.workunit.client.0.smithi028.stdout:9/900: unlink d1/d10/d28/cf7 0 2024-07-03T15:44:18.345 INFO:tasks.workunit.client.0.smithi028.stdout:9/901: chown d1/d16/d57/fb1 4451294 1 2024-07-03T15:44:18.345 INFO:tasks.workunit.client.0.smithi028.stdout:6/982: dread d4/d15/f6a [0,4194304] 0 2024-07-03T15:44:18.345 INFO:tasks.workunit.client.0.smithi028.stdout:9/902: symlink d1/d10/d28/d45/d61/d8c/d77/db7/dab/dcb/l13b 0 2024-07-03T15:44:18.348 INFO:tasks.workunit.client.0.smithi028.stdout:6/983: link d4/d78/d10c/f7c d4/d78/d10c/d38/d59/d91/f161 0 2024-07-03T15:44:18.348 INFO:tasks.workunit.client.0.smithi028.stdout:6/984: truncate d4/d12/d2c/dac/fff 1186881 0 2024-07-03T15:44:18.350 INFO:tasks.workunit.client.0.smithi028.stdout:9/903: rename d1/d16/d57/f66 to d1/d16/d57/d5f/d64/f13c 0 2024-07-03T15:44:18.356 INFO:tasks.workunit.client.0.smithi028.stdout:9/904: mkdir d1/d10/d76/d13d 0 2024-07-03T15:44:18.356 INFO:tasks.workunit.client.0.smithi028.stdout:6/985: dread d4/f10 [0,4194304] 0 2024-07-03T15:44:18.356 INFO:tasks.workunit.client.0.smithi028.stdout:6/986: chown d4/d78/d10c/d38 50317 1 2024-07-03T15:44:18.357 INFO:tasks.workunit.client.0.smithi028.stdout:9/905: truncate d1/d16/d57/f9c 4346816 0 2024-07-03T15:44:18.357 INFO:tasks.workunit.client.0.smithi028.stdout:9/906: chown d1/d16/f2e 57285 1 2024-07-03T15:44:18.360 INFO:tasks.workunit.client.0.smithi028.stdout:6/987: link d4/c3b d4/d78/d10c/d38/d60/dc5/dcf/d136/c162 0 2024-07-03T15:44:18.364 INFO:tasks.workunit.client.0.smithi028.stdout:6/988: link d4/d78/d10c/d13/dbd/c112 d4/d78/d10c/d47/d66/c163 0 2024-07-03T15:44:18.365 INFO:tasks.workunit.client.0.smithi028.stdout:9/907: dread d1/d10/d28/d45/d61/d8c/fce [0,4194304] 0 2024-07-03T15:44:18.366 INFO:tasks.workunit.client.0.smithi028.stdout:6/989: symlink d4/d78/d10c/d38/d60/l164 0 2024-07-03T15:44:18.368 INFO:tasks.workunit.client.0.smithi028.stdout:9/908: unlink d1/d10/d28/d4e/d5b/de9/lfc 0 2024-07-03T15:44:18.372 INFO:tasks.workunit.client.0.smithi028.stdout:9/909: rmdir d1/d10/d28/d4e 39 2024-07-03T15:44:18.372 INFO:tasks.workunit.client.0.smithi028.stdout:9/910: chown d1/d10/d28/d4e/d5b/cb2 0 1 2024-07-03T15:44:18.372 INFO:tasks.workunit.client.0.smithi028.stdout:9/911: write d1/d16/d57/fda [1728130,30222] 0 2024-07-03T15:44:18.373 INFO:tasks.workunit.client.0.smithi028.stdout:6/990: getdents d4/d78/d10c/d38/d60/dc5 0 2024-07-03T15:44:18.373 INFO:tasks.workunit.client.0.smithi028.stdout:6/991: write d4/d78/d10c/d47/d9f/d124/f13d [652868,13084] 0 2024-07-03T15:44:18.373 INFO:tasks.workunit.client.0.smithi028.stdout:6/992: readlink d4/d78/d10c/d38/d59/d143/l154 0 2024-07-03T15:44:18.373 INFO:tasks.workunit.client.0.smithi028.stdout:6/993: write d4/d78/d10c/d1c/f88 [2145083,114277] 0 2024-07-03T15:44:18.374 INFO:tasks.workunit.client.0.smithi028.stdout:9/912: truncate d1/d16/d57/d5f/fd4 2988744 0 2024-07-03T15:44:18.374 INFO:tasks.workunit.client.0.smithi028.stdout:9/913: chown d1/d10/d28/d45/d61/d8c/f6a 12707507 1 2024-07-03T15:44:18.376 INFO:tasks.workunit.client.0.smithi028.stdout:0/846: sync 2024-07-03T15:44:18.376 INFO:tasks.workunit.client.0.smithi028.stdout:3/780: sync 2024-07-03T15:44:18.377 INFO:tasks.workunit.client.0.smithi028.stdout:7/684: sync 2024-07-03T15:44:18.377 INFO:tasks.workunit.client.0.smithi028.stdout:7/685: creat d9/d1c/d45/db5/d92/d99/daf/dc4/fd1 x:0 0 0 2024-07-03T15:44:18.378 INFO:tasks.workunit.client.0.smithi028.stdout:4/919: sync 2024-07-03T15:44:18.378 INFO:tasks.workunit.client.0.smithi028.stdout:5/839: sync 2024-07-03T15:44:18.379 INFO:tasks.workunit.client.0.smithi028.stdout:6/994: creat d4/d78/d10c/d47/d66/f165 x:0 0 0 2024-07-03T15:44:18.386 INFO:tasks.workunit.client.0.smithi028.stdout:0/847: write d2/d3/dc4/d58/d6e/f80 [1622023,53848] 0 2024-07-03T15:44:18.387 INFO:tasks.workunit.client.0.smithi028.stdout:9/914: creat d1/d10/d28/d45/d61/d8c/d77/db7/dab/dca/d11d/f13e x:0 0 0 2024-07-03T15:44:18.393 INFO:tasks.workunit.client.0.smithi028.stdout:9/915: dread d1/dd/fb6 [0,4194304] 0 2024-07-03T15:44:18.393 INFO:tasks.workunit.client.0.smithi028.stdout:9/916: fsync d1/d10/d28/d4e/d5b/f84 0 2024-07-03T15:44:18.394 INFO:tasks.workunit.client.0.smithi028.stdout:5/840: unlink d0/dd/d13/d57/da2/de9/ff3 0 2024-07-03T15:44:18.395 INFO:tasks.workunit.client.0.smithi028.stdout:5/841: creat d0/dd/d13/d57/d26/ddf/f115 x:0 0 0 2024-07-03T15:44:18.396 INFO:tasks.workunit.client.0.smithi028.stdout:6/995: creat d4/d12/d2c/df4/f166 x:0 0 0 2024-07-03T15:44:18.399 INFO:tasks.workunit.client.0.smithi028.stdout:3/781: dwrite d18/d1a/f4f [0,4194304] 0 2024-07-03T15:44:18.403 INFO:tasks.workunit.client.0.smithi028.stdout:7/686: dwrite d9/f25 [0,4194304] 0 2024-07-03T15:44:18.403 INFO:tasks.workunit.client.0.smithi028.stdout:0/848: dwrite d2/d1c/d6c/f79 [0,4194304] 0 2024-07-03T15:44:18.404 INFO:tasks.workunit.client.0.smithi028.stdout:9/917: mkdir d1/d13f 0 2024-07-03T15:44:18.408 INFO:tasks.workunit.client.0.smithi028.stdout:5/842: mknod d0/dd/d13/d29/c116 0 2024-07-03T15:44:18.409 INFO:tasks.workunit.client.0.smithi028.stdout:6/996: symlink d4/d12/d2c/d12d/l167 0 2024-07-03T15:44:18.412 INFO:tasks.workunit.client.0.smithi028.stdout:3/782: mknod d18/d1a/d1e/cfb 0 2024-07-03T15:44:18.413 INFO:tasks.workunit.client.0.smithi028.stdout:7/687: symlink d9/d1c/d45/d71/d78/ld2 0 2024-07-03T15:44:18.413 INFO:tasks.workunit.client.0.smithi028.stdout:7/688: write d9/d1c/d45/db5/fb0 [745972,82633] 0 2024-07-03T15:44:18.414 INFO:tasks.workunit.client.0.smithi028.stdout:0/849: rmdir d2/d3/d4d/d56/d78/da8/dd7 39 2024-07-03T15:44:18.419 INFO:tasks.workunit.client.0.smithi028.stdout:3/783: creat d18/d1a/d1e/d58/df3/ffc x:0 0 0 2024-07-03T15:44:18.422 INFO:tasks.workunit.client.0.smithi028.stdout:7/689: symlink d9/d1c/d86/ld3 0 2024-07-03T15:44:18.423 INFO:tasks.workunit.client.0.smithi028.stdout:0/850: symlink d2/d3/d4d/d56/dcd/l100 0 2024-07-03T15:44:18.423 INFO:tasks.workunit.client.0.smithi028.stdout:0/851: chown d2/d1c/c60 5 1 2024-07-03T15:44:18.423 INFO:tasks.workunit.client.0.smithi028.stdout:0/852: chown d2/d1c/f2d 719 1 2024-07-03T15:44:18.423 INFO:tasks.workunit.client.0.smithi028.stdout:6/997: write d4/d78/d10c/d38/d60/dc5/ffa [1932164,103157] 0 2024-07-03T15:44:18.424 INFO:tasks.workunit.client.0.smithi028.stdout:9/918: dwrite d1/d10/d28/d4e/d4f/f87 [0,4194304] 0 2024-07-03T15:44:18.425 INFO:tasks.workunit.client.0.smithi028.stdout:3/784: symlink d18/d3a/d9a/lfd 0 2024-07-03T15:44:18.425 INFO:tasks.workunit.client.0.smithi028.stdout:3/785: write d18/d1a/d1e/d58/f7f [901594,96456] 0 2024-07-03T15:44:18.426 INFO:tasks.workunit.client.0.smithi028.stdout:7/690: truncate d9/d1c/d45/db5/f2e 1419638 0 2024-07-03T15:44:18.426 INFO:tasks.workunit.client.0.smithi028.stdout:7/691: creat d9/d1c/d45/d71/fd4 x:0 0 0 2024-07-03T15:44:18.428 INFO:tasks.workunit.client.0.smithi028.stdout:5/843: rename d0/dd/d1e/d3b/d3f/d5a to d0/dd/d1e/d8e/d117 0 2024-07-03T15:44:18.432 INFO:tasks.workunit.client.0.smithi028.stdout:0/853: dread d2/d3/dc4/d36/d45/f7b [0,4194304] 0 2024-07-03T15:44:18.432 INFO:tasks.workunit.client.0.smithi028.stdout:0/854: readlink d2/lc7 0 2024-07-03T15:44:18.436 INFO:tasks.workunit.client.0.smithi028.stdout:6/998: write d4/d78/d10c/d1c/f61 [11336406,32239] 0 2024-07-03T15:44:18.436 INFO:tasks.workunit.client.0.smithi028.stdout:6/999: stat d4/d12/f4f 0 2024-07-03T15:44:18.441 INFO:tasks.workunit.client.0.smithi028.stdout:3/786: symlink d18/d1a/d1e/d58/df3/df9/lfe 0 2024-07-03T15:44:18.447 INFO:tasks.workunit.client.0.smithi028.stdout:9/919: rename d1/l15 to d1/d10/d28/d4e/d5b/df0/l140 0 2024-07-03T15:44:18.449 INFO:tasks.workunit.client.0.smithi028.stdout:0/855: mknod d2/d3/dc4/d58/d82/c101 0 2024-07-03T15:44:18.452 INFO:tasks.workunit.client.0.smithi028.stdout:3/787: creat d18/d3a/dc1/dd0/fff x:0 0 0 2024-07-03T15:44:18.455 INFO:tasks.workunit.client.0.smithi028.stdout:5/844: rename d0/dd/d13/d57/da2/fab to d0/dd/d13/d47/d50/f118 0 2024-07-03T15:44:18.456 INFO:tasks.workunit.client.0.smithi028.stdout:9/920: symlink d1/dd/l141 0 2024-07-03T15:44:18.457 INFO:tasks.workunit.client.0.smithi028.stdout:9/921: readlink d1/d16/d57/d5f/d7f/lfa 0 2024-07-03T15:44:18.457 INFO:tasks.workunit.client.0.smithi028.stdout:9/922: write d1/d17/f9a [7223109,88413] 0 2024-07-03T15:44:18.457 INFO:tasks.workunit.client.0.smithi028.stdout:9/923: readlink d1/d16/d57/d5f/d64/l13a 0 2024-07-03T15:44:18.457 INFO:tasks.workunit.client.0.smithi028.stdout:9/924: write d1/d10/d28/d45/d61/d8c/d77/db7/fbd [705441,34218] 0 2024-07-03T15:44:18.460 INFO:tasks.workunit.client.0.smithi028.stdout:3/788: write d18/d3a/d9d/d8c/ff6 [3320258,77811] 0 2024-07-03T15:44:18.462 INFO:tasks.workunit.client.0.smithi028.stdout:0/856: link d2/d3/d4d/f9e d2/d1c/d6c/f102 0 2024-07-03T15:44:18.463 INFO:tasks.workunit.client.0.smithi028.stdout:5/845: link d0/dd/d1e/d3b/de0/dfa/d107/c10e d0/dd/df/da1/daa/c119 0 2024-07-03T15:44:18.471 INFO:tasks.workunit.client.0.smithi028.stdout:0/857: read d2/d1c/d6c/dc5/ffd [3228638,27013] 0 2024-07-03T15:44:18.474 INFO:tasks.workunit.client.0.smithi028.stdout:9/925: unlink d1/d8e/lf4 0 2024-07-03T15:44:18.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.472+0000 7fc4b9473640 1 -- 172.21.15.28:0/872521006 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4b40fc510 msgr2=0x7fc4b40fc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:18.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.472+0000 7fc4b9473640 1 --2- 172.21.15.28:0/872521006 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4b40fc510 0x7fc4b40fc980 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc49c0048d0 tx=0x7fc49c009da0 comp rx=0 tx=0).stop 2024-07-03T15:44:18.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.472+0000 7fc4b9473640 1 -- 172.21.15.28:0/872521006 shutdown_connections 2024-07-03T15:44:18.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.472+0000 7fc4b9473640 1 --2- 172.21.15.28:0/872521006 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4b40fc510 0x7fc4b40fc980 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:18.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.472+0000 7fc4b9473640 1 --2- 172.21.15.28:0/872521006 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4b40fb240 0x7fc4b40fb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:18.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.472+0000 7fc4b9473640 1 -- 172.21.15.28:0/872521006 >> 172.21.15.28:0/872521006 conn(0x7fc4b40f6a20 msgr2=0x7fc4b40f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:44:18.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.472+0000 7fc4b9473640 1 -- 172.21.15.28:0/872521006 shutdown_connections 2024-07-03T15:44:18.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.472+0000 7fc4b9473640 1 -- 172.21.15.28:0/872521006 wait complete. 2024-07-03T15:44:18.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.473+0000 7fc4b9473640 1 Processor -- start 2024-07-03T15:44:18.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.473+0000 7fc4b9473640 1 -- start start 2024-07-03T15:44:18.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.473+0000 7fc4b9473640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4b40fb240 0x7fc4b4113560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:44:18.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.473+0000 7fc4b9473640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4b40fc510 0x7fc4b4113aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:44:18.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.473+0000 7fc4b9473640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b41140f0 con 0x7fc4b40fc510 2024-07-03T15:44:18.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.473+0000 7fc4b9473640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b4114910 con 0x7fc4b40fb240 2024-07-03T15:44:18.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.474+0000 7fc4b27fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4b40fc510 0x7fc4b4113aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:44:18.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.474+0000 7fc4b27fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4b40fc510 0x7fc4b4113aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50502/0 (socket says 172.21.15.28:50502) 2024-07-03T15:44:18.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.474+0000 7fc4b27fc640 1 -- 172.21.15.28:0/2753267334 learned_addr learned my addr 172.21.15.28:0/2753267334 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:44:18.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.474+0000 7fc4b2ffd640 1 --2- 172.21.15.28:0/2753267334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4b40fb240 0x7fc4b4113560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:44:18.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.474+0000 7fc4b27fc640 1 -- 172.21.15.28:0/2753267334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4b40fb240 msgr2=0x7fc4b4113560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:18.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.474+0000 7fc4b27fc640 1 --2- 172.21.15.28:0/2753267334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4b40fb240 0x7fc4b4113560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:18.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.474+0000 7fc4b27fc640 1 -- 172.21.15.28:0/2753267334 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc49c005040 con 0x7fc4b40fc510 2024-07-03T15:44:18.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.475+0000 7fc4b27fc640 1 --2- 172.21.15.28:0/2753267334 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4b40fc510 0x7fc4b4113aa0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc49c0048a0 tx=0x7fc49c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:44:18.479 INFO:tasks.workunit.client.0.smithi028.stdout:3/789: rename d18/d43/f61 to d18/d1a/d5e/d66/da3/f100 0 2024-07-03T15:44:18.479 INFO:tasks.workunit.client.0.smithi028.stdout:3/790: stat d18/d1a/d64/ddc 0 2024-07-03T15:44:18.480 INFO:tasks.workunit.client.0.smithi028.stdout:3/791: read - d18/d6f/d78/d87/d9c/fd4 zero size 2024-07-03T15:44:18.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.476+0000 7fc4ab7fe640 1 -- 172.21.15.28:0/2753267334 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc49c038070 con 0x7fc4b40fc510 2024-07-03T15:44:18.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.476+0000 7fc4ab7fe640 1 -- 172.21.15.28:0/2753267334 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc49c032b10 con 0x7fc4b40fc510 2024-07-03T15:44:18.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.476+0000 7fc4ab7fe640 1 -- 172.21.15.28:0/2753267334 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc49c02a9f0 con 0x7fc4b40fc510 2024-07-03T15:44:18.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.476+0000 7fc4b9473640 1 -- 172.21.15.28:0/2753267334 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4b4114b40 con 0x7fc4b40fc510 2024-07-03T15:44:18.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.477+0000 7fc4b9473640 1 -- 172.21.15.28:0/2753267334 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4b4114fd0 con 0x7fc4b40fc510 2024-07-03T15:44:18.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.479+0000 7fc4ab7fe640 1 -- 172.21.15.28:0/2753267334 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 24) v1 ==== 88617+0+0 (secure 0 0 0) 0x7fc49c030020 con 0x7fc4b40fc510 2024-07-03T15:44:18.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.479+0000 7fc4ab7fe640 1 --2- 172.21.15.28:0/2753267334 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fc4a006a540 0x7fc4a006c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:44:18.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.480+0000 7fc4ab7fe640 1 -- 172.21.15.28:0/2753267334 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fc49c0a8bd0 con 0x7fc4b40fc510 2024-07-03T15:44:18.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.480+0000 7fc4b2ffd640 1 --2- 172.21.15.28:0/2753267334 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fc4a006a540 0x7fc4a006c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:44:18.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.480+0000 7fc4b2ffd640 1 --2- 172.21.15.28:0/2753267334 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fc4a006a540 0x7fc4a006c9f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac007f80 tx=0x7fc4ac004c70 comp rx=0 tx=0).ready entity=mgr.24403 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:44:18.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.481+0000 7fc4b9473640 1 -- 172.21.15.28:0/2753267334 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4b40fa6a0 con 0x7fc4b40fc510 2024-07-03T15:44:18.487 INFO:tasks.workunit.client.0.smithi028.stdout:0/858: dwrite d2/d1c/d6c/f85 [0,4194304] 0 2024-07-03T15:44:18.489 INFO:tasks.workunit.client.0.smithi028.stdout:9/926: truncate d1/dd/f6f 424146 0 2024-07-03T15:44:18.489 INFO:tasks.workunit.client.0.smithi028.stdout:9/927: dread - d1/d10/d28/d4e/d4f/fb3 zero size 2024-07-03T15:44:18.491 INFO:tasks.workunit.client.0.smithi028.stdout:3/792: symlink d18/d6f/l101 0 2024-07-03T15:44:18.491 INFO:tasks.workunit.client.0.smithi028.stdout:7/692: truncate d9/d1c/d45/db5/f2e 1196462 0 2024-07-03T15:44:18.491 INFO:tasks.workunit.client.0.smithi028.stdout:0/859: rmdir d2/d3/d4d/d56 39 2024-07-03T15:44:18.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.490+0000 7fc4ab7fe640 1 -- 172.21.15.28:0/2753267334 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7fc49c072850 con 0x7fc4b40fc510 2024-07-03T15:44:18.495 INFO:tasks.workunit.client.0.smithi028.stdout:9/928: truncate d1/d4a/f65 2312378 0 2024-07-03T15:44:18.495 INFO:tasks.workunit.client.0.smithi028.stdout:3/793: dread f6 [0,4194304] 0 2024-07-03T15:44:18.497 INFO:tasks.workunit.client.0.smithi028.stdout:7/693: unlink d9/d1c/d45/db5/d5d/l68 0 2024-07-03T15:44:18.497 INFO:tasks.workunit.client.0.smithi028.stdout:3/794: dread d18/d3a/d9d/d8c/f92 [4194304,4194304] 0 2024-07-03T15:44:18.499 INFO:tasks.workunit.client.0.smithi028.stdout:0/860: mknod d2/d3/d4d/d56/d78/da8/dd0/df7/c103 0 2024-07-03T15:44:18.502 INFO:tasks.workunit.client.0.smithi028.stdout:9/929: link d1/d16/d57/d5f/dd8/lec d1/d16/d57/d5f/de3/d115/l142 0 2024-07-03T15:44:18.502 INFO:tasks.workunit.client.0.smithi028.stdout:9/930: truncate d1/d96/fc9 209148 0 2024-07-03T15:44:18.502 INFO:tasks.workunit.client.0.smithi028.stdout:9/931: readlink d1/l2c 0 2024-07-03T15:44:18.505 INFO:tasks.workunit.client.0.smithi028.stdout:3/795: creat d18/d3a/ddf/f102 x:0 0 0 2024-07-03T15:44:18.505 INFO:tasks.workunit.client.0.smithi028.stdout:3/796: chown d18/d1a/d1e/d58/d9f/l89 250879252 1 2024-07-03T15:44:18.505 INFO:tasks.workunit.client.0.smithi028.stdout:3/797: readlink d18/d1a/l7c 0 2024-07-03T15:44:18.507 INFO:tasks.workunit.client.0.smithi028.stdout:0/861: dread d2/f49 [0,4194304] 0 2024-07-03T15:44:18.510 INFO:tasks.workunit.client.0.smithi028.stdout:9/932: symlink d1/d8e/dac/l143 0 2024-07-03T15:44:18.512 INFO:tasks.workunit.client.0.smithi028.stdout:7/694: getdents d9/d1c/d7c 0 2024-07-03T15:44:18.512 INFO:tasks.workunit.client.0.smithi028.stdout:7/695: readlink d9/l32 0 2024-07-03T15:44:18.513 INFO:tasks.workunit.client.0.smithi028.stdout:2/649: sync 2024-07-03T15:44:18.514 INFO:tasks.workunit.client.0.smithi028.stdout:3/798: creat d18/d1a/d64/ddc/f103 x:0 0 0 2024-07-03T15:44:18.516 INFO:tasks.workunit.client.0.smithi028.stdout:9/933: creat d1/d10/d28/d45/d61/d8c/d77/db7/dab/dcb/f144 x:0 0 0 2024-07-03T15:44:18.518 INFO:tasks.workunit.client.0.smithi028.stdout:9/934: dread d1/d10/d28/d4e/d5b/fbc [0,4194304] 0 2024-07-03T15:44:18.522 INFO:tasks.workunit.client.0.smithi028.stdout:7/696: symlink d9/d1c/d7c/d8b/ld5 0 2024-07-03T15:44:18.522 INFO:tasks.workunit.client.0.smithi028.stdout:7/697: read d9/d1c/d45/db5/f5e [2660867,16527] 0 2024-07-03T15:44:18.522 INFO:tasks.workunit.client.0.smithi028.stdout:7/698: write d9/d1c/d45/db5/d92/d99/f58 [4807076,11015] 0 2024-07-03T15:44:18.522 INFO:tasks.workunit.client.0.smithi028.stdout:7/699: chown d9/d1c/d45/d71/d78/l95 275696689 1 2024-07-03T15:44:18.522 INFO:tasks.workunit.client.0.smithi028.stdout:7/700: stat d9/d1c/d7c/d8b/da0 0 2024-07-03T15:44:18.522 INFO:tasks.workunit.client.0.smithi028.stdout:2/650: getdents da/d1e/d56/dc0 0 2024-07-03T15:44:18.531 INFO:tasks.workunit.client.0.smithi028.stdout:9/935: unlink d1/d16/d57/d5f/d7f/c93 0 2024-07-03T15:44:18.532 INFO:tasks.workunit.client.0.smithi028.stdout:7/701: symlink d9/d1c/d45/db5/d92/d99/daf/ld6 0 2024-07-03T15:44:18.532 INFO:tasks.workunit.client.0.smithi028.stdout:9/936: creat d1/d10/d28/d4e/d4f/f145 x:0 0 0 2024-07-03T15:44:18.532 INFO:tasks.workunit.client.0.smithi028.stdout:3/799: rmdir d18/d1a/d1e/d58/d9f/da9 39 2024-07-03T15:44:18.532 INFO:tasks.workunit.client.0.smithi028.stdout:2/651: unlink da/d23/d30/f62 0 2024-07-03T15:44:18.537 INFO:tasks.workunit.client.0.smithi028.stdout:5/846: dwrite d0/dd/d13/d57/d26/d41/db0/fba [0,4194304] 0 2024-07-03T15:44:18.537 INFO:tasks.workunit.client.0.smithi028.stdout:2/652: write da/d1e/f7b [2532852,76668] 0 2024-07-03T15:44:18.537 INFO:tasks.workunit.client.0.smithi028.stdout:5/847: fdatasync d0/dd/df/faf 0 2024-07-03T15:44:18.538 INFO:tasks.workunit.client.0.smithi028.stdout:3/800: rmdir d18/d3a/dc1/dd0 39 2024-07-03T15:44:18.538 INFO:tasks.workunit.client.0.smithi028.stdout:3/801: dread - d18/d1a/d1e/fd2 zero size 2024-07-03T15:44:18.538 INFO:tasks.workunit.client.0.smithi028.stdout:9/937: read d1/d16/d57/f9c [3692250,48679] 0 2024-07-03T15:44:18.541 INFO:tasks.workunit.client.0.smithi028.stdout:5/848: mknod d0/dd/d13/c11a 0 2024-07-03T15:44:18.541 INFO:tasks.workunit.client.0.smithi028.stdout:5/849: readlink d0/dd/d13/d57/l3a 0 2024-07-03T15:44:18.546 INFO:tasks.workunit.client.0.smithi028.stdout:5/850: mkdir d0/d8f/db5/ddc/dff/d11b 0 2024-07-03T15:44:18.556 INFO:tasks.workunit.client.0.smithi028.stdout:7/702: dwrite d9/d1c/d6e/f76 [4194304,4194304] 0 2024-07-03T15:44:18.560 INFO:tasks.workunit.client.0.smithi028.stdout:9/938: rename d1/dd to d1/d10/d28/d4e/d5b/df0/d146 0 2024-07-03T15:44:18.561 INFO:tasks.workunit.client.0.smithi028.stdout:9/939: creat d1/d10/f147 x:0 0 0 2024-07-03T15:44:18.562 INFO:tasks.workunit.client.0.smithi028.stdout:3/802: dwrite d18/d1a/d1e/d58/d9f/fa2 [0,4194304] 0 2024-07-03T15:44:18.564 INFO:tasks.workunit.client.0.smithi028.stdout:9/940: mkdir d1/d8e/d148 0 2024-07-03T15:44:18.564 INFO:tasks.workunit.client.0.smithi028.stdout:9/941: stat d1/d16/d57/l5a 0 2024-07-03T15:44:18.564 INFO:tasks.workunit.client.0.smithi028.stdout:5/851: dwrite d0/dd/d13/d57/da2/de9/f101 [0,4194304] 0 2024-07-03T15:44:18.565 INFO:tasks.workunit.client.0.smithi028.stdout:3/803: mknod d18/d3a/ddf/c104 0 2024-07-03T15:44:18.565 INFO:tasks.workunit.client.0.smithi028.stdout:3/804: chown d18/d1a/d1e/d73/c98 95 1 2024-07-03T15:44:18.567 INFO:tasks.workunit.client.0.smithi028.stdout:9/942: symlink d1/d10/d28/d4e/d5b/l149 0 2024-07-03T15:44:18.575 INFO:tasks.workunit.client.0.smithi028.stdout:3/805: mkdir d18/d6f/d78/d105 0 2024-07-03T15:44:18.575 INFO:tasks.workunit.client.0.smithi028.stdout:3/806: chown d18/d6f/d78/d87/dca/fcc 123614046 1 2024-07-03T15:44:18.575 INFO:tasks.workunit.client.0.smithi028.stdout:3/807: read d18/d1a/d1e/f41 [398942,3921] 0 2024-07-03T15:44:18.577 INFO:tasks.workunit.client.0.smithi028.stdout:9/943: mkdir d1/d8e/dac/d14a 0 2024-07-03T15:44:18.577 INFO:tasks.workunit.client.0.smithi028.stdout:9/944: dread - d1/d16/d57/d5f/fe0 zero size 2024-07-03T15:44:18.579 INFO:tasks.workunit.client.0.smithi028.stdout:3/808: symlink d18/d3a/dbb/l106 0 2024-07-03T15:44:18.584 INFO:tasks.workunit.client.0.smithi028.stdout:5/852: dwrite d0/dd/d13/d57/d30/f90 [4194304,4194304] 0 2024-07-03T15:44:18.585 INFO:tasks.workunit.client.0.smithi028.stdout:7/703: dwrite d9/f10 [4194304,4194304] 0 2024-07-03T15:44:18.585 INFO:tasks.workunit.client.0.smithi028.stdout:3/809: dread d18/fb3 [0,4194304] 0 2024-07-03T15:44:18.586 INFO:tasks.workunit.client.0.smithi028.stdout:7/704: readlink d9/l9b 0 2024-07-03T15:44:18.591 INFO:tasks.workunit.client.0.smithi028.stdout:9/945: write d1/d10/d28/d45/d61/d8c/d77/f85 [1238972,71699] 0 2024-07-03T15:44:18.591 INFO:tasks.workunit.client.0.smithi028.stdout:9/946: fsync d1/d16/d57/d5f/d7f/f136 0 2024-07-03T15:44:18.591 INFO:tasks.workunit.client.0.smithi028.stdout:3/810: link d18/d3a/d9a/fa0 d18/d1a/d64/f107 0 2024-07-03T15:44:18.592 INFO:tasks.workunit.client.0.smithi028.stdout:7/705: mkdir d9/d1c/d7c/d8b/dd7 0 2024-07-03T15:44:18.594 INFO:tasks.workunit.client.0.smithi028.stdout:9/947: rename d1/d10/d28/d4e/d5b/df0/d146/l94 to d1/d10/d28/d45/d61/d8c/d77/db7/dab/dcb/l14b 0 2024-07-03T15:44:18.594 INFO:tasks.workunit.client.0.smithi028.stdout:9/948: dread - d1/d16/d57/d5f/d7f/f136 zero size 2024-07-03T15:44:18.596 INFO:tasks.workunit.client.0.smithi028.stdout:3/811: creat d18/f108 x:0 0 0 2024-07-03T15:44:18.598 INFO:tasks.workunit.client.0.smithi028.stdout:7/706: symlink d9/d1c/da9/dc1/dc5/ld8 0 2024-07-03T15:44:18.598 INFO:tasks.workunit.client.0.smithi028.stdout:7/707: getdents d9/d1c/d7c/d8b/dd7 0 2024-07-03T15:44:18.598 INFO:tasks.workunit.client.0.smithi028.stdout:7/708: creat d9/d1c/d45/db5/d92/d99/daf/dc4/fd9 x:0 0 0 2024-07-03T15:44:18.598 INFO:tasks.workunit.client.0.smithi028.stdout:9/949: rename d1/l3f to d1/d16/d57/d5f/dd8/l14c 0 2024-07-03T15:44:18.599 INFO:tasks.workunit.client.0.smithi028.stdout:9/950: chown d1/d10/d28/d45/d61/d8c/d77/db7/dab/dcb/fe2 222490 1 2024-07-03T15:44:18.599 INFO:tasks.workunit.client.0.smithi028.stdout:5/853: dwrite d0/dd/d13/d57/d26/d41/ffc [0,4194304] 0 2024-07-03T15:44:18.604 INFO:tasks.workunit.client.0.smithi028.stdout:3/812: rename d18/d1a/d1e/d58/ddd to d18/d1a/d109 0 2024-07-03T15:44:18.604 INFO:tasks.workunit.client.0.smithi028.stdout:3/813: stat d18/d1a/d1e/d58/f6c 0 2024-07-03T15:44:18.604 INFO:tasks.workunit.client.0.smithi028.stdout:3/814: chown d18/d1a/d5e/d66/f95 10 1 2024-07-03T15:44:18.605 INFO:tasks.workunit.client.0.smithi028.stdout:7/709: dread d9/d1c/d45/db5/d92/d99/d6a/f7d [0,4194304] 0 2024-07-03T15:44:18.605 INFO:tasks.workunit.client.0.smithi028.stdout:9/951: creat d1/d10/dea/f14d x:0 0 0 2024-07-03T15:44:18.606 INFO:tasks.workunit.client.0.smithi028.stdout:5/854: dread d0/dd/d1e/d3b/f4f [0,4194304] 0 2024-07-03T15:44:18.606 INFO:tasks.workunit.client.0.smithi028.stdout:5/855: chown d0/dd/d13/d57/da2 235 1 2024-07-03T15:44:18.609 INFO:tasks.workunit.client.0.smithi028.stdout:3/815: mkdir d18/d1a/d1e/d10a 0 2024-07-03T15:44:18.612 INFO:tasks.workunit.client.0.smithi028.stdout:7/710: unlink d9/d1c/da9/dc1/dc5/ld8 0 2024-07-03T15:44:18.612 INFO:tasks.workunit.client.0.smithi028.stdout:9/952: truncate d1/d10/f3e 745462 0 2024-07-03T15:44:18.616 INFO:tasks.workunit.client.0.smithi028.stdout:5/856: rename d0/dd/df/da7/ff2 to d0/dd/d13/d47/f11c 0 2024-07-03T15:44:18.616 INFO:tasks.workunit.client.0.smithi028.stdout:5/857: write d0/dd/d13/d57/d26/fe7 [493613,34967] 0 2024-07-03T15:44:18.619 INFO:tasks.workunit.client.0.smithi028.stdout:3/816: symlink d18/d6f/d78/l10b 0 2024-07-03T15:44:18.619 INFO:tasks.workunit.client.0.smithi028.stdout:3/817: fsync d18/d1a/d1e/d58/d9f/de2/ff1 0 2024-07-03T15:44:18.619 INFO:tasks.workunit.client.0.smithi028.stdout:7/711: write d9/d1c/f5c [3265828,106752] 0 2024-07-03T15:44:18.622 INFO:tasks.workunit.client.0.smithi028.stdout:9/953: unlink d1/d10/d28/d45/d61/d8c/d77/db7/dab/dcb/ccc 0 2024-07-03T15:44:18.634 INFO:tasks.workunit.client.0.smithi028.stdout:5/858: dread d0/dd/d1e/d8e/d117/f65 [0,4194304] 0 2024-07-03T15:44:18.643 INFO:tasks.workunit.client.0.smithi028.stdout:7/712: dwrite d9/d1c/d45/d71/f7f [0,4194304] 0 2024-07-03T15:44:18.643 INFO:tasks.workunit.client.0.smithi028.stdout:5/859: dread d0/dd/d13/d29/f4e [0,4194304] 0 2024-07-03T15:44:18.643 INFO:tasks.workunit.client.0.smithi028.stdout:7/713: fsync d9/d1c/f3f 0 2024-07-03T15:44:18.643 INFO:tasks.workunit.client.0.smithi028.stdout:7/714: readlink d9/d1c/l66 0 2024-07-03T15:44:18.669 INFO:tasks.workunit.client.0.smithi028.stdout:5/860: dwrite d0/dd/d13/d57/d26/ddf/f10b [0,4194304] 0 2024-07-03T15:44:18.670 INFO:tasks.workunit.client.0.smithi028.stdout:7/715: dwrite d9/d1c/d45/db5/d92/d99/daf/fbd [0,4194304] 0 2024-07-03T15:44:18.670 INFO:tasks.workunit.client.0.smithi028.stdout:7/716: readlink d9/d1c/da9/dc1/lc2 0 2024-07-03T15:44:18.673 INFO:tasks.workunit.client.0.smithi028.stdout:5/861: symlink d0/dd/df/da7/l11d 0 2024-07-03T15:44:18.673 INFO:tasks.workunit.client.0.smithi028.stdout:5/862: write d0/dd/d1e/d3b/de0/ff7 [699622,9099] 0 2024-07-03T15:44:18.681 INFO:tasks.workunit.client.0.smithi028.stdout:5/863: symlink d0/dd/d14/dc0/l11e 0 2024-07-03T15:44:18.684 INFO:tasks.workunit.client.0.smithi028.stdout:7/717: rename d9/d1c/da9/dc1 to d9/d1c/d45/db5/d92/d9e/dda 0 2024-07-03T15:44:18.684 INFO:tasks.workunit.client.0.smithi028.stdout:7/718: readlink d9/d1c/d45/d71/d78/ld2 0 2024-07-03T15:44:18.686 INFO:tasks.workunit.client.0.smithi028.stdout:5/864: getdents d0 0 2024-07-03T15:44:18.686 INFO:tasks.workunit.client.0.smithi028.stdout:5/865: stat d0/dd/d13/d57/d26/fe7 0 2024-07-03T15:44:18.686 INFO:tasks.workunit.client.0.smithi028.stdout:5/866: fsync d0/dd/d1e/f36 0 2024-07-03T15:44:18.687 INFO:tasks.workunit.client.0.smithi028.stdout:7/719: mkdir d9/d1c/d45/ddb 0 2024-07-03T15:44:18.688 INFO:tasks.workunit.client.0.smithi028.stdout:5/867: mkdir d0/dd/d13/d57/d77/d11f 0 2024-07-03T15:44:18.692 INFO:tasks.workunit.client.0.smithi028.stdout:5/868: rename d0/dd/d14/dc0/fd3 to d0/dd/d98/f120 0 2024-07-03T15:44:18.693 INFO:tasks.workunit.client.0.smithi028.stdout:5/869: write d0/dd/d13/d57/d26/ddf/f10b [4243075,42432] 0 2024-07-03T15:44:18.693 INFO:tasks.workunit.client.0.smithi028.stdout:5/870: stat d0/dd/df/f52 0 2024-07-03T15:44:18.693 INFO:tasks.workunit.client.0.smithi028.stdout:7/720: truncate d9/f34 4461469 0 2024-07-03T15:44:18.695 INFO:tasks.workunit.client.0.smithi028.stdout:7/721: creat d9/d1c/d7c/d8b/dd7/fdc x:0 0 0 2024-07-03T15:44:18.695 INFO:tasks.workunit.client.0.smithi028.stdout:7/722: truncate d9/d1c/d45/d71/fac 1015831 0 2024-07-03T15:44:18.696 INFO:tasks.workunit.client.0.smithi028.stdout:7/723: creat d9/d1c/d7c/d8b/da0/fdd x:0 0 0 2024-07-03T15:44:18.696 INFO:tasks.workunit.client.0.smithi028.stdout:7/724: chown d9/fa 3 1 2024-07-03T15:44:18.696 INFO:tasks.workunit.client.0.smithi028.stdout:7/725: chown d9/d1c/d6e/fbc 6 1 2024-07-03T15:44:18.698 INFO:tasks.workunit.client.0.smithi028.stdout:7/726: chown d9/d1c/l28 1 1 2024-07-03T15:44:18.706 INFO:tasks.workunit.client.0.smithi028.stdout:5/871: dwrite d0/dd/d1e/d8e/fb3 [0,4194304] 0 2024-07-03T15:44:18.706 INFO:tasks.workunit.client.0.smithi028.stdout:5/872: write d0/dd/d13/d47/d50/f118 [524852,120006] 0 2024-07-03T15:44:18.715 INFO:tasks.workunit.client.0.smithi028.stdout:5/873: link d0/dd/d13/d57/l3a d0/dd/d13/d29/l121 0 2024-07-03T15:44:18.715 INFO:tasks.workunit.client.0.smithi028.stdout:5/874: write d0/dd/d13/d47/d50/f8c [3540304,54133] 0 2024-07-03T15:44:18.731 INFO:tasks.workunit.client.0.smithi028.stdout:5/875: dwrite d0/dd/d1e/d8e/de4/f10a [0,4194304] 0 2024-07-03T15:44:18.734 INFO:tasks.workunit.client.0.smithi028.stdout:5/876: creat d0/d8f/db5/ddc/dff/f122 x:0 0 0 2024-07-03T15:44:18.735 INFO:tasks.workunit.client.0.smithi028.stdout:5/877: mkdir d0/d8f/db5/ddc/dff/d11b/d123 0 2024-07-03T15:44:18.744 INFO:tasks.workunit.client.0.smithi028.stdout:5/878: dread d0/dd/d1e/f7a [0,4194304] 0 2024-07-03T15:44:18.751 INFO:tasks.workunit.client.0.smithi028.stdout:5/879: rename d0/dd/d13/d57/d26/d41 to d0/dd/d14/dc0/d124 0 2024-07-03T15:44:18.758 INFO:tasks.workunit.client.0.smithi028.stdout:5/880: dread d0/dd/d14/dc0/d124/ffc [0,4194304] 0 2024-07-03T15:44:18.760 INFO:tasks.workunit.client.0.smithi028.stdout:5/881: unlink d0/dd/df/faf 0 2024-07-03T15:44:18.762 INFO:tasks.workunit.client.0.smithi028.stdout:5/882: mknod d0/dd/d13/d29/c125 0 2024-07-03T15:44:18.762 INFO:tasks.workunit.client.0.smithi028.stdout:5/883: chown d0/dd/d13/d47 1307942 1 2024-07-03T15:44:18.763 INFO:tasks.workunit.client.0.smithi028.stdout:5/884: rename d0/dd/d1e/d8e/fc6 to d0/dd/df/da1/f126 0 2024-07-03T15:44:18.765 INFO:tasks.workunit.client.0.smithi028.stdout:5/885: unlink d0/dd/d13/d57/d30/d70/d7f/dbb/f103 0 2024-07-03T15:44:18.766 INFO:tasks.workunit.client.0.smithi028.stdout:5/886: chown d0 798 1 2024-07-03T15:44:18.766 INFO:tasks.workunit.client.0.smithi028.stdout:5/887: chown d0/dd/d1e/d8e/d117/f78 923799 1 2024-07-03T15:44:18.766 INFO:tasks.workunit.client.0.smithi028.stdout:5/888: symlink d0/dd/d13/d57/d30/d70/d7f/d9e/dd6/l127 0 2024-07-03T15:44:18.780 INFO:tasks.workunit.client.0.smithi028.stdout:5/889: dwrite d0/dd/d13/d29/f4e [8388608,4194304] 0 2024-07-03T15:44:18.787 INFO:tasks.workunit.client.0.smithi028.stdout:5/890: dread d0/dd/d14/dc0/d124/f44 [0,4194304] 0 2024-07-03T15:44:18.790 INFO:tasks.workunit.client.0.smithi028.stdout:5/891: creat d0/d8f/db5/f128 x:0 0 0 2024-07-03T15:44:18.796 INFO:tasks.workunit.client.0.smithi028.stdout:5/892: dread d0/d8f/fc9 [0,4194304] 0 2024-07-03T15:44:18.805 INFO:tasks.workunit.client.0.smithi028.stdout:5/893: rename d0/dd/d13/d57/d30/d70/d7f/dbb to d0/dd/d1e/d3b/de0/d129 0 2024-07-03T15:44:18.805 INFO:tasks.workunit.client.0.smithi028.stdout:5/894: fsync d0/dd/f53 0 2024-07-03T15:44:18.810 INFO:tasks.workunit.client.0.smithi028.stdout:5/895: rename d0/dd/d1e/f89 to d0/dd/df/f12a 0 2024-07-03T15:44:18.810 INFO:tasks.workunit.client.0.smithi028.stdout:5/896: read d0/dd/fcd [746571,112464] 0 2024-07-03T15:44:18.812 INFO:tasks.workunit.client.0.smithi028.stdout:5/897: mknod d0/dd/d1e/d94/c12b 0 2024-07-03T15:44:18.815 INFO:tasks.workunit.client.0.smithi028.stdout:5/898: creat d0/dd/d13/d47/dbc/f12c x:0 0 0 2024-07-03T15:44:18.815 INFO:tasks.workunit.client.0.smithi028.stdout:5/899: chown d0/dd/d1e/d86/f88 201583 1 2024-07-03T15:44:18.818 INFO:tasks.workunit.client.0.smithi028.stdout:5/900: dread d0/dd/d14/dc0/d124/f44 [0,4194304] 0 2024-07-03T15:44:18.818 INFO:tasks.workunit.client.0.smithi028.stdout:5/901: creat d0/dd/d13/d29/f12d x:0 0 0 2024-07-03T15:44:18.822 INFO:tasks.workunit.client.0.smithi028.stdout:5/902: rename d0/dd/d13/d57/d77/fd1 to d0/dd/d1e/d8e/de4/d102/f12e 0 2024-07-03T15:44:18.822 INFO:tasks.workunit.client.0.smithi028.stdout:5/903: chown d0/dd/d13/d29/c116 580340 1 2024-07-03T15:44:18.830 INFO:tasks.workunit.client.0.smithi028.stdout:5/904: dread d0/dd/d13/d29/f4e [8388608,4194304] 0 2024-07-03T15:44:18.832 INFO:tasks.workunit.client.0.smithi028.stdout:5/905: mkdir d0/dd/d1e/d3b/de0/dfa/d107/d12f 0 2024-07-03T15:44:18.834 INFO:tasks.workunit.client.0.smithi028.stdout:5/906: creat d0/dd/d1e/d3b/d3f/d71/df4/f130 x:0 0 0 2024-07-03T15:44:18.834 INFO:tasks.workunit.client.0.smithi028.stdout:5/907: creat d0/dd/d13/d57/d26/ddf/f131 x:0 0 0 2024-07-03T15:44:18.834 INFO:tasks.workunit.client.0.smithi028.stdout:5/908: fsync d0/dd/d13/d57/d30/d70/d7f/d9e/fd2 0 2024-07-03T15:44:18.838 INFO:tasks.workunit.client.0.smithi028.stdout:8/932: sync 2024-07-03T15:44:18.840 INFO:tasks.workunit.client.0.smithi028.stdout:8/933: creat d0/d41/d50/d3/dc/d3f/d8e/d5d/f133 x:0 0 0 2024-07-03T15:44:18.849 INFO:tasks.workunit.client.0.smithi028.stdout:5/909: dwrite d0/dd/df/f16 [4194304,4194304] 0 2024-07-03T15:44:18.860 INFO:tasks.workunit.client.0.smithi028.stdout:8/934: dwrite d0/d41/d50/d3/dc/d33/d7a/f56 [0,4194304] 0 2024-07-03T15:44:18.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.862+0000 7fc4b9473640 1 -- 172.21.15.28:0/2753267334 --> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4b4102e50 con 0x7fc4a006a540 2024-07-03T15:44:18.865 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:18 smithi028 ceph-mon[29148]: from='client.14654 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:44:18.866 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:18 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/1760684325' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:44:18.866 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:18 smithi028 ceph-mon[29148]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:44:18.866 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:18 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/3544844550' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-03T15:44:18.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.863+0000 7fc4ab7fe640 1 -- 172.21.15.28:0/2753267334 <== mgr.24403 v2:172.21.15.148:6828/4285739175 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7fc4ac007c10 con 0x7fc4a006a540 2024-07-03T15:44:18.867 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-07-03T15:44:18.867 INFO:teuthology.orchestra.run.smithi028.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2", 2024-07-03T15:44:18.867 INFO:teuthology.orchestra.run.smithi028.stdout: "in_progress": true, 2024-07-03T15:44:18.867 INFO:teuthology.orchestra.run.smithi028.stdout: "which": "Upgrading all daemon types on all hosts", 2024-07-03T15:44:18.867 INFO:teuthology.orchestra.run.smithi028.stdout: "services_complete": [], 2024-07-03T15:44:18.867 INFO:teuthology.orchestra.run.smithi028.stdout: "progress": "1/21 daemons upgraded", 2024-07-03T15:44:18.867 INFO:teuthology.orchestra.run.smithi028.stdout: "message": "", 2024-07-03T15:44:18.867 INFO:teuthology.orchestra.run.smithi028.stdout: "is_paused": false 2024-07-03T15:44:18.867 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-07-03T15:44:18.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.865+0000 7fc4b9473640 1 -- 172.21.15.28:0/2753267334 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fc4a006a540 msgr2=0x7fc4a006c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:18.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.865+0000 7fc4b9473640 1 --2- 172.21.15.28:0/2753267334 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fc4a006a540 0x7fc4a006c9f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac007f80 tx=0x7fc4ac004c70 comp rx=0 tx=0).stop 2024-07-03T15:44:18.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.865+0000 7fc4b9473640 1 -- 172.21.15.28:0/2753267334 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4b40fc510 msgr2=0x7fc4b4113aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:18.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.865+0000 7fc4b9473640 1 --2- 172.21.15.28:0/2753267334 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4b40fc510 0x7fc4b4113aa0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc49c0048a0 tx=0x7fc49c009da0 comp rx=0 tx=0).stop 2024-07-03T15:44:18.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.865+0000 7fc4b9473640 1 -- 172.21.15.28:0/2753267334 shutdown_connections 2024-07-03T15:44:18.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.865+0000 7fc4b9473640 1 --2- 172.21.15.28:0/2753267334 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fc4a006a540 0x7fc4a006c9f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:18.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.865+0000 7fc4b9473640 1 --2- 172.21.15.28:0/2753267334 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4b40fc510 0x7fc4b4113aa0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:18.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.865+0000 7fc4b9473640 1 --2- 172.21.15.28:0/2753267334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4b40fb240 0x7fc4b4113560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:18.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.865+0000 7fc4b9473640 1 -- 172.21.15.28:0/2753267334 >> 172.21.15.28:0/2753267334 conn(0x7fc4b40f6a20 msgr2=0x7fc4b40f8100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:44:18.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.865+0000 7fc4b9473640 1 -- 172.21.15.28:0/2753267334 shutdown_connections 2024-07-03T15:44:18.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.865+0000 7fc4b9473640 1 -- 172.21.15.28:0/2753267334 wait complete. 2024-07-03T15:44:18.872 INFO:tasks.workunit.client.0.smithi028.stdout:5/910: dwrite d0/dd/d13/d57/d30/f90 [0,4194304] 0 2024-07-03T15:44:18.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.949+0000 7fa39c874640 1 -- 172.21.15.28:0/367209362 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3980fc330 msgr2=0x7fa3980fc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:18.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.949+0000 7fa39c874640 1 --2- 172.21.15.28:0/367209362 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3980fc330 0x7fa3980fc740 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa3880048d0 tx=0x7fa388009da0 comp rx=0 tx=0).stop 2024-07-03T15:44:18.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.949+0000 7fa39c874640 1 -- 172.21.15.28:0/367209362 shutdown_connections 2024-07-03T15:44:18.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.949+0000 7fa39c874640 1 --2- 172.21.15.28:0/367209362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3980fd600 0x7fa3980fda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:18.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.949+0000 7fa39c874640 1 --2- 172.21.15.28:0/367209362 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3980fc330 0x7fa3980fc740 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:18.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.949+0000 7fa39c874640 1 -- 172.21.15.28:0/367209362 >> 172.21.15.28:0/367209362 conn(0x7fa3980f7af0 msgr2=0x7fa3980f9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:44:18.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.949+0000 7fa39c874640 1 -- 172.21.15.28:0/367209362 shutdown_connections 2024-07-03T15:44:18.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.949+0000 7fa39c874640 1 -- 172.21.15.28:0/367209362 wait complete. 2024-07-03T15:44:18.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.949+0000 7fa39c874640 1 Processor -- start 2024-07-03T15:44:18.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.950+0000 7fa39c874640 1 -- start start 2024-07-03T15:44:18.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.950+0000 7fa39c874640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3980fc330 0x7fa39810cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:44:18.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.950+0000 7fa39c874640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3980fd600 0x7fa39810d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:44:18.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.950+0000 7fa39c874640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39810dad0 con 0x7fa3980fc330 2024-07-03T15:44:18.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.950+0000 7fa39c874640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39810e2f0 con 0x7fa3980fd600 2024-07-03T15:44:18.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.950+0000 7fa39659c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3980fc330 0x7fa39810cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:44:18.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.950+0000 7fa39659c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3980fc330 0x7fa39810cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50518/0 (socket says 172.21.15.28:50518) 2024-07-03T15:44:18.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.950+0000 7fa39659c640 1 -- 172.21.15.28:0/3192437799 learned_addr learned my addr 172.21.15.28:0/3192437799 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:44:18.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.950+0000 7fa395d9b640 1 --2- 172.21.15.28:0/3192437799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3980fd600 0x7fa39810d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:44:18.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.950+0000 7fa39659c640 1 -- 172.21.15.28:0/3192437799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3980fd600 msgr2=0x7fa39810d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:18.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.950+0000 7fa39659c640 1 --2- 172.21.15.28:0/3192437799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3980fd600 0x7fa39810d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:18.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.950+0000 7fa39659c640 1 -- 172.21.15.28:0/3192437799 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa388005040 con 0x7fa3980fc330 2024-07-03T15:44:18.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.950+0000 7fa395d9b640 1 --2- 172.21.15.28:0/3192437799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3980fd600 0x7fa39810d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-03T15:44:18.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.951+0000 7fa39659c640 1 --2- 172.21.15.28:0/3192437799 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3980fc330 0x7fa39810cf40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa388009f80 tx=0x7fa388009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:44:18.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.951+0000 7fa386ffd640 1 -- 172.21.15.28:0/3192437799 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa388038070 con 0x7fa3980fc330 2024-07-03T15:44:18.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.951+0000 7fa39c874640 1 -- 172.21.15.28:0/3192437799 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa39810e520 con 0x7fa3980fc330 2024-07-03T15:44:18.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.951+0000 7fa39c874640 1 -- 172.21.15.28:0/3192437799 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa39810ea60 con 0x7fa3980fc330 2024-07-03T15:44:18.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.951+0000 7fa386ffd640 1 -- 172.21.15.28:0/3192437799 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa388032970 con 0x7fa3980fc330 2024-07-03T15:44:18.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.951+0000 7fa386ffd640 1 -- 172.21.15.28:0/3192437799 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa388033860 con 0x7fa3980fc330 2024-07-03T15:44:18.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.952+0000 7fa39c874640 1 -- 172.21.15.28:0/3192437799 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3980fb790 con 0x7fa3980fc330 2024-07-03T15:44:18.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.953+0000 7fa386ffd640 1 -- 172.21.15.28:0/3192437799 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 24) v1 ==== 88617+0+0 (secure 0 0 0) 0x7fa388030020 con 0x7fa3980fc330 2024-07-03T15:44:18.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.953+0000 7fa386ffd640 1 --2- 172.21.15.28:0/3192437799 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fa38006a6d0 0x7fa38006cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:44:18.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.953+0000 7fa386ffd640 1 -- 172.21.15.28:0/3192437799 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fa3880a8e60 con 0x7fa3980fc330 2024-07-03T15:44:18.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.953+0000 7fa395d9b640 1 --2- 172.21.15.28:0/3192437799 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fa38006a6d0 0x7fa38006cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:44:18.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.954+0000 7fa395d9b640 1 --2- 172.21.15.28:0/3192437799 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fa38006a6d0 0x7fa38006cb80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa38c009fd0 tx=0x7fa38c009e50 comp rx=0 tx=0).ready entity=mgr.24403 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:44:18.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:18.957+0000 7fa386ffd640 1 -- 172.21.15.28:0/3192437799 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7fa388072ad0 con 0x7fa3980fc330 2024-07-03T15:44:18.982 INFO:tasks.workunit.client.0.smithi028.stdout:8/935: unlink d0/d41/d50/d3/dc/d33/d7a/c12e 0 2024-07-03T15:44:18.982 INFO:tasks.workunit.client.0.smithi028.stdout:8/936: fdatasync d0/d41/d50/daf/feb 0 2024-07-03T15:44:18.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:18 smithi148 ceph-mon[36056]: from='client.14654 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:44:18.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:18 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/1760684325' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:44:18.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:18 smithi148 ceph-mon[36056]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:44:18.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:18 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/3544844550' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-03T15:44:18.983 INFO:tasks.workunit.client.0.smithi028.stdout:5/911: truncate d0/dd/df/fc2 1875578 0 2024-07-03T15:44:18.984 INFO:tasks.workunit.client.0.smithi028.stdout:8/937: symlink d0/d6c/da9/l134 0 2024-07-03T15:44:18.986 INFO:tasks.workunit.client.0.smithi028.stdout:5/912: mkdir d0/dd/d13/d57/da2/d132 0 2024-07-03T15:44:18.986 INFO:tasks.workunit.client.0.smithi028.stdout:5/913: fdatasync d0/dd/d13/d29/f12d 0 2024-07-03T15:44:18.989 INFO:tasks.workunit.client.0.smithi028.stdout:8/938: symlink d0/d41/d50/daf/dc8/da6/da7/l135 0 2024-07-03T15:44:18.991 INFO:tasks.workunit.client.0.smithi028.stdout:5/914: mkdir d0/dd/d14/dc0/d124/db0/d133 0 2024-07-03T15:44:18.993 INFO:tasks.workunit.client.0.smithi028.stdout:8/939: rmdir d0/d41/d50/d3/dc/d3f/d8e/d81/dfc 39 2024-07-03T15:44:19.012 INFO:tasks.workunit.client.0.smithi028.stdout:4/920: sync 2024-07-03T15:44:19.016 INFO:tasks.workunit.client.0.smithi028.stdout:5/915: dwrite d0/dd/f53 [0,4194304] 0 2024-07-03T15:44:19.017 INFO:tasks.workunit.client.0.smithi028.stdout:8/940: dwrite d0/d41/d50/d3/d35/d68/d22/fd5 [0,4194304] 0 2024-07-03T15:44:19.018 INFO:tasks.workunit.client.0.smithi028.stdout:8/941: creat d0/d41/d50/d3/d35/d86/f136 x:0 0 0 2024-07-03T15:44:19.018 INFO:tasks.workunit.client.0.smithi028.stdout:5/916: read d0/dd/d13/d57/d26/fe7 [442322,46842] 0 2024-07-03T15:44:19.018 INFO:tasks.workunit.client.0.smithi028.stdout:5/917: creat d0/dd/d1e/d3b/de0/f134 x:0 0 0 2024-07-03T15:44:19.034 INFO:tasks.workunit.client.0.smithi028.stdout:4/921: dwrite d2/d5a/d45/d84/df4/d92/f136 [0,4194304] 0 2024-07-03T15:44:19.035 INFO:tasks.workunit.client.0.smithi028.stdout:4/922: stat d2/f52 0 2024-07-03T15:44:19.035 INFO:tasks.workunit.client.0.smithi028.stdout:4/923: stat d2/d12/d42/d63/d9e/cad 0 2024-07-03T15:44:19.042 INFO:tasks.workunit.client.0.smithi028.stdout:8/942: dwrite d0/d41/d50/d3/d35/d68/d22/fb1 [0,4194304] 0 2024-07-03T15:44:19.042 INFO:tasks.workunit.client.0.smithi028.stdout:8/943: chown d0/de0/lac 8 1 2024-07-03T15:44:19.042 INFO:tasks.workunit.client.0.smithi028.stdout:8/944: chown d0/l4e 1 1 2024-07-03T15:44:19.047 INFO:tasks.workunit.client.0.smithi028.stdout:8/945: mkdir d0/d41/d50/d3/dc/d3f/d8e/d5d/d137 0 2024-07-03T15:44:19.049 INFO:tasks.workunit.client.0.smithi028.stdout:8/946: mknod d0/d41/d50/daf/dc1/dd1/c138 0 2024-07-03T15:44:19.050 INFO:tasks.workunit.client.0.smithi028.stdout:5/918: dwrite d0/dd/d13/d57/d30/d70/d7f/d9e/fd2 [0,4194304] 0 2024-07-03T15:44:19.058 INFO:tasks.workunit.client.0.smithi028.stdout:5/919: rename d0/dd/d13/d57/d30/d70/d7f/fbd to d0/dd/d14/dc0/d124/f135 0 2024-07-03T15:44:19.059 INFO:tasks.workunit.client.0.smithi028.stdout:5/920: dread d0/dd/d14/fcb [0,4194304] 0 2024-07-03T15:44:19.059 INFO:tasks.workunit.client.0.smithi028.stdout:5/921: write d0/dd/d1e/d3b/de0/fe3 [2794306,50986] 0 2024-07-03T15:44:19.062 INFO:tasks.workunit.client.0.smithi028.stdout:4/924: dwrite d2/d3e/f50 [4194304,4194304] 0 2024-07-03T15:44:19.064 INFO:tasks.workunit.client.0.smithi028.stdout:8/947: dread d0/d41/d50/d3/dc/d33/d7a/f56 [0,4194304] 0 2024-07-03T15:44:19.064 INFO:tasks.workunit.client.0.smithi028.stdout:8/948: getdents d0/d41/d50/d3/dc/d33/dff 0 2024-07-03T15:44:19.068 INFO:tasks.workunit.client.0.smithi028.stdout:5/922: rename d0/dd/d13/d47/c60 to d0/dd/d1e/d8e/de4/d102/c136 0 2024-07-03T15:44:19.068 INFO:tasks.workunit.client.0.smithi028.stdout:5/923: creat d0/d8f/db5/f137 x:0 0 0 2024-07-03T15:44:19.072 INFO:tasks.workunit.client.0.smithi028.stdout:4/925: symlink d2/d12/d5b/d78/de4/d3a/l137 0 2024-07-03T15:44:19.072 INFO:tasks.workunit.client.0.smithi028.stdout:4/926: chown d2/d12/d5b/d98/l11c 40662774 1 2024-07-03T15:44:19.089 INFO:tasks.workunit.client.0.smithi028.stdout:5/924: dwrite d0/dd/d13/d57/d30/fde [0,4194304] 0 2024-07-03T15:44:19.091 INFO:tasks.workunit.client.0.smithi028.stdout:4/927: rename d2/d12/d42/d63/da8 to d2/d12/d42/d63/d97/d138 0 2024-07-03T15:44:19.091 INFO:tasks.workunit.client.0.smithi028.stdout:4/928: truncate d2/d5a/d45/d84/fe1 1247876 0 2024-07-03T15:44:19.091 INFO:tasks.workunit.client.0.smithi028.stdout:8/949: truncate d0/d41/d50/d3/dc/d3f/d8e/d81/d9e/fea 997673 0 2024-07-03T15:44:19.096 INFO:tasks.workunit.client.0.smithi028.stdout:4/929: symlink d2/d12/d5b/d78/de4/l139 0 2024-07-03T15:44:19.097 INFO:tasks.workunit.client.0.smithi028.stdout:4/930: creat d2/d12/d42/d63/d75/f13a x:0 0 0 2024-07-03T15:44:19.097 INFO:tasks.workunit.client.0.smithi028.stdout:4/931: fdatasync d2/d12/da6/fbf 0 2024-07-03T15:44:19.102 INFO:tasks.workunit.client.0.smithi028.stdout:8/950: mkdir d0/d41/d50/d3/d71/d139 0 2024-07-03T15:44:19.128 INFO:tasks.workunit.client.0.smithi028.stdout:4/932: dwrite d2/d3e/f3f [0,4194304] 0 2024-07-03T15:44:19.131 INFO:tasks.workunit.client.0.smithi028.stdout:5/925: dwrite d0/dd/d1e/d3b/f5e [4194304,4194304] 0 2024-07-03T15:44:19.135 INFO:tasks.workunit.client.0.smithi028.stdout:4/933: rename d2/d12/f17 to d2/d5a/d45/d84/df4/d9d/f13b 0 2024-07-03T15:44:19.135 INFO:tasks.workunit.client.0.smithi028.stdout:5/926: dread d0/dd/d1e/d86/f88 [0,4194304] 0 2024-07-03T15:44:19.140 INFO:tasks.workunit.client.0.smithi028.stdout:5/927: symlink d0/dd/d14/dc0/d124/d106/l138 0 2024-07-03T15:44:19.146 INFO:tasks.workunit.client.0.smithi028.stdout:5/928: mkdir d0/dd/df/da1/daa/d111/d139 0 2024-07-03T15:44:19.146 INFO:tasks.workunit.client.0.smithi028.stdout:5/929: truncate d0/dd/d13/d57/d26/ddf/f10b 4778447 0 2024-07-03T15:44:19.152 INFO:tasks.workunit.client.0.smithi028.stdout:5/930: unlink d0/dd/d13/d57/d30/d70/d7f/d9e/fd2 0 2024-07-03T15:44:19.160 INFO:tasks.workunit.client.0.smithi028.stdout:4/934: dwrite d2/d12/d5b/f101 [0,4194304] 0 2024-07-03T15:44:19.165 INFO:tasks.workunit.client.0.smithi028.stdout:4/935: unlink d2/f13 0 2024-07-03T15:44:19.170 INFO:tasks.workunit.client.0.smithi028.stdout:4/936: link d2/d5a/d45/d84/df4/d9d/fd1 d2/d5a/d45/d84/df4/d92/d12e/d106/f13c 0 2024-07-03T15:44:19.176 INFO:tasks.workunit.client.0.smithi028.stdout:5/931: dwrite d0/dd/df/f12a [0,4194304] 0 2024-07-03T15:44:19.176 INFO:tasks.workunit.client.0.smithi028.stdout:5/932: chown d0/dd/d1e/d86/f88 241 1 2024-07-03T15:44:19.176 INFO:tasks.workunit.client.0.smithi028.stdout:5/933: chown d0/dd/d13/d57/d30/fd0 215 1 2024-07-03T15:44:19.176 INFO:tasks.workunit.client.0.smithi028.stdout:5/934: truncate d0/dd/d1e/d8e/fa6 672891 0 2024-07-03T15:44:19.190 INFO:tasks.workunit.client.0.smithi028.stdout:5/935: creat d0/dd/d1e/d3b/ddd/f13a x:0 0 0 2024-07-03T15:44:19.191 INFO:tasks.workunit.client.0.smithi028.stdout:5/936: creat d0/dd/d13/d57/d77/d11f/f13b x:0 0 0 2024-07-03T15:44:19.192 INFO:tasks.workunit.client.0.smithi028.stdout:5/937: creat d0/dd/d13/d57/da2/d132/f13c x:0 0 0 2024-07-03T15:44:19.193 INFO:tasks.workunit.client.0.smithi028.stdout:5/938: creat d0/d8f/db5/ddc/dff/d11b/d123/f13d x:0 0 0 2024-07-03T15:44:19.193 INFO:tasks.workunit.client.0.smithi028.stdout:4/937: dwrite d2/d12/d5b/fea [0,4194304] 0 2024-07-03T15:44:19.194 INFO:tasks.workunit.client.0.smithi028.stdout:5/939: symlink d0/dd/d14/dc0/d124/db0/d133/l13e 0 2024-07-03T15:44:19.194 INFO:tasks.workunit.client.0.smithi028.stdout:5/940: creat d0/dd/df/da7/f13f x:0 0 0 2024-07-03T15:44:19.194 INFO:tasks.workunit.client.0.smithi028.stdout:5/941: write d0/dd/d13/d47/dbc/f12c [667841,37624] 0 2024-07-03T15:44:19.198 INFO:tasks.workunit.client.0.smithi028.stdout:5/942: mkdir d0/d8f/db5/ddc/d140 0 2024-07-03T15:44:19.199 INFO:tasks.workunit.client.0.smithi028.stdout:5/943: readlink d0/dd/d13/d57/l48 0 2024-07-03T15:44:19.199 INFO:tasks.workunit.client.0.smithi028.stdout:4/938: rmdir d2/d12/d42/d63/dfa 39 2024-07-03T15:44:19.207 INFO:tasks.workunit.client.0.smithi028.stdout:5/944: dread d0/dd/d14/dc0/d124/f135 [0,4194304] 0 2024-07-03T15:44:19.209 INFO:tasks.workunit.client.0.smithi028.stdout:5/945: mkdir d0/dd/d14/dc0/d141 0 2024-07-03T15:44:19.209 INFO:tasks.workunit.client.0.smithi028.stdout:5/946: fdatasync d0/d8f/fc9 0 2024-07-03T15:44:19.209 INFO:tasks.workunit.client.0.smithi028.stdout:5/947: dread - d0/d8f/db5/f128 zero size 2024-07-03T15:44:19.219 INFO:tasks.workunit.client.0.smithi028.stdout:5/948: dread d0/dd/d98/f6a [0,4194304] 0 2024-07-03T15:44:19.222 INFO:tasks.workunit.client.0.smithi028.stdout:5/949: truncate d0/dd/df/f12a 2591192 0 2024-07-03T15:44:19.224 INFO:tasks.workunit.client.0.smithi028.stdout:4/939: dwrite d2/d5a/f36 [0,4194304] 0 2024-07-03T15:44:19.224 INFO:tasks.workunit.client.0.smithi028.stdout:4/940: chown d2/l16 418910 1 2024-07-03T15:44:19.224 INFO:tasks.workunit.client.0.smithi028.stdout:4/941: write d2/d12/d5b/d98/fdd [438887,98179] 0 2024-07-03T15:44:19.224 INFO:tasks.workunit.client.0.smithi028.stdout:5/950: truncate d0/dd/df/da1/f126 1803415 0 2024-07-03T15:44:19.234 INFO:tasks.workunit.client.0.smithi028.stdout:4/942: symlink d2/d10d/l13d 0 2024-07-03T15:44:19.237 INFO:tasks.workunit.client.0.smithi028.stdout:5/951: link d0/dd/d1e/c91 d0/dd/d13/d57/d26/ddf/c142 0 2024-07-03T15:44:19.237 INFO:tasks.workunit.client.0.smithi028.stdout:5/952: write d0/dd/d1e/f7a [2155266,66566] 0 2024-07-03T15:44:19.238 INFO:tasks.workunit.client.0.smithi028.stdout:5/953: stat d0/dd/d13/d47/d50/l7c 0 2024-07-03T15:44:19.238 INFO:tasks.workunit.client.0.smithi028.stdout:5/954: write d0/dd/d13/d57/d26/ddf/f131 [497612,20558] 0 2024-07-03T15:44:19.241 INFO:tasks.workunit.client.0.smithi028.stdout:4/943: rename d2/d5a/d45/cc0 to d2/d12/d2e/df3/dfb/d115/c13e 0 2024-07-03T15:44:19.262 INFO:tasks.workunit.client.0.smithi028.stdout:5/955: dwrite d0/dd/d13/d57/fc1 [0,4194304] 0 2024-07-03T15:44:19.270 INFO:tasks.workunit.client.0.smithi028.stdout:5/956: dread d0/dd/d13/d29/f35 [8388608,4194304] 0 2024-07-03T15:44:19.270 INFO:tasks.workunit.client.0.smithi028.stdout:5/957: readlink d0/dd/d13/d57/d30/d70/d7f/lef 0 2024-07-03T15:44:19.270 INFO:tasks.workunit.client.0.smithi028.stdout:5/958: chown d0/l95 0 1 2024-07-03T15:44:19.271 INFO:tasks.workunit.client.0.smithi028.stdout:5/959: symlink d0/dd/d1e/d3b/ddd/l143 0 2024-07-03T15:44:19.273 INFO:tasks.workunit.client.0.smithi028.stdout:5/960: link d0/dd/df/f12a d0/dd/d13/d47/dbc/f144 0 2024-07-03T15:44:19.275 INFO:tasks.workunit.client.0.smithi028.stdout:5/961: getdents d0/dd/d14/dc0 0 2024-07-03T15:44:19.275 INFO:tasks.workunit.client.0.smithi028.stdout:5/962: readlink d0/dd/d14/dc0/l11e 0 2024-07-03T15:44:19.276 INFO:tasks.workunit.client.0.smithi028.stdout:5/963: chown d0/dd/d1e/d8e/d117/l8b 87628 1 2024-07-03T15:44:19.277 INFO:tasks.workunit.client.0.smithi028.stdout:5/964: link d0/dd/d13/c6d d0/dd/d13/d57/c145 0 2024-07-03T15:44:19.277 INFO:tasks.workunit.client.0.smithi028.stdout:5/965: creat d0/dd/df/da7/f146 x:0 0 0 2024-07-03T15:44:19.278 INFO:tasks.workunit.client.0.smithi028.stdout:5/966: write d0/dd/d13/d47/dbc/f12c [248208,60394] 0 2024-07-03T15:44:19.279 INFO:tasks.workunit.client.0.smithi028.stdout:5/967: unlink d0/dd/cb8 0 2024-07-03T15:44:19.281 INFO:tasks.workunit.client.0.smithi028.stdout:5/968: creat d0/dd/d1e/d3b/de0/dfa/f147 x:0 0 0 2024-07-03T15:44:19.281 INFO:tasks.workunit.client.0.smithi028.stdout:5/969: truncate d0/dd/d1e/d3b/de0/ff7 1475730 0 2024-07-03T15:44:19.293 INFO:tasks.workunit.client.0.smithi028.stdout:5/970: dwrite d0/dd/d1e/d3b/d3f/d71/df4/f130 [0,4194304] 0 2024-07-03T15:44:19.296 INFO:tasks.workunit.client.0.smithi028.stdout:5/971: mkdir d0/d148 0 2024-07-03T15:44:19.296 INFO:tasks.workunit.client.0.smithi028.stdout:5/972: chown d0/d8f/db5/f137 1229851 1 2024-07-03T15:44:19.298 INFO:tasks.workunit.client.0.smithi028.stdout:5/973: creat d0/dd/d14/dc0/d124/d106/f149 x:0 0 0 2024-07-03T15:44:19.298 INFO:tasks.workunit.client.0.smithi028.stdout:0/862: sync 2024-07-03T15:44:19.298 INFO:tasks.workunit.client.0.smithi028.stdout:7/727: sync 2024-07-03T15:44:19.298 INFO:tasks.workunit.client.0.smithi028.stdout:2/653: sync 2024-07-03T15:44:19.298 INFO:tasks.workunit.client.0.smithi028.stdout:2/654: creat da/d42/fe2 x:0 0 0 2024-07-03T15:44:19.298 INFO:tasks.workunit.client.0.smithi028.stdout:2/655: truncate da/d42/d47/d78/fa3 1187856 0 2024-07-03T15:44:19.298 INFO:tasks.workunit.client.0.smithi028.stdout:3/818: sync 2024-07-03T15:44:19.298 INFO:tasks.workunit.client.0.smithi028.stdout:9/954: sync 2024-07-03T15:44:19.299 INFO:tasks.workunit.client.0.smithi028.stdout:9/955: fdatasync d1/d10/d28/d45/d56/dcd/dd5/f118 0 2024-07-03T15:44:19.299 INFO:tasks.workunit.client.0.smithi028.stdout:9/956: write d1/d4a/f92 [2413216,66611] 0 2024-07-03T15:44:19.300 INFO:tasks.workunit.client.0.smithi028.stdout:9/957: write d1/d10/d28/fd0 [168721,41220] 0 2024-07-03T15:44:19.300 INFO:tasks.workunit.client.0.smithi028.stdout:9/958: write d1/d16/d57/d5f/d64/dc3/fdc [121641,12900] 0 2024-07-03T15:44:19.301 INFO:tasks.workunit.client.0.smithi028.stdout:0/863: dread d2/d3/d4d/d56/d78/da8/dd7/fe2 [0,4194304] 0 2024-07-03T15:44:19.306 INFO:tasks.workunit.client.0.smithi028.stdout:2/656: rename f9 to da/d1e/d68/fe3 0 2024-07-03T15:44:19.306 INFO:tasks.workunit.client.0.smithi028.stdout:3/819: mknod d18/d3a/db6/c10c 0 2024-07-03T15:44:19.306 INFO:tasks.workunit.client.0.smithi028.stdout:3/820: dread - d18/d1a/d1e/f82 zero size 2024-07-03T15:44:19.308 INFO:tasks.workunit.client.0.smithi028.stdout:0/864: mknod d2/d3/dc4/d36/d45/c104 0 2024-07-03T15:44:19.308 INFO:tasks.workunit.client.0.smithi028.stdout:0/865: chown d2/d3/c3c 115 1 2024-07-03T15:44:19.308 INFO:tasks.workunit.client.0.smithi028.stdout:0/866: chown d2/d3/d4d/f57 20429990 1 2024-07-03T15:44:19.308 INFO:tasks.workunit.client.0.smithi028.stdout:0/867: fdatasync d2/d3/d4d/d56/d78/da8/dd0/ff3 0 2024-07-03T15:44:19.309 INFO:tasks.workunit.client.0.smithi028.stdout:5/974: write d0/dd/df/f16 [7723341,66836] 0 2024-07-03T15:44:19.310 INFO:tasks.workunit.client.0.smithi028.stdout:9/959: dread d1/d10/d28/d45/d61/d8c/d77/db7/dab/dcb/fe4 [0,4194304] 0 2024-07-03T15:44:19.311 INFO:tasks.workunit.client.0.smithi028.stdout:9/960: write d1/d8e/f9e [285905,54378] 0 2024-07-03T15:44:19.311 INFO:tasks.workunit.client.0.smithi028.stdout:9/961: write d1/d10/d28/d45/d61/d8c/d77/db7/f6c [4044278,36250] 0 2024-07-03T15:44:19.313 INFO:tasks.workunit.client.0.smithi028.stdout:5/975: link d0/dd/d13/d57/d30/f90 d0/dd/d13/d29/dc4/f14a 0 2024-07-03T15:44:19.315 INFO:tasks.workunit.client.0.smithi028.stdout:2/657: symlink da/d23/le4 0 2024-07-03T15:44:19.315 INFO:tasks.workunit.client.0.smithi028.stdout:3/821: read d18/d1a/d1e/d58/d9f/fa2 [2078476,65077] 0 2024-07-03T15:44:19.315 INFO:tasks.workunit.client.0.smithi028.stdout:2/658: write da/d74/fc8 [20185,114297] 0 2024-07-03T15:44:19.316 INFO:tasks.workunit.client.0.smithi028.stdout:3/822: fsync d18/d6f/f7d 0 2024-07-03T15:44:19.316 INFO:tasks.workunit.client.0.smithi028.stdout:3/823: creat d18/d1a/d5e/d66/f10d x:0 0 0 2024-07-03T15:44:19.316 INFO:tasks.workunit.client.0.smithi028.stdout:3/824: chown f9 686 1 2024-07-03T15:44:19.317 INFO:tasks.workunit.client.0.smithi028.stdout:5/976: rename d0/dd/df/da1/daa/d111/d139 to d0/dd/d13/d47/d14b 0 2024-07-03T15:44:19.317 INFO:tasks.workunit.client.0.smithi028.stdout:5/977: write d0/d8f/fc9 [4514102,127287] 0 2024-07-03T15:44:19.318 INFO:tasks.workunit.client.0.smithi028.stdout:5/978: write d0/dd/d1e/f7a [3279207,21615] 0 2024-07-03T15:44:19.325 INFO:tasks.workunit.client.0.smithi028.stdout:5/979: dread d0/dd/df/fc2 [0,4194304] 0 2024-07-03T15:44:19.326 INFO:tasks.workunit.client.0.smithi028.stdout:0/868: dread d2/d3/d4d/f57 [0,4194304] 0 2024-07-03T15:44:19.327 INFO:tasks.workunit.client.0.smithi028.stdout:3/825: dread d18/d1a/d1e/f5d [0,4194304] 0 2024-07-03T15:44:19.327 INFO:tasks.workunit.client.0.smithi028.stdout:9/962: dread d1/d10/d28/d45/d61/d8c/fce [0,4194304] 0 2024-07-03T15:44:19.327 INFO:tasks.workunit.client.0.smithi028.stdout:9/963: write d1/d10/d28/d45/d61/d8c/d77/db7/dab/f114 [231377,98958] 0 2024-07-03T15:44:19.328 INFO:tasks.workunit.client.0.smithi028.stdout:7/728: dwrite d9/d1c/d45/db5/d92/d99/d6a/fad [0,4194304] 0 2024-07-03T15:44:19.330 INFO:tasks.workunit.client.0.smithi028.stdout:3/826: mkdir d18/d1a/d64/d10e 0 2024-07-03T15:44:19.330 INFO:tasks.workunit.client.0.smithi028.stdout:3/827: fdatasync d18/d1a/d1e/f41 0 2024-07-03T15:44:19.330 INFO:tasks.workunit.client.0.smithi028.stdout:9/964: link d1/d10/lf2 d1/d10/dea/l14e 0 2024-07-03T15:44:19.334 INFO:tasks.workunit.client.0.smithi028.stdout:3/828: link d18/d1a/d1e/d58/d9f/da9/fe4 d18/d1a/d5e/d66/da3/f10f 0 2024-07-03T15:44:19.334 INFO:tasks.workunit.client.0.smithi028.stdout:3/829: write d18/d1a/d1e/d58/df3/ffc [476383,29191] 0 2024-07-03T15:44:19.334 INFO:tasks.workunit.client.0.smithi028.stdout:3/830: truncate d18/d3a/ddf/f102 828570 0 2024-07-03T15:44:19.336 INFO:tasks.workunit.client.0.smithi028.stdout:2/659: dwrite da/d24/d2c/f97 [0,4194304] 0 2024-07-03T15:44:19.337 INFO:tasks.workunit.client.0.smithi028.stdout:3/831: mkdir d18/d3a/d9d/d110 0 2024-07-03T15:44:19.337 INFO:tasks.workunit.client.0.smithi028.stdout:3/832: fsync f17 0 2024-07-03T15:44:19.337 INFO:tasks.workunit.client.0.smithi028.stdout:3/833: truncate d18/d1a/d1e/f57 1125872 0 2024-07-03T15:44:19.337 INFO:tasks.workunit.client.0.smithi028.stdout:3/834: stat d18/d1a/d5e/f7b 0 2024-07-03T15:44:19.339 INFO:tasks.workunit.client.0.smithi028.stdout:5/980: dread d0/dd/d13/d29/f35 [0,4194304] 0 2024-07-03T15:44:19.339 INFO:tasks.workunit.client.0.smithi028.stdout:5/981: chown d0/dd/d13/d57/d30/d70/cf6 1810025 1 2024-07-03T15:44:19.341 INFO:tasks.workunit.client.0.smithi028.stdout:9/965: rmdir d1/d10/d28/d4e/d5b/de9 39 2024-07-03T15:44:19.346 INFO:tasks.workunit.client.0.smithi028.stdout:2/660: creat da/d24/d2c/dba/fe5 x:0 0 0 2024-07-03T15:44:19.346 INFO:tasks.workunit.client.0.smithi028.stdout:7/729: dread d9/d1c/d45/db5/f72 [0,4194304] 0 2024-07-03T15:44:19.347 INFO:tasks.workunit.client.0.smithi028.stdout:7/730: dread d9/f84 [0,4194304] 0 2024-07-03T15:44:19.348 INFO:tasks.workunit.client.0.smithi028.stdout:0/869: dwrite d2/d3/d4d/d56/fb8 [0,4194304] 0 2024-07-03T15:44:19.354 INFO:tasks.workunit.client.0.smithi028.stdout:3/835: dwrite d18/d3a/d9a/fcf [0,4194304] 0 2024-07-03T15:44:19.359 INFO:tasks.workunit.client.0.smithi028.stdout:9/966: mkdir d1/d8e/dac/d14a/d14f 0 2024-07-03T15:44:19.359 INFO:tasks.workunit.client.0.smithi028.stdout:9/967: stat d1/d8e/dac/fbf 0 2024-07-03T15:44:19.359 INFO:tasks.workunit.client.0.smithi028.stdout:9/968: truncate d1/d10/d28/d45/d61/d8c/d77/db7/dab/dca/d11d/f13e 434780 0 2024-07-03T15:44:19.361 INFO:tasks.workunit.client.0.smithi028.stdout:9/969: dread d1/d4a/f92 [4194304,4194304] 0 2024-07-03T15:44:19.361 INFO:tasks.workunit.client.0.smithi028.stdout:9/970: write d1/d10/d28/d45/d61/d8c/d77/f85 [4982044,28143] 0 2024-07-03T15:44:19.361 INFO:tasks.workunit.client.0.smithi028.stdout:9/971: creat d1/d4a/d7e/f150 x:0 0 0 2024-07-03T15:44:19.363 INFO:tasks.workunit.client.0.smithi028.stdout:2/661: dread da/d23/f3b [0,4194304] 0 2024-07-03T15:44:19.368 INFO:tasks.workunit.client.0.smithi028.stdout:7/731: rename d9/d1c/d45/l91 to d9/d1c/d45/db5/d92/lde 0 2024-07-03T15:44:19.370 INFO:tasks.workunit.client.0.smithi028.stdout:0/870: rmdir d2/d3/dc4/d58 39 2024-07-03T15:44:19.376 INFO:tasks.workunit.client.0.smithi028.stdout:2/662: symlink da/d42/d9e/le6 0 2024-07-03T15:44:19.377 INFO:tasks.workunit.client.0.smithi028.stdout:2/663: creat da/d42/d9e/fe7 x:0 0 0 2024-07-03T15:44:19.379 INFO:tasks.workunit.client.0.smithi028.stdout:3/836: dwrite d18/d3a/ddf/f102 [0,4194304] 0 2024-07-03T15:44:19.379 INFO:tasks.workunit.client.0.smithi028.stdout:3/837: creat d18/d3a/dbb/f111 x:0 0 0 2024-07-03T15:44:19.379 INFO:tasks.workunit.client.0.smithi028.stdout:9/972: dwrite d1/d16/f51 [0,4194304] 0 2024-07-03T15:44:19.380 INFO:tasks.workunit.client.0.smithi028.stdout:9/973: chown d1/d10/d28/d45/d61/d8c/d77/db7/dab/dcb/fe4 0 1 2024-07-03T15:44:19.384 INFO:tasks.workunit.client.0.smithi028.stdout:7/732: write d9/f48 [3492847,54481] 0 2024-07-03T15:44:19.386 INFO:tasks.workunit.client.0.smithi028.stdout:3/838: link d18/d1a/d1e/d73/lc2 d18/d1a/d1e/d73/l112 0 2024-07-03T15:44:19.387 INFO:tasks.workunit.client.0.smithi028.stdout:0/871: dwrite d2/d3/d4d/d56/d78/dcf/fdf [0,4194304] 0 2024-07-03T15:44:19.390 INFO:tasks.workunit.client.0.smithi028.stdout:0/872: chown d2/d3/dc4/d58/d82/de7 2812764 1 2024-07-03T15:44:19.391 INFO:tasks.workunit.client.0.smithi028.stdout:9/974: mkdir d1/d8e/d151 0 2024-07-03T15:44:19.399 INFO:tasks.workunit.client.0.smithi028.stdout:0/873: creat d2/d3/dc4/d58/d82/de7/f105 x:0 0 0 2024-07-03T15:44:19.399 INFO:tasks.workunit.client.0.smithi028.stdout:3/839: read d18/d1a/d1e/f53 [7081780,29417] 0 2024-07-03T15:44:19.399 INFO:tasks.workunit.client.0.smithi028.stdout:3/840: write d18/d1a/d64/ddc/f103 [248583,78034] 0 2024-07-03T15:44:19.399 INFO:tasks.workunit.client.0.smithi028.stdout:3/841: stat d18/d3a/d9d/d8c/faf 0 2024-07-03T15:44:19.399 INFO:tasks.workunit.client.0.smithi028.stdout:3/842: chown d18/d3a/c4b 179692 1 2024-07-03T15:44:19.402 INFO:tasks.workunit.client.0.smithi028.stdout:9/975: getdents d1/d17 0 2024-07-03T15:44:19.402 INFO:tasks.workunit.client.0.smithi028.stdout:9/976: truncate d1/d10/d28/d45/fa8 1909022 0 2024-07-03T15:44:19.403 INFO:tasks.workunit.client.0.smithi028.stdout:9/977: fdatasync d1/d10/d28/d45/d61/d8c/f131 0 2024-07-03T15:44:19.403 INFO:tasks.workunit.client.0.smithi028.stdout:9/978: chown d1/d10/d76/d13d 1 1 2024-07-03T15:44:19.404 INFO:tasks.workunit.client.0.smithi028.stdout:2/664: rename da/d23/d30 to da/d42/d47/de8 0 2024-07-03T15:44:19.404 INFO:tasks.workunit.client.0.smithi028.stdout:2/665: chown da/d42/d9e/le0 52 1 2024-07-03T15:44:19.406 INFO:tasks.workunit.client.0.smithi028.stdout:0/874: unlink d2/lc7 0 2024-07-03T15:44:19.406 INFO:tasks.workunit.client.0.smithi028.stdout:0/875: stat d2/c4e 0 2024-07-03T15:44:19.406 INFO:tasks.workunit.client.0.smithi028.stdout:0/876: fdatasync d2/d3/d4d/d56/d78/d9f/f6f 0 2024-07-03T15:44:19.407 INFO:tasks.workunit.client.0.smithi028.stdout:9/979: unlink d1/d16/d57/f12f 0 2024-07-03T15:44:19.409 INFO:tasks.workunit.client.0.smithi028.stdout:3/843: dread d18/d1a/d1e/f53 [0,4194304] 0 2024-07-03T15:44:19.409 INFO:tasks.workunit.client.0.smithi028.stdout:3/844: creat d18/d1a/d1e/f113 x:0 0 0 2024-07-03T15:44:19.409 INFO:tasks.workunit.client.0.smithi028.stdout:0/877: symlink d2/d3/d4d/d56/d78/da8/dd0/l106 0 2024-07-03T15:44:19.410 INFO:tasks.workunit.client.0.smithi028.stdout:0/878: chown d2/d3/d4d/d56/d78/dbe 32955782 1 2024-07-03T15:44:19.415 INFO:tasks.workunit.client.0.smithi028.stdout:9/980: rename d1/d10/d28/d45/fa8 to d1/d16/d57/d5f/d64/dc3/f152 0 2024-07-03T15:44:19.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:19.411+0000 7fa39c874640 1 -- 172.21.15.28:0/3192437799 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa39804b530 con 0x7fa3980fc330 2024-07-03T15:44:19.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:19.412+0000 7fa386ffd640 1 -- 172.21.15.28:0/3192437799 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fa388076190 con 0x7fa3980fc330 2024-07-03T15:44:19.416 INFO:teuthology.orchestra.run.smithi028.stdout:HEALTH_OK 2024-07-03T15:44:19.418 INFO:tasks.workunit.client.0.smithi028.stdout:3/845: write ff [2998803,35031] 0 2024-07-03T15:44:19.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:19.415+0000 7fa39c874640 1 -- 172.21.15.28:0/3192437799 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fa38006a6d0 msgr2=0x7fa38006cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:19.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:19.415+0000 7fa39c874640 1 --2- 172.21.15.28:0/3192437799 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fa38006a6d0 0x7fa38006cb80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa38c009fd0 tx=0x7fa38c009e50 comp rx=0 tx=0).stop 2024-07-03T15:44:19.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:19.415+0000 7fa39c874640 1 -- 172.21.15.28:0/3192437799 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3980fc330 msgr2=0x7fa39810cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:19.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:19.415+0000 7fa39c874640 1 --2- 172.21.15.28:0/3192437799 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3980fc330 0x7fa39810cf40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa388009f80 tx=0x7fa388009da0 comp rx=0 tx=0).stop 2024-07-03T15:44:19.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:19.415+0000 7fa39c874640 1 -- 172.21.15.28:0/3192437799 shutdown_connections 2024-07-03T15:44:19.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:19.415+0000 7fa39c874640 1 --2- 172.21.15.28:0/3192437799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3980fd600 0x7fa39810d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:19.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:19.415+0000 7fa39c874640 1 --2- 172.21.15.28:0/3192437799 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fa38006a6d0 0x7fa38006cb80 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:19.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:19.415+0000 7fa39c874640 1 --2- 172.21.15.28:0/3192437799 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3980fc330 0x7fa39810cf40 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:19.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:19.415+0000 7fa39c874640 1 -- 172.21.15.28:0/3192437799 >> 172.21.15.28:0/3192437799 conn(0x7fa3980f7af0 msgr2=0x7fa3980f91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:44:19.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:19.416+0000 7fa39c874640 1 -- 172.21.15.28:0/3192437799 shutdown_connections 2024-07-03T15:44:19.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:19.416+0000 7fa39c874640 1 -- 172.21.15.28:0/3192437799 wait complete. 2024-07-03T15:44:19.421 INFO:tasks.workunit.client.0.smithi028.stdout:2/666: dwrite da/d1e/fc9 [0,4194304] 0 2024-07-03T15:44:19.422 INFO:tasks.workunit.client.0.smithi028.stdout:0/879: unlink d2/d3/dc4/d36/d45/lfe 0 2024-07-03T15:44:19.422 INFO:tasks.workunit.client.0.smithi028.stdout:0/880: stat d2/d3/dc4/d36 0 2024-07-03T15:44:19.422 INFO:tasks.workunit.client.0.smithi028.stdout:0/881: creat d2/d3/dc4/d58/d82/d9b/f107 x:0 0 0 2024-07-03T15:44:19.422 INFO:tasks.workunit.client.0.smithi028.stdout:0/882: creat d2/d3/d4d/d56/d78/da8/f108 x:0 0 0 2024-07-03T15:44:19.429 INFO:tasks.workunit.client.0.smithi028.stdout:9/981: mkdir d1/d10/d28/d45/d61/d8c/d153 0 2024-07-03T15:44:19.431 INFO:tasks.workunit.client.0.smithi028.stdout:3/846: write d18/d3a/d9d/d8c/fdb [3373933,78382] 0 2024-07-03T15:44:19.433 INFO:tasks.workunit.client.0.smithi028.stdout:2/667: mknod da/d24/d2c/dba/ce9 0 2024-07-03T15:44:19.433 INFO:tasks.workunit.client.0.smithi028.stdout:2/668: creat da/d24/d2c/dba/fea x:0 0 0 2024-07-03T15:44:19.439 INFO:tasks.workunit.client.0.smithi028.stdout:9/982: creat d1/d10/d28/d45/f154 x:0 0 0 2024-07-03T15:44:19.441 INFO:tasks.workunit.client.0.smithi028.stdout:3/847: rmdir d18/d1a/d5e/de9 0 2024-07-03T15:44:19.441 INFO:tasks.workunit.client.0.smithi028.stdout:3/848: dread - d18/d3a/d9a/fee zero size 2024-07-03T15:44:19.443 INFO:tasks.workunit.client.0.smithi028.stdout:0/883: getdents d2/d1c 0 2024-07-03T15:44:19.443 INFO:tasks.workunit.client.0.smithi028.stdout:0/884: chown d2/d3/d4d/d56/d78/da8/dd7 418143701 1 2024-07-03T15:44:19.443 INFO:tasks.workunit.client.0.smithi028.stdout:0/885: chown d2/d1c/d6c/da9/ced 1964662 1 2024-07-03T15:44:19.446 INFO:tasks.workunit.client.0.smithi028.stdout:9/983: creat d1/d10/d28/d4e/f155 x:0 0 0 2024-07-03T15:44:19.447 INFO:tasks.workunit.client.0.smithi028.stdout:3/849: dread d18/d1a/d1e/f53 [4194304,4194304] 0 2024-07-03T15:44:19.447 INFO:tasks.workunit.client.0.smithi028.stdout:3/850: chown d18/d6f/ca1 15376242 1 2024-07-03T15:44:19.447 INFO:tasks.workunit.client.0.smithi028.stdout:3/851: chown d18/d1a/d64/lbe 2790 1 2024-07-03T15:44:19.447 INFO:tasks.workunit.client.0.smithi028.stdout:0/886: link d2/d3/dc4/d58/d82/fb6 d2/d3/d4d/d56/dcd/de4/f109 0 2024-07-03T15:44:19.450 INFO:tasks.workunit.client.0.smithi028.stdout:9/984: link d1/d10/d28/d45/d61/d8c/l121 d1/d10/d28/d45/d56/dcd/l156 0 2024-07-03T15:44:19.451 INFO:tasks.workunit.client.0.smithi028.stdout:2/669: dwrite da/d1e/f7b [0,4194304] 0 2024-07-03T15:44:19.454 INFO:tasks.workunit.client.0.smithi028.stdout:0/887: creat d2/d3/dc4/d58/d82/f10a x:0 0 0 2024-07-03T15:44:19.457 INFO:tasks.workunit.client.0.smithi028.stdout:9/985: mknod d1/d16/d57/d5f/dd8/c157 0 2024-07-03T15:44:19.464 INFO:tasks.workunit.client.0.smithi028.stdout:3/852: dwrite d18/d3a/d9d/d8c/db4/fe7 [0,4194304] 0 2024-07-03T15:44:19.464 INFO:tasks.workunit.client.0.smithi028.stdout:3/853: fdatasync d18/d1a/fc8 0 2024-07-03T15:44:19.464 INFO:tasks.workunit.client.0.smithi028.stdout:3/854: chown f17 11513684 1 2024-07-03T15:44:19.465 INFO:tasks.workunit.client.0.smithi028.stdout:3/855: write d18/d1a/d1e/d58/f7f [471702,46436] 0 2024-07-03T15:44:19.475 INFO:tasks.workunit.client.0.smithi028.stdout:2/670: dwrite da/d24/f55 [0,4194304] 0 2024-07-03T15:44:19.475 INFO:tasks.workunit.client.0.smithi028.stdout:0/888: dwrite d2/d3/fc6 [4194304,4194304] 0 2024-07-03T15:44:19.475 INFO:tasks.workunit.client.0.smithi028.stdout:0/889: chown d2/d3/d4d/d56/fb8 2098828 1 2024-07-03T15:44:19.480 INFO:tasks.workunit.client.0.smithi028.stdout:3/856: write f14 [3249123,9560] 0 2024-07-03T15:44:19.480 INFO:tasks.workunit.client.0.smithi028.stdout:3/857: fdatasync d18/d3a/d9a/fcf 0 2024-07-03T15:44:19.484 INFO:tasks.workunit.client.0.smithi028.stdout:3/858: mkdir d18/d1a/d1e/d58/d9f/da9/d114 0 2024-07-03T15:44:19.485 INFO:tasks.workunit.client.0.smithi028.stdout:2/671: dread da/d24/f55 [0,4194304] 0 2024-07-03T15:44:19.486 INFO:tasks.workunit.client.0.smithi028.stdout:2/672: creat da/d1e/dd0/feb x:0 0 0 2024-07-03T15:44:19.490 INFO:tasks.workunit.client.0.smithi028.stdout:0/890: dwrite d2/f49 [0,4194304] 0 2024-07-03T15:44:19.492 INFO:tasks.workunit.client.0.smithi028.stdout:0/891: mknod d2/d3/d4d/ddc/c10b 0 2024-07-03T15:44:19.494 INFO:tasks.workunit.client.0.smithi028.stdout:2/673: mkdir da/d1e/dec 0 2024-07-03T15:44:19.495 INFO:tasks.workunit.client.0.smithi028.stdout:2/674: fdatasync da/d42/f46 0 2024-07-03T15:44:19.497 INFO:tasks.workunit.client.0.smithi028.stdout:3/859: dwrite d18/d1a/d1e/d58/d9f/fa5 [0,4194304] 0 2024-07-03T15:44:19.497 INFO:tasks.workunit.client.0.smithi028.stdout:2/675: creat da/fed x:0 0 0 2024-07-03T15:44:19.501 INFO:tasks.workunit.client.0.smithi028.stdout:0/892: dread d2/d3/d4d/d56/d78/da8/dd0/feb [0,4194304] 0 2024-07-03T15:44:19.501 INFO:tasks.workunit.client.0.smithi028.stdout:0/893: stat d2/d3/dc4/d58/d82/fb6 0 2024-07-03T15:44:19.502 INFO:tasks.workunit.client.0.smithi028.stdout:0/894: truncate d2/d3/d4d/d56/d78/da8/ff1 625957 0 2024-07-03T15:44:19.502 INFO:tasks.workunit.client.0.smithi028.stdout:0/895: creat d2/d3/dc4/d58/d82/d9b/f10c x:0 0 0 2024-07-03T15:44:19.502 INFO:tasks.workunit.client.0.smithi028.stdout:0/896: fsync d2/f15 0 2024-07-03T15:44:19.504 INFO:tasks.workunit.client.0.smithi028.stdout:3/860: symlink d18/d1a/d1e/d58/d9f/l115 0 2024-07-03T15:44:19.508 INFO:tasks.workunit.client.0.smithi028.stdout:3/861: symlink d18/d43/l116 0 2024-07-03T15:44:19.511 INFO:tasks.workunit.client.0.smithi028.stdout:3/862: creat d18/d3a/dc1/dd0/f117 x:0 0 0 2024-07-03T15:44:19.517 INFO:tasks.workunit.client.0.smithi028.stdout:0/897: dwrite d2/d3/dc4/d36/d45/da3/fab [0,4194304] 0 2024-07-03T15:44:19.517 INFO:tasks.workunit.client.0.smithi028.stdout:0/898: chown d2/d3/d4d/d56/fb8 4 1 2024-07-03T15:44:19.518 INFO:tasks.workunit.client.0.smithi028.stdout:2/676: dwrite da/d1e/dd0/feb [0,4194304] 0 2024-07-03T15:44:19.521 INFO:tasks.workunit.client.0.smithi028.stdout:2/677: dread da/d42/d47/fde [0,4194304] 0 2024-07-03T15:44:19.521 INFO:tasks.workunit.client.0.smithi028.stdout:2/678: getdents da/d1e/dec 0 2024-07-03T15:44:19.522 INFO:tasks.workunit.client.0.smithi028.stdout:0/899: rename d2/d1c/d6c/dc5/fe3 to d2/d3/d4d/d56/d78/da8/dd7/f10d 0 2024-07-03T15:44:19.525 INFO:tasks.workunit.client.0.smithi028.stdout:3/863: dwrite d18/d6f/f9e [0,4194304] 0 2024-07-03T15:44:19.529 INFO:tasks.workunit.client.0.smithi028.stdout:2/679: rename da/d1e/d68/fe3 to da/d74/fee 0 2024-07-03T15:44:19.529 INFO:tasks.workunit.client.0.smithi028.stdout:2/680: creat da/d42/fef x:0 0 0 2024-07-03T15:44:19.534 INFO:tasks.workunit.client.0.smithi028.stdout:0/900: creat d2/d3/d4d/d56/dcd/f10e x:0 0 0 2024-07-03T15:44:19.536 INFO:tasks.workunit.client.0.smithi028.stdout:3/864: rename d18/d1a/d64/d10e to d18/d6f/d78/d87/d118 0 2024-07-03T15:44:19.536 INFO:tasks.workunit.client.0.smithi028.stdout:3/865: truncate d18/f42 4929913 0 2024-07-03T15:44:19.541 INFO:tasks.workunit.client.0.smithi028.stdout:2/681: mkdir da/d24/d2c/dc6/df0 0 2024-07-03T15:44:19.544 INFO:tasks.workunit.client.0.smithi028.stdout:3/866: mknod d18/d3a/dc1/c119 0 2024-07-03T15:44:19.553 INFO:tasks.workunit.client.0.smithi028.stdout:0/901: dwrite d2/d1c/d6c/f102 [4194304,4194304] 0 2024-07-03T15:44:19.573 INFO:tasks.workunit.client.0.smithi028.stdout:3/867: dwrite d18/d1a/d1e/d58/f96 [0,4194304] 0 2024-07-03T15:44:19.575 INFO:tasks.workunit.client.0.smithi028.stdout:3/868: unlink d18/d6f/d78/d87/d9c/fd4 0 2024-07-03T15:44:19.575 INFO:tasks.workunit.client.0.smithi028.stdout:2/682: dwrite da/d1e/fc9 [0,4194304] 0 2024-07-03T15:44:19.578 INFO:tasks.workunit.client.0.smithi028.stdout:0/902: dwrite d2/d1c/f41 [0,4194304] 0 2024-07-03T15:44:19.578 INFO:tasks.workunit.client.0.smithi028.stdout:0/903: dread - d2/d3/d4d/d56/d78/fa1 zero size 2024-07-03T15:44:19.583 INFO:tasks.workunit.client.0.smithi028.stdout:2/683: symlink da/d23/lf1 0 2024-07-03T15:44:19.583 INFO:tasks.workunit.client.0.smithi028.stdout:4/944: sync 2024-07-03T15:44:19.583 INFO:tasks.workunit.client.0.smithi028.stdout:8/951: sync 2024-07-03T15:44:19.584 INFO:tasks.workunit.client.0.smithi028.stdout:2/684: dread da/d74/fc8 [0,4194304] 0 2024-07-03T15:44:19.584 INFO:tasks.workunit.client.0.smithi028.stdout:2/685: truncate da/d1e/d68/fb3 476044 0 2024-07-03T15:44:19.586 INFO:tasks.workunit.client.0.smithi028.stdout:4/945: creat d2/d3e/d9f/d74/f13f x:0 0 0 2024-07-03T15:44:19.587 INFO:tasks.workunit.client.0.smithi028.stdout:8/952: mknod d0/d41/d50/d3/dc/d3f/d8e/d81/d9e/d119/c13a 0 2024-07-03T15:44:19.589 INFO:tasks.workunit.client.0.smithi028.stdout:2/686: unlink da/d42/c5e 0 2024-07-03T15:44:19.589 INFO:tasks.workunit.client.0.smithi028.stdout:2/687: truncate da/d42/d47/d78/fcd 699587 0 2024-07-03T15:44:19.590 INFO:tasks.workunit.client.0.smithi028.stdout:8/953: getdents d0 0 2024-07-03T15:44:19.590 INFO:tasks.workunit.client.0.smithi028.stdout:8/954: chown d0/d41/d50/d3/d35/d68/d22/d40/f117 3665 1 2024-07-03T15:44:19.594 INFO:tasks.workunit.client.0.smithi028.stdout:2/688: getdents da/d42/d47 0 2024-07-03T15:44:19.594 INFO:tasks.workunit.client.0.smithi028.stdout:2/689: link da/d23/le4 da/d24/d2c/dc6/df0/lf2 0 2024-07-03T15:44:19.595 INFO:tasks.workunit.client.0.smithi028.stdout:2/690: mknod da/d23/d6f/d84/db8/cf3 0 2024-07-03T15:44:19.598 INFO:tasks.workunit.client.0.smithi028.stdout:2/691: creat da/d1e/d56/dc0/ff4 x:0 0 0 2024-07-03T15:44:19.599 INFO:tasks.workunit.client.0.smithi028.stdout:8/955: dread d0/d41/d50/d3/d35/d68/d22/d40/d62/f65 [0,4194304] 0 2024-07-03T15:44:19.600 INFO:tasks.workunit.client.0.smithi028.stdout:2/692: unlink da/d24/d2c/dba/fd6 0 2024-07-03T15:44:19.600 INFO:tasks.workunit.client.0.smithi028.stdout:4/946: write d2/d12/d42/d63/d6d/fd2 [486770,4708] 0 2024-07-03T15:44:19.601 INFO:tasks.workunit.client.0.smithi028.stdout:8/956: getdents d0/d41/d50/d3/dc/d3f/d8e 0 2024-07-03T15:44:19.603 INFO:tasks.workunit.client.0.smithi028.stdout:8/957: mknod d0/d41/d50/d3/d35/d68/d22/d40/dd2/c13b 0 2024-07-03T15:44:19.606 INFO:tasks.workunit.client.0.smithi028.stdout:3/869: dwrite d18/d1a/d5e/d66/f95 [0,4194304] 0 2024-07-03T15:44:19.606 INFO:tasks.workunit.client.0.smithi028.stdout:3/870: stat d18/d3a/l63 0 2024-07-03T15:44:19.606 INFO:tasks.workunit.client.0.smithi028.stdout:0/904: dwrite d2/d1c/f2b [0,4194304] 0 2024-07-03T15:44:19.607 INFO:tasks.workunit.client.0.smithi028.stdout:0/905: readlink d2/d6d/ldd 0 2024-07-03T15:44:19.611 INFO:tasks.workunit.client.0.smithi028.stdout:3/871: symlink d18/d6f/d78/d87/d118/l11a 0 2024-07-03T15:44:19.611 INFO:tasks.workunit.client.0.smithi028.stdout:3/872: truncate d18/d3a/dbb/f111 5378 0 2024-07-03T15:44:19.611 INFO:tasks.workunit.client.0.smithi028.stdout:3/873: write d18/d1a/d5e/f7b [2038969,93946] 0 2024-07-03T15:44:19.613 INFO:tasks.workunit.client.0.smithi028.stdout:8/958: dread d0/d41/d50/d3/dc/d33/d7a/f2f [0,4194304] 0 2024-07-03T15:44:19.613 INFO:tasks.workunit.client.0.smithi028.stdout:8/959: creat d0/d41/d50/d3/d35/d86/f13c x:0 0 0 2024-07-03T15:44:19.613 INFO:tasks.workunit.client.0.smithi028.stdout:8/960: truncate d0/d41/d50/d3/dc/d3f/ff1 872783 0 2024-07-03T15:44:19.613 INFO:tasks.workunit.client.0.smithi028.stdout:0/906: rmdir d2/d3/dc4/d36/d45 39 2024-07-03T15:44:19.614 INFO:tasks.workunit.client.0.smithi028.stdout:2/693: dwrite da/d23/d6f/fae [0,4194304] 0 2024-07-03T15:44:19.620 INFO:tasks.workunit.client.0.smithi028.stdout:0/907: creat d2/d3/d4d/d56/d78/da8/dd7/f10f x:0 0 0 2024-07-03T15:44:19.623 INFO:tasks.workunit.client.0.smithi028.stdout:8/961: write d0/d41/d50/daf/dc8/da6/da7/fda [3666827,100109] 0 2024-07-03T15:44:19.623 INFO:tasks.workunit.client.0.smithi028.stdout:8/962: fdatasync d0/d41/d50/d3/dc/d4d/fc6 0 2024-07-03T15:44:19.624 INFO:tasks.workunit.client.0.smithi028.stdout:3/874: read d18/d3a/ddf/f102 [562596,109416] 0 2024-07-03T15:44:19.633 INFO:tasks.workunit.client.0.smithi028.stdout:0/908: link d2/d3/dc4/d36/d45/f99 d2/d3/d4d/d96/f110 0 2024-07-03T15:44:19.634 INFO:tasks.workunit.client.0.smithi028.stdout:8/963: symlink d0/d41/d50/d3/d35/d68/l13d 0 2024-07-03T15:44:19.635 INFO:tasks.workunit.client.0.smithi028.stdout:2/694: dwrite da/d23/f75 [0,4194304] 0 2024-07-03T15:44:19.637 INFO:tasks.workunit.client.0.smithi028.stdout:3/875: rename d18/d1a/c1c to d18/d1a/d1e/d58/d9f/de2/c11b 0 2024-07-03T15:44:19.643 INFO:tasks.workunit.client.0.smithi028.stdout:0/909: creat d2/d3/d4d/d56/dcd/de4/f111 x:0 0 0 2024-07-03T15:44:19.643 INFO:tasks.workunit.client.0.smithi028.stdout:0/910: write d2/d3/dc4/d36/f88 [1586211,99027] 0 2024-07-03T15:44:19.643 INFO:tasks.workunit.client.0.smithi028.stdout:0/911: creat d2/d3/d4d/d96/dc2/f112 x:0 0 0 2024-07-03T15:44:19.644 INFO:tasks.workunit.client.0.smithi028.stdout:3/876: dread d18/d1a/d1e/d58/d9f/fa5 [0,4194304] 0 2024-07-03T15:44:19.644 INFO:tasks.workunit.client.0.smithi028.stdout:3/877: creat d18/f11c x:0 0 0 2024-07-03T15:44:19.644 INFO:tasks.workunit.client.0.smithi028.stdout:3/878: write d18/d3a/d9d/d8c/ff6 [58384,97744] 0 2024-07-03T15:44:19.645 INFO:tasks.workunit.client.0.smithi028.stdout:8/964: mkdir d0/de0/dcf/d129/d13e 0 2024-07-03T15:44:19.648 INFO:tasks.workunit.client.0.smithi028.stdout:2/695: rmdir da/d42/d47/de8 39 2024-07-03T15:44:19.648 INFO:tasks.workunit.client.0.smithi028.stdout:2/696: chown da/d24/f55 237273 1 2024-07-03T15:44:19.651 INFO:tasks.workunit.client.0.smithi028.stdout:0/912: rename d2/d3/dc4/d36/f61 to d2/d1c/f113 0 2024-07-03T15:44:19.656 INFO:tasks.workunit.client.0.smithi028.stdout:8/965: creat d0/d41/d50/d3/dc/d3f/d8e/d81/dfc/f13f x:0 0 0 2024-07-03T15:44:19.658 INFO:tasks.workunit.client.0.smithi028.stdout:2/697: symlink da/d42/d47/d78/lf5 0 2024-07-03T15:44:19.665 INFO:tasks.workunit.client.0.smithi028.stdout:8/966: dread d0/d41/d50/d3/dc/d3f/d8e/f6d [0,4194304] 0 2024-07-03T15:44:19.667 INFO:tasks.workunit.client.0.smithi028.stdout:3/879: rmdir d18/d1a/d1e/d58 39 2024-07-03T15:44:19.669 INFO:tasks.workunit.client.0.smithi028.stdout:0/913: dwrite d2/d3/dc4/d36/d45/da3/fb9 [0,4194304] 0 2024-07-03T15:44:19.671 INFO:tasks.workunit.client.0.smithi028.stdout:2/698: symlink da/d1e/d68/lf6 0 2024-07-03T15:44:19.672 INFO:tasks.workunit.client.0.smithi028.stdout:2/699: write da/d23/d6f/fae [4661853,87810] 0 2024-07-03T15:44:19.673 INFO:tasks.workunit.client.0.smithi028.stdout:3/880: link d18/d1a/cd9 d18/d3a/d9d/d8c/c11d 0 2024-07-03T15:44:19.679 INFO:tasks.workunit.client.0.smithi028.stdout:3/881: symlink d18/d3a/d9a/l11e 0 2024-07-03T15:44:19.685 INFO:tasks.workunit.client.0.smithi028.stdout:3/882: mkdir d18/d1a/d1e/d10a/d11f 0 2024-07-03T15:44:19.687 INFO:tasks.workunit.client.0.smithi028.stdout:3/883: symlink d18/d1a/d5e/l120 0 2024-07-03T15:44:19.687 INFO:tasks.workunit.client.0.smithi028.stdout:3/884: fdatasync d18/d1a/d1e/f38 0 2024-07-03T15:44:19.689 INFO:tasks.workunit.client.0.smithi028.stdout:3/885: mkdir d18/d1a/d1e/d121 0 2024-07-03T15:44:19.689 INFO:tasks.workunit.client.0.smithi028.stdout:0/914: write d2/d1c/f41 [1486817,71574] 0 2024-07-03T15:44:19.691 INFO:tasks.workunit.client.0.smithi028.stdout:3/886: creat d18/d1a/f122 x:0 0 0 2024-07-03T15:44:19.691 INFO:tasks.workunit.client.0.smithi028.stdout:3/887: stat d18/d6f/d78/d87 0 2024-07-03T15:44:19.691 INFO:tasks.workunit.client.0.smithi028.stdout:3/888: fsync d18/d3a/f76 0 2024-07-03T15:44:19.691 INFO:tasks.workunit.client.0.smithi028.stdout:3/889: creat d18/d6f/f123 x:0 0 0 2024-07-03T15:44:19.691 INFO:tasks.workunit.client.0.smithi028.stdout:3/890: truncate d18/d1a/d5e/d66/fb7 1028112 0 2024-07-03T15:44:19.691 INFO:tasks.workunit.client.0.smithi028.stdout:3/891: write f14 [1177715,60528] 0 2024-07-03T15:44:19.693 INFO:tasks.workunit.client.0.smithi028.stdout:8/967: dwrite d0/d41/d50/d3/dc/d3f/faa [0,4194304] 0 2024-07-03T15:44:19.693 INFO:tasks.workunit.client.0.smithi028.stdout:2/700: dwrite da/d1e/f58 [0,4194304] 0 2024-07-03T15:44:19.693 INFO:tasks.workunit.client.0.smithi028.stdout:2/701: readlink da/d42/d9e/le0 0 2024-07-03T15:44:19.694 INFO:tasks.workunit.client.0.smithi028.stdout:3/892: link d18/d1a/d64/f107 d18/d1a/d1e/d58/d9f/f124 0 2024-07-03T15:44:19.699 INFO:tasks.workunit.client.0.smithi028.stdout:8/968: dread d0/d41/f88 [0,4194304] 0 2024-07-03T15:44:19.705 INFO:tasks.workunit.client.0.smithi028.stdout:0/915: rmdir d2/d3/d4d/d56/d78 39 2024-07-03T15:44:19.706 INFO:tasks.workunit.client.0.smithi028.stdout:0/916: chown d2/d3/d4d/d56/d78/dcf/fdf 38 1 2024-07-03T15:44:19.708 INFO:tasks.workunit.client.0.smithi028.stdout:2/702: mknod da/cf7 0 2024-07-03T15:44:19.709 INFO:tasks.workunit.client.0.smithi028.stdout:3/893: mkdir d18/d1a/d1e/d10a/d11f/d125 0 2024-07-03T15:44:19.711 INFO:tasks.workunit.client.0.smithi028.stdout:8/969: creat d0/de0/dcf/f140 x:0 0 0 2024-07-03T15:44:19.711 INFO:tasks.workunit.client.0.smithi028.stdout:8/970: creat d0/d41/d50/d3/dc/d3f/d8e/d5d/f141 x:0 0 0 2024-07-03T15:44:19.711 INFO:tasks.workunit.client.0.smithi028.stdout:0/917: dread d2/d3/d4d/f57 [0,4194304] 0 2024-07-03T15:44:19.712 INFO:tasks.workunit.client.0.smithi028.stdout:2/703: creat da/d24/d2c/ff8 x:0 0 0 2024-07-03T15:44:19.713 INFO:tasks.workunit.client.0.smithi028.stdout:3/894: rename d18/d1a/d5e/d66/da3/fac to d18/d6f/d78/d87/d118/f126 0 2024-07-03T15:44:19.716 INFO:tasks.workunit.client.0.smithi028.stdout:8/971: truncate d0/d41/d50/d3/dc/d3f/d8e/d81/fbd 3925692 0 2024-07-03T15:44:19.716 INFO:tasks.workunit.client.0.smithi028.stdout:8/972: stat d0/d41/d50/d3/dc/d3f/d8e/f42 0 2024-07-03T15:44:19.716 INFO:tasks.workunit.client.0.smithi028.stdout:8/973: dread - d0/d41/d50/d3/d35/d68/d22/d40/db7/f106 zero size 2024-07-03T15:44:19.717 INFO:tasks.workunit.client.0.smithi028.stdout:0/918: mkdir d2/d3/d4d/d56/d78/da8/dd0/df7/d114 0 2024-07-03T15:44:19.717 INFO:tasks.workunit.client.0.smithi028.stdout:0/919: write d2/d3/dc4/d58/d82/d9b/fe0 [423083,21288] 0 2024-07-03T15:44:19.717 INFO:tasks.workunit.client.0.smithi028.stdout:0/920: write d2/d1c/f20 [999356,112137] 0 2024-07-03T15:44:19.718 INFO:tasks.workunit.client.0.smithi028.stdout:2/704: mknod da/d74/cf9 0 2024-07-03T15:44:19.718 INFO:tasks.workunit.client.0.smithi028.stdout:2/705: chown da/d42/d47/d78 8057 1 2024-07-03T15:44:19.719 INFO:tasks.workunit.client.0.smithi028.stdout:3/895: link d18/d6f/l101 d18/d1a/d5e/d66/l127 0 2024-07-03T15:44:19.723 INFO:tasks.workunit.client.0.smithi028.stdout:2/706: unlink da/d42/d47/de8/l44 0 2024-07-03T15:44:19.724 INFO:tasks.workunit.client.0.smithi028.stdout:2/707: chown da/d42/d47/d78/f9d 333758 1 2024-07-03T15:44:19.725 INFO:tasks.workunit.client.0.smithi028.stdout:8/974: getdents d0/d41/d50/d3/d35/d68/d22/d40/dd2 0 2024-07-03T15:44:19.725 INFO:tasks.workunit.client.0.smithi028.stdout:8/975: stat d0/d41/d50/d3/dc/d33/d7a/f2f 0 2024-07-03T15:44:19.725 INFO:tasks.workunit.client.0.smithi028.stdout:8/976: readlink d0/d41/d50/d3/dc/d33/d7a/lc7 0 2024-07-03T15:44:19.725 INFO:tasks.workunit.client.0.smithi028.stdout:0/921: write d2/d3/dc4/d36/d45/da3/fab [1166034,127716] 0 2024-07-03T15:44:19.727 INFO:tasks.workunit.client.0.smithi028.stdout:2/708: creat da/d24/d2c/dc6/df0/ffa x:0 0 0 2024-07-03T15:44:19.727 INFO:tasks.workunit.client.0.smithi028.stdout:2/709: stat da/fed 0 2024-07-03T15:44:19.727 INFO:tasks.workunit.client.0.smithi028.stdout:2/710: chown da/d42/d47/d78/c95 23917506 1 2024-07-03T15:44:19.727 INFO:tasks.workunit.client.0.smithi028.stdout:2/711: chown da/d42/d47/de8/l40 6 1 2024-07-03T15:44:19.727 INFO:tasks.workunit.client.0.smithi028.stdout:2/712: fdatasync da/d23/fdd 0 2024-07-03T15:44:19.728 INFO:tasks.workunit.client.0.smithi028.stdout:3/896: dread d18/f1d [0,4194304] 0 2024-07-03T15:44:19.730 INFO:tasks.workunit.client.0.smithi028.stdout:0/922: creat d2/d3/d4d/d56/d78/da8/dd0/df7/d114/f115 x:0 0 0 2024-07-03T15:44:19.732 INFO:tasks.workunit.client.0.smithi028.stdout:2/713: mknod da/d42/d47/d78/cfb 0 2024-07-03T15:44:19.732 INFO:tasks.workunit.client.0.smithi028.stdout:2/714: truncate da/f76 32525 0 2024-07-03T15:44:19.732 INFO:tasks.workunit.client.0.smithi028.stdout:2/715: write da/d24/fbe [209704,77109] 0 2024-07-03T15:44:19.732 INFO:tasks.workunit.client.0.smithi028.stdout:2/716: write da/d1e/d56/f9a [3250861,35129] 0 2024-07-03T15:44:19.736 INFO:tasks.workunit.client.0.smithi028.stdout:0/923: getdents d2/d3 0 2024-07-03T15:44:19.743 INFO:tasks.workunit.client.0.smithi028.stdout:0/924: dread d2/d3/dc4/d36/d45/da3/fab [0,4194304] 0 2024-07-03T15:44:19.749 INFO:tasks.workunit.client.0.smithi028.stdout:8/977: dwrite d0/d41/d50/d3/dc/d3f/d8e/d81/d9e/f11e [0,4194304] 0 2024-07-03T15:44:19.752 INFO:tasks.workunit.client.0.smithi028.stdout:3/897: dwrite d18/d1a/d1e/f3e [4194304,4194304] 0 2024-07-03T15:44:19.752 INFO:tasks.workunit.client.0.smithi028.stdout:3/898: chown d18/f42 6570 1 2024-07-03T15:44:19.752 INFO:tasks.workunit.client.0.smithi028.stdout:3/899: write d18/d3a/d9d/fde [515419,1915] 0 2024-07-03T15:44:19.759 INFO:tasks.workunit.client.0.smithi028.stdout:3/900: truncate d18/d43/ff5 781188 0 2024-07-03T15:44:19.759 INFO:tasks.workunit.client.0.smithi028.stdout:3/901: write d18/d6f/d78/d87/fa4 [820844,74007] 0 2024-07-03T15:44:19.759 INFO:tasks.workunit.client.0.smithi028.stdout:3/902: truncate d18/d1a/d1e/f113 949525 0 2024-07-03T15:44:19.759 INFO:tasks.workunit.client.0.smithi028.stdout:3/903: write d18/d3a/d9a/fa0 [1209848,109950] 0 2024-07-03T15:44:19.759 INFO:tasks.workunit.client.0.smithi028.stdout:3/904: read - d18/d6f/f123 zero size 2024-07-03T15:44:19.759 INFO:tasks.workunit.client.0.smithi028.stdout:3/905: chown d18/d6f/f7d 4719702 1 2024-07-03T15:44:19.765 INFO:tasks.workunit.client.0.smithi028.stdout:0/925: dwrite d2/d3/d4d/d56/d78/da8/dd0/df7/d114/f115 [0,4194304] 0 2024-07-03T15:44:19.766 INFO:tasks.workunit.client.0.smithi028.stdout:8/978: dwrite d0/d41/d50/d3/d35/f11a [0,4194304] 0 2024-07-03T15:44:19.768 INFO:tasks.workunit.client.0.smithi028.stdout:0/926: mknod d2/d3/c116 0 2024-07-03T15:44:19.772 INFO:tasks.workunit.client.0.smithi028.stdout:0/927: unlink d2/c5 0 2024-07-03T15:44:19.774 INFO:tasks.workunit.client.0.smithi028.stdout:8/979: link d0/d41/d50/d3/d71/c128 d0/d41/d50/d3/c142 0 2024-07-03T15:44:19.785 INFO:tasks.workunit.client.0.smithi028.stdout:5/982: sync 2024-07-03T15:44:19.786 INFO:tasks.workunit.client.0.smithi028.stdout:5/983: truncate d0/dd/d1e/d8e/de4/f10a 5003729 0 2024-07-03T15:44:19.786 INFO:tasks.workunit.client.0.smithi028.stdout:5/984: write d0/dd/d13/d57/d30/fc5 [1226045,120371] 0 2024-07-03T15:44:19.786 INFO:tasks.workunit.client.0.smithi028.stdout:5/985: chown d0/dd/d13/d29/f35 52061 1 2024-07-03T15:44:19.788 INFO:tasks.workunit.client.0.smithi028.stdout:3/906: truncate d18/d43/ff5 1454139 0 2024-07-03T15:44:19.794 INFO:tasks.workunit.client.0.smithi028.stdout:0/928: dwrite d2/d1c/f113 [0,4194304] 0 2024-07-03T15:44:19.794 INFO:tasks.workunit.client.0.smithi028.stdout:0/929: write d2/d3/dc4/d36/f88 [2519957,107433] 0 2024-07-03T15:44:19.795 INFO:tasks.workunit.client.0.smithi028.stdout:0/930: stat d2/d1c/f2d 0 2024-07-03T15:44:19.795 INFO:tasks.workunit.client.0.smithi028.stdout:0/931: chown d2/d3/d4d/d96/dc2/fea 7 1 2024-07-03T15:44:19.798 INFO:tasks.workunit.client.0.smithi028.stdout:8/980: dwrite d0/d41/d50/d3/d35/d68/d22/d40/db7/f106 [0,4194304] 0 2024-07-03T15:44:19.802 INFO:tasks.workunit.client.0.smithi028.stdout:8/981: mknod d0/d41/d50/d3/d35/d68/d22/d40/d4a/de6/c143 0 2024-07-03T15:44:19.803 INFO:tasks.workunit.client.0.smithi028.stdout:0/932: unlink d2/d3/dc4/d58/c8a 0 2024-07-03T15:44:19.804 INFO:tasks.workunit.client.0.smithi028.stdout:0/933: write d2/d3/dc4/fc1 [442439,67003] 0 2024-07-03T15:44:19.805 INFO:tasks.workunit.client.0.smithi028.stdout:8/982: creat d0/d41/d50/d3/dc/d3f/f144 x:0 0 0 2024-07-03T15:44:19.807 INFO:tasks.workunit.client.0.smithi028.stdout:5/986: dwrite d0/dd/d13/d57/d77/d11f/f13b [0,4194304] 0 2024-07-03T15:44:19.807 INFO:tasks.workunit.client.0.smithi028.stdout:5/987: creat d0/dd/d14/dc0/d124/d106/f14c x:0 0 0 2024-07-03T15:44:19.807 INFO:tasks.workunit.client.0.smithi028.stdout:3/907: dwrite d18/d1a/d1e/d58/d9f/fa5 [0,4194304] 0 2024-07-03T15:44:19.808 INFO:tasks.workunit.client.0.smithi028.stdout:3/908: creat d18/d6f/d78/f128 x:0 0 0 2024-07-03T15:44:19.808 INFO:tasks.workunit.client.0.smithi028.stdout:3/909: dread - d18/d6f/ff2 zero size 2024-07-03T15:44:19.815 INFO:tasks.workunit.client.0.smithi028.stdout:5/988: creat d0/dd/d13/d29/d110/f14d x:0 0 0 2024-07-03T15:44:19.818 INFO:tasks.workunit.client.0.smithi028.stdout:0/934: rename d2/d1c/d6c/da9 to d2/d3/dc4/d58/d117 0 2024-07-03T15:44:19.818 INFO:tasks.workunit.client.0.smithi028.stdout:0/935: write d2/f15 [149453,72961] 0 2024-07-03T15:44:19.822 INFO:tasks.workunit.client.0.smithi028.stdout:0/936: link d2/d1c/f41 d2/d3/d4d/d56/d78/da8/dd0/df7/d114/f118 0 2024-07-03T15:44:19.822 INFO:tasks.workunit.client.0.smithi028.stdout:0/937: stat d2/d3/fac 0 2024-07-03T15:44:19.822 INFO:tasks.workunit.client.0.smithi028.stdout:0/938: fdatasync d2/d3/dc4/d58/d6e/ff6 0 2024-07-03T15:44:19.822 INFO:tasks.workunit.client.0.smithi028.stdout:3/910: dread d18/d1a/d64/f8f [0,4194304] 0 2024-07-03T15:44:19.822 INFO:tasks.workunit.client.0.smithi028.stdout:3/911: fdatasync d18/d1a/d1e/d58/d9f/da9/fd5 0 2024-07-03T15:44:19.822 INFO:tasks.workunit.client.0.smithi028.stdout:8/983: dwrite d0/d41/d50/d3/d35/d68/d22/d40/f117 [0,4194304] 0 2024-07-03T15:44:19.822 INFO:tasks.workunit.client.0.smithi028.stdout:3/912: chown d18/d1a/d1e/d73/c98 759356103 1 2024-07-03T15:44:19.823 INFO:tasks.workunit.client.0.smithi028.stdout:5/989: rmdir d0/dd/d1e/d3b/d3f/d71/df4 39 2024-07-03T15:44:19.837 INFO:tasks.workunit.client.0.smithi028.stdout:0/939: rmdir d2/d3/d4d/d56/dcd/de4 39 2024-07-03T15:44:19.839 INFO:tasks.workunit.client.0.smithi028.stdout:0/940: creat d2/d3/dc4/d58/d117/f119 x:0 0 0 2024-07-03T15:44:19.842 INFO:tasks.workunit.client.0.smithi028.stdout:8/984: dwrite d0/d41/d50/daf/dc8/da6/f9c [0,4194304] 0 2024-07-03T15:44:19.842 INFO:tasks.workunit.client.0.smithi028.stdout:8/985: write d0/d41/d50/d3/dc/d3f/d8e/d5d/f133 [165380,129344] 0 2024-07-03T15:44:19.842 INFO:tasks.workunit.client.0.smithi028.stdout:8/986: write d0/d41/d50/d3/d35/d86/f136 [424276,94487] 0 2024-07-03T15:44:19.843 INFO:tasks.workunit.client.0.smithi028.stdout:8/987: fdatasync d0/d41/d50/d3/d35/d68/d22/f115 0 2024-07-03T15:44:19.843 INFO:tasks.workunit.client.0.smithi028.stdout:8/988: chown d0/d6c 144 1 2024-07-03T15:44:19.844 INFO:tasks.workunit.client.0.smithi028.stdout:8/989: mknod d0/d41/d50/d3/d35/d68/d22/d40/dd2/c145 0 2024-07-03T15:44:19.845 INFO:tasks.workunit.client.0.smithi028.stdout:3/913: dwrite d18/d3a/d9a/fa0 [0,4194304] 0 2024-07-03T15:44:19.845 INFO:tasks.workunit.client.0.smithi028.stdout:3/914: fdatasync d18/d3a/d9a/fee 0 2024-07-03T15:44:19.845 INFO:tasks.workunit.client.0.smithi028.stdout:3/915: write d18/d1a/d1e/d58/df3/ffc [621851,88837] 0 2024-07-03T15:44:19.846 INFO:tasks.workunit.client.0.smithi028.stdout:5/990: dwrite d0/dd/df/da7/f146 [0,4194304] 0 2024-07-03T15:44:19.849 INFO:tasks.workunit.client.0.smithi028.stdout:3/916: rename d18/d6f/d78/d87/d118 to d18/d6f/d78/d129 0 2024-07-03T15:44:19.849 INFO:tasks.workunit.client.0.smithi028.stdout:3/917: readlink d18/d1a/d1e/d58/d9f/la6 0 2024-07-03T15:44:19.852 INFO:tasks.workunit.client.0.smithi028.stdout:3/918: mknod d18/d1a/d1e/d73/c12a 0 2024-07-03T15:44:19.852 INFO:tasks.workunit.client.0.smithi028.stdout:3/919: read - d18/d1a/d5e/d66/fd8 zero size 2024-07-03T15:44:19.853 INFO:tasks.workunit.client.0.smithi028.stdout:0/941: dwrite d2/d3/d4d/d56/d78/da8/f108 [0,4194304] 0 2024-07-03T15:44:19.853 INFO:tasks.workunit.client.0.smithi028.stdout:5/991: rmdir d0/dd/d13/d57/da2/de9 39 2024-07-03T15:44:19.858 INFO:tasks.workunit.client.0.smithi028.stdout:8/990: read d0/d41/d50/d3/d35/d68/d22/d40/f60 [3803589,79318] 0 2024-07-03T15:44:19.862 INFO:tasks.workunit.client.0.smithi028.stdout:9/986: sync 2024-07-03T15:44:19.862 INFO:tasks.workunit.client.0.smithi028.stdout:4/947: sync 2024-07-03T15:44:19.863 INFO:tasks.workunit.client.0.smithi028.stdout:7/733: sync 2024-07-03T15:44:19.863 INFO:tasks.workunit.client.0.smithi028.stdout:4/948: stat d2/d5a/d45/d84/df4/d92/f136 0 2024-07-03T15:44:19.863 INFO:tasks.workunit.client.0.smithi028.stdout:9/987: fsync d1/d10/d28/d45/d61/d8c/d77/f91 0 2024-07-03T15:44:19.863 INFO:tasks.workunit.client.0.smithi028.stdout:4/949: chown d2/d12/d42/d63/d6d/dc8/c108 52088 1 2024-07-03T15:44:19.863 INFO:tasks.workunit.client.0.smithi028.stdout:9/988: chown d1/d10/d28/l54 432697728 1 2024-07-03T15:44:19.863 INFO:tasks.workunit.client.0.smithi028.stdout:7/734: write d9/d1c/d45/db5/f81 [1844886,33366] 0 2024-07-03T15:44:19.863 INFO:tasks.workunit.client.0.smithi028.stdout:9/989: write d1/d10/d28/fd0 [1187569,107844] 0 2024-07-03T15:44:19.867 INFO:tasks.workunit.client.0.smithi028.stdout:3/920: dwrite d18/d3a/d9d/d8c/db4/fe7 [0,4194304] 0 2024-07-03T15:44:19.870 INFO:tasks.workunit.client.0.smithi028.stdout:0/942: rename d2/d1c/f20 to d2/d3/dc4/d36/f11a 0 2024-07-03T15:44:19.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:19 smithi028 ceph-mon[29148]: pgmap v11: 65 pgs: 65 active+clean; 2.9 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 134 MiB/s rd, 171 MiB/s wr, 322 op/s 2024-07-03T15:44:19.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:19 smithi028 ceph-mon[29148]: from='client.14666 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:44:19.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:19 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/3192437799' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-03T15:44:19.879 INFO:tasks.workunit.client.0.smithi028.stdout:2/717: sync 2024-07-03T15:44:19.879 INFO:tasks.workunit.client.0.smithi028.stdout:8/991: mkdir d0/d41/d50/d3/dc/d3f/d8e/d81/dfc/d146 0 2024-07-03T15:44:19.879 INFO:tasks.workunit.client.0.smithi028.stdout:8/992: truncate d0/d41/d50/d3/d35/d86/f136 1541231 0 2024-07-03T15:44:19.880 INFO:tasks.workunit.client.0.smithi028.stdout:7/735: dwrite d9/f3a [0,4194304] 0 2024-07-03T15:44:19.881 INFO:tasks.workunit.client.0.smithi028.stdout:4/950: unlink d2/d3e/f3f 0 2024-07-03T15:44:19.889 INFO:tasks.workunit.client.0.smithi028.stdout:3/921: dwrite d18/d6f/f9e [0,4194304] 0 2024-07-03T15:44:19.889 INFO:tasks.workunit.client.0.smithi028.stdout:9/990: symlink d1/d10/d28/d45/d56/dcd/dd5/l158 0 2024-07-03T15:44:19.892 INFO:tasks.workunit.client.0.smithi028.stdout:0/943: mknod d2/d3/d4d/d56/d78/dcf/c11b 0 2024-07-03T15:44:19.893 INFO:tasks.workunit.client.0.smithi028.stdout:5/992: truncate d0/dd/d13/d57/d26/ddf/ff1 4071301 0 2024-07-03T15:44:19.893 INFO:tasks.workunit.client.0.smithi028.stdout:5/993: write d0/f2c [4611166,91662] 0 2024-07-03T15:44:19.894 INFO:tasks.workunit.client.0.smithi028.stdout:8/993: dread d0/d41/d50/d3/d35/d68/d22/fd5 [0,4194304] 0 2024-07-03T15:44:19.896 INFO:tasks.workunit.client.0.smithi028.stdout:7/736: dwrite d9/d1c/d45/f62 [0,4194304] 0 2024-07-03T15:44:19.899 INFO:tasks.workunit.client.0.smithi028.stdout:2/718: symlink da/d42/d47/de8/lfc 0 2024-07-03T15:44:19.904 INFO:tasks.workunit.client.0.smithi028.stdout:8/994: dread d0/d41/d50/d3/dc/d3f/d8e/d81/fc3 [0,4194304] 0 2024-07-03T15:44:19.904 INFO:tasks.workunit.client.0.smithi028.stdout:8/995: write d0/d6c/da9/fd7 [9810,14423] 0 2024-07-03T15:44:19.905 INFO:tasks.workunit.client.0.smithi028.stdout:8/996: write d0/de0/dcf/dc0/ffd [490595,64714] 0 2024-07-03T15:44:19.907 INFO:tasks.workunit.client.0.smithi028.stdout:4/951: unlink d2/d12/d5b/d78/de4/l46 0 2024-07-03T15:44:19.907 INFO:tasks.workunit.client.0.smithi028.stdout:4/952: fdatasync d2/d12/da6/fcc 0 2024-07-03T15:44:19.907 INFO:tasks.workunit.client.0.smithi028.stdout:4/953: fdatasync d2/d5a/d45/d84/f8f 0 2024-07-03T15:44:19.908 INFO:tasks.workunit.client.0.smithi028.stdout:5/994: dwrite d0/dd/d13/ffd [0,4194304] 0 2024-07-03T15:44:19.911 INFO:tasks.workunit.client.0.smithi028.stdout:3/922: rmdir d18/d6f/d78/d87/dca 39 2024-07-03T15:44:19.913 INFO:tasks.workunit.client.0.smithi028.stdout:9/991: creat d1/d10/d28/d45/d61/d8c/d77/db7/dab/f159 x:0 0 0 2024-07-03T15:44:19.913 INFO:tasks.workunit.client.0.smithi028.stdout:9/992: fdatasync d1/d10/d28/d45/d56/d11a/f125 0 2024-07-03T15:44:19.917 INFO:tasks.workunit.client.0.smithi028.stdout:0/944: rename d2/f3b to d2/d1c/d6c/f11c 0 2024-07-03T15:44:19.920 INFO:tasks.workunit.client.0.smithi028.stdout:2/719: mknod da/d23/d6f/d84/db8/cfd 0 2024-07-03T15:44:19.922 INFO:tasks.workunit.client.0.smithi028.stdout:4/954: dwrite d2/f1b [4194304,4194304] 0 2024-07-03T15:44:19.923 INFO:tasks.workunit.client.0.smithi028.stdout:8/997: mknod d0/d41/d50/d3/dc/d3f/d8e/d81/dfc/d146/c147 0 2024-07-03T15:44:19.926 INFO:tasks.workunit.client.0.smithi028.stdout:5/995: creat d0/dd/d14/dc0/d141/f14e x:0 0 0 2024-07-03T15:44:19.928 INFO:tasks.workunit.client.0.smithi028.stdout:3/923: truncate d18/d43/ff5 124277 0 2024-07-03T15:44:19.928 INFO:tasks.workunit.client.0.smithi028.stdout:3/924: chown d18/d43/l116 235 1 2024-07-03T15:44:19.928 INFO:tasks.workunit.client.0.smithi028.stdout:3/925: dread - d18/d3a/dbb/ffa zero size 2024-07-03T15:44:19.928 INFO:tasks.workunit.client.0.smithi028.stdout:3/926: chown d18/d6f/d78 463607683 1 2024-07-03T15:44:19.931 INFO:tasks.workunit.client.0.smithi028.stdout:9/993: link d1/d10/d28/d4e/d5b/l149 d1/d16/d57/d5f/dd8/l15a 0 2024-07-03T15:44:19.936 INFO:tasks.workunit.client.0.smithi028.stdout:4/955: write d2/f1b [2456154,11610] 0 2024-07-03T15:44:19.937 INFO:tasks.workunit.client.0.smithi028.stdout:7/737: rename d9/d1c/d45/db5/f72 to d9/d1c/d45/db5/d92/d99/daf/dc4/fdf 0 2024-07-03T15:44:19.938 INFO:tasks.workunit.client.0.smithi028.stdout:2/720: mknod da/d23/d6f/d84/db8/cfe 0 2024-07-03T15:44:19.941 INFO:tasks.workunit.client.0.smithi028.stdout:5/996: truncate d0/dd/d13/d57/f1f 693952 0 2024-07-03T15:44:19.943 INFO:tasks.workunit.client.0.smithi028.stdout:0/945: dwrite d2/d3/d4d/d56/d78/dcf/fdf [0,4194304] 0 2024-07-03T15:44:19.943 INFO:tasks.workunit.client.0.smithi028.stdout:3/927: creat d18/d6f/d78/d105/f12b x:0 0 0 2024-07-03T15:44:19.944 INFO:tasks.workunit.client.0.smithi028.stdout:4/956: read d2/d12/d5b/d78/de4/fd3 [423104,20860] 0 2024-07-03T15:44:19.948 INFO:tasks.workunit.client.0.smithi028.stdout:7/738: mkdir d9/d1c/d45/d71/d78/de0 0 2024-07-03T15:44:19.950 INFO:tasks.workunit.client.0.smithi028.stdout:2/721: rename da/d1e/cbf to da/cff 0 2024-07-03T15:44:19.950 INFO:tasks.workunit.client.0.smithi028.stdout:2/722: fdatasync da/d42/f4d 0 2024-07-03T15:44:19.952 INFO:tasks.workunit.client.0.smithi028.stdout:4/957: dread d2/d12/f28 [0,4194304] 0 2024-07-03T15:44:19.952 INFO:tasks.workunit.client.0.smithi028.stdout:4/958: dread - d2/d3e/d9f/f7b zero size 2024-07-03T15:44:19.955 INFO:tasks.workunit.client.0.smithi028.stdout:5/997: truncate d0/dd/d13/d47/d50/f118 307978 0 2024-07-03T15:44:19.955 INFO:tasks.workunit.client.0.smithi028.stdout:5/998: fdatasync d0/dd/d13/d57/d26/f2e 0 2024-07-03T15:44:19.955 INFO:tasks.workunit.client.0.smithi028.stdout:5/999: write d0/dd/d13/d57/d30/fc5 [1225566,68546] 0 2024-07-03T15:44:19.956 INFO:tasks.workunit.client.0.smithi028.stdout:9/994: dwrite d1/d16/d57/d5f/d64/dc3/f152 [0,4194304] 0 2024-07-03T15:44:19.956 INFO:tasks.workunit.client.0.smithi028.stdout:8/998: dwrite d0/f2a [4194304,4194304] 0 2024-07-03T15:44:19.956 INFO:tasks.workunit.client.0.smithi028.stdout:8/999: readlink d0/d41/d50/d3/d35/le2 0 2024-07-03T15:44:19.956 INFO:tasks.workunit.client.0.smithi028.stdout:3/928: dread d18/d1a/d1e/f41 [0,4194304] 0 2024-07-03T15:44:19.957 INFO:tasks.workunit.client.0.smithi028.stdout:0/946: mkdir d2/d3/d4d/d56/d78/dae/d11d 0 2024-07-03T15:44:19.957 INFO:tasks.workunit.client.0.smithi028.stdout:0/947: creat d2/d3/dc4/f11e x:0 0 0 2024-07-03T15:44:19.961 INFO:tasks.workunit.client.0.smithi028.stdout:4/959: dread d2/d12/d2d/d58/f5e [0,4194304] 0 2024-07-03T15:44:19.964 INFO:tasks.workunit.client.0.smithi028.stdout:9/995: getdents d1/d16/d57/d5f/d64/dc3 0 2024-07-03T15:44:19.964 INFO:tasks.workunit.client.0.smithi028.stdout:9/996: creat d1/d10/d28/d4e/f15b x:0 0 0 2024-07-03T15:44:19.966 INFO:tasks.workunit.client.0.smithi028.stdout:4/960: write d2/d12/d2d/d58/f5e [1493950,93868] 0 2024-07-03T15:44:19.967 INFO:tasks.workunit.client.0.smithi028.stdout:0/948: dread d2/d3/dc4/d58/d117/fbf [0,4194304] 0 2024-07-03T15:44:19.972 INFO:tasks.workunit.client.0.smithi028.stdout:9/997: mknod d1/d8e/c15c 0 2024-07-03T15:44:19.974 INFO:tasks.workunit.client.0.smithi028.stdout:7/739: rmdir d9/d1c/d45/d71/d78 39 2024-07-03T15:44:19.976 INFO:tasks.workunit.client.0.smithi028.stdout:4/961: rename d2/d12/d2d/db9/dda to d2/d3e/d140 0 2024-07-03T15:44:19.976 INFO:tasks.workunit.client.0.smithi028.stdout:4/962: stat d2/d12/d2e/fb1 0 2024-07-03T15:44:19.978 INFO:tasks.workunit.client.0.smithi028.stdout:0/949: unlink d2/d3/d4d/d56/d78/da8/lbb 0 2024-07-03T15:44:19.979 INFO:tasks.workunit.client.0.smithi028.stdout:2/723: dwrite da/d42/f65 [0,4194304] 0 2024-07-03T15:44:19.980 INFO:tasks.workunit.client.0.smithi028.stdout:3/929: dwrite d18/d3a/dc1/dd0/f117 [0,4194304] 0 2024-07-03T15:44:19.980 INFO:tasks.workunit.client.0.smithi028.stdout:9/998: dread d1/d10/d28/d45/d56/dcd/f122 [0,4194304] 0 2024-07-03T15:44:19.980 INFO:tasks.workunit.client.0.smithi028.stdout:9/999: write d1/d10/d28/d4e/d5b/df0/d146/f90 [6344515,98487] 0 2024-07-03T15:44:19.981 INFO:tasks.workunit.client.0.smithi028.stdout:4/963: mknod d2/d3e/d9f/d74/c141 0 2024-07-03T15:44:19.982 INFO:tasks.workunit.client.0.smithi028.stdout:4/964: chown d2/d5a/d45/d84/df4/d107 0 1 2024-07-03T15:44:19.982 INFO:tasks.workunit.client.0.smithi028.stdout:4/965: chown d2/d12/la1 125 1 2024-07-03T15:44:19.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:19 smithi148 ceph-mon[36056]: pgmap v11: 65 pgs: 65 active+clean; 2.9 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 134 MiB/s rd, 171 MiB/s wr, 322 op/s 2024-07-03T15:44:19.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:19 smithi148 ceph-mon[36056]: from='client.14666 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:44:19.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:19 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/3192437799' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-03T15:44:19.982 INFO:tasks.workunit.client.0.smithi028.stdout:0/950: mkdir d2/d3/d4d/d56/d78/dae/d11d/d11f 0 2024-07-03T15:44:19.986 INFO:tasks.workunit.client.0.smithi028.stdout:2/724: creat da/d1e/f100 x:0 0 0 2024-07-03T15:44:19.990 INFO:tasks.workunit.client.0.smithi028.stdout:4/966: write d2/d3e/dae/f11d [2803226,5671] 0 2024-07-03T15:44:19.991 INFO:tasks.workunit.client.0.smithi028.stdout:3/930: creat d18/d1a/d1e/d58/f12c x:0 0 0 2024-07-03T15:44:19.991 INFO:tasks.workunit.client.0.smithi028.stdout:4/967: chown d2/d5a/d45/d84/df4/d9d/fd1 6737 1 2024-07-03T15:44:19.991 INFO:tasks.workunit.client.0.smithi028.stdout:7/740: getdents d9/d1c/d45/d71 0 2024-07-03T15:44:19.994 INFO:tasks.workunit.client.0.smithi028.stdout:2/725: rmdir da/d42/d47 39 2024-07-03T15:44:19.996 INFO:tasks.workunit.client.0.smithi028.stdout:4/968: symlink d2/d12/d42/d63/dfa/l142 0 2024-07-03T15:44:19.998 INFO:tasks.workunit.client.0.smithi028.stdout:2/726: dread da/d24/fbe [0,4194304] 0 2024-07-03T15:44:20.007 INFO:tasks.workunit.client.0.smithi028.stdout:4/969: dread d2/d5a/d45/d84/df4/d9d/fd1 [0,4194304] 0 2024-07-03T15:44:20.008 INFO:tasks.workunit.client.0.smithi028.stdout:2/727: mknod da/d42/d47/da2/c101 0 2024-07-03T15:44:20.009 INFO:tasks.workunit.client.0.smithi028.stdout:2/728: creat da/d1e/d56/dc0/f102 x:0 0 0 2024-07-03T15:44:20.019 INFO:tasks.workunit.client.0.smithi028.stdout:0/951: dwrite d2/d3/d4d/d56/d78/da8/dd0/ff3 [0,4194304] 0 2024-07-03T15:44:20.019 INFO:tasks.workunit.client.0.smithi028.stdout:3/931: dwrite d18/d3a/d9d/d8c/f92 [0,4194304] 0 2024-07-03T15:44:20.020 INFO:tasks.workunit.client.0.smithi028.stdout:4/970: read d2/d12/d2e/f38 [1503794,64664] 0 2024-07-03T15:44:20.020 INFO:tasks.workunit.client.0.smithi028.stdout:0/952: creat d2/d3/d4d/d56/d78/da8/f120 x:0 0 0 2024-07-03T15:44:20.020 INFO:tasks.workunit.client.0.smithi028.stdout:3/932: symlink d18/d6f/d78/d87/dca/l12d 0 2024-07-03T15:44:20.020 INFO:tasks.workunit.client.0.smithi028.stdout:4/971: getdents d2/d12/d42/d63/d97/dbe 0 2024-07-03T15:44:20.020 INFO:tasks.workunit.client.0.smithi028.stdout:3/933: truncate d18/f55 18187 0 2024-07-03T15:44:20.024 INFO:tasks.workunit.client.0.smithi028.stdout:3/934: mkdir d18/d6f/d78/d129/d12e 0 2024-07-03T15:44:20.025 INFO:tasks.workunit.client.0.smithi028.stdout:3/935: rename d18/d1a/l49 to d18/d6f/d78/l12f 0 2024-07-03T15:44:20.027 INFO:tasks.workunit.client.0.smithi028.stdout:4/972: write d2/d5a/d45/d84/df4/d92/d12e/fd8 [2994219,72030] 0 2024-07-03T15:44:20.027 INFO:tasks.workunit.client.0.smithi028.stdout:4/973: creat d2/d12/d2d/d58/f143 x:0 0 0 2024-07-03T15:44:20.027 INFO:tasks.workunit.client.0.smithi028.stdout:4/974: chown d2/d5a/d45/ce5 7896658 1 2024-07-03T15:44:20.028 INFO:tasks.workunit.client.0.smithi028.stdout:3/936: link d18/d3a/c4a d18/d1a/d1e/d58/d9f/da9/c130 0 2024-07-03T15:44:20.030 INFO:tasks.workunit.client.0.smithi028.stdout:3/937: write d18/d3a/dc1/fd3 [4929,130145] 0 2024-07-03T15:44:20.033 INFO:tasks.workunit.client.0.smithi028.stdout:0/953: write d2/d1c/f2e [3152669,2174] 0 2024-07-03T15:44:20.033 INFO:tasks.workunit.client.0.smithi028.stdout:0/954: truncate d2/d3/d4d/d56/dcd/f10e 260135 0 2024-07-03T15:44:20.033 INFO:tasks.workunit.client.0.smithi028.stdout:0/955: write d2/d1c/f2b [4280185,105590] 0 2024-07-03T15:44:20.036 INFO:tasks.workunit.client.0.smithi028.stdout:0/956: dread d2/d3/f2f [4194304,4194304] 0 2024-07-03T15:44:20.036 INFO:tasks.workunit.client.0.smithi028.stdout:0/957: chown d2/d3/d4d/d56/d78/da8/dd0/feb 0 1 2024-07-03T15:44:20.036 INFO:tasks.workunit.client.0.smithi028.stdout:0/958: chown d2/d1c/f1e 350956662 1 2024-07-03T15:44:20.036 INFO:tasks.workunit.client.0.smithi028.stdout:0/959: chown d2/cfc 3638367 1 2024-07-03T15:44:20.036 INFO:tasks.workunit.client.0.smithi028.stdout:0/960: truncate d2/d1c/f83 1651809 0 2024-07-03T15:44:20.037 INFO:tasks.workunit.client.0.smithi028.stdout:4/975: write d2/d12/da6/fbf [2740807,93450] 0 2024-07-03T15:44:20.037 INFO:tasks.workunit.client.0.smithi028.stdout:4/976: truncate d2/d5a/f36 5096828 0 2024-07-03T15:44:20.038 INFO:tasks.workunit.client.0.smithi028.stdout:0/961: mkdir d2/d3/d121 0 2024-07-03T15:44:20.046 INFO:tasks.workunit.client.0.smithi028.stdout:3/938: dwrite d18/d3a/d9d/fde [0,4194304] 0 2024-07-03T15:44:20.046 INFO:tasks.workunit.client.0.smithi028.stdout:4/977: write d2/d5a/d45/d84/f8f [412413,119178] 0 2024-07-03T15:44:20.049 INFO:tasks.workunit.client.0.smithi028.stdout:3/939: dread d18/d3a/d9d/d8c/faf [0,4194304] 0 2024-07-03T15:44:20.052 INFO:tasks.workunit.client.0.smithi028.stdout:4/978: mknod d2/d12/d42/d63/c144 0 2024-07-03T15:44:20.052 INFO:tasks.workunit.client.0.smithi028.stdout:4/979: read - d2/d3e/d9f/d74/f13f zero size 2024-07-03T15:44:20.052 INFO:tasks.workunit.client.0.smithi028.stdout:0/962: write d2/d3/dc4/d36/d45/fc9 [3133126,54375] 0 2024-07-03T15:44:20.057 INFO:tasks.workunit.client.0.smithi028.stdout:0/963: symlink d2/d3/dc4/d36/l122 0 2024-07-03T15:44:20.057 INFO:tasks.workunit.client.0.smithi028.stdout:0/964: write d2/d3/dc4/d36/f11a [1179007,107159] 0 2024-07-03T15:44:20.057 INFO:tasks.workunit.client.0.smithi028.stdout:0/965: fsync d2/d1c/f2b 0 2024-07-03T15:44:20.057 INFO:tasks.workunit.client.0.smithi028.stdout:0/966: chown d2/d3/dc4/d58/d6e/df5 112475347 1 2024-07-03T15:44:20.057 INFO:tasks.workunit.client.0.smithi028.stdout:0/967: dread - d2/d3/d4d/d56/d78/da8/dd7/f10d zero size 2024-07-03T15:44:20.058 INFO:tasks.workunit.client.0.smithi028.stdout:3/940: getdents d18/d1a/d1e/d58 0 2024-07-03T15:44:20.058 INFO:tasks.workunit.client.0.smithi028.stdout:3/941: fdatasync d18/d6f/d78/ff0 0 2024-07-03T15:44:20.058 INFO:tasks.workunit.client.0.smithi028.stdout:3/942: chown d18/d6f/d78/d129 167 1 2024-07-03T15:44:20.059 INFO:tasks.workunit.client.0.smithi028.stdout:3/943: fdatasync d18/d1a/d1e/f57 0 2024-07-03T15:44:20.059 INFO:tasks.workunit.client.0.smithi028.stdout:3/944: creat d18/d1a/d1e/f131 x:0 0 0 2024-07-03T15:44:20.059 INFO:tasks.workunit.client.0.smithi028.stdout:3/945: truncate d18/d1a/d1e/f131 487901 0 2024-07-03T15:44:20.063 INFO:tasks.workunit.client.0.smithi028.stdout:3/946: mkdir d18/d1a/d1e/d58/d9f/da9/d132 0 2024-07-03T15:44:20.063 INFO:tasks.workunit.client.0.smithi028.stdout:3/947: chown d18/d6f/d78/d87/dca/fcc 12 1 2024-07-03T15:44:20.063 INFO:tasks.workunit.client.0.smithi028.stdout:3/948: fsync d18/d1a/d1e/f113 0 2024-07-03T15:44:20.064 INFO:tasks.workunit.client.0.smithi028.stdout:0/968: mknod d2/d3/dc4/d58/c123 0 2024-07-03T15:44:20.065 INFO:tasks.workunit.client.0.smithi028.stdout:3/949: rename l13 to d18/d1a/d109/l133 0 2024-07-03T15:44:20.096 INFO:tasks.workunit.client.0.smithi028.stdout:0/969: dwrite d2/d1c/f2d [0,4194304] 0 2024-07-03T15:44:20.097 INFO:tasks.workunit.client.0.smithi028.stdout:3/950: dwrite d18/d1a/d1e/d58/d9f/fa2 [4194304,4194304] 0 2024-07-03T15:44:20.190 INFO:tasks.workunit.client.0.smithi028.stdout:0/970: creat d2/d3/dc4/d36/f124 x:0 0 0 2024-07-03T15:44:20.196 INFO:tasks.workunit.client.0.smithi028.stdout:3/951: rmdir d18/d3a/d9a 39 2024-07-03T15:44:20.217 INFO:tasks.workunit.client.0.smithi028.stdout:0/971: mkdir d2/d3/d4d/d56/d78/da8/dd0/df7/d125 0 2024-07-03T15:44:20.219 INFO:tasks.workunit.client.0.smithi028.stdout:3/952: rename d18/d1a/d1e/d58/f6c to d18/d3a/dc1/f134 0 2024-07-03T15:44:20.220 INFO:tasks.workunit.client.0.smithi028.stdout:3/953: readlink d18/d1a/l56 0 2024-07-03T15:44:20.220 INFO:tasks.workunit.client.0.smithi028.stdout:3/954: read d18/d1a/d64/ddc/f103 [107497,33606] 0 2024-07-03T15:44:20.220 INFO:tasks.workunit.client.0.smithi028.stdout:3/955: read f6 [1720680,40722] 0 2024-07-03T15:44:20.221 INFO:tasks.workunit.client.0.smithi028.stdout:0/972: mknod d2/d3/d4d/d56/d78/dae/d11d/d11f/c126 0 2024-07-03T15:44:20.223 INFO:tasks.workunit.client.0.smithi028.stdout:3/956: mkdir d18/d1a/d1e/d10a/d11f/d125/d135 0 2024-07-03T15:44:20.224 INFO:tasks.workunit.client.0.smithi028.stdout:0/973: mkdir d2/d3/d121/d127 0 2024-07-03T15:44:20.224 INFO:tasks.workunit.client.0.smithi028.stdout:0/974: chown d2/d3/d4d/d56/d78/da8/dd7 404427954 1 2024-07-03T15:44:20.225 INFO:tasks.workunit.client.0.smithi028.stdout:3/957: rename l16 to d18/d1a/d1e/d58/d9f/l136 0 2024-07-03T15:44:20.226 INFO:tasks.workunit.client.0.smithi028.stdout:0/975: creat d2/d3/d4d/d56/d78/da8/dd0/df7/d125/f128 x:0 0 0 2024-07-03T15:44:20.227 INFO:tasks.workunit.client.0.smithi028.stdout:0/976: stat d2/d3/f25 0 2024-07-03T15:44:20.227 INFO:tasks.workunit.client.0.smithi028.stdout:0/977: truncate d2/d3/d4d/fa2 4923966 0 2024-07-03T15:44:20.228 INFO:tasks.workunit.client.0.smithi028.stdout:0/978: creat d2/d3/dc4/d36/d45/dc8/f129 x:0 0 0 2024-07-03T15:44:20.228 INFO:tasks.workunit.client.0.smithi028.stdout:3/958: dread d18/d43/f4d [0,4194304] 0 2024-07-03T15:44:20.229 INFO:tasks.workunit.client.0.smithi028.stdout:0/979: mkdir d2/d3/d4d/d56/d78/da8/dd0/df7/d114/d12a 0 2024-07-03T15:44:20.230 INFO:tasks.workunit.client.0.smithi028.stdout:3/959: truncate f14 2033448 0 2024-07-03T15:44:20.230 INFO:tasks.workunit.client.0.smithi028.stdout:3/960: fsync d18/d6f/f9e 0 2024-07-03T15:44:20.238 INFO:tasks.workunit.client.0.smithi028.stdout:3/961: read d18/d3a/d9d/d8c/db4/fe7 [3575877,105817] 0 2024-07-03T15:44:20.239 INFO:tasks.workunit.client.0.smithi028.stdout:3/962: creat d18/d1a/d1e/d58/d9f/de2/f137 x:0 0 0 2024-07-03T15:44:20.240 INFO:tasks.workunit.client.0.smithi028.stdout:3/963: mkdir d18/d138 0 2024-07-03T15:44:20.240 INFO:tasks.workunit.client.0.smithi028.stdout:3/964: write d18/d3a/dbb/f111 [733240,25604] 0 2024-07-03T15:44:20.252 INFO:tasks.workunit.client.0.smithi028.stdout:3/965: dread ff [0,4194304] 0 2024-07-03T15:44:20.252 INFO:tasks.workunit.client.0.smithi028.stdout:3/966: chown d18/d1a/d1e/d73/c98 36200987 1 2024-07-03T15:44:20.252 INFO:tasks.workunit.client.0.smithi028.stdout:0/980: dwrite d2/d3/d4d/fa2 [4194304,4194304] 0 2024-07-03T15:44:20.254 INFO:tasks.workunit.client.0.smithi028.stdout:0/981: link d2/c4e d2/d3/dc4/d58/d82/c12b 0 2024-07-03T15:44:20.254 INFO:tasks.workunit.client.0.smithi028.stdout:0/982: read - d2/d3/dc4/f11e zero size 2024-07-03T15:44:20.256 INFO:tasks.workunit.client.0.smithi028.stdout:0/983: link d2/d3/dc4/fb3 d2/d3/d4d/d56/d78/dae/d11d/d11f/f12c 0 2024-07-03T15:44:20.257 INFO:tasks.workunit.client.0.smithi028.stdout:0/984: rmdir d2/d3/dc4/d36/d45 39 2024-07-03T15:44:20.257 INFO:tasks.workunit.client.0.smithi028.stdout:0/985: write d2/d3/d4d/d56/d78/ff0 [869590,93821] 0 2024-07-03T15:44:20.269 INFO:tasks.workunit.client.0.smithi028.stdout:3/967: dwrite d18/d1a/d1e/f57 [0,4194304] 0 2024-07-03T15:44:20.270 INFO:tasks.workunit.client.0.smithi028.stdout:3/968: symlink d18/d1a/d5e/l139 0 2024-07-03T15:44:20.278 INFO:tasks.workunit.client.0.smithi028.stdout:0/986: dwrite d2/d3/dc4/d36/d45/f7b [0,4194304] 0 2024-07-03T15:44:20.278 INFO:tasks.workunit.client.0.smithi028.stdout:0/987: write d2/d3/f2f [2423485,53562] 0 2024-07-03T15:44:20.279 INFO:tasks.workunit.client.0.smithi028.stdout:0/988: mkdir d2/d3/d4d/d56/d78/dae/d11d/d11f/d12d 0 2024-07-03T15:44:20.280 INFO:tasks.workunit.client.0.smithi028.stdout:0/989: mknod d2/d6d/c12e 0 2024-07-03T15:44:20.293 INFO:tasks.workunit.client.0.smithi028.stdout:0/990: write d2/d3/d4d/fa2 [7749468,5834] 0 2024-07-03T15:44:20.294 INFO:tasks.workunit.client.0.smithi028.stdout:3/969: dwrite d18/d6f/d78/d129/f126 [0,4194304] 0 2024-07-03T15:44:20.295 INFO:tasks.workunit.client.0.smithi028.stdout:0/991: creat d2/d3/d4d/d56/d78/dbe/f12f x:0 0 0 2024-07-03T15:44:20.298 INFO:tasks.workunit.client.0.smithi028.stdout:3/970: creat d18/d1a/d5e/f13a x:0 0 0 2024-07-03T15:44:20.298 INFO:tasks.workunit.client.0.smithi028.stdout:3/971: fsync d18/d3a/f5a 0 2024-07-03T15:44:20.298 INFO:tasks.workunit.client.0.smithi028.stdout:3/972: write d18/d6f/f9e [5052908,97296] 0 2024-07-03T15:44:20.301 INFO:tasks.workunit.client.0.smithi028.stdout:3/973: truncate d18/d1a/f4f 5175890 0 2024-07-03T15:44:20.305 INFO:tasks.workunit.client.0.smithi028.stdout:3/974: mkdir d18/d1a/d1e/d58/df3/df9/d13b 0 2024-07-03T15:44:20.306 INFO:tasks.workunit.client.0.smithi028.stdout:3/975: write d18/d1a/d1e/d58/f12c [469541,73984] 0 2024-07-03T15:44:20.306 INFO:tasks.workunit.client.0.smithi028.stdout:3/976: fdatasync d18/d6f/f7d 0 2024-07-03T15:44:20.306 INFO:tasks.workunit.client.0.smithi028.stdout:3/977: readlink d18/d1a/d1e/d58/d9f/lc6 0 2024-07-03T15:44:20.306 INFO:tasks.workunit.client.0.smithi028.stdout:3/978: truncate d18/d3a/dc1/f134 2342385 0 2024-07-03T15:44:20.323 INFO:tasks.workunit.client.0.smithi028.stdout:3/979: dwrite d18/d43/f4d [0,4194304] 0 2024-07-03T15:44:20.342 INFO:tasks.workunit.client.0.smithi028.stdout:3/980: dwrite d18/d3a/f76 [0,4194304] 0 2024-07-03T15:44:20.345 INFO:tasks.workunit.client.0.smithi028.stdout:3/981: unlink d18/d3a/dc1/f134 0 2024-07-03T15:44:20.345 INFO:tasks.workunit.client.0.smithi028.stdout:3/982: creat d18/d1a/d64/ddc/f13c x:0 0 0 2024-07-03T15:44:20.346 INFO:tasks.workunit.client.0.smithi028.stdout:3/983: dread d18/d1a/d5e/d66/fb2 [0,4194304] 0 2024-07-03T15:44:20.348 INFO:tasks.workunit.client.0.smithi028.stdout:3/984: creat d18/d6f/d78/d129/d12e/f13d x:0 0 0 2024-07-03T15:44:20.349 INFO:tasks.workunit.client.0.smithi028.stdout:3/985: creat d18/d1a/d1e/d58/d9f/de2/f13e x:0 0 0 2024-07-03T15:44:20.357 INFO:tasks.workunit.client.0.smithi028.stdout:3/986: dread d18/d1a/d1e/f57 [0,4194304] 0 2024-07-03T15:44:20.357 INFO:tasks.workunit.client.0.smithi028.stdout:3/987: chown d18/d6f/d78/d87/d9c 69 1 2024-07-03T15:44:20.358 INFO:tasks.workunit.client.0.smithi028.stdout:3/988: creat d18/d3a/db6/f13f x:0 0 0 2024-07-03T15:44:20.359 INFO:tasks.workunit.client.0.smithi028.stdout:3/989: mknod d18/d1a/d1e/d58/d9f/da9/c140 0 2024-07-03T15:44:20.359 INFO:tasks.workunit.client.0.smithi028.stdout:3/990: fsync d18/d3a/dc1/dd0/f117 0 2024-07-03T15:44:20.361 INFO:tasks.workunit.client.0.smithi028.stdout:3/991: unlink d18/d6f/d78/d87/d9c/lda 0 2024-07-03T15:44:20.361 INFO:tasks.workunit.client.0.smithi028.stdout:3/992: rename d18/d1a/d1e/d58/d9f/de2 to d18/d1a/d1e/d58/d9f/de2/d141 22 2024-07-03T15:44:20.361 INFO:tasks.workunit.client.0.smithi028.stdout:3/993: stat d18/d6f/d78/d105 0 2024-07-03T15:44:20.387 INFO:tasks.workunit.client.0.smithi028.stdout:3/994: dwrite d18/d1a/d1e/d58/f12c [0,4194304] 0 2024-07-03T15:44:20.387 INFO:tasks.workunit.client.0.smithi028.stdout:7/741: sync 2024-07-03T15:44:20.387 INFO:tasks.workunit.client.0.smithi028.stdout:7/742: write d9/d1c/d45/f57 [309794,64021] 0 2024-07-03T15:44:20.387 INFO:tasks.workunit.client.0.smithi028.stdout:7/743: read d9/f48 [2856812,127904] 0 2024-07-03T15:44:20.387 INFO:tasks.workunit.client.0.smithi028.stdout:7/744: write d9/d1c/d45/db5/d92/fc6 [677478,37563] 0 2024-07-03T15:44:20.388 INFO:tasks.workunit.client.0.smithi028.stdout:3/995: rename d18/d1a/d1e/d73/c98 to d18/d1a/c142 0 2024-07-03T15:44:20.388 INFO:tasks.workunit.client.0.smithi028.stdout:3/996: chown d18/d1a/d1e/d58/d9f/de2/f137 92 1 2024-07-03T15:44:20.390 INFO:tasks.workunit.client.0.smithi028.stdout:7/745: rename d9/d1c/d45/d71/fb1 to d9/d1c/d86/fe1 0 2024-07-03T15:44:20.392 INFO:tasks.workunit.client.0.smithi028.stdout:7/746: mkdir d9/d1c/d45/db5/d92/d99/daf/dc4/de2 0 2024-07-03T15:44:20.394 INFO:tasks.workunit.client.0.smithi028.stdout:7/747: creat d9/d1c/d45/db5/d92/d99/daf/dc4/de2/fe3 x:0 0 0 2024-07-03T15:44:20.394 INFO:tasks.workunit.client.0.smithi028.stdout:7/748: chown d9/d1c/d45/db5/d92/d9e/dda/dc5 515597 1 2024-07-03T15:44:20.395 INFO:tasks.workunit.client.0.smithi028.stdout:7/749: mkdir d9/d1c/d45/db5/d92/de4 0 2024-07-03T15:44:20.396 INFO:tasks.workunit.client.0.smithi028.stdout:7/750: symlink d9/d1c/d45/db5/d92/d99/daf/dc4/de2/le5 0 2024-07-03T15:44:20.398 INFO:tasks.workunit.client.0.smithi028.stdout:7/751: rename d9/d1c/d45/fae to d9/d1c/d6e/fe6 0 2024-07-03T15:44:20.400 INFO:tasks.workunit.client.0.smithi028.stdout:7/752: creat d9/d1c/d45/db5/d92/d9e/fe7 x:0 0 0 2024-07-03T15:44:20.400 INFO:tasks.workunit.client.0.smithi028.stdout:7/753: truncate d9/d1c/d86/fbe 130503 0 2024-07-03T15:44:20.400 INFO:tasks.workunit.client.0.smithi028.stdout:7/754: readlink d9/d1c/d86/l96 0 2024-07-03T15:44:20.409 INFO:tasks.workunit.client.0.smithi028.stdout:3/997: dwrite d18/d3a/dc1/dd0/fff [0,4194304] 0 2024-07-03T15:44:20.409 INFO:tasks.workunit.client.0.smithi028.stdout:3/998: dread - d18/d3a/fbf zero size 2024-07-03T15:44:20.414 INFO:tasks.workunit.client.0.smithi028.stdout:3/999: mkdir d18/d1a/d1e/d58/df3/df9/d13b/d143 0 2024-07-03T15:44:20.416 INFO:tasks.workunit.client.0.smithi028.stdout:7/755: truncate d9/d1c/d86/fbe 85536 0 2024-07-03T15:44:20.417 INFO:tasks.workunit.client.0.smithi028.stdout:7/756: read d9/f34 [1720065,50750] 0 2024-07-03T15:44:20.418 INFO:tasks.workunit.client.0.smithi028.stdout:7/757: rename d9/f48 to d9/d1c/d45/db5/d92/d9e/dda/dc5/fe8 0 2024-07-03T15:44:20.431 INFO:tasks.workunit.client.0.smithi028.stdout:7/758: dwrite d9/d1c/d45/db5/fb0 [0,4194304] 0 2024-07-03T15:44:20.432 INFO:tasks.workunit.client.0.smithi028.stdout:7/759: fdatasync d9/d1c/d45/db5/d92/fc6 0 2024-07-03T15:44:20.435 INFO:tasks.workunit.client.0.smithi028.stdout:7/760: rename d9/d1c/d45/db5/d92/d99/d6a/fad to d9/d1c/d45/d71/fe9 0 2024-07-03T15:44:20.437 INFO:tasks.workunit.client.0.smithi028.stdout:7/761: unlink l8 0 2024-07-03T15:44:20.446 INFO:tasks.workunit.client.0.smithi028.stdout:7/762: write d9/d1c/d45/db5/f44 [4019298,24193] 0 2024-07-03T15:44:20.448 INFO:tasks.workunit.client.0.smithi028.stdout:7/763: rename d9/d1c/d6e/f73 to d9/d1c/d45/db5/d92/d99/daf/dc4/de2/fea 0 2024-07-03T15:44:20.461 INFO:tasks.workunit.client.0.smithi028.stdout:7/764: dwrite d9/d1c/f4b [0,4194304] 0 2024-07-03T15:44:20.461 INFO:tasks.workunit.client.0.smithi028.stdout:7/765: readlink d9/d1c/l66 0 2024-07-03T15:44:20.461 INFO:tasks.workunit.client.0.smithi028.stdout:7/766: mknod d9/d1c/d86/ceb 0 2024-07-03T15:44:20.463 INFO:tasks.workunit.client.0.smithi028.stdout:7/767: dread d9/d1c/d45/db5/f31 [0,4194304] 0 2024-07-03T15:44:20.464 INFO:tasks.workunit.client.0.smithi028.stdout:7/768: symlink d9/lec 0 2024-07-03T15:44:20.465 INFO:tasks.workunit.client.0.smithi028.stdout:7/769: mknod d9/d1c/d45/db5/d92/d9e/ced 0 2024-07-03T15:44:20.465 INFO:tasks.workunit.client.0.smithi028.stdout:7/770: write d9/d1c/d6e/f8c [625577,43082] 0 2024-07-03T15:44:20.481 INFO:tasks.workunit.client.0.smithi028.stdout:7/771: dwrite d9/d1c/d45/d71/d78/fbb [0,4194304] 0 2024-07-03T15:44:20.485 INFO:tasks.workunit.client.0.smithi028.stdout:7/772: unlink d9/d1c/d45/c5f 0 2024-07-03T15:44:20.485 INFO:tasks.workunit.client.0.smithi028.stdout:7/773: chown d9/d1c/f6b 988993 1 2024-07-03T15:44:20.486 INFO:tasks.workunit.client.0.smithi028.stdout:7/774: symlink d9/d1c/d45/db5/d92/d9e/dda/lee 0 2024-07-03T15:44:20.486 INFO:tasks.workunit.client.0.smithi028.stdout:7/775: chown d9/d1c/d45/db5/d92/d99/daf/dc4/de2 18 1 2024-07-03T15:44:20.494 INFO:tasks.workunit.client.0.smithi028.stdout:7/776: dread d9/d1c/d45/db5/f3e [0,4194304] 0 2024-07-03T15:44:20.494 INFO:tasks.workunit.client.0.smithi028.stdout:7/777: fsync d9/f10 0 2024-07-03T15:44:20.495 INFO:tasks.workunit.client.0.smithi028.stdout:7/778: fdatasync d9/d1c/f6b 0 2024-07-03T15:44:20.638 INFO:tasks.workunit.client.0.smithi028.stdout:2/729: sync 2024-07-03T15:44:20.655 INFO:tasks.workunit.client.0.smithi028.stdout:2/730: dwrite da/d23/f96 [0,4194304] 0 2024-07-03T15:44:20.656 INFO:tasks.workunit.client.0.smithi028.stdout:2/731: mkdir da/d24/d103 0 2024-07-03T15:44:20.673 INFO:tasks.workunit.client.0.smithi028.stdout:2/732: dwrite da/f15 [0,4194304] 0 2024-07-03T15:44:20.673 INFO:tasks.workunit.client.0.smithi028.stdout:2/733: creat da/f104 x:0 0 0 2024-07-03T15:44:20.682 INFO:tasks.workunit.client.0.smithi028.stdout:2/734: dread da/d23/f75 [0,4194304] 0 2024-07-03T15:44:20.704 INFO:tasks.workunit.client.0.smithi028.stdout:2/735: dwrite da/d24/d2c/f3d [0,4194304] 0 2024-07-03T15:44:20.704 INFO:tasks.workunit.client.0.smithi028.stdout:2/736: fsync da/d23/d6f/d84/fa7 0 2024-07-03T15:44:20.706 INFO:tasks.workunit.client.0.smithi028.stdout:2/737: getdents da/d42/d47/de8 0 2024-07-03T15:44:20.706 INFO:tasks.workunit.client.0.smithi028.stdout:2/738: dread - da/fc4 zero size 2024-07-03T15:44:20.707 INFO:tasks.workunit.client.0.smithi028.stdout:2/739: truncate da/d42/d47/fa5 723095 0 2024-07-03T15:44:20.707 INFO:tasks.workunit.client.0.smithi028.stdout:2/740: chown da/d23/f3b 1849508 1 2024-07-03T15:44:20.707 INFO:tasks.workunit.client.0.smithi028.stdout:2/741: write da/d1e/d56/dc0/f102 [50818,19232] 0 2024-07-03T15:44:20.715 INFO:tasks.workunit.client.0.smithi028.stdout:2/742: dread da/d23/d6f/d84/fa7 [0,4194304] 0 2024-07-03T15:44:20.715 INFO:tasks.workunit.client.0.smithi028.stdout:2/743: write da/f85 [916013,7262] 0 2024-07-03T15:44:20.715 INFO:tasks.workunit.client.0.smithi028.stdout:2/744: creat da/d1e/dd0/f105 x:0 0 0 2024-07-03T15:44:20.717 INFO:tasks.workunit.client.0.smithi028.stdout:2/745: mkdir da/d1e/d56/d106 0 2024-07-03T15:44:20.721 INFO:tasks.workunit.client.0.smithi028.stdout:2/746: creat da/d24/f107 x:0 0 0 2024-07-03T15:44:20.724 INFO:tasks.workunit.client.0.smithi028.stdout:2/747: dread da/d24/f55 [0,4194304] 0 2024-07-03T15:44:20.724 INFO:tasks.workunit.client.0.smithi028.stdout:2/748: write da/d1e/dd0/feb [5061941,118495] 0 2024-07-03T15:44:20.724 INFO:tasks.workunit.client.0.smithi028.stdout:2/749: creat da/d24/d2c/dc6/f108 x:0 0 0 2024-07-03T15:44:20.724 INFO:tasks.workunit.client.0.smithi028.stdout:2/750: stat da/d42/d47/de8/f3e 0 2024-07-03T15:44:20.726 INFO:tasks.workunit.client.0.smithi028.stdout:2/751: symlink da/d74/l109 0 2024-07-03T15:44:20.741 INFO:tasks.workunit.client.0.smithi028.stdout:2/752: dwrite da/d24/fc7 [0,4194304] 0 2024-07-03T15:44:20.743 INFO:tasks.workunit.client.0.smithi028.stdout:2/753: mkdir da/d23/d6f/d84/d10a 0 2024-07-03T15:44:20.747 INFO:tasks.workunit.client.0.smithi028.stdout:2/754: getdents da/d1e/dd0 0 2024-07-03T15:44:20.749 INFO:tasks.workunit.client.0.smithi028.stdout:2/755: stat da/d23/l49 0 2024-07-03T15:44:20.751 INFO:tasks.workunit.client.0.smithi028.stdout:2/756: rmdir da/d1e/d56 39 2024-07-03T15:44:20.751 INFO:tasks.workunit.client.0.smithi028.stdout:2/757: fdatasync da/d24/d2c/f61 0 2024-07-03T15:44:20.753 INFO:tasks.workunit.client.0.smithi028.stdout:2/758: symlink da/d24/d2c/l10b 0 2024-07-03T15:44:20.753 INFO:tasks.workunit.client.0.smithi028.stdout:2/759: chown da/d42/d47/da2 1276434 1 2024-07-03T15:44:20.753 INFO:tasks.workunit.client.0.smithi028.stdout:2/760: write da/d1e/d56/dc0/ff4 [775722,42440] 0 2024-07-03T15:44:20.753 INFO:tasks.workunit.client.0.smithi028.stdout:2/761: creat da/d1e/d68/f10c x:0 0 0 2024-07-03T15:44:20.757 INFO:tasks.workunit.client.0.smithi028.stdout:2/762: mknod da/d1e/d56/c10d 0 2024-07-03T15:44:20.757 INFO:tasks.workunit.client.0.smithi028.stdout:2/763: write da/d42/fe2 [104040,80953] 0 2024-07-03T15:44:20.757 INFO:tasks.workunit.client.0.smithi028.stdout:2/764: chown da/d42/d47/daa 730949 1 2024-07-03T15:44:20.757 INFO:tasks.workunit.client.0.smithi028.stdout:2/765: creat da/d1e/d56/dc0/f10e x:0 0 0 2024-07-03T15:44:20.763 INFO:tasks.workunit.client.0.smithi028.stdout:2/766: creat da/d42/f10f x:0 0 0 2024-07-03T15:44:20.774 INFO:tasks.workunit.client.0.smithi028.stdout:2/767: dwrite da/d24/fc7 [0,4194304] 0 2024-07-03T15:44:20.778 INFO:tasks.workunit.client.0.smithi028.stdout:2/768: link da/d23/d6f/d84/db8/cfe da/d1e/dec/c110 0 2024-07-03T15:44:20.792 INFO:tasks.workunit.client.0.smithi028.stdout:7/779: sync 2024-07-03T15:44:20.792 INFO:tasks.workunit.client.0.smithi028.stdout:0/992: sync 2024-07-03T15:44:20.792 INFO:tasks.workunit.client.0.smithi028.stdout:4/980: sync 2024-07-03T15:44:20.793 INFO:tasks.workunit.client.0.smithi028.stdout:7/780: rename d9/d1c/d45/db5/d5d/f79 to d9/d1c/d45/db5/d92/d99/daf/fef 0 2024-07-03T15:44:20.794 INFO:tasks.workunit.client.0.smithi028.stdout:4/981: link d2/d5a/d45/d84/fec d2/d5a/d45/d84/df4/d107/f145 0 2024-07-03T15:44:20.795 INFO:tasks.workunit.client.0.smithi028.stdout:4/982: truncate d2/d12/d5b/d78/de4/d3a/ff2 692215 0 2024-07-03T15:44:20.796 INFO:tasks.workunit.client.0.smithi028.stdout:7/781: rmdir d9/d1c/d7c/d8b/dd7 39 2024-07-03T15:44:20.797 INFO:tasks.workunit.client.0.smithi028.stdout:4/983: symlink d2/d3e/d9f/d74/l146 0 2024-07-03T15:44:20.798 INFO:tasks.workunit.client.0.smithi028.stdout:4/984: mknod d2/d12/d42/d63/d6d/dc8/c147 0 2024-07-03T15:44:20.798 INFO:tasks.workunit.client.0.smithi028.stdout:4/985: chown d2/d5a/d45/d84/df4/f91 2 1 2024-07-03T15:44:20.798 INFO:tasks.workunit.client.0.smithi028.stdout:4/986: dread - d2/d5a/d45/ff7 zero size 2024-07-03T15:44:20.798 INFO:tasks.workunit.client.0.smithi028.stdout:4/987: creat d2/d12/d5b/f148 x:0 0 0 2024-07-03T15:44:20.800 INFO:tasks.workunit.client.0.smithi028.stdout:2/769: dwrite da/d42/fb0 [0,4194304] 0 2024-07-03T15:44:20.811 INFO:tasks.workunit.client.0.smithi028.stdout:4/988: dread d2/d5a/d45/d84/df4/d92/d12e/d106/f13c [0,4194304] 0 2024-07-03T15:44:20.811 INFO:tasks.workunit.client.0.smithi028.stdout:4/989: stat d2/d5a/d45/d84/df4/d107/f145 0 2024-07-03T15:44:20.811 INFO:tasks.workunit.client.0.smithi028.stdout:4/990: chown d2/d12/d2c/c44 3307731 1 2024-07-03T15:44:20.811 INFO:tasks.workunit.client.0.smithi028.stdout:4/991: symlink d2/d12/d5b/d78/de4/d3a/l149 0 2024-07-03T15:44:20.812 INFO:tasks.workunit.client.0.smithi028.stdout:0/993: dread d2/d3/f73 [0,4194304] 0 2024-07-03T15:44:20.812 INFO:tasks.workunit.client.0.smithi028.stdout:0/994: creat d2/d1c/d6c/dc5/f130 x:0 0 0 2024-07-03T15:44:20.812 INFO:tasks.workunit.client.0.smithi028.stdout:4/992: read d2/d3e/d9f/d74/fce [14205,111745] 0 2024-07-03T15:44:20.814 INFO:tasks.workunit.client.0.smithi028.stdout:0/995: getdents d2/d3/d4d/d56/dcd 0 2024-07-03T15:44:20.815 INFO:tasks.workunit.client.0.smithi028.stdout:2/770: write da/ff [2175700,32801] 0 2024-07-03T15:44:20.816 INFO:tasks.workunit.client.0.smithi028.stdout:7/782: dwrite d9/d1c/f5a [0,4194304] 0 2024-07-03T15:44:20.822 INFO:tasks.workunit.client.0.smithi028.stdout:2/771: unlink da/f12 0 2024-07-03T15:44:20.828 INFO:tasks.workunit.client.0.smithi028.stdout:0/996: dread d2/d3/dc4/d36/d45/f46 [0,4194304] 0 2024-07-03T15:44:20.844 INFO:tasks.workunit.client.0.smithi028.stdout:7/783: dwrite d9/d1c/d45/db5/f44 [4194304,4194304] 0 2024-07-03T15:44:20.845 INFO:tasks.workunit.client.0.smithi028.stdout:7/784: truncate d9/d1c/d45/db5/d92/d99/daf/dc4/fdf 3277135 0 2024-07-03T15:44:20.845 INFO:tasks.workunit.client.0.smithi028.stdout:7/785: chown d9/d1c/l66 0 1 2024-07-03T15:44:20.848 INFO:tasks.workunit.client.0.smithi028.stdout:2/772: dwrite da/d23/d6f/d84/db8/fce [0,4194304] 0 2024-07-03T15:44:20.849 INFO:tasks.workunit.client.0.smithi028.stdout:0/997: dwrite d2/d1c/f83 [0,4194304] 0 2024-07-03T15:44:20.849 INFO:tasks.workunit.client.0.smithi028.stdout:0/998: fdatasync d2/d1c/d6c/dc5/ffd 0 2024-07-03T15:44:20.849 INFO:tasks.workunit.client.0.smithi028.stdout:2/773: write da/d42/d47/d78/f9d [510831,122745] 0 2024-07-03T15:44:20.851 INFO:tasks.workunit.client.0.smithi028.stdout:0/999: dread d2/d3/dc4/d58/d82/d9b/fe0 [0,4194304] 0 2024-07-03T15:44:20.860 INFO:tasks.workunit.client.0.smithi028.stdout:2/774: write da/d23/d6f/d84/db8/fce [2642367,101029] 0 2024-07-03T15:44:20.861 INFO:tasks.workunit.client.0.smithi028.stdout:2/775: chown da/d23/d6f/d84/db8/cfe 101941966 1 2024-07-03T15:44:20.866 INFO:tasks.workunit.client.0.smithi028.stdout:7/786: dwrite d9/d1c/d45/f46 [0,4194304] 0 2024-07-03T15:44:20.866 INFO:tasks.workunit.client.0.smithi028.stdout:7/787: chown d9/d1c/d45/db5/d92/d99/daf/fbd 21 1 2024-07-03T15:44:20.867 INFO:tasks.workunit.client.0.smithi028.stdout:2/776: rename da/d23/d6f/d84/d10a to da/d74/d111 0 2024-07-03T15:44:20.870 INFO:tasks.workunit.client.0.smithi028.stdout:7/788: getdents d9/d1c/da9 0 2024-07-03T15:44:20.872 INFO:tasks.workunit.client.0.smithi028.stdout:2/777: creat da/d23/d6f/f112 x:0 0 0 2024-07-03T15:44:20.877 INFO:tasks.workunit.client.0.smithi028.stdout:2/778: rename da/d42/d47/d78/fa3 to da/d23/f113 0 2024-07-03T15:44:20.877 INFO:tasks.workunit.client.0.smithi028.stdout:2/779: write da/fc4 [174988,106051] 0 2024-07-03T15:44:20.877 INFO:tasks.workunit.client.0.smithi028.stdout:2/780: dread - da/d24/d2c/dba/fe5 zero size 2024-07-03T15:44:20.877 INFO:tasks.workunit.client.0.smithi028.stdout:2/781: stat da/d42/fac 0 2024-07-03T15:44:20.888 INFO:tasks.workunit.client.0.smithi028.stdout:2/782: dwrite da/d24/d2c/f61 [4194304,4194304] 0 2024-07-03T15:44:20.888 INFO:tasks.workunit.client.0.smithi028.stdout:2/783: fsync da/d42/d47/d78/fb1 0 2024-07-03T15:44:20.888 INFO:tasks.workunit.client.0.smithi028.stdout:2/784: chown da/d24/d2c/f3d 7 1 2024-07-03T15:44:20.890 INFO:tasks.workunit.client.0.smithi028.stdout:2/785: mknod da/d74/d111/c114 0 2024-07-03T15:44:20.890 INFO:tasks.workunit.client.0.smithi028.stdout:2/786: readlink da/d1e/d68/lbd 0 2024-07-03T15:44:20.890 INFO:tasks.workunit.client.0.smithi028.stdout:2/787: read - da/d1e/dd0/f105 zero size 2024-07-03T15:44:20.890 INFO:tasks.workunit.client.0.smithi028.stdout:2/788: chown da/d24/d2c/c36 95586 1 2024-07-03T15:44:20.890 INFO:tasks.workunit.client.0.smithi028.stdout:2/789: stat da/d42/d47/de8/c99 0 2024-07-03T15:44:20.892 INFO:tasks.workunit.client.0.smithi028.stdout:2/790: mkdir da/d23/d115 0 2024-07-03T15:44:20.895 INFO:tasks.workunit.client.0.smithi028.stdout:2/791: write da/d24/f55 [963646,103586] 0 2024-07-03T15:44:20.898 INFO:tasks.workunit.client.0.smithi028.stdout:2/792: mknod da/d1e/d56/c116 0 2024-07-03T15:44:20.899 INFO:tasks.workunit.client.0.smithi028.stdout:2/793: creat da/d42/d47/f117 x:0 0 0 2024-07-03T15:44:20.899 INFO:tasks.workunit.client.0.smithi028.stdout:2/794: write da/f85 [1552323,87080] 0 2024-07-03T15:44:20.899 INFO:tasks.workunit.client.0.smithi028.stdout:2/795: creat da/d42/d47/da2/f118 x:0 0 0 2024-07-03T15:44:20.900 INFO:tasks.workunit.client.0.smithi028.stdout:2/796: write da/d42/f46 [4362282,84186] 0 2024-07-03T15:44:20.903 INFO:tasks.workunit.client.0.smithi028.stdout:2/797: creat da/d1e/f119 x:0 0 0 2024-07-03T15:44:20.919 INFO:tasks.workunit.client.0.smithi028.stdout:2/798: dwrite da/d42/d47/de8/f3e [0,4194304] 0 2024-07-03T15:44:20.919 INFO:tasks.workunit.client.0.smithi028.stdout:2/799: readlink da/d24/d2c/l2b 0 2024-07-03T15:44:20.919 INFO:tasks.workunit.client.0.smithi028.stdout:2/800: dread - da/d42/d47/de8/fcc zero size 2024-07-03T15:44:20.919 INFO:tasks.workunit.client.0.smithi028.stdout:2/801: creat da/f11a x:0 0 0 2024-07-03T15:44:20.922 INFO:tasks.workunit.client.0.smithi028.stdout:2/802: mkdir da/d23/d6f/d11b 0 2024-07-03T15:44:20.922 INFO:tasks.workunit.client.0.smithi028.stdout:2/803: readlink da/d1e/d68/l7f 0 2024-07-03T15:44:20.923 INFO:tasks.workunit.client.0.smithi028.stdout:2/804: mknod da/d23/d6f/d11b/c11c 0 2024-07-03T15:44:20.923 INFO:tasks.workunit.client.0.smithi028.stdout:2/805: symlink da/d1e/dd0/l11d 0 2024-07-03T15:44:20.930 INFO:tasks.workunit.client.0.smithi028.stdout:2/806: dread da/d24/d2c/f97 [0,4194304] 0 2024-07-03T15:44:20.930 INFO:tasks.workunit.client.0.smithi028.stdout:2/807: readlink da/d23/d6f/l88 0 2024-07-03T15:44:20.930 INFO:tasks.workunit.client.0.smithi028.stdout:2/808: write da/d42/d47/f117 [923461,119902] 0 2024-07-03T15:44:20.933 INFO:tasks.workunit.client.0.smithi028.stdout:2/809: rmdir da/d1e/d56/dc0 39 2024-07-03T15:44:20.935 INFO:tasks.workunit.client.0.smithi028.stdout:2/810: creat da/d1e/dec/f11e x:0 0 0 2024-07-03T15:44:20.937 INFO:tasks.workunit.client.0.smithi028.stdout:2/811: mknod da/d24/d2c/dc6/c11f 0 2024-07-03T15:44:20.937 INFO:tasks.workunit.client.0.smithi028.stdout:2/812: getdents da/d42/d47/de8 0 2024-07-03T15:44:20.939 INFO:tasks.workunit.client.0.smithi028.stdout:2/813: creat da/d23/d6f/d11b/f120 x:0 0 0 2024-07-03T15:44:20.955 INFO:tasks.workunit.client.0.smithi028.stdout:2/814: dwrite da/d1e/d68/f10c [0,4194304] 0 2024-07-03T15:44:20.965 INFO:tasks.workunit.client.0.smithi028.stdout:2/815: getdents da/d24 0 2024-07-03T15:44:20.978 INFO:tasks.workunit.client.0.smithi028.stdout:2/816: dwrite da/d23/d6f/fae [0,4194304] 0 2024-07-03T15:44:20.987 INFO:tasks.workunit.client.0.smithi028.stdout:2/817: getdents da/d42/d47/daa 0 2024-07-03T15:44:20.987 INFO:tasks.workunit.client.0.smithi028.stdout:2/818: creat da/d23/d6f/d84/db8/f121 x:0 0 0 2024-07-03T15:44:20.987 INFO:tasks.workunit.client.0.smithi028.stdout:2/819: stat da/d24/d2c/dc6/ldb 0 2024-07-03T15:44:20.990 INFO:tasks.workunit.client.0.smithi028.stdout:2/820: mknod da/d1e/d56/c122 0 2024-07-03T15:44:20.990 INFO:tasks.workunit.client.0.smithi028.stdout:2/821: creat da/d23/d6f/d11b/f123 x:0 0 0 2024-07-03T15:44:20.995 INFO:tasks.workunit.client.0.smithi028.stdout:2/822: dread da/d24/f55 [0,4194304] 0 2024-07-03T15:44:20.997 INFO:tasks.workunit.client.0.smithi028.stdout:2/823: unlink da/d42/f57 0 2024-07-03T15:44:20.998 INFO:tasks.workunit.client.0.smithi028.stdout:2/824: creat da/d1e/d56/f124 x:0 0 0 2024-07-03T15:44:20.998 INFO:tasks.workunit.client.0.smithi028.stdout:2/825: readlink da/d74/l109 0 2024-07-03T15:44:20.999 INFO:tasks.workunit.client.0.smithi028.stdout:2/826: symlink da/d24/d2c/dc6/df0/l125 0 2024-07-03T15:44:20.999 INFO:tasks.workunit.client.0.smithi028.stdout:2/827: write da/d42/d47/de8/f8b [233091,36881] 0 2024-07-03T15:44:20.999 INFO:tasks.workunit.client.0.smithi028.stdout:2/828: dread - da/fed zero size 2024-07-03T15:44:20.999 INFO:tasks.workunit.client.0.smithi028.stdout:2/829: write da/d23/d6f/d84/db8/fce [925033,23521] 0 2024-07-03T15:44:21.299 INFO:tasks.workunit.client.0.smithi028.stdout:4/993: sync 2024-07-03T15:44:21.469 INFO:tasks.workunit.client.0.smithi028.stdout:7/789: sync 2024-07-03T15:44:21.469 INFO:tasks.workunit.client.0.smithi028.stdout:2/830: sync 2024-07-03T15:44:21.470 INFO:tasks.workunit.client.0.smithi028.stdout:7/790: chown d9/c12 2 1 2024-07-03T15:44:21.470 INFO:tasks.workunit.client.0.smithi028.stdout:2/831: chown da/d1e/d68/lbd 454053 1 2024-07-03T15:44:21.470 INFO:tasks.workunit.client.0.smithi028.stdout:2/832: write da/d24/d2c/dc6/f108 [612352,46810] 0 2024-07-03T15:44:21.470 INFO:tasks.workunit.client.0.smithi028.stdout:2/833: chown da/cf7 1 1 2024-07-03T15:44:21.470 INFO:tasks.workunit.client.0.smithi028.stdout:7/791: unlink d9/f34 0 2024-07-03T15:44:21.474 INFO:tasks.workunit.client.0.smithi028.stdout:2/834: dread da/d23/d6f/d84/fca [0,4194304] 0 2024-07-03T15:44:21.474 INFO:tasks.workunit.client.0.smithi028.stdout:2/835: chown da/d42/d47/de8/f8b 502922987 1 2024-07-03T15:44:21.479 INFO:tasks.workunit.client.0.smithi028.stdout:2/836: rmdir da/d24/d2c/dc6 39 2024-07-03T15:44:21.481 INFO:tasks.workunit.client.0.smithi028.stdout:2/837: dread da/d42/d47/daa/fdf [0,4194304] 0 2024-07-03T15:44:21.483 INFO:tasks.workunit.client.0.smithi028.stdout:2/838: symlink da/d74/l126 0 2024-07-03T15:44:21.483 INFO:tasks.workunit.client.0.smithi028.stdout:2/839: fsync da/d24/d2c/faf 0 2024-07-03T15:44:21.486 INFO:tasks.workunit.client.0.smithi028.stdout:2/840: mkdir da/d127 0 2024-07-03T15:44:21.486 INFO:tasks.workunit.client.0.smithi028.stdout:2/841: fdatasync da/d1e/dd0/fd4 0 2024-07-03T15:44:21.489 INFO:tasks.workunit.client.0.smithi028.stdout:2/842: rename da/d1e/d68/f10c to da/d42/f128 0 2024-07-03T15:44:21.489 INFO:tasks.workunit.client.0.smithi028.stdout:2/843: fdatasync da/d74/f90 0 2024-07-03T15:44:21.489 INFO:tasks.workunit.client.0.smithi028.stdout:2/844: write da/d1e/dd0/f105 [16236,117214] 0 2024-07-03T15:44:21.490 INFO:tasks.workunit.client.0.smithi028.stdout:2/845: mkdir da/d23/d6f/d84/d129 0 2024-07-03T15:44:21.491 INFO:tasks.workunit.client.0.smithi028.stdout:7/792: dwrite d9/d1c/d45/d71/fe9 [4194304,4194304] 0 2024-07-03T15:44:21.492 INFO:tasks.workunit.client.0.smithi028.stdout:7/793: mkdir d9/d1c/da9/df0 0 2024-07-03T15:44:21.492 INFO:tasks.workunit.client.0.smithi028.stdout:7/794: write d9/f1f [3065758,91765] 0 2024-07-03T15:44:21.501 INFO:tasks.workunit.client.0.smithi028.stdout:2/846: dread da/d24/fa4 [0,4194304] 0 2024-07-03T15:44:21.501 INFO:tasks.workunit.client.0.smithi028.stdout:2/847: stat da/d23/d6f/d84/fca 0 2024-07-03T15:44:21.503 INFO:tasks.workunit.client.0.smithi028.stdout:7/795: write d9/d1c/d86/fb7 [2724817,103521] 0 2024-07-03T15:44:21.506 INFO:tasks.workunit.client.0.smithi028.stdout:7/796: dread d9/d1c/d45/db5/d92/d99/d6a/f7d [0,4194304] 0 2024-07-03T15:44:21.509 INFO:tasks.workunit.client.0.smithi028.stdout:7/797: symlink d9/d1c/d45/db5/d92/d9e/dda/lf1 0 2024-07-03T15:44:21.510 INFO:tasks.workunit.client.0.smithi028.stdout:7/798: mkdir d9/d1c/d45/d71/df2 0 2024-07-03T15:44:21.513 INFO:tasks.workunit.client.0.smithi028.stdout:7/799: mknod d9/d1c/d45/d71/d78/cf3 0 2024-07-03T15:44:21.513 INFO:tasks.workunit.client.0.smithi028.stdout:7/800: fsync d9/d1c/d45/db5/f81 0 2024-07-03T15:44:21.513 INFO:tasks.workunit.client.0.smithi028.stdout:7/801: chown d9/d1c/d86/fb7 1976 1 2024-07-03T15:44:21.513 INFO:tasks.workunit.client.0.smithi028.stdout:7/802: fsync d9/d1c/d45/db5/fb0 0 2024-07-03T15:44:21.514 INFO:tasks.workunit.client.0.smithi028.stdout:7/803: creat d9/d1c/d45/db5/d92/ff4 x:0 0 0 2024-07-03T15:44:21.514 INFO:tasks.workunit.client.0.smithi028.stdout:7/804: fdatasync d9/d1c/d6e/fbf 0 2024-07-03T15:44:21.514 INFO:tasks.workunit.client.0.smithi028.stdout:7/805: rmdir d9/d1c/d7c 39 2024-07-03T15:44:21.518 INFO:tasks.workunit.client.0.smithi028.stdout:2/848: dwrite da/d42/f128 [4194304,4194304] 0 2024-07-03T15:44:21.522 INFO:tasks.workunit.client.0.smithi028.stdout:2/849: mkdir da/d12a 0 2024-07-03T15:44:21.522 INFO:tasks.workunit.client.0.smithi028.stdout:2/850: write da/d1e/d56/dc0/f102 [1092099,89884] 0 2024-07-03T15:44:21.531 INFO:tasks.workunit.client.0.smithi028.stdout:2/851: write da/d1e/dd0/feb [4123590,60937] 0 2024-07-03T15:44:21.531 INFO:tasks.workunit.client.0.smithi028.stdout:2/852: readlink da/d23/le4 0 2024-07-03T15:44:21.536 INFO:tasks.workunit.client.0.smithi028.stdout:7/806: dwrite d9/d1c/d45/db5/d92/d99/d6a/f7b [0,4194304] 0 2024-07-03T15:44:21.536 INFO:tasks.workunit.client.0.smithi028.stdout:7/807: readlink d9/l11 0 2024-07-03T15:44:21.543 INFO:tasks.workunit.client.0.smithi028.stdout:7/808: rename d9/d1c/d45/db5/d92/d99/l64 to d9/d1c/d45/db5/lf5 0 2024-07-03T15:44:21.543 INFO:tasks.workunit.client.0.smithi028.stdout:7/809: creat d9/d1c/d6e/ff6 x:0 0 0 2024-07-03T15:44:21.546 INFO:tasks.workunit.client.0.smithi028.stdout:7/810: unlink d9/d1c/d45/d71/d78/l95 0 2024-07-03T15:44:21.547 INFO:tasks.workunit.client.0.smithi028.stdout:2/853: dwrite da/d24/fa4 [0,4194304] 0 2024-07-03T15:44:21.547 INFO:tasks.workunit.client.0.smithi028.stdout:2/854: fdatasync da/d1e/d56/f124 0 2024-07-03T15:44:21.548 INFO:tasks.workunit.client.0.smithi028.stdout:7/811: rename c3 to d9/d1c/d45/d71/df2/cf7 0 2024-07-03T15:44:21.551 INFO:tasks.workunit.client.0.smithi028.stdout:7/812: creat d9/d1c/d45/db5/d92/de4/ff8 x:0 0 0 2024-07-03T15:44:21.551 INFO:tasks.workunit.client.0.smithi028.stdout:7/813: creat d9/d1c/d7c/d8b/ff9 x:0 0 0 2024-07-03T15:44:21.551 INFO:tasks.workunit.client.0.smithi028.stdout:7/814: fdatasync d9/d1c/d45/db5/d92/d99/daf/fef 0 2024-07-03T15:44:21.555 INFO:tasks.workunit.client.0.smithi028.stdout:7/815: mknod d9/d1c/d6e/cfa 0 2024-07-03T15:44:21.555 INFO:tasks.workunit.client.0.smithi028.stdout:7/816: readlink d9/lec 0 2024-07-03T15:44:21.556 INFO:tasks.workunit.client.0.smithi028.stdout:7/817: rename d9/d1c/d7c/c7e to d9/d1c/d45/db5/d92/d99/daf/dc4/de2/cfb 0 2024-07-03T15:44:21.557 INFO:tasks.workunit.client.0.smithi028.stdout:7/818: truncate d9/d1c/f93 810232 0 2024-07-03T15:44:21.557 INFO:tasks.workunit.client.0.smithi028.stdout:7/819: mkdir d9/d1c/d45/db5/dfc 0 2024-07-03T15:44:21.558 INFO:tasks.workunit.client.0.smithi028.stdout:7/820: unlink d9/d1c/d45/d71/fd4 0 2024-07-03T15:44:21.558 INFO:tasks.workunit.client.0.smithi028.stdout:4/994: sync 2024-07-03T15:44:21.559 INFO:tasks.workunit.client.0.smithi028.stdout:4/995: symlink d2/d5a/d45/d84/df4/d92/d12e/l14a 0 2024-07-03T15:44:21.562 INFO:tasks.workunit.client.0.smithi028.stdout:4/996: symlink d2/l14b 0 2024-07-03T15:44:21.562 INFO:tasks.workunit.client.0.smithi028.stdout:4/997: write d2/d5a/d45/d84/df4/d9d/f13b [1655137,27586] 0 2024-07-03T15:44:21.564 INFO:tasks.workunit.client.0.smithi028.stdout:4/998: getdents d2/d12/d2c 0 2024-07-03T15:44:21.565 INFO:tasks.workunit.client.0.smithi028.stdout:2/855: dwrite da/d1e/d56/dc0/f102 [0,4194304] 0 2024-07-03T15:44:21.567 INFO:tasks.workunit.client.0.smithi028.stdout:7/821: read d9/d1c/d45/db5/d92/d99/d6a/f7b [2789995,129946] 0 2024-07-03T15:44:21.568 INFO:tasks.workunit.client.0.smithi028.stdout:2/856: symlink da/d42/d47/d78/l12b 0 2024-07-03T15:44:21.578 INFO:tasks.workunit.client.0.smithi028.stdout:7/822: dread d9/f69 [0,4194304] 0 2024-07-03T15:44:21.580 INFO:tasks.workunit.client.0.smithi028.stdout:7/823: read d9/d1c/d45/db5/d92/d99/d6a/f7d [302047,69579] 0 2024-07-03T15:44:21.580 INFO:tasks.workunit.client.0.smithi028.stdout:7/824: readlink d9/d1c/l65 0 2024-07-03T15:44:21.583 INFO:tasks.workunit.client.0.smithi028.stdout:2/857: read da/d1e/f58 [2903614,25242] 0 2024-07-03T15:44:21.583 INFO:tasks.workunit.client.0.smithi028.stdout:4/999: dwrite d2/d12/d5b/f7a [0,4194304] 0 2024-07-03T15:44:21.583 INFO:tasks.workunit.client.0.smithi028.stdout:7/825: truncate d9/d1c/f3f 981046 0 2024-07-03T15:44:21.583 INFO:tasks.workunit.client.0.smithi028.stdout:7/826: write d9/d1c/d6e/fbf [105675,40489] 0 2024-07-03T15:44:21.583 INFO:tasks.workunit.client.0.smithi028.stdout:7/827: dread - d9/d1c/d45/db5/d92/ff4 zero size 2024-07-03T15:44:21.583 INFO:tasks.workunit.client.0.smithi028.stdout:7/828: readlink d9/d1c/d7c/lcf 0 2024-07-03T15:44:21.584 INFO:tasks.workunit.client.0.smithi028.stdout:7/829: symlink d9/d1c/d45/db5/d92/d9e/dda/lfd 0 2024-07-03T15:44:21.591 INFO:tasks.workunit.client.0.smithi028.stdout:7/830: getdents d9/d1c/d45/db5/d92/d9e/dda 0 2024-07-03T15:44:21.593 INFO:tasks.workunit.client.0.smithi028.stdout:7/831: mknod d9/d1c/da9/df0/cfe 0 2024-07-03T15:44:21.594 INFO:tasks.workunit.client.0.smithi028.stdout:7/832: symlink d9/d1c/da9/df0/lff 0 2024-07-03T15:44:21.597 INFO:tasks.workunit.client.0.smithi028.stdout:7/833: rename d9/d1c/d45/db5/d92/d99/d6a/l6f to d9/d1c/d45/db5/l100 0 2024-07-03T15:44:21.598 INFO:tasks.workunit.client.0.smithi028.stdout:2/858: dwrite da/f1d [0,4194304] 0 2024-07-03T15:44:21.598 INFO:tasks.workunit.client.0.smithi028.stdout:2/859: readlink da/l9b 0 2024-07-03T15:44:21.598 INFO:tasks.workunit.client.0.smithi028.stdout:2/860: readlink da/d24/d2c/l93 0 2024-07-03T15:44:21.598 INFO:tasks.workunit.client.0.smithi028.stdout:7/834: symlink d9/d1c/d45/db5/l101 0 2024-07-03T15:44:21.603 INFO:tasks.workunit.client.0.smithi028.stdout:2/861: rename da/d24/d2c/ce1 to da/d24/d2c/c12c 0 2024-07-03T15:44:21.603 INFO:tasks.workunit.client.0.smithi028.stdout:2/862: dread - da/d24/d2c/dc6/df0/ffa zero size 2024-07-03T15:44:21.604 INFO:tasks.workunit.client.0.smithi028.stdout:7/835: creat d9/d1c/d45/db5/d92/d9e/dda/f102 x:0 0 0 2024-07-03T15:44:21.608 INFO:tasks.workunit.client.0.smithi028.stdout:2/863: creat da/d23/d6f/d84/d129/f12d x:0 0 0 2024-07-03T15:44:21.624 INFO:tasks.workunit.client.0.smithi028.stdout:2/864: dwrite da/d23/f3b [0,4194304] 0 2024-07-03T15:44:21.624 INFO:tasks.workunit.client.0.smithi028.stdout:2/865: write da/d42/d47/de8/fd2 [366234,60140] 0 2024-07-03T15:44:21.624 INFO:tasks.workunit.client.0.smithi028.stdout:2/866: truncate da/d24/f98 827201 0 2024-07-03T15:44:21.625 INFO:tasks.workunit.client.0.smithi028.stdout:2/867: rmdir da/d1e/dd0 39 2024-07-03T15:44:21.640 INFO:tasks.workunit.client.0.smithi028.stdout:2/868: dwrite da/d42/d47/f6d [0,4194304] 0 2024-07-03T15:44:21.646 INFO:tasks.workunit.client.0.smithi028.stdout:2/869: rename da/d24/d2c/c36 to da/d24/c12e 0 2024-07-03T15:44:21.646 INFO:tasks.workunit.client.0.smithi028.stdout:2/870: chown da/d1e/d56/dc0/f102 71 1 2024-07-03T15:44:21.646 INFO:tasks.workunit.client.0.smithi028.stdout:2/871: write da/d42/f4d [650594,24673] 0 2024-07-03T15:44:21.646 INFO:tasks.workunit.client.0.smithi028.stdout:2/872: dread - da/d1e/dec/f11e zero size 2024-07-03T15:44:21.646 INFO:tasks.workunit.client.0.smithi028.stdout:2/873: fsync da/d1e/fc9 0 2024-07-03T15:44:21.646 INFO:tasks.workunit.client.0.smithi028.stdout:2/874: stat da/d24/l5a 0 2024-07-03T15:44:21.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:21 smithi028 ceph-mon[29148]: pgmap v12: 65 pgs: 65 active+clean; 2.7 GiB data, 9.4 GiB used, 527 GiB / 536 GiB avail; 162 MiB/s rd, 204 MiB/s wr, 354 op/s 2024-07-03T15:44:21.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:21 smithi148 ceph-mon[36056]: pgmap v12: 65 pgs: 65 active+clean; 2.7 GiB data, 9.4 GiB used, 527 GiB / 536 GiB avail; 162 MiB/s rd, 204 MiB/s wr, 354 op/s 2024-07-03T15:44:22.009 INFO:tasks.workunit.client.0.smithi028.stdout:7/836: sync 2024-07-03T15:44:22.011 INFO:tasks.workunit.client.0.smithi028.stdout:7/837: link d9/d1c/d45/db5/d92/d99/daf/dc4/de2/fe3 d9/d1c/d45/db5/f103 0 2024-07-03T15:44:22.013 INFO:tasks.workunit.client.0.smithi028.stdout:7/838: symlink d9/d1c/l104 0 2024-07-03T15:44:22.013 INFO:tasks.workunit.client.0.smithi028.stdout:7/839: creat d9/d1c/d6e/f105 x:0 0 0 2024-07-03T15:44:22.014 INFO:tasks.workunit.client.0.smithi028.stdout:7/840: creat d9/d1c/da9/f106 x:0 0 0 2024-07-03T15:44:22.014 INFO:tasks.workunit.client.0.smithi028.stdout:7/841: mknod d9/d1c/d86/c107 0 2024-07-03T15:44:22.016 INFO:tasks.workunit.client.0.smithi028.stdout:7/842: rmdir d9/d1c/d45/db5/d92/de4 39 2024-07-03T15:44:22.016 INFO:tasks.workunit.client.0.smithi028.stdout:7/843: chown d9/d1c/d45/db5/d92/lde 991096 1 2024-07-03T15:44:22.016 INFO:tasks.workunit.client.0.smithi028.stdout:7/844: truncate d9/d1c/d45/db5/d92/de4/ff8 24868 0 2024-07-03T15:44:22.025 INFO:tasks.workunit.client.0.smithi028.stdout:7/845: write d9/d1c/d6e/fbc [1959137,15133] 0 2024-07-03T15:44:22.025 INFO:tasks.workunit.client.0.smithi028.stdout:7/846: creat d9/d1c/d7c/d8b/da0/f108 x:0 0 0 2024-07-03T15:44:22.028 INFO:tasks.workunit.client.0.smithi028.stdout:7/847: rename d9/d1c/d45/db5/f4f to d9/d1c/d45/db5/d92/d9e/f109 0 2024-07-03T15:44:22.046 INFO:tasks.workunit.client.0.smithi028.stdout:7/848: dwrite d9/d1c/d45/d71/fac [0,4194304] 0 2024-07-03T15:44:22.048 INFO:tasks.workunit.client.0.smithi028.stdout:7/849: dread d9/d1c/d6e/f8c [0,4194304] 0 2024-07-03T15:44:22.054 INFO:tasks.workunit.client.0.smithi028.stdout:7/850: mkdir d9/d1c/d7c/d8b/d10a 0 2024-07-03T15:44:22.055 INFO:tasks.workunit.client.0.smithi028.stdout:7/851: mkdir d9/d1c/d45/db5/d92/d99/d10b 0 2024-07-03T15:44:22.055 INFO:tasks.workunit.client.0.smithi028.stdout:7/852: creat d9/d1c/d45/ddb/f10c x:0 0 0 2024-07-03T15:44:22.055 INFO:tasks.workunit.client.0.smithi028.stdout:7/853: chown d9/d1c/d6e/f8c 181711 1 2024-07-03T15:44:22.055 INFO:tasks.workunit.client.0.smithi028.stdout:7/854: mkdir d9/d1c/da9/df0/d10d 0 2024-07-03T15:44:22.056 INFO:tasks.workunit.client.0.smithi028.stdout:7/855: unlink d9/c5b 0 2024-07-03T15:44:22.058 INFO:tasks.workunit.client.0.smithi028.stdout:7/856: truncate d9/d1c/d45/db5/d92/d99/d6a/f7d 292783 0 2024-07-03T15:44:22.058 INFO:tasks.workunit.client.0.smithi028.stdout:7/857: getdents d9/d1c/d7c 0 2024-07-03T15:44:22.067 INFO:tasks.workunit.client.0.smithi028.stdout:7/858: dread d9/d1c/d6e/f76 [4194304,4194304] 0 2024-07-03T15:44:22.068 INFO:tasks.workunit.client.0.smithi028.stdout:7/859: symlink d9/d1c/d45/d71/d78/de0/l10e 0 2024-07-03T15:44:22.072 INFO:tasks.workunit.client.0.smithi028.stdout:2/875: sync 2024-07-03T15:44:22.072 INFO:tasks.workunit.client.0.smithi028.stdout:2/876: creat da/d1e/dec/f12f x:0 0 0 2024-07-03T15:44:22.076 INFO:tasks.workunit.client.0.smithi028.stdout:2/877: mknod da/d74/d111/c130 0 2024-07-03T15:44:22.076 INFO:tasks.workunit.client.0.smithi028.stdout:2/878: write da/d42/fa6 [4410062,54019] 0 2024-07-03T15:44:22.079 INFO:tasks.workunit.client.0.smithi028.stdout:2/879: symlink da/d1e/d56/dc0/l131 0 2024-07-03T15:44:22.088 INFO:tasks.workunit.client.0.smithi028.stdout:2/880: write da/d23/d6f/fae [4046251,16190] 0 2024-07-03T15:44:22.089 INFO:tasks.workunit.client.0.smithi028.stdout:2/881: write da/d42/d47/fde [103281,57972] 0 2024-07-03T15:44:22.089 INFO:tasks.workunit.client.0.smithi028.stdout:2/882: fdatasync f4 0 2024-07-03T15:44:22.089 INFO:tasks.workunit.client.0.smithi028.stdout:2/883: write da/d24/f107 [954220,26476] 0 2024-07-03T15:44:22.090 INFO:tasks.workunit.client.0.smithi028.stdout:7/860: dwrite d9/d1c/d45/db5/d92/d99/d6a/f7b [4194304,4194304] 0 2024-07-03T15:44:22.090 INFO:tasks.workunit.client.0.smithi028.stdout:7/861: creat d9/d1c/d45/db5/d5d/f10f x:0 0 0 2024-07-03T15:44:22.090 INFO:tasks.workunit.client.0.smithi028.stdout:7/862: chown d9/d1c/d86 3 1 2024-07-03T15:44:22.093 INFO:tasks.workunit.client.0.smithi028.stdout:7/863: truncate d9/fa 1004427 0 2024-07-03T15:44:22.094 INFO:tasks.workunit.client.0.smithi028.stdout:7/864: creat d9/d1c/d45/db5/d92/d99/d6a/f110 x:0 0 0 2024-07-03T15:44:22.095 INFO:tasks.workunit.client.0.smithi028.stdout:7/865: write d9/d1c/d86/fbe [731970,48557] 0 2024-07-03T15:44:22.096 INFO:tasks.workunit.client.0.smithi028.stdout:7/866: symlink d9/d1c/d45/d71/d78/de0/l111 0 2024-07-03T15:44:22.096 INFO:tasks.workunit.client.0.smithi028.stdout:7/867: stat d9/d1c/d45/d71 0 2024-07-03T15:44:22.096 INFO:tasks.workunit.client.0.smithi028.stdout:7/868: write d9/d1c/f4b [4912261,106411] 0 2024-07-03T15:44:22.096 INFO:tasks.workunit.client.0.smithi028.stdout:7/869: chown d9/d1c/l54 214154 1 2024-07-03T15:44:22.096 INFO:tasks.workunit.client.0.smithi028.stdout:7/870: mknod d9/d1c/d7c/d8b/da0/c112 0 2024-07-03T15:44:22.096 INFO:tasks.workunit.client.0.smithi028.stdout:7/871: readlink d9/d1c/d45/d71/l88 0 2024-07-03T15:44:22.108 INFO:tasks.workunit.client.0.smithi028.stdout:2/884: dwrite da/d42/d9e/fa0 [0,4194304] 0 2024-07-03T15:44:22.108 INFO:tasks.workunit.client.0.smithi028.stdout:2/885: write da/f85 [2656246,51734] 0 2024-07-03T15:44:22.111 INFO:tasks.workunit.client.0.smithi028.stdout:7/872: dwrite d9/d1c/d6e/f8a [0,4194304] 0 2024-07-03T15:44:22.111 INFO:tasks.workunit.client.0.smithi028.stdout:7/873: chown d9/d1c/d45/db5/d92/d99/daf/fef 0 1 2024-07-03T15:44:22.111 INFO:tasks.workunit.client.0.smithi028.stdout:2/886: symlink da/d74/l132 0 2024-07-03T15:44:22.115 INFO:tasks.workunit.client.0.smithi028.stdout:7/874: rename f6 to d9/d1c/d45/db5/d92/d99/daf/f113 0 2024-07-03T15:44:22.117 INFO:tasks.workunit.client.0.smithi028.stdout:7/875: symlink d9/d1c/da9/df0/l114 0 2024-07-03T15:44:22.117 INFO:tasks.workunit.client.0.smithi028.stdout:7/876: chown d9/d1c/d45/d71/l88 19029597 1 2024-07-03T15:44:22.119 INFO:tasks.workunit.client.0.smithi028.stdout:2/887: rename da/d1e/c2f to da/d12a/c133 0 2024-07-03T15:44:22.120 INFO:tasks.workunit.client.0.smithi028.stdout:7/877: truncate d9/d1c/d45/db5/f44 2383323 0 2024-07-03T15:44:22.144 INFO:tasks.workunit.client.0.smithi028.stdout:2/888: dwrite da/d23/d6f/d84/fca [0,4194304] 0 2024-07-03T15:44:22.147 INFO:tasks.workunit.client.0.smithi028.stdout:7/878: dwrite d9/d1c/d45/db5/d92/d99/f58 [0,4194304] 0 2024-07-03T15:44:22.149 INFO:tasks.workunit.client.0.smithi028.stdout:2/889: symlink da/d42/d47/l134 0 2024-07-03T15:44:22.149 INFO:tasks.workunit.client.0.smithi028.stdout:2/890: creat da/d42/f135 x:0 0 0 2024-07-03T15:44:22.149 INFO:tasks.workunit.client.0.smithi028.stdout:2/891: write da/d1e/f58 [1886984,54052] 0 2024-07-03T15:44:22.149 INFO:tasks.workunit.client.0.smithi028.stdout:2/892: creat da/d23/d6f/d84/d129/f136 x:0 0 0 2024-07-03T15:44:22.150 INFO:tasks.workunit.client.0.smithi028.stdout:2/893: fsync da/d42/d47/de8/fd2 0 2024-07-03T15:44:22.150 INFO:tasks.workunit.client.0.smithi028.stdout:2/894: fsync da/f10 0 2024-07-03T15:44:22.156 INFO:tasks.workunit.client.0.smithi028.stdout:7/879: rmdir d9/d1c/d45/ddb 39 2024-07-03T15:44:22.156 INFO:tasks.workunit.client.0.smithi028.stdout:7/880: read - d9/f22 zero size 2024-07-03T15:44:22.157 INFO:tasks.workunit.client.0.smithi028.stdout:7/881: read d9/d1c/d45/db5/f31 [331245,129135] 0 2024-07-03T15:44:22.157 INFO:tasks.workunit.client.0.smithi028.stdout:7/882: creat d9/d1c/da9/f115 x:0 0 0 2024-07-03T15:44:22.444 INFO:tasks.workunit.client.0.smithi028.stdout:2/895: sync 2024-07-03T15:44:22.445 INFO:tasks.workunit.client.0.smithi028.stdout:2/896: mknod da/d24/d2c/dc6/c137 0 2024-07-03T15:44:22.445 INFO:tasks.workunit.client.0.smithi028.stdout:2/897: stat da/d24/f66 0 2024-07-03T15:44:22.445 INFO:tasks.workunit.client.0.smithi028.stdout:2/898: chown da/d1e/f119 7363 1 2024-07-03T15:44:22.446 INFO:tasks.workunit.client.0.smithi028.stdout:2/899: creat da/d42/d47/de8/f138 x:0 0 0 2024-07-03T15:44:22.467 INFO:tasks.workunit.client.0.smithi028.stdout:2/900: dwrite da/d23/f3b [0,4194304] 0 2024-07-03T15:44:22.467 INFO:tasks.workunit.client.0.smithi028.stdout:2/901: chown da/fc4 2852263 1 2024-07-03T15:44:22.469 INFO:tasks.workunit.client.0.smithi028.stdout:2/902: read da/d42/d47/f81 [173012,52796] 0 2024-07-03T15:44:22.470 INFO:tasks.workunit.client.0.smithi028.stdout:2/903: rename da/d1e/f45 to da/d24/d2c/dc6/f139 0 2024-07-03T15:44:22.470 INFO:tasks.workunit.client.0.smithi028.stdout:2/904: creat da/d42/f13a x:0 0 0 2024-07-03T15:44:22.474 INFO:tasks.workunit.client.0.smithi028.stdout:2/905: symlink da/d1e/l13b 0 2024-07-03T15:44:22.474 INFO:tasks.workunit.client.0.smithi028.stdout:2/906: chown da/d1e/dec/f12f 179 1 2024-07-03T15:44:22.488 INFO:tasks.workunit.client.0.smithi028.stdout:2/907: dwrite da/d24/d2c/dc6/f139 [0,4194304] 0 2024-07-03T15:44:22.488 INFO:tasks.workunit.client.0.smithi028.stdout:2/908: creat da/d42/f13c x:0 0 0 2024-07-03T15:44:22.496 INFO:tasks.workunit.client.0.smithi028.stdout:2/909: write da/d24/f98 [1664267,91799] 0 2024-07-03T15:44:22.498 INFO:tasks.workunit.client.0.smithi028.stdout:2/910: getdents da/d42/d47/da2 0 2024-07-03T15:44:22.501 INFO:tasks.workunit.client.0.smithi028.stdout:2/911: getdents da/d1e/d56 0 2024-07-03T15:44:22.503 INFO:tasks.workunit.client.0.smithi028.stdout:2/912: mknod da/d24/c13d 0 2024-07-03T15:44:22.505 INFO:tasks.workunit.client.0.smithi028.stdout:2/913: rename da/d42/d47/d78/c95 to da/d23/c13e 0 2024-07-03T15:44:22.506 INFO:tasks.workunit.client.0.smithi028.stdout:2/914: rename da/d1e/d68/lbd to da/d1e/d56/l13f 0 2024-07-03T15:44:22.506 INFO:tasks.workunit.client.0.smithi028.stdout:2/915: fsync da/d23/d6f/fae 0 2024-07-03T15:44:22.506 INFO:tasks.workunit.client.0.smithi028.stdout:2/916: fdatasync da/d42/f46 0 2024-07-03T15:44:22.507 INFO:tasks.workunit.client.0.smithi028.stdout:2/917: symlink da/d23/l140 0 2024-07-03T15:44:22.508 INFO:tasks.workunit.client.0.smithi028.stdout:2/918: symlink da/d24/d103/l141 0 2024-07-03T15:44:22.509 INFO:tasks.workunit.client.0.smithi028.stdout:2/919: truncate da/d42/f13c 538063 0 2024-07-03T15:44:22.509 INFO:tasks.workunit.client.0.smithi028.stdout:2/920: creat da/d42/d47/daa/f142 x:0 0 0 2024-07-03T15:44:22.510 INFO:tasks.workunit.client.0.smithi028.stdout:2/921: dread - da/d1e/dec/f12f zero size 2024-07-03T15:44:22.510 INFO:tasks.workunit.client.0.smithi028.stdout:2/922: stat da/d24/d2c/cbb 0 2024-07-03T15:44:22.510 INFO:tasks.workunit.client.0.smithi028.stdout:2/923: chown da/d42/d47/de8/l40 8335 1 2024-07-03T15:44:22.510 INFO:tasks.workunit.client.0.smithi028.stdout:2/924: stat da/f15 0 2024-07-03T15:44:22.510 INFO:tasks.workunit.client.0.smithi028.stdout:2/925: mknod da/d1e/dec/c143 0 2024-07-03T15:44:22.511 INFO:tasks.workunit.client.0.smithi028.stdout:7/883: sync 2024-07-03T15:44:22.511 INFO:tasks.workunit.client.0.smithi028.stdout:7/884: stat d9/d1c/d45/db5/d92/d9e 0 2024-07-03T15:44:22.512 INFO:tasks.workunit.client.0.smithi028.stdout:7/885: dread d9/d1c/f20 [0,4194304] 0 2024-07-03T15:44:22.515 INFO:tasks.workunit.client.0.smithi028.stdout:7/886: dread d9/d1c/d6e/fe6 [0,4194304] 0 2024-07-03T15:44:22.517 INFO:tasks.workunit.client.0.smithi028.stdout:7/887: rename d9/d1c/d6e/f76 to d9/d1c/d45/db5/d92/d9e/dda/f116 0 2024-07-03T15:44:22.517 INFO:tasks.workunit.client.0.smithi028.stdout:7/888: readlink d9/d1c/d45/db5/lf5 0 2024-07-03T15:44:22.517 INFO:tasks.workunit.client.0.smithi028.stdout:7/889: chown d9/d1c/d45/db5/d92/d99/c59 743722 1 2024-07-03T15:44:22.517 INFO:tasks.workunit.client.0.smithi028.stdout:7/890: symlink d9/d1c/d45/ddb/l117 0 2024-07-03T15:44:22.518 INFO:tasks.workunit.client.0.smithi028.stdout:7/891: creat d9/d1c/f118 x:0 0 0 2024-07-03T15:44:22.519 INFO:tasks.workunit.client.0.smithi028.stdout:7/892: mkdir d9/d1c/d7c/d8b/d119 0 2024-07-03T15:44:22.520 INFO:tasks.workunit.client.0.smithi028.stdout:7/893: rmdir d9/d1c/da9/df0 39 2024-07-03T15:44:22.521 INFO:tasks.workunit.client.0.smithi028.stdout:7/894: mkdir d9/d1c/d45/ddb/d11a 0 2024-07-03T15:44:22.522 INFO:tasks.workunit.client.0.smithi028.stdout:7/895: creat d9/d1c/d45/db5/d92/d99/daf/dc4/f11b x:0 0 0 2024-07-03T15:44:22.523 INFO:tasks.workunit.client.0.smithi028.stdout:7/896: symlink d9/d1c/d7c/l11c 0 2024-07-03T15:44:22.524 INFO:tasks.workunit.client.0.smithi028.stdout:7/897: symlink d9/d1c/d86/l11d 0 2024-07-03T15:44:22.527 INFO:tasks.workunit.client.0.smithi028.stdout:2/926: dwrite da/d42/f128 [4194304,4194304] 0 2024-07-03T15:44:22.529 INFO:tasks.workunit.client.0.smithi028.stdout:2/927: mkdir da/d42/d47/daa/d144 0 2024-07-03T15:44:22.529 INFO:tasks.workunit.client.0.smithi028.stdout:2/928: readlink da/d42/d9e/le0 0 2024-07-03T15:44:22.534 INFO:tasks.workunit.client.0.smithi028.stdout:2/929: read da/d74/f8d [3489449,90174] 0 2024-07-03T15:44:22.536 INFO:tasks.workunit.client.0.smithi028.stdout:2/930: mkdir da/d24/d2c/dc6/df0/d145 0 2024-07-03T15:44:22.536 INFO:tasks.workunit.client.0.smithi028.stdout:7/898: dwrite d9/d1c/d45/db5/d92/ff4 [0,4194304] 0 2024-07-03T15:44:22.543 INFO:tasks.workunit.client.0.smithi028.stdout:2/931: rmdir da/d23/d6f 39 2024-07-03T15:44:22.543 INFO:tasks.workunit.client.0.smithi028.stdout:2/932: write da/d1e/dec/f11e [28694,14284] 0 2024-07-03T15:44:22.545 INFO:tasks.workunit.client.0.smithi028.stdout:2/933: creat da/d24/d2c/dc6/f146 x:0 0 0 2024-07-03T15:44:22.552 INFO:tasks.workunit.client.0.smithi028.stdout:7/899: dwrite d9/d1c/d45/db5/d92/d99/daf/dc4/fd9 [0,4194304] 0 2024-07-03T15:44:22.552 INFO:tasks.workunit.client.0.smithi028.stdout:7/900: read d9/d1c/f3f [896587,23809] 0 2024-07-03T15:44:22.557 INFO:tasks.workunit.client.0.smithi028.stdout:2/934: dread da/d42/f65 [0,4194304] 0 2024-07-03T15:44:22.562 INFO:tasks.workunit.client.0.smithi028.stdout:7/901: write d9/d1c/d45/db5/d92/d9e/fa1 [1271313,95896] 0 2024-07-03T15:44:22.562 INFO:tasks.workunit.client.0.smithi028.stdout:7/902: truncate d9/d1c/f6b 4264193 0 2024-07-03T15:44:22.568 INFO:tasks.workunit.client.0.smithi028.stdout:7/903: rename d9/d1c/d45/db5/f77 to d9/d1c/d45/db5/d5d/f11e 0 2024-07-03T15:44:22.570 INFO:tasks.workunit.client.0.smithi028.stdout:7/904: creat d9/d1c/d45/f11f x:0 0 0 2024-07-03T15:44:22.584 INFO:tasks.workunit.client.0.smithi028.stdout:7/905: dwrite d9/d1c/f93 [0,4194304] 0 2024-07-03T15:44:22.584 INFO:tasks.workunit.client.0.smithi028.stdout:7/906: fsync d9/d1c/d45/db5/d5d/fc8 0 2024-07-03T15:44:22.592 INFO:tasks.workunit.client.0.smithi028.stdout:7/907: dread d9/d1c/d45/db5/d92/d9e/f109 [0,4194304] 0 2024-07-03T15:44:22.595 INFO:tasks.workunit.client.0.smithi028.stdout:7/908: creat d9/d1c/d45/db5/dfc/f120 x:0 0 0 2024-07-03T15:44:22.595 INFO:tasks.workunit.client.0.smithi028.stdout:7/909: write d9/d1c/d45/db5/d92/d99/daf/f113 [901362,91248] 0 2024-07-03T15:44:22.595 INFO:tasks.workunit.client.0.smithi028.stdout:7/910: fsync d9/d1c/f5a 0 2024-07-03T15:44:22.597 INFO:tasks.workunit.client.0.smithi028.stdout:7/911: mkdir d9/d1c/d45/ddb/d121 0 2024-07-03T15:44:22.597 INFO:tasks.workunit.client.0.smithi028.stdout:7/912: fdatasync d9/f41 0 2024-07-03T15:44:22.597 INFO:tasks.workunit.client.0.smithi028.stdout:7/913: creat d9/d1c/d45/db5/dfc/f122 x:0 0 0 2024-07-03T15:44:22.597 INFO:tasks.workunit.client.0.smithi028.stdout:7/914: write d9/d1c/d45/db5/f31 [1396893,54503] 0 2024-07-03T15:44:22.642 INFO:tasks.workunit.client.0.smithi028.stdout:7/915: dwrite d9/d1c/d45/db5/d92/d99/d6a/f110 [0,4194304] 0 2024-07-03T15:44:22.642 INFO:tasks.workunit.client.0.smithi028.stdout:7/916: dread - d9/d1c/f118 zero size 2024-07-03T15:44:22.643 INFO:tasks.workunit.client.0.smithi028.stdout:7/917: truncate d9/d1c/d45/db5/d92/d99/daf/dc4/fd1 711197 0 2024-07-03T15:44:22.643 INFO:tasks.workunit.client.0.smithi028.stdout:7/918: write d9/d1c/d45/d71/f7f [4807094,105816] 0 2024-07-03T15:44:22.643 INFO:tasks.workunit.client.0.smithi028.stdout:7/919: creat d9/d1c/d7c/d8b/f123 x:0 0 0 2024-07-03T15:44:22.643 INFO:tasks.workunit.client.0.smithi028.stdout:7/920: truncate d9/d1c/d45/ddb/f10c 256093 0 2024-07-03T15:44:22.643 INFO:tasks.workunit.client.0.smithi028.stdout:7/921: symlink d9/d1c/d45/db5/d92/d9e/l124 0 2024-07-03T15:44:22.645 INFO:tasks.workunit.client.0.smithi028.stdout:7/922: truncate d9/d1c/d45/db5/f44 72920 0 2024-07-03T15:44:22.657 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-07-03T15:44:22.658 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-07-03T15:44:22.662 INFO:tasks.workunit.client.0.smithi028.stdout:7/923: dwrite d9/d1c/f118 [0,4194304] 0 2024-07-03T15:44:22.662 INFO:tasks.workunit.client.0.smithi028.stdout:7/924: chown d9/d1c/d86/fbe 115 1 2024-07-03T15:44:22.667 INFO:tasks.workunit.client.0.smithi028.stdout:7/925: rename d9/d1c/d45/d71/df2/cf7 to d9/d1c/d45/db5/d92/d9e/dda/c125 0 2024-07-03T15:44:22.679 INFO:tasks.workunit.client.0.smithi028.stdout:7/926: dwrite d9/d1c/d45/db5/d5d/f10f [0,4194304] 0 2024-07-03T15:44:22.682 INFO:tasks.workunit.client.0.smithi028.stdout:7/927: dread d9/d1c/d86/fbe [0,4194304] 0 2024-07-03T15:44:22.682 INFO:tasks.workunit.client.0.smithi028.stdout:7/928: dread - d9/d1c/d7c/d8b/da0/f108 zero size 2024-07-03T15:44:22.693 INFO:tasks.workunit.client.0.smithi028.stdout:7/929: dwrite d9/d1c/d45/db5/f42 [0,4194304] 0 2024-07-03T15:44:22.699 INFO:tasks.workunit.client.0.smithi028.stdout:7/930: write d9/f25 [2765723,100152] 0 2024-07-03T15:44:22.699 INFO:tasks.workunit.client.0.smithi028.stdout:7/931: chown d9/d1c/d6e/c9f 130659 1 2024-07-03T15:44:22.700 INFO:tasks.workunit.client.0.smithi028.stdout:7/932: write d9/d1c/d6e/f105 [755849,30906] 0 2024-07-03T15:44:22.718 INFO:tasks.workunit.client.0.smithi028.stdout:7/933: dwrite d9/d1c/d45/db5/d92/d99/daf/f113 [0,4194304] 0 2024-07-03T15:44:22.718 INFO:tasks.workunit.client.0.smithi028.stdout:7/934: creat d9/d1c/d45/db5/d92/d9e/f126 x:0 0 0 2024-07-03T15:44:22.719 INFO:tasks.workunit.client.0.smithi028.stdout:7/935: creat d9/d1c/d45/db5/d92/d99/f127 x:0 0 0 2024-07-03T15:44:22.719 INFO:tasks.workunit.client.0.smithi028.stdout:7/936: write d9/d1c/d6e/fbf [718528,104402] 0 2024-07-03T15:44:22.719 INFO:tasks.workunit.client.0.smithi028.stdout:7/937: creat d9/d1c/d45/db5/dfc/f128 x:0 0 0 2024-07-03T15:44:22.733 INFO:tasks.workunit.client.0.smithi028.stdout:7/938: dwrite d9/d1c/d45/db5/d92/ff4 [0,4194304] 0 2024-07-03T15:44:22.747 INFO:tasks.workunit.client.0.smithi028.stdout:7/939: dwrite d9/d1c/d45/db5/d92/d99/d6a/f110 [4194304,4194304] 0 2024-07-03T15:44:22.747 INFO:tasks.workunit.client.0.smithi028.stdout:7/940: write d9/d1c/d45/db5/d5d/fc8 [892555,45304] 0 2024-07-03T15:44:22.749 INFO:tasks.workunit.client.0.smithi028.stdout:7/941: mknod d9/c129 0 2024-07-03T15:44:22.750 INFO:tasks.workunit.client.0.smithi028.stdout:7/942: symlink d9/d1c/d45/db5/d92/d9e/l12a 0 2024-07-03T15:44:22.750 INFO:tasks.workunit.client.0.smithi028.stdout:7/943: read - d9/d1c/fc9 zero size 2024-07-03T15:44:22.750 INFO:tasks.workunit.client.0.smithi028.stdout:7/944: readlink d9/ld 0 2024-07-03T15:44:22.752 INFO:tasks.workunit.client.0.smithi028.stdout:7/945: getdents d9/d1c/d45/db5/d92/d99/daf 0 2024-07-03T15:44:22.752 INFO:tasks.workunit.client.0.smithi028.stdout:7/946: creat d9/d1c/d86/f12b x:0 0 0 2024-07-03T15:44:22.752 INFO:tasks.workunit.client.0.smithi028.stdout:7/947: write d9/f25 [1049663,37906] 0 2024-07-03T15:44:22.753 INFO:tasks.workunit.client.0.smithi028.stdout:7/948: rename d9/d1c to d9/d1c/d12c 22 2024-07-03T15:44:22.754 INFO:tasks.workunit.client.0.smithi028.stdout:7/949: write d9/d1c/f20 [175958,63476] 0 2024-07-03T15:44:22.763 INFO:tasks.workunit.client.0.smithi028.stdout:7/950: read d9/d1c/d45/d71/d78/f9a [327535,108258] 0 2024-07-03T15:44:22.764 INFO:tasks.workunit.client.0.smithi028.stdout:7/951: readlink d9/d1c/d45/db5/d92/d99/daf/dc4/de2/le5 0 2024-07-03T15:44:22.764 INFO:tasks.workunit.client.0.smithi028.stdout:7/952: symlink d9/d1c/da9/l12d 0 2024-07-03T15:44:22.780 INFO:tasks.workunit.client.0.smithi028.stdout:7/953: dwrite d9/d1c/d45/f57 [0,4194304] 0 2024-07-03T15:44:22.791 INFO:tasks.workunit.client.0.smithi028.stdout:7/954: dwrite d9/d1c/d45/db5/d92/d99/f127 [0,4194304] 0 2024-07-03T15:44:22.791 INFO:tasks.workunit.client.0.smithi028.stdout:7/955: creat d9/d1c/d45/db5/d92/d99/daf/dc4/f12e x:0 0 0 2024-07-03T15:44:22.804 INFO:tasks.workunit.client.0.smithi028.stdout:7/956: dread d9/d1c/d45/db5/d5d/f11e [0,4194304] 0 2024-07-03T15:44:22.804 INFO:tasks.workunit.client.0.smithi028.stdout:7/957: creat d9/d1c/d45/db5/dfc/f12f x:0 0 0 2024-07-03T15:44:22.805 INFO:tasks.workunit.client.0.smithi028.stdout:7/958: dread - d9/d1c/d6e/ff6 zero size 2024-07-03T15:44:22.805 INFO:tasks.workunit.client.0.smithi028.stdout:7/959: write d9/d1c/d45/db5/f103 [740671,121579] 0 2024-07-03T15:44:22.811 INFO:tasks.workunit.client.0.smithi028.stdout:7/960: dread d9/d1c/d45/db5/d92/ff4 [0,4194304] 0 2024-07-03T15:44:22.816 INFO:tasks.workunit.client.0.smithi028.stdout:7/961: getdents d9/d1c/d45/db5/dfc 0 2024-07-03T15:44:22.816 INFO:tasks.workunit.client.0.smithi028.stdout:7/962: readlink d9/d1c/d45/l50 0 2024-07-03T15:44:22.840 INFO:tasks.workunit.client.0.smithi028.stdout:7/963: dwrite d9/d1c/d6e/f105 [0,4194304] 0 2024-07-03T15:44:22.851 INFO:tasks.workunit.client.0.smithi028.stdout:7/964: write d9/d1c/d45/db5/d92/d99/d6a/f7b [7669993,116139] 0 2024-07-03T15:44:22.861 INFO:tasks.workunit.client.0.smithi028.stdout:7/965: write d9/d1c/d45/db5/d5d/f11e [2074331,104418] 0 2024-07-03T15:44:22.862 INFO:tasks.workunit.client.0.smithi028.stdout:7/966: mkdir d9/d1c/d45/ddb/d11a/d130 0 2024-07-03T15:44:22.871 INFO:tasks.workunit.client.0.smithi028.stdout:7/967: dread d9/d1c/d45/db5/d92/d99/daf/dc4/de2/fe3 [0,4194304] 0 2024-07-03T15:44:22.871 INFO:tasks.workunit.client.0.smithi028.stdout:7/968: creat d9/d1c/d45/db5/f131 x:0 0 0 2024-07-03T15:44:22.871 INFO:tasks.workunit.client.0.smithi028.stdout:7/969: stat d9/d1c/d45/db5/d92/d99/d6a/c6d 0 2024-07-03T15:44:22.872 INFO:tasks.workunit.client.0.smithi028.stdout:7/970: creat d9/d1c/d7c/f132 x:0 0 0 2024-07-03T15:44:22.873 INFO:tasks.workunit.client.0.smithi028.stdout:7/971: chown d9/fa 167973 1 2024-07-03T15:44:22.874 INFO:tasks.workunit.client.0.smithi028.stdout:7/972: fsync d9/d1c/d45/d71/f7f 0 2024-07-03T15:44:22.885 INFO:tasks.workunit.client.0.smithi028.stdout:7/973: write d9/d1c/d45/f57 [1000519,5375] 0 2024-07-03T15:44:22.885 INFO:tasks.workunit.client.0.smithi028.stdout:7/974: dread - d9/d1c/d86/f12b zero size 2024-07-03T15:44:22.885 INFO:tasks.workunit.client.0.smithi028.stdout:7/975: dread - d9/f22 zero size 2024-07-03T15:44:22.885 INFO:tasks.workunit.client.0.smithi028.stdout:7/976: creat d9/d1c/d86/f133 x:0 0 0 2024-07-03T15:44:22.904 INFO:tasks.workunit.client.0.smithi028.stdout:7/977: dwrite d9/d1c/f5a [0,4194304] 0 2024-07-03T15:44:22.904 INFO:tasks.workunit.client.0.smithi028.stdout:7/978: write d9/d1c/d86/fb7 [1194996,1228] 0 2024-07-03T15:44:22.904 INFO:tasks.workunit.client.0.smithi028.stdout:7/979: chown d9/d1c/d45/d71/d78/de0/l10e 2 1 2024-07-03T15:44:22.904 INFO:tasks.workunit.client.0.smithi028.stdout:7/980: fdatasync d9/d1c/d45/db5/d92/d99/daf/dc4/fd1 0 2024-07-03T15:44:22.904 INFO:tasks.workunit.client.0.smithi028.stdout:7/981: write d9/d1c/d45/db5/f3e [3705755,27472] 0 2024-07-03T15:44:22.921 INFO:tasks.workunit.client.0.smithi028.stdout:7/982: dwrite d9/f1f [0,4194304] 0 2024-07-03T15:44:22.923 INFO:tasks.workunit.client.0.smithi028.stdout:7/983: rename d9/d1c/d45/db5/d5d/fc8 to d9/d1c/d45/ddb/d121/f134 0 2024-07-03T15:44:22.923 INFO:tasks.workunit.client.0.smithi028.stdout:7/984: truncate d9/d1c/d45/ddb/f10c 701654 0 2024-07-03T15:44:22.937 INFO:tasks.workunit.client.0.smithi028.stdout:7/985: dwrite d9/d1c/d45/d71/f7f [4194304,4194304] 0 2024-07-03T15:44:22.939 INFO:tasks.workunit.client.0.smithi028.stdout:7/986: rename d9/d1c/d45/db5/f42 to d9/d1c/da9/f135 0 2024-07-03T15:44:22.943 INFO:tasks.workunit.client.0.smithi028.stdout:7/987: rename d9/d1c/d45/d71 to d9/d1c/d45/db5/d92/d99/daf/d136 0 2024-07-03T15:44:22.958 INFO:tasks.workunit.client.0.smithi028.stdout:7/988: dwrite d9/fa3 [0,4194304] 0 2024-07-03T15:44:22.965 INFO:tasks.workunit.client.0.smithi028.stdout:7/989: unlink d9/d1c/d45/db5/c47 0 2024-07-03T15:44:22.969 INFO:tasks.workunit.client.0.smithi028.stdout:2/935: sync 2024-07-03T15:44:22.971 INFO:tasks.workunit.client.0.smithi028.stdout:2/936: creat da/d23/d6f/f147 x:0 0 0 2024-07-03T15:44:22.980 INFO:tasks.workunit.client.0.smithi028.stdout:7/990: dwrite d9/d1c/d45/db5/f81 [0,4194304] 0 2024-07-03T15:44:22.993 INFO:tasks.workunit.client.0.smithi028.stdout:2/937: dwrite da/d24/d2c/f97 [4194304,4194304] 0 2024-07-03T15:44:22.993 INFO:tasks.workunit.client.0.smithi028.stdout:2/938: fdatasync da/d42/d47/de8/f3e 0 2024-07-03T15:44:22.998 INFO:tasks.workunit.client.0.smithi028.stdout:2/939: rmdir da/d74 39 2024-07-03T15:44:22.999 INFO:tasks.workunit.client.0.smithi028.stdout:2/940: rename da/d24/d2c/dc6/f108 to da/d23/d6f/f148 0 2024-07-03T15:44:23.000 INFO:tasks.workunit.client.0.smithi028.stdout:7/991: dwrite d9/d1c/d6e/fe6 [0,4194304] 0 2024-07-03T15:44:23.000 INFO:tasks.workunit.client.0.smithi028.stdout:7/992: dread - d9/d1c/d7c/d8b/dd7/fdc zero size 2024-07-03T15:44:23.001 INFO:tasks.workunit.client.0.smithi028.stdout:2/941: symlink da/d42/d47/daa/l149 0 2024-07-03T15:44:23.010 INFO:tasks.workunit.client.0.smithi028.stdout:7/993: dread d9/d1c/d45/db5/d92/d99/daf/d136/d78/fbb [0,4194304] 0 2024-07-03T15:44:23.010 INFO:tasks.workunit.client.0.smithi028.stdout:7/994: creat d9/d1c/d45/db5/d92/d99/daf/dc4/f137 x:0 0 0 2024-07-03T15:44:23.013 INFO:tasks.workunit.client.0.smithi028.stdout:2/942: rmdir da/d23/d115 0 2024-07-03T15:44:23.013 INFO:tasks.workunit.client.0.smithi028.stdout:2/943: write da/d1e/dd0/f105 [757707,83126] 0 2024-07-03T15:44:23.013 INFO:tasks.workunit.client.0.smithi028.stdout:2/944: write da/d1e/f58 [4397615,8861] 0 2024-07-03T15:44:23.016 INFO:tasks.workunit.client.0.smithi028.stdout:2/945: truncate da/d74/f90 2679314 0 2024-07-03T15:44:23.016 INFO:tasks.workunit.client.0.smithi028.stdout:2/946: truncate da/d42/f10f 911074 0 2024-07-03T15:44:23.018 INFO:tasks.workunit.client.0.smithi028.stdout:7/995: dread d9/d1c/d86/fe1 [0,4194304] 0 2024-07-03T15:44:23.018 INFO:tasks.workunit.client.0.smithi028.stdout:7/996: readlink d9/d1c/d45/db5/d92/lde 0 2024-07-03T15:44:23.043 INFO:tasks.workunit.client.0.smithi028.stdout:7/997: dwrite d9/d1c/da9/f106 [0,4194304] 0 2024-07-03T15:44:23.044 INFO:tasks.workunit.client.0.smithi028.stdout:2/947: dwrite da/d42/d47/de8/f138 [0,4194304] 0 2024-07-03T15:44:23.048 INFO:tasks.workunit.client.0.smithi028.stdout:7/998: symlink d9/d1c/da9/df0/l138 0 2024-07-03T15:44:23.054 INFO:tasks.workunit.client.0.smithi028.stdout:2/948: creat da/d23/d6f/d11b/f14a x:0 0 0 2024-07-03T15:44:23.055 INFO:tasks.workunit.client.0.smithi028.stdout:2/949: symlink da/d42/d47/da2/l14b 0 2024-07-03T15:44:23.057 INFO:tasks.workunit.client.0.smithi028.stdout:2/950: rename da/d1e/d68/c73 to da/d127/c14c 0 2024-07-03T15:44:23.058 INFO:tasks.workunit.client.0.smithi028.stdout:7/999: read d9/f3a [4072907,9482] 0 2024-07-03T15:44:23.058 INFO:tasks.workunit.client.0.smithi028.stdout:2/951: rename da/f11a to da/d23/f14d 0 2024-07-03T15:44:23.060 INFO:tasks.workunit.client.0.smithi028.stdout:2/952: getdents da/d1e/dec 0 2024-07-03T15:44:23.081 INFO:tasks.workunit.client.0.smithi028.stdout:2/953: dwrite da/d23/d6f/d84/d129/f136 [0,4194304] 0 2024-07-03T15:44:23.415 INFO:tasks.workunit.client.0.smithi028.stdout:2/954: sync 2024-07-03T15:44:23.417 INFO:tasks.workunit.client.0.smithi028.stdout:2/955: symlink da/d12a/l14e 0 2024-07-03T15:44:23.420 INFO:tasks.workunit.client.0.smithi028.stdout:2/956: mkdir da/d42/d47/daa/d14f 0 2024-07-03T15:44:23.422 INFO:tasks.workunit.client.0.smithi028.stdout:2/957: rename da/d74/l109 to da/d1e/d56/dc0/l150 0 2024-07-03T15:44:23.423 INFO:tasks.workunit.client.0.smithi028.stdout:2/958: getdents da/d24/d103 0 2024-07-03T15:44:23.431 INFO:tasks.workunit.client.0.smithi028.stdout:2/959: write da/d42/d47/d78/fb1 [8674563,3965] 0 2024-07-03T15:44:23.440 INFO:tasks.workunit.client.0.smithi028.stdout:2/960: dread da/f15 [0,4194304] 0 2024-07-03T15:44:23.440 INFO:tasks.workunit.client.0.smithi028.stdout:2/961: creat da/d42/d9e/f151 x:0 0 0 2024-07-03T15:44:23.441 INFO:tasks.workunit.client.0.smithi028.stdout:2/962: write da/d42/d47/de8/f8b [218903,87344] 0 2024-07-03T15:44:23.447 INFO:tasks.workunit.client.0.smithi028.stdout:2/963: dread da/d24/f66 [0,4194304] 0 2024-07-03T15:44:23.450 INFO:tasks.workunit.client.0.smithi028.stdout:2/964: rename da/d42/d47/da2/f118 to da/d127/f152 0 2024-07-03T15:44:23.450 INFO:tasks.workunit.client.0.smithi028.stdout:2/965: creat da/d1e/d68/f153 x:0 0 0 2024-07-03T15:44:23.450 INFO:tasks.workunit.client.0.smithi028.stdout:2/966: write da/fc4 [199571,66772] 0 2024-07-03T15:44:23.450 INFO:tasks.workunit.client.0.smithi028.stdout:2/967: write da/d23/d6f/d84/d129/f12d [877027,18452] 0 2024-07-03T15:44:23.452 INFO:tasks.workunit.client.0.smithi028.stdout:2/968: stat da/d24/c12e 0 2024-07-03T15:44:23.454 INFO:tasks.workunit.client.0.smithi028.stdout:2/969: link da/d42/f10f da/d42/d47/daa/d144/f154 0 2024-07-03T15:44:23.461 INFO:tasks.workunit.client.0.smithi028.stdout:2/970: dread da/d24/d2c/f3d [0,4194304] 0 2024-07-03T15:44:23.464 INFO:tasks.workunit.client.0.smithi028.stdout:2/971: getdents da/d24/d2c/dba 0 2024-07-03T15:44:23.465 INFO:tasks.workunit.client.0.smithi028.stdout:2/972: mkdir da/d42/d47/daa/d155 0 2024-07-03T15:44:23.468 INFO:tasks.workunit.client.0.smithi028.stdout:2/973: dread da/d42/d47/d78/f9d [0,4194304] 0 2024-07-03T15:44:23.468 INFO:tasks.workunit.client.0.smithi028.stdout:2/974: chown da/d1e/d68/l9f 161513975 1 2024-07-03T15:44:23.468 INFO:tasks.workunit.client.0.smithi028.stdout:2/975: chown da/d24/f98 172527505 1 2024-07-03T15:44:23.470 INFO:tasks.workunit.client.0.smithi028.stdout:2/976: rename da/d23/d6f/d84/fa7 to da/d42/d47/daa/d144/f156 0 2024-07-03T15:44:23.477 INFO:tasks.workunit.client.0.smithi028.stdout:2/977: dread da/d24/d2c/f61 [0,4194304] 0 2024-07-03T15:44:23.478 INFO:tasks.workunit.client.0.smithi028.stdout:2/978: mknod da/d42/d47/daa/c157 0 2024-07-03T15:44:23.478 INFO:tasks.workunit.client.0.smithi028.stdout:2/979: fdatasync da/d42/d47/d78/f9d 0 2024-07-03T15:44:23.479 INFO:tasks.workunit.client.0.smithi028.stdout:2/980: creat da/d12a/f158 x:0 0 0 2024-07-03T15:44:23.480 INFO:tasks.workunit.client.0.smithi028.stdout:2/981: creat da/d42/d47/da2/f159 x:0 0 0 2024-07-03T15:44:23.482 INFO:tasks.workunit.client.0.smithi028.stdout:2/982: rename da/d42/d9e/cd7 to da/d12a/c15a 0 2024-07-03T15:44:23.508 INFO:tasks.workunit.client.0.smithi028.stdout:2/983: dwrite da/d42/d47/daa/d144/f154 [0,4194304] 0 2024-07-03T15:44:23.511 INFO:tasks.workunit.client.0.smithi028.stdout:2/984: rename da/d24/f55 to da/d42/d47/daa/d144/f15b 0 2024-07-03T15:44:23.511 INFO:tasks.workunit.client.0.smithi028.stdout:2/985: fdatasync da/d1e/d56/dc0/f10e 0 2024-07-03T15:44:23.511 INFO:tasks.workunit.client.0.smithi028.stdout:2/986: fdatasync da/d42/d47/daa/f142 0 2024-07-03T15:44:23.512 INFO:tasks.workunit.client.0.smithi028.stdout:2/987: rmdir da/d23/d6f/d84 39 2024-07-03T15:44:23.512 INFO:tasks.workunit.client.0.smithi028.stdout:2/988: write da/d1e/dd0/fdc [327245,106013] 0 2024-07-03T15:44:23.512 INFO:tasks.workunit.client.0.smithi028.stdout:2/989: dread - da/d1e/f119 zero size 2024-07-03T15:44:23.518 INFO:tasks.workunit.client.0.smithi028.stdout:2/990: truncate da/d23/d6f/d84/d129/f12d 322683 0 2024-07-03T15:44:23.520 INFO:tasks.workunit.client.0.smithi028.stdout:2/991: symlink da/d1e/dec/l15c 0 2024-07-03T15:44:23.526 INFO:tasks.workunit.client.0.smithi028.stdout:2/992: dread da/d24/d2c/faf [0,4194304] 0 2024-07-03T15:44:23.529 INFO:tasks.workunit.client.0.smithi028.stdout:2/993: mkdir da/d42/d47/d15d 0 2024-07-03T15:44:23.529 INFO:tasks.workunit.client.0.smithi028.stdout:2/994: write da/d1e/d56/dc0/ff4 [947548,43234] 0 2024-07-03T15:44:23.529 INFO:tasks.workunit.client.0.smithi028.stdout:2/995: truncate da/d42/d47/da2/fab 203496 0 2024-07-03T15:44:23.530 INFO:tasks.workunit.client.0.smithi028.stdout:2/996: symlink da/d42/d9e/l15e 0 2024-07-03T15:44:23.547 INFO:tasks.workunit.client.0.smithi028.stdout:2/997: dwrite da/d1e/d56/f124 [0,4194304] 0 2024-07-03T15:44:23.550 INFO:tasks.workunit.client.0.smithi028.stdout:2/998: dread da/d1e/dd0/fdc [0,4194304] 0 2024-07-03T15:44:23.553 INFO:tasks.workunit.client.0.smithi028.stdout:2/999: rename da/d1e/d56/l13f to da/d23/d6f/l15f 0 2024-07-03T15:44:23.556 INFO:tasks.workunit.client.0.smithi028.stderr:+ rm -rf -- ./tmp.K6T0EhjVPM 2024-07-03T15:44:23.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:23 smithi028 ceph-mon[29148]: pgmap v13: 65 pgs: 65 active+clean; 2.7 GiB data, 9.4 GiB used, 527 GiB / 536 GiB avail; 109 MiB/s rd, 135 MiB/s wr, 270 op/s 2024-07-03T15:44:23.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:23 smithi148 ceph-mon[36056]: pgmap v13: 65 pgs: 65 active+clean; 2.7 GiB data, 9.4 GiB used, 527 GiB / 536 GiB avail; 109 MiB/s rd, 135 MiB/s wr, 270 op/s 2024-07-03T15:44:26.014 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:25 smithi148 ceph-mon[36056]: pgmap v14: 65 pgs: 65 active+clean; 2.3 GiB data, 8.8 GiB used, 528 GiB / 536 GiB avail; 149 MiB/s rd, 188 MiB/s wr, 396 op/s 2024-07-03T15:44:26.014 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:25 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:26.014 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:25 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:26.015 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:25 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:26.015 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:25 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:26.015 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:25 smithi148 ceph-mon[36056]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:44:26.015 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:25 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:44:26.015 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:25 smithi148 ceph-mon[36056]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:44:26.015 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:25 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:44:26.015 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:25 smithi148 ceph-mon[36056]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:44:26.015 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:25 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:44:26.015 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:25 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:26.015 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:25 smithi148 ceph-mon[36056]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:44:26.015 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:25 smithi148 ceph-mon[36056]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:44:26.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:25 smithi028 ceph-mon[29148]: pgmap v14: 65 pgs: 65 active+clean; 2.3 GiB data, 8.8 GiB used, 528 GiB / 536 GiB avail; 149 MiB/s rd, 188 MiB/s wr, 396 op/s 2024-07-03T15:44:26.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:25 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:26.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:25 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:26.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:25 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:26.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:25 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:26.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:25 smithi028 ceph-mon[29148]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:44:26.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:25 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:44:26.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:25 smithi028 ceph-mon[29148]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:44:26.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:25 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:44:26.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:25 smithi028 ceph-mon[29148]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:44:26.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:25 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-03T15:44:26.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:25 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:26.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:25 smithi028 ceph-mon[29148]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:44:26.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:25 smithi028 ceph-mon[29148]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:44:27.086 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:26 smithi028 ceph-mon[29148]: Adjusting osd_memory_target on smithi028 to 1558M 2024-07-03T15:44:27.086 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:26 smithi028 ceph-mon[29148]: Updating smithi028:/etc/ceph/ceph.conf 2024-07-03T15:44:27.086 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:26 smithi028 ceph-mon[29148]: Updating smithi148:/etc/ceph/ceph.conf 2024-07-03T15:44:27.086 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:26 smithi028 ceph-mon[29148]: Updating smithi148:/var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-03T15:44:27.087 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:26 smithi028 ceph-mon[29148]: Updating smithi028:/var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-03T15:44:27.087 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:26 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:27.087 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:26 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:27.087 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:26 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:27.087 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:26 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:27.087 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:26 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:27.087 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:26 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:27.087 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:26 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:27.087 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:26 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:27.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:26 smithi148 ceph-mon[36056]: Adjusting osd_memory_target on smithi028 to 1558M 2024-07-03T15:44:27.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:26 smithi148 ceph-mon[36056]: Updating smithi028:/etc/ceph/ceph.conf 2024-07-03T15:44:27.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:26 smithi148 ceph-mon[36056]: Updating smithi148:/etc/ceph/ceph.conf 2024-07-03T15:44:27.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:26 smithi148 ceph-mon[36056]: Updating smithi148:/var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-03T15:44:27.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:26 smithi148 ceph-mon[36056]: Updating smithi028:/var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-03T15:44:27.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:26 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:27.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:26 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:27.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:26 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:27.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:26 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:27.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:26 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:27.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:26 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:27.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:26 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:27.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:26 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:28.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:27 smithi028 ceph-mon[29148]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-07-03T15:44:28.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:27 smithi028 ceph-mon[29148]: Updating smithi028:/etc/ceph/ceph.client.admin.keyring 2024-07-03T15:44:28.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:27 smithi028 ceph-mon[29148]: Updating smithi148:/var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/config/ceph.client.admin.keyring 2024-07-03T15:44:28.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:27 smithi028 ceph-mon[29148]: Updating smithi028:/var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/config/ceph.client.admin.keyring 2024-07-03T15:44:28.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:27 smithi028 ceph-mon[29148]: pgmap v15: 65 pgs: 65 active+clean; 2.3 GiB data, 8.8 GiB used, 528 GiB / 536 GiB avail; 95 MiB/s rd, 117 MiB/s wr, 252 op/s 2024-07-03T15:44:28.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:27 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:28.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:27 smithi028 ceph-mon[29148]: Reconfiguring node-exporter.smithi028 (dependencies changed)... 2024-07-03T15:44:28.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:27 smithi028 ceph-mon[29148]: Reconfiguring daemon node-exporter.smithi028 on smithi028 2024-07-03T15:44:28.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:27 smithi148 ceph-mon[36056]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-07-03T15:44:28.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:27 smithi148 ceph-mon[36056]: Updating smithi028:/etc/ceph/ceph.client.admin.keyring 2024-07-03T15:44:28.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:27 smithi148 ceph-mon[36056]: Updating smithi148:/var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/config/ceph.client.admin.keyring 2024-07-03T15:44:28.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:27 smithi148 ceph-mon[36056]: Updating smithi028:/var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/config/ceph.client.admin.keyring 2024-07-03T15:44:28.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:27 smithi148 ceph-mon[36056]: pgmap v15: 65 pgs: 65 active+clean; 2.3 GiB data, 8.8 GiB used, 528 GiB / 536 GiB avail; 95 MiB/s rd, 117 MiB/s wr, 252 op/s 2024-07-03T15:44:28.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:27 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:28.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:27 smithi148 ceph-mon[36056]: Reconfiguring node-exporter.smithi028 (dependencies changed)... 2024-07-03T15:44:28.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:27 smithi148 ceph-mon[36056]: Reconfiguring daemon node-exporter.smithi028 on smithi028 2024-07-03T15:44:30.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:30 smithi028 ceph-mon[29148]: pgmap v16: 65 pgs: 65 active+clean; 2.2 GiB data, 8.6 GiB used, 528 GiB / 536 GiB avail; 96 MiB/s rd, 117 MiB/s wr, 278 op/s 2024-07-03T15:44:30.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:30 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:30.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:30 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:30.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:30 smithi028 ceph-mon[29148]: Reconfiguring alertmanager.smithi028 (dependencies changed)... 2024-07-03T15:44:30.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:30 smithi028 ceph-mon[29148]: Reconfiguring daemon alertmanager.smithi028 on smithi028 2024-07-03T15:44:30.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:30 smithi148 ceph-mon[36056]: pgmap v16: 65 pgs: 65 active+clean; 2.2 GiB data, 8.6 GiB used, 528 GiB / 536 GiB avail; 96 MiB/s rd, 117 MiB/s wr, 278 op/s 2024-07-03T15:44:30.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:30 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:30.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:30 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:30.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:30 smithi148 ceph-mon[36056]: Reconfiguring alertmanager.smithi028 (dependencies changed)... 2024-07-03T15:44:30.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:30 smithi148 ceph-mon[36056]: Reconfiguring daemon alertmanager.smithi028 on smithi028 2024-07-03T15:44:32.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:32 smithi028 ceph-mon[29148]: pgmap v17: 65 pgs: 65 active+clean; 1.8 GiB data, 7.5 GiB used, 529 GiB / 536 GiB avail; 83 MiB/s rd, 103 MiB/s wr, 369 op/s 2024-07-03T15:44:32.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:32 smithi148 ceph-mon[36056]: pgmap v17: 65 pgs: 65 active+clean; 1.8 GiB data, 7.5 GiB used, 529 GiB / 536 GiB avail; 83 MiB/s rd, 103 MiB/s wr, 369 op/s 2024-07-03T15:44:33.265 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:33 smithi028 ceph-mon[29148]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:44:33.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:33 smithi148 ceph-mon[36056]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:44:34.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:34 smithi028 ceph-mon[29148]: pgmap v18: 65 pgs: 65 active+clean; 1.8 GiB data, 7.5 GiB used, 529 GiB / 536 GiB avail; 42 MiB/s rd, 55 MiB/s wr, 258 op/s 2024-07-03T15:44:34.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:34 smithi148 ceph-mon[36056]: pgmap v18: 65 pgs: 65 active+clean; 1.8 GiB data, 7.5 GiB used, 529 GiB / 536 GiB avail; 42 MiB/s rd, 55 MiB/s wr, 258 op/s 2024-07-03T15:44:35.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:35 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:35.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:35 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:35.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:35 smithi148 ceph-mon[36056]: Reconfiguring grafana.smithi028 (dependencies changed)... 2024-07-03T15:44:35.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:35 smithi148 ceph-mon[36056]: pgmap v19: 65 pgs: 65 active+clean; 1.4 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 43 MiB/s rd, 56 MiB/s wr, 326 op/s 2024-07-03T15:44:35.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:35 smithi148 ceph-mon[36056]: Reconfiguring daemon grafana.smithi028 on smithi028 2024-07-03T15:44:35.841 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:35 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:35.841 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:35 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:35.841 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:35 smithi028 ceph-mon[29148]: Reconfiguring grafana.smithi028 (dependencies changed)... 2024-07-03T15:44:35.841 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:35 smithi028 ceph-mon[29148]: pgmap v19: 65 pgs: 65 active+clean; 1.4 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 43 MiB/s rd, 56 MiB/s wr, 326 op/s 2024-07-03T15:44:35.841 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:35 smithi028 ceph-mon[29148]: Reconfiguring daemon grafana.smithi028 on smithi028 2024-07-03T15:44:37.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:37 smithi028 ceph-mon[29148]: pgmap v20: 65 pgs: 65 active+clean; 1.4 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 200 op/s 2024-07-03T15:44:37.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:37 smithi148 ceph-mon[36056]: pgmap v20: 65 pgs: 65 active+clean; 1.4 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 200 op/s 2024-07-03T15:44:39.845 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:39 smithi028 ceph-mon[29148]: pgmap v21: 65 pgs: 65 active+clean; 1.3 GiB data, 6.1 GiB used, 530 GiB / 536 GiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 247 op/s 2024-07-03T15:44:39.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:39 smithi148 ceph-mon[36056]: pgmap v21: 65 pgs: 65 active+clean; 1.3 GiB data, 6.1 GiB used, 530 GiB / 536 GiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 247 op/s 2024-07-03T15:44:40.962 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:40 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:40.963 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:40 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:40.963 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:40 smithi028 ceph-mon[29148]: Reconfiguring prometheus.smithi028 (dependencies changed)... 2024-07-03T15:44:40.963 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:40 smithi028 ceph-mon[29148]: Reconfiguring daemon prometheus.smithi028 on smithi028 2024-07-03T15:44:40.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:40 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:40.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:40 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:40.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:40 smithi148 ceph-mon[36056]: Reconfiguring prometheus.smithi028 (dependencies changed)... 2024-07-03T15:44:40.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:40 smithi148 ceph-mon[36056]: Reconfiguring daemon prometheus.smithi028 on smithi028 2024-07-03T15:44:41.853 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:41 smithi028 ceph-mon[29148]: pgmap v22: 65 pgs: 65 active+clean; 957 MiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 316 op/s 2024-07-03T15:44:41.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:41 smithi148 ceph-mon[36056]: pgmap v22: 65 pgs: 65 active+clean; 957 MiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 316 op/s 2024-07-03T15:44:42.713 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-07-03T15:44:42.714 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-07-03T15:44:43.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:43 smithi028 ceph-mon[29148]: pgmap v23: 65 pgs: 65 active+clean; 957 MiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 209 op/s 2024-07-03T15:44:43.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:43 smithi148 ceph-mon[36056]: pgmap v23: 65 pgs: 65 active+clean; 957 MiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 209 op/s 2024-07-03T15:44:45.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:45 smithi148 ceph-mon[36056]: pgmap v24: 65 pgs: 65 active+clean; 494 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 348 op/s 2024-07-03T15:44:46.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:45 smithi028 ceph-mon[29148]: pgmap v24: 65 pgs: 65 active+clean; 494 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 348 op/s 2024-07-03T15:44:47.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:47 smithi148 ceph-mon[36056]: pgmap v25: 65 pgs: 65 active+clean; 494 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 280 op/s 2024-07-03T15:44:48.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:47 smithi028 ceph-mon[29148]: pgmap v25: 65 pgs: 65 active+clean; 494 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 280 op/s 2024-07-03T15:44:48.737 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:48 smithi028 ceph-mon[29148]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:44:48.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:48 smithi148 ceph-mon[36056]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:44:49.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.524+0000 7f849c21c640 1 -- 172.21.15.28:0/2448622362 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f84940fc330 msgr2=0x7f84940fc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:49.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.524+0000 7f849c21c640 1 --2- 172.21.15.28:0/2448622362 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f84940fc330 0x7f84940fc740 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f84900048d0 tx=0x7f8490009da0 comp rx=0 tx=0).stop 2024-07-03T15:44:49.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.524+0000 7f849c21c640 1 -- 172.21.15.28:0/2448622362 shutdown_connections 2024-07-03T15:44:49.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.524+0000 7f849c21c640 1 --2- 172.21.15.28:0/2448622362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84940fd600 0x7f84940fda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:49.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.524+0000 7f849c21c640 1 --2- 172.21.15.28:0/2448622362 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f84940fc330 0x7f84940fc740 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:49.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.524+0000 7f849c21c640 1 -- 172.21.15.28:0/2448622362 >> 172.21.15.28:0/2448622362 conn(0x7f84940f7af0 msgr2=0x7f84940f9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:44:49.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.524+0000 7f849c21c640 1 -- 172.21.15.28:0/2448622362 shutdown_connections 2024-07-03T15:44:49.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.524+0000 7f849c21c640 1 -- 172.21.15.28:0/2448622362 wait complete. 2024-07-03T15:44:49.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.525+0000 7f849c21c640 1 Processor -- start 2024-07-03T15:44:49.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.525+0000 7f849c21c640 1 -- start start 2024-07-03T15:44:49.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.525+0000 7f849c21c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f84940fc330 0x7f8494071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:44:49.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.525+0000 7f849c21c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84940fd600 0x7f849406f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:44:49.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.525+0000 7f849c21c640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849406fe00 con 0x7f84940fc330 2024-07-03T15:44:49.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.525+0000 7f849c21c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849406ff40 con 0x7f84940fd600 2024-07-03T15:44:49.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.525+0000 7f8499fb8640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f84940fc330 0x7f8494071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:44:49.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.525+0000 7f8499fb8640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f84940fc330 0x7f8494071380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38148/0 (socket says 172.21.15.28:38148) 2024-07-03T15:44:49.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.525+0000 7f8499fb8640 1 -- 172.21.15.28:0/990665874 learned_addr learned my addr 172.21.15.28:0/990665874 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:44:49.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.525+0000 7f84997b7640 1 --2- 172.21.15.28:0/990665874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84940fd600 0x7f849406f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:44:49.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.526+0000 7f8499fb8640 1 -- 172.21.15.28:0/990665874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84940fd600 msgr2=0x7f849406f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:49.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.526+0000 7f8499fb8640 1 --2- 172.21.15.28:0/990665874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84940fd600 0x7f849406f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:49.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.526+0000 7f8499fb8640 1 -- 172.21.15.28:0/990665874 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8490005040 con 0x7f84940fc330 2024-07-03T15:44:49.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.526+0000 7f8499fb8640 1 --2- 172.21.15.28:0/990665874 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f84940fc330 0x7f8494071380 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f84900048a0 tx=0x7f8490009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:44:49.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.526+0000 7f848a7fc640 1 -- 172.21.15.28:0/990665874 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8490038070 con 0x7f84940fc330 2024-07-03T15:44:49.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.526+0000 7f848a7fc640 1 -- 172.21.15.28:0/990665874 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f84900092b0 con 0x7f84940fc330 2024-07-03T15:44:49.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.526+0000 7f849c21c640 1 -- 172.21.15.28:0/990665874 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84940701c0 con 0x7f84940fc330 2024-07-03T15:44:49.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.526+0000 7f848a7fc640 1 -- 172.21.15.28:0/990665874 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84900336c0 con 0x7f84940fc330 2024-07-03T15:44:49.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.527+0000 7f849c21c640 1 -- 172.21.15.28:0/990665874 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8494114f90 con 0x7f84940fc330 2024-07-03T15:44:49.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.528+0000 7f849c21c640 1 -- 172.21.15.28:0/990665874 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84940fb790 con 0x7f84940fc330 2024-07-03T15:44:49.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.529+0000 7f848a7fc640 1 -- 172.21.15.28:0/990665874 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 24) v1 ==== 88617+0+0 (secure 0 0 0) 0x7f8490030020 con 0x7f84940fc330 2024-07-03T15:44:49.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.529+0000 7f848a7fc640 1 --2- 172.21.15.28:0/990665874 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f848006a7d0 0x7f848006cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:44:49.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.530+0000 7f848a7fc640 1 -- 172.21.15.28:0/990665874 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f84900a8f10 con 0x7f84940fc330 2024-07-03T15:44:49.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.530+0000 7f84997b7640 1 --2- 172.21.15.28:0/990665874 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f848006a7d0 0x7f848006cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:44:49.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.531+0000 7f84997b7640 1 --2- 172.21.15.28:0/990665874 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f848006a7d0 0x7f848006cc80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8484004eb0 tx=0x7f848400b040 comp rx=0 tx=0).ready entity=mgr.24403 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:44:49.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.535+0000 7f848a7fc640 1 -- 172.21.15.28:0/990665874 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7f8490072bc0 con 0x7f84940fc330 2024-07-03T15:44:49.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.790+0000 7f849c21c640 1 -- 172.21.15.28:0/990665874 --> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8494101930 con 0x7f848006a7d0 2024-07-03T15:44:49.794 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:49 smithi028 ceph-mon[29148]: pgmap v26: 65 pgs: 65 active+clean; 463 MiB data, 3.1 GiB used, 533 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 322 op/s 2024-07-03T15:44:49.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.791+0000 7f848a7fc640 1 -- 172.21.15.28:0/990665874 <== mgr.24403 v2:172.21.15.148:6828/4285739175 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f8494101930 con 0x7f848006a7d0 2024-07-03T15:44:49.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.794+0000 7f849c21c640 1 -- 172.21.15.28:0/990665874 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f848006a7d0 msgr2=0x7f848006cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:49.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.794+0000 7f849c21c640 1 --2- 172.21.15.28:0/990665874 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f848006a7d0 0x7f848006cc80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8484004eb0 tx=0x7f848400b040 comp rx=0 tx=0).stop 2024-07-03T15:44:49.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.794+0000 7f849c21c640 1 -- 172.21.15.28:0/990665874 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f84940fc330 msgr2=0x7f8494071380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:49.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.794+0000 7f849c21c640 1 --2- 172.21.15.28:0/990665874 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f84940fc330 0x7f8494071380 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f84900048a0 tx=0x7f8490009da0 comp rx=0 tx=0).stop 2024-07-03T15:44:49.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.794+0000 7f849c21c640 1 -- 172.21.15.28:0/990665874 shutdown_connections 2024-07-03T15:44:49.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.794+0000 7f849c21c640 1 --2- 172.21.15.28:0/990665874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84940fd600 0x7f849406f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:49.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.794+0000 7f849c21c640 1 --2- 172.21.15.28:0/990665874 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f848006a7d0 0x7f848006cc80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:49.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.794+0000 7f849c21c640 1 --2- 172.21.15.28:0/990665874 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f84940fc330 0x7f8494071380 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:49.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.794+0000 7f849c21c640 1 -- 172.21.15.28:0/990665874 >> 172.21.15.28:0/990665874 conn(0x7f84940f7af0 msgr2=0x7f84940f91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:44:49.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.794+0000 7f849c21c640 1 -- 172.21.15.28:0/990665874 shutdown_connections 2024-07-03T15:44:49.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.794+0000 7f849c21c640 1 -- 172.21.15.28:0/990665874 wait complete. 2024-07-03T15:44:49.808 INFO:teuthology.orchestra.run.smithi028.stdout:true 2024-07-03T15:44:49.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.913+0000 7f41f1d48640 1 -- 172.21.15.28:0/3025243194 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41ec0fc330 msgr2=0x7f41ec0fc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:49.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.913+0000 7f41f1d48640 1 --2- 172.21.15.28:0/3025243194 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41ec0fc330 0x7f41ec0fc740 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f41dc0048d0 tx=0x7f41dc009da0 comp rx=0 tx=0).stop 2024-07-03T15:44:49.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.913+0000 7f41f1d48640 1 -- 172.21.15.28:0/3025243194 shutdown_connections 2024-07-03T15:44:49.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.913+0000 7f41f1d48640 1 --2- 172.21.15.28:0/3025243194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41ec0fd600 0x7f41ec0fda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:49.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.913+0000 7f41f1d48640 1 --2- 172.21.15.28:0/3025243194 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41ec0fc330 0x7f41ec0fc740 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:49.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.913+0000 7f41f1d48640 1 -- 172.21.15.28:0/3025243194 >> 172.21.15.28:0/3025243194 conn(0x7f41ec0f7af0 msgr2=0x7f41ec0f9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:44:49.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.914+0000 7f41f1d48640 1 -- 172.21.15.28:0/3025243194 shutdown_connections 2024-07-03T15:44:49.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.914+0000 7f41f1d48640 1 -- 172.21.15.28:0/3025243194 wait complete. 2024-07-03T15:44:49.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.914+0000 7f41f1d48640 1 Processor -- start 2024-07-03T15:44:49.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.914+0000 7f41f1d48640 1 -- start start 2024-07-03T15:44:49.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.914+0000 7f41f1d48640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41ec0fc330 0x7f41ec10f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:44:49.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.914+0000 7f41f1d48640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41ec0fd600 0x7f41ec10f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:44:49.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.915+0000 7f41f1d48640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41ec10fd40 con 0x7f41ec0fc330 2024-07-03T15:44:49.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.915+0000 7f41f1d48640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41ec110560 con 0x7f41ec0fd600 2024-07-03T15:44:49.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.915+0000 7f41eb7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41ec0fc330 0x7f41ec10f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:44:49.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.915+0000 7f41eb7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41ec0fc330 0x7f41ec10f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:60146/0 (socket says 172.21.15.28:60146) 2024-07-03T15:44:49.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.915+0000 7f41eb7fe640 1 -- 172.21.15.28:0/2739885991 learned_addr learned my addr 172.21.15.28:0/2739885991 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:44:49.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.915+0000 7f41eaffd640 1 --2- 172.21.15.28:0/2739885991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41ec0fd600 0x7f41ec10f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:44:49.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.915+0000 7f41eb7fe640 1 -- 172.21.15.28:0/2739885991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41ec0fd600 msgr2=0x7f41ec10f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:49.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.915+0000 7f41eb7fe640 1 --2- 172.21.15.28:0/2739885991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41ec0fd600 0x7f41ec10f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:49.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.915+0000 7f41eb7fe640 1 -- 172.21.15.28:0/2739885991 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41dc005040 con 0x7f41ec0fc330 2024-07-03T15:44:49.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.915+0000 7f41eb7fe640 1 --2- 172.21.15.28:0/2739885991 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41ec0fc330 0x7f41ec10f1b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f41dc0048a0 tx=0x7f41dc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:44:49.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.915+0000 7f41d3fff640 1 -- 172.21.15.28:0/2739885991 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41dc038070 con 0x7f41ec0fc330 2024-07-03T15:44:49.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.915+0000 7f41f1d48640 1 -- 172.21.15.28:0/2739885991 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41ec110790 con 0x7f41ec0fc330 2024-07-03T15:44:49.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.915+0000 7f41d3fff640 1 -- 172.21.15.28:0/2739885991 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f41dc0092b0 con 0x7f41ec0fc330 2024-07-03T15:44:49.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.915+0000 7f41d3fff640 1 -- 172.21.15.28:0/2739885991 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41dc0338a0 con 0x7f41ec0fc330 2024-07-03T15:44:49.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.915+0000 7f41f1d48640 1 -- 172.21.15.28:0/2739885991 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41ec110cd0 con 0x7f41ec0fc330 2024-07-03T15:44:49.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.917+0000 7f41f1d48640 1 -- 172.21.15.28:0/2739885991 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41ec0fb790 con 0x7f41ec0fc330 2024-07-03T15:44:49.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.917+0000 7f41d3fff640 1 -- 172.21.15.28:0/2739885991 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 24) v1 ==== 88617+0+0 (secure 0 0 0) 0x7f41dc030020 con 0x7f41ec0fc330 2024-07-03T15:44:49.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.917+0000 7f41d3fff640 1 --2- 172.21.15.28:0/2739885991 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f41d406a6d0 0x7f41d406cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:44:49.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.918+0000 7f41d3fff640 1 -- 172.21.15.28:0/2739885991 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f41dc0a8d90 con 0x7f41ec0fc330 2024-07-03T15:44:49.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.918+0000 7f41eaffd640 1 --2- 172.21.15.28:0/2739885991 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f41d406a6d0 0x7f41d406cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:44:49.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.918+0000 7f41eaffd640 1 --2- 172.21.15.28:0/2739885991 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f41d406a6d0 0x7f41d406cb80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f41ec10ef60 tx=0x7f41e000b040 comp rx=0 tx=0).ready entity=mgr.24403 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:44:49.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:49.922+0000 7f41d3fff640 1 -- 172.21.15.28:0/2739885991 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7f41dc072a60 con 0x7f41ec0fc330 2024-07-03T15:44:49.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:49 smithi148 ceph-mon[36056]: pgmap v26: 65 pgs: 65 active+clean; 463 MiB data, 3.1 GiB used, 533 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 322 op/s 2024-07-03T15:44:50.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.203+0000 7f41f1d48640 1 -- 172.21.15.28:0/2739885991 --> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f41ec110ef0 con 0x7f41d406a6d0 2024-07-03T15:44:50.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.204+0000 7f41d3fff640 1 -- 172.21.15.28:0/2739885991 <== mgr.24403 v2:172.21.15.148:6828/4285739175 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f41ec110ef0 con 0x7f41d406a6d0 2024-07-03T15:44:50.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.206+0000 7f41f1d48640 1 -- 172.21.15.28:0/2739885991 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f41d406a6d0 msgr2=0x7f41d406cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:50.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.206+0000 7f41f1d48640 1 --2- 172.21.15.28:0/2739885991 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f41d406a6d0 0x7f41d406cb80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f41ec10ef60 tx=0x7f41e000b040 comp rx=0 tx=0).stop 2024-07-03T15:44:50.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.206+0000 7f41f1d48640 1 -- 172.21.15.28:0/2739885991 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41ec0fc330 msgr2=0x7f41ec10f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:50.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.206+0000 7f41f1d48640 1 --2- 172.21.15.28:0/2739885991 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41ec0fc330 0x7f41ec10f1b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f41dc0048a0 tx=0x7f41dc009da0 comp rx=0 tx=0).stop 2024-07-03T15:44:50.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.206+0000 7f41f1d48640 1 -- 172.21.15.28:0/2739885991 shutdown_connections 2024-07-03T15:44:50.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.206+0000 7f41f1d48640 1 --2- 172.21.15.28:0/2739885991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41ec0fd600 0x7f41ec10f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:50.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.206+0000 7f41f1d48640 1 --2- 172.21.15.28:0/2739885991 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f41d406a6d0 0x7f41d406cb80 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:50.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.206+0000 7f41f1d48640 1 --2- 172.21.15.28:0/2739885991 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41ec0fc330 0x7f41ec10f1b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:50.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.206+0000 7f41f1d48640 1 -- 172.21.15.28:0/2739885991 >> 172.21.15.28:0/2739885991 conn(0x7f41ec0f7af0 msgr2=0x7f41ec0f9160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:44:50.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.206+0000 7f41f1d48640 1 -- 172.21.15.28:0/2739885991 shutdown_connections 2024-07-03T15:44:50.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.206+0000 7f41f1d48640 1 -- 172.21.15.28:0/2739885991 wait complete. 2024-07-03T15:44:50.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.309+0000 7f374c51d640 1 -- 172.21.15.28:0/743474406 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37440fd650 msgr2=0x7f37440fdac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:50.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.309+0000 7f374c51d640 1 --2- 172.21.15.28:0/743474406 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37440fd650 0x7f37440fdac0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f37340048d0 tx=0x7f3734009da0 comp rx=0 tx=0).stop 2024-07-03T15:44:50.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.310+0000 7f374c51d640 1 -- 172.21.15.28:0/743474406 shutdown_connections 2024-07-03T15:44:50.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.310+0000 7f374c51d640 1 --2- 172.21.15.28:0/743474406 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37440fd650 0x7f37440fdac0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:50.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.310+0000 7f374c51d640 1 --2- 172.21.15.28:0/743474406 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37440fc380 0x7f37440fc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:50.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.310+0000 7f374c51d640 1 -- 172.21.15.28:0/743474406 >> 172.21.15.28:0/743474406 conn(0x7f37440f7b40 msgr2=0x7f37440f9f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:44:50.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.310+0000 7f374c51d640 1 -- 172.21.15.28:0/743474406 shutdown_connections 2024-07-03T15:44:50.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.310+0000 7f374c51d640 1 -- 172.21.15.28:0/743474406 wait complete. 2024-07-03T15:44:50.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.310+0000 7f374c51d640 1 Processor -- start 2024-07-03T15:44:50.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.311+0000 7f374c51d640 1 -- start start 2024-07-03T15:44:50.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.311+0000 7f374c51d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37440fc380 0x7f374410cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:44:50.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.311+0000 7f374c51d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37440fd650 0x7f374410d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:44:50.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.311+0000 7f374c51d640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374410dab0 con 0x7f37440fd650 2024-07-03T15:44:50.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.311+0000 7f374c51d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374410e2d0 con 0x7f37440fc380 2024-07-03T15:44:50.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.311+0000 7f374a2b9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37440fc380 0x7f374410cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:44:50.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.311+0000 7f3749ab8640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37440fd650 0x7f374410d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:44:50.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.311+0000 7f374a2b9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37440fc380 0x7f374410cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.28:57288/0 (socket says 172.21.15.28:57288) 2024-07-03T15:44:50.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.311+0000 7f374a2b9640 1 -- 172.21.15.28:0/1193483391 learned_addr learned my addr 172.21.15.28:0/1193483391 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:44:50.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.311+0000 7f374a2b9640 1 -- 172.21.15.28:0/1193483391 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37440fd650 msgr2=0x7f374410d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:50.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.311+0000 7f374a2b9640 1 --2- 172.21.15.28:0/1193483391 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37440fd650 0x7f374410d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:50.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.311+0000 7f374a2b9640 1 -- 172.21.15.28:0/1193483391 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3734005040 con 0x7f37440fc380 2024-07-03T15:44:50.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.312+0000 7f374a2b9640 1 --2- 172.21.15.28:0/1193483391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37440fc380 0x7f374410cf20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f374000eb70 tx=0x7f374000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:44:50.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.312+0000 7f373affd640 1 -- 172.21.15.28:0/1193483391 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3740007c30 con 0x7f37440fc380 2024-07-03T15:44:50.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.312+0000 7f373affd640 1 -- 172.21.15.28:0/1193483391 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f374000f040 con 0x7f37440fc380 2024-07-03T15:44:50.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.312+0000 7f374c51d640 1 -- 172.21.15.28:0/1193483391 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f374410e500 con 0x7f37440fc380 2024-07-03T15:44:50.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.312+0000 7f373affd640 1 -- 172.21.15.28:0/1193483391 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37400095f0 con 0x7f37440fc380 2024-07-03T15:44:50.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.312+0000 7f374c51d640 1 -- 172.21.15.28:0/1193483391 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f374410eaa0 con 0x7f37440fc380 2024-07-03T15:44:50.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.313+0000 7f374c51d640 1 -- 172.21.15.28:0/1193483391 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37440fb7e0 con 0x7f37440fc380 2024-07-03T15:44:50.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.313+0000 7f373affd640 1 -- 172.21.15.28:0/1193483391 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 24) v1 ==== 88617+0+0 (secure 0 0 0) 0x7f3740009790 con 0x7f37440fc380 2024-07-03T15:44:50.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.314+0000 7f373affd640 1 --2- 172.21.15.28:0/1193483391 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f373006a6d0 0x7f373006cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:44:50.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.314+0000 7f373affd640 1 -- 172.21.15.28:0/1193483391 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f3740088de0 con 0x7f37440fc380 2024-07-03T15:44:50.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.314+0000 7f3749ab8640 1 --2- 172.21.15.28:0/1193483391 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f373006a6d0 0x7f373006cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:44:50.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.315+0000 7f3749ab8640 1 --2- 172.21.15.28:0/1193483391 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f373006a6d0 0x7f373006cb80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3734007ac0 tx=0x7f3734035040 comp rx=0 tx=0).ready entity=mgr.24403 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:44:50.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.318+0000 7f373affd640 1 -- 172.21.15.28:0/1193483391 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7f3740052900 con 0x7f37440fc380 2024-07-03T15:44:50.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.563+0000 7f374c51d640 1 -- 172.21.15.28:0/1193483391 --> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f374410ee50 con 0x7f373006a6d0 2024-07-03T15:44:50.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.573+0000 7f373affd640 1 -- 172.21.15.28:0/1193483391 <== mgr.24403 v2:172.21.15.148:6828/4285739175 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f374410ee50 con 0x7f373006a6d0 2024-07-03T15:44:50.577 INFO:teuthology.orchestra.run.smithi028.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-03T15:44:50.577 INFO:teuthology.orchestra.run.smithi028.stdout:alertmanager.smithi028 smithi028 *:9093,9094 starting - - - - 2024-07-03T15:44:50.577 INFO:teuthology.orchestra.run.smithi028.stdout:crash.smithi028 smithi028 running (8m) 25s ago 8m 6899k - 17.2.7-1047-gd4b063ab 4ca399d9e65d 535660e448f9 2024-07-03T15:44:50.577 INFO:teuthology.orchestra.run.smithi028.stdout:crash.smithi148 smithi148 running (7m) 42s ago 7m 6899k - 17.2.7-1047-gd4b063ab 4ca399d9e65d 4bb7675895e7 2024-07-03T15:44:50.578 INFO:teuthology.orchestra.run.smithi028.stdout:grafana.smithi028 smithi028 *:3000 starting - - - - 2024-07-03T15:44:50.578 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi028.rhpvve smithi028 running (3m) 25s ago 3m 217M - 17.2.7-1047-gd4b063ab 4ca399d9e65d 3ed5abd02cfb 2024-07-03T15:44:50.578 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi028.wmbxew smithi028 running (3m) 25s ago 3m 139M - 17.2.7-1047-gd4b063ab 4ca399d9e65d b3bf736998f9 2024-07-03T15:44:50.578 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi148.szomfl smithi148 running (2m) 42s ago 2m 13.9M - 17.2.7-1047-gd4b063ab 4ca399d9e65d d94f5c95fa8d 2024-07-03T15:44:50.578 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi148.xgpuld smithi148 running (3m) 42s ago 3m 11.9M - 17.2.7-1047-gd4b063ab 4ca399d9e65d fc70360a8de0 2024-07-03T15:44:50.578 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.smithi028.vtqrtz smithi028 *:9283 running (9m) 25s ago 9m 240M - 17.2.7-1047-gd4b063ab 4ca399d9e65d 0d26e74cb154 2024-07-03T15:44:50.578 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.smithi148.rjzsar smithi148 *:8443,9283 running (63s) 42s ago 7m 477M - 19.0.0-2550-gc00506dc 9d49405603e8 1fe24bbddd1a 2024-07-03T15:44:50.579 INFO:teuthology.orchestra.run.smithi028.stdout:mon.smithi028 smithi028 running (9m) 25s ago 9m 47.6M 2048M 17.2.7-1047-gd4b063ab 4ca399d9e65d 7461f9bef4e0 2024-07-03T15:44:50.579 INFO:teuthology.orchestra.run.smithi028.stdout:mon.smithi148 smithi148 running (7m) 42s ago 7m 37.6M 2048M 17.2.7-1047-gd4b063ab 4ca399d9e65d 4c6c8ed2ab85 2024-07-03T15:44:50.579 INFO:teuthology.orchestra.run.smithi028.stdout:node-exporter.smithi028 smithi028 *:9100 starting - - - - 2024-07-03T15:44:50.579 INFO:teuthology.orchestra.run.smithi028.stdout:node-exporter.smithi148 smithi148 *:9100 running (7m) 42s ago 7m 17.1M - 1.5.0 0da6a335fe13 4994e60bd57c 2024-07-03T15:44:50.579 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0 smithi028 running (6m) 25s ago 6m 235M 1558M 17.2.7-1047-gd4b063ab 4ca399d9e65d b18089a439ba 2024-07-03T15:44:50.579 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1 smithi028 running (6m) 25s ago 6m 244M 1558M 17.2.7-1047-gd4b063ab 4ca399d9e65d 87d415dde231 2024-07-03T15:44:50.579 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2 smithi028 running (5m) 25s ago 5m 206M 1558M 17.2.7-1047-gd4b063ab 4ca399d9e65d b2cbb312212d 2024-07-03T15:44:50.580 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3 smithi148 running (5m) 42s ago 5m 274M 2581M 17.2.7-1047-gd4b063ab 4ca399d9e65d 5be69a27df31 2024-07-03T15:44:50.580 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4 smithi148 running (5m) 42s ago 5m 219M 2581M 17.2.7-1047-gd4b063ab 4ca399d9e65d 54f7c26c77ca 2024-07-03T15:44:50.580 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5 smithi148 running (4m) 42s ago 4m 279M 2581M 17.2.7-1047-gd4b063ab 4ca399d9e65d d185c6905384 2024-07-03T15:44:50.580 INFO:teuthology.orchestra.run.smithi028.stdout:prometheus.smithi028 smithi028 *:9095 running (6m) 25s ago 7m 58.6M - 2.43.0 a07b618ecd1d 97f838e118a3 2024-07-03T15:44:50.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.575+0000 7f374c51d640 1 -- 172.21.15.28:0/1193483391 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f373006a6d0 msgr2=0x7f373006cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:50.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.575+0000 7f374c51d640 1 --2- 172.21.15.28:0/1193483391 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f373006a6d0 0x7f373006cb80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3734007ac0 tx=0x7f3734035040 comp rx=0 tx=0).stop 2024-07-03T15:44:50.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.575+0000 7f374c51d640 1 -- 172.21.15.28:0/1193483391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37440fc380 msgr2=0x7f374410cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:50.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.575+0000 7f374c51d640 1 --2- 172.21.15.28:0/1193483391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37440fc380 0x7f374410cf20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f374000eb70 tx=0x7f374000b040 comp rx=0 tx=0).stop 2024-07-03T15:44:50.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.575+0000 7f374c51d640 1 -- 172.21.15.28:0/1193483391 shutdown_connections 2024-07-03T15:44:50.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.575+0000 7f374c51d640 1 --2- 172.21.15.28:0/1193483391 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f373006a6d0 0x7f373006cb80 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:50.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.575+0000 7f374c51d640 1 --2- 172.21.15.28:0/1193483391 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37440fd650 0x7f374410d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:50.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.575+0000 7f374c51d640 1 --2- 172.21.15.28:0/1193483391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37440fc380 0x7f374410cf20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:50.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.575+0000 7f374c51d640 1 -- 172.21.15.28:0/1193483391 >> 172.21.15.28:0/1193483391 conn(0x7f37440f7b40 msgr2=0x7f37440f91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:44:50.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.575+0000 7f374c51d640 1 -- 172.21.15.28:0/1193483391 shutdown_connections 2024-07-03T15:44:50.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.575+0000 7f374c51d640 1 -- 172.21.15.28:0/1193483391 wait complete. 2024-07-03T15:44:50.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.688+0000 7f9bb44b2640 1 -- 172.21.15.28:0/1214324865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bac06e350 msgr2=0x7f9bac108540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:50.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.688+0000 7f9bb44b2640 1 --2- 172.21.15.28:0/1214324865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bac06e350 0x7f9bac108540 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c0048d0 tx=0x7f9b9c009da0 comp rx=0 tx=0).stop 2024-07-03T15:44:50.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.688+0000 7f9bb44b2640 1 -- 172.21.15.28:0/1214324865 shutdown_connections 2024-07-03T15:44:50.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.688+0000 7f9bb44b2640 1 --2- 172.21.15.28:0/1214324865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bac06e350 0x7f9bac108540 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:50.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.688+0000 7f9bb44b2640 1 --2- 172.21.15.28:0/1214324865 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bac06da00 0x7f9bac06de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:50.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.688+0000 7f9bb44b2640 1 -- 172.21.15.28:0/1214324865 >> 172.21.15.28:0/1214324865 conn(0x7f9bac069370 msgr2=0x7f9bac06b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:44:50.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.688+0000 7f9bb44b2640 1 -- 172.21.15.28:0/1214324865 shutdown_connections 2024-07-03T15:44:50.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.688+0000 7f9bb44b2640 1 -- 172.21.15.28:0/1214324865 wait complete. 2024-07-03T15:44:50.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.689+0000 7f9bb44b2640 1 Processor -- start 2024-07-03T15:44:50.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.689+0000 7f9bb44b2640 1 -- start start 2024-07-03T15:44:50.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.689+0000 7f9bb44b2640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bac06da00 0x7f9bac11bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:44:50.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.689+0000 7f9bb44b2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bac06e350 0x7f9bac11c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:44:50.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.689+0000 7f9bb44b2640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bac11c940 con 0x7f9bac06da00 2024-07-03T15:44:50.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.689+0000 7f9bb44b2640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bac11d080 con 0x7f9bac06e350 2024-07-03T15:44:50.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.690+0000 7f9bb224e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bac06da00 0x7f9bac11bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:44:50.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.690+0000 7f9bb224e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bac06da00 0x7f9bac11bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:60184/0 (socket says 172.21.15.28:60184) 2024-07-03T15:44:50.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.690+0000 7f9bb224e640 1 -- 172.21.15.28:0/404186519 learned_addr learned my addr 172.21.15.28:0/404186519 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:44:50.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.690+0000 7f9bb224e640 1 -- 172.21.15.28:0/404186519 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bac06e350 msgr2=0x7f9bac11c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:50.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.690+0000 7f9bb224e640 1 --2- 172.21.15.28:0/404186519 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bac06e350 0x7f9bac11c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:50.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.690+0000 7f9bb224e640 1 -- 172.21.15.28:0/404186519 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b9c005040 con 0x7f9bac06da00 2024-07-03T15:44:50.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.691+0000 7f9bb224e640 1 --2- 172.21.15.28:0/404186519 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bac06da00 0x7f9bac11bdb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9ba400ecb0 tx=0x7f9ba400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:44:50.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.695+0000 7f9ba2ffd640 1 -- 172.21.15.28:0/404186519 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ba400cb30 con 0x7f9bac06da00 2024-07-03T15:44:50.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.695+0000 7f9ba2ffd640 1 -- 172.21.15.28:0/404186519 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9ba400f040 con 0x7f9bac06da00 2024-07-03T15:44:50.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.695+0000 7f9ba2ffd640 1 -- 172.21.15.28:0/404186519 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ba40097c0 con 0x7f9bac06da00 2024-07-03T15:44:50.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.695+0000 7f9bb44b2640 1 -- 172.21.15.28:0/404186519 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bac11d310 con 0x7f9bac06da00 2024-07-03T15:44:50.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.695+0000 7f9bb44b2640 1 -- 172.21.15.28:0/404186519 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bac11d860 con 0x7f9bac06da00 2024-07-03T15:44:50.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.700+0000 7f9bb44b2640 1 -- 172.21.15.28:0/404186519 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b90005240 con 0x7f9bac06da00 2024-07-03T15:44:50.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.700+0000 7f9ba2ffd640 1 -- 172.21.15.28:0/404186519 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 24) v1 ==== 88617+0+0 (secure 0 0 0) 0x7f9ba4007890 con 0x7f9bac06da00 2024-07-03T15:44:50.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.701+0000 7f9ba2ffd640 1 --2- 172.21.15.28:0/404186519 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f9b9806a8a0 0x7f9b9806cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:44:50.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.701+0000 7f9ba2ffd640 1 -- 172.21.15.28:0/404186519 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f9ba4089730 con 0x7f9bac06da00 2024-07-03T15:44:50.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.701+0000 7f9bb1a4d640 1 --2- 172.21.15.28:0/404186519 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f9b9806a8a0 0x7f9b9806cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:44:50.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.703+0000 7f9bb1a4d640 1 --2- 172.21.15.28:0/404186519 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f9b9806a8a0 0x7f9b9806cd50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c0047f0 tx=0x7f9b9c009da0 comp rx=0 tx=0).ready entity=mgr.24403 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:44:50.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.710+0000 7f9ba2ffd640 1 -- 172.21.15.28:0/404186519 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7f9ba40532a0 con 0x7f9bac06da00 2024-07-03T15:44:50.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:50 smithi028 ceph-mon[29148]: from='client.14674 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:44:50.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:50 smithi148 ceph-mon[36056]: from='client.14674 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:44:50.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.995+0000 7f9bb44b2640 1 -- 172.21.15.28:0/404186519 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9b900057c0 con 0x7f9bac06da00 2024-07-03T15:44:51.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.996+0000 7f9ba2ffd640 1 -- 172.21.15.28:0/404186519 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+849 (secure 0 0 0) 0x7f9ba4056960 con 0x7f9bac06da00 2024-07-03T15:44:51.000 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-07-03T15:44:51.000 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": { 2024-07-03T15:44:51.000 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 2 2024-07-03T15:44:51.000 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:44:51.000 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": { 2024-07-03T15:44:51.000 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 1, 2024-07-03T15:44:51.000 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 1 2024-07-03T15:44:51.000 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:44:51.001 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": { 2024-07-03T15:44:51.001 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 6 2024-07-03T15:44:51.001 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:44:51.001 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": { 2024-07-03T15:44:51.001 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 4 2024-07-03T15:44:51.001 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:44:51.001 INFO:teuthology.orchestra.run.smithi028.stdout: "overall": { 2024-07-03T15:44:51.001 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 13, 2024-07-03T15:44:51.001 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 1 2024-07-03T15:44:51.001 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-07-03T15:44:51.001 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-07-03T15:44:51.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.999+0000 7f9bb44b2640 1 -- 172.21.15.28:0/404186519 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f9b9806a8a0 msgr2=0x7f9b9806cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:51.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.999+0000 7f9bb44b2640 1 --2- 172.21.15.28:0/404186519 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f9b9806a8a0 0x7f9b9806cd50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c0047f0 tx=0x7f9b9c009da0 comp rx=0 tx=0).stop 2024-07-03T15:44:51.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.999+0000 7f9bb44b2640 1 -- 172.21.15.28:0/404186519 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bac06da00 msgr2=0x7f9bac11bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:51.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.999+0000 7f9bb44b2640 1 --2- 172.21.15.28:0/404186519 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bac06da00 0x7f9bac11bdb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9ba400ecb0 tx=0x7f9ba400b040 comp rx=0 tx=0).stop 2024-07-03T15:44:51.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.999+0000 7f9bb44b2640 1 -- 172.21.15.28:0/404186519 shutdown_connections 2024-07-03T15:44:51.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.999+0000 7f9bb44b2640 1 --2- 172.21.15.28:0/404186519 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bac06e350 0x7f9bac11c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:51.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.999+0000 7f9bb44b2640 1 --2- 172.21.15.28:0/404186519 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f9b9806a8a0 0x7f9b9806cd50 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:51.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.999+0000 7f9bb44b2640 1 --2- 172.21.15.28:0/404186519 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bac06da00 0x7f9bac11bdb0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:51.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.999+0000 7f9bb44b2640 1 -- 172.21.15.28:0/404186519 >> 172.21.15.28:0/404186519 conn(0x7f9bac069370 msgr2=0x7f9bac106600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:44:51.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.999+0000 7f9bb44b2640 1 -- 172.21.15.28:0/404186519 shutdown_connections 2024-07-03T15:44:51.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:50.999+0000 7f9bb44b2640 1 -- 172.21.15.28:0/404186519 wait complete. 2024-07-03T15:44:51.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.099+0000 7fe97711b640 1 -- 172.21.15.28:0/930698696 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe9700659f0 msgr2=0x7fe9701013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:51.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.099+0000 7fe97711b640 1 --2- 172.21.15.28:0/930698696 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe9700659f0 0x7fe9701013b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe9640048d0 tx=0x7fe964009da0 comp rx=0 tx=0).stop 2024-07-03T15:44:51.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.099+0000 7fe97711b640 1 -- 172.21.15.28:0/930698696 shutdown_connections 2024-07-03T15:44:51.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.099+0000 7fe97711b640 1 --2- 172.21.15.28:0/930698696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9701018f0 0x7fe970103cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:51.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.099+0000 7fe97711b640 1 --2- 172.21.15.28:0/930698696 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe9700659f0 0x7fe9701013b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:51.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.099+0000 7fe97711b640 1 -- 172.21.15.28:0/930698696 >> 172.21.15.28:0/930698696 conn(0x7fe9700f6a20 msgr2=0x7fe9700f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:44:51.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.099+0000 7fe97711b640 1 -- 172.21.15.28:0/930698696 shutdown_connections 2024-07-03T15:44:51.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.099+0000 7fe97711b640 1 -- 172.21.15.28:0/930698696 wait complete. 2024-07-03T15:44:51.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.100+0000 7fe97711b640 1 Processor -- start 2024-07-03T15:44:51.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.100+0000 7fe97711b640 1 -- start start 2024-07-03T15:44:51.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.100+0000 7fe97711b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9700659f0 0x7fe97010f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:44:51.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.100+0000 7fe97711b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe9701018f0 0x7fe97010f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:44:51.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.100+0000 7fe97711b640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97010fd20 con 0x7fe9701018f0 2024-07-03T15:44:51.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.100+0000 7fe97711b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe970110540 con 0x7fe9700659f0 2024-07-03T15:44:51.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.101+0000 7fe96ffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe9701018f0 0x7fe97010f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:44:51.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.101+0000 7fe96ffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe9701018f0 0x7fe97010f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:60200/0 (socket says 172.21.15.28:60200) 2024-07-03T15:44:51.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.101+0000 7fe96ffff640 1 -- 172.21.15.28:0/3224498373 learned_addr learned my addr 172.21.15.28:0/3224498373 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:44:51.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.101+0000 7fe974eb7640 1 --2- 172.21.15.28:0/3224498373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9700659f0 0x7fe97010f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:44:51.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.101+0000 7fe96ffff640 1 -- 172.21.15.28:0/3224498373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9700659f0 msgr2=0x7fe97010f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:51.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.101+0000 7fe96ffff640 1 --2- 172.21.15.28:0/3224498373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9700659f0 0x7fe97010f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:51.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.101+0000 7fe96ffff640 1 -- 172.21.15.28:0/3224498373 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe964005040 con 0x7fe9701018f0 2024-07-03T15:44:51.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.101+0000 7fe96ffff640 1 --2- 172.21.15.28:0/3224498373 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe9701018f0 0x7fe97010f6d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe95c00eb70 tx=0x7fe95c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:44:51.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.101+0000 7fe96d7fa640 1 -- 172.21.15.28:0/3224498373 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe95c007c30 con 0x7fe9701018f0 2024-07-03T15:44:51.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.101+0000 7fe97711b640 1 -- 172.21.15.28:0/3224498373 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe970110770 con 0x7fe9701018f0 2024-07-03T15:44:51.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.101+0000 7fe97711b640 1 -- 172.21.15.28:0/3224498373 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe970110d10 con 0x7fe9701018f0 2024-07-03T15:44:51.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.101+0000 7fe96d7fa640 1 -- 172.21.15.28:0/3224498373 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe95c00f040 con 0x7fe9701018f0 2024-07-03T15:44:51.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.102+0000 7fe96d7fa640 1 -- 172.21.15.28:0/3224498373 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe95c00cad0 con 0x7fe9701018f0 2024-07-03T15:44:51.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.103+0000 7fe97711b640 1 -- 172.21.15.28:0/3224498373 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe954005240 con 0x7fe9701018f0 2024-07-03T15:44:51.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.103+0000 7fe96d7fa640 1 -- 172.21.15.28:0/3224498373 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 24) v1 ==== 88617+0+0 (secure 0 0 0) 0x7fe95c00cc30 con 0x7fe9701018f0 2024-07-03T15:44:51.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.104+0000 7fe96d7fa640 1 --2- 172.21.15.28:0/3224498373 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fe96006a6d0 0x7fe96006cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:44:51.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.104+0000 7fe96d7fa640 1 -- 172.21.15.28:0/3224498373 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fe95c088b20 con 0x7fe9701018f0 2024-07-03T15:44:51.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.104+0000 7fe974eb7640 1 --2- 172.21.15.28:0/3224498373 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fe96006a6d0 0x7fe96006cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:44:51.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.105+0000 7fe974eb7640 1 --2- 172.21.15.28:0/3224498373 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fe96006a6d0 0x7fe96006cb80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe964009a10 tx=0x7fe964035040 comp rx=0 tx=0).ready entity=mgr.24403 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:44:51.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.108+0000 7fe96d7fa640 1 -- 172.21.15.28:0/3224498373 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7fe95c052760 con 0x7fe9701018f0 2024-07-03T15:44:51.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.422+0000 7fe97711b640 1 -- 172.21.15.28:0/3224498373 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe9540057c0 con 0x7fe9701018f0 2024-07-03T15:44:51.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.423+0000 7fe96d7fa640 1 -- 172.21.15.28:0/3224498373 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1838 (secure 0 0 0) 0x7fe95c019740 con 0x7fe9701018f0 2024-07-03T15:44:51.427 INFO:teuthology.orchestra.run.smithi028.stdout:e17 2024-07-03T15:44:51.427 INFO:teuthology.orchestra.run.smithi028.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-03T15:44:51.427 INFO:teuthology.orchestra.run.smithi028.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:44:51.427 INFO:teuthology.orchestra.run.smithi028.stdout:legacy client fscid: 1 2024-07-03T15:44:51.427 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:44:51.427 INFO:teuthology.orchestra.run.smithi028.stdout:Filesystem 'cephfs' (1) 2024-07-03T15:44:51.427 INFO:teuthology.orchestra.run.smithi028.stdout:fs_name cephfs 2024-07-03T15:44:51.427 INFO:teuthology.orchestra.run.smithi028.stdout:epoch 16 2024-07-03T15:44:51.427 INFO:teuthology.orchestra.run.smithi028.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-03T15:44:51.427 INFO:teuthology.orchestra.run.smithi028.stdout:created 2024-07-03T15:41:40.656298+0000 2024-07-03T15:44:51.427 INFO:teuthology.orchestra.run.smithi028.stdout:modified 2024-07-03T15:41:56.454984+0000 2024-07-03T15:44:51.427 INFO:teuthology.orchestra.run.smithi028.stdout:tableserver 0 2024-07-03T15:44:51.428 INFO:teuthology.orchestra.run.smithi028.stdout:root 0 2024-07-03T15:44:51.428 INFO:teuthology.orchestra.run.smithi028.stdout:session_timeout 60 2024-07-03T15:44:51.428 INFO:teuthology.orchestra.run.smithi028.stdout:session_autoclose 300 2024-07-03T15:44:51.428 INFO:teuthology.orchestra.run.smithi028.stdout:max_file_size 1099511627776 2024-07-03T15:44:51.428 INFO:teuthology.orchestra.run.smithi028.stdout:required_client_features {} 2024-07-03T15:44:51.428 INFO:teuthology.orchestra.run.smithi028.stdout:last_failure 0 2024-07-03T15:44:51.428 INFO:teuthology.orchestra.run.smithi028.stdout:last_failure_osd_epoch 0 2024-07-03T15:44:51.428 INFO:teuthology.orchestra.run.smithi028.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:44:51.428 INFO:teuthology.orchestra.run.smithi028.stdout:max_mds 1 2024-07-03T15:44:51.428 INFO:teuthology.orchestra.run.smithi028.stdout:in 0 2024-07-03T15:44:51.428 INFO:teuthology.orchestra.run.smithi028.stdout:up {0=14450} 2024-07-03T15:44:51.428 INFO:teuthology.orchestra.run.smithi028.stdout:failed 2024-07-03T15:44:51.428 INFO:teuthology.orchestra.run.smithi028.stdout:damaged 2024-07-03T15:44:51.428 INFO:teuthology.orchestra.run.smithi028.stdout:stopped 2024-07-03T15:44:51.429 INFO:teuthology.orchestra.run.smithi028.stdout:data_pools [3] 2024-07-03T15:44:51.429 INFO:teuthology.orchestra.run.smithi028.stdout:metadata_pool 2 2024-07-03T15:44:51.429 INFO:teuthology.orchestra.run.smithi028.stdout:inline_data disabled 2024-07-03T15:44:51.429 INFO:teuthology.orchestra.run.smithi028.stdout:balancer 2024-07-03T15:44:51.429 INFO:teuthology.orchestra.run.smithi028.stdout:standby_count_wanted 1 2024-07-03T15:44:51.429 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi028.rhpvve{0:14450} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.28:6826/1613936042,v1:172.21.15.28:6827/1613936042] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:44:51.429 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi028.wmbxew{0:14460} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.28:6828/1730336821,v1:172.21.15.28:6829/1730336821] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:44:51.429 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:44:51.429 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:44:51.429 INFO:teuthology.orchestra.run.smithi028.stdout:Standby daemons: 2024-07-03T15:44:51.429 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:44:51.429 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi148.szomfl{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/4171231858,v1:172.21.15.148:6827/4171231858] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:44:51.429 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi148.xgpuld{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/2347536398,v1:172.21.15.148:6825/2347536398] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:44:51.430 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 17 2024-07-03T15:44:51.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.425+0000 7fe97711b640 1 -- 172.21.15.28:0/3224498373 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fe96006a6d0 msgr2=0x7fe96006cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:51.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.425+0000 7fe97711b640 1 --2- 172.21.15.28:0/3224498373 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fe96006a6d0 0x7fe96006cb80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe964009a10 tx=0x7fe964035040 comp rx=0 tx=0).stop 2024-07-03T15:44:51.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.425+0000 7fe97711b640 1 -- 172.21.15.28:0/3224498373 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe9701018f0 msgr2=0x7fe97010f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:51.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.425+0000 7fe97711b640 1 --2- 172.21.15.28:0/3224498373 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe9701018f0 0x7fe97010f6d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe95c00eb70 tx=0x7fe95c00b040 comp rx=0 tx=0).stop 2024-07-03T15:44:51.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.425+0000 7fe97711b640 1 -- 172.21.15.28:0/3224498373 shutdown_connections 2024-07-03T15:44:51.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.425+0000 7fe97711b640 1 --2- 172.21.15.28:0/3224498373 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fe96006a6d0 0x7fe96006cb80 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:51.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.425+0000 7fe97711b640 1 --2- 172.21.15.28:0/3224498373 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe9701018f0 0x7fe97010f6d0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:51.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.425+0000 7fe97711b640 1 --2- 172.21.15.28:0/3224498373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9700659f0 0x7fe97010f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:51.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.425+0000 7fe97711b640 1 -- 172.21.15.28:0/3224498373 >> 172.21.15.28:0/3224498373 conn(0x7fe9700f6a20 msgr2=0x7fe9700f6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:44:51.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.425+0000 7fe97711b640 1 -- 172.21.15.28:0/3224498373 shutdown_connections 2024-07-03T15:44:51.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.425+0000 7fe97711b640 1 -- 172.21.15.28:0/3224498373 wait complete. 2024-07-03T15:44:51.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.543+0000 7ff27e720640 1 -- 172.21.15.28:0/886824892 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2780fb8c0 msgr2=0x7ff2780fbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:51.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.543+0000 7ff27e720640 1 --2- 172.21.15.28:0/886824892 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2780fb8c0 0x7ff2780fbcd0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff2680048d0 tx=0x7ff268009da0 comp rx=0 tx=0).stop 2024-07-03T15:44:51.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.543+0000 7ff27e720640 1 -- 172.21.15.28:0/886824892 shutdown_connections 2024-07-03T15:44:51.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.543+0000 7ff27e720640 1 --2- 172.21.15.28:0/886824892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2780fc210 0x7ff2780fc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:51.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.543+0000 7ff27e720640 1 --2- 172.21.15.28:0/886824892 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2780fb8c0 0x7ff2780fbcd0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:51.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.543+0000 7ff27e720640 1 -- 172.21.15.28:0/886824892 >> 172.21.15.28:0/886824892 conn(0x7ff2780f61b0 msgr2=0x7ff2780f85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:44:51.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.543+0000 7ff27e720640 1 -- 172.21.15.28:0/886824892 shutdown_connections 2024-07-03T15:44:51.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.543+0000 7ff27e720640 1 -- 172.21.15.28:0/886824892 wait complete. 2024-07-03T15:44:51.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.543+0000 7ff27e720640 1 Processor -- start 2024-07-03T15:44:51.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.544+0000 7ff27e720640 1 -- start start 2024-07-03T15:44:51.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.544+0000 7ff27e720640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2780fb8c0 0x7ff2780ff4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:44:51.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.544+0000 7ff27e720640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2780fc210 0x7ff2780fd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:44:51.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.544+0000 7ff27e720640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2780fdf20 con 0x7ff2780fb8c0 2024-07-03T15:44:51.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.544+0000 7ff27e720640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2780fe060 con 0x7ff2780fc210 2024-07-03T15:44:51.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.544+0000 7ff277fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2780fb8c0 0x7ff2780ff4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:44:51.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.544+0000 7ff277fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2780fb8c0 0x7ff2780ff4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:60228/0 (socket says 172.21.15.28:60228) 2024-07-03T15:44:51.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.544+0000 7ff277fff640 1 -- 172.21.15.28:0/1587043899 learned_addr learned my addr 172.21.15.28:0/1587043899 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:44:51.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.544+0000 7ff2777fe640 1 --2- 172.21.15.28:0/1587043899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2780fc210 0x7ff2780fd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:44:51.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.544+0000 7ff277fff640 1 -- 172.21.15.28:0/1587043899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2780fc210 msgr2=0x7ff2780fd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:51.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.544+0000 7ff277fff640 1 --2- 172.21.15.28:0/1587043899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2780fc210 0x7ff2780fd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:51.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.544+0000 7ff277fff640 1 -- 172.21.15.28:0/1587043899 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff268005040 con 0x7ff2780fb8c0 2024-07-03T15:44:51.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.545+0000 7ff277fff640 1 --2- 172.21.15.28:0/1587043899 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2780fb8c0 0x7ff2780ff4a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff2680041d0 tx=0x7ff268009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:44:51.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.545+0000 7ff274ff9640 1 -- 172.21.15.28:0/1587043899 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff268038070 con 0x7ff2780fb8c0 2024-07-03T15:44:51.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.545+0000 7ff27e720640 1 -- 172.21.15.28:0/1587043899 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2780fe2e0 con 0x7ff2780fb8c0 2024-07-03T15:44:51.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.545+0000 7ff274ff9640 1 -- 172.21.15.28:0/1587043899 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff2680092b0 con 0x7ff2780fb8c0 2024-07-03T15:44:51.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.545+0000 7ff27e720640 1 -- 172.21.15.28:0/1587043899 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2780fe820 con 0x7ff2780fb8c0 2024-07-03T15:44:51.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.545+0000 7ff274ff9640 1 -- 172.21.15.28:0/1587043899 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff268033710 con 0x7ff2780fb8c0 2024-07-03T15:44:51.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.546+0000 7ff27e720640 1 -- 172.21.15.28:0/1587043899 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2780f9e50 con 0x7ff2780fb8c0 2024-07-03T15:44:51.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.547+0000 7ff274ff9640 1 -- 172.21.15.28:0/1587043899 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 24) v1 ==== 88617+0+0 (secure 0 0 0) 0x7ff268030020 con 0x7ff2780fb8c0 2024-07-03T15:44:51.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.548+0000 7ff274ff9640 1 --2- 172.21.15.28:0/1587043899 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7ff26006a6d0 0x7ff26006cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:44:51.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.548+0000 7ff274ff9640 1 -- 172.21.15.28:0/1587043899 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7ff2680a8e30 con 0x7ff2780fb8c0 2024-07-03T15:44:51.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.548+0000 7ff2777fe640 1 --2- 172.21.15.28:0/1587043899 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7ff26006a6d0 0x7ff26006cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:44:51.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.549+0000 7ff2777fe640 1 --2- 172.21.15.28:0/1587043899 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7ff26006a6d0 0x7ff26006cb80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff26c004a70 tx=0x7ff26c00b040 comp rx=0 tx=0).ready entity=mgr.24403 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:44:51.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.552+0000 7ff274ff9640 1 -- 172.21.15.28:0/1587043899 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7ff268072a50 con 0x7ff2780fb8c0 2024-07-03T15:44:51.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.813+0000 7ff27e720640 1 -- 172.21.15.28:0/1587043899 --> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff27805d960 con 0x7ff26006a6d0 2024-07-03T15:44:51.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.815+0000 7ff274ff9640 1 -- 172.21.15.28:0/1587043899 <== mgr.24403 v2:172.21.15.148:6828/4285739175 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7ff27805d960 con 0x7ff26006a6d0 2024-07-03T15:44:51.818 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-07-03T15:44:51.818 INFO:teuthology.orchestra.run.smithi028.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2", 2024-07-03T15:44:51.818 INFO:teuthology.orchestra.run.smithi028.stdout: "in_progress": true, 2024-07-03T15:44:51.819 INFO:teuthology.orchestra.run.smithi028.stdout: "which": "Upgrading all daemon types on all hosts", 2024-07-03T15:44:51.819 INFO:teuthology.orchestra.run.smithi028.stdout: "services_complete": [], 2024-07-03T15:44:51.819 INFO:teuthology.orchestra.run.smithi028.stdout: "progress": "1/21 daemons upgraded", 2024-07-03T15:44:51.819 INFO:teuthology.orchestra.run.smithi028.stdout: "message": "", 2024-07-03T15:44:51.819 INFO:teuthology.orchestra.run.smithi028.stdout: "is_paused": false 2024-07-03T15:44:51.819 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-07-03T15:44:51.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.817+0000 7ff27e720640 1 -- 172.21.15.28:0/1587043899 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7ff26006a6d0 msgr2=0x7ff26006cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:51.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.817+0000 7ff27e720640 1 --2- 172.21.15.28:0/1587043899 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7ff26006a6d0 0x7ff26006cb80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff26c004a70 tx=0x7ff26c00b040 comp rx=0 tx=0).stop 2024-07-03T15:44:51.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.817+0000 7ff27e720640 1 -- 172.21.15.28:0/1587043899 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2780fb8c0 msgr2=0x7ff2780ff4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:51.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.817+0000 7ff27e720640 1 --2- 172.21.15.28:0/1587043899 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2780fb8c0 0x7ff2780ff4a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff2680041d0 tx=0x7ff268009da0 comp rx=0 tx=0).stop 2024-07-03T15:44:51.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.817+0000 7ff27e720640 1 -- 172.21.15.28:0/1587043899 shutdown_connections 2024-07-03T15:44:51.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.817+0000 7ff27e720640 1 --2- 172.21.15.28:0/1587043899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2780fc210 0x7ff2780fd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:51.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.817+0000 7ff27e720640 1 --2- 172.21.15.28:0/1587043899 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7ff26006a6d0 0x7ff26006cb80 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:51.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.817+0000 7ff27e720640 1 --2- 172.21.15.28:0/1587043899 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2780fb8c0 0x7ff2780ff4a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:51.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.817+0000 7ff27e720640 1 -- 172.21.15.28:0/1587043899 >> 172.21.15.28:0/1587043899 conn(0x7ff2780f61b0 msgr2=0x7ff2780fab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:44:51.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.817+0000 7ff27e720640 1 -- 172.21.15.28:0/1587043899 shutdown_connections 2024-07-03T15:44:51.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.817+0000 7ff27e720640 1 -- 172.21.15.28:0/1587043899 wait complete. 2024-07-03T15:44:51.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.964+0000 7f8580519640 1 -- 172.21.15.28:0/3531990259 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f857806f320 msgr2=0x7f857806d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:51.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.964+0000 7f8580519640 1 --2- 172.21.15.28:0/3531990259 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f857806f320 0x7f857806d850 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f85680048d0 tx=0x7f8568009da0 comp rx=0 tx=0).stop 2024-07-03T15:44:51.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.965+0000 7f8580519640 1 -- 172.21.15.28:0/3531990259 shutdown_connections 2024-07-03T15:44:51.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.965+0000 7f8580519640 1 --2- 172.21.15.28:0/3531990259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f857806dd90 0x7f857806e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:51.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.965+0000 7f8580519640 1 --2- 172.21.15.28:0/3531990259 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f857806f320 0x7f857806d850 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:51.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.965+0000 7f8580519640 1 -- 172.21.15.28:0/3531990259 >> 172.21.15.28:0/3531990259 conn(0x7f8578069370 msgr2=0x7f857806b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:44:51.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.965+0000 7f8580519640 1 -- 172.21.15.28:0/3531990259 shutdown_connections 2024-07-03T15:44:51.969 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:51 smithi028 ceph-mon[29148]: from='client.14678 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:44:51.969 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:51 smithi028 ceph-mon[29148]: pgmap v27: 65 pgs: 65 active+clean; 311 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 401 op/s 2024-07-03T15:44:51.970 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:51 smithi028 ceph-mon[29148]: from='client.24455 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:44:51.970 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:51 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/404186519' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:44:51.970 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:51 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/3224498373' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-03T15:44:51.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.965+0000 7f8580519640 1 -- 172.21.15.28:0/3531990259 wait complete. 2024-07-03T15:44:51.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.965+0000 7f8580519640 1 Processor -- start 2024-07-03T15:44:51.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.966+0000 7f8580519640 1 -- start start 2024-07-03T15:44:51.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.966+0000 7f8580519640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f857806dd90 0x7f8578080000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:44:51.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.966+0000 7f8580519640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f857806f320 0x7f857807e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:44:51.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.966+0000 7f8580519640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8578080600 con 0x7f857806f320 2024-07-03T15:44:51.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.966+0000 7f8580519640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f857807ea60 con 0x7f857806dd90 2024-07-03T15:44:51.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.966+0000 7f857e2b5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f857806dd90 0x7f8578080000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:44:51.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.966+0000 7f857e2b5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f857806dd90 0x7f8578080000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.28:57342/0 (socket says 172.21.15.28:57342) 2024-07-03T15:44:51.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.966+0000 7f857e2b5640 1 -- 172.21.15.28:0/231459923 learned_addr learned my addr 172.21.15.28:0/231459923 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:44:51.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.967+0000 7f857e2b5640 1 -- 172.21.15.28:0/231459923 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f857806f320 msgr2=0x7f857807e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:51.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.967+0000 7f857e2b5640 1 --2- 172.21.15.28:0/231459923 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f857806f320 0x7f857807e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:51.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.967+0000 7f857e2b5640 1 -- 172.21.15.28:0/231459923 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8568005040 con 0x7f857806dd90 2024-07-03T15:44:51.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.967+0000 7f857e2b5640 1 --2- 172.21.15.28:0/231459923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f857806dd90 0x7f8578080000 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f85680047f0 tx=0x7f8568009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:44:51.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.968+0000 7f856effd640 1 -- 172.21.15.28:0/231459923 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8568038070 con 0x7f857806dd90 2024-07-03T15:44:51.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.968+0000 7f856effd640 1 -- 172.21.15.28:0/231459923 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8568032970 con 0x7f857806dd90 2024-07-03T15:44:51.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.968+0000 7f856effd640 1 -- 172.21.15.28:0/231459923 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f856803da80 con 0x7f857806dd90 2024-07-03T15:44:51.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.969+0000 7f8580519640 1 -- 172.21.15.28:0/231459923 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f857807ece0 con 0x7f857806dd90 2024-07-03T15:44:51.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.969+0000 7f8580519640 1 -- 172.21.15.28:0/231459923 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f857807f220 con 0x7f857806dd90 2024-07-03T15:44:51.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.973+0000 7f8580519640 1 -- 172.21.15.28:0/231459923 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f857806cfc0 con 0x7f857806dd90 2024-07-03T15:44:51.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.973+0000 7f856effd640 1 -- 172.21.15.28:0/231459923 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 24) v1 ==== 88617+0+0 (secure 0 0 0) 0x7f8568030020 con 0x7f857806dd90 2024-07-03T15:44:51.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.974+0000 7f856effd640 1 --2- 172.21.15.28:0/231459923 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f856406a7e0 0x7f856406cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:44:51.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.974+0000 7f856effd640 1 -- 172.21.15.28:0/231459923 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f85680a9740 con 0x7f857806dd90 2024-07-03T15:44:51.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.976+0000 7f857dab4640 1 --2- 172.21.15.28:0/231459923 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f856406a7e0 0x7f856406cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:44:51.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.977+0000 7f857dab4640 1 --2- 172.21.15.28:0/231459923 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f856406a7e0 0x7f856406cc90 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8574004d40 tx=0x7f857400b040 comp rx=0 tx=0).ready entity=mgr.24403 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:44:51.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:51 smithi148 ceph-mon[36056]: from='client.14678 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:44:51.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:51 smithi148 ceph-mon[36056]: pgmap v27: 65 pgs: 65 active+clean; 311 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 401 op/s 2024-07-03T15:44:51.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:51 smithi148 ceph-mon[36056]: from='client.24455 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:44:51.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:51 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/404186519' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:44:51.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:51 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/3224498373' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-03T15:44:51.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:51.981+0000 7f856effd640 1 -- 172.21.15.28:0/231459923 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7f8568073340 con 0x7f857806dd90 2024-07-03T15:44:52.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:52.350+0000 7f8580519640 1 -- 172.21.15.28:0/231459923 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f8578072dd0 con 0x7f857806dd90 2024-07-03T15:44:52.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:52.351+0000 7f856effd640 1 -- 172.21.15.28:0/231459923 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f8568076a00 con 0x7f857806dd90 2024-07-03T15:44:52.355 INFO:teuthology.orchestra.run.smithi028.stdout:HEALTH_OK 2024-07-03T15:44:52.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:52.353+0000 7f8580519640 1 -- 172.21.15.28:0/231459923 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f856406a7e0 msgr2=0x7f856406cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:52.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:52.353+0000 7f8580519640 1 --2- 172.21.15.28:0/231459923 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f856406a7e0 0x7f856406cc90 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8574004d40 tx=0x7f857400b040 comp rx=0 tx=0).stop 2024-07-03T15:44:52.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:52.353+0000 7f8580519640 1 -- 172.21.15.28:0/231459923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f857806dd90 msgr2=0x7f8578080000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:44:52.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:52.353+0000 7f8580519640 1 --2- 172.21.15.28:0/231459923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f857806dd90 0x7f8578080000 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f85680047f0 tx=0x7f8568009da0 comp rx=0 tx=0).stop 2024-07-03T15:44:52.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:52.353+0000 7f8580519640 1 -- 172.21.15.28:0/231459923 shutdown_connections 2024-07-03T15:44:52.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:52.353+0000 7f8580519640 1 --2- 172.21.15.28:0/231459923 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f856406a7e0 0x7f856406cc90 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:52.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:52.353+0000 7f8580519640 1 --2- 172.21.15.28:0/231459923 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f857806f320 0x7f857807e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:52.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:52.353+0000 7f8580519640 1 --2- 172.21.15.28:0/231459923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f857806dd90 0x7f8578080000 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:44:52.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:52.353+0000 7f8580519640 1 -- 172.21.15.28:0/231459923 >> 172.21.15.28:0/231459923 conn(0x7f8578069370 msgr2=0x7f8578069e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:44:52.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:52.353+0000 7f8580519640 1 -- 172.21.15.28:0/231459923 shutdown_connections 2024-07-03T15:44:52.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:44:52.353+0000 7f8580519640 1 -- 172.21.15.28:0/231459923 wait complete. 2024-07-03T15:44:52.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:52 smithi148 ceph-mon[36056]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:44:52.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:52 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/231459923' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-03T15:44:53.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:52 smithi028 ceph-mon[29148]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:44:53.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:52 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/231459923' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-03T15:44:54.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:53 smithi028 ceph-mon[29148]: pgmap v28: 65 pgs: 65 active+clean; 311 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 307 op/s 2024-07-03T15:44:54.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:53 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:54.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:53 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:54.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:53 smithi028 ceph-mon[29148]: Reconfiguring node-exporter.smithi148 (dependencies changed)... 2024-07-03T15:44:54.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:53 smithi028 ceph-mon[29148]: Reconfiguring daemon node-exporter.smithi148 on smithi148 2024-07-03T15:44:54.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:53 smithi148 ceph-mon[36056]: pgmap v28: 65 pgs: 65 active+clean; 311 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 307 op/s 2024-07-03T15:44:54.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:53 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:54.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:53 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:54.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:53 smithi148 ceph-mon[36056]: Reconfiguring node-exporter.smithi148 (dependencies changed)... 2024-07-03T15:44:54.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:53 smithi148 ceph-mon[36056]: Reconfiguring daemon node-exporter.smithi148 on smithi148 2024-07-03T15:44:56.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:55 smithi028 ceph-mon[29148]: pgmap v29: 65 pgs: 65 active+clean; 310 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 472 op/s 2024-07-03T15:44:56.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:55 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:56.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:55 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:56.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:55 smithi028 ceph-mon[29148]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-07-03T15:44:56.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:55 smithi028 ceph-mon[29148]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-07-03T15:44:56.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:55 smithi028 ceph-mon[29148]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-07-03T15:44:56.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:55 smithi028 ceph-mon[29148]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:44:56.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:55 smithi148 ceph-mon[36056]: pgmap v29: 65 pgs: 65 active+clean; 310 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 472 op/s 2024-07-03T15:44:56.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:55 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:56.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:55 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:44:56.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:55 smithi148 ceph-mon[36056]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-07-03T15:44:56.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:55 smithi148 ceph-mon[36056]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-07-03T15:44:56.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:55 smithi148 ceph-mon[36056]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-07-03T15:44:56.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:55 smithi148 ceph-mon[36056]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:44:57.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:56 smithi028 ceph-mon[29148]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-07-03T15:44:57.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:56 smithi028 ceph-mon[29148]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-07-03T15:44:57.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:56 smithi028 ceph-mon[29148]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-07-03T15:44:57.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:56 smithi028 ceph-mon[29148]: Upgrade: Need to upgrade myself (mgr.smithi148.rjzsar) 2024-07-03T15:44:57.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:56 smithi028 ceph-mon[29148]: Upgrade: Need to upgrade myself (mgr.smithi148.rjzsar) 2024-07-03T15:44:57.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:56 smithi148 ceph-mon[36056]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-07-03T15:44:57.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:56 smithi148 ceph-mon[36056]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-07-03T15:44:57.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:56 smithi148 ceph-mon[36056]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-07-03T15:44:57.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:56 smithi148 ceph-mon[36056]: Upgrade: Need to upgrade myself (mgr.smithi148.rjzsar) 2024-07-03T15:44:57.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:56 smithi148 ceph-mon[36056]: Upgrade: Need to upgrade myself (mgr.smithi148.rjzsar) 2024-07-03T15:44:58.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:57 smithi028 ceph-mon[29148]: pgmap v30: 65 pgs: 65 active+clean; 310 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 334 op/s 2024-07-03T15:44:58.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:57 smithi148 ceph-mon[36056]: pgmap v30: 65 pgs: 65 active+clean; 310 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 334 op/s 2024-07-03T15:45:00.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:44:59 smithi148 ceph-mon[36056]: pgmap v31: 65 pgs: 65 active+clean; 318 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 412 op/s 2024-07-03T15:45:00.288 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:44:59 smithi028 ceph-mon[29148]: pgmap v31: 65 pgs: 65 active+clean; 318 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 412 op/s 2024-07-03T15:45:02.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:01 smithi148 ceph-mon[36056]: pgmap v32: 65 pgs: 65 active+clean; 319 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 388 op/s 2024-07-03T15:45:02.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:01 smithi028 ceph-mon[29148]: pgmap v32: 65 pgs: 65 active+clean; 319 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 388 op/s 2024-07-03T15:45:04.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:03 smithi148 ceph-mon[36056]: pgmap v33: 65 pgs: 65 active+clean; 319 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 260 op/s 2024-07-03T15:45:04.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:03 smithi148 ceph-mon[36056]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:45:04.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:03 smithi028 ceph-mon[29148]: pgmap v33: 65 pgs: 65 active+clean; 319 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 260 op/s 2024-07-03T15:45:04.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:03 smithi028 ceph-mon[29148]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:45:06.027 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:05 smithi028 ceph-mon[29148]: pgmap v34: 65 pgs: 65 active+clean; 324 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 569 op/s 2024-07-03T15:45:06.028 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:05 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:45:06.028 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:05 smithi028 ceph-mon[29148]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi028.vtqrtz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-03T15:45:06.028 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:05 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi028.vtqrtz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-03T15:45:06.028 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:05 smithi028 ceph-mon[29148]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-03T15:45:06.028 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:05 smithi028 ceph-mon[29148]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:45:06.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:05 smithi148 ceph-mon[36056]: pgmap v34: 65 pgs: 65 active+clean; 324 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 569 op/s 2024-07-03T15:45:06.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:05 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:45:06.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:05 smithi148 ceph-mon[36056]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi028.vtqrtz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-03T15:45:06.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:05 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi028.vtqrtz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-03T15:45:06.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:05 smithi148 ceph-mon[36056]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-03T15:45:06.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:05 smithi148 ceph-mon[36056]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:45:07.118 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:06 smithi028 ceph-mon[29148]: Upgrade: Updating mgr.smithi028.vtqrtz 2024-07-03T15:45:07.118 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:06 smithi028 ceph-mon[29148]: Deploying daemon mgr.smithi028.vtqrtz on smithi028 2024-07-03T15:45:07.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:06 smithi148 ceph-mon[36056]: Upgrade: Updating mgr.smithi028.vtqrtz 2024-07-03T15:45:07.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:06 smithi148 ceph-mon[36056]: Deploying daemon mgr.smithi028.vtqrtz on smithi028 2024-07-03T15:45:08.089 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:07 smithi028 ceph-mon[29148]: pgmap v35: 65 pgs: 65 active+clean; 324 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 403 op/s 2024-07-03T15:45:08.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:07 smithi148 ceph-mon[36056]: pgmap v35: 65 pgs: 65 active+clean; 324 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 403 op/s 2024-07-03T15:45:10.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:09 smithi028 ceph-mon[29148]: pgmap v36: 65 pgs: 65 active+clean; 317 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 599 op/s 2024-07-03T15:45:10.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:09 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:45:10.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:09 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:45:10.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:09 smithi028 ceph-mon[29148]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:45:10.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:09 smithi148 ceph-mon[36056]: pgmap v36: 65 pgs: 65 active+clean; 317 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 599 op/s 2024-07-03T15:45:10.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:09 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:45:10.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:09 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:45:10.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:09 smithi148 ceph-mon[36056]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:45:12.129 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:11 smithi148 ceph-mon[36056]: pgmap v37: 65 pgs: 65 active+clean; 317 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 521 op/s 2024-07-03T15:45:12.225 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:11 smithi028 ceph-mon[29148]: pgmap v37: 65 pgs: 65 active+clean; 317 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 521 op/s 2024-07-03T15:45:13.868 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:13 smithi028 ceph-mon[29148]: pgmap v38: 65 pgs: 65 active+clean; 317 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 504 op/s 2024-07-03T15:45:13.868 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:13 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:45:13.869 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:13 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:45:13.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:13 smithi148 ceph-mon[36056]: pgmap v38: 65 pgs: 65 active+clean; 317 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 504 op/s 2024-07-03T15:45:13.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:13 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:45:13.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:13 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:45:15.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:15 smithi028 ceph-mon[29148]: pgmap v39: 65 pgs: 65 active+clean; 316 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 1.17k op/s 2024-07-03T15:45:15.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:15 smithi148 ceph-mon[36056]: pgmap v39: 65 pgs: 65 active+clean; 316 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 1.17k op/s 2024-07-03T15:45:16.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:16 smithi028 ceph-mon[29148]: Standby manager daemon smithi028.vtqrtz restarted 2024-07-03T15:45:16.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:16 smithi028 ceph-mon[29148]: Standby manager daemon smithi028.vtqrtz started 2024-07-03T15:45:16.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:16 smithi028 ceph-mon[29148]: from='mgr.? 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi028.vtqrtz/crt"}]: dispatch 2024-07-03T15:45:16.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:16 smithi028 ceph-mon[29148]: from='mgr.? 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-03T15:45:16.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:16 smithi028 ceph-mon[29148]: from='mgr.? 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi028.vtqrtz/key"}]: dispatch 2024-07-03T15:45:16.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:16 smithi028 ceph-mon[29148]: from='mgr.? 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-03T15:45:16.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:16 smithi148 ceph-mon[36056]: Standby manager daemon smithi028.vtqrtz restarted 2024-07-03T15:45:16.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:16 smithi148 ceph-mon[36056]: Standby manager daemon smithi028.vtqrtz started 2024-07-03T15:45:16.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:16 smithi148 ceph-mon[36056]: from='mgr.? 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi028.vtqrtz/crt"}]: dispatch 2024-07-03T15:45:16.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:16 smithi148 ceph-mon[36056]: from='mgr.? 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-03T15:45:16.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:16 smithi148 ceph-mon[36056]: from='mgr.? 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi028.vtqrtz/key"}]: dispatch 2024-07-03T15:45:16.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:16 smithi148 ceph-mon[36056]: from='mgr.? 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-03T15:45:17.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:17 smithi028 ceph-mon[29148]: pgmap v40: 65 pgs: 65 active+clean; 316 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 860 op/s 2024-07-03T15:45:17.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:17 smithi028 ceph-mon[29148]: mgrmap e25: smithi148.rjzsar(active, since 74s), standbys: smithi028.vtqrtz 2024-07-03T15:45:17.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:17 smithi148 ceph-mon[36056]: pgmap v40: 65 pgs: 65 active+clean; 316 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 860 op/s 2024-07-03T15:45:17.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:17 smithi148 ceph-mon[36056]: mgrmap e25: smithi148.rjzsar(active, since 74s), standbys: smithi028.vtqrtz 2024-07-03T15:45:18.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:18 smithi148 ceph-mon[36056]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:45:18.794 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:18 smithi028 ceph-mon[29148]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:45:19.902 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:19 smithi028 ceph-mon[29148]: pgmap v41: 65 pgs: 65 active+clean; 284 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 1.18k op/s 2024-07-03T15:45:19.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:19 smithi148 ceph-mon[36056]: pgmap v41: 65 pgs: 65 active+clean; 284 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 1.18k op/s 2024-07-03T15:45:21.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:21 smithi148 ceph-mon[36056]: pgmap v42: 65 pgs: 65 active+clean; 284 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 984 op/s 2024-07-03T15:45:22.087 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:21 smithi028 ceph-mon[29148]: pgmap v42: 65 pgs: 65 active+clean; 284 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 984 op/s 2024-07-03T15:45:22.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.490+0000 7f06f6ff8640 1 -- 172.21.15.28:0/33162132 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f06f00fb0d0 msgr2=0x7f06f0101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:22.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.490+0000 7f06f6ff8640 1 --2- 172.21.15.28:0/33162132 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f06f00fb0d0 0x7f06f0101900 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f06e00048d0 tx=0x7f06e0009da0 comp rx=0 tx=0).stop 2024-07-03T15:45:22.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.490+0000 7f06f6ff8640 1 -- 172.21.15.28:0/33162132 shutdown_connections 2024-07-03T15:45:22.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.490+0000 7f06f6ff8640 1 --2- 172.21.15.28:0/33162132 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f06f00fb0d0 0x7f06f0101900 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:22.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.490+0000 7f06f6ff8640 1 --2- 172.21.15.28:0/33162132 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06f00fa760 0x7f06f00fab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:22.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.490+0000 7f06f6ff8640 1 -- 172.21.15.28:0/33162132 >> 172.21.15.28:0/33162132 conn(0x7f06f00f61a0 msgr2=0x7f06f00f85b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:45:22.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.491+0000 7f06f6ff8640 1 -- 172.21.15.28:0/33162132 shutdown_connections 2024-07-03T15:45:22.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.491+0000 7f06f6ff8640 1 -- 172.21.15.28:0/33162132 wait complete. 2024-07-03T15:45:22.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.491+0000 7f06f6ff8640 1 Processor -- start 2024-07-03T15:45:22.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.492+0000 7f06f6ff8640 1 -- start start 2024-07-03T15:45:22.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.492+0000 7f06f6ff8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06f00fa760 0x7f06f00fd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:45:22.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.492+0000 7f06f6ff8640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f06f00fb0d0 0x7f06f00fb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:45:22.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.492+0000 7f06f6ff8640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f00fbcf0 con 0x7f06f00fb0d0 2024-07-03T15:45:22.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.492+0000 7f06f6ff8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f00fbe30 con 0x7f06f00fa760 2024-07-03T15:45:22.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.492+0000 7f06e7fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f06f00fb0d0 0x7f06f00fb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:45:22.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.492+0000 7f06f4d94640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06f00fa760 0x7f06f00fd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:45:22.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.492+0000 7f06e7fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f06f00fb0d0 0x7f06f00fb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50428/0 (socket says 172.21.15.28:50428) 2024-07-03T15:45:22.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.492+0000 7f06e7fff640 1 -- 172.21.15.28:0/3509173657 learned_addr learned my addr 172.21.15.28:0/3509173657 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:45:22.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.493+0000 7f06f4d94640 1 -- 172.21.15.28:0/3509173657 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f06f00fb0d0 msgr2=0x7f06f00fb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:22.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.493+0000 7f06f4d94640 1 --2- 172.21.15.28:0/3509173657 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f06f00fb0d0 0x7f06f00fb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:22.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.493+0000 7f06f4d94640 1 -- 172.21.15.28:0/3509173657 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e0005040 con 0x7f06f00fa760 2024-07-03T15:45:22.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.493+0000 7f06f4d94640 1 --2- 172.21.15.28:0/3509173657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06f00fa760 0x7f06f00fd270 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f06f00fd870 tx=0x7f06ec00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:45:22.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.494+0000 7f06e57fa640 1 -- 172.21.15.28:0/3509173657 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06ec019070 con 0x7f06f00fa760 2024-07-03T15:45:22.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.494+0000 7f06f6ff8640 1 -- 172.21.15.28:0/3509173657 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06f00fc0b0 con 0x7f06f00fa760 2024-07-03T15:45:22.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.494+0000 7f06e57fa640 1 -- 172.21.15.28:0/3509173657 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f06ec0095a0 con 0x7f06f00fa760 2024-07-03T15:45:22.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.494+0000 7f06e57fa640 1 -- 172.21.15.28:0/3509173657 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06ec019070 con 0x7f06f00fa760 2024-07-03T15:45:22.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.494+0000 7f06f6ff8640 1 -- 172.21.15.28:0/3509173657 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06f010cb80 con 0x7f06f00fa760 2024-07-03T15:45:22.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.496+0000 7f06f6ff8640 1 -- 172.21.15.28:0/3509173657 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06dc005240 con 0x7f06f00fa760 2024-07-03T15:45:22.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.496+0000 7f06e57fa640 1 -- 172.21.15.28:0/3509173657 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 25) v1 ==== 92029+0+0 (secure 0 0 0) 0x7f06ec01d9e0 con 0x7f06f00fa760 2024-07-03T15:45:22.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.497+0000 7f06e57fa640 1 --2- 172.21.15.28:0/3509173657 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f06d806db80 0x7f06d8070030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:45:22.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.497+0000 7f06e57fa640 1 -- 172.21.15.28:0/3509173657 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f06ec095760 con 0x7f06f00fa760 2024-07-03T15:45:22.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.497+0000 7f06e7fff640 1 --2- 172.21.15.28:0/3509173657 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f06d806db80 0x7f06d8070030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:45:22.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.498+0000 7f06e7fff640 1 --2- 172.21.15.28:0/3509173657 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f06d806db80 0x7f06d8070030 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f06e0003080 tx=0x7f06e0035040 comp rx=0 tx=0).ready entity=mgr.24403 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:45:22.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.502+0000 7f06e57fa640 1 -- 172.21.15.28:0/3509173657 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7f06ec062d00 con 0x7f06f00fa760 2024-07-03T15:45:22.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.821+0000 7f06f6ff8640 1 -- 172.21.15.28:0/3509173657 --> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f06dc000bf0 con 0x7f06d806db80 2024-07-03T15:45:22.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.822+0000 7f06e57fa640 1 -- 172.21.15.28:0/3509173657 <== mgr.24403 v2:172.21.15.148:6828/4285739175 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+328 (secure 0 0 0) 0x7f06dc000bf0 con 0x7f06d806db80 2024-07-03T15:45:22.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.824+0000 7f06f6ff8640 1 -- 172.21.15.28:0/3509173657 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f06d806db80 msgr2=0x7f06d8070030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:22.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.824+0000 7f06f6ff8640 1 --2- 172.21.15.28:0/3509173657 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f06d806db80 0x7f06d8070030 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f06e0003080 tx=0x7f06e0035040 comp rx=0 tx=0).stop 2024-07-03T15:45:22.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.825+0000 7f06f6ff8640 1 -- 172.21.15.28:0/3509173657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06f00fa760 msgr2=0x7f06f00fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:22.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.825+0000 7f06f6ff8640 1 --2- 172.21.15.28:0/3509173657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06f00fa760 0x7f06f00fd270 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f06f00fd870 tx=0x7f06ec00b040 comp rx=0 tx=0).stop 2024-07-03T15:45:22.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.825+0000 7f06f6ff8640 1 -- 172.21.15.28:0/3509173657 shutdown_connections 2024-07-03T15:45:22.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.825+0000 7f06f6ff8640 1 --2- 172.21.15.28:0/3509173657 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f06d806db80 0x7f06d8070030 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:22.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.825+0000 7f06f6ff8640 1 --2- 172.21.15.28:0/3509173657 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f06f00fb0d0 0x7f06f00fb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:22.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.825+0000 7f06f6ff8640 1 --2- 172.21.15.28:0/3509173657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06f00fa760 0x7f06f00fd270 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:22.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.825+0000 7f06f6ff8640 1 -- 172.21.15.28:0/3509173657 >> 172.21.15.28:0/3509173657 conn(0x7f06f00f61a0 msgr2=0x7f06f00ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:45:22.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.825+0000 7f06f6ff8640 1 -- 172.21.15.28:0/3509173657 shutdown_connections 2024-07-03T15:45:22.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.825+0000 7f06f6ff8640 1 -- 172.21.15.28:0/3509173657 wait complete. 2024-07-03T15:45:22.838 INFO:teuthology.orchestra.run.smithi028.stdout:true 2024-07-03T15:45:22.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.930+0000 7f5b1bc43640 1 -- 172.21.15.28:0/2052740742 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b140ff4b0 msgr2=0x7f5b14101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:22.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.930+0000 7f5b1bc43640 1 --2- 172.21.15.28:0/2052740742 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b140ff4b0 0x7f5b14101890 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5b040048d0 tx=0x7f5b04009da0 comp rx=0 tx=0).stop 2024-07-03T15:45:22.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.930+0000 7f5b1bc43640 1 -- 172.21.15.28:0/2052740742 shutdown_connections 2024-07-03T15:45:22.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.930+0000 7f5b1bc43640 1 --2- 172.21.15.28:0/2052740742 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b140ff4b0 0x7f5b14101890 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:22.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.930+0000 7f5b1bc43640 1 --2- 172.21.15.28:0/2052740742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b14066540 0x7f5b140fef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:22.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.930+0000 7f5b1bc43640 1 -- 172.21.15.28:0/2052740742 >> 172.21.15.28:0/2052740742 conn(0x7f5b140f6a20 msgr2=0x7f5b140f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:45:22.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.931+0000 7f5b1bc43640 1 -- 172.21.15.28:0/2052740742 shutdown_connections 2024-07-03T15:45:22.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.931+0000 7f5b1bc43640 1 -- 172.21.15.28:0/2052740742 wait complete. 2024-07-03T15:45:22.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.931+0000 7f5b1bc43640 1 Processor -- start 2024-07-03T15:45:22.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.931+0000 7f5b1bc43640 1 -- start start 2024-07-03T15:45:22.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.931+0000 7f5b1bc43640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b14066540 0x7f5b141135a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:45:22.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.932+0000 7f5b1bc43640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b140ff4b0 0x7f5b14113ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:45:22.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.932+0000 7f5b1bc43640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b14114130 con 0x7f5b14066540 2024-07-03T15:45:22.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.932+0000 7f5b1bc43640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b14114950 con 0x7f5b140ff4b0 2024-07-03T15:45:22.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.932+0000 7f5b199df640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b14066540 0x7f5b141135a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:45:22.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.932+0000 7f5b199df640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b14066540 0x7f5b141135a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50450/0 (socket says 172.21.15.28:50450) 2024-07-03T15:45:22.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.932+0000 7f5b199df640 1 -- 172.21.15.28:0/2642630962 learned_addr learned my addr 172.21.15.28:0/2642630962 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:45:22.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.932+0000 7f5b191de640 1 --2- 172.21.15.28:0/2642630962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b140ff4b0 0x7f5b14113ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:45:22.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.932+0000 7f5b199df640 1 -- 172.21.15.28:0/2642630962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b140ff4b0 msgr2=0x7f5b14113ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:22.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.932+0000 7f5b199df640 1 --2- 172.21.15.28:0/2642630962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b140ff4b0 0x7f5b14113ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:22.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.932+0000 7f5b199df640 1 -- 172.21.15.28:0/2642630962 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b04005040 con 0x7f5b14066540 2024-07-03T15:45:22.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.932+0000 7f5b191de640 1 --2- 172.21.15.28:0/2642630962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b140ff4b0 0x7f5b14113ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-03T15:45:22.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.932+0000 7f5b199df640 1 --2- 172.21.15.28:0/2642630962 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b14066540 0x7f5b141135a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5b1000eb70 tx=0x7f5b1000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:45:22.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.932+0000 7f5b0a7fc640 1 -- 172.21.15.28:0/2642630962 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b10007c30 con 0x7f5b14066540 2024-07-03T15:45:22.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.932+0000 7f5b1bc43640 1 -- 172.21.15.28:0/2642630962 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b14114be0 con 0x7f5b14066540 2024-07-03T15:45:22.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.933+0000 7f5b1bc43640 1 -- 172.21.15.28:0/2642630962 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b14115180 con 0x7f5b14066540 2024-07-03T15:45:22.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.933+0000 7f5b0a7fc640 1 -- 172.21.15.28:0/2642630962 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5b1000f040 con 0x7f5b14066540 2024-07-03T15:45:22.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.933+0000 7f5b0a7fc640 1 -- 172.21.15.28:0/2642630962 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b1000cb90 con 0x7f5b14066540 2024-07-03T15:45:22.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.934+0000 7f5b1bc43640 1 -- 172.21.15.28:0/2642630962 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5af8005240 con 0x7f5b14066540 2024-07-03T15:45:22.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.935+0000 7f5b0a7fc640 1 -- 172.21.15.28:0/2642630962 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 25) v1 ==== 92029+0+0 (secure 0 0 0) 0x7f5b100096e0 con 0x7f5b14066540 2024-07-03T15:45:22.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.935+0000 7f5b0a7fc640 1 --2- 172.21.15.28:0/2642630962 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f5b0006dae0 0x7f5b0006ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:45:22.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.935+0000 7f5b0a7fc640 1 -- 172.21.15.28:0/2642630962 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f5b1008d540 con 0x7f5b14066540 2024-07-03T15:45:22.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.935+0000 7f5b191de640 1 --2- 172.21.15.28:0/2642630962 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f5b0006dae0 0x7f5b0006ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:45:22.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.936+0000 7f5b191de640 1 --2- 172.21.15.28:0/2642630962 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f5b0006dae0 0x7f5b0006ff90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5b04000c00 tx=0x7f5b04009da0 comp rx=0 tx=0).ready entity=mgr.24403 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:45:22.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:22.940+0000 7f5b0a7fc640 1 -- 172.21.15.28:0/2642630962 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7f5b100570d0 con 0x7f5b14066540 2024-07-03T15:45:23.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.266+0000 7f5b1bc43640 1 -- 172.21.15.28:0/2642630962 --> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5af8000bf0 con 0x7f5b0006dae0 2024-07-03T15:45:23.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.269+0000 7f5b0a7fc640 1 -- 172.21.15.28:0/2642630962 <== mgr.24403 v2:172.21.15.148:6828/4285739175 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+328 (secure 0 0 0) 0x7f5af8000bf0 con 0x7f5b0006dae0 2024-07-03T15:45:23.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.273+0000 7f5b1bc43640 1 -- 172.21.15.28:0/2642630962 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f5b0006dae0 msgr2=0x7f5b0006ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:23.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.273+0000 7f5b1bc43640 1 --2- 172.21.15.28:0/2642630962 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f5b0006dae0 0x7f5b0006ff90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5b04000c00 tx=0x7f5b04009da0 comp rx=0 tx=0).stop 2024-07-03T15:45:23.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.273+0000 7f5b1bc43640 1 -- 172.21.15.28:0/2642630962 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b14066540 msgr2=0x7f5b141135a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:23.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.273+0000 7f5b1bc43640 1 --2- 172.21.15.28:0/2642630962 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b14066540 0x7f5b141135a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5b1000eb70 tx=0x7f5b1000b040 comp rx=0 tx=0).stop 2024-07-03T15:45:23.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.273+0000 7f5b1bc43640 1 -- 172.21.15.28:0/2642630962 shutdown_connections 2024-07-03T15:45:23.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.273+0000 7f5b1bc43640 1 --2- 172.21.15.28:0/2642630962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b140ff4b0 0x7f5b14113ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:23.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.273+0000 7f5b1bc43640 1 --2- 172.21.15.28:0/2642630962 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f5b0006dae0 0x7f5b0006ff90 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:23.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.273+0000 7f5b1bc43640 1 --2- 172.21.15.28:0/2642630962 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b14066540 0x7f5b141135a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:23.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.273+0000 7f5b1bc43640 1 -- 172.21.15.28:0/2642630962 >> 172.21.15.28:0/2642630962 conn(0x7f5b140f6a20 msgr2=0x7f5b140fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:45:23.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.275+0000 7f5b1bc43640 1 -- 172.21.15.28:0/2642630962 shutdown_connections 2024-07-03T15:45:23.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.275+0000 7f5b1bc43640 1 -- 172.21.15.28:0/2642630962 wait complete. 2024-07-03T15:45:23.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.382+0000 7fd1420d4640 1 -- 172.21.15.28:0/736558319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd13c0fc510 msgr2=0x7fd13c0fc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:23.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.382+0000 7fd1420d4640 1 --2- 172.21.15.28:0/736558319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd13c0fc510 0x7fd13c0fc980 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd130004810 tx=0x7fd130009da0 comp rx=0 tx=0).stop 2024-07-03T15:45:23.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.382+0000 7fd1420d4640 1 -- 172.21.15.28:0/736558319 shutdown_connections 2024-07-03T15:45:23.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.382+0000 7fd1420d4640 1 --2- 172.21.15.28:0/736558319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd13c0fc510 0x7fd13c0fc980 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:23.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.382+0000 7fd1420d4640 1 --2- 172.21.15.28:0/736558319 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd13c0fb240 0x7fd13c0fb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:23.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.382+0000 7fd1420d4640 1 -- 172.21.15.28:0/736558319 >> 172.21.15.28:0/736558319 conn(0x7fd13c0f6a20 msgr2=0x7fd13c0f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:45:23.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.382+0000 7fd1420d4640 1 -- 172.21.15.28:0/736558319 shutdown_connections 2024-07-03T15:45:23.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.382+0000 7fd1420d4640 1 -- 172.21.15.28:0/736558319 wait complete. 2024-07-03T15:45:23.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.383+0000 7fd1420d4640 1 Processor -- start 2024-07-03T15:45:23.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.383+0000 7fd1420d4640 1 -- start start 2024-07-03T15:45:23.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.384+0000 7fd1420d4640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd13c0fb240 0x7fd13c10cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:45:23.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.384+0000 7fd1420d4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd13c0fc510 0x7fd13c10d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:45:23.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.384+0000 7fd1420d4640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13c10dae0 con 0x7fd13c0fb240 2024-07-03T15:45:23.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.384+0000 7fd1420d4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13c10e300 con 0x7fd13c0fc510 2024-07-03T15:45:23.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.384+0000 7fd13b7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd13c0fb240 0x7fd13c10cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:45:23.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.384+0000 7fd13b7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd13c0fb240 0x7fd13c10cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50476/0 (socket says 172.21.15.28:50476) 2024-07-03T15:45:23.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.384+0000 7fd13b7fe640 1 -- 172.21.15.28:0/3027512004 learned_addr learned my addr 172.21.15.28:0/3027512004 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:45:23.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.384+0000 7fd13affd640 1 --2- 172.21.15.28:0/3027512004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd13c0fc510 0x7fd13c10d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:45:23.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.385+0000 7fd13affd640 1 -- 172.21.15.28:0/3027512004 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd13c0fb240 msgr2=0x7fd13c10cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:23.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.385+0000 7fd13affd640 1 --2- 172.21.15.28:0/3027512004 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd13c0fb240 0x7fd13c10cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:23.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.385+0000 7fd13affd640 1 -- 172.21.15.28:0/3027512004 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd130005040 con 0x7fd13c0fc510 2024-07-03T15:45:23.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.385+0000 7fd13affd640 1 --2- 172.21.15.28:0/3027512004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd13c0fc510 0x7fd13c10d490 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd130004810 tx=0x7fd130009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:45:23.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.385+0000 7fd123fff640 1 -- 172.21.15.28:0/3027512004 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd130038070 con 0x7fd13c0fc510 2024-07-03T15:45:23.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.385+0000 7fd123fff640 1 -- 172.21.15.28:0/3027512004 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd13002ad70 con 0x7fd13c0fc510 2024-07-03T15:45:23.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.386+0000 7fd1420d4640 1 -- 172.21.15.28:0/3027512004 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd13c10e530 con 0x7fd13c0fc510 2024-07-03T15:45:23.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.386+0000 7fd123fff640 1 -- 172.21.15.28:0/3027512004 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd130033660 con 0x7fd13c0fc510 2024-07-03T15:45:23.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.386+0000 7fd1420d4640 1 -- 172.21.15.28:0/3027512004 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd13c10e990 con 0x7fd13c0fc510 2024-07-03T15:45:23.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.387+0000 7fd1420d4640 1 -- 172.21.15.28:0/3027512004 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd13c0fa6a0 con 0x7fd13c0fc510 2024-07-03T15:45:23.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.388+0000 7fd123fff640 1 -- 172.21.15.28:0/3027512004 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 25) v1 ==== 92029+0+0 (secure 0 0 0) 0x7fd130030020 con 0x7fd13c0fc510 2024-07-03T15:45:23.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.389+0000 7fd123fff640 1 --2- 172.21.15.28:0/3027512004 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fd12406db30 0x7fd12406ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:45:23.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.389+0000 7fd123fff640 1 -- 172.21.15.28:0/3027512004 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fd1300ace20 con 0x7fd13c0fc510 2024-07-03T15:45:23.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.389+0000 7fd13b7fe640 1 --2- 172.21.15.28:0/3027512004 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fd12406db30 0x7fd12406ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:45:23.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.390+0000 7fd13b7fe640 1 --2- 172.21.15.28:0/3027512004 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fd12406db30 0x7fd12406ffe0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd12c009af0 tx=0x7fd12c00b040 comp rx=0 tx=0).ready entity=mgr.24403 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:45:23.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.393+0000 7fd123fff640 1 -- 172.21.15.28:0/3027512004 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7fd130076a50 con 0x7fd13c0fc510 2024-07-03T15:45:23.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.691+0000 7fd1420d4640 1 -- 172.21.15.28:0/3027512004 --> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd13c10ede0 con 0x7fd12406db30 2024-07-03T15:45:23.696 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:23 smithi028 ceph-mon[29148]: pgmap v43: 65 pgs: 65 active+clean; 284 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 984 op/s 2024-07-03T15:45:23.696 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:23 smithi028 ceph-mon[29148]: from='client.24477 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:45:23.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.701+0000 7fd123fff640 1 -- 172.21.15.28:0/3027512004 <== mgr.24403 v2:172.21.15.148:6828/4285739175 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fd13c10ede0 con 0x7fd12406db30 2024-07-03T15:45:23.706 INFO:teuthology.orchestra.run.smithi028.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-03T15:45:23.706 INFO:teuthology.orchestra.run.smithi028.stdout:alertmanager.smithi028 smithi028 *:9093,9094 running (49s) 0s ago 8m 17.8M - 0.25.0 c8568f914cd2 1425315a0de9 2024-07-03T15:45:23.706 INFO:teuthology.orchestra.run.smithi028.stdout:crash.smithi028 smithi028 running (9m) 0s ago 9m 6899k - 17.2.7-1047-gd4b063ab 4ca399d9e65d 535660e448f9 2024-07-03T15:45:23.706 INFO:teuthology.orchestra.run.smithi028.stdout:crash.smithi148 smithi148 running (8m) 11s ago 8m 6899k - 17.2.7-1047-gd4b063ab 4ca399d9e65d 4bb7675895e7 2024-07-03T15:45:23.706 INFO:teuthology.orchestra.run.smithi028.stdout:grafana.smithi028 smithi028 *:3000 running (44s) 0s ago 8m 73.9M - 9.4.7 954c08fa6188 f9f17c92cfea 2024-07-03T15:45:23.706 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi028.rhpvve smithi028 running (3m) 0s ago 3m 211M - 17.2.7-1047-gd4b063ab 4ca399d9e65d 3ed5abd02cfb 2024-07-03T15:45:23.706 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi028.wmbxew smithi028 running (3m) 0s ago 3m 146M - 17.2.7-1047-gd4b063ab 4ca399d9e65d b3bf736998f9 2024-07-03T15:45:23.706 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi148.szomfl smithi148 running (3m) 11s ago 3m 14.4M - 17.2.7-1047-gd4b063ab 4ca399d9e65d d94f5c95fa8d 2024-07-03T15:45:23.706 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi148.xgpuld smithi148 running (3m) 11s ago 3m 12.2M - 17.2.7-1047-gd4b063ab 4ca399d9e65d fc70360a8de0 2024-07-03T15:45:23.707 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.smithi028.vtqrtz smithi028 *:8443,9283,8765 running (14s) 0s ago 10m 53.2M - 19.0.0-2550-gc00506dc 9d49405603e8 14d0363c5b77 2024-07-03T15:45:23.707 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.smithi148.rjzsar smithi148 *:8443,9283 running (96s) 11s ago 8m 497M - 19.0.0-2550-gc00506dc 9d49405603e8 1fe24bbddd1a 2024-07-03T15:45:23.707 INFO:teuthology.orchestra.run.smithi028.stdout:mon.smithi028 smithi028 running (10m) 0s ago 10m 50.9M 2048M 17.2.7-1047-gd4b063ab 4ca399d9e65d 7461f9bef4e0 2024-07-03T15:45:23.707 INFO:teuthology.orchestra.run.smithi028.stdout:mon.smithi148 smithi148 running (7m) 11s ago 7m 41.3M 2048M 17.2.7-1047-gd4b063ab 4ca399d9e65d 4c6c8ed2ab85 2024-07-03T15:45:23.707 INFO:teuthology.orchestra.run.smithi028.stdout:node-exporter.smithi028 smithi028 *:9100 running (55s) 0s ago 9m 15.1M - 1.5.0 0da6a335fe13 91ca1f49de0e 2024-07-03T15:45:23.707 INFO:teuthology.orchestra.run.smithi028.stdout:node-exporter.smithi148 smithi148 *:9100 running (28s) 11s ago 8m 8154k - 1.5.0 0da6a335fe13 1eaaf7b1b561 2024-07-03T15:45:23.707 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0 smithi028 running (7m) 0s ago 7m 220M 1558M 17.2.7-1047-gd4b063ab 4ca399d9e65d b18089a439ba 2024-07-03T15:45:23.707 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1 smithi028 running (6m) 0s ago 6m 227M 1558M 17.2.7-1047-gd4b063ab 4ca399d9e65d 87d415dde231 2024-07-03T15:45:23.707 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2 smithi028 running (6m) 0s ago 6m 187M 1558M 17.2.7-1047-gd4b063ab 4ca399d9e65d b2cbb312212d 2024-07-03T15:45:23.707 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3 smithi148 running (6m) 11s ago 6m 248M 2581M 17.2.7-1047-gd4b063ab 4ca399d9e65d 5be69a27df31 2024-07-03T15:45:23.707 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4 smithi148 running (5m) 11s ago 5m 220M 2581M 17.2.7-1047-gd4b063ab 4ca399d9e65d 54f7c26c77ca 2024-07-03T15:45:23.707 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5 smithi148 running (5m) 11s ago 5m 212M 2581M 17.2.7-1047-gd4b063ab 4ca399d9e65d d185c6905384 2024-07-03T15:45:23.708 INFO:teuthology.orchestra.run.smithi028.stdout:prometheus.smithi028 smithi028 *:9095 running (35s) 0s ago 8m 46.6M - 2.43.0 a07b618ecd1d 57f08be67c0f 2024-07-03T15:45:23.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.704+0000 7fd1420d4640 1 -- 172.21.15.28:0/3027512004 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fd12406db30 msgr2=0x7fd12406ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:23.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.704+0000 7fd1420d4640 1 --2- 172.21.15.28:0/3027512004 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fd12406db30 0x7fd12406ffe0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd12c009af0 tx=0x7fd12c00b040 comp rx=0 tx=0).stop 2024-07-03T15:45:23.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.704+0000 7fd1420d4640 1 -- 172.21.15.28:0/3027512004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd13c0fc510 msgr2=0x7fd13c10d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:23.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.704+0000 7fd1420d4640 1 --2- 172.21.15.28:0/3027512004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd13c0fc510 0x7fd13c10d490 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd130004810 tx=0x7fd130009da0 comp rx=0 tx=0).stop 2024-07-03T15:45:23.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.704+0000 7fd1420d4640 1 -- 172.21.15.28:0/3027512004 shutdown_connections 2024-07-03T15:45:23.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.704+0000 7fd1420d4640 1 --2- 172.21.15.28:0/3027512004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd13c0fc510 0x7fd13c10d490 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:23.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.704+0000 7fd1420d4640 1 --2- 172.21.15.28:0/3027512004 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fd12406db30 0x7fd12406ffe0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:23.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.704+0000 7fd1420d4640 1 --2- 172.21.15.28:0/3027512004 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd13c0fb240 0x7fd13c10cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:23.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.704+0000 7fd1420d4640 1 -- 172.21.15.28:0/3027512004 >> 172.21.15.28:0/3027512004 conn(0x7fd13c0f6a20 msgr2=0x7fd13c0f80a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:45:23.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.704+0000 7fd1420d4640 1 -- 172.21.15.28:0/3027512004 shutdown_connections 2024-07-03T15:45:23.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.704+0000 7fd1420d4640 1 -- 172.21.15.28:0/3027512004 wait complete. 2024-07-03T15:45:23.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.832+0000 7f8d63981640 1 -- 172.21.15.28:0/3742211804 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d5c0ff4f0 msgr2=0x7f8d5c1018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:23.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.832+0000 7f8d63981640 1 --2- 172.21.15.28:0/3742211804 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d5c0ff4f0 0x7f8d5c1018d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c0048d0 tx=0x7f8d4c009da0 comp rx=0 tx=0).stop 2024-07-03T15:45:23.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.832+0000 7f8d63981640 1 -- 172.21.15.28:0/3742211804 shutdown_connections 2024-07-03T15:45:23.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.832+0000 7f8d63981640 1 --2- 172.21.15.28:0/3742211804 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d5c0ff4f0 0x7f8d5c1018d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:23.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.832+0000 7f8d63981640 1 --2- 172.21.15.28:0/3742211804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d5c0659f0 0x7f8d5c0fefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:23.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.832+0000 7f8d63981640 1 -- 172.21.15.28:0/3742211804 >> 172.21.15.28:0/3742211804 conn(0x7f8d5c0f6a20 msgr2=0x7f8d5c0f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:45:23.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.832+0000 7f8d63981640 1 -- 172.21.15.28:0/3742211804 shutdown_connections 2024-07-03T15:45:23.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.832+0000 7f8d63981640 1 -- 172.21.15.28:0/3742211804 wait complete. 2024-07-03T15:45:23.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.833+0000 7f8d63981640 1 Processor -- start 2024-07-03T15:45:23.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.833+0000 7f8d63981640 1 -- start start 2024-07-03T15:45:23.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.834+0000 7f8d63981640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d5c0659f0 0x7f8d5c10cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:45:23.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.834+0000 7f8d63981640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d5c0ff4f0 0x7f8d5c10d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:45:23.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.834+0000 7f8d63981640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5c10db30 con 0x7f8d5c0ff4f0 2024-07-03T15:45:23.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.834+0000 7f8d63981640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5c10e300 con 0x7f8d5c0659f0 2024-07-03T15:45:23.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.834+0000 7f8d60f1c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d5c0ff4f0 0x7f8d5c10d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:45:23.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.834+0000 7f8d6171d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d5c0659f0 0x7f8d5c10cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:45:23.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.834+0000 7f8d60f1c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d5c0ff4f0 0x7f8d5c10d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50488/0 (socket says 172.21.15.28:50488) 2024-07-03T15:45:23.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.834+0000 7f8d60f1c640 1 -- 172.21.15.28:0/364103162 learned_addr learned my addr 172.21.15.28:0/364103162 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:45:23.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.834+0000 7f8d60f1c640 1 -- 172.21.15.28:0/364103162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d5c0659f0 msgr2=0x7f8d5c10cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:23.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.834+0000 7f8d60f1c640 1 --2- 172.21.15.28:0/364103162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d5c0659f0 0x7f8d5c10cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:23.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.834+0000 7f8d60f1c640 1 -- 172.21.15.28:0/364103162 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d4c005040 con 0x7f8d5c0ff4f0 2024-07-03T15:45:23.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.835+0000 7f8d60f1c640 1 --2- 172.21.15.28:0/364103162 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d5c0ff4f0 0x7f8d5c10d4e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c0048a0 tx=0x7f8d4c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:45:23.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.835+0000 7f8d51ffb640 1 -- 172.21.15.28:0/364103162 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d4c038070 con 0x7f8d5c0ff4f0 2024-07-03T15:45:23.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.835+0000 7f8d63981640 1 -- 172.21.15.28:0/364103162 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d5c10e530 con 0x7f8d5c0ff4f0 2024-07-03T15:45:23.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.835+0000 7f8d51ffb640 1 -- 172.21.15.28:0/364103162 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d4c032d60 con 0x7f8d5c0ff4f0 2024-07-03T15:45:23.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.835+0000 7f8d51ffb640 1 -- 172.21.15.28:0/364103162 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d4c03ed00 con 0x7f8d5c0ff4f0 2024-07-03T15:45:23.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.835+0000 7f8d63981640 1 -- 172.21.15.28:0/364103162 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d5c10ea50 con 0x7f8d5c0ff4f0 2024-07-03T15:45:23.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.837+0000 7f8d63981640 1 -- 172.21.15.28:0/364103162 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d5c0fa6a0 con 0x7f8d5c0ff4f0 2024-07-03T15:45:23.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.838+0000 7f8d51ffb640 1 -- 172.21.15.28:0/364103162 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 25) v1 ==== 92029+0+0 (secure 0 0 0) 0x7f8d4c030020 con 0x7f8d5c0ff4f0 2024-07-03T15:45:23.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.839+0000 7f8d51ffb640 1 --2- 172.21.15.28:0/364103162 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f8d4806dae0 0x7f8d4806ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:45:23.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.839+0000 7f8d51ffb640 1 -- 172.21.15.28:0/364103162 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f8d4c0ad910 con 0x7f8d5c0ff4f0 2024-07-03T15:45:23.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.839+0000 7f8d6171d640 1 --2- 172.21.15.28:0/364103162 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f8d4806dae0 0x7f8d4806ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:45:23.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.839+0000 7f8d6171d640 1 --2- 172.21.15.28:0/364103162 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f8d4806dae0 0x7f8d4806ff90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8d58009af0 tx=0x7f8d5800b040 comp rx=0 tx=0).ready entity=mgr.24403 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:45:23.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:23.846+0000 7f8d51ffb640 1 -- 172.21.15.28:0/364103162 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7f8d4c0775b0 con 0x7f8d5c0ff4f0 2024-07-03T15:45:23.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:23 smithi148 ceph-mon[36056]: pgmap v43: 65 pgs: 65 active+clean; 284 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 984 op/s 2024-07-03T15:45:23.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:23 smithi148 ceph-mon[36056]: from='client.24477 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:45:24.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.264+0000 7f8d63981640 1 -- 172.21.15.28:0/364103162 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8d5c10ec30 con 0x7f8d5c0ff4f0 2024-07-03T15:45:24.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.264+0000 7f8d51ffb640 1 -- 172.21.15.28:0/364103162 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+741 (secure 0 0 0) 0x7f8d4c03ee60 con 0x7f8d5c0ff4f0 2024-07-03T15:45:24.268 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-07-03T15:45:24.268 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": { 2024-07-03T15:45:24.269 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 2 2024-07-03T15:45:24.269 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:45:24.269 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": { 2024-07-03T15:45:24.269 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 2 2024-07-03T15:45:24.269 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:45:24.269 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": { 2024-07-03T15:45:24.269 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 6 2024-07-03T15:45:24.269 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:45:24.269 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": { 2024-07-03T15:45:24.269 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 4 2024-07-03T15:45:24.269 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:45:24.269 INFO:teuthology.orchestra.run.smithi028.stdout: "overall": { 2024-07-03T15:45:24.269 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 12, 2024-07-03T15:45:24.270 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 2 2024-07-03T15:45:24.270 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-07-03T15:45:24.270 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-07-03T15:45:24.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.268+0000 7f8d63981640 1 -- 172.21.15.28:0/364103162 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f8d4806dae0 msgr2=0x7f8d4806ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:24.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.268+0000 7f8d63981640 1 --2- 172.21.15.28:0/364103162 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f8d4806dae0 0x7f8d4806ff90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8d58009af0 tx=0x7f8d5800b040 comp rx=0 tx=0).stop 2024-07-03T15:45:24.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.268+0000 7f8d63981640 1 -- 172.21.15.28:0/364103162 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d5c0ff4f0 msgr2=0x7f8d5c10d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:24.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.268+0000 7f8d63981640 1 --2- 172.21.15.28:0/364103162 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d5c0ff4f0 0x7f8d5c10d4e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c0048a0 tx=0x7f8d4c009da0 comp rx=0 tx=0).stop 2024-07-03T15:45:24.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.268+0000 7f8d63981640 1 -- 172.21.15.28:0/364103162 shutdown_connections 2024-07-03T15:45:24.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.268+0000 7f8d63981640 1 --2- 172.21.15.28:0/364103162 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7f8d4806dae0 0x7f8d4806ff90 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:24.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.268+0000 7f8d63981640 1 --2- 172.21.15.28:0/364103162 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d5c0ff4f0 0x7f8d5c10d4e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:24.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.268+0000 7f8d63981640 1 --2- 172.21.15.28:0/364103162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d5c0659f0 0x7f8d5c10cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:24.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.268+0000 7f8d63981640 1 -- 172.21.15.28:0/364103162 >> 172.21.15.28:0/364103162 conn(0x7f8d5c0f6a20 msgr2=0x7f8d5c0f6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:45:24.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.268+0000 7f8d63981640 1 -- 172.21.15.28:0/364103162 shutdown_connections 2024-07-03T15:45:24.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.268+0000 7f8d63981640 1 -- 172.21.15.28:0/364103162 wait complete. 2024-07-03T15:45:24.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.373+0000 7fda6a33c640 1 -- 172.21.15.28:0/3403902817 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda640fc510 msgr2=0x7fda640fc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:24.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.373+0000 7fda6a33c640 1 --2- 172.21.15.28:0/3403902817 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda640fc510 0x7fda640fc980 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fda580048d0 tx=0x7fda58009da0 comp rx=0 tx=0).stop 2024-07-03T15:45:24.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.373+0000 7fda6a33c640 1 -- 172.21.15.28:0/3403902817 shutdown_connections 2024-07-03T15:45:24.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.373+0000 7fda6a33c640 1 --2- 172.21.15.28:0/3403902817 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda640fc510 0x7fda640fc980 secure :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fda580048d0 tx=0x7fda58009da0 comp rx=0 tx=0).stop 2024-07-03T15:45:24.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.373+0000 7fda6a33c640 1 --2- 172.21.15.28:0/3403902817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda640fb240 0x7fda640fb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:24.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.373+0000 7fda6a33c640 1 -- 172.21.15.28:0/3403902817 >> 172.21.15.28:0/3403902817 conn(0x7fda640f6a20 msgr2=0x7fda640f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:45:24.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.374+0000 7fda6a33c640 1 -- 172.21.15.28:0/3403902817 shutdown_connections 2024-07-03T15:45:24.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.374+0000 7fda6a33c640 1 -- 172.21.15.28:0/3403902817 wait complete. 2024-07-03T15:45:24.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.374+0000 7fda6a33c640 1 Processor -- start 2024-07-03T15:45:24.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.375+0000 7fda6a33c640 1 -- start start 2024-07-03T15:45:24.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.375+0000 7fda6a33c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda640fb240 0x7fda6410d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:45:24.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.375+0000 7fda6a33c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda6410d600 0x7fda6410e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:45:24.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.375+0000 7fda6a33c640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6410dac0 con 0x7fda6410d600 2024-07-03T15:45:24.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.375+0000 7fda6a33c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6410dc00 con 0x7fda640fb240 2024-07-03T15:45:24.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.375+0000 7fda637fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda6410d600 0x7fda6410e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:45:24.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.375+0000 7fda637fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda6410d600 0x7fda6410e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50518/0 (socket says 172.21.15.28:50518) 2024-07-03T15:45:24.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.375+0000 7fda637fe640 1 -- 172.21.15.28:0/517719737 learned_addr learned my addr 172.21.15.28:0/517719737 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:45:24.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.375+0000 7fda63fff640 1 --2- 172.21.15.28:0/517719737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda640fb240 0x7fda6410d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:45:24.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.376+0000 7fda637fe640 1 -- 172.21.15.28:0/517719737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda640fb240 msgr2=0x7fda6410d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:24.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.376+0000 7fda637fe640 1 --2- 172.21.15.28:0/517719737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda640fb240 0x7fda6410d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:24.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.376+0000 7fda637fe640 1 -- 172.21.15.28:0/517719737 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda58005040 con 0x7fda6410d600 2024-07-03T15:45:24.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.376+0000 7fda637fe640 1 --2- 172.21.15.28:0/517719737 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda6410d600 0x7fda6410e470 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fda58009040 tx=0x7fda58009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:45:24.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.376+0000 7fda60ff9640 1 -- 172.21.15.28:0/517719737 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda58038070 con 0x7fda6410d600 2024-07-03T15:45:24.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.376+0000 7fda6a33c640 1 -- 172.21.15.28:0/517719737 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda6410ea10 con 0x7fda6410d600 2024-07-03T15:45:24.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.376+0000 7fda6a33c640 1 -- 172.21.15.28:0/517719737 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda6410ef50 con 0x7fda6410d600 2024-07-03T15:45:24.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.376+0000 7fda60ff9640 1 -- 172.21.15.28:0/517719737 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fda580092b0 con 0x7fda6410d600 2024-07-03T15:45:24.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.376+0000 7fda60ff9640 1 -- 172.21.15.28:0/517719737 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda58033660 con 0x7fda6410d600 2024-07-03T15:45:24.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.378+0000 7fda6a33c640 1 -- 172.21.15.28:0/517719737 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda640fa6a0 con 0x7fda6410d600 2024-07-03T15:45:24.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.378+0000 7fda60ff9640 1 -- 172.21.15.28:0/517719737 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 25) v1 ==== 92029+0+0 (secure 0 0 0) 0x7fda58030020 con 0x7fda6410d600 2024-07-03T15:45:24.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.379+0000 7fda60ff9640 1 --2- 172.21.15.28:0/517719737 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fda4c06dae0 0x7fda4c06ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:45:24.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.379+0000 7fda60ff9640 1 -- 172.21.15.28:0/517719737 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fda580ace50 con 0x7fda6410d600 2024-07-03T15:45:24.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.379+0000 7fda63fff640 1 --2- 172.21.15.28:0/517719737 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fda4c06dae0 0x7fda4c06ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:45:24.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.380+0000 7fda63fff640 1 --2- 172.21.15.28:0/517719737 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fda4c06dae0 0x7fda4c06ff90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fda54009af0 tx=0x7fda5400b040 comp rx=0 tx=0).ready entity=mgr.24403 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:45:24.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.388+0000 7fda60ff9640 1 -- 172.21.15.28:0/517719737 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7fda58076ba0 con 0x7fda6410d600 2024-07-03T15:45:24.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.725+0000 7fda6a33c640 1 -- 172.21.15.28:0/517719737 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fda6404b530 con 0x7fda6410d600 2024-07-03T15:45:24.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.725+0000 7fda60ff9640 1 -- 172.21.15.28:0/517719737 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1838 (secure 0 0 0) 0x7fda5807a260 con 0x7fda6410d600 2024-07-03T15:45:24.729 INFO:teuthology.orchestra.run.smithi028.stdout:e17 2024-07-03T15:45:24.729 INFO:teuthology.orchestra.run.smithi028.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-03T15:45:24.730 INFO:teuthology.orchestra.run.smithi028.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:45:24.730 INFO:teuthology.orchestra.run.smithi028.stdout:legacy client fscid: 1 2024-07-03T15:45:24.730 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:45:24.730 INFO:teuthology.orchestra.run.smithi028.stdout:Filesystem 'cephfs' (1) 2024-07-03T15:45:24.730 INFO:teuthology.orchestra.run.smithi028.stdout:fs_name cephfs 2024-07-03T15:45:24.730 INFO:teuthology.orchestra.run.smithi028.stdout:epoch 16 2024-07-03T15:45:24.730 INFO:teuthology.orchestra.run.smithi028.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-03T15:45:24.730 INFO:teuthology.orchestra.run.smithi028.stdout:created 2024-07-03T15:41:40.656298+0000 2024-07-03T15:45:24.730 INFO:teuthology.orchestra.run.smithi028.stdout:modified 2024-07-03T15:41:56.454984+0000 2024-07-03T15:45:24.730 INFO:teuthology.orchestra.run.smithi028.stdout:tableserver 0 2024-07-03T15:45:24.730 INFO:teuthology.orchestra.run.smithi028.stdout:root 0 2024-07-03T15:45:24.730 INFO:teuthology.orchestra.run.smithi028.stdout:session_timeout 60 2024-07-03T15:45:24.731 INFO:teuthology.orchestra.run.smithi028.stdout:session_autoclose 300 2024-07-03T15:45:24.731 INFO:teuthology.orchestra.run.smithi028.stdout:max_file_size 1099511627776 2024-07-03T15:45:24.731 INFO:teuthology.orchestra.run.smithi028.stdout:required_client_features {} 2024-07-03T15:45:24.731 INFO:teuthology.orchestra.run.smithi028.stdout:last_failure 0 2024-07-03T15:45:24.731 INFO:teuthology.orchestra.run.smithi028.stdout:last_failure_osd_epoch 0 2024-07-03T15:45:24.731 INFO:teuthology.orchestra.run.smithi028.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:45:24.731 INFO:teuthology.orchestra.run.smithi028.stdout:max_mds 1 2024-07-03T15:45:24.731 INFO:teuthology.orchestra.run.smithi028.stdout:in 0 2024-07-03T15:45:24.731 INFO:teuthology.orchestra.run.smithi028.stdout:up {0=14450} 2024-07-03T15:45:24.731 INFO:teuthology.orchestra.run.smithi028.stdout:failed 2024-07-03T15:45:24.731 INFO:teuthology.orchestra.run.smithi028.stdout:damaged 2024-07-03T15:45:24.731 INFO:teuthology.orchestra.run.smithi028.stdout:stopped 2024-07-03T15:45:24.731 INFO:teuthology.orchestra.run.smithi028.stdout:data_pools [3] 2024-07-03T15:45:24.732 INFO:teuthology.orchestra.run.smithi028.stdout:metadata_pool 2 2024-07-03T15:45:24.732 INFO:teuthology.orchestra.run.smithi028.stdout:inline_data disabled 2024-07-03T15:45:24.732 INFO:teuthology.orchestra.run.smithi028.stdout:balancer 2024-07-03T15:45:24.732 INFO:teuthology.orchestra.run.smithi028.stdout:standby_count_wanted 1 2024-07-03T15:45:24.732 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi028.rhpvve{0:14450} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.28:6826/1613936042,v1:172.21.15.28:6827/1613936042] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:45:24.732 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi028.wmbxew{0:14460} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.28:6828/1730336821,v1:172.21.15.28:6829/1730336821] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:45:24.732 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:45:24.732 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:45:24.732 INFO:teuthology.orchestra.run.smithi028.stdout:Standby daemons: 2024-07-03T15:45:24.732 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:45:24.732 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi148.szomfl{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/4171231858,v1:172.21.15.148:6827/4171231858] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:45:24.732 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi148.xgpuld{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/2347536398,v1:172.21.15.148:6825/2347536398] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:45:24.733 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 17 2024-07-03T15:45:24.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.729+0000 7fda6a33c640 1 -- 172.21.15.28:0/517719737 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fda4c06dae0 msgr2=0x7fda4c06ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:24.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.729+0000 7fda6a33c640 1 --2- 172.21.15.28:0/517719737 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fda4c06dae0 0x7fda4c06ff90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fda54009af0 tx=0x7fda5400b040 comp rx=0 tx=0).stop 2024-07-03T15:45:24.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.729+0000 7fda6a33c640 1 -- 172.21.15.28:0/517719737 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda6410d600 msgr2=0x7fda6410e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:24.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.729+0000 7fda6a33c640 1 --2- 172.21.15.28:0/517719737 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda6410d600 0x7fda6410e470 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fda58009040 tx=0x7fda58009da0 comp rx=0 tx=0).stop 2024-07-03T15:45:24.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.729+0000 7fda6a33c640 1 -- 172.21.15.28:0/517719737 shutdown_connections 2024-07-03T15:45:24.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.729+0000 7fda6a33c640 1 --2- 172.21.15.28:0/517719737 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fda4c06dae0 0x7fda4c06ff90 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:24.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.729+0000 7fda6a33c640 1 --2- 172.21.15.28:0/517719737 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda6410d600 0x7fda6410e470 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:24.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.729+0000 7fda6a33c640 1 --2- 172.21.15.28:0/517719737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda640fb240 0x7fda6410d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:24.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.729+0000 7fda6a33c640 1 -- 172.21.15.28:0/517719737 >> 172.21.15.28:0/517719737 conn(0x7fda640f6a20 msgr2=0x7fda64065480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:45:24.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.729+0000 7fda6a33c640 1 -- 172.21.15.28:0/517719737 shutdown_connections 2024-07-03T15:45:24.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.729+0000 7fda6a33c640 1 -- 172.21.15.28:0/517719737 wait complete. 2024-07-03T15:45:24.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.853+0000 7fe89ba24640 1 -- 172.21.15.28:0/921237461 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe894102970 msgr2=0x7fe894102d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:24.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.853+0000 7fe89ba24640 1 --2- 172.21.15.28:0/921237461 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe894102970 0x7fe894102d80 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe884009c50 tx=0x7fe884004d10 comp rx=0 tx=0).stop 2024-07-03T15:45:24.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.853+0000 7fe89ba24640 1 -- 172.21.15.28:0/921237461 shutdown_connections 2024-07-03T15:45:24.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.853+0000 7fe89ba24640 1 --2- 172.21.15.28:0/921237461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe894103c40 0x7fe8941040b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:24.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.853+0000 7fe89ba24640 1 --2- 172.21.15.28:0/921237461 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe894102970 0x7fe894102d80 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:24.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.853+0000 7fe89ba24640 1 -- 172.21.15.28:0/921237461 >> 172.21.15.28:0/921237461 conn(0x7fe894071bd0 msgr2=0x7fe894074000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:45:24.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.853+0000 7fe89ba24640 1 -- 172.21.15.28:0/921237461 shutdown_connections 2024-07-03T15:45:24.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.853+0000 7fe89ba24640 1 -- 172.21.15.28:0/921237461 wait complete. 2024-07-03T15:45:24.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.854+0000 7fe89ba24640 1 Processor -- start 2024-07-03T15:45:24.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.854+0000 7fe89ba24640 1 -- start start 2024-07-03T15:45:24.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.855+0000 7fe89ba24640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe894102970 0x7fe894113350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:45:24.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.855+0000 7fe89ba24640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe894103c40 0x7fe894113890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:45:24.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.855+0000 7fe89ba24640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe894113ee0 con 0x7fe894103c40 2024-07-03T15:45:24.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.855+0000 7fe89ba24640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe894114700 con 0x7fe894102970 2024-07-03T15:45:24.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.855+0000 7fe8997c0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe894102970 0x7fe894113350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:45:24.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.855+0000 7fe8997c0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe894102970 0x7fe894113350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.28:43728/0 (socket says 172.21.15.28:43728) 2024-07-03T15:45:24.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.855+0000 7fe8997c0640 1 -- 172.21.15.28:0/7317393 learned_addr learned my addr 172.21.15.28:0/7317393 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:45:24.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.855+0000 7fe898fbf640 1 --2- 172.21.15.28:0/7317393 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe894103c40 0x7fe894113890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:45:24.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.856+0000 7fe8997c0640 1 -- 172.21.15.28:0/7317393 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe894103c40 msgr2=0x7fe894113890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:24.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.856+0000 7fe8997c0640 1 --2- 172.21.15.28:0/7317393 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe894103c40 0x7fe894113890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:24.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.856+0000 7fe8997c0640 1 -- 172.21.15.28:0/7317393 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8840098b0 con 0x7fe894102970 2024-07-03T15:45:24.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.856+0000 7fe8997c0640 1 --2- 172.21.15.28:0/7317393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe894102970 0x7fe894113350 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe88402a8b0 tx=0x7fe884004d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:45:24.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.857+0000 7fe891ffb640 1 -- 172.21.15.28:0/7317393 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe884032910 con 0x7fe894102970 2024-07-03T15:45:24.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.857+0000 7fe891ffb640 1 -- 172.21.15.28:0/7317393 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe88403d420 con 0x7fe894102970 2024-07-03T15:45:24.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.857+0000 7fe89ba24640 1 -- 172.21.15.28:0/7317393 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe894114930 con 0x7fe894102970 2024-07-03T15:45:24.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.857+0000 7fe891ffb640 1 -- 172.21.15.28:0/7317393 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe884031610 con 0x7fe894102970 2024-07-03T15:45:24.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.857+0000 7fe89ba24640 1 -- 172.21.15.28:0/7317393 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe894114d30 con 0x7fe894102970 2024-07-03T15:45:24.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.859+0000 7fe891ffb640 1 -- 172.21.15.28:0/7317393 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 25) v1 ==== 92029+0+0 (secure 0 0 0) 0x7fe88402f030 con 0x7fe894102970 2024-07-03T15:45:24.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.860+0000 7fe891ffb640 1 --2- 172.21.15.28:0/7317393 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fe87c06dbb0 0x7fe87c070060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:45:24.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.860+0000 7fe891ffb640 1 -- 172.21.15.28:0/7317393 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fe8840ae710 con 0x7fe894102970 2024-07-03T15:45:24.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.860+0000 7fe898fbf640 1 --2- 172.21.15.28:0/7317393 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fe87c06dbb0 0x7fe87c070060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:45:24.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.861+0000 7fe89ba24640 1 -- 172.21.15.28:0/7317393 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe894075800 con 0x7fe894102970 2024-07-03T15:45:24.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.861+0000 7fe898fbf640 1 --2- 172.21.15.28:0/7317393 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fe87c06dbb0 0x7fe87c070060 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe894113100 tx=0x7fe88800b040 comp rx=0 tx=0).ready entity=mgr.24403 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:45:24.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:24.866+0000 7fe891ffb640 1 -- 172.21.15.28:0/7317393 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7fe8840782c0 con 0x7fe894102970 2024-07-03T15:45:24.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:24 smithi148 ceph-mon[36056]: from='client.14708 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:45:24.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:24 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:45:24.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:24 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:45:24.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:24 smithi148 ceph-mon[36056]: from='client.24483 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:45:24.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:24 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/364103162' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:45:24.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:24 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:45:24.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:24 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:45:25.030 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:24 smithi028 ceph-mon[29148]: from='client.14708 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:45:25.031 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:24 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:45:25.031 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:24 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:45:25.031 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:24 smithi028 ceph-mon[29148]: from='client.24483 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:45:25.031 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:24 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/364103162' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:45:25.031 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:24 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:45:25.031 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:24 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:45:25.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.209+0000 7fe89ba24640 1 -- 172.21.15.28:0/7317393 --> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe894115070 con 0x7fe87c06dbb0 2024-07-03T15:45:25.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.211+0000 7fe891ffb640 1 -- 172.21.15.28:0/7317393 <== mgr.24403 v2:172.21.15.148:6828/4285739175 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fe894115070 con 0x7fe87c06dbb0 2024-07-03T15:45:25.215 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-07-03T15:45:25.215 INFO:teuthology.orchestra.run.smithi028.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2", 2024-07-03T15:45:25.216 INFO:teuthology.orchestra.run.smithi028.stdout: "in_progress": true, 2024-07-03T15:45:25.216 INFO:teuthology.orchestra.run.smithi028.stdout: "which": "Upgrading all daemon types on all hosts", 2024-07-03T15:45:25.216 INFO:teuthology.orchestra.run.smithi028.stdout: "services_complete": [ 2024-07-03T15:45:25.216 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr" 2024-07-03T15:45:25.216 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-07-03T15:45:25.216 INFO:teuthology.orchestra.run.smithi028.stdout: "progress": "2/21 daemons upgraded", 2024-07-03T15:45:25.216 INFO:teuthology.orchestra.run.smithi028.stdout: "message": "Currently upgrading mgr daemons", 2024-07-03T15:45:25.216 INFO:teuthology.orchestra.run.smithi028.stdout: "is_paused": false 2024-07-03T15:45:25.216 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-07-03T15:45:25.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.215+0000 7fe89ba24640 1 -- 172.21.15.28:0/7317393 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fe87c06dbb0 msgr2=0x7fe87c070060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:25.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.215+0000 7fe89ba24640 1 --2- 172.21.15.28:0/7317393 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fe87c06dbb0 0x7fe87c070060 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe894113100 tx=0x7fe88800b040 comp rx=0 tx=0).stop 2024-07-03T15:45:25.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.215+0000 7fe89ba24640 1 -- 172.21.15.28:0/7317393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe894102970 msgr2=0x7fe894113350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:25.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.215+0000 7fe89ba24640 1 --2- 172.21.15.28:0/7317393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe894102970 0x7fe894113350 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe88402a8b0 tx=0x7fe884004d10 comp rx=0 tx=0).stop 2024-07-03T15:45:25.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.215+0000 7fe89ba24640 1 -- 172.21.15.28:0/7317393 shutdown_connections 2024-07-03T15:45:25.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.215+0000 7fe89ba24640 1 --2- 172.21.15.28:0/7317393 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fe87c06dbb0 0x7fe87c070060 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:25.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.215+0000 7fe89ba24640 1 --2- 172.21.15.28:0/7317393 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe894103c40 0x7fe894113890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:25.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.215+0000 7fe89ba24640 1 --2- 172.21.15.28:0/7317393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe894102970 0x7fe894113350 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:25.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.215+0000 7fe89ba24640 1 -- 172.21.15.28:0/7317393 >> 172.21.15.28:0/7317393 conn(0x7fe894071bd0 msgr2=0x7fe894072f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:45:25.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.215+0000 7fe89ba24640 1 -- 172.21.15.28:0/7317393 shutdown_connections 2024-07-03T15:45:25.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.215+0000 7fe89ba24640 1 -- 172.21.15.28:0/7317393 wait complete. 2024-07-03T15:45:25.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.326+0000 7fa831193640 1 -- 172.21.15.28:0/2394273704 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa820032720 con 0x7fa82c0fb230 2024-07-03T15:45:25.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.327+0000 7fa8343f9640 1 -- 172.21.15.28:0/2394273704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa82c0fb230 msgr2=0x7fa82c0fb640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:25.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.327+0000 7fa8343f9640 1 --2- 172.21.15.28:0/2394273704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa82c0fb230 0x7fa82c0fb640 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa8200048d0 tx=0x7fa820009da0 comp rx=0 tx=0).stop 2024-07-03T15:45:25.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.327+0000 7fa8343f9640 1 -- 172.21.15.28:0/2394273704 shutdown_connections 2024-07-03T15:45:25.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.327+0000 7fa8343f9640 1 --2- 172.21.15.28:0/2394273704 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa82c0fc500 0x7fa82c0fc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:25.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.327+0000 7fa8343f9640 1 --2- 172.21.15.28:0/2394273704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa82c0fb230 0x7fa82c0fb640 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:25.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.327+0000 7fa8343f9640 1 -- 172.21.15.28:0/2394273704 >> 172.21.15.28:0/2394273704 conn(0x7fa82c0f6a10 msgr2=0x7fa82c0f8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:45:25.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.327+0000 7fa8343f9640 1 -- 172.21.15.28:0/2394273704 shutdown_connections 2024-07-03T15:45:25.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.327+0000 7fa8343f9640 1 -- 172.21.15.28:0/2394273704 wait complete. 2024-07-03T15:45:25.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.328+0000 7fa8343f9640 1 Processor -- start 2024-07-03T15:45:25.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.329+0000 7fa8343f9640 1 -- start start 2024-07-03T15:45:25.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.330+0000 7fa8343f9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa82c0fb230 0x7fa82c10ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:45:25.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.330+0000 7fa8343f9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa82c0fc500 0x7fa82c10b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:45:25.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.330+0000 7fa8343f9640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c10b8a0 con 0x7fa82c0fb230 2024-07-03T15:45:25.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.330+0000 7fa8343f9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c10c110 con 0x7fa82c0fc500 2024-07-03T15:45:25.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.330+0000 7fa832195640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa82c0fb230 0x7fa82c10ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:45:25.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.330+0000 7fa832195640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa82c0fb230 0x7fa82c10ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50570/0 (socket says 172.21.15.28:50570) 2024-07-03T15:45:25.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.331+0000 7fa832195640 1 -- 172.21.15.28:0/2771070096 learned_addr learned my addr 172.21.15.28:0/2771070096 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:45:25.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.331+0000 7fa832195640 1 -- 172.21.15.28:0/2771070096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa82c0fc500 msgr2=0x7fa82c10b2a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-03T15:45:25.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.331+0000 7fa832195640 1 --2- 172.21.15.28:0/2771070096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa82c0fc500 0x7fa82c10b2a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:25.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.331+0000 7fa832195640 1 -- 172.21.15.28:0/2771070096 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa820005040 con 0x7fa82c0fb230 2024-07-03T15:45:25.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.332+0000 7fa832195640 1 --2- 172.21.15.28:0/2771070096 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa82c0fb230 0x7fa82c10ad60 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa8200041d0 tx=0x7fa820009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:45:25.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.332+0000 7fa81effd640 1 -- 172.21.15.28:0/2771070096 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa820038070 con 0x7fa82c0fb230 2024-07-03T15:45:25.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.332+0000 7fa81effd640 1 -- 172.21.15.28:0/2771070096 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa8200092b0 con 0x7fa82c0fb230 2024-07-03T15:45:25.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.332+0000 7fa81effd640 1 -- 172.21.15.28:0/2771070096 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8200336d0 con 0x7fa82c0fb230 2024-07-03T15:45:25.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.333+0000 7fa8343f9640 1 -- 172.21.15.28:0/2771070096 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa82c10c2b0 con 0x7fa82c0fb230 2024-07-03T15:45:25.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.333+0000 7fa8343f9640 1 -- 172.21.15.28:0/2771070096 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa82c10c850 con 0x7fa82c0fb230 2024-07-03T15:45:25.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.334+0000 7fa8343f9640 1 -- 172.21.15.28:0/2771070096 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa82c0fa690 con 0x7fa82c0fb230 2024-07-03T15:45:25.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.336+0000 7fa81effd640 1 -- 172.21.15.28:0/2771070096 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 25) v1 ==== 92029+0+0 (secure 0 0 0) 0x7fa820030020 con 0x7fa82c0fb230 2024-07-03T15:45:25.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.337+0000 7fa81effd640 1 --2- 172.21.15.28:0/2771070096 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fa81806db90 0x7fa818070040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:45:25.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.337+0000 7fa81effd640 1 -- 172.21.15.28:0/2771070096 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fa8200ad440 con 0x7fa82c0fb230 2024-07-03T15:45:25.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.340+0000 7fa831994640 1 --2- 172.21.15.28:0/2771070096 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fa81806db90 0x7fa818070040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:45:25.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.343+0000 7fa81effd640 1 -- 172.21.15.28:0/2771070096 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7fa8200770c0 con 0x7fa82c0fb230 2024-07-03T15:45:25.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.348+0000 7fa831994640 1 --2- 172.21.15.28:0/2771070096 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fa81806db90 0x7fa818070040 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa828004d40 tx=0x7fa82800b040 comp rx=0 tx=0).ready entity=mgr.24403 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:45:25.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.655+0000 7fa8343f9640 1 -- 172.21.15.28:0/2771070096 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa82c04b530 con 0x7fa82c0fb230 2024-07-03T15:45:25.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.655+0000 7fa81effd640 1 -- 172.21.15.28:0/2771070096 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fa82007a780 con 0x7fa82c0fb230 2024-07-03T15:45:25.659 INFO:teuthology.orchestra.run.smithi028.stdout:HEALTH_OK 2024-07-03T15:45:25.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.658+0000 7fa8343f9640 1 -- 172.21.15.28:0/2771070096 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fa81806db90 msgr2=0x7fa818070040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:25.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.658+0000 7fa8343f9640 1 --2- 172.21.15.28:0/2771070096 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fa81806db90 0x7fa818070040 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa828004d40 tx=0x7fa82800b040 comp rx=0 tx=0).stop 2024-07-03T15:45:25.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.658+0000 7fa8343f9640 1 -- 172.21.15.28:0/2771070096 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa82c0fb230 msgr2=0x7fa82c10ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:25.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.658+0000 7fa8343f9640 1 --2- 172.21.15.28:0/2771070096 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa82c0fb230 0x7fa82c10ad60 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa8200041d0 tx=0x7fa820009da0 comp rx=0 tx=0).stop 2024-07-03T15:45:25.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.658+0000 7fa8343f9640 1 -- 172.21.15.28:0/2771070096 shutdown_connections 2024-07-03T15:45:25.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.658+0000 7fa8343f9640 1 --2- 172.21.15.28:0/2771070096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa82c0fc500 0x7fa82c10b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:25.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.658+0000 7fa8343f9640 1 --2- 172.21.15.28:0/2771070096 >> [v2:172.21.15.148:6828/4285739175,v1:172.21.15.148:6829/4285739175] conn(0x7fa81806db90 0x7fa818070040 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:25.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.658+0000 7fa8343f9640 1 --2- 172.21.15.28:0/2771070096 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa82c0fb230 0x7fa82c10ad60 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:25.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.658+0000 7fa8343f9640 1 -- 172.21.15.28:0/2771070096 >> 172.21.15.28:0/2771070096 conn(0x7fa82c0f6a10 msgr2=0x7fa82c0f7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:45:25.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.658+0000 7fa8343f9640 1 -- 172.21.15.28:0/2771070096 shutdown_connections 2024-07-03T15:45:25.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:25.658+0000 7fa8343f9640 1 -- 172.21.15.28:0/2771070096 wait complete. 2024-07-03T15:45:25.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:25 smithi148 ceph-mon[36056]: pgmap v44: 65 pgs: 65 active+clean; 272 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 1.39k op/s 2024-07-03T15:45:25.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:25 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/517719737' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-03T15:45:26.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:25 smithi028 ceph-mon[29148]: pgmap v44: 65 pgs: 65 active+clean; 272 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 1.39k op/s 2024-07-03T15:45:26.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:25 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/517719737' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-03T15:45:26.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:26 smithi148 ceph-mon[36056]: from='client.24495 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:45:26.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:26 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/2771070096' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-03T15:45:26.996 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:26 smithi028 ceph-mon[29148]: from='client.24495 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:45:26.996 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:26 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/2771070096' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-03T15:45:28.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:27 smithi028 ceph-mon[29148]: pgmap v45: 65 pgs: 65 active+clean; 272 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 721 op/s 2024-07-03T15:45:28.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:27 smithi148 ceph-mon[36056]: pgmap v45: 65 pgs: 65 active+clean; 272 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 721 op/s 2024-07-03T15:45:30.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:29 smithi028 ceph-mon[29148]: pgmap v46: 65 pgs: 65 active+clean; 263 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 917 op/s 2024-07-03T15:45:30.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:29 smithi148 ceph-mon[36056]: pgmap v46: 65 pgs: 65 active+clean; 263 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 917 op/s 2024-07-03T15:45:31.958 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:31 smithi148 ceph-mon[36056]: pgmap v47: 65 pgs: 65 active+clean; 263 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 596 op/s 2024-07-03T15:45:31.959 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:31 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:45:31.959 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:31 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:45:31.959 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:31 smithi148 ceph-mon[36056]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:45:31.959 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:31 smithi148 ceph-mon[36056]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:45:31.959 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:31 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:45:31.959 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:31 smithi148 ceph-mon[36056]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:45:31.959 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:31 smithi148 ceph-mon[36056]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "mgr fail", "who": "smithi148.rjzsar"}]: dispatch 2024-07-03T15:45:31.959 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:31 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "mgr fail", "who": "smithi148.rjzsar"}]: dispatch 2024-07-03T15:45:31.959 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:31 smithi148 ceph-mon[36056]: Activating manager daemon smithi028.vtqrtz 2024-07-03T15:45:31.959 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:31 smithi148 ceph-mon[36056]: osdmap e47: 6 total, 6 up, 6 in 2024-07-03T15:45:31.959 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:31 smithi148 ceph-mon[36056]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' cmd='[{"prefix": "mgr fail", "who": "smithi148.rjzsar"}]': finished 2024-07-03T15:45:31.959 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:31 smithi148 ceph-mon[36056]: mgrmap e26: smithi028.vtqrtz(active, starting, since 0.00669852s) 2024-07-03T15:45:31.960 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:31 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi028"}]: dispatch 2024-07-03T15:45:31.960 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:31 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-07-03T15:45:31.960 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:31 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi028.rhpvve"}]: dispatch 2024-07-03T15:45:31.960 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:31 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi028.wmbxew"}]: dispatch 2024-07-03T15:45:31.960 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:31 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.szomfl"}]: dispatch 2024-07-03T15:45:31.960 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:31 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.xgpuld"}]: dispatch 2024-07-03T15:45:31.960 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:31 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mgr metadata", "who": "smithi028.vtqrtz", "id": "smithi028.vtqrtz"}]: dispatch 2024-07-03T15:45:31.960 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:31 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-03T15:45:31.960 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:31 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-03T15:45:31.960 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:31 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-03T15:45:31.960 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:31 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-03T15:45:31.960 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:31 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-03T15:45:31.960 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:31 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-03T15:45:31.961 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:31 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-03T15:45:31.961 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:31 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-03T15:45:31.961 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:31 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-03T15:45:32.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:31 smithi028 ceph-mon[29148]: pgmap v47: 65 pgs: 65 active+clean; 263 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 596 op/s 2024-07-03T15:45:32.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:31 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:45:32.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:31 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:45:32.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:31 smithi028 ceph-mon[29148]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:45:32.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:31 smithi028 ceph-mon[29148]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:45:32.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:31 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' 2024-07-03T15:45:32.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:31 smithi028 ceph-mon[29148]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:45:32.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:31 smithi028 ceph-mon[29148]: from='mgr.24403 172.21.15.148:0/3455361573' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "mgr fail", "who": "smithi148.rjzsar"}]: dispatch 2024-07-03T15:45:32.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:31 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "mgr fail", "who": "smithi148.rjzsar"}]: dispatch 2024-07-03T15:45:32.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:31 smithi028 ceph-mon[29148]: Activating manager daemon smithi028.vtqrtz 2024-07-03T15:45:32.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:31 smithi028 ceph-mon[29148]: osdmap e47: 6 total, 6 up, 6 in 2024-07-03T15:45:32.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:31 smithi028 ceph-mon[29148]: from='mgr.24403 ' entity='mgr.smithi148.rjzsar' cmd='[{"prefix": "mgr fail", "who": "smithi148.rjzsar"}]': finished 2024-07-03T15:45:32.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:31 smithi028 ceph-mon[29148]: mgrmap e26: smithi028.vtqrtz(active, starting, since 0.00669852s) 2024-07-03T15:45:32.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:31 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi028"}]: dispatch 2024-07-03T15:45:32.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:31 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-07-03T15:45:32.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:31 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi028.rhpvve"}]: dispatch 2024-07-03T15:45:32.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:31 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi028.wmbxew"}]: dispatch 2024-07-03T15:45:32.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:31 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.szomfl"}]: dispatch 2024-07-03T15:45:32.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:31 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.xgpuld"}]: dispatch 2024-07-03T15:45:32.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:31 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mgr metadata", "who": "smithi028.vtqrtz", "id": "smithi028.vtqrtz"}]: dispatch 2024-07-03T15:45:32.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:31 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-03T15:45:32.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:31 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-03T15:45:32.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:31 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-03T15:45:32.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:31 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-03T15:45:32.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:31 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-03T15:45:32.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:31 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-03T15:45:32.125 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:31 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-03T15:45:32.125 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:31 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-03T15:45:32.125 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:31 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-03T15:45:32.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:32 smithi148 ceph-mon[36056]: Manager daemon smithi028.vtqrtz is now available 2024-07-03T15:45:32.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:32 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:45:32.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:32 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:45:32.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:32 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi028.vtqrtz/mirror_snapshot_schedule"}]: dispatch 2024-07-03T15:45:32.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:32 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi028.vtqrtz/trash_purge_schedule"}]: dispatch 2024-07-03T15:45:33.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:32 smithi028 ceph-mon[29148]: Manager daemon smithi028.vtqrtz is now available 2024-07-03T15:45:33.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:32 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:45:33.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:32 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:45:33.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:32 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi028.vtqrtz/mirror_snapshot_schedule"}]: dispatch 2024-07-03T15:45:33.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:32 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi028.vtqrtz/trash_purge_schedule"}]: dispatch 2024-07-03T15:45:34.170 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:33 smithi148 ceph-mon[36056]: mgrmap e27: smithi028.vtqrtz(active, since 1.2363s) 2024-07-03T15:45:34.170 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:33 smithi148 ceph-mon[36056]: pgmap v3: 65 pgs: 65 active+clean; 252 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-07-03T15:45:34.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:33 smithi028 ceph-mon[29148]: mgrmap e27: smithi028.vtqrtz(active, since 1.2363s) 2024-07-03T15:45:34.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:33 smithi028 ceph-mon[29148]: pgmap v3: 65 pgs: 65 active+clean; 252 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-07-03T15:45:35.032 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:34 smithi148 ceph-mon[36056]: pgmap v4: 65 pgs: 65 active+clean; 252 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-07-03T15:45:35.159 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:34 smithi028 ceph-mon[29148]: pgmap v4: 65 pgs: 65 active+clean; 252 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-07-03T15:45:35.464 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-07-03T15:45:35.464 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-07-03T15:45:35.835 DEBUG:teuthology.parallel:result is None 2024-07-03T15:45:36.154 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:35 smithi148 ceph-mon[36056]: mgrmap e28: smithi028.vtqrtz(active, since 3s) 2024-07-03T15:45:36.154 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:35 smithi148 ceph-mon[36056]: [03/Jul/2024:15:45:35] ENGINE Bus STARTING 2024-07-03T15:45:36.154 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:35 smithi148 ceph-mon[36056]: [03/Jul/2024:15:45:35] ENGINE Serving on https://172.21.15.28:7150 2024-07-03T15:45:36.154 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:35 smithi148 ceph-mon[36056]: [03/Jul/2024:15:45:35] ENGINE Error in HTTPServer.serve 2024-07-03T15:45:36.154 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Traceback (most recent call last): 2024-07-03T15:45:36.154 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-03T15:45:36.154 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: self._connections.run(self.expiration_interval) 2024-07-03T15:45:36.154 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-03T15:45:36.155 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: self._run(expiration_interval) 2024-07-03T15:45:36.155 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-03T15:45:36.155 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-07-03T15:45:36.155 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-03T15:45:36.155 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-03T15:45:36.155 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-03T15:45:36.155 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: s = self.context.wrap_socket( 2024-07-03T15:45:36.155 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-03T15:45:36.155 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: return self.sslsocket_class._create( 2024-07-03T15:45:36.155 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-03T15:45:36.155 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: self.do_handshake() 2024-07-03T15:45:36.155 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-03T15:45:36.155 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: self._sslobj.do_handshake() 2024-07-03T15:45:36.156 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-03T15:45:36.156 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:35 smithi148 ceph-mon[36056]: [03/Jul/2024:15:45:35] ENGINE Serving on http://172.21.15.28:8765 2024-07-03T15:45:36.156 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:35 smithi148 ceph-mon[36056]: [03/Jul/2024:15:45:35] ENGINE Bus STARTED 2024-07-03T15:45:36.156 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:35 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:45:36.156 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:35 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:45:36.176 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:35 smithi028 ceph-mon[29148]: mgrmap e28: smithi028.vtqrtz(active, since 3s) 2024-07-03T15:45:36.176 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:35 smithi028 ceph-mon[29148]: [03/Jul/2024:15:45:35] ENGINE Bus STARTING 2024-07-03T15:45:36.176 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:35 smithi028 ceph-mon[29148]: [03/Jul/2024:15:45:35] ENGINE Serving on https://172.21.15.28:7150 2024-07-03T15:45:36.176 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:35 smithi028 ceph-mon[29148]: [03/Jul/2024:15:45:35] ENGINE Error in HTTPServer.serve 2024-07-03T15:45:36.176 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Traceback (most recent call last): 2024-07-03T15:45:36.176 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-03T15:45:36.176 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: self._connections.run(self.expiration_interval) 2024-07-03T15:45:36.176 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-03T15:45:36.176 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: self._run(expiration_interval) 2024-07-03T15:45:36.177 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-03T15:45:36.177 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-07-03T15:45:36.177 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-03T15:45:36.177 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-03T15:45:36.177 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-03T15:45:36.177 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: s = self.context.wrap_socket( 2024-07-03T15:45:36.177 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-03T15:45:36.177 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: return self.sslsocket_class._create( 2024-07-03T15:45:36.177 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-03T15:45:36.177 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: self.do_handshake() 2024-07-03T15:45:36.177 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-03T15:45:36.178 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: self._sslobj.do_handshake() 2024-07-03T15:45:36.178 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-03T15:45:36.178 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:35 smithi028 ceph-mon[29148]: [03/Jul/2024:15:45:35] ENGINE Serving on http://172.21.15.28:8765 2024-07-03T15:45:36.178 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:35 smithi028 ceph-mon[29148]: [03/Jul/2024:15:45:35] ENGINE Bus STARTED 2024-07-03T15:45:36.178 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:35 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:45:36.178 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:35 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:45:36.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:36 smithi148 ceph-mon[36056]: pgmap v5: 65 pgs: 65 active+clean; 252 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-07-03T15:45:36.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:36 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:45:36.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:36 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:45:37.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:36 smithi028 ceph-mon[29148]: pgmap v5: 65 pgs: 65 active+clean; 252 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-07-03T15:45:37.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:36 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:45:37.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:36 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:45:38.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:37 smithi028 ceph-mon[29148]: Standby manager daemon smithi148.rjzsar started 2024-07-03T15:45:38.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:37 smithi028 ceph-mon[29148]: from='mgr.? 172.21.15.148:0/3621915413' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.rjzsar/crt"}]: dispatch 2024-07-03T15:45:38.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:37 smithi028 ceph-mon[29148]: from='mgr.? 172.21.15.148:0/3621915413' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-03T15:45:38.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:37 smithi028 ceph-mon[29148]: from='mgr.? 172.21.15.148:0/3621915413' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.rjzsar/key"}]: dispatch 2024-07-03T15:45:38.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:37 smithi028 ceph-mon[29148]: from='mgr.? 172.21.15.148:0/3621915413' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-03T15:45:38.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:37 smithi148 ceph-mon[36056]: Standby manager daemon smithi148.rjzsar started 2024-07-03T15:45:38.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:37 smithi148 ceph-mon[36056]: from='mgr.? 172.21.15.148:0/3621915413' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.rjzsar/crt"}]: dispatch 2024-07-03T15:45:38.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:37 smithi148 ceph-mon[36056]: from='mgr.? 172.21.15.148:0/3621915413' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-03T15:45:38.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:37 smithi148 ceph-mon[36056]: from='mgr.? 172.21.15.148:0/3621915413' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.rjzsar/key"}]: dispatch 2024-07-03T15:45:38.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:37 smithi148 ceph-mon[36056]: from='mgr.? 172.21.15.148:0/3621915413' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-03T15:45:39.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:38 smithi148 ceph-mon[36056]: pgmap v6: 65 pgs: 65 active+clean; 252 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-07-03T15:45:39.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:38 smithi148 ceph-mon[36056]: mgrmap e29: smithi028.vtqrtz(active, since 6s), standbys: smithi148.rjzsar 2024-07-03T15:45:39.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:38 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mgr metadata", "who": "smithi148.rjzsar", "id": "smithi148.rjzsar"}]: dispatch 2024-07-03T15:45:39.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:38 smithi028 ceph-mon[29148]: pgmap v6: 65 pgs: 65 active+clean; 252 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-07-03T15:45:39.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:38 smithi028 ceph-mon[29148]: mgrmap e29: smithi028.vtqrtz(active, since 6s), standbys: smithi148.rjzsar 2024-07-03T15:45:39.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:38 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mgr metadata", "who": "smithi148.rjzsar", "id": "smithi148.rjzsar"}]: dispatch 2024-07-03T15:45:40.376 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:40 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:45:40.377 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:40 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:45:40.377 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:40 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:45:40.377 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:40 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:45:40.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:40 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:45:40.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:40 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:45:40.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:40 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:45:40.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:40 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:45:41.181 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:41 smithi028 ceph-mon[29148]: pgmap v7: 65 pgs: 65 active+clean; 238 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 541 op/s 2024-07-03T15:45:41.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:41 smithi148 ceph-mon[36056]: pgmap v7: 65 pgs: 65 active+clean; 238 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 541 op/s 2024-07-03T15:45:43.267 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:43 smithi028 ceph-mon[29148]: pgmap v8: 65 pgs: 65 active+clean; 238 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 923 KiB/s rd, 1.1 MiB/s wr, 417 op/s 2024-07-03T15:45:43.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:43 smithi148 ceph-mon[36056]: pgmap v8: 65 pgs: 65 active+clean; 238 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 923 KiB/s rd, 1.1 MiB/s wr, 417 op/s 2024-07-03T15:45:44.766 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-07-03T15:45:44.767 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-07-03T15:45:45.129 DEBUG:teuthology.parallel:result is None 2024-07-03T15:45:45.129 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-07-03T15:45:45.158 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-07-03T15:45:45.158 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-07-03T15:45:45.194 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-07-03T15:45:45.194 DEBUG:teuthology.parallel:result is None 2024-07-03T15:45:45.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:45 smithi028 ceph-mon[29148]: pgmap v9: 65 pgs: 65 active+clean; 225 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 750 op/s 2024-07-03T15:45:45.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:45 smithi148 ceph-mon[36056]: pgmap v9: 65 pgs: 65 active+clean; 225 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 750 op/s 2024-07-03T15:45:46.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:46 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:45:46.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:46 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:45:46.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:46 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:45:46.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:46 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:45:46.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:46 smithi148 ceph-mon[36056]: Updating smithi028:/etc/ceph/ceph.conf 2024-07-03T15:45:46.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:46 smithi148 ceph-mon[36056]: Updating smithi148:/etc/ceph/ceph.conf 2024-07-03T15:45:46.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:46 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:45:46.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:46 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:45:46.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:46 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:45:46.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:46 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:45:46.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:46 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:45:46.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:46 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:45:46.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:46 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:45:46.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:46 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:45:46.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:46 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:45:46.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:46 smithi028 ceph-mon[29148]: Updating smithi028:/etc/ceph/ceph.conf 2024-07-03T15:45:46.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:46 smithi028 ceph-mon[29148]: Updating smithi148:/etc/ceph/ceph.conf 2024-07-03T15:45:46.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:46 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:45:46.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:46 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:45:46.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:46 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:45:46.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:46 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:45:46.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:46 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:45:47.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:47 smithi148 ceph-mon[36056]: Updating smithi148:/var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-03T15:45:47.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:47 smithi148 ceph-mon[36056]: Updating smithi028:/var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-03T15:45:47.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:47 smithi148 ceph-mon[36056]: pgmap v10: 65 pgs: 65 active+clean; 225 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 672 op/s 2024-07-03T15:45:47.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:47 smithi148 ceph-mon[36056]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-07-03T15:45:47.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:47 smithi148 ceph-mon[36056]: Updating smithi028:/etc/ceph/ceph.client.admin.keyring 2024-07-03T15:45:47.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:47 smithi148 ceph-mon[36056]: Updating smithi148:/var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/config/ceph.client.admin.keyring 2024-07-03T15:45:47.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:47 smithi148 ceph-mon[36056]: Updating smithi028:/var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/config/ceph.client.admin.keyring 2024-07-03T15:45:47.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:47 smithi148 ceph-mon[36056]: Reconfiguring prometheus.smithi028 (dependencies changed)... 2024-07-03T15:45:47.380 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:47 smithi028 ceph-mon[29148]: Updating smithi148:/var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-03T15:45:47.380 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:47 smithi028 ceph-mon[29148]: Updating smithi028:/var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-03T15:45:47.380 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:47 smithi028 ceph-mon[29148]: pgmap v10: 65 pgs: 65 active+clean; 225 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 672 op/s 2024-07-03T15:45:47.380 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:47 smithi028 ceph-mon[29148]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-07-03T15:45:47.380 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:47 smithi028 ceph-mon[29148]: Updating smithi028:/etc/ceph/ceph.client.admin.keyring 2024-07-03T15:45:47.380 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:47 smithi028 ceph-mon[29148]: Updating smithi148:/var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/config/ceph.client.admin.keyring 2024-07-03T15:45:47.380 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:47 smithi028 ceph-mon[29148]: Updating smithi028:/var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/config/ceph.client.admin.keyring 2024-07-03T15:45:47.380 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:47 smithi028 ceph-mon[29148]: Reconfiguring prometheus.smithi028 (dependencies changed)... 2024-07-03T15:45:48.462 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:48 smithi028 ceph-mon[29148]: Reconfiguring daemon prometheus.smithi028 on smithi028 2024-07-03T15:45:48.462 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:48 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:45:48.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:48 smithi148 ceph-mon[36056]: Reconfiguring daemon prometheus.smithi028 on smithi028 2024-07-03T15:45:48.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:48 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:45:49.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:49 smithi148 ceph-mon[36056]: pgmap v11: 65 pgs: 65 active+clean; 225 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 672 op/s 2024-07-03T15:45:49.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:49 smithi028 ceph-mon[29148]: pgmap v11: 65 pgs: 65 active+clean; 225 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 672 op/s 2024-07-03T15:45:51.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:51 smithi148 ceph-mon[36056]: pgmap v12: 65 pgs: 65 active+clean; 209 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 756 op/s 2024-07-03T15:45:51.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:51 smithi028 ceph-mon[29148]: pgmap v12: 65 pgs: 65 active+clean; 209 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 756 op/s 2024-07-03T15:45:53.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:53 smithi148 ceph-mon[36056]: pgmap v13: 65 pgs: 65 active+clean; 209 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1018 KiB/s rd, 998 KiB/s wr, 451 op/s 2024-07-03T15:45:53.509 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:53 smithi028 ceph-mon[29148]: pgmap v13: 65 pgs: 65 active+clean; 209 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1018 KiB/s rd, 998 KiB/s wr, 451 op/s 2024-07-03T15:45:55.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:55 smithi148 ceph-mon[36056]: pgmap v14: 65 pgs: 65 active+clean; 197 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 1009 KiB/s wr, 454 op/s 2024-07-03T15:45:55.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:55 smithi028 ceph-mon[29148]: pgmap v14: 65 pgs: 65 active+clean; 197 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 1009 KiB/s wr, 454 op/s 2024-07-03T15:45:55.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.757+0000 7fe037e03640 1 -- 172.21.15.28:0/1197948180 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe030101890 msgr2=0x7fe030103c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:55.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.757+0000 7fe037e03640 1 --2- 172.21.15.28:0/1197948180 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe030101890 0x7fe030103c70 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe0200048d0 tx=0x7fe020009da0 comp rx=0 tx=0).stop 2024-07-03T15:45:55.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.757+0000 7fe037e03640 1 -- 172.21.15.28:0/1197948180 shutdown_connections 2024-07-03T15:45:55.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.757+0000 7fe037e03640 1 --2- 172.21.15.28:0/1197948180 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe030101890 0x7fe030103c70 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:55.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.757+0000 7fe037e03640 1 --2- 172.21.15.28:0/1197948180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe030065a50 0x7fe030101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:55.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.757+0000 7fe037e03640 1 -- 172.21.15.28:0/1197948180 >> 172.21.15.28:0/1197948180 conn(0x7fe0300f6a20 msgr2=0x7fe0300f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:45:55.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.757+0000 7fe037e03640 1 -- 172.21.15.28:0/1197948180 shutdown_connections 2024-07-03T15:45:55.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.757+0000 7fe037e03640 1 -- 172.21.15.28:0/1197948180 wait complete. 2024-07-03T15:45:55.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.757+0000 7fe037e03640 1 Processor -- start 2024-07-03T15:45:55.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.758+0000 7fe037e03640 1 -- start start 2024-07-03T15:45:55.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.758+0000 7fe037e03640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe030065a50 0x7fe03010f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:45:55.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.758+0000 7fe037e03640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe030101890 0x7fe03010f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:45:55.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.758+0000 7fe037e03640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe030110540 con 0x7fe030101890 2024-07-03T15:45:55.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.758+0000 7fe037e03640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0301106b0 con 0x7fe030065a50 2024-07-03T15:45:55.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.758+0000 7fe03539e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe030101890 0x7fe03010f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:45:55.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.758+0000 7fe03539e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe030101890 0x7fe03010f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39726/0 (socket says 172.21.15.28:39726) 2024-07-03T15:45:55.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.758+0000 7fe03539e640 1 -- 172.21.15.28:0/2695172937 learned_addr learned my addr 172.21.15.28:0/2695172937 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:45:55.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.758+0000 7fe035b9f640 1 --2- 172.21.15.28:0/2695172937 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe030065a50 0x7fe03010f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:45:55.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.758+0000 7fe03539e640 1 -- 172.21.15.28:0/2695172937 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe030065a50 msgr2=0x7fe03010f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:55.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.758+0000 7fe03539e640 1 --2- 172.21.15.28:0/2695172937 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe030065a50 0x7fe03010f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:55.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.758+0000 7fe03539e640 1 -- 172.21.15.28:0/2695172937 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe020005040 con 0x7fe030101890 2024-07-03T15:45:55.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.758+0000 7fe03539e640 1 --2- 172.21.15.28:0/2695172937 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe030101890 0x7fe03010f6d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe020003080 tx=0x7fe020009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:45:55.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.759+0000 7fe0267fc640 1 -- 172.21.15.28:0/2695172937 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe020038070 con 0x7fe030101890 2024-07-03T15:45:55.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.759+0000 7fe0267fc640 1 -- 172.21.15.28:0/2695172937 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe020032970 con 0x7fe030101890 2024-07-03T15:45:55.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.759+0000 7fe037e03640 1 -- 172.21.15.28:0/2695172937 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe030110930 con 0x7fe030101890 2024-07-03T15:45:55.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.759+0000 7fe0267fc640 1 -- 172.21.15.28:0/2695172937 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe020033900 con 0x7fe030101890 2024-07-03T15:45:55.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.759+0000 7fe037e03640 1 -- 172.21.15.28:0/2695172937 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe030110df0 con 0x7fe030101890 2024-07-03T15:45:55.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.760+0000 7fe037e03640 1 -- 172.21.15.28:0/2695172937 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0300fa6a0 con 0x7fe030101890 2024-07-03T15:45:55.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.760+0000 7fe0267fc640 1 -- 172.21.15.28:0/2695172937 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 29) v1 ==== 92027+0+0 (secure 0 0 0) 0x7fe020030020 con 0x7fe030101890 2024-07-03T15:45:55.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.761+0000 7fe0267fc640 1 --2- 172.21.15.28:0/2695172937 >> [v2:172.21.15.28:6800/1642892231,v1:172.21.15.28:6801/1642892231] conn(0x7fe01c06dae0 0x7fe01c06ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:45:55.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.761+0000 7fe0267fc640 1 -- 172.21.15.28:0/2695172937 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fe0200ad3e0 con 0x7fe030101890 2024-07-03T15:45:55.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.761+0000 7fe035b9f640 1 --2- 172.21.15.28:0/2695172937 >> [v2:172.21.15.28:6800/1642892231,v1:172.21.15.28:6801/1642892231] conn(0x7fe01c06dae0 0x7fe01c06ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:45:55.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.762+0000 7fe035b9f640 1 --2- 172.21.15.28:0/2695172937 >> [v2:172.21.15.28:6800/1642892231,v1:172.21.15.28:6801/1642892231] conn(0x7fe01c06dae0 0x7fe01c06ff90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe02c006fd0 tx=0x7fe02c008040 comp rx=0 tx=0).ready entity=mgr.14700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:45:55.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.765+0000 7fe0267fc640 1 -- 172.21.15.28:0/2695172937 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7fe020076ea0 con 0x7fe030101890 2024-07-03T15:45:55.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.967+0000 7fe037e03640 1 -- 172.21.15.28:0/2695172937 --> [v2:172.21.15.28:6800/1642892231,v1:172.21.15.28:6801/1642892231] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe030107270 con 0x7fe01c06dae0 2024-07-03T15:45:55.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.969+0000 7fe0267fc640 1 -- 172.21.15.28:0/2695172937 <== mgr.14700 v2:172.21.15.28:6800/1642892231 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7fe030107270 con 0x7fe01c06dae0 2024-07-03T15:45:55.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.971+0000 7fe037e03640 1 -- 172.21.15.28:0/2695172937 >> [v2:172.21.15.28:6800/1642892231,v1:172.21.15.28:6801/1642892231] conn(0x7fe01c06dae0 msgr2=0x7fe01c06ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:55.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.971+0000 7fe037e03640 1 --2- 172.21.15.28:0/2695172937 >> [v2:172.21.15.28:6800/1642892231,v1:172.21.15.28:6801/1642892231] conn(0x7fe01c06dae0 0x7fe01c06ff90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe02c006fd0 tx=0x7fe02c008040 comp rx=0 tx=0).stop 2024-07-03T15:45:55.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.971+0000 7fe037e03640 1 -- 172.21.15.28:0/2695172937 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe030101890 msgr2=0x7fe03010f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:55.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.971+0000 7fe037e03640 1 --2- 172.21.15.28:0/2695172937 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe030101890 0x7fe03010f6d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe020003080 tx=0x7fe020009da0 comp rx=0 tx=0).stop 2024-07-03T15:45:55.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.971+0000 7fe037e03640 1 -- 172.21.15.28:0/2695172937 shutdown_connections 2024-07-03T15:45:55.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.971+0000 7fe037e03640 1 --2- 172.21.15.28:0/2695172937 >> [v2:172.21.15.28:6800/1642892231,v1:172.21.15.28:6801/1642892231] conn(0x7fe01c06dae0 0x7fe01c06ff90 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:55.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.971+0000 7fe037e03640 1 --2- 172.21.15.28:0/2695172937 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe030101890 0x7fe03010f6d0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:55.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.971+0000 7fe037e03640 1 --2- 172.21.15.28:0/2695172937 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe030065a50 0x7fe03010f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:55.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.971+0000 7fe037e03640 1 -- 172.21.15.28:0/2695172937 >> 172.21.15.28:0/2695172937 conn(0x7fe0300f6a20 msgr2=0x7fe0300f6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:45:55.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.971+0000 7fe037e03640 1 -- 172.21.15.28:0/2695172937 shutdown_connections 2024-07-03T15:45:55.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:55.971+0000 7fe037e03640 1 -- 172.21.15.28:0/2695172937 wait complete. 2024-07-03T15:45:55.984 INFO:teuthology.orchestra.run.smithi028.stdout:true 2024-07-03T15:45:56.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.055+0000 7f27dcfab640 1 -- 172.21.15.28:0/13092969 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f27d80fc500 msgr2=0x7f27d80fc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:56.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.055+0000 7f27dcfab640 1 --2- 172.21.15.28:0/13092969 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f27d80fc500 0x7f27d80fc970 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f27c00048d0 tx=0x7f27c0009da0 comp rx=0 tx=0).stop 2024-07-03T15:45:56.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.055+0000 7f27dcfab640 1 -- 172.21.15.28:0/13092969 shutdown_connections 2024-07-03T15:45:56.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.055+0000 7f27dcfab640 1 --2- 172.21.15.28:0/13092969 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f27d80fc500 0x7f27d80fc970 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:56.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.055+0000 7f27dcfab640 1 --2- 172.21.15.28:0/13092969 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27d80fb230 0x7f27d80fb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:56.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.055+0000 7f27dcfab640 1 -- 172.21.15.28:0/13092969 >> 172.21.15.28:0/13092969 conn(0x7f27d80f6a10 msgr2=0x7f27d80f8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:45:56.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.055+0000 7f27dcfab640 1 -- 172.21.15.28:0/13092969 shutdown_connections 2024-07-03T15:45:56.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.055+0000 7f27dcfab640 1 -- 172.21.15.28:0/13092969 wait complete. 2024-07-03T15:45:56.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.055+0000 7f27dcfab640 1 Processor -- start 2024-07-03T15:45:56.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.055+0000 7f27dcfab640 1 -- start start 2024-07-03T15:45:56.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.056+0000 7f27dcfab640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f27d80fb230 0x7f27d810cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:45:56.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.056+0000 7f27dcfab640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27d80fc500 0x7f27d810d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:45:56.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.056+0000 7f27dcfab640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d810e360 con 0x7f27d80fb230 2024-07-03T15:45:56.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.056+0000 7f27dcfab640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d810e4d0 con 0x7f27d80fc500 2024-07-03T15:45:56.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.056+0000 7f27d659c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f27d80fb230 0x7f27d810cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:45:56.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.056+0000 7f27d659c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f27d80fb230 0x7f27d810cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39742/0 (socket says 172.21.15.28:39742) 2024-07-03T15:45:56.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.056+0000 7f27d659c640 1 -- 172.21.15.28:0/1129796837 learned_addr learned my addr 172.21.15.28:0/1129796837 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:45:56.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.056+0000 7f27cffff640 1 --2- 172.21.15.28:0/1129796837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27d80fc500 0x7f27d810d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:45:56.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.056+0000 7f27d659c640 1 -- 172.21.15.28:0/1129796837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27d80fc500 msgr2=0x7f27d810d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:56.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.056+0000 7f27d659c640 1 --2- 172.21.15.28:0/1129796837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27d80fc500 0x7f27d810d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:56.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.056+0000 7f27d659c640 1 -- 172.21.15.28:0/1129796837 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c0005040 con 0x7f27d80fb230 2024-07-03T15:45:56.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.056+0000 7f27d659c640 1 --2- 172.21.15.28:0/1129796837 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f27d80fb230 0x7f27d810cfb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f27c800b790 tx=0x7f27c80091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:45:56.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.056+0000 7f27ceffd640 1 -- 172.21.15.28:0/1129796837 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27c800c700 con 0x7f27d80fb230 2024-07-03T15:45:56.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.056+0000 7f27ceffd640 1 -- 172.21.15.28:0/1129796837 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f27c800cd30 con 0x7f27d80fb230 2024-07-03T15:45:56.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.056+0000 7f27dcfab640 1 -- 172.21.15.28:0/1129796837 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27d810e7b0 con 0x7f27d80fb230 2024-07-03T15:45:56.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.056+0000 7f27ceffd640 1 -- 172.21.15.28:0/1129796837 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27c8013370 con 0x7f27d80fb230 2024-07-03T15:45:56.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.056+0000 7f27dcfab640 1 -- 172.21.15.28:0/1129796837 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27d810ecd0 con 0x7f27d80fb230 2024-07-03T15:45:56.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.057+0000 7f27dcfab640 1 -- 172.21.15.28:0/1129796837 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27d8104610 con 0x7f27d80fb230 2024-07-03T15:45:56.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.058+0000 7f27ceffd640 1 -- 172.21.15.28:0/1129796837 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 29) v1 ==== 92027+0+0 (secure 0 0 0) 0x7f27c8012070 con 0x7f27d80fb230 2024-07-03T15:45:56.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.059+0000 7f27ceffd640 1 --2- 172.21.15.28:0/1129796837 >> [v2:172.21.15.28:6800/1642892231,v1:172.21.15.28:6801/1642892231] conn(0x7f27c406dae0 0x7f27c406ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:45:56.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.059+0000 7f27ceffd640 1 -- 172.21.15.28:0/1129796837 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f27c808cd80 con 0x7f27d80fb230 2024-07-03T15:45:56.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.059+0000 7f27cffff640 1 --2- 172.21.15.28:0/1129796837 >> [v2:172.21.15.28:6800/1642892231,v1:172.21.15.28:6801/1642892231] conn(0x7f27c406dae0 0x7f27c406ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:45:56.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.059+0000 7f27cffff640 1 --2- 172.21.15.28:0/1129796837 >> [v2:172.21.15.28:6800/1642892231,v1:172.21.15.28:6801/1642892231] conn(0x7f27c406dae0 0x7f27c406ff90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f27c0009f80 tx=0x7f27c0035040 comp rx=0 tx=0).ready entity=mgr.14700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:45:56.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.063+0000 7f27ceffd640 1 -- 172.21.15.28:0/1129796837 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7f27c8056840 con 0x7f27d80fb230 2024-07-03T15:45:56.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.268+0000 7f27dcfab640 1 -- 172.21.15.28:0/1129796837 --> [v2:172.21.15.28:6800/1642892231,v1:172.21.15.28:6801/1642892231] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f27d8103e00 con 0x7f27c406dae0 2024-07-03T15:45:56.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.270+0000 7f27ceffd640 1 -- 172.21.15.28:0/1129796837 <== mgr.14700 v2:172.21.15.28:6800/1642892231 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f27d8103e00 con 0x7f27c406dae0 2024-07-03T15:45:56.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.272+0000 7f27dcfab640 1 -- 172.21.15.28:0/1129796837 >> [v2:172.21.15.28:6800/1642892231,v1:172.21.15.28:6801/1642892231] conn(0x7f27c406dae0 msgr2=0x7f27c406ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:56.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.272+0000 7f27dcfab640 1 --2- 172.21.15.28:0/1129796837 >> [v2:172.21.15.28:6800/1642892231,v1:172.21.15.28:6801/1642892231] conn(0x7f27c406dae0 0x7f27c406ff90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f27c0009f80 tx=0x7f27c0035040 comp rx=0 tx=0).stop 2024-07-03T15:45:56.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.272+0000 7f27dcfab640 1 -- 172.21.15.28:0/1129796837 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f27d80fb230 msgr2=0x7f27d810cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:56.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.272+0000 7f27dcfab640 1 --2- 172.21.15.28:0/1129796837 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f27d80fb230 0x7f27d810cfb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f27c800b790 tx=0x7f27c80091d0 comp rx=0 tx=0).stop 2024-07-03T15:45:56.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.272+0000 7f27dcfab640 1 -- 172.21.15.28:0/1129796837 shutdown_connections 2024-07-03T15:45:56.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.272+0000 7f27dcfab640 1 --2- 172.21.15.28:0/1129796837 >> [v2:172.21.15.28:6800/1642892231,v1:172.21.15.28:6801/1642892231] conn(0x7f27c406dae0 0x7f27c406ff90 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:56.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.272+0000 7f27dcfab640 1 --2- 172.21.15.28:0/1129796837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27d80fc500 0x7f27d810d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:56.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.272+0000 7f27dcfab640 1 --2- 172.21.15.28:0/1129796837 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f27d80fb230 0x7f27d810cfb0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:56.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.272+0000 7f27dcfab640 1 -- 172.21.15.28:0/1129796837 >> 172.21.15.28:0/1129796837 conn(0x7f27d80f6a10 msgr2=0x7f27d80f8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:45:56.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.272+0000 7f27dcfab640 1 -- 172.21.15.28:0/1129796837 shutdown_connections 2024-07-03T15:45:56.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.272+0000 7f27dcfab640 1 -- 172.21.15.28:0/1129796837 wait complete. 2024-07-03T15:45:56.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.366+0000 7f0b8cfcf640 1 -- 172.21.15.28:0/1530179223 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b880fc370 msgr2=0x7f0b880fc780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:56.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.366+0000 7f0b8cfcf640 1 --2- 172.21.15.28:0/1530179223 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b880fc370 0x7f0b880fc780 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0b780048d0 tx=0x7f0b78009da0 comp rx=0 tx=0).stop 2024-07-03T15:45:56.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.366+0000 7f0b8cfcf640 1 -- 172.21.15.28:0/1530179223 shutdown_connections 2024-07-03T15:45:56.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.366+0000 7f0b8cfcf640 1 --2- 172.21.15.28:0/1530179223 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b880fd640 0x7f0b880fdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:56.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.366+0000 7f0b8cfcf640 1 --2- 172.21.15.28:0/1530179223 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b880fc370 0x7f0b880fc780 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:56.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.366+0000 7f0b8cfcf640 1 -- 172.21.15.28:0/1530179223 >> 172.21.15.28:0/1530179223 conn(0x7f0b880f7b50 msgr2=0x7f0b880f9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:45:56.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.367+0000 7f0b8cfcf640 1 -- 172.21.15.28:0/1530179223 shutdown_connections 2024-07-03T15:45:56.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.367+0000 7f0b8cfcf640 1 -- 172.21.15.28:0/1530179223 wait complete. 2024-07-03T15:45:56.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.367+0000 7f0b8cfcf640 1 Processor -- start 2024-07-03T15:45:56.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.367+0000 7f0b8cfcf640 1 -- start start 2024-07-03T15:45:56.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.367+0000 7f0b8cfcf640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b880fc370 0x7f0b88071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:45:56.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.367+0000 7f0b8cfcf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b880fd640 0x7f0b8806f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:45:56.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.367+0000 7f0b8cfcf640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8806fe00 con 0x7f0b880fc370 2024-07-03T15:45:56.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.367+0000 7f0b8cfcf640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8806ff40 con 0x7f0b880fd640 2024-07-03T15:45:56.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.368+0000 7f0b8659c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b880fc370 0x7f0b88071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:45:56.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.368+0000 7f0b8659c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b880fc370 0x7f0b88071380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39764/0 (socket says 172.21.15.28:39764) 2024-07-03T15:45:56.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.368+0000 7f0b8659c640 1 -- 172.21.15.28:0/3270123726 learned_addr learned my addr 172.21.15.28:0/3270123726 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:45:56.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.368+0000 7f0b85d9b640 1 --2- 172.21.15.28:0/3270123726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b880fd640 0x7f0b8806f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:45:56.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.368+0000 7f0b8659c640 1 -- 172.21.15.28:0/3270123726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b880fd640 msgr2=0x7f0b8806f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:56.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.368+0000 7f0b8659c640 1 --2- 172.21.15.28:0/3270123726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b880fd640 0x7f0b8806f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:56.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.368+0000 7f0b8659c640 1 -- 172.21.15.28:0/3270123726 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b78005040 con 0x7f0b880fc370 2024-07-03T15:45:56.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.368+0000 7f0b85d9b640 1 --2- 172.21.15.28:0/3270123726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b880fd640 0x7f0b8806f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-03T15:45:56.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.368+0000 7f0b8659c640 1 --2- 172.21.15.28:0/3270123726 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b880fc370 0x7f0b88071380 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0b780048a0 tx=0x7f0b78009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:45:56.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.368+0000 7f0b76ffd640 1 -- 172.21.15.28:0/3270123726 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b78038070 con 0x7f0b880fc370 2024-07-03T15:45:56.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.368+0000 7f0b76ffd640 1 -- 172.21.15.28:0/3270123726 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0b780092b0 con 0x7f0b880fc370 2024-07-03T15:45:56.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.368+0000 7f0b8cfcf640 1 -- 172.21.15.28:0/3270123726 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b880701c0 con 0x7f0b880fc370 2024-07-03T15:45:56.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.368+0000 7f0b76ffd640 1 -- 172.21.15.28:0/3270123726 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b780338a0 con 0x7f0b880fc370 2024-07-03T15:45:56.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.369+0000 7f0b8cfcf640 1 -- 172.21.15.28:0/3270123726 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b88114fc0 con 0x7f0b880fc370 2024-07-03T15:45:56.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.369+0000 7f0b8cfcf640 1 -- 172.21.15.28:0/3270123726 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b880fb7d0 con 0x7f0b880fc370 2024-07-03T15:45:56.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.370+0000 7f0b76ffd640 1 -- 172.21.15.28:0/3270123726 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 29) v1 ==== 92027+0+0 (secure 0 0 0) 0x7f0b78030020 con 0x7f0b880fc370 2024-07-03T15:45:56.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.371+0000 7f0b76ffd640 1 --2- 172.21.15.28:0/3270123726 >> [v2:172.21.15.28:6800/1642892231,v1:172.21.15.28:6801/1642892231] conn(0x7f0b7006dae0 0x7f0b7006ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:45:56.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.371+0000 7f0b76ffd640 1 -- 172.21.15.28:0/3270123726 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f0b780ad120 con 0x7f0b880fc370 2024-07-03T15:45:56.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.371+0000 7f0b85d9b640 1 --2- 172.21.15.28:0/3270123726 >> [v2:172.21.15.28:6800/1642892231,v1:172.21.15.28:6801/1642892231] conn(0x7f0b7006dae0 0x7f0b7006ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:45:56.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.371+0000 7f0b85d9b640 1 --2- 172.21.15.28:0/3270123726 >> [v2:172.21.15.28:6800/1642892231,v1:172.21.15.28:6801/1642892231] conn(0x7f0b7006dae0 0x7f0b7006ff90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c004d80 tx=0x7f0b7c00b040 comp rx=0 tx=0).ready entity=mgr.14700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:45:56.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.375+0000 7f0b76ffd640 1 -- 172.21.15.28:0/3270123726 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7f0b78076c90 con 0x7f0b880fc370 2024-07-03T15:45:56.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.582+0000 7f0b8cfcf640 1 -- 172.21.15.28:0/3270123726 --> [v2:172.21.15.28:6800/1642892231,v1:172.21.15.28:6801/1642892231] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b88101950 con 0x7f0b7006dae0 2024-07-03T15:45:56.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.592+0000 7f0b76ffd640 1 -- 172.21.15.28:0/3270123726 <== mgr.14700 v2:172.21.15.28:6800/1642892231 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f0b88101950 con 0x7f0b7006dae0 2024-07-03T15:45:56.596 INFO:teuthology.orchestra.run.smithi028.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-03T15:45:56.596 INFO:teuthology.orchestra.run.smithi028.stdout:alertmanager.smithi028 smithi028 *:9093,9094 running (82s) 17s ago 9m 25.7M - 0.25.0 c8568f914cd2 1425315a0de9 2024-07-03T15:45:56.596 INFO:teuthology.orchestra.run.smithi028.stdout:crash.smithi028 smithi028 running (9m) 17s ago 9m 6899k - 17.2.7-1047-gd4b063ab 4ca399d9e65d 535660e448f9 2024-07-03T15:45:56.596 INFO:teuthology.orchestra.run.smithi028.stdout:crash.smithi148 smithi148 running (8m) 20s ago 8m 6899k - 17.2.7-1047-gd4b063ab 4ca399d9e65d 4bb7675895e7 2024-07-03T15:45:56.596 INFO:teuthology.orchestra.run.smithi028.stdout:grafana.smithi028 smithi028 *:3000 running (77s) 17s ago 9m 77.0M - 9.4.7 954c08fa6188 f9f17c92cfea 2024-07-03T15:45:56.596 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi028.rhpvve smithi028 running (4m) 17s ago 4m 185M - 17.2.7-1047-gd4b063ab 4ca399d9e65d 3ed5abd02cfb 2024-07-03T15:45:56.596 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi028.wmbxew smithi028 running (4m) 17s ago 4m 126M - 17.2.7-1047-gd4b063ab 4ca399d9e65d b3bf736998f9 2024-07-03T15:45:56.596 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi148.szomfl smithi148 running (4m) 20s ago 4m 14.7M - 17.2.7-1047-gd4b063ab 4ca399d9e65d d94f5c95fa8d 2024-07-03T15:45:56.597 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi148.xgpuld smithi148 running (4m) 20s ago 4m 12.5M - 17.2.7-1047-gd4b063ab 4ca399d9e65d fc70360a8de0 2024-07-03T15:45:56.597 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.smithi028.vtqrtz smithi028 *:8443,9283,8765 running (47s) 17s ago 10m 474M - 19.0.0-2550-gc00506dc 9d49405603e8 14d0363c5b77 2024-07-03T15:45:56.597 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.smithi148.rjzsar smithi148 *:8443,9283 running (2m) 20s ago 8m 137M - 19.0.0-2550-gc00506dc 9d49405603e8 1fe24bbddd1a 2024-07-03T15:45:56.597 INFO:teuthology.orchestra.run.smithi028.stdout:mon.smithi028 smithi028 running (10m) 17s ago 11m 53.6M 2048M 17.2.7-1047-gd4b063ab 4ca399d9e65d 7461f9bef4e0 2024-07-03T15:45:56.597 INFO:teuthology.orchestra.run.smithi028.stdout:mon.smithi148 smithi148 running (8m) 20s ago 8m 43.5M 2048M 17.2.7-1047-gd4b063ab 4ca399d9e65d 4c6c8ed2ab85 2024-07-03T15:45:56.597 INFO:teuthology.orchestra.run.smithi028.stdout:node-exporter.smithi028 smithi028 *:9100 running (88s) 17s ago 9m 15.2M - 1.5.0 0da6a335fe13 91ca1f49de0e 2024-07-03T15:45:56.597 INFO:teuthology.orchestra.run.smithi028.stdout:node-exporter.smithi148 smithi148 *:9100 running (61s) 20s ago 8m 7990k - 1.5.0 0da6a335fe13 1eaaf7b1b561 2024-07-03T15:45:56.597 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0 smithi028 running (7m) 17s ago 7m 246M 1558M 17.2.7-1047-gd4b063ab 4ca399d9e65d b18089a439ba 2024-07-03T15:45:56.597 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1 smithi028 running (7m) 17s ago 7m 250M 1558M 17.2.7-1047-gd4b063ab 4ca399d9e65d 87d415dde231 2024-07-03T15:45:56.597 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2 smithi028 running (6m) 17s ago 6m 207M 1558M 17.2.7-1047-gd4b063ab 4ca399d9e65d b2cbb312212d 2024-07-03T15:45:56.597 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3 smithi148 running (6m) 20s ago 6m 268M 2581M 17.2.7-1047-gd4b063ab 4ca399d9e65d 5be69a27df31 2024-07-03T15:45:56.597 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4 smithi148 running (6m) 20s ago 6m 239M 2581M 17.2.7-1047-gd4b063ab 4ca399d9e65d 54f7c26c77ca 2024-07-03T15:45:56.598 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5 smithi148 running (5m) 20s ago 5m 231M 2581M 17.2.7-1047-gd4b063ab 4ca399d9e65d d185c6905384 2024-07-03T15:45:56.598 INFO:teuthology.orchestra.run.smithi028.stdout:prometheus.smithi028 smithi028 *:9095 running (67s) 17s ago 9m 46.7M - 2.43.0 a07b618ecd1d 57f08be67c0f 2024-07-03T15:45:56.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.594+0000 7f0b8cfcf640 1 -- 172.21.15.28:0/3270123726 >> [v2:172.21.15.28:6800/1642892231,v1:172.21.15.28:6801/1642892231] conn(0x7f0b7006dae0 msgr2=0x7f0b7006ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:56.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.594+0000 7f0b8cfcf640 1 --2- 172.21.15.28:0/3270123726 >> [v2:172.21.15.28:6800/1642892231,v1:172.21.15.28:6801/1642892231] conn(0x7f0b7006dae0 0x7f0b7006ff90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c004d80 tx=0x7f0b7c00b040 comp rx=0 tx=0).stop 2024-07-03T15:45:56.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.594+0000 7f0b8cfcf640 1 -- 172.21.15.28:0/3270123726 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b880fc370 msgr2=0x7f0b88071380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:56.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.594+0000 7f0b8cfcf640 1 --2- 172.21.15.28:0/3270123726 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b880fc370 0x7f0b88071380 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0b780048a0 tx=0x7f0b78009da0 comp rx=0 tx=0).stop 2024-07-03T15:45:56.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.594+0000 7f0b8cfcf640 1 -- 172.21.15.28:0/3270123726 shutdown_connections 2024-07-03T15:45:56.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.594+0000 7f0b8cfcf640 1 --2- 172.21.15.28:0/3270123726 >> [v2:172.21.15.28:6800/1642892231,v1:172.21.15.28:6801/1642892231] conn(0x7f0b7006dae0 0x7f0b7006ff90 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:56.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.594+0000 7f0b8cfcf640 1 --2- 172.21.15.28:0/3270123726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b880fd640 0x7f0b8806f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:56.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.594+0000 7f0b8cfcf640 1 --2- 172.21.15.28:0/3270123726 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b880fc370 0x7f0b88071380 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:56.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.594+0000 7f0b8cfcf640 1 -- 172.21.15.28:0/3270123726 >> 172.21.15.28:0/3270123726 conn(0x7f0b880f7b50 msgr2=0x7f0b880f9180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:45:56.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.594+0000 7f0b8cfcf640 1 -- 172.21.15.28:0/3270123726 shutdown_connections 2024-07-03T15:45:56.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.594+0000 7f0b8cfcf640 1 -- 172.21.15.28:0/3270123726 wait complete. 2024-07-03T15:45:56.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.676+0000 7fcf07166640 1 -- 172.21.15.28:0/2785892597 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf000fd9b0 msgr2=0x7fcf000fddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:56.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.676+0000 7fcf07166640 1 --2- 172.21.15.28:0/2785892597 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf000fd9b0 0x7fcf000fddc0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fcef40048d0 tx=0x7fcef4009da0 comp rx=0 tx=0).stop 2024-07-03T15:45:56.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.676+0000 7fcf07166640 1 -- 172.21.15.28:0/2785892597 shutdown_connections 2024-07-03T15:45:56.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.676+0000 7fcf07166640 1 --2- 172.21.15.28:0/2785892597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf000fe300 0x7fcf000fa460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:56.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.676+0000 7fcf07166640 1 --2- 172.21.15.28:0/2785892597 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf000fd9b0 0x7fcf000fddc0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:56.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.676+0000 7fcf07166640 1 -- 172.21.15.28:0/2785892597 >> 172.21.15.28:0/2785892597 conn(0x7fcf000f61b0 msgr2=0x7fcf000f85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:45:56.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.676+0000 7fcf07166640 1 -- 172.21.15.28:0/2785892597 shutdown_connections 2024-07-03T15:45:56.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.676+0000 7fcf07166640 1 -- 172.21.15.28:0/2785892597 wait complete. 2024-07-03T15:45:56.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.677+0000 7fcf07166640 1 Processor -- start 2024-07-03T15:45:56.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.677+0000 7fcf07166640 1 -- start start 2024-07-03T15:45:56.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.677+0000 7fcf07166640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf000fd9b0 0x7fcf0010d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:45:56.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.677+0000 7fcf07166640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf000fe300 0x7fcf0010d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:45:56.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.677+0000 7fcf07166640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0010dbc0 con 0x7fcf000fe300 2024-07-03T15:45:56.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.677+0000 7fcf07166640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0010e390 con 0x7fcf000fd9b0 2024-07-03T15:45:56.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.677+0000 7fcefffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf000fe300 0x7fcf0010d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:45:56.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.677+0000 7fcefffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf000fe300 0x7fcf0010d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39782/0 (socket says 172.21.15.28:39782) 2024-07-03T15:45:56.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.677+0000 7fcefffff640 1 -- 172.21.15.28:0/2341029674 learned_addr learned my addr 172.21.15.28:0/2341029674 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:45:56.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.677+0000 7fcf04f02640 1 --2- 172.21.15.28:0/2341029674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf000fd9b0 0x7fcf0010d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:45:56.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.678+0000 7fcefffff640 1 -- 172.21.15.28:0/2341029674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf000fd9b0 msgr2=0x7fcf0010d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:56.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.678+0000 7fcefffff640 1 --2- 172.21.15.28:0/2341029674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf000fd9b0 0x7fcf0010d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:56.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.678+0000 7fcefffff640 1 -- 172.21.15.28:0/2341029674 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef4005040 con 0x7fcf000fe300 2024-07-03T15:45:56.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.678+0000 7fcefffff640 1 --2- 172.21.15.28:0/2341029674 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf000fe300 0x7fcf0010d570 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fceec002f40 tx=0x7fceec00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:45:56.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.678+0000 7fcefd7fa640 1 -- 172.21.15.28:0/2341029674 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fceec007c30 con 0x7fcf000fe300 2024-07-03T15:45:56.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.678+0000 7fcf07166640 1 -- 172.21.15.28:0/2341029674 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf0010e5c0 con 0x7fcf000fe300 2024-07-03T15:45:56.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.678+0000 7fcefd7fa640 1 -- 172.21.15.28:0/2341029674 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fceec00f040 con 0x7fcf000fe300 2024-07-03T15:45:56.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.678+0000 7fcefd7fa640 1 -- 172.21.15.28:0/2341029674 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fceec00cc30 con 0x7fcf000fe300 2024-07-03T15:45:56.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.678+0000 7fcf07166640 1 -- 172.21.15.28:0/2341029674 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf0010eb60 con 0x7fcf000fe300 2024-07-03T15:45:56.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.679+0000 7fcf07166640 1 -- 172.21.15.28:0/2341029674 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf000f9e50 con 0x7fcf000fe300 2024-07-03T15:45:56.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.680+0000 7fcefd7fa640 1 -- 172.21.15.28:0/2341029674 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 29) v1 ==== 92027+0+0 (secure 0 0 0) 0x7fceec0090d0 con 0x7fcf000fe300 2024-07-03T15:45:56.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.680+0000 7fcefd7fa640 1 --2- 172.21.15.28:0/2341029674 >> [v2:172.21.15.28:6800/1642892231,v1:172.21.15.28:6801/1642892231] conn(0x7fcef006dae0 0x7fcef006ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:45:56.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.681+0000 7fcefd7fa640 1 -- 172.21.15.28:0/2341029674 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fceec08cf80 con 0x7fcf000fe300 2024-07-03T15:45:56.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.681+0000 7fcf04f02640 1 --2- 172.21.15.28:0/2341029674 >> [v2:172.21.15.28:6800/1642892231,v1:172.21.15.28:6801/1642892231] conn(0x7fcef006dae0 0x7fcef006ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:45:56.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.681+0000 7fcf04f02640 1 --2- 172.21.15.28:0/2341029674 >> [v2:172.21.15.28:6800/1642892231,v1:172.21.15.28:6801/1642892231] conn(0x7fcef006dae0 0x7fcef006ff90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fcef4009f80 tx=0x7fcef4035040 comp rx=0 tx=0).ready entity=mgr.14700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:45:56.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.686+0000 7fcefd7fa640 1 -- 172.21.15.28:0/2341029674 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7fceec056a60 con 0x7fcf000fe300 2024-07-03T15:45:56.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.947+0000 7fcf07166640 1 -- 172.21.15.28:0/2341029674 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fcf00104f10 con 0x7fcf000fe300 2024-07-03T15:45:56.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.947+0000 7fcefd7fa640 1 -- 172.21.15.28:0/2341029674 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+741 (secure 0 0 0) 0x7fceec05a120 con 0x7fcf000fe300 2024-07-03T15:45:56.951 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-07-03T15:45:56.952 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": { 2024-07-03T15:45:56.952 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 2 2024-07-03T15:45:56.952 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:45:56.952 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": { 2024-07-03T15:45:56.952 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 2 2024-07-03T15:45:56.952 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:45:56.952 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": { 2024-07-03T15:45:56.952 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 6 2024-07-03T15:45:56.952 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:45:56.952 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": { 2024-07-03T15:45:56.952 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 4 2024-07-03T15:45:56.952 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:45:56.953 INFO:teuthology.orchestra.run.smithi028.stdout: "overall": { 2024-07-03T15:45:56.953 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 12, 2024-07-03T15:45:56.953 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 2 2024-07-03T15:45:56.953 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-07-03T15:45:56.953 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-07-03T15:45:56.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.949+0000 7fcf07166640 1 -- 172.21.15.28:0/2341029674 >> [v2:172.21.15.28:6800/1642892231,v1:172.21.15.28:6801/1642892231] conn(0x7fcef006dae0 msgr2=0x7fcef006ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:56.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.949+0000 7fcf07166640 1 --2- 172.21.15.28:0/2341029674 >> [v2:172.21.15.28:6800/1642892231,v1:172.21.15.28:6801/1642892231] conn(0x7fcef006dae0 0x7fcef006ff90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fcef4009f80 tx=0x7fcef4035040 comp rx=0 tx=0).stop 2024-07-03T15:45:56.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.949+0000 7fcf07166640 1 -- 172.21.15.28:0/2341029674 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf000fe300 msgr2=0x7fcf0010d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:56.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.949+0000 7fcf07166640 1 --2- 172.21.15.28:0/2341029674 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf000fe300 0x7fcf0010d570 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fceec002f40 tx=0x7fceec00b040 comp rx=0 tx=0).stop 2024-07-03T15:45:56.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.949+0000 7fcf07166640 1 -- 172.21.15.28:0/2341029674 shutdown_connections 2024-07-03T15:45:56.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.949+0000 7fcf07166640 1 --2- 172.21.15.28:0/2341029674 >> [v2:172.21.15.28:6800/1642892231,v1:172.21.15.28:6801/1642892231] conn(0x7fcef006dae0 0x7fcef006ff90 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:56.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.949+0000 7fcf07166640 1 --2- 172.21.15.28:0/2341029674 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf000fe300 0x7fcf0010d570 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:56.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.949+0000 7fcf07166640 1 --2- 172.21.15.28:0/2341029674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf000fd9b0 0x7fcf0010d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:56.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.949+0000 7fcf07166640 1 -- 172.21.15.28:0/2341029674 >> 172.21.15.28:0/2341029674 conn(0x7fcf000f61b0 msgr2=0x7fcf000f7ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:45:56.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.950+0000 7fcf07166640 1 -- 172.21.15.28:0/2341029674 shutdown_connections 2024-07-03T15:45:56.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:56.950+0000 7fcf07166640 1 -- 172.21.15.28:0/2341029674 wait complete. 2024-07-03T15:45:57.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.032+0000 7f0046051640 1 -- 172.21.15.28:0/2314250231 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00400fd640 msgr2=0x7f00400fdab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:57.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.032+0000 7f0046051640 1 --2- 172.21.15.28:0/2314250231 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00400fd640 0x7f00400fdab0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f00340048d0 tx=0x7f0034009da0 comp rx=0 tx=0).stop 2024-07-03T15:45:57.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.032+0000 7f0046051640 1 -- 172.21.15.28:0/2314250231 shutdown_connections 2024-07-03T15:45:57.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.032+0000 7f0046051640 1 --2- 172.21.15.28:0/2314250231 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00400fd640 0x7f00400fdab0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:57.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.032+0000 7f0046051640 1 --2- 172.21.15.28:0/2314250231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00400fc370 0x7f00400fc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:57.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.032+0000 7f0046051640 1 -- 172.21.15.28:0/2314250231 >> 172.21.15.28:0/2314250231 conn(0x7f00400f7b50 msgr2=0x7f00400f9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:45:57.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.032+0000 7f0046051640 1 -- 172.21.15.28:0/2314250231 shutdown_connections 2024-07-03T15:45:57.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.032+0000 7f0046051640 1 -- 172.21.15.28:0/2314250231 wait complete. 2024-07-03T15:45:57.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.033+0000 7f0046051640 1 Processor -- start 2024-07-03T15:45:57.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.033+0000 7f0046051640 1 -- start start 2024-07-03T15:45:57.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.033+0000 7f0046051640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00400fc370 0x7f004010cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:45:57.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.033+0000 7f0046051640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00400fd640 0x7f004010d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:45:57.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.033+0000 7f0046051640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004010db00 con 0x7f00400fc370 2024-07-03T15:45:57.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.033+0000 7f0046051640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004010e320 con 0x7f00400fd640 2024-07-03T15:45:57.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.033+0000 7f003f7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00400fc370 0x7f004010cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:45:57.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.033+0000 7f003f7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00400fc370 0x7f004010cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39790/0 (socket says 172.21.15.28:39790) 2024-07-03T15:45:57.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.033+0000 7f003f7fe640 1 -- 172.21.15.28:0/1118782719 learned_addr learned my addr 172.21.15.28:0/1118782719 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:45:57.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.033+0000 7f003effd640 1 --2- 172.21.15.28:0/1118782719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00400fd640 0x7f004010d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:45:57.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.034+0000 7f003f7fe640 1 -- 172.21.15.28:0/1118782719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00400fd640 msgr2=0x7f004010d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:57.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.034+0000 7f003f7fe640 1 --2- 172.21.15.28:0/1118782719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00400fd640 0x7f004010d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:57.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.034+0000 7f003f7fe640 1 -- 172.21.15.28:0/1118782719 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0034005040 con 0x7f00400fc370 2024-07-03T15:45:57.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.034+0000 7f003f7fe640 1 --2- 172.21.15.28:0/1118782719 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00400fc370 0x7f004010cf70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0030002a50 tx=0x7f0030004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:45:57.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.034+0000 7f0027fff640 1 -- 172.21.15.28:0/1118782719 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0030007c30 con 0x7f00400fc370 2024-07-03T15:45:57.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.034+0000 7f0027fff640 1 -- 172.21.15.28:0/1118782719 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f003000f040 con 0x7f00400fc370 2024-07-03T15:45:57.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.034+0000 7f0046051640 1 -- 172.21.15.28:0/1118782719 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f004010e5b0 con 0x7f00400fc370 2024-07-03T15:45:57.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.034+0000 7f0027fff640 1 -- 172.21.15.28:0/1118782719 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f003000dad0 con 0x7f00400fc370 2024-07-03T15:45:57.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.034+0000 7f0046051640 1 -- 172.21.15.28:0/1118782719 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f004010eb20 con 0x7f00400fc370 2024-07-03T15:45:57.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.035+0000 7f0046051640 1 -- 172.21.15.28:0/1118782719 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00401056f0 con 0x7f00400fc370 2024-07-03T15:45:57.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.036+0000 7f0027fff640 1 -- 172.21.15.28:0/1118782719 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 29) v1 ==== 92027+0+0 (secure 0 0 0) 0x7f003000dc70 con 0x7f00400fc370 2024-07-03T15:45:57.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.036+0000 7f0027fff640 1 --2- 172.21.15.28:0/1118782719 >> [v2:172.21.15.28:6800/1642892231,v1:172.21.15.28:6801/1642892231] conn(0x7f002806dae0 0x7f002806ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:45:57.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.036+0000 7f0027fff640 1 -- 172.21.15.28:0/1118782719 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f003008cb40 con 0x7f00400fc370 2024-07-03T15:45:57.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.036+0000 7f003effd640 1 --2- 172.21.15.28:0/1118782719 >> [v2:172.21.15.28:6800/1642892231,v1:172.21.15.28:6801/1642892231] conn(0x7f002806dae0 0x7f002806ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:45:57.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.037+0000 7f003effd640 1 --2- 172.21.15.28:0/1118782719 >> [v2:172.21.15.28:6800/1642892231,v1:172.21.15.28:6801/1642892231] conn(0x7f002806dae0 0x7f002806ff90 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f00340043d0 tx=0x7f0034035040 comp rx=0 tx=0).ready entity=mgr.14700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:45:57.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.040+0000 7f0027fff640 1 -- 172.21.15.28:0/1118782719 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7f0030056620 con 0x7f00400fc370 2024-07-03T15:45:57.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.277+0000 7f0046051640 1 -- 172.21.15.28:0/1118782719 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f004004b580 con 0x7f00400fc370 2024-07-03T15:45:57.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.277+0000 7f0027fff640 1 -- 172.21.15.28:0/1118782719 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1838 (secure 0 0 0) 0x7f0030059ce0 con 0x7f00400fc370 2024-07-03T15:45:57.282 INFO:teuthology.orchestra.run.smithi028.stdout:e17 2024-07-03T15:45:57.282 INFO:teuthology.orchestra.run.smithi028.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-03T15:45:57.282 INFO:teuthology.orchestra.run.smithi028.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:45:57.282 INFO:teuthology.orchestra.run.smithi028.stdout:legacy client fscid: 1 2024-07-03T15:45:57.282 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:45:57.282 INFO:teuthology.orchestra.run.smithi028.stdout:Filesystem 'cephfs' (1) 2024-07-03T15:45:57.282 INFO:teuthology.orchestra.run.smithi028.stdout:fs_name cephfs 2024-07-03T15:45:57.282 INFO:teuthology.orchestra.run.smithi028.stdout:epoch 16 2024-07-03T15:45:57.282 INFO:teuthology.orchestra.run.smithi028.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-03T15:45:57.283 INFO:teuthology.orchestra.run.smithi028.stdout:created 2024-07-03T15:41:40.656298+0000 2024-07-03T15:45:57.283 INFO:teuthology.orchestra.run.smithi028.stdout:modified 2024-07-03T15:41:56.454984+0000 2024-07-03T15:45:57.283 INFO:teuthology.orchestra.run.smithi028.stdout:tableserver 0 2024-07-03T15:45:57.283 INFO:teuthology.orchestra.run.smithi028.stdout:root 0 2024-07-03T15:45:57.283 INFO:teuthology.orchestra.run.smithi028.stdout:session_timeout 60 2024-07-03T15:45:57.283 INFO:teuthology.orchestra.run.smithi028.stdout:session_autoclose 300 2024-07-03T15:45:57.283 INFO:teuthology.orchestra.run.smithi028.stdout:max_file_size 1099511627776 2024-07-03T15:45:57.283 INFO:teuthology.orchestra.run.smithi028.stdout:required_client_features {} 2024-07-03T15:45:57.283 INFO:teuthology.orchestra.run.smithi028.stdout:last_failure 0 2024-07-03T15:45:57.283 INFO:teuthology.orchestra.run.smithi028.stdout:last_failure_osd_epoch 0 2024-07-03T15:45:57.283 INFO:teuthology.orchestra.run.smithi028.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:45:57.283 INFO:teuthology.orchestra.run.smithi028.stdout:max_mds 1 2024-07-03T15:45:57.283 INFO:teuthology.orchestra.run.smithi028.stdout:in 0 2024-07-03T15:45:57.284 INFO:teuthology.orchestra.run.smithi028.stdout:up {0=14450} 2024-07-03T15:45:57.284 INFO:teuthology.orchestra.run.smithi028.stdout:failed 2024-07-03T15:45:57.284 INFO:teuthology.orchestra.run.smithi028.stdout:damaged 2024-07-03T15:45:57.284 INFO:teuthology.orchestra.run.smithi028.stdout:stopped 2024-07-03T15:45:57.284 INFO:teuthology.orchestra.run.smithi028.stdout:data_pools [3] 2024-07-03T15:45:57.284 INFO:teuthology.orchestra.run.smithi028.stdout:metadata_pool 2 2024-07-03T15:45:57.284 INFO:teuthology.orchestra.run.smithi028.stdout:inline_data disabled 2024-07-03T15:45:57.284 INFO:teuthology.orchestra.run.smithi028.stdout:balancer 2024-07-03T15:45:57.284 INFO:teuthology.orchestra.run.smithi028.stdout:standby_count_wanted 1 2024-07-03T15:45:57.284 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi028.rhpvve{0:14450} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.28:6826/1613936042,v1:172.21.15.28:6827/1613936042] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:45:57.284 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi028.wmbxew{0:14460} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.28:6828/1730336821,v1:172.21.15.28:6829/1730336821] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:45:57.284 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:45:57.284 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:45:57.285 INFO:teuthology.orchestra.run.smithi028.stdout:Standby daemons: 2024-07-03T15:45:57.285 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:45:57.285 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi148.szomfl{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/4171231858,v1:172.21.15.148:6827/4171231858] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:45:57.285 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi148.xgpuld{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/2347536398,v1:172.21.15.148:6825/2347536398] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:45:57.285 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 17 2024-07-03T15:45:57.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.280+0000 7f0046051640 1 -- 172.21.15.28:0/1118782719 >> [v2:172.21.15.28:6800/1642892231,v1:172.21.15.28:6801/1642892231] conn(0x7f002806dae0 msgr2=0x7f002806ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:57.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.280+0000 7f0046051640 1 --2- 172.21.15.28:0/1118782719 >> [v2:172.21.15.28:6800/1642892231,v1:172.21.15.28:6801/1642892231] conn(0x7f002806dae0 0x7f002806ff90 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f00340043d0 tx=0x7f0034035040 comp rx=0 tx=0).stop 2024-07-03T15:45:57.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.280+0000 7f0046051640 1 -- 172.21.15.28:0/1118782719 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00400fc370 msgr2=0x7f004010cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:57.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.280+0000 7f0046051640 1 --2- 172.21.15.28:0/1118782719 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00400fc370 0x7f004010cf70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0030002a50 tx=0x7f0030004600 comp rx=0 tx=0).stop 2024-07-03T15:45:57.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.280+0000 7f0046051640 1 -- 172.21.15.28:0/1118782719 shutdown_connections 2024-07-03T15:45:57.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.280+0000 7f0046051640 1 --2- 172.21.15.28:0/1118782719 >> [v2:172.21.15.28:6800/1642892231,v1:172.21.15.28:6801/1642892231] conn(0x7f002806dae0 0x7f002806ff90 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:57.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.280+0000 7f0046051640 1 --2- 172.21.15.28:0/1118782719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00400fd640 0x7f004010d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:57.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.280+0000 7f0046051640 1 --2- 172.21.15.28:0/1118782719 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00400fc370 0x7f004010cf70 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:57.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.280+0000 7f0046051640 1 -- 172.21.15.28:0/1118782719 >> 172.21.15.28:0/1118782719 conn(0x7f00400f7b50 msgr2=0x7f00400f91a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:45:57.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.280+0000 7f0046051640 1 -- 172.21.15.28:0/1118782719 shutdown_connections 2024-07-03T15:45:57.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.280+0000 7f0046051640 1 -- 172.21.15.28:0/1118782719 wait complete. 2024-07-03T15:45:57.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.364+0000 7f104ce92640 1 -- 172.21.15.28:0/1658907982 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f10480fc360 msgr2=0x7f10480fc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:57.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.364+0000 7f104ce92640 1 --2- 172.21.15.28:0/1658907982 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f10480fc360 0x7f10480fc770 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f10380048d0 tx=0x7f1038009da0 comp rx=0 tx=0).stop 2024-07-03T15:45:57.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.364+0000 7f104ce92640 1 -- 172.21.15.28:0/1658907982 shutdown_connections 2024-07-03T15:45:57.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.364+0000 7f104ce92640 1 --2- 172.21.15.28:0/1658907982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10480fd630 0x7f10480fdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:57.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.364+0000 7f104ce92640 1 --2- 172.21.15.28:0/1658907982 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f10480fc360 0x7f10480fc770 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:57.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.364+0000 7f104ce92640 1 -- 172.21.15.28:0/1658907982 >> 172.21.15.28:0/1658907982 conn(0x7f10480f7b40 msgr2=0x7f10480f9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:45:57.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.364+0000 7f104ce92640 1 -- 172.21.15.28:0/1658907982 shutdown_connections 2024-07-03T15:45:57.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.364+0000 7f104ce92640 1 -- 172.21.15.28:0/1658907982 wait complete. 2024-07-03T15:45:57.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.365+0000 7f104ce92640 1 Processor -- start 2024-07-03T15:45:57.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.365+0000 7f104ce92640 1 -- start start 2024-07-03T15:45:57.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.365+0000 7f104ce92640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f10480fc360 0x7f104810d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:45:57.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.365+0000 7f104ce92640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10480fd630 0x7f104810d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:45:57.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.365+0000 7f104ce92640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104810e3b0 con 0x7f10480fc360 2024-07-03T15:45:57.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.365+0000 7f104ce92640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104810e520 con 0x7f10480fd630 2024-07-03T15:45:57.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.365+0000 7f104659c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f10480fc360 0x7f104810d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:45:57.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.365+0000 7f104659c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f10480fc360 0x7f104810d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39820/0 (socket says 172.21.15.28:39820) 2024-07-03T15:45:57.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.365+0000 7f104659c640 1 -- 172.21.15.28:0/3693267589 learned_addr learned my addr 172.21.15.28:0/3693267589 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:45:57.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.365+0000 7f103ffff640 1 --2- 172.21.15.28:0/3693267589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10480fd630 0x7f104810d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:45:57.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.365+0000 7f104659c640 1 -- 172.21.15.28:0/3693267589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10480fd630 msgr2=0x7f104810d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:57.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.366+0000 7f104659c640 1 --2- 172.21.15.28:0/3693267589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10480fd630 0x7f104810d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:57.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.366+0000 7f104659c640 1 -- 172.21.15.28:0/3693267589 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1038005040 con 0x7f10480fc360 2024-07-03T15:45:57.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.366+0000 7f103ffff640 1 --2- 172.21.15.28:0/3693267589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10480fd630 0x7f104810d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-03T15:45:57.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.366+0000 7f104659c640 1 --2- 172.21.15.28:0/3693267589 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f10480fc360 0x7f104810d050 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1038003080 tx=0x7f1038009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:45:57.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.366+0000 7f103effd640 1 -- 172.21.15.28:0/3693267589 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1038038070 con 0x7f10480fc360 2024-07-03T15:45:57.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.366+0000 7f103effd640 1 -- 172.21.15.28:0/3693267589 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1038032970 con 0x7f10480fc360 2024-07-03T15:45:57.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.366+0000 7f104ce92640 1 -- 172.21.15.28:0/3693267589 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f104810e7a0 con 0x7f10480fc360 2024-07-03T15:45:57.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.366+0000 7f103effd640 1 -- 172.21.15.28:0/3693267589 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1038033900 con 0x7f10480fc360 2024-07-03T15:45:57.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.366+0000 7f104ce92640 1 -- 172.21.15.28:0/3693267589 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f104810ec60 con 0x7f10480fc360 2024-07-03T15:45:57.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.367+0000 7f104ce92640 1 -- 172.21.15.28:0/3693267589 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1048105740 con 0x7f10480fc360 2024-07-03T15:45:57.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.368+0000 7f103effd640 1 -- 172.21.15.28:0/3693267589 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 29) v1 ==== 92027+0+0 (secure 0 0 0) 0x7f1038030020 con 0x7f10480fc360 2024-07-03T15:45:57.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.368+0000 7f103effd640 1 --2- 172.21.15.28:0/3693267589 >> [v2:172.21.15.28:6800/1642892231,v1:172.21.15.28:6801/1642892231] conn(0x7f103406dae0 0x7f103406ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:45:57.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.368+0000 7f103effd640 1 -- 172.21.15.28:0/3693267589 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f10380ad3e0 con 0x7f10480fc360 2024-07-03T15:45:57.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.368+0000 7f103ffff640 1 --2- 172.21.15.28:0/3693267589 >> [v2:172.21.15.28:6800/1642892231,v1:172.21.15.28:6801/1642892231] conn(0x7f103406dae0 0x7f103406ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:45:57.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.369+0000 7f103ffff640 1 --2- 172.21.15.28:0/3693267589 >> [v2:172.21.15.28:6800/1642892231,v1:172.21.15.28:6801/1642892231] conn(0x7f103406dae0 0x7f103406ff90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1030006fd0 tx=0x7f1030008040 comp rx=0 tx=0).ready entity=mgr.14700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:45:57.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.372+0000 7f103effd640 1 -- 172.21.15.28:0/3693267589 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7f1038076ea0 con 0x7f10480fc360 2024-07-03T15:45:57.480 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:57 smithi028 ceph-mon[29148]: pgmap v15: 65 pgs: 65 active+clean; 197 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 301 KiB/s rd, 286 KiB/s wr, 87 op/s 2024-07-03T15:45:57.480 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:57 smithi028 ceph-mon[29148]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:45:57.480 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:57 smithi028 ceph-mon[29148]: from='client.14764 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:45:57.480 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:57 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/2341029674' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:45:57.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:57 smithi148 ceph-mon[36056]: pgmap v15: 65 pgs: 65 active+clean; 197 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 301 KiB/s rd, 286 KiB/s wr, 87 op/s 2024-07-03T15:45:57.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:57 smithi148 ceph-mon[36056]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:45:57.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:57 smithi148 ceph-mon[36056]: from='client.14764 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:45:57.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:57 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/2341029674' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:45:57.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.575+0000 7f104ce92640 1 -- 172.21.15.28:0/3693267589 --> [v2:172.21.15.28:6800/1642892231,v1:172.21.15.28:6801/1642892231] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f10481050f0 con 0x7f103406dae0 2024-07-03T15:45:57.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.576+0000 7f103effd640 1 -- 172.21.15.28:0/3693267589 <== mgr.14700 v2:172.21.15.28:6800/1642892231 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f10481050f0 con 0x7f103406dae0 2024-07-03T15:45:57.580 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-07-03T15:45:57.580 INFO:teuthology.orchestra.run.smithi028.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2", 2024-07-03T15:45:57.580 INFO:teuthology.orchestra.run.smithi028.stdout: "in_progress": true, 2024-07-03T15:45:57.580 INFO:teuthology.orchestra.run.smithi028.stdout: "which": "Upgrading all daemon types on all hosts", 2024-07-03T15:45:57.580 INFO:teuthology.orchestra.run.smithi028.stdout: "services_complete": [ 2024-07-03T15:45:57.581 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr" 2024-07-03T15:45:57.581 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-07-03T15:45:57.581 INFO:teuthology.orchestra.run.smithi028.stdout: "progress": "2/21 daemons upgraded", 2024-07-03T15:45:57.581 INFO:teuthology.orchestra.run.smithi028.stdout: "message": "", 2024-07-03T15:45:57.581 INFO:teuthology.orchestra.run.smithi028.stdout: "is_paused": false 2024-07-03T15:45:57.581 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-07-03T15:45:57.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.578+0000 7f104ce92640 1 -- 172.21.15.28:0/3693267589 >> [v2:172.21.15.28:6800/1642892231,v1:172.21.15.28:6801/1642892231] conn(0x7f103406dae0 msgr2=0x7f103406ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:57.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.578+0000 7f104ce92640 1 --2- 172.21.15.28:0/3693267589 >> [v2:172.21.15.28:6800/1642892231,v1:172.21.15.28:6801/1642892231] conn(0x7f103406dae0 0x7f103406ff90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1030006fd0 tx=0x7f1030008040 comp rx=0 tx=0).stop 2024-07-03T15:45:57.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.578+0000 7f104ce92640 1 -- 172.21.15.28:0/3693267589 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f10480fc360 msgr2=0x7f104810d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:57.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.578+0000 7f104ce92640 1 --2- 172.21.15.28:0/3693267589 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f10480fc360 0x7f104810d050 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1038003080 tx=0x7f1038009da0 comp rx=0 tx=0).stop 2024-07-03T15:45:57.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.578+0000 7f104ce92640 1 -- 172.21.15.28:0/3693267589 shutdown_connections 2024-07-03T15:45:57.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.578+0000 7f104ce92640 1 --2- 172.21.15.28:0/3693267589 >> [v2:172.21.15.28:6800/1642892231,v1:172.21.15.28:6801/1642892231] conn(0x7f103406dae0 0x7f103406ff90 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:57.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.578+0000 7f104ce92640 1 --2- 172.21.15.28:0/3693267589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10480fd630 0x7f104810d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:57.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.578+0000 7f104ce92640 1 --2- 172.21.15.28:0/3693267589 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f10480fc360 0x7f104810d050 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:57.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.578+0000 7f104ce92640 1 -- 172.21.15.28:0/3693267589 >> 172.21.15.28:0/3693267589 conn(0x7f10480f7b40 msgr2=0x7f10480f9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:45:57.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.578+0000 7f104ce92640 1 -- 172.21.15.28:0/3693267589 shutdown_connections 2024-07-03T15:45:57.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.578+0000 7f104ce92640 1 -- 172.21.15.28:0/3693267589 wait complete. 2024-07-03T15:45:57.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.661+0000 7fb7eee8f640 1 -- 172.21.15.28:0/1034945613 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7e8065990 msgr2=0x7fb7e80ff010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:57.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.661+0000 7fb7eee8f640 1 --2- 172.21.15.28:0/1034945613 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7e8065990 0x7fb7e80ff010 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb7e40048d0 tx=0x7fb7e4009da0 comp rx=0 tx=0).stop 2024-07-03T15:45:57.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.661+0000 7fb7eee8f640 1 -- 172.21.15.28:0/1034945613 shutdown_connections 2024-07-03T15:45:57.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.661+0000 7fb7eee8f640 1 --2- 172.21.15.28:0/1034945613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7e80ff550 0x7fb7e8101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:57.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.661+0000 7fb7eee8f640 1 --2- 172.21.15.28:0/1034945613 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7e8065990 0x7fb7e80ff010 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:57.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.661+0000 7fb7eee8f640 1 -- 172.21.15.28:0/1034945613 >> 172.21.15.28:0/1034945613 conn(0x7fb7e80f6a20 msgr2=0x7fb7e80f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:45:57.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.661+0000 7fb7eee8f640 1 -- 172.21.15.28:0/1034945613 shutdown_connections 2024-07-03T15:45:57.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.661+0000 7fb7eee8f640 1 -- 172.21.15.28:0/1034945613 wait complete. 2024-07-03T15:45:57.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.662+0000 7fb7eee8f640 1 Processor -- start 2024-07-03T15:45:57.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.662+0000 7fb7eee8f640 1 -- start start 2024-07-03T15:45:57.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.662+0000 7fb7eee8f640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7e8065990 0x7fb7e80fca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:45:57.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.662+0000 7fb7eee8f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7e80ff550 0x7fb7e80faf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:45:57.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.662+0000 7fb7eee8f640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e80fb530 con 0x7fb7e8065990 2024-07-03T15:45:57.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.662+0000 7fb7eee8f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e80fb670 con 0x7fb7e80ff550 2024-07-03T15:45:57.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.662+0000 7fb7ecc2b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7e8065990 0x7fb7e80fca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:45:57.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.662+0000 7fb7ecc2b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7e8065990 0x7fb7e80fca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39840/0 (socket says 172.21.15.28:39840) 2024-07-03T15:45:57.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.662+0000 7fb7ecc2b640 1 -- 172.21.15.28:0/95175473 learned_addr learned my addr 172.21.15.28:0/95175473 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:45:57.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.663+0000 7fb7ecc2b640 1 -- 172.21.15.28:0/95175473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7e80ff550 msgr2=0x7fb7e80faf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:57.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.663+0000 7fb7dffff640 1 --2- 172.21.15.28:0/95175473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7e80ff550 0x7fb7e80faf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:45:57.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.663+0000 7fb7ecc2b640 1 --2- 172.21.15.28:0/95175473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7e80ff550 0x7fb7e80faf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:57.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.663+0000 7fb7ecc2b640 1 -- 172.21.15.28:0/95175473 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e4005040 con 0x7fb7e8065990 2024-07-03T15:45:57.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.663+0000 7fb7dffff640 1 --2- 172.21.15.28:0/95175473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7e80ff550 0x7fb7e80faf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-03T15:45:57.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.663+0000 7fb7ecc2b640 1 --2- 172.21.15.28:0/95175473 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7e8065990 0x7fb7e80fca20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4000c00 tx=0x7fb7e4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:45:57.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.663+0000 7fb7dd7fa640 1 -- 172.21.15.28:0/95175473 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7e4038070 con 0x7fb7e8065990 2024-07-03T15:45:57.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.663+0000 7fb7dd7fa640 1 -- 172.21.15.28:0/95175473 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb7e4032970 con 0x7fb7e8065990 2024-07-03T15:45:57.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.663+0000 7fb7eee8f640 1 -- 172.21.15.28:0/95175473 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7e80fb8f0 con 0x7fb7e8065990 2024-07-03T15:45:57.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.663+0000 7fb7dd7fa640 1 -- 172.21.15.28:0/95175473 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7e4033950 con 0x7fb7e8065990 2024-07-03T15:45:57.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.663+0000 7fb7eee8f640 1 -- 172.21.15.28:0/95175473 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7e8115030 con 0x7fb7e8065990 2024-07-03T15:45:57.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.664+0000 7fb7eee8f640 1 -- 172.21.15.28:0/95175473 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7e80fa6a0 con 0x7fb7e8065990 2024-07-03T15:45:57.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.665+0000 7fb7dd7fa640 1 -- 172.21.15.28:0/95175473 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 29) v1 ==== 92027+0+0 (secure 0 0 0) 0x7fb7e4030020 con 0x7fb7e8065990 2024-07-03T15:45:57.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.665+0000 7fb7dd7fa640 1 --2- 172.21.15.28:0/95175473 >> [v2:172.21.15.28:6800/1642892231,v1:172.21.15.28:6801/1642892231] conn(0x7fb7d006dae0 0x7fb7d006ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:45:57.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.665+0000 7fb7dd7fa640 1 -- 172.21.15.28:0/95175473 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fb7e40ad370 con 0x7fb7e8065990 2024-07-03T15:45:57.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.665+0000 7fb7dffff640 1 --2- 172.21.15.28:0/95175473 >> [v2:172.21.15.28:6800/1642892231,v1:172.21.15.28:6801/1642892231] conn(0x7fb7d006dae0 0x7fb7d006ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:45:57.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.666+0000 7fb7dffff640 1 --2- 172.21.15.28:0/95175473 >> [v2:172.21.15.28:6800/1642892231,v1:172.21.15.28:6801/1642892231] conn(0x7fb7d006dae0 0x7fb7d006ff90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8006fd0 tx=0x7fb7d8008040 comp rx=0 tx=0).ready entity=mgr.14700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:45:57.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.670+0000 7fb7dd7fa640 1 -- 172.21.15.28:0/95175473 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184824 (secure 0 0 0) 0x7fb7e4076e10 con 0x7fb7e8065990 2024-07-03T15:45:57.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.940+0000 7fb7eee8f640 1 -- 172.21.15.28:0/95175473 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb7e804b530 con 0x7fb7e8065990 2024-07-03T15:45:57.945 INFO:teuthology.orchestra.run.smithi028.stdout:HEALTH_OK 2024-07-03T15:45:57.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.940+0000 7fb7dd7fa640 1 -- 172.21.15.28:0/95175473 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fb7e407a4d0 con 0x7fb7e8065990 2024-07-03T15:45:57.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.942+0000 7fb7eee8f640 1 -- 172.21.15.28:0/95175473 >> [v2:172.21.15.28:6800/1642892231,v1:172.21.15.28:6801/1642892231] conn(0x7fb7d006dae0 msgr2=0x7fb7d006ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:57.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.942+0000 7fb7eee8f640 1 --2- 172.21.15.28:0/95175473 >> [v2:172.21.15.28:6800/1642892231,v1:172.21.15.28:6801/1642892231] conn(0x7fb7d006dae0 0x7fb7d006ff90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8006fd0 tx=0x7fb7d8008040 comp rx=0 tx=0).stop 2024-07-03T15:45:57.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.942+0000 7fb7eee8f640 1 -- 172.21.15.28:0/95175473 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7e8065990 msgr2=0x7fb7e80fca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:45:57.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.942+0000 7fb7eee8f640 1 --2- 172.21.15.28:0/95175473 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7e8065990 0x7fb7e80fca20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4000c00 tx=0x7fb7e4009da0 comp rx=0 tx=0).stop 2024-07-03T15:45:57.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.942+0000 7fb7eee8f640 1 -- 172.21.15.28:0/95175473 shutdown_connections 2024-07-03T15:45:57.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.942+0000 7fb7eee8f640 1 --2- 172.21.15.28:0/95175473 >> [v2:172.21.15.28:6800/1642892231,v1:172.21.15.28:6801/1642892231] conn(0x7fb7d006dae0 0x7fb7d006ff90 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:57.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.942+0000 7fb7eee8f640 1 --2- 172.21.15.28:0/95175473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7e80ff550 0x7fb7e80faf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:57.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.942+0000 7fb7eee8f640 1 --2- 172.21.15.28:0/95175473 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7e8065990 0x7fb7e80fca20 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:45:57.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.942+0000 7fb7eee8f640 1 -- 172.21.15.28:0/95175473 >> 172.21.15.28:0/95175473 conn(0x7fb7e80f6a20 msgr2=0x7fb7e80f6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:45:57.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.942+0000 7fb7eee8f640 1 -- 172.21.15.28:0/95175473 shutdown_connections 2024-07-03T15:45:57.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:45:57.942+0000 7fb7eee8f640 1 -- 172.21.15.28:0/95175473 wait complete. 2024-07-03T15:45:58.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:58 smithi148 ceph-mon[36056]: from='client.14768 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:45:58.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:58 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/1118782719' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-03T15:45:58.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:58 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/95175473' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-03T15:45:58.502 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:58 smithi028 ceph-mon[29148]: from='client.14768 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:45:58.503 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:58 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/1118782719' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-03T15:45:58.503 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:58 smithi028 ceph-mon[29148]: from='client.? 172.21.15.28:0/95175473' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-03T15:45:59.430 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:59 smithi148 ceph-mon[36056]: from='client.14780 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:45:59.430 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:59 smithi148 ceph-mon[36056]: pgmap v16: 65 pgs: 65 active+clean; 197 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 301 KiB/s rd, 286 KiB/s wr, 87 op/s 2024-07-03T15:45:59.430 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:59 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:45:59.430 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:59 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:45:59.430 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:59 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-07-03T15:45:59.430 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:45:59 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:45:59.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:59 smithi028 ceph-mon[29148]: from='client.14780 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:45:59.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:59 smithi028 ceph-mon[29148]: pgmap v16: 65 pgs: 65 active+clean; 197 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 301 KiB/s rd, 286 KiB/s wr, 87 op/s 2024-07-03T15:45:59.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:59 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:45:59.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:59 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:45:59.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:59 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-07-03T15:45:59.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:45:59 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:46:00.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:00 smithi148 ceph-mon[36056]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-07-03T15:46:00.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:00 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi148.rjzsar", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-03T15:46:00.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:00 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-03T15:46:00.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:00 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:46:00.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:00 smithi028 ceph-mon[29148]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-07-03T15:46:00.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:00 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi148.rjzsar", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-03T15:46:00.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:00 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-03T15:46:00.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:00 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:46:01.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:01 smithi148 ceph-mon[36056]: pgmap v17: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 301 KiB/s rd, 286 KiB/s wr, 88 op/s 2024-07-03T15:46:01.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:01 smithi148 ceph-mon[36056]: Upgrade: Updating mgr.smithi148.rjzsar 2024-07-03T15:46:01.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:01 smithi148 ceph-mon[36056]: Deploying daemon mgr.smithi148.rjzsar on smithi148 2024-07-03T15:46:01.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:01 smithi028 ceph-mon[29148]: pgmap v17: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 301 KiB/s rd, 286 KiB/s wr, 88 op/s 2024-07-03T15:46:01.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:01 smithi028 ceph-mon[29148]: Upgrade: Updating mgr.smithi148.rjzsar 2024-07-03T15:46:01.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:01 smithi028 ceph-mon[29148]: Deploying daemon mgr.smithi148.rjzsar on smithi148 2024-07-03T15:46:02.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:02 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:46:02.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:02 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:46:03.508 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:03 smithi028 ceph-mon[29148]: pgmap v18: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 11 KiB/s rd, 11 KiB/s wr, 5 op/s 2024-07-03T15:46:03.521 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:03 smithi148 ceph-mon[36056]: pgmap v18: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 11 KiB/s rd, 11 KiB/s wr, 5 op/s 2024-07-03T15:46:04.619 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:04 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:04.619 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:04 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:04.619 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:04 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:46:04.619 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:04 smithi148 ceph-mon[36056]: pgmap v19: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 12 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-07-03T15:46:04.619 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:04 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:04.620 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:04 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:04.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:04 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:04.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:04 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:04.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:04 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:46:04.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:04 smithi028 ceph-mon[29148]: pgmap v19: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 12 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-07-03T15:46:04.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:04 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:04.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:04 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:07.582 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:07 smithi148 ceph-mon[36056]: pgmap v20: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 426 B/s wr, 2 op/s 2024-07-03T15:46:07.582 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:07 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:07.583 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:07 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:07.583 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:07 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:07.583 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:07 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:07.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:07 smithi028 ceph-mon[29148]: pgmap v20: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 426 B/s wr, 2 op/s 2024-07-03T15:46:07.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:07 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:07.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:07 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:07.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:07 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:07.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:07 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:09.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:09 smithi148 ceph-mon[36056]: pgmap v21: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 426 B/s wr, 2 op/s 2024-07-03T15:46:09.734 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:09 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:09.734 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:09 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:09.734 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:09 smithi148 ceph-mon[36056]: Standby manager daemon smithi148.rjzsar restarted 2024-07-03T15:46:09.734 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:09 smithi148 ceph-mon[36056]: Standby manager daemon smithi148.rjzsar started 2024-07-03T15:46:09.734 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:09 smithi148 ceph-mon[36056]: from='mgr.? 172.21.15.148:0/2607701708' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.rjzsar/crt"}]: dispatch 2024-07-03T15:46:09.734 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:09 smithi148 ceph-mon[36056]: from='mgr.? 172.21.15.148:0/2607701708' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-03T15:46:09.734 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:09 smithi148 ceph-mon[36056]: from='mgr.? 172.21.15.148:0/2607701708' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.rjzsar/key"}]: dispatch 2024-07-03T15:46:09.734 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:09 smithi148 ceph-mon[36056]: from='mgr.? 172.21.15.148:0/2607701708' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-03T15:46:09.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:09 smithi028 ceph-mon[29148]: pgmap v21: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 426 B/s wr, 2 op/s 2024-07-03T15:46:09.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:09 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:09.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:09 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:09.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:09 smithi028 ceph-mon[29148]: Standby manager daemon smithi148.rjzsar restarted 2024-07-03T15:46:09.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:09 smithi028 ceph-mon[29148]: Standby manager daemon smithi148.rjzsar started 2024-07-03T15:46:09.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:09 smithi028 ceph-mon[29148]: from='mgr.? 172.21.15.148:0/2607701708' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.rjzsar/crt"}]: dispatch 2024-07-03T15:46:09.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:09 smithi028 ceph-mon[29148]: from='mgr.? 172.21.15.148:0/2607701708' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-03T15:46:09.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:09 smithi028 ceph-mon[29148]: from='mgr.? 172.21.15.148:0/2607701708' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.rjzsar/key"}]: dispatch 2024-07-03T15:46:09.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:09 smithi028 ceph-mon[29148]: from='mgr.? 172.21.15.148:0/2607701708' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-03T15:46:10.833 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:10 smithi028 ceph-mon[29148]: mgrmap e30: smithi028.vtqrtz(active, since 37s), standbys: smithi148.rjzsar 2024-07-03T15:46:10.833 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:10 smithi028 ceph-mon[29148]: pgmap v22: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.4 KiB/s rd, 426 B/s wr, 3 op/s 2024-07-03T15:46:10.833 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:10 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:10.833 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:10 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:10.833 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:10 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:46:10.833 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:10 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:46:10.833 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:10 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:10.834 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:10 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:46:10.834 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:10 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:46:10.834 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:10 smithi028 ceph-mon[29148]: Upgrade: Setting container_image for all mgr 2024-07-03T15:46:10.834 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:10 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:10.834 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:10 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi028.vtqrtz"}]: dispatch 2024-07-03T15:46:10.834 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:10 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi028.vtqrtz"}]': finished 2024-07-03T15:46:10.834 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:10 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi148.rjzsar"}]: dispatch 2024-07-03T15:46:10.834 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:10 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi148.rjzsar"}]': finished 2024-07-03T15:46:10.834 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:10 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-07-03T15:46:10.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:10 smithi148 ceph-mon[36056]: mgrmap e30: smithi028.vtqrtz(active, since 37s), standbys: smithi148.rjzsar 2024-07-03T15:46:10.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:10 smithi148 ceph-mon[36056]: pgmap v22: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.4 KiB/s rd, 426 B/s wr, 3 op/s 2024-07-03T15:46:10.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:10 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:10.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:10 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:10.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:10 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:46:10.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:10 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:46:10.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:10 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:10.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:10 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:46:10.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:10 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:46:10.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:10 smithi148 ceph-mon[36056]: Upgrade: Setting container_image for all mgr 2024-07-03T15:46:10.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:10 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:10.984 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:10 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi028.vtqrtz"}]: dispatch 2024-07-03T15:46:10.984 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:10 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi028.vtqrtz"}]': finished 2024-07-03T15:46:10.984 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:10 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi148.rjzsar"}]: dispatch 2024-07-03T15:46:10.984 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:10 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi148.rjzsar"}]': finished 2024-07-03T15:46:10.984 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:10 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-07-03T15:46:12.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:12 smithi148 ceph-mon[36056]: pgmap v23: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-07-03T15:46:12.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:12 smithi148 ceph-mon[36056]: Upgrade: Updating mon.smithi028 2024-07-03T15:46:12.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:12 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:12.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:12 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-03T15:46:12.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:12 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-03T15:46:12.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:12 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:46:12.984 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:12 smithi148 ceph-mon[36056]: Deploying daemon mon.smithi028 on smithi028 2024-07-03T15:46:13.060 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:12 smithi028 ceph-mon[29148]: pgmap v23: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-07-03T15:46:13.060 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:12 smithi028 ceph-mon[29148]: Upgrade: Updating mon.smithi028 2024-07-03T15:46:13.060 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:12 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:13.060 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:12 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-03T15:46:13.061 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:12 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-03T15:46:13.061 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:12 smithi028 ceph-mon[29148]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:46:13.061 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:12 smithi028 ceph-mon[29148]: Deploying daemon mon.smithi028 on smithi028 2024-07-03T15:46:13.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:13 smithi028 systemd[1]: Stopping Ceph mon.smithi028 for a85e4be6-3951-11ef-bcab-c7b262605968... 2024-07-03T15:46:14.110 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:13 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-mon-smithi028[29144]: 2024-07-03T15:46:13.644+0000 7f30914fb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi028 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-07-03T15:46:14.110 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:13 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-mon-smithi028[29144]: 2024-07-03T15:46:13.644+0000 7f30914fb640 -1 mon.smithi028@0(leader) e2 *** Got Signal Terminated *** 2024-07-03T15:46:14.110 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:13 smithi028 podman[92688]: 2024-07-03 15:46:13.880342265 +0000 UTC m=+0.372767247 container died 7461f9bef4e0a95623bbd52ef076537cfa9eb65ae7b952d69e006ee86776c794 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-mon-smithi028, org.label-schema.schema-version=1.0, RELEASE=quincy-d4b063a, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240625, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-07-03T15:46:14.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:14 smithi028 podman[92688]: 2024-07-03 15:46:14.31923336 +0000 UTC m=+0.811658343 container cleanup 7461f9bef4e0a95623bbd52ef076537cfa9eb65ae7b952d69e006ee86776c794 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-mon-smithi028, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=quincy-d4b063a, org.label-schema.build-date=20240625, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-07-03T15:46:14.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:14 smithi028 bash[92688]: ceph-a85e4be6-3951-11ef-bcab-c7b262605968-mon-smithi028 2024-07-03T15:46:14.750 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:14 smithi028 podman[92702]: 2024-07-03 15:46:14.487716967 +0000 UTC m=+0.604826809 container remove 7461f9bef4e0a95623bbd52ef076537cfa9eb65ae7b952d69e006ee86776c794 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-mon-smithi028, ceph=True, org.label-schema.build-date=20240625, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, RELEASE=quincy-d4b063a, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_CLEAN=True) 2024-07-03T15:46:15.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:14 smithi028 systemd[1]: ceph-a85e4be6-3951-11ef-bcab-c7b262605968@mon.smithi028.service: Deactivated successfully. 2024-07-03T15:46:15.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:14 smithi028 systemd[1]: Stopped Ceph mon.smithi028 for a85e4be6-3951-11ef-bcab-c7b262605968. 2024-07-03T15:46:15.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:14 smithi028 systemd[1]: ceph-a85e4be6-3951-11ef-bcab-c7b262605968@mon.smithi028.service: Consumed 11.516s CPU time. 2024-07-03T15:46:15.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:14 smithi028 systemd[1]: Starting Ceph mon.smithi028 for a85e4be6-3951-11ef-bcab-c7b262605968... 2024-07-03T15:46:15.537 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 podman[92839]: 2024-07-03 15:46:15.163534056 +0000 UTC m=+0.102897015 container create 5093f7bf47d7d127fd7bce60c3a831acac39c7f908d95617cea2dfb1ac776bb4 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-mon-smithi028, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.build-date=20240625, CEPH_POINT_RELEASE=, RELEASE=squid-release-c00506d, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2) 2024-07-03T15:46:15.537 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 podman[92839]: 2024-07-03 15:46:15.078729205 +0000 UTC m=+0.018092182 image pull 9d49405603e8445bc510362cdeb05f243619be76b0f3897dd4c152b00cffe45f quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2 2024-07-03T15:46:15.537 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 podman[92839]: 2024-07-03 15:46:15.332669011 +0000 UTC m=+0.272031982 container init 5093f7bf47d7d127fd7bce60c3a831acac39c7f908d95617cea2dfb1ac776bb4 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-mon-smithi028, CEPH_POINT_RELEASE=, RELEASE=squid-release-c00506d, io.buildah.version=1.36.0, org.label-schema.build-date=20240625, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.vendor=CentOS) 2024-07-03T15:46:15.538 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 podman[92839]: 2024-07-03 15:46:15.335918476 +0000 UTC m=+0.275281444 container start 5093f7bf47d7d127fd7bce60c3a831acac39c7f908d95617cea2dfb1ac776bb4 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-mon-smithi028, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240625, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-release-c00506d) 2024-07-03T15:46:15.538 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: set uid:gid to 167:167 (ceph:ceph) 2024-07-03T15:46:15.538 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc), process ceph-mon, pid 2 2024-07-03T15:46:15.538 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: pidfile_write: ignore empty --pid-file 2024-07-03T15:46:15.538 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: load: jerasure load: lrc 2024-07-03T15:46:15.538 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: RocksDB version: 7.9.2 2024-07-03T15:46:15.538 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Git sha 0 2024-07-03T15:46:15.538 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Compile date 2024-07-01 20:58:27 2024-07-03T15:46:15.538 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: DB SUMMARY 2024-07-03T15:46:15.538 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: DB Session ID: NX3RLOVX6IB1QHMD4L8Z 2024-07-03T15:46:15.538 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: CURRENT file: CURRENT 2024-07-03T15:46:15.539 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: IDENTITY file: IDENTITY 2024-07-03T15:46:15.539 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: MANIFEST file: MANIFEST-000015 size: 972 Bytes 2024-07-03T15:46:15.539 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi028/store.db dir, Total Num: 1, files: 000030.sst 2024-07-03T15:46:15.539 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi028/store.db: 000028.log size: 2049365 ; 2024-07-03T15:46:15.539 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.error_if_exists: 0 2024-07-03T15:46:15.539 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.create_if_missing: 0 2024-07-03T15:46:15.539 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.paranoid_checks: 1 2024-07-03T15:46:15.539 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.flush_verify_memtable_count: 1 2024-07-03T15:46:15.539 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-07-03T15:46:15.539 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-07-03T15:46:15.539 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.env: 0x55c05a91d740 2024-07-03T15:46:15.539 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.fs: PosixFileSystem 2024-07-03T15:46:15.539 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.info_log: 0x55c05cf752c0 2024-07-03T15:46:15.539 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.max_file_opening_threads: 16 2024-07-03T15:46:15.540 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.statistics: (nil) 2024-07-03T15:46:15.540 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.use_fsync: 0 2024-07-03T15:46:15.540 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.max_log_file_size: 0 2024-07-03T15:46:15.540 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-07-03T15:46:15.540 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.log_file_time_to_roll: 0 2024-07-03T15:46:15.540 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.keep_log_file_num: 1000 2024-07-03T15:46:15.540 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.recycle_log_file_num: 0 2024-07-03T15:46:15.540 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.allow_fallocate: 1 2024-07-03T15:46:15.540 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.allow_mmap_reads: 0 2024-07-03T15:46:15.540 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.allow_mmap_writes: 0 2024-07-03T15:46:15.540 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.use_direct_reads: 0 2024-07-03T15:46:15.540 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-07-03T15:46:15.540 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.create_missing_column_families: 0 2024-07-03T15:46:15.540 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.db_log_dir: 2024-07-03T15:46:15.541 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.wal_dir: 2024-07-03T15:46:15.541 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.table_cache_numshardbits: 6 2024-07-03T15:46:15.541 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.WAL_ttl_seconds: 0 2024-07-03T15:46:15.541 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.WAL_size_limit_MB: 0 2024-07-03T15:46:15.541 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-07-03T15:46:15.541 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-07-03T15:46:15.541 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.is_fd_close_on_exec: 1 2024-07-03T15:46:15.541 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.advise_random_on_open: 1 2024-07-03T15:46:15.541 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.db_write_buffer_size: 0 2024-07-03T15:46:15.541 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.write_buffer_manager: 0x55c05cf841e0 2024-07-03T15:46:15.541 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-07-03T15:46:15.541 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-07-03T15:46:15.541 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.use_adaptive_mutex: 0 2024-07-03T15:46:15.541 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.rate_limiter: (nil) 2024-07-03T15:46:15.542 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-07-03T15:46:15.543 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.wal_recovery_mode: 2 2024-07-03T15:46:15.543 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.enable_thread_tracking: 0 2024-07-03T15:46:15.543 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.enable_pipelined_write: 0 2024-07-03T15:46:15.543 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.unordered_write: 0 2024-07-03T15:46:15.543 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-07-03T15:46:15.543 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-07-03T15:46:15.543 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-07-03T15:46:15.543 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-07-03T15:46:15.544 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.row_cache: None 2024-07-03T15:46:15.544 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.wal_filter: None 2024-07-03T15:46:15.544 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-07-03T15:46:15.544 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.allow_ingest_behind: 0 2024-07-03T15:46:15.544 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.two_write_queues: 0 2024-07-03T15:46:15.544 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.manual_wal_flush: 0 2024-07-03T15:46:15.544 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.wal_compression: 0 2024-07-03T15:46:15.544 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.atomic_flush: 0 2024-07-03T15:46:15.544 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-07-03T15:46:15.544 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.persist_stats_to_disk: 0 2024-07-03T15:46:15.544 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.write_dbid_to_manifest: 0 2024-07-03T15:46:15.544 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.log_readahead_size: 0 2024-07-03T15:46:15.544 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-07-03T15:46:15.545 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.best_efforts_recovery: 0 2024-07-03T15:46:15.545 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-07-03T15:46:15.545 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-07-03T15:46:15.545 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.allow_data_in_errors: 0 2024-07-03T15:46:15.545 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.db_host_id: __hostname__ 2024-07-03T15:46:15.545 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.enforce_single_del_contracts: true 2024-07-03T15:46:15.545 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.max_background_jobs: 2 2024-07-03T15:46:15.545 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.max_background_compactions: -1 2024-07-03T15:46:15.545 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.max_subcompactions: 1 2024-07-03T15:46:15.545 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-07-03T15:46:15.545 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-07-03T15:46:15.545 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.delayed_write_rate : 16777216 2024-07-03T15:46:15.545 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.max_total_wal_size: 0 2024-07-03T15:46:15.546 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-07-03T15:46:15.546 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.stats_dump_period_sec: 600 2024-07-03T15:46:15.546 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.stats_persist_period_sec: 600 2024-07-03T15:46:15.546 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-07-03T15:46:15.546 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.max_open_files: -1 2024-07-03T15:46:15.546 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.bytes_per_sync: 0 2024-07-03T15:46:15.546 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.wal_bytes_per_sync: 0 2024-07-03T15:46:15.546 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.strict_bytes_per_sync: 0 2024-07-03T15:46:15.546 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.compaction_readahead_size: 0 2024-07-03T15:46:15.546 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.max_background_flushes: -1 2024-07-03T15:46:15.546 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Compression algorithms supported: 2024-07-03T15:46:15.546 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: kZSTD supported: 0 2024-07-03T15:46:15.546 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: kXpressCompression supported: 0 2024-07-03T15:46:15.547 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: kBZip2Compression supported: 0 2024-07-03T15:46:15.547 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-07-03T15:46:15.547 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: kLZ4Compression supported: 1 2024-07-03T15:46:15.547 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: kZlibCompression supported: 1 2024-07-03T15:46:15.547 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: kLZ4HCCompression supported: 1 2024-07-03T15:46:15.547 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: kSnappyCompression supported: 1 2024-07-03T15:46:15.547 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Fast CRC32 supported: Supported on x86 2024-07-03T15:46:15.547 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: DMutex implementation: pthread_mutex_t 2024-07-03T15:46:15.547 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi028/store.db/MANIFEST-000015 2024-07-03T15:46:15.547 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-07-03T15:46:15.547 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-07-03T15:46:15.547 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.merge_operator: 2024-07-03T15:46:15.547 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.compaction_filter: None 2024-07-03T15:46:15.547 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.compaction_filter_factory: None 2024-07-03T15:46:15.548 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.sst_partitioner_factory: None 2024-07-03T15:46:15.548 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.memtable_factory: SkipListFactory 2024-07-03T15:46:15.548 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.table_factory: BlockBasedTable 2024-07-03T15:46:15.548 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c05cf74f60) 2024-07-03T15:46:15.548 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: cache_index_and_filter_blocks: 1 2024-07-03T15:46:15.548 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-07-03T15:46:15.548 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-07-03T15:46:15.548 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: pin_top_level_index_and_filter: 1 2024-07-03T15:46:15.548 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: index_type: 0 2024-07-03T15:46:15.548 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: data_block_index_type: 0 2024-07-03T15:46:15.548 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: index_shortening: 1 2024-07-03T15:46:15.548 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: data_block_hash_table_util_ratio: 0.750000 2024-07-03T15:46:15.548 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: checksum: 4 2024-07-03T15:46:15.549 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: no_block_cache: 0 2024-07-03T15:46:15.549 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: block_cache: 0x55c05cf6d1f0 2024-07-03T15:46:15.549 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: block_cache_name: BinnedLRUCache 2024-07-03T15:46:15.549 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: block_cache_options: 2024-07-03T15:46:15.549 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: capacity : 536870912 2024-07-03T15:46:15.549 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: num_shard_bits : 4 2024-07-03T15:46:15.549 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: strict_capacity_limit : 0 2024-07-03T15:46:15.549 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: high_pri_pool_ratio: 0.000 2024-07-03T15:46:15.549 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: block_cache_compressed: (nil) 2024-07-03T15:46:15.549 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: persistent_cache: (nil) 2024-07-03T15:46:15.549 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: block_size: 4096 2024-07-03T15:46:15.549 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: block_size_deviation: 10 2024-07-03T15:46:15.549 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: block_restart_interval: 16 2024-07-03T15:46:15.549 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: index_block_restart_interval: 1 2024-07-03T15:46:15.549 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: metadata_block_size: 4096 2024-07-03T15:46:15.550 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: partition_filters: 0 2024-07-03T15:46:15.550 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: use_delta_encoding: 1 2024-07-03T15:46:15.550 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: filter_policy: bloomfilter 2024-07-03T15:46:15.550 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: whole_key_filtering: 1 2024-07-03T15:46:15.550 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: verify_compression: 0 2024-07-03T15:46:15.550 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: read_amp_bytes_per_bit: 0 2024-07-03T15:46:15.550 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: format_version: 5 2024-07-03T15:46:15.550 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: enable_index_compression: 1 2024-07-03T15:46:15.550 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: block_align: 0 2024-07-03T15:46:15.550 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: max_auto_readahead_size: 262144 2024-07-03T15:46:15.550 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: prepopulate_block_cache: 0 2024-07-03T15:46:15.550 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: initial_auto_readahead_size: 8192 2024-07-03T15:46:15.550 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: num_file_reads_for_auto_readahead: 2 2024-07-03T15:46:15.550 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.write_buffer_size: 33554432 2024-07-03T15:46:15.551 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.max_write_buffer_number: 2 2024-07-03T15:46:15.551 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.compression: NoCompression 2024-07-03T15:46:15.551 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.bottommost_compression: Disabled 2024-07-03T15:46:15.551 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.prefix_extractor: nullptr 2024-07-03T15:46:15.551 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-07-03T15:46:15.551 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.num_levels: 7 2024-07-03T15:46:15.551 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-07-03T15:46:15.551 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-07-03T15:46:15.551 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-07-03T15:46:15.551 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-07-03T15:46:15.551 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-07-03T15:46:15.551 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-07-03T15:46:15.551 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-07-03T15:46:15.551 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-07-03T15:46:15.552 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-07-03T15:46:15.552 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-07-03T15:46:15.552 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-07-03T15:46:15.552 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-07-03T15:46:15.552 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.compression_opts.window_bits: -14 2024-07-03T15:46:15.552 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.compression_opts.level: 32767 2024-07-03T15:46:15.552 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.compression_opts.strategy: 0 2024-07-03T15:46:15.552 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-07-03T15:46:15.552 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-07-03T15:46:15.552 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-07-03T15:46:15.552 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-07-03T15:46:15.552 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.compression_opts.enabled: false 2024-07-03T15:46:15.552 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-07-03T15:46:15.552 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-07-03T15:46:15.553 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-07-03T15:46:15.553 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-07-03T15:46:15.553 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.target_file_size_base: 67108864 2024-07-03T15:46:15.553 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.target_file_size_multiplier: 1 2024-07-03T15:46:15.553 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-07-03T15:46:15.553 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-07-03T15:46:15.553 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-07-03T15:46:15.553 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-07-03T15:46:15.553 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-07-03T15:46:15.553 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-07-03T15:46:15.553 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-07-03T15:46:15.553 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-07-03T15:46:15.553 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-07-03T15:46:15.553 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-07-03T15:46:15.554 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-07-03T15:46:15.554 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-07-03T15:46:15.554 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-07-03T15:46:15.554 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.arena_block_size: 1048576 2024-07-03T15:46:15.554 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-07-03T15:46:15.554 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-07-03T15:46:15.554 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.disable_auto_compactions: 0 2024-07-03T15:46:15.554 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-07-03T15:46:15.554 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-07-03T15:46:15.554 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-07-03T15:46:15.554 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-07-03T15:46:15.554 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-07-03T15:46:15.555 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-07-03T15:46:15.555 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-07-03T15:46:15.555 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-07-03T15:46:15.555 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-07-03T15:46:15.555 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-07-03T15:46:15.555 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.table_properties_collectors: 2024-07-03T15:46:15.555 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.inplace_update_support: 0 2024-07-03T15:46:15.555 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.inplace_update_num_locks: 10000 2024-07-03T15:46:15.555 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-07-03T15:46:15.555 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-07-03T15:46:15.555 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.memtable_huge_page_size: 0 2024-07-03T15:46:15.555 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.bloom_locality: 0 2024-07-03T15:46:15.555 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.max_successive_merges: 0 2024-07-03T15:46:15.555 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.optimize_filters_for_hits: 0 2024-07-03T15:46:15.556 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.paranoid_file_checks: 0 2024-07-03T15:46:15.556 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.force_consistency_checks: 1 2024-07-03T15:46:15.556 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.report_bg_io_stats: 0 2024-07-03T15:46:15.556 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.ttl: 2592000 2024-07-03T15:46:15.556 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.periodic_compaction_seconds: 0 2024-07-03T15:46:15.556 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-07-03T15:46:15.556 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-07-03T15:46:15.556 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.enable_blob_files: false 2024-07-03T15:46:15.556 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.min_blob_size: 0 2024-07-03T15:46:15.556 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.blob_file_size: 268435456 2024-07-03T15:46:15.556 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.blob_compression_type: NoCompression 2024-07-03T15:46:15.556 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.enable_blob_garbage_collection: false 2024-07-03T15:46:15.556 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-07-03T15:46:15.556 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-07-03T15:46:15.557 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-07-03T15:46:15.557 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.blob_file_starting_level: 0 2024-07-03T15:46:15.557 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-07-03T15:46:15.557 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 30.sst 2024-07-03T15:46:15.557 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-07-03T15:46:15.557 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi028/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 32, last_sequence is 10268, log_number is 28,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-07-03T15:46:15.557 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 28 2024-07-03T15:46:15.557 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 2f240546-3b3a-499f-a585-5ecb60da485f 2024-07-03T15:46:15.557 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: EVENT_LOG_v1 {"time_micros": 1720021575363293, "job": 1, "event": "recovery_started", "wal_files": [28]} 2024-07-03T15:46:15.557 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #28 mode 2 2024-07-03T15:46:15.557 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: EVENT_LOG_v1 {"time_micros": 1720021575369766, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 33, "file_size": 1869152, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 10269, "largest_seqno": 11114, "table_properties": {"data_size": 1864528, "index_size": 2586, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1029, "raw_key_size": 9214, "raw_average_key_size": 23, "raw_value_size": 1856039, "raw_average_value_size": 4722, "num_data_blocks": 118, "num_entries": 393, "num_filter_entries": 393, "num_deletions": 2, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1720021575, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "2f240546-3b3a-499f-a585-5ecb60da485f", "db_session_id": "NX3RLOVX6IB1QHMD4L8Z", "orig_file_number": 33, "seqno_to_time_mapping": "N/A"}} 2024-07-03T15:46:15.557 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: EVENT_LOG_v1 {"time_micros": 1720021575369885, "job": 1, "event": "recovery_finished"} 2024-07-03T15:46:15.558 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: [db/version_set.cc:5047] Creating manifest 35 2024-07-03T15:46:15.558 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-07-03T15:46:15.558 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi028/store.db/000028.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-07-03T15:46:15.558 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55c05cf98e00 2024-07-03T15:46:15.558 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: DB pointer 0x55c05d08e000 2024-07-03T15:46:15.558 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-07-03T15:46:15.558 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-07-03T15:46:15.558 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: ** DB Stats ** 2024-07-03T15:46:15.558 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-07-03T15:46:15.558 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-07-03T15:46:15.558 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-07-03T15:46:15.558 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-07-03T15:46:15.559 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-07-03T15:46:15.559 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-07-03T15:46:15.559 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-07-03T15:46:15.559 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: 2024-07-03T15:46:15.559 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: ** Compaction Stats [default] ** 2024-07-03T15:46:15.559 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-07-03T15:46:15.559 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-07-03T15:46:15.559 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: L0 1/0 1.78 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 520.5 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-07-03T15:46:15.559 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: L6 1/0 5.25 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-07-03T15:46:15.559 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Sum 2/0 7.03 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 520.5 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-07-03T15:46:15.559 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 520.5 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-07-03T15:46:15.559 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: 2024-07-03T15:46:15.559 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: ** Compaction Stats [default] ** 2024-07-03T15:46:15.559 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-07-03T15:46:15.560 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-07-03T15:46:15.560 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 520.5 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-07-03T15:46:15.560 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: 2024-07-03T15:46:15.560 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-07-03T15:46:15.560 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: 2024-07-03T15:46:15.560 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-07-03T15:46:15.560 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Flush(GB): cumulative 0.002, interval 0.002 2024-07-03T15:46:15.560 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-07-03T15:46:15.560 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-07-03T15:46:15.560 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-07-03T15:46:15.560 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: AddFile(Keys): cumulative 0, interval 0 2024-07-03T15:46:15.560 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Cumulative compaction: 0.00 GB write, 198.11 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-07-03T15:46:15.560 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Interval compaction: 0.00 GB write, 198.11 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-07-03T15:46:15.561 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-07-03T15:46:15.561 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Block cache BinnedLRUCache@0x55c05cf6d1f0#2 capacity: 512.00 MB usage: 3.98 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-07-03T15:46:15.561 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,1.16 KB,0.000220537%) IndexBlock(1,2.83 KB,0.000539422%) Misc(1,0.00 KB,0%) 2024-07-03T15:46:15.561 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: 2024-07-03T15:46:15.561 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: ** File Read Latency Histogram By Level [default] ** 2024-07-03T15:46:15.561 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: starting mon.smithi028 rank 0 at public addrs [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] at bind addrs [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi028 fsid a85e4be6-3951-11ef-bcab-c7b262605968 2024-07-03T15:46:15.561 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: mon.smithi028@-1(???) e2 preinit fsid a85e4be6-3951-11ef-bcab-c7b262605968 2024-07-03T15:46:15.561 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: mon.smithi028@-1(???).mds e17 new map 2024-07-03T15:46:15.561 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: mon.smithi028@-1(???).mds e17 print_map 2024-07-03T15:46:15.561 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: e17 2024-07-03T15:46:15.561 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-07-03T15:46:15.561 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:46:15.561 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: legacy client fscid: 1 2024-07-03T15:46:15.562 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: 2024-07-03T15:46:15.562 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Filesystem 'cephfs' (1) 2024-07-03T15:46:15.562 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: fs_name cephfs 2024-07-03T15:46:15.562 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: epoch 16 2024-07-03T15:46:15.562 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-03T15:46:15.562 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: created 2024-07-03T15:41:40.656298+0000 2024-07-03T15:46:15.562 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: modified 2024-07-03T15:41:56.454984+0000 2024-07-03T15:46:15.562 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: tableserver 0 2024-07-03T15:46:15.562 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: root 0 2024-07-03T15:46:15.562 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: session_timeout 60 2024-07-03T15:46:15.562 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: session_autoclose 300 2024-07-03T15:46:15.562 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: max_file_size 1099511627776 2024-07-03T15:46:15.562 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: max_xattr_size 65536 2024-07-03T15:46:15.563 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: required_client_features {} 2024-07-03T15:46:15.563 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: last_failure 0 2024-07-03T15:46:15.563 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: last_failure_osd_epoch 0 2024-07-03T15:46:15.563 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:46:15.563 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: max_mds 1 2024-07-03T15:46:15.563 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: in 0 2024-07-03T15:46:15.563 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: up {0=14450} 2024-07-03T15:46:15.563 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: failed 2024-07-03T15:46:15.563 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: damaged 2024-07-03T15:46:15.563 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: stopped 2024-07-03T15:46:15.563 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: data_pools [3] 2024-07-03T15:46:15.563 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: metadata_pool 2 2024-07-03T15:46:15.563 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: inline_data disabled 2024-07-03T15:46:15.563 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: balancer 2024-07-03T15:46:15.564 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: bal_rank_mask -1 2024-07-03T15:46:15.564 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: standby_count_wanted 1 2024-07-03T15:46:15.564 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: qdb_cluster leader: 0 members: 2024-07-03T15:46:15.564 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: [mds.cephfs.smithi028.rhpvve{0:14450} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.28:6826/1613936042,v1:172.21.15.28:6827/1613936042] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:46:15.564 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: [mds.cephfs.smithi028.wmbxew{0:14460} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.28:6828/1730336821,v1:172.21.15.28:6829/1730336821] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:46:15.564 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: 2024-07-03T15:46:15.564 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: 2024-07-03T15:46:15.564 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Standby daemons: 2024-07-03T15:46:15.564 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: 2024-07-03T15:46:15.564 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: [mds.cephfs.smithi148.szomfl{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/4171231858,v1:172.21.15.148:6827/4171231858] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:46:15.564 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: [mds.cephfs.smithi148.xgpuld{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/2347536398,v1:172.21.15.148:6825/2347536398] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:46:15.564 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: mon.smithi028@-1(???).osd e47 crush map has features 3314933000852226048, adjusting msgr requires 2024-07-03T15:46:15.564 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: mon.smithi028@-1(???).osd e47 crush map has features 288514051259236352, adjusting msgr requires 2024-07-03T15:46:15.564 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: mon.smithi028@-1(???).osd e47 crush map has features 288514051259236352, adjusting msgr requires 2024-07-03T15:46:15.565 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: mon.smithi028@-1(???).osd e47 crush map has features 288514051259236352, adjusting msgr requires 2024-07-03T15:46:15.565 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 ceph-mon[92858]: mon.smithi028@-1(???).paxosservice(auth 1..20) refresh upgraded, format 0 -> 3 2024-07-03T15:46:15.565 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 bash[92839]: 5093f7bf47d7d127fd7bce60c3a831acac39c7f908d95617cea2dfb1ac776bb4 2024-07-03T15:46:15.565 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:15 smithi028 systemd[1]: Started Ceph mon.smithi028 for a85e4be6-3951-11ef-bcab-c7b262605968. 2024-07-03T15:46:16.853 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:16 smithi028 ceph-mon[92858]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi028"}]: dispatch 2024-07-03T15:46:16.853 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:16 smithi028 ceph-mon[92858]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-07-03T15:46:16.853 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:16 smithi028 ceph-mon[92858]: pgmap v25: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:46:16.854 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:16 smithi028 ceph-mon[92858]: mon.smithi028 calling monitor election 2024-07-03T15:46:16.854 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:16 smithi028 ceph-mon[92858]: mon.smithi028 is new leader, mons smithi028,smithi148 in quorum (ranks 0,1) 2024-07-03T15:46:16.854 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:16 smithi028 ceph-mon[92858]: monmap epoch 2 2024-07-03T15:46:16.854 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:16 smithi028 ceph-mon[92858]: fsid a85e4be6-3951-11ef-bcab-c7b262605968 2024-07-03T15:46:16.854 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:16 smithi028 ceph-mon[92858]: last_changed 2024-07-03T15:37:24.935576+0000 2024-07-03T15:46:16.854 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:16 smithi028 ceph-mon[92858]: created 2024-07-03T15:34:53.167064+0000 2024-07-03T15:46:16.854 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:16 smithi028 ceph-mon[92858]: min_mon_release 17 (quincy) 2024-07-03T15:46:16.854 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:16 smithi028 ceph-mon[92858]: election_strategy: 1 2024-07-03T15:46:16.854 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:16 smithi028 ceph-mon[92858]: 0: [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] mon.smithi028 2024-07-03T15:46:16.854 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:16 smithi028 ceph-mon[92858]: 1: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.smithi148 2024-07-03T15:46:16.854 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:16 smithi028 ceph-mon[92858]: fsmap cephfs:1 {0=cephfs.smithi028.rhpvve=up:active} 1 up:standby-replay 2 up:standby 2024-07-03T15:46:16.854 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:16 smithi028 ceph-mon[92858]: osdmap e47: 6 total, 6 up, 6 in 2024-07-03T15:46:16.854 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:16 smithi028 ceph-mon[92858]: mgrmap e30: smithi028.vtqrtz(active, since 45s), standbys: smithi148.rjzsar 2024-07-03T15:46:16.855 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:16 smithi028 ceph-mon[92858]: overall HEALTH_OK 2024-07-03T15:46:16.855 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:16 smithi028 ceph-mon[92858]: from='mgr.14700 ' entity='' 2024-07-03T15:46:16.855 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:16 smithi028 ceph-mon[92858]: mgrmap e31: smithi028.vtqrtz(active, since 45s), standbys: smithi148.rjzsar 2024-07-03T15:46:17.105 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:16 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi028"}]: dispatch 2024-07-03T15:46:17.105 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:16 smithi148 ceph-mon[36056]: from='mgr.14700 172.21.15.28:0/4272894678' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-07-03T15:46:17.105 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:16 smithi148 ceph-mon[36056]: pgmap v25: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:46:17.105 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:16 smithi148 ceph-mon[36056]: mon.smithi028 calling monitor election 2024-07-03T15:46:17.105 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:16 smithi148 ceph-mon[36056]: mon.smithi028 is new leader, mons smithi028,smithi148 in quorum (ranks 0,1) 2024-07-03T15:46:17.105 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:16 smithi148 ceph-mon[36056]: monmap epoch 2 2024-07-03T15:46:17.105 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:16 smithi148 ceph-mon[36056]: fsid a85e4be6-3951-11ef-bcab-c7b262605968 2024-07-03T15:46:17.106 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:16 smithi148 ceph-mon[36056]: last_changed 2024-07-03T15:37:24.935576+0000 2024-07-03T15:46:17.106 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:16 smithi148 ceph-mon[36056]: created 2024-07-03T15:34:53.167064+0000 2024-07-03T15:46:17.106 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:16 smithi148 ceph-mon[36056]: min_mon_release 17 (quincy) 2024-07-03T15:46:17.106 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:16 smithi148 ceph-mon[36056]: election_strategy: 1 2024-07-03T15:46:17.106 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:16 smithi148 ceph-mon[36056]: 0: [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] mon.smithi028 2024-07-03T15:46:17.106 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:16 smithi148 ceph-mon[36056]: 1: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.smithi148 2024-07-03T15:46:17.106 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:16 smithi148 ceph-mon[36056]: fsmap cephfs:1 {0=cephfs.smithi028.rhpvve=up:active} 1 up:standby-replay 2 up:standby 2024-07-03T15:46:17.106 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:16 smithi148 ceph-mon[36056]: osdmap e47: 6 total, 6 up, 6 in 2024-07-03T15:46:17.106 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:16 smithi148 ceph-mon[36056]: mgrmap e30: smithi028.vtqrtz(active, since 45s), standbys: smithi148.rjzsar 2024-07-03T15:46:17.106 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:16 smithi148 ceph-mon[36056]: overall HEALTH_OK 2024-07-03T15:46:17.106 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:16 smithi148 ceph-mon[36056]: from='mgr.14700 ' entity='' 2024-07-03T15:46:17.106 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:16 smithi148 ceph-mon[36056]: mgrmap e31: smithi028.vtqrtz(active, since 45s), standbys: smithi148.rjzsar 2024-07-03T15:46:21.814 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:21 smithi148 ceph-mon[36056]: Standby manager daemon smithi148.rjzsar restarted 2024-07-03T15:46:21.814 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:21 smithi148 ceph-mon[36056]: Standby manager daemon smithi148.rjzsar started 2024-07-03T15:46:21.814 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:21 smithi148 ceph-mon[36056]: from='mgr.? 172.21.15.148:0/4046617249' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.rjzsar/crt"}]: dispatch 2024-07-03T15:46:21.814 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:21 smithi148 ceph-mon[36056]: from='mgr.? 172.21.15.148:0/4046617249' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-03T15:46:21.814 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:21 smithi148 ceph-mon[36056]: from='mgr.? 172.21.15.148:0/4046617249' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.rjzsar/key"}]: dispatch 2024-07-03T15:46:21.815 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:21 smithi148 ceph-mon[36056]: from='mgr.? 172.21.15.148:0/4046617249' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-03T15:46:22.045 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:21 smithi028 ceph-mon[92858]: Standby manager daemon smithi148.rjzsar restarted 2024-07-03T15:46:22.046 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:21 smithi028 ceph-mon[92858]: Standby manager daemon smithi148.rjzsar started 2024-07-03T15:46:22.046 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:21 smithi028 ceph-mon[92858]: from='mgr.? 172.21.15.148:0/4046617249' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.rjzsar/crt"}]: dispatch 2024-07-03T15:46:22.046 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:21 smithi028 ceph-mon[92858]: from='mgr.? 172.21.15.148:0/4046617249' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-03T15:46:22.046 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:21 smithi028 ceph-mon[92858]: from='mgr.? 172.21.15.148:0/4046617249' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.rjzsar/key"}]: dispatch 2024-07-03T15:46:22.046 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:21 smithi028 ceph-mon[92858]: from='mgr.? 172.21.15.148:0/4046617249' entity='mgr.smithi148.rjzsar' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-03T15:46:23.061 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:22 smithi028 ceph-mon[92858]: mgrmap e32: smithi028.vtqrtz(active, since 50s), standbys: smithi148.rjzsar 2024-07-03T15:46:23.062 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:22 smithi028 ceph-mon[92858]: Active manager daemon smithi028.vtqrtz restarted 2024-07-03T15:46:23.062 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:22 smithi028 ceph-mon[92858]: Activating manager daemon smithi028.vtqrtz 2024-07-03T15:46:23.062 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:22 smithi028 ceph-mon[92858]: osdmap e48: 6 total, 6 up, 6 in 2024-07-03T15:46:23.062 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:22 smithi028 ceph-mon[92858]: mgrmap e33: smithi028.vtqrtz(active, starting, since 0.00562276s), standbys: smithi148.rjzsar 2024-07-03T15:46:23.062 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:22 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi028"}]: dispatch 2024-07-03T15:46:23.062 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:22 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-07-03T15:46:23.062 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:22 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi028.rhpvve"}]: dispatch 2024-07-03T15:46:23.062 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:22 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi028.wmbxew"}]: dispatch 2024-07-03T15:46:23.062 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:22 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.szomfl"}]: dispatch 2024-07-03T15:46:23.062 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:22 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.xgpuld"}]: dispatch 2024-07-03T15:46:23.062 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:22 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mgr metadata", "who": "smithi028.vtqrtz", "id": "smithi028.vtqrtz"}]: dispatch 2024-07-03T15:46:23.062 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:22 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mgr metadata", "who": "smithi148.rjzsar", "id": "smithi148.rjzsar"}]: dispatch 2024-07-03T15:46:23.063 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:22 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-03T15:46:23.063 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:22 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-03T15:46:23.063 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:22 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-03T15:46:23.063 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:22 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-03T15:46:23.063 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:22 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-03T15:46:23.063 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:22 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-03T15:46:23.063 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:22 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-03T15:46:23.063 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:22 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-03T15:46:23.063 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:22 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-03T15:46:23.063 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:22 smithi028 ceph-mon[92858]: Manager daemon smithi028.vtqrtz is now available 2024-07-03T15:46:23.063 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:22 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:46:23.063 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:22 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:46:23.063 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:22 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi028.vtqrtz/mirror_snapshot_schedule"}]: dispatch 2024-07-03T15:46:23.064 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:22 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi028.vtqrtz/trash_purge_schedule"}]: dispatch 2024-07-03T15:46:23.142 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:22 smithi148 ceph-mon[36056]: mgrmap e32: smithi028.vtqrtz(active, since 50s), standbys: smithi148.rjzsar 2024-07-03T15:46:23.142 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:22 smithi148 ceph-mon[36056]: Active manager daemon smithi028.vtqrtz restarted 2024-07-03T15:46:23.142 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:22 smithi148 ceph-mon[36056]: Activating manager daemon smithi028.vtqrtz 2024-07-03T15:46:23.142 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:22 smithi148 ceph-mon[36056]: osdmap e48: 6 total, 6 up, 6 in 2024-07-03T15:46:23.142 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:22 smithi148 ceph-mon[36056]: mgrmap e33: smithi028.vtqrtz(active, starting, since 0.00562276s), standbys: smithi148.rjzsar 2024-07-03T15:46:23.142 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:22 smithi148 ceph-mon[36056]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi028"}]: dispatch 2024-07-03T15:46:23.142 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:22 smithi148 ceph-mon[36056]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-07-03T15:46:23.142 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:22 smithi148 ceph-mon[36056]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi028.rhpvve"}]: dispatch 2024-07-03T15:46:23.142 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:22 smithi148 ceph-mon[36056]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi028.wmbxew"}]: dispatch 2024-07-03T15:46:23.142 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:22 smithi148 ceph-mon[36056]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.szomfl"}]: dispatch 2024-07-03T15:46:23.142 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:22 smithi148 ceph-mon[36056]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.xgpuld"}]: dispatch 2024-07-03T15:46:23.143 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:22 smithi148 ceph-mon[36056]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mgr metadata", "who": "smithi028.vtqrtz", "id": "smithi028.vtqrtz"}]: dispatch 2024-07-03T15:46:23.143 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:22 smithi148 ceph-mon[36056]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mgr metadata", "who": "smithi148.rjzsar", "id": "smithi148.rjzsar"}]: dispatch 2024-07-03T15:46:23.143 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:22 smithi148 ceph-mon[36056]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-03T15:46:23.143 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:22 smithi148 ceph-mon[36056]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-03T15:46:23.143 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:22 smithi148 ceph-mon[36056]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-03T15:46:23.143 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:22 smithi148 ceph-mon[36056]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-03T15:46:23.143 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:22 smithi148 ceph-mon[36056]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-03T15:46:23.143 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:22 smithi148 ceph-mon[36056]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-03T15:46:23.143 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:22 smithi148 ceph-mon[36056]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-03T15:46:23.143 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:22 smithi148 ceph-mon[36056]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-03T15:46:23.143 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:22 smithi148 ceph-mon[36056]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-03T15:46:23.143 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:22 smithi148 ceph-mon[36056]: Manager daemon smithi028.vtqrtz is now available 2024-07-03T15:46:23.143 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:22 smithi148 ceph-mon[36056]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:46:23.143 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:22 smithi148 ceph-mon[36056]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:46:23.144 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:22 smithi148 ceph-mon[36056]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi028.vtqrtz/mirror_snapshot_schedule"}]: dispatch 2024-07-03T15:46:23.144 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:22 smithi148 ceph-mon[36056]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi028.vtqrtz/trash_purge_schedule"}]: dispatch 2024-07-03T15:46:24.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:24 smithi028 ceph-mon[92858]: mgrmap e34: smithi028.vtqrtz(active, since 1.01084s), standbys: smithi148.rjzsar 2024-07-03T15:46:24.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:24 smithi028 ceph-mon[92858]: [03/Jul/2024:15:46:23] ENGINE Bus STARTING 2024-07-03T15:46:24.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:24 smithi028 ceph-mon[92858]: [03/Jul/2024:15:46:23] ENGINE Serving on http://172.21.15.28:8765 2024-07-03T15:46:24.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:24 smithi028 ceph-mon[92858]: [03/Jul/2024:15:46:23] ENGINE Serving on https://172.21.15.28:7150 2024-07-03T15:46:24.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:24 smithi028 ceph-mon[92858]: [03/Jul/2024:15:46:23] ENGINE Bus STARTED 2024-07-03T15:46:24.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:24 smithi028 ceph-mon[92858]: [03/Jul/2024:15:46:23] ENGINE Error in HTTPServer.serve 2024-07-03T15:46:24.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: Traceback (most recent call last): 2024-07-03T15:46:24.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-03T15:46:24.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: self._connections.run(self.expiration_interval) 2024-07-03T15:46:24.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-03T15:46:24.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: self._run(expiration_interval) 2024-07-03T15:46:24.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-03T15:46:24.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-07-03T15:46:24.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-03T15:46:24.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-03T15:46:24.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-03T15:46:24.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: s = self.context.wrap_socket( 2024-07-03T15:46:24.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-03T15:46:24.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: return self.sslsocket_class._create( 2024-07-03T15:46:24.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-03T15:46:24.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: self.do_handshake() 2024-07-03T15:46:24.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-03T15:46:24.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: self._sslobj.do_handshake() 2024-07-03T15:46:24.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-03T15:46:24.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:24 smithi148 ceph-mon[36056]: mgrmap e34: smithi028.vtqrtz(active, since 1.01084s), standbys: smithi148.rjzsar 2024-07-03T15:46:24.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:24 smithi148 ceph-mon[36056]: [03/Jul/2024:15:46:23] ENGINE Bus STARTING 2024-07-03T15:46:24.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:24 smithi148 ceph-mon[36056]: [03/Jul/2024:15:46:23] ENGINE Serving on http://172.21.15.28:8765 2024-07-03T15:46:24.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:24 smithi148 ceph-mon[36056]: [03/Jul/2024:15:46:23] ENGINE Serving on https://172.21.15.28:7150 2024-07-03T15:46:24.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:24 smithi148 ceph-mon[36056]: [03/Jul/2024:15:46:23] ENGINE Bus STARTED 2024-07-03T15:46:24.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:24 smithi148 ceph-mon[36056]: [03/Jul/2024:15:46:23] ENGINE Error in HTTPServer.serve 2024-07-03T15:46:24.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Traceback (most recent call last): 2024-07-03T15:46:24.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-03T15:46:24.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: self._connections.run(self.expiration_interval) 2024-07-03T15:46:24.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-03T15:46:24.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: self._run(expiration_interval) 2024-07-03T15:46:24.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-03T15:46:24.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-07-03T15:46:24.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-03T15:46:24.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-03T15:46:24.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-03T15:46:24.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: s = self.context.wrap_socket( 2024-07-03T15:46:24.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-03T15:46:24.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: return self.sslsocket_class._create( 2024-07-03T15:46:24.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-03T15:46:24.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: self.do_handshake() 2024-07-03T15:46:24.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-03T15:46:24.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: self._sslobj.do_handshake() 2024-07-03T15:46:24.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-03T15:46:25.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:25 smithi028 ceph-mon[92858]: pgmap v4: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-07-03T15:46:25.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:25 smithi028 ceph-mon[92858]: mgrmap e35: smithi028.vtqrtz(active, since 2s), standbys: smithi148.rjzsar 2024-07-03T15:46:25.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:25 smithi148 ceph-mon[36056]: pgmap v4: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-07-03T15:46:25.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:25 smithi148 ceph-mon[36056]: mgrmap e35: smithi028.vtqrtz(active, since 2s), standbys: smithi148.rjzsar 2024-07-03T15:46:26.570 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:26 smithi148 ceph-mon[36056]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:26.570 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:26 smithi148 ceph-mon[36056]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:26.570 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:26 smithi148 ceph-mon[36056]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:26.570 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:26 smithi148 ceph-mon[36056]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:26.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:26 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:26.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:26 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:26.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:26 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:26.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:26 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:27.376 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:27 smithi148 ceph-mon[36056]: pgmap v5: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-07-03T15:46:27.376 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:27 smithi148 ceph-mon[36056]: mgrmap e36: smithi028.vtqrtz(active, since 4s), standbys: smithi148.rjzsar 2024-07-03T15:46:27.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:27 smithi028 ceph-mon[92858]: pgmap v5: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-07-03T15:46:27.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:27 smithi028 ceph-mon[92858]: mgrmap e36: smithi028.vtqrtz(active, since 4s), standbys: smithi148.rjzsar 2024-07-03T15:46:28.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.027+0000 7f98be6fd640 1 -- 172.21.15.28:0/292952137 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f98b80fd640 msgr2=0x7f98b80fdab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:46:28.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.027+0000 7f98be6fd640 1 --2- 172.21.15.28:0/292952137 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f98b80fd640 0x7f98b80fdab0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f98ac0048d0 tx=0x7f98ac009da0 comp rx=0 tx=0).stop 2024-07-03T15:46:28.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.027+0000 7f98be6fd640 1 -- 172.21.15.28:0/292952137 shutdown_connections 2024-07-03T15:46:28.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.027+0000 7f98be6fd640 1 --2- 172.21.15.28:0/292952137 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f98b80fd640 0x7f98b80fdab0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:46:28.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.027+0000 7f98be6fd640 1 --2- 172.21.15.28:0/292952137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98b80fc370 0x7f98b80fc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:46:28.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.027+0000 7f98be6fd640 1 -- 172.21.15.28:0/292952137 >> 172.21.15.28:0/292952137 conn(0x7f98b80f7b50 msgr2=0x7f98b80f9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:46:28.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.027+0000 7f98be6fd640 1 -- 172.21.15.28:0/292952137 shutdown_connections 2024-07-03T15:46:28.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.027+0000 7f98be6fd640 1 -- 172.21.15.28:0/292952137 wait complete. 2024-07-03T15:46:28.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.028+0000 7f98be6fd640 1 Processor -- start 2024-07-03T15:46:28.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.028+0000 7f98be6fd640 1 -- start start 2024-07-03T15:46:28.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.028+0000 7f98be6fd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f98b80fc370 0x7f98b8071310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:46:28.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.028+0000 7f98be6fd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98b80fd640 0x7f98b806f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:46:28.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.028+0000 7f98be6fd640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b806fdb0 con 0x7f98b80fc370 2024-07-03T15:46:28.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.028+0000 7f98be6fd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b806ff20 con 0x7f98b80fd640 2024-07-03T15:46:28.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.028+0000 7f98b7fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f98b80fc370 0x7f98b8071310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:46:28.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.028+0000 7f98b7fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f98b80fc370 0x7f98b8071310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49644/0 (socket says 172.21.15.28:49644) 2024-07-03T15:46:28.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.028+0000 7f98b7fff640 1 -- 172.21.15.28:0/664764939 learned_addr learned my addr 172.21.15.28:0/664764939 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:46:28.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.028+0000 7f98b77fe640 1 --2- 172.21.15.28:0/664764939 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98b80fd640 0x7f98b806f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:46:28.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.028+0000 7f98b7fff640 1 -- 172.21.15.28:0/664764939 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98b80fd640 msgr2=0x7f98b806f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:46:28.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.028+0000 7f98b7fff640 1 --2- 172.21.15.28:0/664764939 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98b80fd640 0x7f98b806f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:46:28.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.028+0000 7f98b7fff640 1 -- 172.21.15.28:0/664764939 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98ac005040 con 0x7f98b80fc370 2024-07-03T15:46:28.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.029+0000 7f98b7fff640 1 --2- 172.21.15.28:0/664764939 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f98b80fc370 0x7f98b8071310 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f98a8002a50 tx=0x7f98a8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:46:28.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.029+0000 7f98b4ff9640 1 -- 172.21.15.28:0/664764939 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98a8007c30 con 0x7f98b80fc370 2024-07-03T15:46:28.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.029+0000 7f98b4ff9640 1 -- 172.21.15.28:0/664764939 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f98a800f040 con 0x7f98b80fc370 2024-07-03T15:46:28.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.029+0000 7f98be6fd640 1 -- 172.21.15.28:0/664764939 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98b80701b0 con 0x7f98b80fc370 2024-07-03T15:46:28.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.029+0000 7f98b4ff9640 1 -- 172.21.15.28:0/664764939 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98a800dad0 con 0x7f98b80fc370 2024-07-03T15:46:28.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.029+0000 7f98be6fd640 1 -- 172.21.15.28:0/664764939 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98b8114fb0 con 0x7f98b80fc370 2024-07-03T15:46:28.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.030+0000 7f98be6fd640 1 -- 172.21.15.28:0/664764939 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98b80fb7d0 con 0x7f98b80fc370 2024-07-03T15:46:28.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.030+0000 7f98b4ff9640 1 -- 172.21.15.28:0/664764939 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 36) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f98a800dc70 con 0x7f98b80fc370 2024-07-03T15:46:28.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.031+0000 7f98b4ff9640 1 --2- 172.21.15.28:0/664764939 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f98a006e1e0 0x7f98a0070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:46:28.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.031+0000 7f98b4ff9640 1 -- 172.21.15.28:0/664764939 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f98a808d500 con 0x7f98b80fc370 2024-07-03T15:46:28.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.031+0000 7f98b77fe640 1 --2- 172.21.15.28:0/664764939 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f98a006e1e0 0x7f98a0070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:46:28.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.032+0000 7f98b77fe640 1 --2- 172.21.15.28:0/664764939 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f98a006e1e0 0x7f98a0070690 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f98ac0043d0 tx=0x7f98ac035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:46:28.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.036+0000 7f98b4ff9640 1 -- 172.21.15.28:0/664764939 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f98a8056790 con 0x7f98b80fc370 2024-07-03T15:46:28.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.241+0000 7f98be6fd640 1 -- 172.21.15.28:0/664764939 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f98b8101920 con 0x7f98a006e1e0 2024-07-03T15:46:28.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.243+0000 7f98b4ff9640 1 -- 172.21.15.28:0/664764939 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f98b8101920 con 0x7f98a006e1e0 2024-07-03T15:46:28.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.245+0000 7f98be6fd640 1 -- 172.21.15.28:0/664764939 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f98a006e1e0 msgr2=0x7f98a0070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:46:28.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.245+0000 7f98be6fd640 1 --2- 172.21.15.28:0/664764939 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f98a006e1e0 0x7f98a0070690 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f98ac0043d0 tx=0x7f98ac035040 comp rx=0 tx=0).stop 2024-07-03T15:46:28.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.245+0000 7f98be6fd640 1 -- 172.21.15.28:0/664764939 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f98b80fc370 msgr2=0x7f98b8071310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:46:28.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.245+0000 7f98be6fd640 1 --2- 172.21.15.28:0/664764939 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f98b80fc370 0x7f98b8071310 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f98a8002a50 tx=0x7f98a8004600 comp rx=0 tx=0).stop 2024-07-03T15:46:28.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.245+0000 7f98be6fd640 1 -- 172.21.15.28:0/664764939 shutdown_connections 2024-07-03T15:46:28.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.245+0000 7f98be6fd640 1 --2- 172.21.15.28:0/664764939 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f98a006e1e0 0x7f98a0070690 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:46:28.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.245+0000 7f98be6fd640 1 --2- 172.21.15.28:0/664764939 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98b80fd640 0x7f98b806f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:46:28.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.245+0000 7f98be6fd640 1 --2- 172.21.15.28:0/664764939 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f98b80fc370 0x7f98b8071310 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:46:28.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.245+0000 7f98be6fd640 1 -- 172.21.15.28:0/664764939 >> 172.21.15.28:0/664764939 conn(0x7f98b80f7b50 msgr2=0x7f98b80f91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:46:28.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.245+0000 7f98be6fd640 1 -- 172.21.15.28:0/664764939 shutdown_connections 2024-07-03T15:46:28.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.245+0000 7f98be6fd640 1 -- 172.21.15.28:0/664764939 wait complete. 2024-07-03T15:46:28.259 INFO:teuthology.orchestra.run.smithi028.stdout:true 2024-07-03T15:46:28.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.330+0000 7fcd779d1640 1 -- 172.21.15.28:0/886483544 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd700fc560 msgr2=0x7fcd700fc9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:46:28.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.330+0000 7fcd779d1640 1 --2- 172.21.15.28:0/886483544 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd700fc560 0x7fcd700fc9d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcd640048d0 tx=0x7fcd64009da0 comp rx=0 tx=0).stop 2024-07-03T15:46:28.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.330+0000 7fcd779d1640 1 -- 172.21.15.28:0/886483544 shutdown_connections 2024-07-03T15:46:28.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.330+0000 7fcd779d1640 1 --2- 172.21.15.28:0/886483544 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd700fc560 0x7fcd700fc9d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:46:28.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.330+0000 7fcd779d1640 1 --2- 172.21.15.28:0/886483544 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd700fb290 0x7fcd700fb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:46:28.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.330+0000 7fcd779d1640 1 -- 172.21.15.28:0/886483544 >> 172.21.15.28:0/886483544 conn(0x7fcd700f6a50 msgr2=0x7fcd700f8e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:46:28.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.330+0000 7fcd779d1640 1 -- 172.21.15.28:0/886483544 shutdown_connections 2024-07-03T15:46:28.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.330+0000 7fcd779d1640 1 -- 172.21.15.28:0/886483544 wait complete. 2024-07-03T15:46:28.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.331+0000 7fcd779d1640 1 Processor -- start 2024-07-03T15:46:28.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.331+0000 7fcd779d1640 1 -- start start 2024-07-03T15:46:28.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.331+0000 7fcd779d1640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd700fb290 0x7fcd70066430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:46:28.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.331+0000 7fcd779d1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd700fc560 0x7fcd70111d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:46:28.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.331+0000 7fcd779d1640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd70112380 con 0x7fcd700fb290 2024-07-03T15:46:28.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.331+0000 7fcd779d1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd701124c0 con 0x7fcd700fc560 2024-07-03T15:46:28.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.331+0000 7fcd7576d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd700fb290 0x7fcd70066430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:46:28.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.331+0000 7fcd7576d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd700fb290 0x7fcd70066430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49664/0 (socket says 172.21.15.28:49664) 2024-07-03T15:46:28.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.331+0000 7fcd7576d640 1 -- 172.21.15.28:0/40498329 learned_addr learned my addr 172.21.15.28:0/40498329 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:46:28.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.331+0000 7fcd74f6c640 1 --2- 172.21.15.28:0/40498329 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd700fc560 0x7fcd70111d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:46:28.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.331+0000 7fcd7576d640 1 -- 172.21.15.28:0/40498329 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd700fc560 msgr2=0x7fcd70111d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:46:28.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.331+0000 7fcd7576d640 1 --2- 172.21.15.28:0/40498329 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd700fc560 0x7fcd70111d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:46:28.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.331+0000 7fcd7576d640 1 -- 172.21.15.28:0/40498329 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd64005040 con 0x7fcd700fb290 2024-07-03T15:46:28.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.332+0000 7fcd7576d640 1 --2- 172.21.15.28:0/40498329 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd700fb290 0x7fcd70066430 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00c830 tx=0x7fcd6c004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:46:28.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.332+0000 7fcd61ffb640 1 -- 172.21.15.28:0/40498329 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd6c00a850 con 0x7fcd700fb290 2024-07-03T15:46:28.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.332+0000 7fcd61ffb640 1 -- 172.21.15.28:0/40498329 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcd6c00ae80 con 0x7fcd700fb290 2024-07-03T15:46:28.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.332+0000 7fcd779d1640 1 -- 172.21.15.28:0/40498329 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd701127a0 con 0x7fcd700fb290 2024-07-03T15:46:28.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.332+0000 7fcd61ffb640 1 -- 172.21.15.28:0/40498329 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd6c012ea0 con 0x7fcd700fb290 2024-07-03T15:46:28.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.332+0000 7fcd779d1640 1 -- 172.21.15.28:0/40498329 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd70112d40 con 0x7fcd700fb290 2024-07-03T15:46:28.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.333+0000 7fcd779d1640 1 -- 172.21.15.28:0/40498329 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd700fa6f0 con 0x7fcd700fb290 2024-07-03T15:46:28.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.333+0000 7fcd61ffb640 1 -- 172.21.15.28:0/40498329 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 36) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fcd6c012400 con 0x7fcd700fb290 2024-07-03T15:46:28.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.334+0000 7fcd61ffb640 1 --2- 172.21.15.28:0/40498329 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fcd5c06e1e0 0x7fcd5c070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:46:28.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.334+0000 7fcd61ffb640 1 -- 172.21.15.28:0/40498329 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fcd6c08df10 con 0x7fcd700fb290 2024-07-03T15:46:28.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.334+0000 7fcd74f6c640 1 --2- 172.21.15.28:0/40498329 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fcd5c06e1e0 0x7fcd5c070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:46:28.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.334+0000 7fcd74f6c640 1 --2- 172.21.15.28:0/40498329 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fcd5c06e1e0 0x7fcd5c070690 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcd6402a530 tx=0x7fcd64009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:46:28.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.339+0000 7fcd61ffb640 1 -- 172.21.15.28:0/40498329 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fcd6c057200 con 0x7fcd700fb290 2024-07-03T15:46:28.393 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:28 smithi148 ceph-mon[36056]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:28.393 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:28 smithi148 ceph-mon[36056]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:28.393 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:28 smithi148 ceph-mon[36056]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:28.393 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:28 smithi148 ceph-mon[36056]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:28.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.538+0000 7fcd779d1640 1 -- 172.21.15.28:0/40498329 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcd701008c0 con 0x7fcd5c06e1e0 2024-07-03T15:46:28.544 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:28 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:28.544 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:28 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:28.544 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:28 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:28.544 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:28 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:28.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.540+0000 7fcd61ffb640 1 -- 172.21.15.28:0/40498329 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7fcd701008c0 con 0x7fcd5c06e1e0 2024-07-03T15:46:28.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.542+0000 7fcd779d1640 1 -- 172.21.15.28:0/40498329 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fcd5c06e1e0 msgr2=0x7fcd5c070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:46:28.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.542+0000 7fcd779d1640 1 --2- 172.21.15.28:0/40498329 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fcd5c06e1e0 0x7fcd5c070690 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcd6402a530 tx=0x7fcd64009da0 comp rx=0 tx=0).stop 2024-07-03T15:46:28.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.542+0000 7fcd779d1640 1 -- 172.21.15.28:0/40498329 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd700fb290 msgr2=0x7fcd70066430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:46:28.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.542+0000 7fcd779d1640 1 --2- 172.21.15.28:0/40498329 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd700fb290 0x7fcd70066430 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00c830 tx=0x7fcd6c004430 comp rx=0 tx=0).stop 2024-07-03T15:46:28.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.542+0000 7fcd779d1640 1 -- 172.21.15.28:0/40498329 shutdown_connections 2024-07-03T15:46:28.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.542+0000 7fcd779d1640 1 --2- 172.21.15.28:0/40498329 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fcd5c06e1e0 0x7fcd5c070690 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:46:28.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.542+0000 7fcd779d1640 1 --2- 172.21.15.28:0/40498329 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd700fc560 0x7fcd70111d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:46:28.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.542+0000 7fcd779d1640 1 --2- 172.21.15.28:0/40498329 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd700fb290 0x7fcd70066430 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:46:28.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.542+0000 7fcd779d1640 1 -- 172.21.15.28:0/40498329 >> 172.21.15.28:0/40498329 conn(0x7fcd700f6a50 msgr2=0x7fcd700f8100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:46:28.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.542+0000 7fcd779d1640 1 -- 172.21.15.28:0/40498329 shutdown_connections 2024-07-03T15:46:28.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.542+0000 7fcd779d1640 1 -- 172.21.15.28:0/40498329 wait complete. 2024-07-03T15:46:28.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.624+0000 7f4fcf186640 1 -- 172.21.15.28:0/2665828601 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4fc80fc510 msgr2=0x7f4fc80fc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:46:28.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.624+0000 7f4fcf186640 1 --2- 172.21.15.28:0/2665828601 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4fc80fc510 0x7f4fc80fc980 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4fb40048d0 tx=0x7f4fb4009da0 comp rx=0 tx=0).stop 2024-07-03T15:46:28.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.625+0000 7f4fcf186640 1 -- 172.21.15.28:0/2665828601 shutdown_connections 2024-07-03T15:46:28.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.625+0000 7f4fcf186640 1 --2- 172.21.15.28:0/2665828601 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4fc80fc510 0x7f4fc80fc980 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:46:28.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.625+0000 7f4fcf186640 1 --2- 172.21.15.28:0/2665828601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fc80fb240 0x7f4fc80fb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:46:28.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.625+0000 7f4fcf186640 1 -- 172.21.15.28:0/2665828601 >> 172.21.15.28:0/2665828601 conn(0x7f4fc80f6a20 msgr2=0x7f4fc80f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:46:28.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.625+0000 7f4fcf186640 1 -- 172.21.15.28:0/2665828601 shutdown_connections 2024-07-03T15:46:28.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.625+0000 7f4fcf186640 1 -- 172.21.15.28:0/2665828601 wait complete. 2024-07-03T15:46:28.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.625+0000 7f4fcf186640 1 Processor -- start 2024-07-03T15:46:28.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.625+0000 7f4fcf186640 1 -- start start 2024-07-03T15:46:28.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.626+0000 7f4fcf186640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fc80fb240 0x7f4fc810cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:46:28.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.626+0000 7f4fcf186640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4fc80fc510 0x7f4fc810d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:46:28.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.626+0000 7f4fcf186640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc810db70 con 0x7f4fc80fc510 2024-07-03T15:46:28.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.626+0000 7f4fcf186640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc810e340 con 0x7f4fc80fb240 2024-07-03T15:46:28.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.626+0000 7f4fc7fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4fc80fc510 0x7f4fc810d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:46:28.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.626+0000 7f4fc7fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4fc80fc510 0x7f4fc810d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49684/0 (socket says 172.21.15.28:49684) 2024-07-03T15:46:28.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.626+0000 7f4fc7fff640 1 -- 172.21.15.28:0/1377095035 learned_addr learned my addr 172.21.15.28:0/1377095035 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:46:28.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.626+0000 7f4fccf22640 1 --2- 172.21.15.28:0/1377095035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fc80fb240 0x7f4fc810cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:46:28.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.626+0000 7f4fc7fff640 1 -- 172.21.15.28:0/1377095035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fc80fb240 msgr2=0x7f4fc810cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:46:28.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.626+0000 7f4fc7fff640 1 --2- 172.21.15.28:0/1377095035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fc80fb240 0x7f4fc810cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:46:28.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.626+0000 7f4fc7fff640 1 -- 172.21.15.28:0/1377095035 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb4005040 con 0x7f4fc80fc510 2024-07-03T15:46:28.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.626+0000 7f4fc7fff640 1 --2- 172.21.15.28:0/1377095035 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4fc80fc510 0x7f4fc810d520 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4009f80 tx=0x7f4fb4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:46:28.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.626+0000 7f4fc57fa640 1 -- 172.21.15.28:0/1377095035 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fb4038070 con 0x7f4fc80fc510 2024-07-03T15:46:28.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.626+0000 7f4fc57fa640 1 -- 172.21.15.28:0/1377095035 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4fb4032970 con 0x7f4fc80fc510 2024-07-03T15:46:28.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.626+0000 7f4fcf186640 1 -- 172.21.15.28:0/1377095035 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc810e570 con 0x7f4fc80fc510 2024-07-03T15:46:28.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.627+0000 7f4fc57fa640 1 -- 172.21.15.28:0/1377095035 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fb4033860 con 0x7f4fc80fc510 2024-07-03T15:46:28.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.627+0000 7f4fcf186640 1 -- 172.21.15.28:0/1377095035 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc810eab0 con 0x7f4fc80fc510 2024-07-03T15:46:28.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.628+0000 7f4fcf186640 1 -- 172.21.15.28:0/1377095035 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fc80fa6a0 con 0x7f4fc80fc510 2024-07-03T15:46:28.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.628+0000 7f4fc57fa640 1 -- 172.21.15.28:0/1377095035 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 36) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f4fb4030020 con 0x7f4fc80fc510 2024-07-03T15:46:28.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.629+0000 7f4fc57fa640 1 --2- 172.21.15.28:0/1377095035 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f4fb806e1e0 0x7f4fb8070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:46:28.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.629+0000 7f4fc57fa640 1 -- 172.21.15.28:0/1377095035 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f4fb40ada70 con 0x7f4fc80fc510 2024-07-03T15:46:28.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.629+0000 7f4fccf22640 1 --2- 172.21.15.28:0/1377095035 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f4fb806e1e0 0x7f4fb8070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:46:28.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.630+0000 7f4fccf22640 1 --2- 172.21.15.28:0/1377095035 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f4fb806e1e0 0x7f4fb8070690 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc004ed0 tx=0x7f4fbc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:46:28.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.634+0000 7f4fc57fa640 1 -- 172.21.15.28:0/1377095035 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f4fb4076e10 con 0x7f4fc80fc510 2024-07-03T15:46:28.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.832+0000 7f4fcf186640 1 -- 172.21.15.28:0/1377095035 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4fc8103f20 con 0x7f4fb806e1e0 2024-07-03T15:46:28.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.843+0000 7f4fc57fa640 1 -- 172.21.15.28:0/1377095035 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f4fc8103f20 con 0x7f4fb806e1e0 2024-07-03T15:46:28.847 INFO:teuthology.orchestra.run.smithi028.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-03T15:46:28.848 INFO:teuthology.orchestra.run.smithi028.stdout:alertmanager.smithi028 smithi028 *:9093,9094 running (115s) 1s ago 10m 25.7M - 0.25.0 c8568f914cd2 1425315a0de9 2024-07-03T15:46:28.848 INFO:teuthology.orchestra.run.smithi028.stdout:crash.smithi028 smithi028 running (10m) 1s ago 10m 6899k - 17.2.7-1047-gd4b063ab 4ca399d9e65d 535660e448f9 2024-07-03T15:46:28.848 INFO:teuthology.orchestra.run.smithi028.stdout:crash.smithi148 smithi148 running (9m) 3s ago 9m 6899k - 17.2.7-1047-gd4b063ab 4ca399d9e65d 4bb7675895e7 2024-07-03T15:46:28.848 INFO:teuthology.orchestra.run.smithi028.stdout:grafana.smithi028 smithi028 *:3000 running (110s) 1s ago 9m 73.4M - 9.4.7 954c08fa6188 f9f17c92cfea 2024-07-03T15:46:28.848 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi028.rhpvve smithi028 running (4m) 1s ago 4m 152M - 17.2.7-1047-gd4b063ab 4ca399d9e65d 3ed5abd02cfb 2024-07-03T15:46:28.848 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi028.wmbxew smithi028 running (4m) 1s ago 4m 110M - 17.2.7-1047-gd4b063ab 4ca399d9e65d b3bf736998f9 2024-07-03T15:46:28.848 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi148.szomfl smithi148 running (4m) 3s ago 4m 15.2M - 17.2.7-1047-gd4b063ab 4ca399d9e65d d94f5c95fa8d 2024-07-03T15:46:28.848 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi148.xgpuld smithi148 running (4m) 3s ago 4m 12.9M - 17.2.7-1047-gd4b063ab 4ca399d9e65d fc70360a8de0 2024-07-03T15:46:28.848 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.smithi028.vtqrtz smithi028 *:8443,9283,8765 running (79s) 1s ago 11m 471M - 19.0.0-2550-gc00506dc 9d49405603e8 14d0363c5b77 2024-07-03T15:46:28.848 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.smithi148.rjzsar smithi148 *:8443,9283,8765 running (25s) 3s ago 9m 436M - 19.0.0-2550-gc00506dc 9d49405603e8 c0193aa8a5dd 2024-07-03T15:46:28.848 INFO:teuthology.orchestra.run.smithi028.stdout:mon.smithi028 smithi028 running (13s) 1s ago 11m 28.7M 2048M 19.0.0-2550-gc00506dc 9d49405603e8 5093f7bf47d7 2024-07-03T15:46:28.848 INFO:teuthology.orchestra.run.smithi028.stdout:mon.smithi148 smithi148 running (9m) 3s ago 9m 45.2M 2048M 17.2.7-1047-gd4b063ab 4ca399d9e65d 4c6c8ed2ab85 2024-07-03T15:46:28.849 INFO:teuthology.orchestra.run.smithi028.stdout:node-exporter.smithi028 smithi028 *:9100 running (2m) 1s ago 10m 15.3M - 1.5.0 0da6a335fe13 91ca1f49de0e 2024-07-03T15:46:28.849 INFO:teuthology.orchestra.run.smithi028.stdout:node-exporter.smithi148 smithi148 *:9100 running (93s) 3s ago 9m 16.2M - 1.5.0 0da6a335fe13 1eaaf7b1b561 2024-07-03T15:46:28.849 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0 smithi028 running (8m) 1s ago 8m 255M 1558M 17.2.7-1047-gd4b063ab 4ca399d9e65d b18089a439ba 2024-07-03T15:46:28.850 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1 smithi028 running (7m) 1s ago 7m 254M 1558M 17.2.7-1047-gd4b063ab 4ca399d9e65d 87d415dde231 2024-07-03T15:46:28.850 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2 smithi028 running (7m) 1s ago 7m 218M 1558M 17.2.7-1047-gd4b063ab 4ca399d9e65d b2cbb312212d 2024-07-03T15:46:28.850 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3 smithi148 running (7m) 3s ago 7m 277M 2581M 17.2.7-1047-gd4b063ab 4ca399d9e65d 5be69a27df31 2024-07-03T15:46:28.850 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4 smithi148 running (6m) 3s ago 6m 249M 2581M 17.2.7-1047-gd4b063ab 4ca399d9e65d 54f7c26c77ca 2024-07-03T15:46:28.850 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5 smithi148 running (6m) 3s ago 6m 238M 2581M 17.2.7-1047-gd4b063ab 4ca399d9e65d d185c6905384 2024-07-03T15:46:28.850 INFO:teuthology.orchestra.run.smithi028.stdout:prometheus.smithi028 smithi028 *:9095 running (30s) 1s ago 9m 46.8M - 2.43.0 a07b618ecd1d 2b0eb2a95afd 2024-07-03T15:46:28.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.845+0000 7f4fcf186640 1 -- 172.21.15.28:0/1377095035 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f4fb806e1e0 msgr2=0x7f4fb8070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:46:28.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.845+0000 7f4fcf186640 1 --2- 172.21.15.28:0/1377095035 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f4fb806e1e0 0x7f4fb8070690 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc004ed0 tx=0x7f4fbc00b040 comp rx=0 tx=0).stop 2024-07-03T15:46:28.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.845+0000 7f4fcf186640 1 -- 172.21.15.28:0/1377095035 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4fc80fc510 msgr2=0x7f4fc810d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:46:28.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.845+0000 7f4fcf186640 1 --2- 172.21.15.28:0/1377095035 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4fc80fc510 0x7f4fc810d520 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4009f80 tx=0x7f4fb4009da0 comp rx=0 tx=0).stop 2024-07-03T15:46:28.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.845+0000 7f4fcf186640 1 -- 172.21.15.28:0/1377095035 shutdown_connections 2024-07-03T15:46:28.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.845+0000 7f4fcf186640 1 --2- 172.21.15.28:0/1377095035 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f4fb806e1e0 0x7f4fb8070690 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:46:28.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.845+0000 7f4fcf186640 1 --2- 172.21.15.28:0/1377095035 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4fc80fc510 0x7f4fc810d520 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:46:28.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.845+0000 7f4fcf186640 1 --2- 172.21.15.28:0/1377095035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fc80fb240 0x7f4fc810cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:46:28.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.845+0000 7f4fcf186640 1 -- 172.21.15.28:0/1377095035 >> 172.21.15.28:0/1377095035 conn(0x7f4fc80f6a20 msgr2=0x7f4fc8065380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:46:28.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.845+0000 7f4fcf186640 1 -- 172.21.15.28:0/1377095035 shutdown_connections 2024-07-03T15:46:28.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.846+0000 7f4fcf186640 1 -- 172.21.15.28:0/1377095035 wait complete. 2024-07-03T15:46:28.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.930+0000 7fa255c0c640 1 -- 172.21.15.28:0/2457277733 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2500ff490 msgr2=0x7fa250101870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:46:28.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.930+0000 7fa255c0c640 1 --2- 172.21.15.28:0/2457277733 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2500ff490 0x7fa250101870 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa2380048d0 tx=0x7fa238009da0 comp rx=0 tx=0).stop 2024-07-03T15:46:28.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.930+0000 7fa255c0c640 1 -- 172.21.15.28:0/2457277733 shutdown_connections 2024-07-03T15:46:28.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.930+0000 7fa255c0c640 1 --2- 172.21.15.28:0/2457277733 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2500ff490 0x7fa250101870 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:46:28.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.930+0000 7fa255c0c640 1 --2- 172.21.15.28:0/2457277733 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa250065a50 0x7fa2500fef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:46:28.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.930+0000 7fa255c0c640 1 -- 172.21.15.28:0/2457277733 >> 172.21.15.28:0/2457277733 conn(0x7fa2500f6a20 msgr2=0x7fa2500f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:46:28.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.930+0000 7fa255c0c640 1 -- 172.21.15.28:0/2457277733 shutdown_connections 2024-07-03T15:46:28.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.930+0000 7fa255c0c640 1 -- 172.21.15.28:0/2457277733 wait complete. 2024-07-03T15:46:28.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.931+0000 7fa255c0c640 1 Processor -- start 2024-07-03T15:46:28.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.931+0000 7fa255c0c640 1 -- start start 2024-07-03T15:46:28.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.931+0000 7fa255c0c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa250065a50 0x7fa25010cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:46:28.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.931+0000 7fa255c0c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2500ff490 0x7fa25010d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:46:28.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.931+0000 7fa255c0c640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25010db20 con 0x7fa2500ff490 2024-07-03T15:46:28.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.931+0000 7fa255c0c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25010e2f0 con 0x7fa250065a50 2024-07-03T15:46:28.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.931+0000 7fa247fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2500ff490 0x7fa25010d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:46:28.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.931+0000 7fa247fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2500ff490 0x7fa25010d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49700/0 (socket says 172.21.15.28:49700) 2024-07-03T15:46:28.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.931+0000 7fa24f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa250065a50 0x7fa25010cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:46:28.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.931+0000 7fa247fff640 1 -- 172.21.15.28:0/4078779833 learned_addr learned my addr 172.21.15.28:0/4078779833 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:46:28.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.932+0000 7fa247fff640 1 -- 172.21.15.28:0/4078779833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa250065a50 msgr2=0x7fa25010cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:46:28.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.932+0000 7fa247fff640 1 --2- 172.21.15.28:0/4078779833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa250065a50 0x7fa25010cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:46:28.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.932+0000 7fa247fff640 1 -- 172.21.15.28:0/4078779833 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa238005040 con 0x7fa2500ff490 2024-07-03T15:46:28.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.932+0000 7fa24f7fe640 1 --2- 172.21.15.28:0/4078779833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa250065a50 0x7fa25010cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-03T15:46:28.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.932+0000 7fa247fff640 1 --2- 172.21.15.28:0/4078779833 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2500ff490 0x7fa25010d4d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa2380048a0 tx=0x7fa238009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:46:28.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.932+0000 7fa24cff9640 1 -- 172.21.15.28:0/4078779833 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa238038070 con 0x7fa2500ff490 2024-07-03T15:46:28.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.932+0000 7fa24cff9640 1 -- 172.21.15.28:0/4078779833 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa2380092b0 con 0x7fa2500ff490 2024-07-03T15:46:28.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.932+0000 7fa255c0c640 1 -- 172.21.15.28:0/4078779833 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa25010e520 con 0x7fa2500ff490 2024-07-03T15:46:28.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.932+0000 7fa24cff9640 1 -- 172.21.15.28:0/4078779833 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa238033780 con 0x7fa2500ff490 2024-07-03T15:46:28.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.932+0000 7fa255c0c640 1 -- 172.21.15.28:0/4078779833 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa25010ea60 con 0x7fa2500ff490 2024-07-03T15:46:28.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.934+0000 7fa24cff9640 1 -- 172.21.15.28:0/4078779833 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 36) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fa238030020 con 0x7fa2500ff490 2024-07-03T15:46:28.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.934+0000 7fa255c0c640 1 -- 172.21.15.28:0/4078779833 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2500fa6a0 con 0x7fa2500ff490 2024-07-03T15:46:28.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.935+0000 7fa24cff9640 1 --2- 172.21.15.28:0/4078779833 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fa23c06e1e0 0x7fa23c070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:46:28.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.935+0000 7fa24cff9640 1 -- 172.21.15.28:0/4078779833 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fa2380adad0 con 0x7fa2500ff490 2024-07-03T15:46:28.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.935+0000 7fa24f7fe640 1 --2- 172.21.15.28:0/4078779833 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fa23c06e1e0 0x7fa23c070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:46:28.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.936+0000 7fa24f7fe640 1 --2- 172.21.15.28:0/4078779833 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fa23c06e1e0 0x7fa23c070690 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa240004fd0 tx=0x7fa24000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:46:28.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:28.939+0000 7fa24cff9640 1 -- 172.21.15.28:0/4078779833 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fa238076d90 con 0x7fa2500ff490 2024-07-03T15:46:29.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.202+0000 7fa255c0c640 1 -- 172.21.15.28:0/4078779833 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa25010ec80 con 0x7fa2500ff490 2024-07-03T15:46:29.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.202+0000 7fa24cff9640 1 -- 172.21.15.28:0/4078779833 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+844 (secure 0 0 0) 0x7fa23807a450 con 0x7fa2500ff490 2024-07-03T15:46:29.206 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-07-03T15:46:29.207 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": { 2024-07-03T15:46:29.207 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 1, 2024-07-03T15:46:29.207 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 1 2024-07-03T15:46:29.207 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:46:29.207 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": { 2024-07-03T15:46:29.207 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 2 2024-07-03T15:46:29.207 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:46:29.207 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": { 2024-07-03T15:46:29.207 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 6 2024-07-03T15:46:29.207 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:46:29.207 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": { 2024-07-03T15:46:29.207 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 4 2024-07-03T15:46:29.208 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:46:29.208 INFO:teuthology.orchestra.run.smithi028.stdout: "overall": { 2024-07-03T15:46:29.208 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 11, 2024-07-03T15:46:29.208 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 3 2024-07-03T15:46:29.208 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-07-03T15:46:29.208 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-07-03T15:46:29.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.205+0000 7fa255c0c640 1 -- 172.21.15.28:0/4078779833 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fa23c06e1e0 msgr2=0x7fa23c070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:46:29.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.205+0000 7fa255c0c640 1 --2- 172.21.15.28:0/4078779833 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fa23c06e1e0 0x7fa23c070690 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa240004fd0 tx=0x7fa24000b040 comp rx=0 tx=0).stop 2024-07-03T15:46:29.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.205+0000 7fa255c0c640 1 -- 172.21.15.28:0/4078779833 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2500ff490 msgr2=0x7fa25010d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:46:29.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.205+0000 7fa255c0c640 1 --2- 172.21.15.28:0/4078779833 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2500ff490 0x7fa25010d4d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa2380048a0 tx=0x7fa238009da0 comp rx=0 tx=0).stop 2024-07-03T15:46:29.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.205+0000 7fa255c0c640 1 -- 172.21.15.28:0/4078779833 shutdown_connections 2024-07-03T15:46:29.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.205+0000 7fa255c0c640 1 --2- 172.21.15.28:0/4078779833 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fa23c06e1e0 0x7fa23c070690 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:46:29.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.205+0000 7fa255c0c640 1 --2- 172.21.15.28:0/4078779833 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2500ff490 0x7fa25010d4d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:46:29.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.205+0000 7fa255c0c640 1 --2- 172.21.15.28:0/4078779833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa250065a50 0x7fa25010cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:46:29.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.205+0000 7fa255c0c640 1 -- 172.21.15.28:0/4078779833 >> 172.21.15.28:0/4078779833 conn(0x7fa2500f6a20 msgr2=0x7fa2500f6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:46:29.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.205+0000 7fa255c0c640 1 -- 172.21.15.28:0/4078779833 shutdown_connections 2024-07-03T15:46:29.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.205+0000 7fa255c0c640 1 -- 172.21.15.28:0/4078779833 wait complete. 2024-07-03T15:46:29.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.291+0000 7f35e4c62640 1 -- 172.21.15.28:0/1201250492 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35e00fb230 msgr2=0x7f35e00fb640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:46:29.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.291+0000 7f35e4c62640 1 --2- 172.21.15.28:0/1201250492 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35e00fb230 0x7f35e00fb640 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f35d00048d0 tx=0x7f35d0009da0 comp rx=0 tx=0).stop 2024-07-03T15:46:29.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.291+0000 7f35e4c62640 1 -- 172.21.15.28:0/1201250492 shutdown_connections 2024-07-03T15:46:29.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.291+0000 7f35e4c62640 1 --2- 172.21.15.28:0/1201250492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35e00fc500 0x7f35e00fc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:46:29.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.291+0000 7f35e4c62640 1 --2- 172.21.15.28:0/1201250492 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35e00fb230 0x7f35e00fb640 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:46:29.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.292+0000 7f35e4c62640 1 -- 172.21.15.28:0/1201250492 >> 172.21.15.28:0/1201250492 conn(0x7f35e00f6a10 msgr2=0x7f35e00f8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:46:29.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.292+0000 7f35e4c62640 1 -- 172.21.15.28:0/1201250492 shutdown_connections 2024-07-03T15:46:29.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.292+0000 7f35e4c62640 1 -- 172.21.15.28:0/1201250492 wait complete. 2024-07-03T15:46:29.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.292+0000 7f35e4c62640 1 Processor -- start 2024-07-03T15:46:29.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.292+0000 7f35e4c62640 1 -- start start 2024-07-03T15:46:29.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.292+0000 7f35e4c62640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35e00fb230 0x7f35e00ff300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:46:29.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.292+0000 7f35e4c62640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35e00fc500 0x7f35e00fd840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:46:29.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.292+0000 7f35e4c62640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e00fdd80 con 0x7f35e00fc500 2024-07-03T15:46:29.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.292+0000 7f35e4c62640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e00fdec0 con 0x7f35e00fb230 2024-07-03T15:46:29.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.292+0000 7f35ddd9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35e00fc500 0x7f35e00fd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:46:29.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.293+0000 7f35de59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35e00fb230 0x7f35e00ff300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:46:29.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.293+0000 7f35ddd9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35e00fc500 0x7f35e00fd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49730/0 (socket says 172.21.15.28:49730) 2024-07-03T15:46:29.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.293+0000 7f35ddd9b640 1 -- 172.21.15.28:0/1145449614 learned_addr learned my addr 172.21.15.28:0/1145449614 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:46:29.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.293+0000 7f35de59c640 1 -- 172.21.15.28:0/1145449614 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35e00fc500 msgr2=0x7f35e00fd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:46:29.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.293+0000 7f35de59c640 1 --2- 172.21.15.28:0/1145449614 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35e00fc500 0x7f35e00fd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:46:29.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.293+0000 7f35de59c640 1 -- 172.21.15.28:0/1145449614 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d0005040 con 0x7f35e00fb230 2024-07-03T15:46:29.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.293+0000 7f35ddd9b640 1 --2- 172.21.15.28:0/1145449614 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35e00fc500 0x7f35e00fd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-03T15:46:29.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.293+0000 7f35de59c640 1 --2- 172.21.15.28:0/1145449614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35e00fb230 0x7f35e00ff300 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f35d00048a0 tx=0x7f35d0009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:46:29.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.293+0000 7f35ceffd640 1 -- 172.21.15.28:0/1145449614 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35d0038070 con 0x7f35e00fb230 2024-07-03T15:46:29.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.293+0000 7f35e4c62640 1 -- 172.21.15.28:0/1145449614 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35e00fe140 con 0x7f35e00fb230 2024-07-03T15:46:29.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.293+0000 7f35ceffd640 1 -- 172.21.15.28:0/1145449614 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f35d0032970 con 0x7f35e00fb230 2024-07-03T15:46:29.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.293+0000 7f35ceffd640 1 -- 172.21.15.28:0/1145449614 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35d0033780 con 0x7f35e00fb230 2024-07-03T15:46:29.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.293+0000 7f35e4c62640 1 -- 172.21.15.28:0/1145449614 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35e00fe650 con 0x7f35e00fb230 2024-07-03T15:46:29.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.295+0000 7f35e4c62640 1 -- 172.21.15.28:0/1145449614 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35e00fa690 con 0x7f35e00fb230 2024-07-03T15:46:29.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.295+0000 7f35ceffd640 1 -- 172.21.15.28:0/1145449614 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 36) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f35d0030020 con 0x7f35e00fb230 2024-07-03T15:46:29.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.296+0000 7f35ceffd640 1 --2- 172.21.15.28:0/1145449614 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f35c806e1e0 0x7f35c8070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:46:29.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.296+0000 7f35ceffd640 1 -- 172.21.15.28:0/1145449614 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f35d00ada20 con 0x7f35e00fb230 2024-07-03T15:46:29.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.296+0000 7f35ddd9b640 1 --2- 172.21.15.28:0/1145449614 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f35c806e1e0 0x7f35c8070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:46:29.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.296+0000 7f35ddd9b640 1 --2- 172.21.15.28:0/1145449614 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f35c806e1e0 0x7f35c8070690 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f35d4004fd0 tx=0x7f35d400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:46:29.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.300+0000 7f35ceffd640 1 -- 172.21.15.28:0/1145449614 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f35d0076dc0 con 0x7f35e00fb230 2024-07-03T15:46:29.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.535+0000 7f35e4c62640 1 -- 172.21.15.28:0/1145449614 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f35e0100fb0 con 0x7f35e00fb230 2024-07-03T15:46:29.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.536+0000 7f35ceffd640 1 -- 172.21.15.28:0/1145449614 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1838 (secure 0 0 0) 0x7f35d007a480 con 0x7f35e00fb230 2024-07-03T15:46:29.540 INFO:teuthology.orchestra.run.smithi028.stdout:e17 2024-07-03T15:46:29.540 INFO:teuthology.orchestra.run.smithi028.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-03T15:46:29.540 INFO:teuthology.orchestra.run.smithi028.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:46:29.540 INFO:teuthology.orchestra.run.smithi028.stdout:legacy client fscid: 1 2024-07-03T15:46:29.540 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:46:29.540 INFO:teuthology.orchestra.run.smithi028.stdout:Filesystem 'cephfs' (1) 2024-07-03T15:46:29.540 INFO:teuthology.orchestra.run.smithi028.stdout:fs_name cephfs 2024-07-03T15:46:29.540 INFO:teuthology.orchestra.run.smithi028.stdout:epoch 16 2024-07-03T15:46:29.541 INFO:teuthology.orchestra.run.smithi028.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-03T15:46:29.541 INFO:teuthology.orchestra.run.smithi028.stdout:created 2024-07-03T15:41:40.656298+0000 2024-07-03T15:46:29.541 INFO:teuthology.orchestra.run.smithi028.stdout:modified 2024-07-03T15:41:56.454984+0000 2024-07-03T15:46:29.541 INFO:teuthology.orchestra.run.smithi028.stdout:tableserver 0 2024-07-03T15:46:29.541 INFO:teuthology.orchestra.run.smithi028.stdout:root 0 2024-07-03T15:46:29.541 INFO:teuthology.orchestra.run.smithi028.stdout:session_timeout 60 2024-07-03T15:46:29.541 INFO:teuthology.orchestra.run.smithi028.stdout:session_autoclose 300 2024-07-03T15:46:29.541 INFO:teuthology.orchestra.run.smithi028.stdout:max_file_size 1099511627776 2024-07-03T15:46:29.541 INFO:teuthology.orchestra.run.smithi028.stdout:required_client_features {} 2024-07-03T15:46:29.541 INFO:teuthology.orchestra.run.smithi028.stdout:last_failure 0 2024-07-03T15:46:29.541 INFO:teuthology.orchestra.run.smithi028.stdout:last_failure_osd_epoch 0 2024-07-03T15:46:29.541 INFO:teuthology.orchestra.run.smithi028.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:46:29.541 INFO:teuthology.orchestra.run.smithi028.stdout:max_mds 1 2024-07-03T15:46:29.541 INFO:teuthology.orchestra.run.smithi028.stdout:in 0 2024-07-03T15:46:29.541 INFO:teuthology.orchestra.run.smithi028.stdout:up {0=14450} 2024-07-03T15:46:29.542 INFO:teuthology.orchestra.run.smithi028.stdout:failed 2024-07-03T15:46:29.542 INFO:teuthology.orchestra.run.smithi028.stdout:damaged 2024-07-03T15:46:29.542 INFO:teuthology.orchestra.run.smithi028.stdout:stopped 2024-07-03T15:46:29.542 INFO:teuthology.orchestra.run.smithi028.stdout:data_pools [3] 2024-07-03T15:46:29.542 INFO:teuthology.orchestra.run.smithi028.stdout:metadata_pool 2 2024-07-03T15:46:29.542 INFO:teuthology.orchestra.run.smithi028.stdout:inline_data disabled 2024-07-03T15:46:29.542 INFO:teuthology.orchestra.run.smithi028.stdout:balancer 2024-07-03T15:46:29.542 INFO:teuthology.orchestra.run.smithi028.stdout:standby_count_wanted 1 2024-07-03T15:46:29.542 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi028.rhpvve{0:14450} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.28:6826/1613936042,v1:172.21.15.28:6827/1613936042] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:46:29.542 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi028.wmbxew{0:14460} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.28:6828/1730336821,v1:172.21.15.28:6829/1730336821] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:46:29.542 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:46:29.542 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:46:29.543 INFO:teuthology.orchestra.run.smithi028.stdout:Standby daemons: 2024-07-03T15:46:29.543 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:46:29.543 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi148.szomfl{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/4171231858,v1:172.21.15.148:6827/4171231858] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:46:29.543 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi148.xgpuld{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/2347536398,v1:172.21.15.148:6825/2347536398] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:46:29.543 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 17 2024-07-03T15:46:29.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.538+0000 7f35e4c62640 1 -- 172.21.15.28:0/1145449614 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f35c806e1e0 msgr2=0x7f35c8070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:46:29.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.538+0000 7f35e4c62640 1 --2- 172.21.15.28:0/1145449614 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f35c806e1e0 0x7f35c8070690 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f35d4004fd0 tx=0x7f35d400b040 comp rx=0 tx=0).stop 2024-07-03T15:46:29.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.538+0000 7f35e4c62640 1 -- 172.21.15.28:0/1145449614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35e00fb230 msgr2=0x7f35e00ff300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:46:29.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.538+0000 7f35e4c62640 1 --2- 172.21.15.28:0/1145449614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35e00fb230 0x7f35e00ff300 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f35d00048a0 tx=0x7f35d0009da0 comp rx=0 tx=0).stop 2024-07-03T15:46:29.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.538+0000 7f35e4c62640 1 -- 172.21.15.28:0/1145449614 shutdown_connections 2024-07-03T15:46:29.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.538+0000 7f35e4c62640 1 --2- 172.21.15.28:0/1145449614 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f35c806e1e0 0x7f35c8070690 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:46:29.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.538+0000 7f35e4c62640 1 --2- 172.21.15.28:0/1145449614 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35e00fc500 0x7f35e00fd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:46:29.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.538+0000 7f35e4c62640 1 --2- 172.21.15.28:0/1145449614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35e00fb230 0x7f35e00ff300 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:46:29.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.538+0000 7f35e4c62640 1 -- 172.21.15.28:0/1145449614 >> 172.21.15.28:0/1145449614 conn(0x7f35e00f6a10 msgr2=0x7f35e0065bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:46:29.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.538+0000 7f35e4c62640 1 -- 172.21.15.28:0/1145449614 shutdown_connections 2024-07-03T15:46:29.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.538+0000 7f35e4c62640 1 -- 172.21.15.28:0/1145449614 wait complete. 2024-07-03T15:46:29.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.629+0000 7f08a70ea640 1 -- 172.21.15.28:0/3756150351 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f08a00fc370 msgr2=0x7f08a00fc780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:46:29.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.629+0000 7f08a70ea640 1 --2- 172.21.15.28:0/3756150351 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f08a00fc370 0x7f08a00fc780 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f08900048d0 tx=0x7f0890009da0 comp rx=0 tx=0).stop 2024-07-03T15:46:29.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.629+0000 7f08a70ea640 1 -- 172.21.15.28:0/3756150351 shutdown_connections 2024-07-03T15:46:29.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.629+0000 7f08a70ea640 1 --2- 172.21.15.28:0/3756150351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08a00fd640 0x7f08a00fdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:46:29.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.629+0000 7f08a70ea640 1 --2- 172.21.15.28:0/3756150351 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f08a00fc370 0x7f08a00fc780 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:46:29.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.629+0000 7f08a70ea640 1 -- 172.21.15.28:0/3756150351 >> 172.21.15.28:0/3756150351 conn(0x7f08a00f7b30 msgr2=0x7f08a00f9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:46:29.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.629+0000 7f08a70ea640 1 -- 172.21.15.28:0/3756150351 shutdown_connections 2024-07-03T15:46:29.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.629+0000 7f08a70ea640 1 -- 172.21.15.28:0/3756150351 wait complete. 2024-07-03T15:46:29.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.630+0000 7f08a70ea640 1 Processor -- start 2024-07-03T15:46:29.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.630+0000 7f08a70ea640 1 -- start start 2024-07-03T15:46:29.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.630+0000 7f08a70ea640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f08a00fc370 0x7f08a010cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:46:29.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.630+0000 7f08a70ea640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08a00fd640 0x7f08a010d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:46:29.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.630+0000 7f08a70ea640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a010daf0 con 0x7f08a00fc370 2024-07-03T15:46:29.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.630+0000 7f08a70ea640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a010e310 con 0x7f08a00fd640 2024-07-03T15:46:29.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.630+0000 7f08a4e86640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f08a00fc370 0x7f08a010cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:46:29.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.630+0000 7f08a4e86640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f08a00fc370 0x7f08a010cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49752/0 (socket says 172.21.15.28:49752) 2024-07-03T15:46:29.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.631+0000 7f08a4e86640 1 -- 172.21.15.28:0/1214549117 learned_addr learned my addr 172.21.15.28:0/1214549117 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:46:29.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.631+0000 7f089ffff640 1 --2- 172.21.15.28:0/1214549117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08a00fd640 0x7f08a010d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:46:29.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.631+0000 7f08a4e86640 1 -- 172.21.15.28:0/1214549117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08a00fd640 msgr2=0x7f08a010d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:46:29.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.631+0000 7f08a4e86640 1 --2- 172.21.15.28:0/1214549117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08a00fd640 0x7f08a010d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:46:29.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.631+0000 7f08a4e86640 1 -- 172.21.15.28:0/1214549117 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0890005040 con 0x7f08a00fc370 2024-07-03T15:46:29.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.631+0000 7f08a4e86640 1 --2- 172.21.15.28:0/1214549117 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f08a00fc370 0x7f08a010cf60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f08900041d0 tx=0x7f0890009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:46:29.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.631+0000 7f089d7fa640 1 -- 172.21.15.28:0/1214549117 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0890038070 con 0x7f08a00fc370 2024-07-03T15:46:29.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.631+0000 7f089d7fa640 1 -- 172.21.15.28:0/1214549117 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f08900092b0 con 0x7f08a00fc370 2024-07-03T15:46:29.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.631+0000 7f089d7fa640 1 -- 172.21.15.28:0/1214549117 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0890033710 con 0x7f08a00fc370 2024-07-03T15:46:29.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.631+0000 7f08a70ea640 1 -- 172.21.15.28:0/1214549117 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08a010e540 con 0x7f08a00fc370 2024-07-03T15:46:29.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.631+0000 7f08a70ea640 1 -- 172.21.15.28:0/1214549117 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08a010ea80 con 0x7f08a00fc370 2024-07-03T15:46:29.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.632+0000 7f08a70ea640 1 -- 172.21.15.28:0/1214549117 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08a00fb7d0 con 0x7f08a00fc370 2024-07-03T15:46:29.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.633+0000 7f089d7fa640 1 -- 172.21.15.28:0/1214549117 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 36) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f0890030020 con 0x7f08a00fc370 2024-07-03T15:46:29.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.633+0000 7f089d7fa640 1 --2- 172.21.15.28:0/1214549117 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f088806e1e0 0x7f0888070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:46:29.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.634+0000 7f089d7fa640 1 -- 172.21.15.28:0/1214549117 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f08900ad960 con 0x7f08a00fc370 2024-07-03T15:46:29.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.634+0000 7f089ffff640 1 --2- 172.21.15.28:0/1214549117 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f088806e1e0 0x7f0888070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:46:29.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.634+0000 7f089ffff640 1 --2- 172.21.15.28:0/1214549117 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f088806e1e0 0x7f0888070690 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f08a010cd10 tx=0x7f089400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:46:29.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.638+0000 7f089d7fa640 1 -- 172.21.15.28:0/1214549117 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0890076d00 con 0x7f08a00fc370 2024-07-03T15:46:29.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.851+0000 7f08a70ea640 1 -- 172.21.15.28:0/1214549117 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f08a010ee20 con 0x7f088806e1e0 2024-07-03T15:46:29.855 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:29 smithi028 ceph-mon[92858]: pgmap v6: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-07-03T15:46:29.855 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:29 smithi028 ceph-mon[92858]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:46:29.855 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:29 smithi028 ceph-mon[92858]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:46:29.855 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:29 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:29.855 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:29 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:29.855 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:29 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:46:29.855 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:29 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:46:29.856 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:29 smithi028 ceph-mon[92858]: Updating smithi028:/etc/ceph/ceph.conf 2024-07-03T15:46:29.856 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:29 smithi028 ceph-mon[92858]: Updating smithi148:/etc/ceph/ceph.conf 2024-07-03T15:46:29.856 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:29 smithi028 ceph-mon[92858]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:46:29.856 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:29 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/4078779833' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:46:29.856 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:29 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/1145449614' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-03T15:46:29.856 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:29 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:29.856 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:29 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:29.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.852+0000 7f089d7fa640 1 -- 172.21.15.28:0/1214549117 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f08a010ee20 con 0x7f088806e1e0 2024-07-03T15:46:29.857 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-07-03T15:46:29.857 INFO:teuthology.orchestra.run.smithi028.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2", 2024-07-03T15:46:29.857 INFO:teuthology.orchestra.run.smithi028.stdout: "in_progress": true, 2024-07-03T15:46:29.857 INFO:teuthology.orchestra.run.smithi028.stdout: "which": "Upgrading all daemon types on all hosts", 2024-07-03T15:46:29.857 INFO:teuthology.orchestra.run.smithi028.stdout: "services_complete": [ 2024-07-03T15:46:29.857 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr" 2024-07-03T15:46:29.857 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-07-03T15:46:29.857 INFO:teuthology.orchestra.run.smithi028.stdout: "progress": "3/21 daemons upgraded", 2024-07-03T15:46:29.857 INFO:teuthology.orchestra.run.smithi028.stdout: "message": "", 2024-07-03T15:46:29.857 INFO:teuthology.orchestra.run.smithi028.stdout: "is_paused": false 2024-07-03T15:46:29.857 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-07-03T15:46:29.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.854+0000 7f08a70ea640 1 -- 172.21.15.28:0/1214549117 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f088806e1e0 msgr2=0x7f0888070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:46:29.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.854+0000 7f08a70ea640 1 --2- 172.21.15.28:0/1214549117 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f088806e1e0 0x7f0888070690 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f08a010cd10 tx=0x7f089400b040 comp rx=0 tx=0).stop 2024-07-03T15:46:29.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.854+0000 7f08a70ea640 1 -- 172.21.15.28:0/1214549117 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f08a00fc370 msgr2=0x7f08a010cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:46:29.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.854+0000 7f08a70ea640 1 --2- 172.21.15.28:0/1214549117 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f08a00fc370 0x7f08a010cf60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f08900041d0 tx=0x7f0890009da0 comp rx=0 tx=0).stop 2024-07-03T15:46:29.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.855+0000 7f08a70ea640 1 -- 172.21.15.28:0/1214549117 shutdown_connections 2024-07-03T15:46:29.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.855+0000 7f08a70ea640 1 --2- 172.21.15.28:0/1214549117 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f088806e1e0 0x7f0888070690 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:46:29.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.855+0000 7f08a70ea640 1 --2- 172.21.15.28:0/1214549117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08a00fd640 0x7f08a010d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:46:29.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.855+0000 7f08a70ea640 1 --2- 172.21.15.28:0/1214549117 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f08a00fc370 0x7f08a010cf60 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:46:29.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.855+0000 7f08a70ea640 1 -- 172.21.15.28:0/1214549117 >> 172.21.15.28:0/1214549117 conn(0x7f08a00f7b30 msgr2=0x7f08a00f91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:46:29.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.855+0000 7f08a70ea640 1 -- 172.21.15.28:0/1214549117 shutdown_connections 2024-07-03T15:46:29.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.855+0000 7f08a70ea640 1 -- 172.21.15.28:0/1214549117 wait complete. 2024-07-03T15:46:29.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.938+0000 7fab435e1640 1 -- 172.21.15.28:0/4113763080 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fab3c0fb260 msgr2=0x7fab3c0fb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:46:29.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.938+0000 7fab435e1640 1 --2- 172.21.15.28:0/4113763080 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fab3c0fb260 0x7fab3c0fb670 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fab380048d0 tx=0x7fab38009da0 comp rx=0 tx=0).stop 2024-07-03T15:46:29.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.938+0000 7fab435e1640 1 -- 172.21.15.28:0/4113763080 shutdown_connections 2024-07-03T15:46:29.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.938+0000 7fab435e1640 1 --2- 172.21.15.28:0/4113763080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab3c0fc530 0x7fab3c0fc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:46:29.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.938+0000 7fab435e1640 1 --2- 172.21.15.28:0/4113763080 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fab3c0fb260 0x7fab3c0fb670 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:46:29.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.938+0000 7fab435e1640 1 -- 172.21.15.28:0/4113763080 >> 172.21.15.28:0/4113763080 conn(0x7fab3c0f6a20 msgr2=0x7fab3c0f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:46:29.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.938+0000 7fab435e1640 1 -- 172.21.15.28:0/4113763080 shutdown_connections 2024-07-03T15:46:29.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.938+0000 7fab435e1640 1 -- 172.21.15.28:0/4113763080 wait complete. 2024-07-03T15:46:29.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.939+0000 7fab435e1640 1 Processor -- start 2024-07-03T15:46:29.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.939+0000 7fab435e1640 1 -- start start 2024-07-03T15:46:29.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.939+0000 7fab435e1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab3c0fb260 0x7fab3c0ff380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:46:29.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.939+0000 7fab435e1640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fab3c0fc530 0x7fab3c0fd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:46:29.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.939+0000 7fab435e1640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c0fde00 con 0x7fab3c0fc530 2024-07-03T15:46:29.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.939+0000 7fab435e1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c0fdf40 con 0x7fab3c0fb260 2024-07-03T15:46:29.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.939+0000 7fab40b7c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fab3c0fc530 0x7fab3c0fd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:46:29.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.939+0000 7fab40b7c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fab3c0fc530 0x7fab3c0fd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38376/0 (socket says 172.21.15.28:38376) 2024-07-03T15:46:29.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.939+0000 7fab40b7c640 1 -- 172.21.15.28:0/923779582 learned_addr learned my addr 172.21.15.28:0/923779582 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:46:29.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.939+0000 7fab4137d640 1 --2- 172.21.15.28:0/923779582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab3c0fb260 0x7fab3c0ff380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:46:29.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.939+0000 7fab40b7c640 1 -- 172.21.15.28:0/923779582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab3c0fb260 msgr2=0x7fab3c0ff380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:46:29.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.939+0000 7fab40b7c640 1 --2- 172.21.15.28:0/923779582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab3c0fb260 0x7fab3c0ff380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:46:29.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.939+0000 7fab40b7c640 1 -- 172.21.15.28:0/923779582 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab38005040 con 0x7fab3c0fc530 2024-07-03T15:46:29.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.940+0000 7fab40b7c640 1 --2- 172.21.15.28:0/923779582 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fab3c0fc530 0x7fab3c0fd870 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fab2c002a50 tx=0x7fab2c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:46:29.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.940+0000 7fab31ffb640 1 -- 172.21.15.28:0/923779582 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab2c007c30 con 0x7fab3c0fc530 2024-07-03T15:46:29.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.940+0000 7fab435e1640 1 -- 172.21.15.28:0/923779582 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab3c0fe220 con 0x7fab3c0fc530 2024-07-03T15:46:29.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.940+0000 7fab31ffb640 1 -- 172.21.15.28:0/923779582 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab2c00f040 con 0x7fab3c0fc530 2024-07-03T15:46:29.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.940+0000 7fab31ffb640 1 -- 172.21.15.28:0/923779582 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab2c00dad0 con 0x7fab3c0fc530 2024-07-03T15:46:29.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.940+0000 7fab435e1640 1 -- 172.21.15.28:0/923779582 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab3c115010 con 0x7fab3c0fc530 2024-07-03T15:46:29.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.941+0000 7fab435e1640 1 -- 172.21.15.28:0/923779582 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab3c0fa6c0 con 0x7fab3c0fc530 2024-07-03T15:46:29.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.942+0000 7fab31ffb640 1 -- 172.21.15.28:0/923779582 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 36) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fab2c00dd40 con 0x7fab3c0fc530 2024-07-03T15:46:29.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.942+0000 7fab31ffb640 1 --2- 172.21.15.28:0/923779582 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fab2806e1e0 0x7fab28070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:46:29.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.942+0000 7fab31ffb640 1 -- 172.21.15.28:0/923779582 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fab2c08d5c0 con 0x7fab3c0fc530 2024-07-03T15:46:29.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.942+0000 7fab4137d640 1 --2- 172.21.15.28:0/923779582 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fab2806e1e0 0x7fab28070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:46:29.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.943+0000 7fab4137d640 1 --2- 172.21.15.28:0/923779582 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fab2806e1e0 0x7fab28070690 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fab380043d0 tx=0x7fab38035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:46:29.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:29.946+0000 7fab31ffb640 1 -- 172.21.15.28:0/923779582 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fab2c0567d0 con 0x7fab3c0fc530 2024-07-03T15:46:30.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:30.201+0000 7fab435e1640 1 -- 172.21.15.28:0/923779582 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fab3c04b530 con 0x7fab3c0fc530 2024-07-03T15:46:30.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:30.202+0000 7fab31ffb640 1 -- 172.21.15.28:0/923779582 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fab2c059e90 con 0x7fab3c0fc530 2024-07-03T15:46:30.206 INFO:teuthology.orchestra.run.smithi028.stdout:HEALTH_OK 2024-07-03T15:46:30.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:30.204+0000 7fab435e1640 1 -- 172.21.15.28:0/923779582 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fab2806e1e0 msgr2=0x7fab28070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:46:30.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:30.204+0000 7fab435e1640 1 --2- 172.21.15.28:0/923779582 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fab2806e1e0 0x7fab28070690 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fab380043d0 tx=0x7fab38035040 comp rx=0 tx=0).stop 2024-07-03T15:46:30.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:30.204+0000 7fab435e1640 1 -- 172.21.15.28:0/923779582 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fab3c0fc530 msgr2=0x7fab3c0fd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:46:30.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:30.204+0000 7fab435e1640 1 --2- 172.21.15.28:0/923779582 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fab3c0fc530 0x7fab3c0fd870 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fab2c002a50 tx=0x7fab2c004600 comp rx=0 tx=0).stop 2024-07-03T15:46:30.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:30.204+0000 7fab435e1640 1 -- 172.21.15.28:0/923779582 shutdown_connections 2024-07-03T15:46:30.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:30.204+0000 7fab435e1640 1 --2- 172.21.15.28:0/923779582 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fab2806e1e0 0x7fab28070690 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:46:30.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:30.204+0000 7fab435e1640 1 --2- 172.21.15.28:0/923779582 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fab3c0fc530 0x7fab3c0fd870 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:46:30.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:30.204+0000 7fab435e1640 1 --2- 172.21.15.28:0/923779582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab3c0fb260 0x7fab3c0ff380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:46:30.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:30.204+0000 7fab435e1640 1 -- 172.21.15.28:0/923779582 >> 172.21.15.28:0/923779582 conn(0x7fab3c0f6a20 msgr2=0x7fab3c065e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:46:30.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:30.204+0000 7fab435e1640 1 -- 172.21.15.28:0/923779582 shutdown_connections 2024-07-03T15:46:30.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:46:30.204+0000 7fab435e1640 1 -- 172.21.15.28:0/923779582 wait complete. 2024-07-03T15:46:30.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:29 smithi148 ceph-mon[36056]: pgmap v6: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-07-03T15:46:30.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:29 smithi148 ceph-mon[36056]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:46:30.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:29 smithi148 ceph-mon[36056]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:46:30.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:29 smithi148 ceph-mon[36056]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:30.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:29 smithi148 ceph-mon[36056]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:30.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:29 smithi148 ceph-mon[36056]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:46:30.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:29 smithi148 ceph-mon[36056]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:46:30.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:29 smithi148 ceph-mon[36056]: Updating smithi028:/etc/ceph/ceph.conf 2024-07-03T15:46:30.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:29 smithi148 ceph-mon[36056]: Updating smithi148:/etc/ceph/ceph.conf 2024-07-03T15:46:30.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:29 smithi148 ceph-mon[36056]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:46:30.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:29 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/4078779833' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:46:30.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:29 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/1145449614' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-03T15:46:30.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:29 smithi148 ceph-mon[36056]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:30.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:29 smithi148 ceph-mon[36056]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:30.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:30 smithi148 ceph-mon[36056]: Updating smithi028:/var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-03T15:46:30.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:30 smithi148 ceph-mon[36056]: Updating smithi148:/var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-03T15:46:30.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:30 smithi148 ceph-mon[36056]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-07-03T15:46:30.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:30 smithi148 ceph-mon[36056]: Updating smithi028:/etc/ceph/ceph.client.admin.keyring 2024-07-03T15:46:30.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:30 smithi148 ceph-mon[36056]: Updating smithi148:/var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/config/ceph.client.admin.keyring 2024-07-03T15:46:30.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:30 smithi148 ceph-mon[36056]: Updating smithi028:/var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/config/ceph.client.admin.keyring 2024-07-03T15:46:30.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:30 smithi148 ceph-mon[36056]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:30.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:30 smithi148 ceph-mon[36056]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:30.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:30 smithi148 ceph-mon[36056]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:30.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:30 smithi148 ceph-mon[36056]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:46:30.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:30 smithi148 ceph-mon[36056]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:46:30.984 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:30 smithi148 ceph-mon[36056]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:46:30.984 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:30 smithi148 ceph-mon[36056]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-07-03T15:46:30.984 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:30 smithi148 ceph-mon[36056]: from='client.? 172.21.15.28:0/923779582' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-03T15:46:31.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:30 smithi028 ceph-mon[92858]: Updating smithi028:/var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-03T15:46:31.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:30 smithi028 ceph-mon[92858]: Updating smithi148:/var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-03T15:46:31.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:30 smithi028 ceph-mon[92858]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-07-03T15:46:31.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:30 smithi028 ceph-mon[92858]: Updating smithi028:/etc/ceph/ceph.client.admin.keyring 2024-07-03T15:46:31.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:30 smithi028 ceph-mon[92858]: Updating smithi148:/var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/config/ceph.client.admin.keyring 2024-07-03T15:46:31.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:30 smithi028 ceph-mon[92858]: Updating smithi028:/var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/config/ceph.client.admin.keyring 2024-07-03T15:46:31.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:30 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:31.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:30 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:31.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:30 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:31.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:30 smithi028 ceph-mon[92858]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:46:31.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:30 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:46:31.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:30 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:46:31.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:30 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-07-03T15:46:31.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:30 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/923779582' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-03T15:46:31.868 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:31 smithi148 ceph-mon[36056]: pgmap v7: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 0 B/s wr, 18 op/s 2024-07-03T15:46:31.868 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:31 smithi148 ceph-mon[36056]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:31.868 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:31 smithi148 ceph-mon[36056]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-03T15:46:31.868 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:31 smithi148 ceph-mon[36056]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-03T15:46:31.868 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:31 smithi148 ceph-mon[36056]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:46:32.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:31 smithi028 ceph-mon[92858]: pgmap v7: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 0 B/s wr, 18 op/s 2024-07-03T15:46:32.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:31 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:32.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:31 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-03T15:46:32.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:31 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-03T15:46:32.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:31 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:46:32.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:32 smithi148 ceph-mon[36056]: Upgrade: Updating mon.smithi148 2024-07-03T15:46:32.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:32 smithi148 ceph-mon[36056]: Deploying daemon mon.smithi148 on smithi148 2024-07-03T15:46:33.053 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:32 smithi028 ceph-mon[92858]: Upgrade: Updating mon.smithi148 2024-07-03T15:46:33.053 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:32 smithi028 ceph-mon[92858]: Deploying daemon mon.smithi148 on smithi148 2024-07-03T15:46:33.074 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:32 smithi148 systemd[1]: Stopping Ceph mon.smithi148 for a85e4be6-3951-11ef-bcab-c7b262605968... 2024-07-03T15:46:33.074 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:33 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-mon-smithi148[36006]: 2024-07-03T15:46:33.047+0000 7f51a5817640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi148 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-07-03T15:46:33.074 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:33 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-mon-smithi148[36006]: 2024-07-03T15:46:33.047+0000 7f51a5817640 -1 mon.smithi148@1(peon) e2 *** Got Signal Terminated *** 2024-07-03T15:46:33.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:33 smithi148 podman[80317]: 2024-07-03 15:46:33.262279157 +0000 UTC m=+0.313376973 container died 4c6c8ed2ab8514f8579f2aa29e5e662df817a7f4b577e99d77cf7dcf90067096 (image=quay.ceph.io/ceph-ci/ceph@sha256:0b7651540cdb24c1b97466c79c298d54c7794c5f53d5d6b6c5d76a2d3b130d85, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-mon-smithi148, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.build-date=20240625, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=quincy-d4b063a, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-03T15:46:33.811 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:33 smithi148 podman[80317]: 2024-07-03 15:46:33.60616298 +0000 UTC m=+0.657260792 container cleanup 4c6c8ed2ab8514f8579f2aa29e5e662df817a7f4b577e99d77cf7dcf90067096 (image=quay.ceph.io/ceph-ci/ceph@sha256:0b7651540cdb24c1b97466c79c298d54c7794c5f53d5d6b6c5d76a2d3b130d85, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-mon-smithi148, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=quincy-d4b063a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, org.label-schema.build-date=20240625, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-03T15:46:33.811 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:33 smithi148 bash[80317]: ceph-a85e4be6-3951-11ef-bcab-c7b262605968-mon-smithi148 2024-07-03T15:46:34.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:33 smithi148 podman[80329]: 2024-07-03 15:46:33.807625358 +0000 UTC m=+0.539978558 container remove 4c6c8ed2ab8514f8579f2aa29e5e662df817a7f4b577e99d77cf7dcf90067096 (image=quay.ceph.io/ceph-ci/ceph@sha256:0b7651540cdb24c1b97466c79c298d54c7794c5f53d5d6b6c5d76a2d3b130d85, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-mon-smithi148, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240625, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=quincy-d4b063a, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-07-03T15:46:34.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 systemd[1]: ceph-a85e4be6-3951-11ef-bcab-c7b262605968@mon.smithi148.service: Deactivated successfully. 2024-07-03T15:46:34.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 systemd[1]: Stopped Ceph mon.smithi148 for a85e4be6-3951-11ef-bcab-c7b262605968. 2024-07-03T15:46:34.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 systemd[1]: ceph-a85e4be6-3951-11ef-bcab-c7b262605968@mon.smithi148.service: Consumed 5.575s CPU time. 2024-07-03T15:46:34.503 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 systemd[1]: Starting Ceph mon.smithi148 for a85e4be6-3951-11ef-bcab-c7b262605968... 2024-07-03T15:46:34.503 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 podman[80461]: 2024-07-03 15:46:34.374498952 +0000 UTC m=+0.017859508 image pull 9d49405603e8445bc510362cdeb05f243619be76b0f3897dd4c152b00cffe45f quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2 2024-07-03T15:46:34.503 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 podman[80461]: 2024-07-03 15:46:34.500266491 +0000 UTC m=+0.143627045 container create d338d2c648dab4f59e621f5c94b28fea53c4543e6132d5f525d1b021a3a18467 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-mon-smithi148, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-release-c00506d, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240625, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0) 2024-07-03T15:46:34.755 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 podman[80461]: 2024-07-03 15:46:34.711216896 +0000 UTC m=+0.354577471 container init d338d2c648dab4f59e621f5c94b28fea53c4543e6132d5f525d1b021a3a18467 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-mon-smithi148, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-release-c00506d, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240625, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0) 2024-07-03T15:46:34.755 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 podman[80461]: 2024-07-03 15:46:34.714544231 +0000 UTC m=+0.357904794 container start d338d2c648dab4f59e621f5c94b28fea53c4543e6132d5f525d1b021a3a18467 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-mon-smithi148, org.label-schema.license=GPLv2, org.label-schema.build-date=20240625, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, RELEASE=squid-release-c00506d, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-03T15:46:34.755 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: set uid:gid to 167:167 (ceph:ceph) 2024-07-03T15:46:34.755 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc), process ceph-mon, pid 2 2024-07-03T15:46:34.755 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: pidfile_write: ignore empty --pid-file 2024-07-03T15:46:34.756 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: load: jerasure load: lrc 2024-07-03T15:46:34.756 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: RocksDB version: 7.9.2 2024-07-03T15:46:34.756 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Git sha 0 2024-07-03T15:46:34.756 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Compile date 2024-07-01 20:58:27 2024-07-03T15:46:34.756 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: DB SUMMARY 2024-07-03T15:46:34.756 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: DB Session ID: 538JZ7I890DYIMW45ZH6 2024-07-03T15:46:34.756 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: CURRENT file: CURRENT 2024-07-03T15:46:34.756 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: IDENTITY file: IDENTITY 2024-07-03T15:46:34.756 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: MANIFEST file: MANIFEST-000009 size: 897 Bytes 2024-07-03T15:46:34.756 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi148/store.db dir, Total Num: 1, files: 000024.sst 2024-07-03T15:46:34.756 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi148/store.db: 000022.log size: 5887534 ; 2024-07-03T15:46:34.756 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.error_if_exists: 0 2024-07-03T15:46:34.756 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.create_if_missing: 0 2024-07-03T15:46:34.757 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.paranoid_checks: 1 2024-07-03T15:46:34.757 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.flush_verify_memtable_count: 1 2024-07-03T15:46:34.757 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-07-03T15:46:34.757 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-07-03T15:46:34.757 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.env: 0x55e1ecf28740 2024-07-03T15:46:34.757 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.fs: PosixFileSystem 2024-07-03T15:46:34.757 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.info_log: 0x55e1eed252c0 2024-07-03T15:46:34.757 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.max_file_opening_threads: 16 2024-07-03T15:46:34.757 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.statistics: (nil) 2024-07-03T15:46:34.757 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.use_fsync: 0 2024-07-03T15:46:34.757 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.max_log_file_size: 0 2024-07-03T15:46:34.757 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-07-03T15:46:34.757 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.log_file_time_to_roll: 0 2024-07-03T15:46:34.758 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.keep_log_file_num: 1000 2024-07-03T15:46:34.758 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.recycle_log_file_num: 0 2024-07-03T15:46:34.758 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.allow_fallocate: 1 2024-07-03T15:46:34.758 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.allow_mmap_reads: 0 2024-07-03T15:46:34.758 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.allow_mmap_writes: 0 2024-07-03T15:46:34.758 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.use_direct_reads: 0 2024-07-03T15:46:34.758 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-07-03T15:46:34.758 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.create_missing_column_families: 0 2024-07-03T15:46:34.758 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.db_log_dir: 2024-07-03T15:46:34.758 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.wal_dir: 2024-07-03T15:46:34.758 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.table_cache_numshardbits: 6 2024-07-03T15:46:34.758 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.WAL_ttl_seconds: 0 2024-07-03T15:46:34.759 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.WAL_size_limit_MB: 0 2024-07-03T15:46:34.759 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-07-03T15:46:34.759 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-07-03T15:46:34.759 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.is_fd_close_on_exec: 1 2024-07-03T15:46:34.759 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.advise_random_on_open: 1 2024-07-03T15:46:34.759 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.db_write_buffer_size: 0 2024-07-03T15:46:34.759 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.write_buffer_manager: 0x55e1eed341e0 2024-07-03T15:46:34.759 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-07-03T15:46:34.759 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-07-03T15:46:34.759 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.use_adaptive_mutex: 0 2024-07-03T15:46:34.759 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.rate_limiter: (nil) 2024-07-03T15:46:34.759 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-07-03T15:46:34.759 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.wal_recovery_mode: 2 2024-07-03T15:46:34.760 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.enable_thread_tracking: 0 2024-07-03T15:46:34.760 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.enable_pipelined_write: 0 2024-07-03T15:46:34.760 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.unordered_write: 0 2024-07-03T15:46:34.760 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-07-03T15:46:34.760 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-07-03T15:46:34.760 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-07-03T15:46:34.760 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-07-03T15:46:34.760 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.row_cache: None 2024-07-03T15:46:34.760 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.wal_filter: None 2024-07-03T15:46:34.761 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-07-03T15:46:34.761 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.allow_ingest_behind: 0 2024-07-03T15:46:34.761 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.two_write_queues: 0 2024-07-03T15:46:34.761 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.manual_wal_flush: 0 2024-07-03T15:46:34.761 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.wal_compression: 0 2024-07-03T15:46:34.761 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.atomic_flush: 0 2024-07-03T15:46:34.761 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-07-03T15:46:34.762 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.persist_stats_to_disk: 0 2024-07-03T15:46:34.762 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.write_dbid_to_manifest: 0 2024-07-03T15:46:34.762 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.log_readahead_size: 0 2024-07-03T15:46:34.762 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-07-03T15:46:34.762 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.best_efforts_recovery: 0 2024-07-03T15:46:34.762 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-07-03T15:46:34.762 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-07-03T15:46:34.762 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.allow_data_in_errors: 0 2024-07-03T15:46:34.762 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.db_host_id: __hostname__ 2024-07-03T15:46:34.762 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.enforce_single_del_contracts: true 2024-07-03T15:46:34.762 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.max_background_jobs: 2 2024-07-03T15:46:34.762 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.max_background_compactions: -1 2024-07-03T15:46:34.762 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.max_subcompactions: 1 2024-07-03T15:46:34.763 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-07-03T15:46:34.763 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-07-03T15:46:34.763 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.delayed_write_rate : 16777216 2024-07-03T15:46:34.763 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.max_total_wal_size: 0 2024-07-03T15:46:34.763 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-07-03T15:46:34.763 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.stats_dump_period_sec: 600 2024-07-03T15:46:34.763 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.stats_persist_period_sec: 600 2024-07-03T15:46:34.763 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-07-03T15:46:34.763 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.max_open_files: -1 2024-07-03T15:46:34.763 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.bytes_per_sync: 0 2024-07-03T15:46:34.763 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.wal_bytes_per_sync: 0 2024-07-03T15:46:34.763 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.strict_bytes_per_sync: 0 2024-07-03T15:46:34.763 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.compaction_readahead_size: 0 2024-07-03T15:46:34.764 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.max_background_flushes: -1 2024-07-03T15:46:34.764 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Compression algorithms supported: 2024-07-03T15:46:34.764 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: kZSTD supported: 0 2024-07-03T15:46:34.764 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: kXpressCompression supported: 0 2024-07-03T15:46:34.764 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: kBZip2Compression supported: 0 2024-07-03T15:46:34.764 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-07-03T15:46:34.764 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: kLZ4Compression supported: 1 2024-07-03T15:46:34.764 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: kZlibCompression supported: 1 2024-07-03T15:46:34.764 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: kLZ4HCCompression supported: 1 2024-07-03T15:46:34.764 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: kSnappyCompression supported: 1 2024-07-03T15:46:34.764 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Fast CRC32 supported: Supported on x86 2024-07-03T15:46:34.764 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: DMutex implementation: pthread_mutex_t 2024-07-03T15:46:34.764 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi148/store.db/MANIFEST-000009 2024-07-03T15:46:34.764 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-07-03T15:46:34.765 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-07-03T15:46:34.765 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.merge_operator: 2024-07-03T15:46:34.765 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.compaction_filter: None 2024-07-03T15:46:34.765 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.compaction_filter_factory: None 2024-07-03T15:46:34.765 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.sst_partitioner_factory: None 2024-07-03T15:46:34.765 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.memtable_factory: SkipListFactory 2024-07-03T15:46:34.765 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.table_factory: BlockBasedTable 2024-07-03T15:46:34.765 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e1eed24f60) 2024-07-03T15:46:34.765 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: cache_index_and_filter_blocks: 1 2024-07-03T15:46:34.765 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-07-03T15:46:34.765 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-07-03T15:46:34.765 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: pin_top_level_index_and_filter: 1 2024-07-03T15:46:34.765 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: index_type: 0 2024-07-03T15:46:34.766 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: data_block_index_type: 0 2024-07-03T15:46:34.766 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: index_shortening: 1 2024-07-03T15:46:34.766 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: data_block_hash_table_util_ratio: 0.750000 2024-07-03T15:46:34.766 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: checksum: 4 2024-07-03T15:46:34.766 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: no_block_cache: 0 2024-07-03T15:46:34.766 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: block_cache: 0x55e1eed1d1f0 2024-07-03T15:46:34.766 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: block_cache_name: BinnedLRUCache 2024-07-03T15:46:34.766 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: block_cache_options: 2024-07-03T15:46:34.766 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: capacity : 536870912 2024-07-03T15:46:34.766 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: num_shard_bits : 4 2024-07-03T15:46:34.766 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: strict_capacity_limit : 0 2024-07-03T15:46:34.766 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: high_pri_pool_ratio: 0.000 2024-07-03T15:46:34.766 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: block_cache_compressed: (nil) 2024-07-03T15:46:34.766 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: persistent_cache: (nil) 2024-07-03T15:46:34.767 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: block_size: 4096 2024-07-03T15:46:34.767 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: block_size_deviation: 10 2024-07-03T15:46:34.767 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: block_restart_interval: 16 2024-07-03T15:46:34.767 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: index_block_restart_interval: 1 2024-07-03T15:46:34.767 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: metadata_block_size: 4096 2024-07-03T15:46:34.767 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: partition_filters: 0 2024-07-03T15:46:34.767 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: use_delta_encoding: 1 2024-07-03T15:46:34.767 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: filter_policy: bloomfilter 2024-07-03T15:46:34.767 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: whole_key_filtering: 1 2024-07-03T15:46:34.767 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: verify_compression: 0 2024-07-03T15:46:34.767 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: read_amp_bytes_per_bit: 0 2024-07-03T15:46:34.767 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: format_version: 5 2024-07-03T15:46:34.767 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: enable_index_compression: 1 2024-07-03T15:46:34.768 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: block_align: 0 2024-07-03T15:46:34.768 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: max_auto_readahead_size: 262144 2024-07-03T15:46:34.768 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: prepopulate_block_cache: 0 2024-07-03T15:46:34.768 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: initial_auto_readahead_size: 8192 2024-07-03T15:46:34.768 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: num_file_reads_for_auto_readahead: 2 2024-07-03T15:46:34.768 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.write_buffer_size: 33554432 2024-07-03T15:46:34.768 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.max_write_buffer_number: 2 2024-07-03T15:46:34.768 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.compression: NoCompression 2024-07-03T15:46:34.768 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.bottommost_compression: Disabled 2024-07-03T15:46:34.768 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.prefix_extractor: nullptr 2024-07-03T15:46:34.768 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-07-03T15:46:34.768 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.num_levels: 7 2024-07-03T15:46:34.769 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-07-03T15:46:34.769 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-07-03T15:46:34.769 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-07-03T15:46:34.769 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-07-03T15:46:34.769 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-07-03T15:46:34.769 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-07-03T15:46:34.769 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-07-03T15:46:34.769 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-07-03T15:46:34.769 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-07-03T15:46:34.769 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-07-03T15:46:34.769 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-07-03T15:46:34.769 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-07-03T15:46:34.769 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.compression_opts.window_bits: -14 2024-07-03T15:46:34.770 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.compression_opts.level: 32767 2024-07-03T15:46:34.770 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.compression_opts.strategy: 0 2024-07-03T15:46:34.770 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-07-03T15:46:34.770 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-07-03T15:46:34.770 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-07-03T15:46:34.770 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-07-03T15:46:34.770 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.compression_opts.enabled: false 2024-07-03T15:46:34.770 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-07-03T15:46:34.770 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-07-03T15:46:34.770 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-07-03T15:46:34.770 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-07-03T15:46:34.770 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.target_file_size_base: 67108864 2024-07-03T15:46:34.770 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.target_file_size_multiplier: 1 2024-07-03T15:46:34.771 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-07-03T15:46:34.771 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-07-03T15:46:34.771 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-07-03T15:46:34.771 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-07-03T15:46:34.771 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-07-03T15:46:34.771 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-07-03T15:46:34.771 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-07-03T15:46:34.771 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-07-03T15:46:34.771 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-07-03T15:46:34.771 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-07-03T15:46:34.771 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-07-03T15:46:34.771 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-07-03T15:46:34.771 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-07-03T15:46:34.771 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.arena_block_size: 1048576 2024-07-03T15:46:34.772 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-07-03T15:46:34.772 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-07-03T15:46:34.772 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.disable_auto_compactions: 0 2024-07-03T15:46:34.772 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-07-03T15:46:34.772 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-07-03T15:46:34.772 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-07-03T15:46:34.772 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-07-03T15:46:34.772 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-07-03T15:46:34.772 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-07-03T15:46:34.772 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-07-03T15:46:34.773 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-07-03T15:46:34.773 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-07-03T15:46:34.773 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-07-03T15:46:34.773 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.table_properties_collectors: 2024-07-03T15:46:34.773 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.inplace_update_support: 0 2024-07-03T15:46:34.773 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.inplace_update_num_locks: 10000 2024-07-03T15:46:34.773 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-07-03T15:46:34.773 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-07-03T15:46:34.773 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.memtable_huge_page_size: 0 2024-07-03T15:46:34.773 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.bloom_locality: 0 2024-07-03T15:46:34.773 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.max_successive_merges: 0 2024-07-03T15:46:34.773 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.optimize_filters_for_hits: 0 2024-07-03T15:46:34.773 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.paranoid_file_checks: 0 2024-07-03T15:46:34.774 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.force_consistency_checks: 1 2024-07-03T15:46:34.774 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.report_bg_io_stats: 0 2024-07-03T15:46:34.774 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.ttl: 2592000 2024-07-03T15:46:34.774 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.periodic_compaction_seconds: 0 2024-07-03T15:46:34.774 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-07-03T15:46:34.774 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-07-03T15:46:34.774 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.enable_blob_files: false 2024-07-03T15:46:34.774 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.min_blob_size: 0 2024-07-03T15:46:34.774 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.blob_file_size: 268435456 2024-07-03T15:46:34.774 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.blob_compression_type: NoCompression 2024-07-03T15:46:34.774 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.enable_blob_garbage_collection: false 2024-07-03T15:46:34.774 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-07-03T15:46:34.774 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-07-03T15:46:34.775 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-07-03T15:46:34.775 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.blob_file_starting_level: 0 2024-07-03T15:46:34.775 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-07-03T15:46:34.775 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 24.sst 2024-07-03T15:46:34.775 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-07-03T15:46:34.775 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi148/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 26, last_sequence is 10336, log_number is 22,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-07-03T15:46:34.775 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 22 2024-07-03T15:46:34.775 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: aa7f1c5d-03d7-4cdd-9d82-e6d25900988e 2024-07-03T15:46:34.775 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: EVENT_LOG_v1 {"time_micros": 1720021594751594, "job": 1, "event": "recovery_started", "wal_files": [22]} 2024-07-03T15:46:34.775 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #22 mode 2 2024-07-03T15:46:35.179 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: EVENT_LOG_v1 {"time_micros": 1720021594764331, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 27, "file_size": 3548227, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 10341, "largest_seqno": 11733, "table_properties": {"data_size": 3541666, "index_size": 4006, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1541, "raw_key_size": 13951, "raw_average_key_size": 23, "raw_value_size": 3528964, "raw_average_value_size": 5941, "num_data_blocks": 182, "num_entries": 594, "num_filter_entries": 594, "num_deletions": 2, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1720021594, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "aa7f1c5d-03d7-4cdd-9d82-e6d25900988e", "db_session_id": "538JZ7I890DYIMW45ZH6", "orig_file_number": 27, "seqno_to_time_mapping": "N/A"}} 2024-07-03T15:46:35.179 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: EVENT_LOG_v1 {"time_micros": 1720021594764461, "job": 1, "event": "recovery_finished"} 2024-07-03T15:46:35.179 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: [db/version_set.cc:5047] Creating manifest 29 2024-07-03T15:46:35.179 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-07-03T15:46:35.179 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi148/store.db/000022.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-07-03T15:46:35.180 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55e1eed48e00 2024-07-03T15:46:35.180 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: DB pointer 0x55e1eee3e000 2024-07-03T15:46:35.180 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-07-03T15:46:35.180 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-07-03T15:46:35.180 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: ** DB Stats ** 2024-07-03T15:46:35.180 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-07-03T15:46:35.180 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-07-03T15:46:35.180 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-07-03T15:46:35.180 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-07-03T15:46:35.180 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-07-03T15:46:35.180 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-07-03T15:46:35.181 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-07-03T15:46:35.181 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: 2024-07-03T15:46:35.181 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: ** Compaction Stats [default] ** 2024-07-03T15:46:35.181 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-07-03T15:46:35.181 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-07-03T15:46:35.181 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: L0 1/0 3.38 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 658.1 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-07-03T15:46:35.181 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: L6 1/0 5.25 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-07-03T15:46:35.181 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Sum 2/0 8.63 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 658.1 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-07-03T15:46:35.181 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 658.1 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-07-03T15:46:35.181 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: 2024-07-03T15:46:35.181 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: ** Compaction Stats [default] ** 2024-07-03T15:46:35.181 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-07-03T15:46:35.181 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-07-03T15:46:35.181 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 658.1 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-07-03T15:46:35.182 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: 2024-07-03T15:46:35.182 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-07-03T15:46:35.182 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: 2024-07-03T15:46:35.182 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-07-03T15:46:35.182 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Flush(GB): cumulative 0.003, interval 0.003 2024-07-03T15:46:35.182 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-07-03T15:46:35.182 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-07-03T15:46:35.182 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-07-03T15:46:35.182 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: AddFile(Keys): cumulative 0, interval 0 2024-07-03T15:46:35.182 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Cumulative compaction: 0.00 GB write, 124.53 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-07-03T15:46:35.182 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Interval compaction: 0.00 GB write, 124.53 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-07-03T15:46:35.182 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-07-03T15:46:35.182 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Block cache BinnedLRUCache@0x55e1eed1d1f0#2 capacity: 512.00 MB usage: 5.86 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-07-03T15:46:35.183 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,1.78 KB,0.000339746%) IndexBlock(1,4.08 KB,0.000777841%) Misc(1,0.00 KB,0%) 2024-07-03T15:46:35.183 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: 2024-07-03T15:46:35.183 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: ** File Read Latency Histogram By Level [default] ** 2024-07-03T15:46:35.183 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: starting mon.smithi148 rank 1 at public addrs [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] at bind addrs [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi148 fsid a85e4be6-3951-11ef-bcab-c7b262605968 2024-07-03T15:46:35.183 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: mon.smithi148@-1(???) e2 preinit fsid a85e4be6-3951-11ef-bcab-c7b262605968 2024-07-03T15:46:35.183 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: mon.smithi148@-1(???).mds e17 new map 2024-07-03T15:46:35.183 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: mon.smithi148@-1(???).mds e17 print_map 2024-07-03T15:46:35.183 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: e17 2024-07-03T15:46:35.183 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-07-03T15:46:35.183 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:46:35.183 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: legacy client fscid: 1 2024-07-03T15:46:35.183 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: 2024-07-03T15:46:35.183 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Filesystem 'cephfs' (1) 2024-07-03T15:46:35.184 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: fs_name cephfs 2024-07-03T15:46:35.184 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: epoch 16 2024-07-03T15:46:35.184 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-03T15:46:35.184 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: created 2024-07-03T15:41:40.656298+0000 2024-07-03T15:46:35.184 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: modified 2024-07-03T15:41:56.454984+0000 2024-07-03T15:46:35.184 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: tableserver 0 2024-07-03T15:46:35.184 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: root 0 2024-07-03T15:46:35.184 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: session_timeout 60 2024-07-03T15:46:35.184 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: session_autoclose 300 2024-07-03T15:46:35.184 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: max_file_size 1099511627776 2024-07-03T15:46:35.184 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: max_xattr_size 65536 2024-07-03T15:46:35.184 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: required_client_features {} 2024-07-03T15:46:35.184 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: last_failure 0 2024-07-03T15:46:35.184 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: last_failure_osd_epoch 0 2024-07-03T15:46:35.184 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:46:35.185 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: max_mds 1 2024-07-03T15:46:35.185 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: in 0 2024-07-03T15:46:35.185 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: up {0=14450} 2024-07-03T15:46:35.185 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: failed 2024-07-03T15:46:35.185 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: damaged 2024-07-03T15:46:35.185 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: stopped 2024-07-03T15:46:35.185 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: data_pools [3] 2024-07-03T15:46:35.185 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: metadata_pool 2 2024-07-03T15:46:35.185 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: inline_data disabled 2024-07-03T15:46:35.185 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: balancer 2024-07-03T15:46:35.185 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: bal_rank_mask -1 2024-07-03T15:46:35.185 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: standby_count_wanted 1 2024-07-03T15:46:35.185 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: qdb_cluster leader: 0 members: 2024-07-03T15:46:35.185 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: [mds.cephfs.smithi028.rhpvve{0:14450} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.28:6826/1613936042,v1:172.21.15.28:6827/1613936042] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:46:35.186 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: [mds.cephfs.smithi028.wmbxew{0:14460} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.28:6828/1730336821,v1:172.21.15.28:6829/1730336821] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:46:35.186 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: 2024-07-03T15:46:35.186 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: 2024-07-03T15:46:35.186 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Standby daemons: 2024-07-03T15:46:35.186 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: 2024-07-03T15:46:35.186 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: [mds.cephfs.smithi148.szomfl{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/4171231858,v1:172.21.15.148:6827/4171231858] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:46:35.186 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: [mds.cephfs.smithi148.xgpuld{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/2347536398,v1:172.21.15.148:6825/2347536398] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:46:35.186 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: mon.smithi148@-1(???).osd e48 crush map has features 3314933000852226048, adjusting msgr requires 2024-07-03T15:46:35.186 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: mon.smithi148@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-07-03T15:46:35.186 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: mon.smithi148@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-07-03T15:46:35.186 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: mon.smithi148@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-07-03T15:46:35.186 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 ceph-mon[80490]: mon.smithi148@-1(???).paxosservice(auth 1..21) refresh upgraded, format 0 -> 3 2024-07-03T15:46:35.186 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 bash[80461]: d338d2c648dab4f59e621f5c94b28fea53c4543e6132d5f525d1b021a3a18467 2024-07-03T15:46:35.186 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:34 smithi148 systemd[1]: Started Ceph mon.smithi148 for a85e4be6-3951-11ef-bcab-c7b262605968. 2024-07-03T15:46:35.187 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:35 smithi148 ceph-mon[80490]: pgmap v8: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 0 B/s wr, 14 op/s 2024-07-03T15:46:35.595 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:35 smithi028 ceph-mon[92858]: pgmap v8: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 0 B/s wr, 14 op/s 2024-07-03T15:46:36.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:36 smithi148 ceph-mon[80490]: osdmap e48: 6 total, 6 up, 6 in 2024-07-03T15:46:36.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:36 smithi148 ceph-mon[80490]: mgrmap e36: smithi028.vtqrtz(active, since 13s), standbys: smithi148.rjzsar 2024-07-03T15:46:36.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:36 smithi148 ceph-mon[80490]: overall HEALTH_OK 2024-07-03T15:46:36.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:36 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi028"}]: dispatch 2024-07-03T15:46:36.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:36 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-07-03T15:46:36.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:36 smithi148 ceph-mon[80490]: mon.smithi028 calling monitor election 2024-07-03T15:46:36.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:36 smithi148 ceph-mon[80490]: mon.smithi148 calling monitor election 2024-07-03T15:46:36.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:36 smithi148 ceph-mon[80490]: mon.smithi028 is new leader, mons smithi028,smithi148 in quorum (ranks 0,1) 2024-07-03T15:46:36.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:36 smithi148 ceph-mon[80490]: monmap epoch 3 2024-07-03T15:46:36.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:36 smithi148 ceph-mon[80490]: fsid a85e4be6-3951-11ef-bcab-c7b262605968 2024-07-03T15:46:36.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:36 smithi148 ceph-mon[80490]: last_changed 2024-07-03T15:46:35.178149+0000 2024-07-03T15:46:36.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:36 smithi148 ceph-mon[80490]: created 2024-07-03T15:34:53.167064+0000 2024-07-03T15:46:36.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:36 smithi148 ceph-mon[80490]: min_mon_release 19 (squid) 2024-07-03T15:46:36.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:36 smithi148 ceph-mon[80490]: election_strategy: 1 2024-07-03T15:46:36.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:36 smithi148 ceph-mon[80490]: 0: [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] mon.smithi028 2024-07-03T15:46:36.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:36 smithi148 ceph-mon[80490]: 1: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.smithi148 2024-07-03T15:46:36.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:36 smithi148 ceph-mon[80490]: fsmap cephfs:1 {0=cephfs.smithi028.rhpvve=up:active} 1 up:standby-replay 2 up:standby 2024-07-03T15:46:36.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:36 smithi148 ceph-mon[80490]: osdmap e48: 6 total, 6 up, 6 in 2024-07-03T15:46:36.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:36 smithi148 ceph-mon[80490]: mgrmap e36: smithi028.vtqrtz(active, since 13s), standbys: smithi148.rjzsar 2024-07-03T15:46:36.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:36 smithi148 ceph-mon[80490]: overall HEALTH_OK 2024-07-03T15:46:36.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:36 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:36.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:36 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:36.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:36 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:46:36.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:36 smithi028 ceph-mon[92858]: osdmap e48: 6 total, 6 up, 6 in 2024-07-03T15:46:36.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:36 smithi028 ceph-mon[92858]: mgrmap e36: smithi028.vtqrtz(active, since 13s), standbys: smithi148.rjzsar 2024-07-03T15:46:36.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:36 smithi028 ceph-mon[92858]: overall HEALTH_OK 2024-07-03T15:46:36.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:36 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi028"}]: dispatch 2024-07-03T15:46:36.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:36 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-07-03T15:46:36.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:36 smithi028 ceph-mon[92858]: mon.smithi028 calling monitor election 2024-07-03T15:46:36.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:36 smithi028 ceph-mon[92858]: mon.smithi148 calling monitor election 2024-07-03T15:46:36.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:36 smithi028 ceph-mon[92858]: mon.smithi028 is new leader, mons smithi028,smithi148 in quorum (ranks 0,1) 2024-07-03T15:46:36.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:36 smithi028 ceph-mon[92858]: monmap epoch 3 2024-07-03T15:46:36.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:36 smithi028 ceph-mon[92858]: fsid a85e4be6-3951-11ef-bcab-c7b262605968 2024-07-03T15:46:36.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:36 smithi028 ceph-mon[92858]: last_changed 2024-07-03T15:46:35.178149+0000 2024-07-03T15:46:36.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:36 smithi028 ceph-mon[92858]: created 2024-07-03T15:34:53.167064+0000 2024-07-03T15:46:36.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:36 smithi028 ceph-mon[92858]: min_mon_release 19 (squid) 2024-07-03T15:46:36.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:36 smithi028 ceph-mon[92858]: election_strategy: 1 2024-07-03T15:46:36.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:36 smithi028 ceph-mon[92858]: 0: [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] mon.smithi028 2024-07-03T15:46:36.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:36 smithi028 ceph-mon[92858]: 1: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.smithi148 2024-07-03T15:46:36.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:36 smithi028 ceph-mon[92858]: fsmap cephfs:1 {0=cephfs.smithi028.rhpvve=up:active} 1 up:standby-replay 2 up:standby 2024-07-03T15:46:36.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:36 smithi028 ceph-mon[92858]: osdmap e48: 6 total, 6 up, 6 in 2024-07-03T15:46:36.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:36 smithi028 ceph-mon[92858]: mgrmap e36: smithi028.vtqrtz(active, since 13s), standbys: smithi148.rjzsar 2024-07-03T15:46:36.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:36 smithi028 ceph-mon[92858]: overall HEALTH_OK 2024-07-03T15:46:36.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:36 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:36.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:36 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:36.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:36 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:46:37.473 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:37 smithi148 ceph-mon[80490]: pgmap v10: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 18 KiB/s rd, 0 B/s wr, 11 op/s 2024-07-03T15:46:37.474 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:37 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:46:37.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:37 smithi028 ceph-mon[92858]: pgmap v10: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 18 KiB/s rd, 0 B/s wr, 11 op/s 2024-07-03T15:46:37.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:37 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:46:39.345 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:39 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:39.345 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:39 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:39.345 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:39 smithi148 ceph-mon[80490]: pgmap v11: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 18 KiB/s rd, 0 B/s wr, 11 op/s 2024-07-03T15:46:39.345 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:39 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:39.345 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:39 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:39.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:39 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:39.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:39 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:39.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:39 smithi028 ceph-mon[92858]: pgmap v11: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 18 KiB/s rd, 0 B/s wr, 11 op/s 2024-07-03T15:46:39.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:39 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:39.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:39 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:41.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:41 smithi148 ceph-mon[80490]: pgmap v12: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 18 KiB/s rd, 0 B/s wr, 12 op/s 2024-07-03T15:46:41.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:41 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:41.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:41 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:41.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:41 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:46:41.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:41 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:46:41.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:41 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:41.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:41 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-03T15:46:41.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:41 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-03T15:46:41.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:41 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:46:42.078 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:41 smithi028 ceph-mon[92858]: pgmap v12: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 18 KiB/s rd, 0 B/s wr, 12 op/s 2024-07-03T15:46:42.078 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:41 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:42.078 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:41 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:42.078 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:41 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:46:42.079 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:41 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:46:42.079 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:41 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:42.079 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:41 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-03T15:46:42.079 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:41 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-03T15:46:42.079 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:41 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:46:43.076 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:42 smithi028 ceph-mon[92858]: Reconfiguring mon.smithi028 (monmap changed)... 2024-07-03T15:46:43.076 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:42 smithi028 ceph-mon[92858]: Reconfiguring daemon mon.smithi028 on smithi028 2024-07-03T15:46:43.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:42 smithi148 ceph-mon[80490]: Reconfiguring mon.smithi028 (monmap changed)... 2024-07-03T15:46:43.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:42 smithi148 ceph-mon[80490]: Reconfiguring daemon mon.smithi028 on smithi028 2024-07-03T15:46:44.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:43 smithi028 ceph-mon[92858]: pgmap v13: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:46:44.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:43 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:44.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:43 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:44.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:43 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi028.vtqrtz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-03T15:46:44.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:43 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-03T15:46:44.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:43 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:46:44.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:43 smithi148 ceph-mon[80490]: pgmap v13: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:46:44.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:43 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:44.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:43 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:44.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:43 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi028.vtqrtz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-03T15:46:44.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:43 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-03T15:46:44.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:43 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:46:45.085 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:44 smithi028 ceph-mon[92858]: Reconfiguring mgr.smithi028.vtqrtz (monmap changed)... 2024-07-03T15:46:45.085 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:44 smithi028 ceph-mon[92858]: Reconfiguring daemon mgr.smithi028.vtqrtz on smithi028 2024-07-03T15:46:45.085 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:44 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:45.085 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:44 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:45.086 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:44 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi028", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-07-03T15:46:45.086 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:44 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:46:45.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:44 smithi148 ceph-mon[80490]: Reconfiguring mgr.smithi028.vtqrtz (monmap changed)... 2024-07-03T15:46:45.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:44 smithi148 ceph-mon[80490]: Reconfiguring daemon mgr.smithi028.vtqrtz on smithi028 2024-07-03T15:46:45.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:44 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:45.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:44 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:45.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:44 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi028", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-07-03T15:46:45.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:44 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:46:45.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:45 smithi028 ceph-mon[92858]: pgmap v14: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:46:45.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:45 smithi028 ceph-mon[92858]: Reconfiguring crash.smithi028 (monmap changed)... 2024-07-03T15:46:45.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:45 smithi028 ceph-mon[92858]: Reconfiguring daemon crash.smithi028 on smithi028 2024-07-03T15:46:46.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:45 smithi148 ceph-mon[80490]: pgmap v14: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:46:46.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:45 smithi148 ceph-mon[80490]: Reconfiguring crash.smithi028 (monmap changed)... 2024-07-03T15:46:46.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:45 smithi148 ceph-mon[80490]: Reconfiguring daemon crash.smithi028 on smithi028 2024-07-03T15:46:47.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:46 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:47.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:46 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:47.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:46 smithi028 ceph-mon[92858]: Reconfiguring osd.0 (monmap changed)... 2024-07-03T15:46:47.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:46 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-07-03T15:46:47.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:46 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:46:47.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:46 smithi028 ceph-mon[92858]: Reconfiguring daemon osd.0 on smithi028 2024-07-03T15:46:47.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:46 smithi028 ceph-mon[92858]: pgmap v15: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:46:47.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:46 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:47.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:46 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:47.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:46 smithi148 ceph-mon[80490]: Reconfiguring osd.0 (monmap changed)... 2024-07-03T15:46:47.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:46 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-07-03T15:46:47.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:46 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:46:47.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:46 smithi148 ceph-mon[80490]: Reconfiguring daemon osd.0 on smithi028 2024-07-03T15:46:47.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:46 smithi148 ceph-mon[80490]: pgmap v15: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:46:48.663 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:48 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:48.664 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:48 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:48.664 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:48 smithi028 ceph-mon[92858]: Reconfiguring osd.1 (monmap changed)... 2024-07-03T15:46:48.664 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:48 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-07-03T15:46:48.664 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:48 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:46:48.664 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:48 smithi028 ceph-mon[92858]: Reconfiguring daemon osd.1 on smithi028 2024-07-03T15:46:48.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:48 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:48.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:48 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:48.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:48 smithi148 ceph-mon[80490]: Reconfiguring osd.1 (monmap changed)... 2024-07-03T15:46:48.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:48 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-07-03T15:46:48.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:48 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:46:48.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:48 smithi148 ceph-mon[80490]: Reconfiguring daemon osd.1 on smithi028 2024-07-03T15:46:50.057 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:49 smithi028 ceph-mon[92858]: pgmap v16: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:46:50.058 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:50.058 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:50.058 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:49 smithi028 ceph-mon[92858]: Reconfiguring osd.2 (monmap changed)... 2024-07-03T15:46:50.058 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-07-03T15:46:50.058 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:46:50.058 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:49 smithi028 ceph-mon[92858]: Reconfiguring daemon osd.2 on smithi028 2024-07-03T15:46:50.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:49 smithi148 ceph-mon[80490]: pgmap v16: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:46:50.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:50.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:50.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:49 smithi148 ceph-mon[80490]: Reconfiguring osd.2 (monmap changed)... 2024-07-03T15:46:50.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-07-03T15:46:50.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:46:50.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:49 smithi148 ceph-mon[80490]: Reconfiguring daemon osd.2 on smithi028 2024-07-03T15:46:51.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:51 smithi028 ceph-mon[92858]: pgmap v17: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:46:51.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:51 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:51.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:51 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:51.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:51 smithi028 ceph-mon[92858]: Reconfiguring mds.cephfs.smithi028.rhpvve (monmap changed)... 2024-07-03T15:46:51.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:51 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi028.rhpvve", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-03T15:46:51.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:51 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:46:51.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:51 smithi028 ceph-mon[92858]: Reconfiguring daemon mds.cephfs.smithi028.rhpvve on smithi028 2024-07-03T15:46:51.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:51 smithi148 ceph-mon[80490]: pgmap v17: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:46:51.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:51 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:51.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:51 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:51.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:51 smithi148 ceph-mon[80490]: Reconfiguring mds.cephfs.smithi028.rhpvve (monmap changed)... 2024-07-03T15:46:51.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:51 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi028.rhpvve", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-03T15:46:51.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:51 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:46:51.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:51 smithi148 ceph-mon[80490]: Reconfiguring daemon mds.cephfs.smithi028.rhpvve on smithi028 2024-07-03T15:46:52.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:52 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:52.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:52 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:52.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:52 smithi148 ceph-mon[80490]: Reconfiguring mds.cephfs.smithi028.wmbxew (monmap changed)... 2024-07-03T15:46:52.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:52 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi028.wmbxew", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-03T15:46:52.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:52 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:46:52.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:52 smithi148 ceph-mon[80490]: Reconfiguring daemon mds.cephfs.smithi028.wmbxew on smithi028 2024-07-03T15:46:52.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:52 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:46:53.018 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:52 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:53.018 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:52 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:53.018 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:52 smithi028 ceph-mon[92858]: Reconfiguring mds.cephfs.smithi028.wmbxew (monmap changed)... 2024-07-03T15:46:53.018 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:52 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi028.wmbxew", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-03T15:46:53.018 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:52 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:46:53.019 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:52 smithi028 ceph-mon[92858]: Reconfiguring daemon mds.cephfs.smithi028.wmbxew on smithi028 2024-07-03T15:46:53.019 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:52 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:46:53.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:53 smithi028 ceph-mon[92858]: pgmap v18: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:46:53.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:53 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:53.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:53 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:53.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:53 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi148", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-07-03T15:46:53.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:53 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:46:53.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:53 smithi148 ceph-mon[80490]: pgmap v18: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:46:53.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:53 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:53.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:53 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:53.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:53 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi148", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-07-03T15:46:53.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:53 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:46:55.005 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:54 smithi148 ceph-mon[80490]: Reconfiguring crash.smithi148 (monmap changed)... 2024-07-03T15:46:55.005 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:54 smithi148 ceph-mon[80490]: Reconfiguring daemon crash.smithi148 on smithi148 2024-07-03T15:46:55.005 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:54 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:55.005 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:54 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:55.006 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:54 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi148.rjzsar", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-03T15:46:55.006 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:54 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-03T15:46:55.006 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:54 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:46:55.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:54 smithi028 ceph-mon[92858]: Reconfiguring crash.smithi148 (monmap changed)... 2024-07-03T15:46:55.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:54 smithi028 ceph-mon[92858]: Reconfiguring daemon crash.smithi148 on smithi148 2024-07-03T15:46:55.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:54 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:55.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:54 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:55.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:54 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi148.rjzsar", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-03T15:46:55.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:54 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-03T15:46:55.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:54 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:46:55.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:55 smithi028 ceph-mon[92858]: pgmap v19: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:46:55.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:55 smithi028 ceph-mon[92858]: Reconfiguring mgr.smithi148.rjzsar (monmap changed)... 2024-07-03T15:46:55.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:55 smithi028 ceph-mon[92858]: Reconfiguring daemon mgr.smithi148.rjzsar on smithi148 2024-07-03T15:46:55.903 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:55 smithi148 ceph-mon[80490]: pgmap v19: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:46:55.903 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:55 smithi148 ceph-mon[80490]: Reconfiguring mgr.smithi148.rjzsar (monmap changed)... 2024-07-03T15:46:55.903 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:55 smithi148 ceph-mon[80490]: Reconfiguring daemon mgr.smithi148.rjzsar on smithi148 2024-07-03T15:46:57.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:56 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:57.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:56 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:57.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:56 smithi148 ceph-mon[80490]: Reconfiguring mon.smithi148 (monmap changed)... 2024-07-03T15:46:57.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:56 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-03T15:46:57.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:56 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-03T15:46:57.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:56 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:46:57.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:56 smithi148 ceph-mon[80490]: Reconfiguring daemon mon.smithi148 on smithi148 2024-07-03T15:46:57.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:56 smithi148 ceph-mon[80490]: pgmap v20: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:46:57.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:56 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:57.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:56 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:57.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:56 smithi028 ceph-mon[92858]: Reconfiguring mon.smithi148 (monmap changed)... 2024-07-03T15:46:57.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:56 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-03T15:46:57.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:56 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-03T15:46:57.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:56 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:46:57.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:56 smithi028 ceph-mon[92858]: Reconfiguring daemon mon.smithi148 on smithi148 2024-07-03T15:46:57.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:56 smithi028 ceph-mon[92858]: pgmap v20: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:46:58.663 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:58 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:58.664 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:58 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:58.664 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:58 smithi028 ceph-mon[92858]: Reconfiguring osd.3 (monmap changed)... 2024-07-03T15:46:58.664 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:58 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-07-03T15:46:58.664 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:58 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:46:58.664 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:58 smithi028 ceph-mon[92858]: Reconfiguring daemon osd.3 on smithi148 2024-07-03T15:46:58.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:58 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:58.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:58 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:58.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:58 smithi148 ceph-mon[80490]: Reconfiguring osd.3 (monmap changed)... 2024-07-03T15:46:58.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:58 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-07-03T15:46:58.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:58 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:46:58.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:58 smithi148 ceph-mon[80490]: Reconfiguring daemon osd.3 on smithi148 2024-07-03T15:46:59.611 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:59 smithi148 ceph-mon[80490]: pgmap v21: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:46:59.611 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:59 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:59.611 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:59 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:59.612 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:59 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-07-03T15:46:59.612 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:46:59 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:46:59.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:59 smithi028 ceph-mon[92858]: pgmap v21: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:46:59.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:59 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:59.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:59 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:46:59.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:59 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-07-03T15:46:59.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:46:59 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:47:00.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.295+0000 7fb5df02a640 1 -- 172.21.15.28:0/1689845915 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5d80fb290 msgr2=0x7fb5d80fb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:00.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.295+0000 7fb5df02a640 1 --2- 172.21.15.28:0/1689845915 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5d80fb290 0x7fb5d80fb6a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb5c80048d0 tx=0x7fb5c8009da0 comp rx=0 tx=0).stop 2024-07-03T15:47:00.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.295+0000 7fb5df02a640 1 -- 172.21.15.28:0/1689845915 shutdown_connections 2024-07-03T15:47:00.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.295+0000 7fb5df02a640 1 --2- 172.21.15.28:0/1689845915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5d80fc560 0x7fb5d80fc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:00.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.295+0000 7fb5df02a640 1 --2- 172.21.15.28:0/1689845915 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5d80fb290 0x7fb5d80fb6a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:00.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.295+0000 7fb5df02a640 1 -- 172.21.15.28:0/1689845915 >> 172.21.15.28:0/1689845915 conn(0x7fb5d80f6a50 msgr2=0x7fb5d80f8e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:47:00.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.295+0000 7fb5df02a640 1 -- 172.21.15.28:0/1689845915 shutdown_connections 2024-07-03T15:47:00.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.295+0000 7fb5df02a640 1 -- 172.21.15.28:0/1689845915 wait complete. 2024-07-03T15:47:00.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.296+0000 7fb5df02a640 1 Processor -- start 2024-07-03T15:47:00.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.296+0000 7fb5df02a640 1 -- start start 2024-07-03T15:47:00.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.296+0000 7fb5df02a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5d80fb290 0x7fb5d810cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:47:00.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.296+0000 7fb5df02a640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5d80fc560 0x7fb5d810d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:47:00.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.296+0000 7fb5df02a640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d810db60 con 0x7fb5d80fc560 2024-07-03T15:47:00.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.296+0000 7fb5df02a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d810e380 con 0x7fb5d80fb290 2024-07-03T15:47:00.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.296+0000 7fb5d7fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5d80fc560 0x7fb5d810d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:47:00.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.296+0000 7fb5d7fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5d80fc560 0x7fb5d810d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56882/0 (socket says 172.21.15.28:56882) 2024-07-03T15:47:00.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.296+0000 7fb5d7fff640 1 -- 172.21.15.28:0/2760609819 learned_addr learned my addr 172.21.15.28:0/2760609819 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:47:00.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.296+0000 7fb5dcdc6640 1 --2- 172.21.15.28:0/2760609819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5d80fb290 0x7fb5d810cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:47:00.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.296+0000 7fb5d7fff640 1 -- 172.21.15.28:0/2760609819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5d80fb290 msgr2=0x7fb5d810cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:00.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.296+0000 7fb5d7fff640 1 --2- 172.21.15.28:0/2760609819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5d80fb290 0x7fb5d810cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:00.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.297+0000 7fb5d7fff640 1 -- 172.21.15.28:0/2760609819 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c8005040 con 0x7fb5d80fc560 2024-07-03T15:47:00.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.297+0000 7fb5d7fff640 1 --2- 172.21.15.28:0/2760609819 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5d80fc560 0x7fb5d810d510 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0002a50 tx=0x7fb5c0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:47:00.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.297+0000 7fb5d57fa640 1 -- 172.21.15.28:0/2760609819 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5c0007c30 con 0x7fb5d80fc560 2024-07-03T15:47:00.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.297+0000 7fb5d57fa640 1 -- 172.21.15.28:0/2760609819 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb5c000f040 con 0x7fb5d80fc560 2024-07-03T15:47:00.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.297+0000 7fb5df02a640 1 -- 172.21.15.28:0/2760609819 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5d810e610 con 0x7fb5d80fc560 2024-07-03T15:47:00.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.297+0000 7fb5d57fa640 1 -- 172.21.15.28:0/2760609819 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5c000dad0 con 0x7fb5d80fc560 2024-07-03T15:47:00.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.297+0000 7fb5df02a640 1 -- 172.21.15.28:0/2760609819 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5d810ebb0 con 0x7fb5d80fc560 2024-07-03T15:47:00.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.298+0000 7fb5df02a640 1 -- 172.21.15.28:0/2760609819 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5d80fa6f0 con 0x7fb5d80fc560 2024-07-03T15:47:00.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.299+0000 7fb5d57fa640 1 -- 172.21.15.28:0/2760609819 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 36) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fb5c000dc70 con 0x7fb5d80fc560 2024-07-03T15:47:00.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.299+0000 7fb5d57fa640 1 --2- 172.21.15.28:0/2760609819 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fb5c406e1e0 0x7fb5c4070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:47:00.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.299+0000 7fb5d57fa640 1 -- 172.21.15.28:0/2760609819 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fb5c008d4b0 con 0x7fb5d80fc560 2024-07-03T15:47:00.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.299+0000 7fb5dcdc6640 1 --2- 172.21.15.28:0/2760609819 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fb5c406e1e0 0x7fb5c4070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:47:00.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.300+0000 7fb5dcdc6640 1 --2- 172.21.15.28:0/2760609819 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fb5c406e1e0 0x7fb5c4070690 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb5c80043d0 tx=0x7fb5c8035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:47:00.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.304+0000 7fb5d57fa640 1 -- 172.21.15.28:0/2760609819 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fb5c0056740 con 0x7fb5d80fc560 2024-07-03T15:47:00.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.513+0000 7fb5df02a640 1 -- 172.21.15.28:0/2760609819 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb5d810ee20 con 0x7fb5c406e1e0 2024-07-03T15:47:00.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.515+0000 7fb5d57fa640 1 -- 172.21.15.28:0/2760609819 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fb5d810ee20 con 0x7fb5c406e1e0 2024-07-03T15:47:00.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.517+0000 7fb5df02a640 1 -- 172.21.15.28:0/2760609819 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fb5c406e1e0 msgr2=0x7fb5c4070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:00.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.517+0000 7fb5df02a640 1 --2- 172.21.15.28:0/2760609819 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fb5c406e1e0 0x7fb5c4070690 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb5c80043d0 tx=0x7fb5c8035040 comp rx=0 tx=0).stop 2024-07-03T15:47:00.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.517+0000 7fb5df02a640 1 -- 172.21.15.28:0/2760609819 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5d80fc560 msgr2=0x7fb5d810d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:00.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.517+0000 7fb5df02a640 1 --2- 172.21.15.28:0/2760609819 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5d80fc560 0x7fb5d810d510 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0002a50 tx=0x7fb5c0004600 comp rx=0 tx=0).stop 2024-07-03T15:47:00.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.517+0000 7fb5df02a640 1 -- 172.21.15.28:0/2760609819 shutdown_connections 2024-07-03T15:47:00.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.517+0000 7fb5df02a640 1 --2- 172.21.15.28:0/2760609819 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fb5c406e1e0 0x7fb5c4070690 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:00.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.517+0000 7fb5df02a640 1 --2- 172.21.15.28:0/2760609819 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5d80fc560 0x7fb5d810d510 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:00.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.517+0000 7fb5df02a640 1 --2- 172.21.15.28:0/2760609819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5d80fb290 0x7fb5d810cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:00.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.517+0000 7fb5df02a640 1 -- 172.21.15.28:0/2760609819 >> 172.21.15.28:0/2760609819 conn(0x7fb5d80f6a50 msgr2=0x7fb5d80f8e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:47:00.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.517+0000 7fb5df02a640 1 -- 172.21.15.28:0/2760609819 shutdown_connections 2024-07-03T15:47:00.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.517+0000 7fb5df02a640 1 -- 172.21.15.28:0/2760609819 wait complete. 2024-07-03T15:47:00.530 INFO:teuthology.orchestra.run.smithi028.stdout:true 2024-07-03T15:47:00.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.600+0000 7fba424e1640 1 -- 172.21.15.28:0/1238524383 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba3c0fc370 msgr2=0x7fba3c0fc780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:00.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.600+0000 7fba424e1640 1 --2- 172.21.15.28:0/1238524383 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba3c0fc370 0x7fba3c0fc780 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fba2c0048d0 tx=0x7fba2c009da0 comp rx=0 tx=0).stop 2024-07-03T15:47:00.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.600+0000 7fba424e1640 1 -- 172.21.15.28:0/1238524383 shutdown_connections 2024-07-03T15:47:00.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.600+0000 7fba424e1640 1 --2- 172.21.15.28:0/1238524383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba3c0fd640 0x7fba3c0fdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:00.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.600+0000 7fba424e1640 1 --2- 172.21.15.28:0/1238524383 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba3c0fc370 0x7fba3c0fc780 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:00.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.600+0000 7fba424e1640 1 -- 172.21.15.28:0/1238524383 >> 172.21.15.28:0/1238524383 conn(0x7fba3c0f7b50 msgr2=0x7fba3c0f9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:47:00.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.600+0000 7fba424e1640 1 -- 172.21.15.28:0/1238524383 shutdown_connections 2024-07-03T15:47:00.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.600+0000 7fba424e1640 1 -- 172.21.15.28:0/1238524383 wait complete. 2024-07-03T15:47:00.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.601+0000 7fba424e1640 1 Processor -- start 2024-07-03T15:47:00.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.601+0000 7fba424e1640 1 -- start start 2024-07-03T15:47:00.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.601+0000 7fba424e1640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba3c0fc370 0x7fba3c10f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:47:00.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.601+0000 7fba424e1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba3c0fd640 0x7fba3c10f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:47:00.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.601+0000 7fba424e1640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3c10fcd0 con 0x7fba3c0fc370 2024-07-03T15:47:00.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.601+0000 7fba424e1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3c1104f0 con 0x7fba3c0fd640 2024-07-03T15:47:00.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.601+0000 7fba3bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba3c0fc370 0x7fba3c10f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:47:00.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.601+0000 7fba3bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba3c0fc370 0x7fba3c10f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56898/0 (socket says 172.21.15.28:56898) 2024-07-03T15:47:00.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.601+0000 7fba3bfff640 1 -- 172.21.15.28:0/1159168902 learned_addr learned my addr 172.21.15.28:0/1159168902 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:47:00.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.602+0000 7fba3b7fe640 1 --2- 172.21.15.28:0/1159168902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba3c0fd640 0x7fba3c10f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:47:00.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.602+0000 7fba3bfff640 1 -- 172.21.15.28:0/1159168902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba3c0fd640 msgr2=0x7fba3c10f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:00.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.602+0000 7fba3bfff640 1 --2- 172.21.15.28:0/1159168902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba3c0fd640 0x7fba3c10f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:00.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.602+0000 7fba3bfff640 1 -- 172.21.15.28:0/1159168902 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba2c005040 con 0x7fba3c0fc370 2024-07-03T15:47:00.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.602+0000 7fba3bfff640 1 --2- 172.21.15.28:0/1159168902 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba3c0fc370 0x7fba3c10f140 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fba2c0048a0 tx=0x7fba2c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:47:00.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.602+0000 7fba38ff9640 1 -- 172.21.15.28:0/1159168902 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba2c038070 con 0x7fba3c0fc370 2024-07-03T15:47:00.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.602+0000 7fba38ff9640 1 -- 172.21.15.28:0/1159168902 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fba2c0092b0 con 0x7fba3c0fc370 2024-07-03T15:47:00.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.602+0000 7fba424e1640 1 -- 172.21.15.28:0/1159168902 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba3c110720 con 0x7fba3c0fc370 2024-07-03T15:47:00.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.602+0000 7fba38ff9640 1 -- 172.21.15.28:0/1159168902 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba2c0338a0 con 0x7fba3c0fc370 2024-07-03T15:47:00.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.602+0000 7fba424e1640 1 -- 172.21.15.28:0/1159168902 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba3c110c60 con 0x7fba3c0fc370 2024-07-03T15:47:00.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.603+0000 7fba424e1640 1 -- 172.21.15.28:0/1159168902 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba3c0fb7d0 con 0x7fba3c0fc370 2024-07-03T15:47:00.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.604+0000 7fba38ff9640 1 -- 172.21.15.28:0/1159168902 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 36) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fba2c030020 con 0x7fba3c0fc370 2024-07-03T15:47:00.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.604+0000 7fba38ff9640 1 --2- 172.21.15.28:0/1159168902 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fba2406e1e0 0x7fba24070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:47:00.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.605+0000 7fba38ff9640 1 -- 172.21.15.28:0/1159168902 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fba2c0ada90 con 0x7fba3c0fc370 2024-07-03T15:47:00.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.605+0000 7fba3b7fe640 1 --2- 172.21.15.28:0/1159168902 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fba2406e1e0 0x7fba24070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:47:00.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.605+0000 7fba3b7fe640 1 --2- 172.21.15.28:0/1159168902 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fba2406e1e0 0x7fba24070690 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fba3c10eef0 tx=0x7fba3000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:47:00.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.609+0000 7fba38ff9640 1 -- 172.21.15.28:0/1159168902 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fba2c076d90 con 0x7fba3c0fc370 2024-07-03T15:47:00.686 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:00 smithi148 ceph-mon[80490]: Reconfiguring osd.4 (monmap changed)... 2024-07-03T15:47:00.686 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:00 smithi148 ceph-mon[80490]: Reconfiguring daemon osd.4 on smithi148 2024-07-03T15:47:00.687 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:00 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:00.687 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:00 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:00.687 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:00 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-07-03T15:47:00.687 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:00 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:47:00.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.810+0000 7fba424e1640 1 -- 172.21.15.28:0/1159168902 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fba3c104fd0 con 0x7fba2406e1e0 2024-07-03T15:47:00.816 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:00 smithi028 ceph-mon[92858]: Reconfiguring osd.4 (monmap changed)... 2024-07-03T15:47:00.816 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:00 smithi028 ceph-mon[92858]: Reconfiguring daemon osd.4 on smithi148 2024-07-03T15:47:00.817 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:00 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:00.817 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:00 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:00.817 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:00 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-07-03T15:47:00.817 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:00 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:47:00.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.812+0000 7fba38ff9640 1 -- 172.21.15.28:0/1159168902 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fba3c104fd0 con 0x7fba2406e1e0 2024-07-03T15:47:00.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.814+0000 7fba424e1640 1 -- 172.21.15.28:0/1159168902 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fba2406e1e0 msgr2=0x7fba24070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:00.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.814+0000 7fba424e1640 1 --2- 172.21.15.28:0/1159168902 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fba2406e1e0 0x7fba24070690 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fba3c10eef0 tx=0x7fba3000b040 comp rx=0 tx=0).stop 2024-07-03T15:47:00.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.814+0000 7fba424e1640 1 -- 172.21.15.28:0/1159168902 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba3c0fc370 msgr2=0x7fba3c10f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:00.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.814+0000 7fba424e1640 1 --2- 172.21.15.28:0/1159168902 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba3c0fc370 0x7fba3c10f140 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fba2c0048a0 tx=0x7fba2c009da0 comp rx=0 tx=0).stop 2024-07-03T15:47:00.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.814+0000 7fba424e1640 1 -- 172.21.15.28:0/1159168902 shutdown_connections 2024-07-03T15:47:00.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.814+0000 7fba424e1640 1 --2- 172.21.15.28:0/1159168902 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fba2406e1e0 0x7fba24070690 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:00.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.814+0000 7fba424e1640 1 --2- 172.21.15.28:0/1159168902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba3c0fd640 0x7fba3c10f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:00.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.814+0000 7fba424e1640 1 --2- 172.21.15.28:0/1159168902 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba3c0fc370 0x7fba3c10f140 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:00.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.814+0000 7fba424e1640 1 -- 172.21.15.28:0/1159168902 >> 172.21.15.28:0/1159168902 conn(0x7fba3c0f7b50 msgr2=0x7fba3c0f9200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:47:00.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.814+0000 7fba424e1640 1 -- 172.21.15.28:0/1159168902 shutdown_connections 2024-07-03T15:47:00.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.814+0000 7fba424e1640 1 -- 172.21.15.28:0/1159168902 wait complete. 2024-07-03T15:47:00.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.896+0000 7f0d407e9640 1 -- 172.21.15.28:0/4025574140 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d380ff530 msgr2=0x7f0d38101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:00.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.896+0000 7f0d407e9640 1 --2- 172.21.15.28:0/4025574140 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d380ff530 0x7f0d38101910 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c0048d0 tx=0x7f0d2c009da0 comp rx=0 tx=0).stop 2024-07-03T15:47:00.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.897+0000 7f0d407e9640 1 -- 172.21.15.28:0/4025574140 shutdown_connections 2024-07-03T15:47:00.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.897+0000 7f0d407e9640 1 --2- 172.21.15.28:0/4025574140 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d380ff530 0x7f0d38101910 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:00.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.897+0000 7f0d407e9640 1 --2- 172.21.15.28:0/4025574140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d380659b0 0x7f0d380feff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:00.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.897+0000 7f0d407e9640 1 -- 172.21.15.28:0/4025574140 >> 172.21.15.28:0/4025574140 conn(0x7f0d380f6a20 msgr2=0x7f0d380f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:47:00.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.897+0000 7f0d407e9640 1 -- 172.21.15.28:0/4025574140 shutdown_connections 2024-07-03T15:47:00.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.897+0000 7f0d407e9640 1 -- 172.21.15.28:0/4025574140 wait complete. 2024-07-03T15:47:00.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.897+0000 7f0d407e9640 1 Processor -- start 2024-07-03T15:47:00.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.897+0000 7f0d407e9640 1 -- start start 2024-07-03T15:47:00.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.897+0000 7f0d407e9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d380659b0 0x7f0d3810f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:47:00.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.897+0000 7f0d407e9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d380ff530 0x7f0d3810f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:47:00.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.897+0000 7f0d407e9640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3810fcf0 con 0x7f0d380659b0 2024-07-03T15:47:00.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.897+0000 7f0d407e9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d38110510 con 0x7f0d380ff530 2024-07-03T15:47:00.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.897+0000 7f0d3e585640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d380659b0 0x7f0d3810f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:47:00.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.897+0000 7f0d3e585640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d380659b0 0x7f0d3810f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56918/0 (socket says 172.21.15.28:56918) 2024-07-03T15:47:00.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.897+0000 7f0d3e585640 1 -- 172.21.15.28:0/2077405307 learned_addr learned my addr 172.21.15.28:0/2077405307 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:47:00.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.898+0000 7f0d3dd84640 1 --2- 172.21.15.28:0/2077405307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d380ff530 0x7f0d3810f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:47:00.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.898+0000 7f0d3e585640 1 -- 172.21.15.28:0/2077405307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d380ff530 msgr2=0x7f0d3810f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:00.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.898+0000 7f0d3e585640 1 --2- 172.21.15.28:0/2077405307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d380ff530 0x7f0d3810f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:00.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.898+0000 7f0d3e585640 1 -- 172.21.15.28:0/2077405307 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d2c005040 con 0x7f0d380659b0 2024-07-03T15:47:00.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.898+0000 7f0d3e585640 1 --2- 172.21.15.28:0/2077405307 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d380659b0 0x7f0d3810f160 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0d30002a80 tx=0x7f0d30004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:47:00.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.898+0000 7f0d36ffd640 1 -- 172.21.15.28:0/2077405307 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d30007c30 con 0x7f0d380659b0 2024-07-03T15:47:00.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.898+0000 7f0d407e9640 1 -- 172.21.15.28:0/2077405307 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d381107a0 con 0x7f0d380659b0 2024-07-03T15:47:00.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.898+0000 7f0d36ffd640 1 -- 172.21.15.28:0/2077405307 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d3000f040 con 0x7f0d380659b0 2024-07-03T15:47:00.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.898+0000 7f0d36ffd640 1 -- 172.21.15.28:0/2077405307 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d3000dba0 con 0x7f0d380659b0 2024-07-03T15:47:00.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.898+0000 7f0d407e9640 1 -- 172.21.15.28:0/2077405307 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d38110d10 con 0x7f0d380659b0 2024-07-03T15:47:00.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.899+0000 7f0d407e9640 1 -- 172.21.15.28:0/2077405307 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d381056b0 con 0x7f0d380659b0 2024-07-03T15:47:00.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.900+0000 7f0d36ffd640 1 -- 172.21.15.28:0/2077405307 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 36) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f0d3000dd00 con 0x7f0d380659b0 2024-07-03T15:47:00.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.900+0000 7f0d36ffd640 1 --2- 172.21.15.28:0/2077405307 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f0d2006e1e0 0x7f0d20070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:47:00.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.900+0000 7f0d36ffd640 1 -- 172.21.15.28:0/2077405307 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f0d3008dc40 con 0x7f0d380659b0 2024-07-03T15:47:00.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.900+0000 7f0d3dd84640 1 --2- 172.21.15.28:0/2077405307 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f0d2006e1e0 0x7f0d20070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:47:00.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.901+0000 7f0d3dd84640 1 --2- 172.21.15.28:0/2077405307 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f0d2006e1e0 0x7f0d20070690 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c000c00 tx=0x7f0d2c035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:47:00.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:00.905+0000 7f0d36ffd640 1 -- 172.21.15.28:0/2077405307 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0d30056da0 con 0x7f0d380659b0 2024-07-03T15:47:01.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.099+0000 7f0d407e9640 1 -- 172.21.15.28:0/2077405307 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d38111070 con 0x7f0d2006e1e0 2024-07-03T15:47:01.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.110+0000 7f0d36ffd640 1 -- 172.21.15.28:0/2077405307 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f0d38111070 con 0x7f0d2006e1e0 2024-07-03T15:47:01.115 INFO:teuthology.orchestra.run.smithi028.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-03T15:47:01.115 INFO:teuthology.orchestra.run.smithi028.stdout:alertmanager.smithi028 smithi028 *:9093,9094 running (2m) 33s ago 10m 25.7M - 0.25.0 c8568f914cd2 1425315a0de9 2024-07-03T15:47:01.115 INFO:teuthology.orchestra.run.smithi028.stdout:crash.smithi028 smithi028 running (10m) 33s ago 10m 6899k - 17.2.7-1047-gd4b063ab 4ca399d9e65d 535660e448f9 2024-07-03T15:47:01.115 INFO:teuthology.orchestra.run.smithi028.stdout:crash.smithi148 smithi148 running (9m) 23s ago 9m 6899k - 17.2.7-1047-gd4b063ab 4ca399d9e65d 4bb7675895e7 2024-07-03T15:47:01.115 INFO:teuthology.orchestra.run.smithi028.stdout:grafana.smithi028 smithi028 *:3000 running (2m) 33s ago 10m 73.4M - 9.4.7 954c08fa6188 f9f17c92cfea 2024-07-03T15:47:01.115 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi028.rhpvve smithi028 running (5m) 33s ago 5m 152M - 17.2.7-1047-gd4b063ab 4ca399d9e65d 3ed5abd02cfb 2024-07-03T15:47:01.115 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi028.wmbxew smithi028 running (5m) 33s ago 5m 110M - 17.2.7-1047-gd4b063ab 4ca399d9e65d b3bf736998f9 2024-07-03T15:47:01.115 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi148.szomfl smithi148 running (5m) 23s ago 5m 15.6M - 17.2.7-1047-gd4b063ab 4ca399d9e65d d94f5c95fa8d 2024-07-03T15:47:01.116 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi148.xgpuld smithi148 running (5m) 23s ago 5m 13.0M - 17.2.7-1047-gd4b063ab 4ca399d9e65d fc70360a8de0 2024-07-03T15:47:01.116 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.smithi028.vtqrtz smithi028 *:8443,9283,8765 running (112s) 33s ago 11m 471M - 19.0.0-2550-gc00506dc 9d49405603e8 14d0363c5b77 2024-07-03T15:47:01.116 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.smithi148.rjzsar smithi148 *:8443,9283,8765 running (58s) 23s ago 9m 437M - 19.0.0-2550-gc00506dc 9d49405603e8 c0193aa8a5dd 2024-07-03T15:47:01.116 INFO:teuthology.orchestra.run.smithi028.stdout:mon.smithi028 smithi028 running (46s) 33s ago 12m 28.7M 2048M 19.0.0-2550-gc00506dc 9d49405603e8 5093f7bf47d7 2024-07-03T15:47:01.116 INFO:teuthology.orchestra.run.smithi028.stdout:mon.smithi148 smithi148 running (26s) 23s ago 9m 22.3M 2048M 19.0.0-2550-gc00506dc 9d49405603e8 d338d2c648da 2024-07-03T15:47:01.116 INFO:teuthology.orchestra.run.smithi028.stdout:node-exporter.smithi028 smithi028 *:9100 running (2m) 33s ago 10m 15.3M - 1.5.0 0da6a335fe13 91ca1f49de0e 2024-07-03T15:47:01.116 INFO:teuthology.orchestra.run.smithi028.stdout:node-exporter.smithi148 smithi148 *:9100 running (2m) 23s ago 9m 15.9M - 1.5.0 0da6a335fe13 1eaaf7b1b561 2024-07-03T15:47:01.116 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0 smithi028 running (8m) 33s ago 8m 255M 1558M 17.2.7-1047-gd4b063ab 4ca399d9e65d b18089a439ba 2024-07-03T15:47:01.116 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1 smithi028 running (8m) 33s ago 8m 254M 1558M 17.2.7-1047-gd4b063ab 4ca399d9e65d 87d415dde231 2024-07-03T15:47:01.116 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2 smithi028 running (8m) 33s ago 8m 218M 1558M 17.2.7-1047-gd4b063ab 4ca399d9e65d b2cbb312212d 2024-07-03T15:47:01.116 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3 smithi148 running (7m) 23s ago 7m 278M 2581M 17.2.7-1047-gd4b063ab 4ca399d9e65d 5be69a27df31 2024-07-03T15:47:01.116 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4 smithi148 running (7m) 23s ago 7m 250M 2581M 17.2.7-1047-gd4b063ab 4ca399d9e65d 54f7c26c77ca 2024-07-03T15:47:01.116 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5 smithi148 running (6m) 23s ago 6m 238M 2581M 17.2.7-1047-gd4b063ab 4ca399d9e65d d185c6905384 2024-07-03T15:47:01.116 INFO:teuthology.orchestra.run.smithi028.stdout:prometheus.smithi028 smithi028 *:9095 running (62s) 33s ago 10m 46.8M - 2.43.0 a07b618ecd1d 2b0eb2a95afd 2024-07-03T15:47:01.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.112+0000 7f0d407e9640 1 -- 172.21.15.28:0/2077405307 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f0d2006e1e0 msgr2=0x7f0d20070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:01.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.112+0000 7f0d407e9640 1 --2- 172.21.15.28:0/2077405307 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f0d2006e1e0 0x7f0d20070690 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c000c00 tx=0x7f0d2c035040 comp rx=0 tx=0).stop 2024-07-03T15:47:01.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.112+0000 7f0d407e9640 1 -- 172.21.15.28:0/2077405307 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d380659b0 msgr2=0x7f0d3810f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:01.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.112+0000 7f0d407e9640 1 --2- 172.21.15.28:0/2077405307 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d380659b0 0x7f0d3810f160 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0d30002a80 tx=0x7f0d30004570 comp rx=0 tx=0).stop 2024-07-03T15:47:01.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.112+0000 7f0d407e9640 1 -- 172.21.15.28:0/2077405307 shutdown_connections 2024-07-03T15:47:01.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.112+0000 7f0d407e9640 1 --2- 172.21.15.28:0/2077405307 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f0d2006e1e0 0x7f0d20070690 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:01.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.112+0000 7f0d407e9640 1 --2- 172.21.15.28:0/2077405307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d380ff530 0x7f0d3810f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:01.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.112+0000 7f0d407e9640 1 --2- 172.21.15.28:0/2077405307 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d380659b0 0x7f0d3810f160 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:01.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.112+0000 7f0d407e9640 1 -- 172.21.15.28:0/2077405307 >> 172.21.15.28:0/2077405307 conn(0x7f0d380f6a20 msgr2=0x7f0d380fb090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:47:01.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.112+0000 7f0d407e9640 1 -- 172.21.15.28:0/2077405307 shutdown_connections 2024-07-03T15:47:01.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.112+0000 7f0d407e9640 1 -- 172.21.15.28:0/2077405307 wait complete. 2024-07-03T15:47:01.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.193+0000 7fc812643640 1 -- 172.21.15.28:0/2985011724 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc80c06fe50 msgr2=0x7fc80c0702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:01.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.193+0000 7fc812643640 1 --2- 172.21.15.28:0/2985011724 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc80c06fe50 0x7fc80c0702c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc8000048d0 tx=0x7fc800009da0 comp rx=0 tx=0).stop 2024-07-03T15:47:01.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.193+0000 7fc812643640 1 -- 172.21.15.28:0/2985011724 shutdown_connections 2024-07-03T15:47:01.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.193+0000 7fc812643640 1 --2- 172.21.15.28:0/2985011724 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc80c06fe50 0x7fc80c0702c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:01.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.193+0000 7fc812643640 1 --2- 172.21.15.28:0/2985011724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc80c0713e0 0x7fc80c06f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:01.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.193+0000 7fc812643640 1 -- 172.21.15.28:0/2985011724 >> 172.21.15.28:0/2985011724 conn(0x7fc80c0f7b40 msgr2=0x7fc80c0f9f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:47:01.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.193+0000 7fc812643640 1 -- 172.21.15.28:0/2985011724 shutdown_connections 2024-07-03T15:47:01.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.193+0000 7fc812643640 1 -- 172.21.15.28:0/2985011724 wait complete. 2024-07-03T15:47:01.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.194+0000 7fc812643640 1 Processor -- start 2024-07-03T15:47:01.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.194+0000 7fc812643640 1 -- start start 2024-07-03T15:47:01.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.194+0000 7fc812643640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc80c06fe50 0x7fc80c113570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:47:01.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.194+0000 7fc812643640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc80c0713e0 0x7fc80c113ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:47:01.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.194+0000 7fc812643640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80c114100 con 0x7fc80c0713e0 2024-07-03T15:47:01.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.194+0000 7fc812643640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80c114920 con 0x7fc80c06fe50 2024-07-03T15:47:01.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.194+0000 7fc80b7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc80c0713e0 0x7fc80c113ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:47:01.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.194+0000 7fc80b7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc80c0713e0 0x7fc80c113ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56934/0 (socket says 172.21.15.28:56934) 2024-07-03T15:47:01.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.194+0000 7fc80b7fe640 1 -- 172.21.15.28:0/958959493 learned_addr learned my addr 172.21.15.28:0/958959493 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:47:01.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.195+0000 7fc80bfff640 1 --2- 172.21.15.28:0/958959493 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc80c06fe50 0x7fc80c113570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:47:01.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.195+0000 7fc80b7fe640 1 -- 172.21.15.28:0/958959493 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc80c06fe50 msgr2=0x7fc80c113570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:01.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.195+0000 7fc80b7fe640 1 --2- 172.21.15.28:0/958959493 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc80c06fe50 0x7fc80c113570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:01.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.195+0000 7fc80b7fe640 1 -- 172.21.15.28:0/958959493 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc800005040 con 0x7fc80c0713e0 2024-07-03T15:47:01.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.195+0000 7fc80b7fe640 1 --2- 172.21.15.28:0/958959493 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc80c0713e0 0x7fc80c113ab0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc800003080 tx=0x7fc800009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:47:01.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.195+0000 7fc808ff9640 1 -- 172.21.15.28:0/958959493 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc800038070 con 0x7fc80c0713e0 2024-07-03T15:47:01.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.195+0000 7fc812643640 1 -- 172.21.15.28:0/958959493 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc80c114b50 con 0x7fc80c0713e0 2024-07-03T15:47:01.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.195+0000 7fc808ff9640 1 -- 172.21.15.28:0/958959493 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc800032970 con 0x7fc80c0713e0 2024-07-03T15:47:01.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.195+0000 7fc808ff9640 1 -- 172.21.15.28:0/958959493 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc800033900 con 0x7fc80c0713e0 2024-07-03T15:47:01.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.195+0000 7fc812643640 1 -- 172.21.15.28:0/958959493 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc80c115090 con 0x7fc80c0713e0 2024-07-03T15:47:01.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.196+0000 7fc812643640 1 -- 172.21.15.28:0/958959493 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc80c0fb7e0 con 0x7fc80c0713e0 2024-07-03T15:47:01.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.197+0000 7fc808ff9640 1 -- 172.21.15.28:0/958959493 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 36) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fc800030020 con 0x7fc80c0713e0 2024-07-03T15:47:01.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.197+0000 7fc808ff9640 1 --2- 172.21.15.28:0/958959493 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc7f406e1e0 0x7fc7f4070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:47:01.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.198+0000 7fc808ff9640 1 -- 172.21.15.28:0/958959493 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fc8000addd0 con 0x7fc80c0713e0 2024-07-03T15:47:01.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.198+0000 7fc80bfff640 1 --2- 172.21.15.28:0/958959493 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc7f406e1e0 0x7fc7f4070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:47:01.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.198+0000 7fc80bfff640 1 --2- 172.21.15.28:0/958959493 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc7f406e1e0 0x7fc7f4070690 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc80c071140 tx=0x7fc7fc009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:47:01.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.202+0000 7fc808ff9640 1 -- 172.21.15.28:0/958959493 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc800077040 con 0x7fc80c0713e0 2024-07-03T15:47:01.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.463+0000 7fc812643640 1 -- 172.21.15.28:0/958959493 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc80c10b4f0 con 0x7fc80c0713e0 2024-07-03T15:47:01.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.464+0000 7fc808ff9640 1 -- 172.21.15.28:0/958959493 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+736 (secure 0 0 0) 0x7fc80007a700 con 0x7fc80c0713e0 2024-07-03T15:47:01.469 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-07-03T15:47:01.469 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": { 2024-07-03T15:47:01.469 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 2 2024-07-03T15:47:01.470 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:47:01.470 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": { 2024-07-03T15:47:01.470 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 2 2024-07-03T15:47:01.470 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:47:01.470 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": { 2024-07-03T15:47:01.470 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 6 2024-07-03T15:47:01.470 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:47:01.470 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": { 2024-07-03T15:47:01.470 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 4 2024-07-03T15:47:01.470 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:47:01.470 INFO:teuthology.orchestra.run.smithi028.stdout: "overall": { 2024-07-03T15:47:01.470 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 10, 2024-07-03T15:47:01.471 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 4 2024-07-03T15:47:01.471 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-07-03T15:47:01.471 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-07-03T15:47:01.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.466+0000 7fc812643640 1 -- 172.21.15.28:0/958959493 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc7f406e1e0 msgr2=0x7fc7f4070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:01.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.466+0000 7fc812643640 1 --2- 172.21.15.28:0/958959493 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc7f406e1e0 0x7fc7f4070690 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc80c071140 tx=0x7fc7fc009040 comp rx=0 tx=0).stop 2024-07-03T15:47:01.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.466+0000 7fc812643640 1 -- 172.21.15.28:0/958959493 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc80c0713e0 msgr2=0x7fc80c113ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:01.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.466+0000 7fc812643640 1 --2- 172.21.15.28:0/958959493 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc80c0713e0 0x7fc80c113ab0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc800003080 tx=0x7fc800009da0 comp rx=0 tx=0).stop 2024-07-03T15:47:01.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.466+0000 7fc812643640 1 -- 172.21.15.28:0/958959493 shutdown_connections 2024-07-03T15:47:01.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.466+0000 7fc812643640 1 --2- 172.21.15.28:0/958959493 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc7f406e1e0 0x7fc7f4070690 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:01.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.466+0000 7fc812643640 1 --2- 172.21.15.28:0/958959493 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc80c0713e0 0x7fc80c113ab0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:01.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.466+0000 7fc812643640 1 --2- 172.21.15.28:0/958959493 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc80c06fe50 0x7fc80c113570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:01.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.466+0000 7fc812643640 1 -- 172.21.15.28:0/958959493 >> 172.21.15.28:0/958959493 conn(0x7fc80c0f7b40 msgr2=0x7fc80c0f9300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:47:01.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.466+0000 7fc812643640 1 -- 172.21.15.28:0/958959493 shutdown_connections 2024-07-03T15:47:01.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.466+0000 7fc812643640 1 -- 172.21.15.28:0/958959493 wait complete. 2024-07-03T15:47:01.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.548+0000 7f2632693640 1 -- 172.21.15.28:0/2441065466 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f262c0fe300 msgr2=0x7f262c0fa460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:01.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.548+0000 7f2632693640 1 --2- 172.21.15.28:0/2441065466 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f262c0fe300 0x7f262c0fa460 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f26200048d0 tx=0x7f2620009da0 comp rx=0 tx=0).stop 2024-07-03T15:47:01.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.548+0000 7f2632693640 1 -- 172.21.15.28:0/2441065466 shutdown_connections 2024-07-03T15:47:01.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.548+0000 7f2632693640 1 --2- 172.21.15.28:0/2441065466 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f262c0fe300 0x7f262c0fa460 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:01.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.548+0000 7f2632693640 1 --2- 172.21.15.28:0/2441065466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f262c0fd9b0 0x7f262c0fddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:01.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.548+0000 7f2632693640 1 -- 172.21.15.28:0/2441065466 >> 172.21.15.28:0/2441065466 conn(0x7f262c0f61b0 msgr2=0x7f262c0f85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:47:01.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.548+0000 7f2632693640 1 -- 172.21.15.28:0/2441065466 shutdown_connections 2024-07-03T15:47:01.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.548+0000 7f2632693640 1 -- 172.21.15.28:0/2441065466 wait complete. 2024-07-03T15:47:01.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.549+0000 7f2632693640 1 Processor -- start 2024-07-03T15:47:01.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.549+0000 7f2632693640 1 -- start start 2024-07-03T15:47:01.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.549+0000 7f2632693640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f262c0fd9b0 0x7f262c0fd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:47:01.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.549+0000 7f2632693640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f262c0fe300 0x7f262c0fb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:47:01.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.549+0000 7f2632693640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262c0fbc20 con 0x7f262c0fe300 2024-07-03T15:47:01.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.549+0000 7f2632693640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262c0fbd60 con 0x7f262c0fd9b0 2024-07-03T15:47:01.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.549+0000 7f262b7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f262c0fe300 0x7f262c0fb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:47:01.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.549+0000 7f262bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f262c0fd9b0 0x7f262c0fd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:47:01.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.549+0000 7f262b7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f262c0fe300 0x7f262c0fb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56960/0 (socket says 172.21.15.28:56960) 2024-07-03T15:47:01.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.549+0000 7f262b7fe640 1 -- 172.21.15.28:0/3059924289 learned_addr learned my addr 172.21.15.28:0/3059924289 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:47:01.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.550+0000 7f262b7fe640 1 -- 172.21.15.28:0/3059924289 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f262c0fd9b0 msgr2=0x7f262c0fd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:01.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.550+0000 7f262b7fe640 1 --2- 172.21.15.28:0/3059924289 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f262c0fd9b0 0x7f262c0fd1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:01.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.550+0000 7f262b7fe640 1 -- 172.21.15.28:0/3059924289 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2620005040 con 0x7f262c0fe300 2024-07-03T15:47:01.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.550+0000 7f262b7fe640 1 --2- 172.21.15.28:0/3059924289 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f262c0fe300 0x7f262c0fb690 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2620000c00 tx=0x7f2620009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:47:01.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.550+0000 7f2628ff9640 1 -- 172.21.15.28:0/3059924289 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2620038070 con 0x7f262c0fe300 2024-07-03T15:47:01.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.550+0000 7f2628ff9640 1 -- 172.21.15.28:0/3059924289 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2620032970 con 0x7f262c0fe300 2024-07-03T15:47:01.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.550+0000 7f2632693640 1 -- 172.21.15.28:0/3059924289 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f262c0fbfe0 con 0x7f262c0fe300 2024-07-03T15:47:01.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.550+0000 7f2628ff9640 1 -- 172.21.15.28:0/3059924289 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26200338e0 con 0x7f262c0fe300 2024-07-03T15:47:01.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.550+0000 7f2632693640 1 -- 172.21.15.28:0/3059924289 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f262c0fc4a0 con 0x7f262c0fe300 2024-07-03T15:47:01.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.551+0000 7f2632693640 1 -- 172.21.15.28:0/3059924289 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f262c0f9e50 con 0x7f262c0fe300 2024-07-03T15:47:01.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.552+0000 7f2628ff9640 1 -- 172.21.15.28:0/3059924289 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 36) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f2620030020 con 0x7f262c0fe300 2024-07-03T15:47:01.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.552+0000 7f2628ff9640 1 --2- 172.21.15.28:0/3059924289 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f261406e1e0 0x7f2614070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:47:01.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.553+0000 7f2628ff9640 1 -- 172.21.15.28:0/3059924289 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f26200ad9f0 con 0x7f262c0fe300 2024-07-03T15:47:01.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.553+0000 7f262bfff640 1 --2- 172.21.15.28:0/3059924289 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f261406e1e0 0x7f2614070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:47:01.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.553+0000 7f262bfff640 1 --2- 172.21.15.28:0/3059924289 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f261406e1e0 0x7f2614070690 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f262c0fcf50 tx=0x7f261c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:47:01.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.557+0000 7f2628ff9640 1 -- 172.21.15.28:0/3059924289 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f26200afe60 con 0x7f262c0fe300 2024-07-03T15:47:01.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:01 smithi148 ceph-mon[80490]: pgmap v22: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:47:01.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:01 smithi148 ceph-mon[80490]: Reconfiguring osd.5 (monmap changed)... 2024-07-03T15:47:01.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:01 smithi148 ceph-mon[80490]: Reconfiguring daemon osd.5 on smithi148 2024-07-03T15:47:01.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:01 smithi148 ceph-mon[80490]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:47:01.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:01 smithi148 ceph-mon[80490]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:47:01.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.783+0000 7f2632693640 1 -- 172.21.15.28:0/3059924289 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f262c04b530 con 0x7f262c0fe300 2024-07-03T15:47:01.788 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:01 smithi028 ceph-mon[92858]: pgmap v22: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:47:01.788 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:01 smithi028 ceph-mon[92858]: Reconfiguring osd.5 (monmap changed)... 2024-07-03T15:47:01.789 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:01 smithi028 ceph-mon[92858]: Reconfiguring daemon osd.5 on smithi148 2024-07-03T15:47:01.789 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:01 smithi028 ceph-mon[92858]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:47:01.789 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:01 smithi028 ceph-mon[92858]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:47:01.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.783+0000 7f2628ff9640 1 -- 172.21.15.28:0/3059924289 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1908 (secure 0 0 0) 0x7f2620076d90 con 0x7f262c0fe300 2024-07-03T15:47:01.789 INFO:teuthology.orchestra.run.smithi028.stdout:e17 2024-07-03T15:47:01.789 INFO:teuthology.orchestra.run.smithi028.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-03T15:47:01.790 INFO:teuthology.orchestra.run.smithi028.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:47:01.790 INFO:teuthology.orchestra.run.smithi028.stdout:legacy client fscid: 1 2024-07-03T15:47:01.790 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:47:01.790 INFO:teuthology.orchestra.run.smithi028.stdout:Filesystem 'cephfs' (1) 2024-07-03T15:47:01.790 INFO:teuthology.orchestra.run.smithi028.stdout:fs_name cephfs 2024-07-03T15:47:01.790 INFO:teuthology.orchestra.run.smithi028.stdout:epoch 16 2024-07-03T15:47:01.790 INFO:teuthology.orchestra.run.smithi028.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-03T15:47:01.790 INFO:teuthology.orchestra.run.smithi028.stdout:created 2024-07-03T15:41:40.656298+0000 2024-07-03T15:47:01.790 INFO:teuthology.orchestra.run.smithi028.stdout:modified 2024-07-03T15:41:56.454984+0000 2024-07-03T15:47:01.791 INFO:teuthology.orchestra.run.smithi028.stdout:tableserver 0 2024-07-03T15:47:01.791 INFO:teuthology.orchestra.run.smithi028.stdout:root 0 2024-07-03T15:47:01.791 INFO:teuthology.orchestra.run.smithi028.stdout:session_timeout 60 2024-07-03T15:47:01.791 INFO:teuthology.orchestra.run.smithi028.stdout:session_autoclose 300 2024-07-03T15:47:01.791 INFO:teuthology.orchestra.run.smithi028.stdout:max_file_size 1099511627776 2024-07-03T15:47:01.791 INFO:teuthology.orchestra.run.smithi028.stdout:max_xattr_size 65536 2024-07-03T15:47:01.791 INFO:teuthology.orchestra.run.smithi028.stdout:required_client_features {} 2024-07-03T15:47:01.791 INFO:teuthology.orchestra.run.smithi028.stdout:last_failure 0 2024-07-03T15:47:01.791 INFO:teuthology.orchestra.run.smithi028.stdout:last_failure_osd_epoch 0 2024-07-03T15:47:01.791 INFO:teuthology.orchestra.run.smithi028.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:47:01.791 INFO:teuthology.orchestra.run.smithi028.stdout:max_mds 1 2024-07-03T15:47:01.791 INFO:teuthology.orchestra.run.smithi028.stdout:in 0 2024-07-03T15:47:01.792 INFO:teuthology.orchestra.run.smithi028.stdout:up {0=14450} 2024-07-03T15:47:01.792 INFO:teuthology.orchestra.run.smithi028.stdout:failed 2024-07-03T15:47:01.792 INFO:teuthology.orchestra.run.smithi028.stdout:damaged 2024-07-03T15:47:01.792 INFO:teuthology.orchestra.run.smithi028.stdout:stopped 2024-07-03T15:47:01.792 INFO:teuthology.orchestra.run.smithi028.stdout:data_pools [3] 2024-07-03T15:47:01.792 INFO:teuthology.orchestra.run.smithi028.stdout:metadata_pool 2 2024-07-03T15:47:01.792 INFO:teuthology.orchestra.run.smithi028.stdout:inline_data disabled 2024-07-03T15:47:01.792 INFO:teuthology.orchestra.run.smithi028.stdout:balancer 2024-07-03T15:47:01.792 INFO:teuthology.orchestra.run.smithi028.stdout:bal_rank_mask -1 2024-07-03T15:47:01.792 INFO:teuthology.orchestra.run.smithi028.stdout:standby_count_wanted 1 2024-07-03T15:47:01.792 INFO:teuthology.orchestra.run.smithi028.stdout:qdb_cluster leader: 0 members: 2024-07-03T15:47:01.792 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi028.rhpvve{0:14450} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.28:6826/1613936042,v1:172.21.15.28:6827/1613936042] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:47:01.792 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi028.wmbxew{0:14460} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.28:6828/1730336821,v1:172.21.15.28:6829/1730336821] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:47:01.792 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:47:01.792 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:47:01.793 INFO:teuthology.orchestra.run.smithi028.stdout:Standby daemons: 2024-07-03T15:47:01.793 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:47:01.793 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi148.szomfl{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/4171231858,v1:172.21.15.148:6827/4171231858] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:47:01.793 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi148.xgpuld{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/2347536398,v1:172.21.15.148:6825/2347536398] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:47:01.794 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 17 2024-07-03T15:47:01.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.785+0000 7f2632693640 1 -- 172.21.15.28:0/3059924289 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f261406e1e0 msgr2=0x7f2614070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:01.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.785+0000 7f2632693640 1 --2- 172.21.15.28:0/3059924289 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f261406e1e0 0x7f2614070690 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f262c0fcf50 tx=0x7f261c00b040 comp rx=0 tx=0).stop 2024-07-03T15:47:01.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.785+0000 7f2632693640 1 -- 172.21.15.28:0/3059924289 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f262c0fe300 msgr2=0x7f262c0fb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:01.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.785+0000 7f2632693640 1 --2- 172.21.15.28:0/3059924289 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f262c0fe300 0x7f262c0fb690 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2620000c00 tx=0x7f2620009da0 comp rx=0 tx=0).stop 2024-07-03T15:47:01.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.785+0000 7f2632693640 1 -- 172.21.15.28:0/3059924289 shutdown_connections 2024-07-03T15:47:01.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.785+0000 7f2632693640 1 --2- 172.21.15.28:0/3059924289 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f261406e1e0 0x7f2614070690 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:01.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.785+0000 7f2632693640 1 --2- 172.21.15.28:0/3059924289 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f262c0fe300 0x7f262c0fb690 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:01.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.785+0000 7f2632693640 1 --2- 172.21.15.28:0/3059924289 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f262c0fd9b0 0x7f262c0fd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:01.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.785+0000 7f2632693640 1 -- 172.21.15.28:0/3059924289 >> 172.21.15.28:0/3059924289 conn(0x7f262c0f61b0 msgr2=0x7f262c0f7830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:47:01.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.785+0000 7f2632693640 1 -- 172.21.15.28:0/3059924289 shutdown_connections 2024-07-03T15:47:01.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.785+0000 7f2632693640 1 -- 172.21.15.28:0/3059924289 wait complete. 2024-07-03T15:47:01.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.868+0000 7fc4ddd17640 1 -- 172.21.15.28:0/394983024 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4d80fd9b0 msgr2=0x7fc4d80fddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:01.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.868+0000 7fc4ddd17640 1 --2- 172.21.15.28:0/394983024 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4d80fd9b0 0x7fc4d80fddc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc4c80048d0 tx=0x7fc4c8009da0 comp rx=0 tx=0).stop 2024-07-03T15:47:01.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.868+0000 7fc4ddd17640 1 -- 172.21.15.28:0/394983024 shutdown_connections 2024-07-03T15:47:01.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.868+0000 7fc4ddd17640 1 --2- 172.21.15.28:0/394983024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4d80fe300 0x7fc4d80fa460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:01.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.868+0000 7fc4ddd17640 1 --2- 172.21.15.28:0/394983024 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4d80fd9b0 0x7fc4d80fddc0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:01.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.868+0000 7fc4ddd17640 1 -- 172.21.15.28:0/394983024 >> 172.21.15.28:0/394983024 conn(0x7fc4d80f61b0 msgr2=0x7fc4d80f85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:47:01.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.868+0000 7fc4ddd17640 1 -- 172.21.15.28:0/394983024 shutdown_connections 2024-07-03T15:47:01.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.868+0000 7fc4ddd17640 1 -- 172.21.15.28:0/394983024 wait complete. 2024-07-03T15:47:01.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.869+0000 7fc4ddd17640 1 Processor -- start 2024-07-03T15:47:01.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.869+0000 7fc4ddd17640 1 -- start start 2024-07-03T15:47:01.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.869+0000 7fc4ddd17640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4d80fd9b0 0x7fc4d810cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:47:01.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.869+0000 7fc4ddd17640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4d80fe300 0x7fc4d810d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:47:01.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.869+0000 7fc4ddd17640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d810db40 con 0x7fc4d80fd9b0 2024-07-03T15:47:01.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.869+0000 7fc4ddd17640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d810e360 con 0x7fc4d80fe300 2024-07-03T15:47:01.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.869+0000 7fc4d77fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4d80fd9b0 0x7fc4d810cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:47:01.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.869+0000 7fc4d77fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4d80fd9b0 0x7fc4d810cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56988/0 (socket says 172.21.15.28:56988) 2024-07-03T15:47:01.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.869+0000 7fc4d77fe640 1 -- 172.21.15.28:0/1278204703 learned_addr learned my addr 172.21.15.28:0/1278204703 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:47:01.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.870+0000 7fc4ceffd640 1 --2- 172.21.15.28:0/1278204703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4d80fe300 0x7fc4d810d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:47:01.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.870+0000 7fc4d77fe640 1 -- 172.21.15.28:0/1278204703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4d80fe300 msgr2=0x7fc4d810d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:01.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.870+0000 7fc4d77fe640 1 --2- 172.21.15.28:0/1278204703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4d80fe300 0x7fc4d810d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:01.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.870+0000 7fc4d77fe640 1 -- 172.21.15.28:0/1278204703 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c8005040 con 0x7fc4d80fd9b0 2024-07-03T15:47:01.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.870+0000 7fc4d77fe640 1 --2- 172.21.15.28:0/1278204703 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4d80fd9b0 0x7fc4d810cfb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8009f80 tx=0x7fc4c8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:47:01.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.870+0000 7fc4d4ff9640 1 -- 172.21.15.28:0/1278204703 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4c8038070 con 0x7fc4d80fd9b0 2024-07-03T15:47:01.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.870+0000 7fc4d4ff9640 1 -- 172.21.15.28:0/1278204703 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc4c8032970 con 0x7fc4d80fd9b0 2024-07-03T15:47:01.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.870+0000 7fc4ddd17640 1 -- 172.21.15.28:0/1278204703 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4d810e590 con 0x7fc4d80fd9b0 2024-07-03T15:47:01.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.870+0000 7fc4d4ff9640 1 -- 172.21.15.28:0/1278204703 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4c8033860 con 0x7fc4d80fd9b0 2024-07-03T15:47:01.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.870+0000 7fc4ddd17640 1 -- 172.21.15.28:0/1278204703 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4d810ead0 con 0x7fc4d80fd9b0 2024-07-03T15:47:01.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.871+0000 7fc4ddd17640 1 -- 172.21.15.28:0/1278204703 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4d80f9e50 con 0x7fc4d80fd9b0 2024-07-03T15:47:01.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.872+0000 7fc4d4ff9640 1 -- 172.21.15.28:0/1278204703 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 36) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fc4c8030020 con 0x7fc4d80fd9b0 2024-07-03T15:47:01.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.872+0000 7fc4d4ff9640 1 --2- 172.21.15.28:0/1278204703 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc4c406e1e0 0x7fc4c4070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:47:01.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.872+0000 7fc4d4ff9640 1 -- 172.21.15.28:0/1278204703 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fc4c80adb40 con 0x7fc4d80fd9b0 2024-07-03T15:47:01.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.872+0000 7fc4ceffd640 1 --2- 172.21.15.28:0/1278204703 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc4c406e1e0 0x7fc4c4070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:47:01.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.873+0000 7fc4ceffd640 1 --2- 172.21.15.28:0/1278204703 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc4c406e1e0 0x7fc4c4070690 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0007fd0 tx=0x7fc4c0009e90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:47:01.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:01.877+0000 7fc4d4ff9640 1 -- 172.21.15.28:0/1278204703 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc4c8076e30 con 0x7fc4d80fd9b0 2024-07-03T15:47:02.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.078+0000 7fc4ddd17640 1 -- 172.21.15.28:0/1278204703 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4d810ee50 con 0x7fc4c406e1e0 2024-07-03T15:47:02.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.079+0000 7fc4d4ff9640 1 -- 172.21.15.28:0/1278204703 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fc4d810ee50 con 0x7fc4c406e1e0 2024-07-03T15:47:02.085 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-07-03T15:47:02.085 INFO:teuthology.orchestra.run.smithi028.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2", 2024-07-03T15:47:02.085 INFO:teuthology.orchestra.run.smithi028.stdout: "in_progress": true, 2024-07-03T15:47:02.085 INFO:teuthology.orchestra.run.smithi028.stdout: "which": "Upgrading all daemon types on all hosts", 2024-07-03T15:47:02.085 INFO:teuthology.orchestra.run.smithi028.stdout: "services_complete": [ 2024-07-03T15:47:02.085 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr", 2024-07-03T15:47:02.085 INFO:teuthology.orchestra.run.smithi028.stdout: "mon" 2024-07-03T15:47:02.085 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-07-03T15:47:02.085 INFO:teuthology.orchestra.run.smithi028.stdout: "progress": "4/21 daemons upgraded", 2024-07-03T15:47:02.085 INFO:teuthology.orchestra.run.smithi028.stdout: "message": "Currently upgrading mon daemons", 2024-07-03T15:47:02.085 INFO:teuthology.orchestra.run.smithi028.stdout: "is_paused": false 2024-07-03T15:47:02.085 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-07-03T15:47:02.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.082+0000 7fc4ddd17640 1 -- 172.21.15.28:0/1278204703 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc4c406e1e0 msgr2=0x7fc4c4070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:02.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.082+0000 7fc4ddd17640 1 --2- 172.21.15.28:0/1278204703 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc4c406e1e0 0x7fc4c4070690 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0007fd0 tx=0x7fc4c0009e90 comp rx=0 tx=0).stop 2024-07-03T15:47:02.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.082+0000 7fc4ddd17640 1 -- 172.21.15.28:0/1278204703 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4d80fd9b0 msgr2=0x7fc4d810cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:02.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.082+0000 7fc4ddd17640 1 --2- 172.21.15.28:0/1278204703 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4d80fd9b0 0x7fc4d810cfb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8009f80 tx=0x7fc4c8009da0 comp rx=0 tx=0).stop 2024-07-03T15:47:02.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.082+0000 7fc4ddd17640 1 -- 172.21.15.28:0/1278204703 shutdown_connections 2024-07-03T15:47:02.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.083+0000 7fc4ddd17640 1 --2- 172.21.15.28:0/1278204703 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc4c406e1e0 0x7fc4c4070690 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:02.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.083+0000 7fc4ddd17640 1 --2- 172.21.15.28:0/1278204703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4d80fe300 0x7fc4d810d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:02.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.083+0000 7fc4ddd17640 1 --2- 172.21.15.28:0/1278204703 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4d80fd9b0 0x7fc4d810cfb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:02.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.083+0000 7fc4ddd17640 1 -- 172.21.15.28:0/1278204703 >> 172.21.15.28:0/1278204703 conn(0x7fc4d80f61b0 msgr2=0x7fc4d80f7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:47:02.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.083+0000 7fc4ddd17640 1 -- 172.21.15.28:0/1278204703 shutdown_connections 2024-07-03T15:47:02.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.083+0000 7fc4ddd17640 1 -- 172.21.15.28:0/1278204703 wait complete. 2024-07-03T15:47:02.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.165+0000 7f6b79762640 1 -- 172.21.15.28:0/1095922746 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6b740fb0d0 msgr2=0x7f6b740fd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:02.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.165+0000 7f6b79762640 1 --2- 172.21.15.28:0/1095922746 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6b740fb0d0 0x7f6b740fd950 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c0048d0 tx=0x7f6b5c009da0 comp rx=0 tx=0).stop 2024-07-03T15:47:02.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.165+0000 7f6b79762640 1 -- 172.21.15.28:0/1095922746 shutdown_connections 2024-07-03T15:47:02.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.165+0000 7f6b79762640 1 --2- 172.21.15.28:0/1095922746 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6b740fb0d0 0x7f6b740fd950 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:02.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.165+0000 7f6b79762640 1 --2- 172.21.15.28:0/1095922746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b740fa780 0x7f6b740fab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:02.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.165+0000 7f6b79762640 1 -- 172.21.15.28:0/1095922746 >> 172.21.15.28:0/1095922746 conn(0x7f6b740f61c0 msgr2=0x7f6b740f85f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:47:02.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.165+0000 7f6b79762640 1 -- 172.21.15.28:0/1095922746 shutdown_connections 2024-07-03T15:47:02.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.165+0000 7f6b79762640 1 -- 172.21.15.28:0/1095922746 wait complete. 2024-07-03T15:47:02.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.166+0000 7f6b79762640 1 Processor -- start 2024-07-03T15:47:02.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.166+0000 7f6b79762640 1 -- start start 2024-07-03T15:47:02.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.166+0000 7f6b79762640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b740fa780 0x7f6b7410d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:47:02.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.166+0000 7f6b79762640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6b740fb0d0 0x7f6b7410d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:47:02.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.166+0000 7f6b79762640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7410e5c0 con 0x7f6b740fb0d0 2024-07-03T15:47:02.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.166+0000 7f6b79762640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7410e730 con 0x7f6b740fa780 2024-07-03T15:47:02.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.166+0000 7f6b727fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6b740fb0d0 0x7f6b7410d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:47:02.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.166+0000 7f6b727fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6b740fb0d0 0x7f6b7410d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:57016/0 (socket says 172.21.15.28:57016) 2024-07-03T15:47:02.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.166+0000 7f6b727fc640 1 -- 172.21.15.28:0/2225098746 learned_addr learned my addr 172.21.15.28:0/2225098746 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:47:02.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.166+0000 7f6b72ffd640 1 --2- 172.21.15.28:0/2225098746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b740fa780 0x7f6b7410d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:47:02.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.166+0000 7f6b727fc640 1 -- 172.21.15.28:0/2225098746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b740fa780 msgr2=0x7f6b7410d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:02.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.166+0000 7f6b727fc640 1 --2- 172.21.15.28:0/2225098746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b740fa780 0x7f6b7410d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:02.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.166+0000 7f6b727fc640 1 -- 172.21.15.28:0/2225098746 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b5c005040 con 0x7f6b740fb0d0 2024-07-03T15:47:02.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.166+0000 7f6b72ffd640 1 --2- 172.21.15.28:0/2225098746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b740fa780 0x7f6b7410d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-03T15:47:02.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.166+0000 7f6b727fc640 1 --2- 172.21.15.28:0/2225098746 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6b740fb0d0 0x7f6b7410d750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c0048a0 tx=0x7f6b5c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:47:02.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.167+0000 7f6b6b7fe640 1 -- 172.21.15.28:0/2225098746 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b5c038070 con 0x7f6b740fb0d0 2024-07-03T15:47:02.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.167+0000 7f6b6b7fe640 1 -- 172.21.15.28:0/2225098746 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6b5c032970 con 0x7f6b740fb0d0 2024-07-03T15:47:02.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.167+0000 7f6b79762640 1 -- 172.21.15.28:0/2225098746 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b7410e9b0 con 0x7f6b740fb0d0 2024-07-03T15:47:02.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.167+0000 7f6b6b7fe640 1 -- 172.21.15.28:0/2225098746 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b5c03d8c0 con 0x7f6b740fb0d0 2024-07-03T15:47:02.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.167+0000 7f6b79762640 1 -- 172.21.15.28:0/2225098746 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b7410eed0 con 0x7f6b740fb0d0 2024-07-03T15:47:02.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.168+0000 7f6b79762640 1 -- 172.21.15.28:0/2225098746 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b740f9e60 con 0x7f6b740fb0d0 2024-07-03T15:47:02.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.168+0000 7f6b6b7fe640 1 -- 172.21.15.28:0/2225098746 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 36) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f6b5c030020 con 0x7f6b740fb0d0 2024-07-03T15:47:02.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.169+0000 7f6b6b7fe640 1 --2- 172.21.15.28:0/2225098746 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f6b6006e1e0 0x7f6b60070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:47:02.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.169+0000 7f6b6b7fe640 1 -- 172.21.15.28:0/2225098746 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f6b5c0ae4a0 con 0x7f6b740fb0d0 2024-07-03T15:47:02.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.169+0000 7f6b72ffd640 1 --2- 172.21.15.28:0/2225098746 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f6b6006e1e0 0x7f6b60070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:47:02.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.169+0000 7f6b72ffd640 1 --2- 172.21.15.28:0/2225098746 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f6b6006e1e0 0x7f6b60070690 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6b64009a80 tx=0x7f6b64008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:47:02.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.173+0000 7f6b6b7fe640 1 -- 172.21.15.28:0/2225098746 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f6b5c0777d0 con 0x7f6b740fb0d0 2024-07-03T15:47:02.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.431+0000 7f6b79762640 1 -- 172.21.15.28:0/2225098746 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f6b74065680 con 0x7f6b740fb0d0 2024-07-03T15:47:02.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.431+0000 7f6b6b7fe640 1 -- 172.21.15.28:0/2225098746 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f6b5c07ae90 con 0x7f6b740fb0d0 2024-07-03T15:47:02.436 INFO:teuthology.orchestra.run.smithi028.stdout:HEALTH_OK 2024-07-03T15:47:02.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.433+0000 7f6b79762640 1 -- 172.21.15.28:0/2225098746 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f6b6006e1e0 msgr2=0x7f6b60070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:02.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.433+0000 7f6b79762640 1 --2- 172.21.15.28:0/2225098746 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f6b6006e1e0 0x7f6b60070690 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6b64009a80 tx=0x7f6b64008040 comp rx=0 tx=0).stop 2024-07-03T15:47:02.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.433+0000 7f6b79762640 1 -- 172.21.15.28:0/2225098746 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6b740fb0d0 msgr2=0x7f6b7410d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:02.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.433+0000 7f6b79762640 1 --2- 172.21.15.28:0/2225098746 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6b740fb0d0 0x7f6b7410d750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c0048a0 tx=0x7f6b5c009da0 comp rx=0 tx=0).stop 2024-07-03T15:47:02.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.433+0000 7f6b79762640 1 -- 172.21.15.28:0/2225098746 shutdown_connections 2024-07-03T15:47:02.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.433+0000 7f6b79762640 1 --2- 172.21.15.28:0/2225098746 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f6b6006e1e0 0x7f6b60070690 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:02.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.433+0000 7f6b79762640 1 --2- 172.21.15.28:0/2225098746 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6b740fb0d0 0x7f6b7410d750 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:02.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.433+0000 7f6b79762640 1 --2- 172.21.15.28:0/2225098746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b740fa780 0x7f6b7410d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:02.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.433+0000 7f6b79762640 1 -- 172.21.15.28:0/2225098746 >> 172.21.15.28:0/2225098746 conn(0x7f6b740f61c0 msgr2=0x7f6b740f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:47:02.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.433+0000 7f6b79762640 1 -- 172.21.15.28:0/2225098746 shutdown_connections 2024-07-03T15:47:02.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:02.433+0000 7f6b79762640 1 -- 172.21.15.28:0/2225098746 wait complete. 2024-07-03T15:47:02.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:02 smithi148 ceph-mon[80490]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:47:02.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:02 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/958959493' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:47:02.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:02 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:02.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:02 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:02.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:02 smithi148 ceph-mon[80490]: Reconfiguring mds.cephfs.smithi148.xgpuld (monmap changed)... 2024-07-03T15:47:02.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:02 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.xgpuld", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-03T15:47:02.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:02 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:47:02.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:02 smithi148 ceph-mon[80490]: Reconfiguring daemon mds.cephfs.smithi148.xgpuld on smithi148 2024-07-03T15:47:02.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:02 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/3059924289' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-03T15:47:02.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:02 smithi028 ceph-mon[92858]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:47:02.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:02 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/958959493' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:47:02.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:02 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:02.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:02 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:02.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:02 smithi028 ceph-mon[92858]: Reconfiguring mds.cephfs.smithi148.xgpuld (monmap changed)... 2024-07-03T15:47:02.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:02 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.xgpuld", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-03T15:47:02.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:02 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:47:02.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:02 smithi028 ceph-mon[92858]: Reconfiguring daemon mds.cephfs.smithi148.xgpuld on smithi148 2024-07-03T15:47:02.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:02 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/3059924289' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-03T15:47:03.721 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:03 smithi148 ceph-mon[80490]: pgmap v23: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:47:03.721 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:03 smithi148 ceph-mon[80490]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:47:03.721 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:03 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/2225098746' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-03T15:47:03.721 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:03 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:03.722 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:03 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:03.722 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:03 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.szomfl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-03T15:47:03.722 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:03 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:47:03.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:03 smithi028 ceph-mon[92858]: pgmap v23: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:47:03.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:03 smithi028 ceph-mon[92858]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:47:03.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:03 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/2225098746' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-03T15:47:03.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:03 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:03.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:03 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:03.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:03 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.szomfl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-03T15:47:03.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:03 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:47:04.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:04 smithi148 ceph-mon[80490]: Reconfiguring mds.cephfs.smithi148.szomfl (monmap changed)... 2024-07-03T15:47:04.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:04 smithi148 ceph-mon[80490]: Reconfiguring daemon mds.cephfs.smithi148.szomfl on smithi148 2024-07-03T15:47:04.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:04 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:04.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:04 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:04.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:04 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:47:04.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:04 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:47:04.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:04 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:47:04.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:04 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:04.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:04 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi028"}]: dispatch 2024-07-03T15:47:04.734 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:04 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi028"}]': finished 2024-07-03T15:47:04.787 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:04 smithi028 ceph-mon[92858]: Reconfiguring mds.cephfs.smithi148.szomfl (monmap changed)... 2024-07-03T15:47:04.787 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:04 smithi028 ceph-mon[92858]: Reconfiguring daemon mds.cephfs.smithi148.szomfl on smithi148 2024-07-03T15:47:04.788 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:04 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:04.788 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:04 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:04.788 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:04 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:47:04.788 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:04 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:47:04.788 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:04 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:47:04.788 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:04 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:04.789 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:04 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi028"}]: dispatch 2024-07-03T15:47:04.789 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:04 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi028"}]': finished 2024-07-03T15:47:05.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:05 smithi148 ceph-mon[80490]: pgmap v24: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:47:05.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:05 smithi148 ceph-mon[80490]: Upgrade: Setting container_image for all mon 2024-07-03T15:47:05.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:05 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi148"}]: dispatch 2024-07-03T15:47:05.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:05 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi148"}]': finished 2024-07-03T15:47:05.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:05 smithi028 ceph-mon[92858]: pgmap v24: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:47:05.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:05 smithi028 ceph-mon[92858]: Upgrade: Setting container_image for all mon 2024-07-03T15:47:05.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:05 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi148"}]: dispatch 2024-07-03T15:47:05.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:05 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi148"}]': finished 2024-07-03T15:47:07.283 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:06 smithi028 ceph-mon[92858]: Upgrade: Updating crash.smithi028 (1/2) 2024-07-03T15:47:07.283 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:06 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:07.283 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:06 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi028", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-07-03T15:47:07.283 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:06 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:47:07.283 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:06 smithi028 ceph-mon[92858]: Deploying daemon crash.smithi028 on smithi028 2024-07-03T15:47:07.283 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:06 smithi028 ceph-mon[92858]: pgmap v25: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:47:07.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:06 smithi148 ceph-mon[80490]: Upgrade: Updating crash.smithi028 (1/2) 2024-07-03T15:47:07.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:06 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:07.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:06 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi028", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-07-03T15:47:07.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:06 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:47:07.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:06 smithi148 ceph-mon[80490]: Deploying daemon crash.smithi028 on smithi028 2024-07-03T15:47:07.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:06 smithi148 ceph-mon[80490]: pgmap v25: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:47:08.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:07 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:47:08.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:07 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:47:09.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:08 smithi148 ceph-mon[80490]: pgmap v26: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:47:09.256 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:08 smithi028 ceph-mon[92858]: pgmap v26: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:47:10.841 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:10 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:10.841 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:10 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:10.844 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:10 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:10.844 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:10 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:11.819 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:11 smithi028 ceph-mon[92858]: pgmap v27: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:47:11.819 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:11 smithi028 ceph-mon[92858]: Upgrade: Updating crash.smithi148 (2/2) 2024-07-03T15:47:11.819 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:11 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:11.820 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:11 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi148", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-07-03T15:47:11.820 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:11 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:47:11.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:11 smithi148 ceph-mon[80490]: pgmap v27: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:47:11.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:11 smithi148 ceph-mon[80490]: Upgrade: Updating crash.smithi148 (2/2) 2024-07-03T15:47:11.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:11 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:11.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:11 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi148", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-07-03T15:47:11.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:11 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:47:12.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:12 smithi028 ceph-mon[92858]: Deploying daemon crash.smithi148 on smithi148 2024-07-03T15:47:12.978 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:12 smithi148 ceph-mon[80490]: Deploying daemon crash.smithi148 on smithi148 2024-07-03T15:47:13.728 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:13 smithi148 ceph-mon[80490]: pgmap v28: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:47:13.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:13 smithi028 ceph-mon[92858]: pgmap v28: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:47:15.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:15 smithi148 ceph-mon[80490]: pgmap v29: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:47:15.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:15 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:15.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:15 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:15.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:15 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:47:15.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:15 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:15.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:15 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:15.596 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:15 smithi028 ceph-mon[92858]: pgmap v29: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:47:15.596 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:15 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:15.596 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:15 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:15.596 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:15 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:47:15.596 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:15 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:15.596 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:15 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:17.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:17 smithi028 ceph-mon[92858]: pgmap v30: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:47:17.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:17 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:17.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:17 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:17.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:17 smithi148 ceph-mon[80490]: pgmap v30: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:47:17.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:17 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:17.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:17 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:19.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:18 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:19.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:18 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:19.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:18 smithi148 ceph-mon[80490]: pgmap v31: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:47:19.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:18 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:19.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:18 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:19.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:18 smithi028 ceph-mon[92858]: pgmap v31: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:47:20.558 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:20 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:20.559 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:20 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:20.559 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:20 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:20.559 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:20 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:20.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:20 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:20.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:20 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:20.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:20 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:20.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:20 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:21.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:21 smithi148 ceph-mon[80490]: pgmap v32: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:47:21.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:21 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:21.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:21 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:21.563 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:21 smithi028 ceph-mon[92858]: pgmap v32: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:47:21.563 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:21 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:21.563 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:21 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:22.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:22 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:47:22.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:22 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:47:23.508 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:23 smithi028 ceph-mon[92858]: pgmap v33: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:47:23.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:23 smithi148 ceph-mon[80490]: pgmap v33: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:47:24.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:24 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:24.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:24 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:24.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:24 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:47:24.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:24 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:47:24.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:24 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:24.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:24 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:47:24.374 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:24 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:47:24.374 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:24 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:47:24.374 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:24 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:47:24.374 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:24 smithi028 ceph-mon[92858]: Upgrade: Setting container_image for all crash 2024-07-03T15:47:24.374 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:24 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:24.374 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:24 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi028"}]: dispatch 2024-07-03T15:47:24.374 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:24 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi028"}]': finished 2024-07-03T15:47:24.374 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:24 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi148"}]: dispatch 2024-07-03T15:47:24.374 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:24 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi148"}]': finished 2024-07-03T15:47:24.374 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:24 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-07-03T15:47:24.374 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:24 smithi028 ceph-mon[92858]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-07-03T15:47:24.374 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:24 smithi028 ceph-mon[92858]: Upgrade: osd.0 is safe to restart 2024-07-03T15:47:24.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:24 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:24.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:24 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:24.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:24 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:47:24.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:24 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:47:24.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:24 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:24.734 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:24 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:47:24.734 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:24 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:47:24.734 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:24 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:47:24.734 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:24 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:47:24.734 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:24 smithi148 ceph-mon[80490]: Upgrade: Setting container_image for all crash 2024-07-03T15:47:24.734 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:24 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:24.734 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:24 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi028"}]: dispatch 2024-07-03T15:47:24.734 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:24 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi028"}]': finished 2024-07-03T15:47:24.734 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:24 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi148"}]: dispatch 2024-07-03T15:47:24.735 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:24 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi148"}]': finished 2024-07-03T15:47:24.735 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:24 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-07-03T15:47:24.735 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:24 smithi148 ceph-mon[80490]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-07-03T15:47:24.735 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:24 smithi148 ceph-mon[80490]: Upgrade: osd.0 is safe to restart 2024-07-03T15:47:25.563 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:25 smithi028 ceph-mon[92858]: pgmap v34: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:47:25.563 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:25 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:25.563 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:25 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-07-03T15:47:25.563 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:25 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:47:25.723 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:25 smithi148 ceph-mon[80490]: pgmap v34: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:47:25.723 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:25 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:25.723 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:25 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-07-03T15:47:25.723 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:25 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:47:26.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:26 smithi148 ceph-mon[80490]: Upgrade: Updating osd.0 2024-07-03T15:47:26.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:26 smithi148 ceph-mon[80490]: Deploying daemon osd.0 on smithi028 2024-07-03T15:47:26.772 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:26 smithi028 ceph-mon[92858]: Upgrade: Updating osd.0 2024-07-03T15:47:26.772 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:26 smithi028 ceph-mon[92858]: Deploying daemon osd.0 on smithi028 2024-07-03T15:47:27.669 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:27 smithi028 ceph-mon[92858]: pgmap v35: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:47:27.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:27 smithi148 ceph-mon[80490]: pgmap v35: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:47:28.316 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:28 smithi028 systemd[1]: Stopping Ceph osd.0 for a85e4be6-3951-11ef-bcab-c7b262605968... 2024-07-03T15:47:28.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:28 smithi028 ceph-mon[92858]: osd.0 marked itself down and dead 2024-07-03T15:47:28.623 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:28 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-0[47392]: 2024-07-03T15:47:28.311+0000 7f7f2eb4b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-03T15:47:28.623 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:28 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-0[47392]: 2024-07-03T15:47:28.311+0000 7f7f2eb4b640 -1 osd.0 48 *** Got signal Terminated *** 2024-07-03T15:47:28.623 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:28 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-0[47392]: 2024-07-03T15:47:28.311+0000 7f7f2eb4b640 -1 osd.0 48 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-03T15:47:28.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:28 smithi148 ceph-mon[80490]: osd.0 marked itself down and dead 2024-07-03T15:47:29.050 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:28 smithi028 podman[100791]: 2024-07-03 15:47:28.782758764 +0000 UTC m=+0.590646901 container died b18089a439bae2d0136b3dbdcc8810882de6a6a13cd87ce43d05adad40a43141 (image=quay.ceph.io/ceph-ci/ceph@sha256:0b7651540cdb24c1b97466c79c298d54c7794c5f53d5d6b6c5d76a2d3b130d85, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240625, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=quincy-d4b063a, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS) 2024-07-03T15:47:29.372 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:29 smithi028 podman[100791]: 2024-07-03 15:47:29.195469485 +0000 UTC m=+1.003357663 container cleanup b18089a439bae2d0136b3dbdcc8810882de6a6a13cd87ce43d05adad40a43141 (image=quay.ceph.io/ceph-ci/ceph@sha256:0b7651540cdb24c1b97466c79c298d54c7794c5f53d5d6b6c5d76a2d3b130d85, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-0, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=quincy-d4b063a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240625, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-07-03T15:47:29.373 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:29 smithi028 bash[100791]: ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-0 2024-07-03T15:47:29.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:29 smithi148 ceph-mon[80490]: pgmap v36: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:47:29.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:29 smithi148 ceph-mon[80490]: Health check failed: 1 osds down (OSD_DOWN) 2024-07-03T15:47:29.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:29 smithi148 ceph-mon[80490]: osdmap e49: 6 total, 5 up, 6 in 2024-07-03T15:47:29.770 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:29 smithi028 ceph-mon[92858]: pgmap v36: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:47:29.770 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:29 smithi028 ceph-mon[92858]: Health check failed: 1 osds down (OSD_DOWN) 2024-07-03T15:47:29.770 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:29 smithi028 ceph-mon[92858]: osdmap e49: 6 total, 5 up, 6 in 2024-07-03T15:47:29.770 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:29 smithi028 podman[100808]: 2024-07-03 15:47:29.405315278 +0000 UTC m=+0.615966911 container remove b18089a439bae2d0136b3dbdcc8810882de6a6a13cd87ce43d05adad40a43141 (image=quay.ceph.io/ceph-ci/ceph@sha256:0b7651540cdb24c1b97466c79c298d54c7794c5f53d5d6b6c5d76a2d3b130d85, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-0, org.label-schema.build-date=20240625, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=quincy-d4b063a, ceph=True, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-07-03T15:47:30.087 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:29 smithi028 podman[100895]: 2024-07-03 15:47:29.76502461 +0000 UTC m=+0.112343352 container create 1e55289c1d29988f6adbf2812ee680f3cffacc59b844cb4d63bb5620e2ca997f (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-0-deactivate, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-release-c00506d, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240625, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True) 2024-07-03T15:47:30.087 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:29 smithi028 podman[100895]: 2024-07-03 15:47:29.670852143 +0000 UTC m=+0.018170855 image pull 9d49405603e8445bc510362cdeb05f243619be76b0f3897dd4c152b00cffe45f quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2 2024-07-03T15:47:30.087 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:30 smithi028 podman[100895]: 2024-07-03 15:47:30.000796962 +0000 UTC m=+0.348115678 container init 1e55289c1d29988f6adbf2812ee680f3cffacc59b844cb4d63bb5620e2ca997f (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, RELEASE=squid-release-c00506d, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240625, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-07-03T15:47:30.087 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:30 smithi028 podman[100895]: 2024-07-03 15:47:30.004254694 +0000 UTC m=+0.351573394 container start 1e55289c1d29988f6adbf2812ee680f3cffacc59b844cb4d63bb5620e2ca997f (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-release-c00506d, io.buildah.version=1.36.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240625) 2024-07-03T15:47:30.088 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:30 smithi028 podman[100895]: 2024-07-03 15:47:30.084126576 +0000 UTC m=+0.431445290 container attach 1e55289c1d29988f6adbf2812ee680f3cffacc59b844cb4d63bb5620e2ca997f (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-0-deactivate, ceph=True, RELEASE=squid-release-c00506d, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240625, io.buildah.version=1.36.0, GIT_BRANCH=HEAD) 2024-07-03T15:47:30.372 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:30 smithi028 podman[100895]: 2024-07-03 15:47:30.275548893 +0000 UTC m=+0.622867617 container died 1e55289c1d29988f6adbf2812ee680f3cffacc59b844cb4d63bb5620e2ca997f (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240625, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-release-c00506d, io.buildah.version=1.36.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-07-03T15:47:30.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:30 smithi148 ceph-mon[80490]: osdmap e50: 6 total, 5 up, 6 in 2024-07-03T15:47:30.771 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:30 smithi028 ceph-mon[92858]: osdmap e50: 6 total, 5 up, 6 in 2024-07-03T15:47:31.122 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:30 smithi028 podman[100919]: 2024-07-03 15:47:30.888291709 +0000 UTC m=+0.606452014 container remove 1e55289c1d29988f6adbf2812ee680f3cffacc59b844cb4d63bb5620e2ca997f (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240625, GIT_BRANCH=HEAD, RELEASE=squid-release-c00506d, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0) 2024-07-03T15:47:31.123 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:30 smithi028 systemd[1]: ceph-a85e4be6-3951-11ef-bcab-c7b262605968@osd.0.service: Deactivated successfully. 2024-07-03T15:47:31.123 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:30 smithi028 systemd[1]: Stopped Ceph osd.0 for a85e4be6-3951-11ef-bcab-c7b262605968. 2024-07-03T15:47:31.123 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:30 smithi028 systemd[1]: ceph-a85e4be6-3951-11ef-bcab-c7b262605968@osd.0.service: Consumed 1min 1.573s CPU time. 2024-07-03T15:47:31.401 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:31 smithi028 systemd[1]: Starting Ceph osd.0 for a85e4be6-3951-11ef-bcab-c7b262605968... 2024-07-03T15:47:31.401 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:31 smithi028 podman[101020]: 2024-07-03 15:47:31.265663713 +0000 UTC m=+0.018612398 image pull 9d49405603e8445bc510362cdeb05f243619be76b0f3897dd4c152b00cffe45f quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2 2024-07-03T15:47:31.401 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:31 smithi028 podman[101020]: 2024-07-03 15:47:31.397616587 +0000 UTC m=+0.150565270 container create 45c32c40472c2ae3e1238cca35300c5745a450e0363f427ae0e68bc9ea174c0b (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-0-activate, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, ceph=True, RELEASE=squid-release-c00506d, GIT_BRANCH=HEAD, org.label-schema.build-date=20240625, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-07-03T15:47:31.712 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:31 smithi028 ceph-mon[92858]: pgmap v39: 65 pgs: 9 stale+active+clean, 56 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:47:31.712 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:31 smithi028 podman[101020]: 2024-07-03 15:47:31.608514749 +0000 UTC m=+0.361463441 container init 45c32c40472c2ae3e1238cca35300c5745a450e0363f427ae0e68bc9ea174c0b (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-0-activate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-release-c00506d, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240625, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-03T15:47:31.712 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:31 smithi028 podman[101020]: 2024-07-03 15:47:31.612155481 +0000 UTC m=+0.365104172 container start 45c32c40472c2ae3e1238cca35300c5745a450e0363f427ae0e68bc9ea174c0b (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-0-activate, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.build-date=20240625, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-release-c00506d, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=) 2024-07-03T15:47:31.712 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:31 smithi028 podman[101020]: 2024-07-03 15:47:31.708643857 +0000 UTC m=+0.461592542 container attach 45c32c40472c2ae3e1238cca35300c5745a450e0363f427ae0e68bc9ea174c0b (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-0-activate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-release-c00506d, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240625) 2024-07-03T15:47:31.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:31 smithi148 ceph-mon[80490]: pgmap v39: 65 pgs: 9 stale+active+clean, 56 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:47:32.373 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:31 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-0-activate[101035]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-07-03T15:47:32.373 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:31 smithi028 bash[101020]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-07-03T15:47:32.373 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:31 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-0-activate[101035]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-07-03T15:47:32.373 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:31 smithi028 bash[101020]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-07-03T15:47:32.373 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:32 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-0-activate[101035]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-07-03T15:47:32.373 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:32 smithi028 bash[101020]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-07-03T15:47:32.374 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:32 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-0-activate[101035]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-07-03T15:47:32.374 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:32 smithi028 bash[101020]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-07-03T15:47:32.374 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:32 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-0-activate[101035]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-07-03T15:47:32.374 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:32 smithi028 bash[101020]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-07-03T15:47:32.374 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:32 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-0-activate[101035]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-07-03T15:47:32.374 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:32 smithi028 bash[101020]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-07-03T15:47:32.374 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:32 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-0-activate[101035]: --> ceph-volume raw activate successful for osd ID: 0 2024-07-03T15:47:32.374 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:32 smithi028 bash[101020]: --> ceph-volume raw activate successful for osd ID: 0 2024-07-03T15:47:32.374 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:32 smithi028 podman[101020]: 2024-07-03 15:47:32.047490508 +0000 UTC m=+0.800439193 container died 45c32c40472c2ae3e1238cca35300c5745a450e0363f427ae0e68bc9ea174c0b (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-0-activate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240625, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, ceph=True, RELEASE=squid-release-c00506d, org.label-schema.license=GPLv2) 2024-07-03T15:47:32.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.524+0000 7febb01af640 1 -- 172.21.15.28:0/3291195206 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feba80fc360 msgr2=0x7feba80fc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:32.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.524+0000 7febb01af640 1 --2- 172.21.15.28:0/3291195206 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feba80fc360 0x7feba80fc770 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7feba40048d0 tx=0x7feba4009da0 comp rx=0 tx=0).stop 2024-07-03T15:47:32.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.524+0000 7febb01af640 1 -- 172.21.15.28:0/3291195206 shutdown_connections 2024-07-03T15:47:32.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.524+0000 7febb01af640 1 --2- 172.21.15.28:0/3291195206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feba80fd630 0x7feba80fdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:32.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.524+0000 7febb01af640 1 --2- 172.21.15.28:0/3291195206 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feba80fc360 0x7feba80fc770 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:32.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.524+0000 7febb01af640 1 -- 172.21.15.28:0/3291195206 >> 172.21.15.28:0/3291195206 conn(0x7feba80f7b40 msgr2=0x7feba80f9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:47:32.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.524+0000 7febb01af640 1 -- 172.21.15.28:0/3291195206 shutdown_connections 2024-07-03T15:47:32.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.524+0000 7febb01af640 1 -- 172.21.15.28:0/3291195206 wait complete. 2024-07-03T15:47:32.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.525+0000 7febb01af640 1 Processor -- start 2024-07-03T15:47:32.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.525+0000 7febb01af640 1 -- start start 2024-07-03T15:47:32.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.525+0000 7febb01af640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feba80fc360 0x7feba810cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:47:32.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.525+0000 7febb01af640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feba80fd630 0x7feba810d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:47:32.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.525+0000 7febb01af640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba810db80 con 0x7feba80fd630 2024-07-03T15:47:32.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.525+0000 7febb01af640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba810e350 con 0x7feba80fc360 2024-07-03T15:47:32.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.525+0000 7febad74a640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feba80fd630 0x7feba810d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:47:32.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.525+0000 7febad74a640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feba80fd630 0x7feba810d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48392/0 (socket says 172.21.15.28:48392) 2024-07-03T15:47:32.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.525+0000 7febad74a640 1 -- 172.21.15.28:0/3917773768 learned_addr learned my addr 172.21.15.28:0/3917773768 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:47:32.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.525+0000 7febadf4b640 1 --2- 172.21.15.28:0/3917773768 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feba80fc360 0x7feba810cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:47:32.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.525+0000 7febad74a640 1 -- 172.21.15.28:0/3917773768 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feba80fc360 msgr2=0x7feba810cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:32.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.525+0000 7febad74a640 1 --2- 172.21.15.28:0/3917773768 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feba80fc360 0x7feba810cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:32.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.525+0000 7febad74a640 1 -- 172.21.15.28:0/3917773768 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba4005040 con 0x7feba80fd630 2024-07-03T15:47:32.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.525+0000 7febad74a640 1 --2- 172.21.15.28:0/3917773768 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feba80fd630 0x7feba810d530 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7feb9800c790 tx=0x7feb980043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:47:32.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.526+0000 7feb9e7fc640 1 -- 172.21.15.28:0/3917773768 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb9800a820 con 0x7feba80fd630 2024-07-03T15:47:32.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.526+0000 7feb9e7fc640 1 -- 172.21.15.28:0/3917773768 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feb9800ae50 con 0x7feba80fd630 2024-07-03T15:47:32.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.526+0000 7febb01af640 1 -- 172.21.15.28:0/3917773768 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feba810e5e0 con 0x7feba80fd630 2024-07-03T15:47:32.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.526+0000 7feb9e7fc640 1 -- 172.21.15.28:0/3917773768 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb980115e0 con 0x7feba80fd630 2024-07-03T15:47:32.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.526+0000 7febb01af640 1 -- 172.21.15.28:0/3917773768 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feba810eb80 con 0x7feba80fd630 2024-07-03T15:47:32.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.527+0000 7febb01af640 1 -- 172.21.15.28:0/3917773768 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feba80fb7c0 con 0x7feba80fd630 2024-07-03T15:47:32.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.527+0000 7feb9e7fc640 1 -- 172.21.15.28:0/3917773768 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 36) v1 ==== 92305+0+0 (secure 0 0 0) 0x7feb98012400 con 0x7feba80fd630 2024-07-03T15:47:32.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.528+0000 7feb9e7fc640 1 --2- 172.21.15.28:0/3917773768 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7feb9406e1e0 0x7feb94070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:47:32.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.528+0000 7feb9e7fc640 1 -- 172.21.15.28:0/3917773768 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6410+0+0 (secure 0 0 0) 0x7feb9808dd30 con 0x7feba80fd630 2024-07-03T15:47:32.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.528+0000 7febadf4b640 1 --2- 172.21.15.28:0/3917773768 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7feb9406e1e0 0x7feb94070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:47:32.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.528+0000 7febadf4b640 1 --2- 172.21.15.28:0/3917773768 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7feb9406e1e0 0x7feb94070690 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7feba4000c00 tx=0x7feba4035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:47:32.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.533+0000 7feb9e7fc640 1 -- 172.21.15.28:0/3917773768 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7feb980570c0 con 0x7feba80fd630 2024-07-03T15:47:32.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.737+0000 7febb01af640 1 -- 172.21.15.28:0/3917773768 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7feba810ef50 con 0x7feb9406e1e0 2024-07-03T15:47:32.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.738+0000 7feb9e7fc640 1 -- 172.21.15.28:0/3917773768 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7feba810ef50 con 0x7feb9406e1e0 2024-07-03T15:47:32.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.740+0000 7febb01af640 1 -- 172.21.15.28:0/3917773768 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7feb9406e1e0 msgr2=0x7feb94070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:32.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.740+0000 7febb01af640 1 --2- 172.21.15.28:0/3917773768 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7feb9406e1e0 0x7feb94070690 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7feba4000c00 tx=0x7feba4035040 comp rx=0 tx=0).stop 2024-07-03T15:47:32.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.740+0000 7febb01af640 1 -- 172.21.15.28:0/3917773768 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feba80fd630 msgr2=0x7feba810d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:32.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.740+0000 7febb01af640 1 --2- 172.21.15.28:0/3917773768 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feba80fd630 0x7feba810d530 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7feb9800c790 tx=0x7feb980043c0 comp rx=0 tx=0).stop 2024-07-03T15:47:32.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.740+0000 7febb01af640 1 -- 172.21.15.28:0/3917773768 shutdown_connections 2024-07-03T15:47:32.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.740+0000 7febb01af640 1 --2- 172.21.15.28:0/3917773768 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7feb9406e1e0 0x7feb94070690 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:32.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.740+0000 7febb01af640 1 --2- 172.21.15.28:0/3917773768 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feba80fd630 0x7feba810d530 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:32.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.740+0000 7febb01af640 1 --2- 172.21.15.28:0/3917773768 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feba80fc360 0x7feba810cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:32.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.740+0000 7febb01af640 1 -- 172.21.15.28:0/3917773768 >> 172.21.15.28:0/3917773768 conn(0x7feba80f7b40 msgr2=0x7feba80f9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:47:32.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.740+0000 7febb01af640 1 -- 172.21.15.28:0/3917773768 shutdown_connections 2024-07-03T15:47:32.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.740+0000 7febb01af640 1 -- 172.21.15.28:0/3917773768 wait complete. 2024-07-03T15:47:32.753 INFO:teuthology.orchestra.run.smithi028.stdout:true 2024-07-03T15:47:32.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.824+0000 7fd2b70b9640 1 -- 172.21.15.28:0/4109982367 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2b00fb240 msgr2=0x7fd2b00fb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:32.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.824+0000 7fd2b70b9640 1 --2- 172.21.15.28:0/4109982367 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2b00fb240 0x7fd2b00fb650 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd2a00048d0 tx=0x7fd2a0009da0 comp rx=0 tx=0).stop 2024-07-03T15:47:32.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.824+0000 7fd2b70b9640 1 -- 172.21.15.28:0/4109982367 shutdown_connections 2024-07-03T15:47:32.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.824+0000 7fd2b70b9640 1 --2- 172.21.15.28:0/4109982367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2b00fc510 0x7fd2b00fc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:32.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.824+0000 7fd2b70b9640 1 --2- 172.21.15.28:0/4109982367 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2b00fb240 0x7fd2b00fb650 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:32.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.824+0000 7fd2b70b9640 1 -- 172.21.15.28:0/4109982367 >> 172.21.15.28:0/4109982367 conn(0x7fd2b00f6a20 msgr2=0x7fd2b00f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:47:32.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.824+0000 7fd2b70b9640 1 -- 172.21.15.28:0/4109982367 shutdown_connections 2024-07-03T15:47:32.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.824+0000 7fd2b70b9640 1 -- 172.21.15.28:0/4109982367 wait complete. 2024-07-03T15:47:32.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.824+0000 7fd2b70b9640 1 Processor -- start 2024-07-03T15:47:32.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.824+0000 7fd2b70b9640 1 -- start start 2024-07-03T15:47:32.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.825+0000 7fd2b70b9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2b00fb240 0x7fd2b010fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:47:32.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.825+0000 7fd2b70b9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2b00fc510 0x7fd2b0112230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:47:32.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.825+0000 7fd2b70b9640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b0065b20 con 0x7fd2b00fc510 2024-07-03T15:47:32.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.825+0000 7fd2b70b9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b0065c90 con 0x7fd2b00fb240 2024-07-03T15:47:32.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.825+0000 7fd2affff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2b00fc510 0x7fd2b0112230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:47:32.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.825+0000 7fd2affff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2b00fc510 0x7fd2b0112230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48400/0 (socket says 172.21.15.28:48400) 2024-07-03T15:47:32.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.825+0000 7fd2affff640 1 -- 172.21.15.28:0/3314115198 learned_addr learned my addr 172.21.15.28:0/3314115198 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:47:32.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.825+0000 7fd2b4e55640 1 --2- 172.21.15.28:0/3314115198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2b00fb240 0x7fd2b010fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:47:32.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.825+0000 7fd2affff640 1 -- 172.21.15.28:0/3314115198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2b00fb240 msgr2=0x7fd2b010fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:32.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.825+0000 7fd2affff640 1 --2- 172.21.15.28:0/3314115198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2b00fb240 0x7fd2b010fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:32.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.825+0000 7fd2affff640 1 -- 172.21.15.28:0/3314115198 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a0005040 con 0x7fd2b00fc510 2024-07-03T15:47:32.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.825+0000 7fd2b4e55640 1 --2- 172.21.15.28:0/3314115198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2b00fb240 0x7fd2b010fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-03T15:47:32.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.825+0000 7fd2affff640 1 --2- 172.21.15.28:0/3314115198 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2b00fc510 0x7fd2b0112230 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd2a400bbb0 tx=0x7fd2a4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:47:32.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.825+0000 7fd2ad7fa640 1 -- 172.21.15.28:0/3314115198 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2a400c9c0 con 0x7fd2b00fc510 2024-07-03T15:47:32.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.825+0000 7fd2ad7fa640 1 -- 172.21.15.28:0/3314115198 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd2a40092e0 con 0x7fd2b00fc510 2024-07-03T15:47:32.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.826+0000 7fd2b70b9640 1 -- 172.21.15.28:0/3314115198 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2b0112890 con 0x7fd2b00fc510 2024-07-03T15:47:32.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.826+0000 7fd2ad7fa640 1 -- 172.21.15.28:0/3314115198 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2a4007640 con 0x7fd2b00fc510 2024-07-03T15:47:32.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.826+0000 7fd2b70b9640 1 -- 172.21.15.28:0/3314115198 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2b0112d60 con 0x7fd2b00fc510 2024-07-03T15:47:32.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.827+0000 7fd2b70b9640 1 -- 172.21.15.28:0/3314115198 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2b00fa6a0 con 0x7fd2b00fc510 2024-07-03T15:47:32.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.827+0000 7fd2ad7fa640 1 -- 172.21.15.28:0/3314115198 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 36) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fd2a4012070 con 0x7fd2b00fc510 2024-07-03T15:47:32.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.828+0000 7fd2ad7fa640 1 --2- 172.21.15.28:0/3314115198 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fd29806e1e0 0x7fd298070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:47:32.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.828+0000 7fd2ad7fa640 1 -- 172.21.15.28:0/3314115198 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fd2a408dcd0 con 0x7fd2b00fc510 2024-07-03T15:47:32.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.828+0000 7fd2b4e55640 1 --2- 172.21.15.28:0/3314115198 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fd29806e1e0 0x7fd298070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:47:32.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.828+0000 7fd2b4e55640 1 --2- 172.21.15.28:0/3314115198 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fd29806e1e0 0x7fd298070690 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0000c00 tx=0x7fd2a0035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:47:32.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:32.832+0000 7fd2ad7fa640 1 -- 172.21.15.28:0/3314115198 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd2a4056f60 con 0x7fd2b00fc510 2024-07-03T15:47:33.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.061+0000 7fd2b70b9640 1 -- 172.21.15.28:0/3314115198 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd2b0100830 con 0x7fd29806e1e0 2024-07-03T15:47:33.065 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:32 smithi028 podman[101165]: 2024-07-03 15:47:32.922063975 +0000 UTC m=+0.868394181 container remove 45c32c40472c2ae3e1238cca35300c5745a450e0363f427ae0e68bc9ea174c0b (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-0-activate, RELEASE=squid-release-c00506d, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240625, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-07-03T15:47:33.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.062+0000 7fd2ad7fa640 1 -- 172.21.15.28:0/3314115198 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fd2b0100830 con 0x7fd29806e1e0 2024-07-03T15:47:33.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.064+0000 7fd2b70b9640 1 -- 172.21.15.28:0/3314115198 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fd29806e1e0 msgr2=0x7fd298070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:33.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.064+0000 7fd2b70b9640 1 --2- 172.21.15.28:0/3314115198 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fd29806e1e0 0x7fd298070690 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0000c00 tx=0x7fd2a0035040 comp rx=0 tx=0).stop 2024-07-03T15:47:33.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.064+0000 7fd2b70b9640 1 -- 172.21.15.28:0/3314115198 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2b00fc510 msgr2=0x7fd2b0112230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:33.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.064+0000 7fd2b70b9640 1 --2- 172.21.15.28:0/3314115198 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2b00fc510 0x7fd2b0112230 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd2a400bbb0 tx=0x7fd2a4009840 comp rx=0 tx=0).stop 2024-07-03T15:47:33.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.064+0000 7fd2b70b9640 1 -- 172.21.15.28:0/3314115198 shutdown_connections 2024-07-03T15:47:33.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.064+0000 7fd2b70b9640 1 --2- 172.21.15.28:0/3314115198 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fd29806e1e0 0x7fd298070690 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:33.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.064+0000 7fd2b70b9640 1 --2- 172.21.15.28:0/3314115198 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2b00fc510 0x7fd2b0112230 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:33.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.064+0000 7fd2b70b9640 1 --2- 172.21.15.28:0/3314115198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2b00fb240 0x7fd2b010fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:33.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.064+0000 7fd2b70b9640 1 -- 172.21.15.28:0/3314115198 >> 172.21.15.28:0/3314115198 conn(0x7fd2b00f6a20 msgr2=0x7fd2b00f80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:47:33.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.064+0000 7fd2b70b9640 1 -- 172.21.15.28:0/3314115198 shutdown_connections 2024-07-03T15:47:33.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.064+0000 7fd2b70b9640 1 -- 172.21.15.28:0/3314115198 wait complete. 2024-07-03T15:47:33.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.147+0000 7f23d80d0640 1 -- 172.21.15.28:0/3366638770 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f23d00fd600 msgr2=0x7f23d00fda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:33.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.147+0000 7f23d80d0640 1 --2- 172.21.15.28:0/3366638770 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f23d00fd600 0x7f23d00fda70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f23c00048d0 tx=0x7f23c0009da0 comp rx=0 tx=0).stop 2024-07-03T15:47:33.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.147+0000 7f23d80d0640 1 -- 172.21.15.28:0/3366638770 shutdown_connections 2024-07-03T15:47:33.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.147+0000 7f23d80d0640 1 --2- 172.21.15.28:0/3366638770 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f23d00fd600 0x7f23d00fda70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:33.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.147+0000 7f23d80d0640 1 --2- 172.21.15.28:0/3366638770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23d00fc330 0x7f23d00fc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:33.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.147+0000 7f23d80d0640 1 -- 172.21.15.28:0/3366638770 >> 172.21.15.28:0/3366638770 conn(0x7f23d00f7af0 msgr2=0x7f23d00f9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:47:33.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.147+0000 7f23d80d0640 1 -- 172.21.15.28:0/3366638770 shutdown_connections 2024-07-03T15:47:33.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.147+0000 7f23d80d0640 1 -- 172.21.15.28:0/3366638770 wait complete. 2024-07-03T15:47:33.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.147+0000 7f23d80d0640 1 Processor -- start 2024-07-03T15:47:33.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.147+0000 7f23d80d0640 1 -- start start 2024-07-03T15:47:33.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.148+0000 7f23d80d0640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f23d00fc330 0x7f23d0071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:47:33.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.148+0000 7f23d80d0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23d00fd600 0x7f23d006f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:47:33.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.148+0000 7f23d80d0640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d006fe00 con 0x7f23d00fc330 2024-07-03T15:47:33.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.148+0000 7f23d80d0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d006ff40 con 0x7f23d00fd600 2024-07-03T15:47:33.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.148+0000 7f23d5e6c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f23d00fc330 0x7f23d0071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:47:33.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.148+0000 7f23d5e6c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f23d00fc330 0x7f23d0071380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48418/0 (socket says 172.21.15.28:48418) 2024-07-03T15:47:33.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.148+0000 7f23d5e6c640 1 -- 172.21.15.28:0/1068843586 learned_addr learned my addr 172.21.15.28:0/1068843586 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:47:33.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.148+0000 7f23d566b640 1 --2- 172.21.15.28:0/1068843586 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23d00fd600 0x7f23d006f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:47:33.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.148+0000 7f23d5e6c640 1 -- 172.21.15.28:0/1068843586 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23d00fd600 msgr2=0x7f23d006f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:33.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.148+0000 7f23d5e6c640 1 --2- 172.21.15.28:0/1068843586 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23d00fd600 0x7f23d006f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:33.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.148+0000 7f23d5e6c640 1 -- 172.21.15.28:0/1068843586 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c0005040 con 0x7f23d00fc330 2024-07-03T15:47:33.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.148+0000 7f23d5e6c640 1 --2- 172.21.15.28:0/1068843586 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f23d00fc330 0x7f23d0071380 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f23cc002a50 tx=0x7f23cc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:47:33.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.148+0000 7f23c67fc640 1 -- 172.21.15.28:0/1068843586 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23cc007c30 con 0x7f23d00fc330 2024-07-03T15:47:33.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.148+0000 7f23c67fc640 1 -- 172.21.15.28:0/1068843586 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f23cc00f040 con 0x7f23d00fc330 2024-07-03T15:47:33.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.148+0000 7f23d80d0640 1 -- 172.21.15.28:0/1068843586 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23d0070220 con 0x7f23d00fc330 2024-07-03T15:47:33.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.148+0000 7f23c67fc640 1 -- 172.21.15.28:0/1068843586 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23cc00db50 con 0x7f23d00fc330 2024-07-03T15:47:33.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.149+0000 7f23d80d0640 1 -- 172.21.15.28:0/1068843586 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d0114fa0 con 0x7f23d00fc330 2024-07-03T15:47:33.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.149+0000 7f23d80d0640 1 -- 172.21.15.28:0/1068843586 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23d00fb790 con 0x7f23d00fc330 2024-07-03T15:47:33.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.150+0000 7f23c67fc640 1 -- 172.21.15.28:0/1068843586 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 36) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f23cc01c400 con 0x7f23d00fc330 2024-07-03T15:47:33.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.150+0000 7f23c67fc640 1 --2- 172.21.15.28:0/1068843586 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f23bc06e1e0 0x7f23bc070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:47:33.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.150+0000 7f23c67fc640 1 -- 172.21.15.28:0/1068843586 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f23cc08d6b0 con 0x7f23d00fc330 2024-07-03T15:47:33.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.151+0000 7f23d566b640 1 --2- 172.21.15.28:0/1068843586 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f23bc06e1e0 0x7f23bc070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:47:33.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.151+0000 7f23d566b640 1 --2- 172.21.15.28:0/1068843586 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f23bc06e1e0 0x7f23bc070690 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f23c002a530 tx=0x7f23c0009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:47:33.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.155+0000 7f23c67fc640 1 -- 172.21.15.28:0/1068843586 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f23cc056810 con 0x7f23d00fc330 2024-07-03T15:47:33.325 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:33 smithi028 podman[101271]: 2024-07-03 15:47:33.054388098 +0000 UTC m=+0.018469334 image pull 9d49405603e8445bc510362cdeb05f243619be76b0f3897dd4c152b00cffe45f quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2 2024-07-03T15:47:33.325 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:33 smithi028 podman[101271]: 2024-07-03 15:47:33.173427204 +0000 UTC m=+0.137508422 container create 4454cecff8fcd7ef64517202475d69270ead15f9771e4e2f283fdb197351ff30 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-0, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240625, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-release-c00506d, GIT_CLEAN=True) 2024-07-03T15:47:33.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.353+0000 7f23d80d0640 1 -- 172.21.15.28:0/1068843586 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f23d0101920 con 0x7f23bc06e1e0 2024-07-03T15:47:33.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.362+0000 7f23c67fc640 1 -- 172.21.15.28:0/1068843586 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f23d0101920 con 0x7f23bc06e1e0 2024-07-03T15:47:33.367 INFO:teuthology.orchestra.run.smithi028.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-03T15:47:33.367 INFO:teuthology.orchestra.run.smithi028.stdout:alertmanager.smithi028 smithi028 *:9093,9094 running (2m) 14s ago 11m 25.8M - 0.25.0 c8568f914cd2 1425315a0de9 2024-07-03T15:47:33.367 INFO:teuthology.orchestra.run.smithi028.stdout:crash.smithi028 smithi028 running (24s) 14s ago 11m 6895k - 19.0.0-2550-gc00506dc 9d49405603e8 c62c2f45cabe 2024-07-03T15:47:33.367 INFO:teuthology.orchestra.run.smithi028.stdout:crash.smithi148 smithi148 running (19s) 16s ago 10m 6895k - 19.0.0-2550-gc00506dc 9d49405603e8 b1318dcaacb6 2024-07-03T15:47:33.367 INFO:teuthology.orchestra.run.smithi028.stdout:grafana.smithi028 smithi028 *:3000 running (2m) 14s ago 10m 80.5M - 9.4.7 954c08fa6188 f9f17c92cfea 2024-07-03T15:47:33.367 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi028.rhpvve smithi028 running (5m) 14s ago 5m 152M - 17.2.7-1047-gd4b063ab 4ca399d9e65d 3ed5abd02cfb 2024-07-03T15:47:33.367 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi028.wmbxew smithi028 running (5m) 14s ago 5m 110M - 17.2.7-1047-gd4b063ab 4ca399d9e65d b3bf736998f9 2024-07-03T15:47:33.367 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi148.szomfl smithi148 running (5m) 16s ago 5m 15.7M - 17.2.7-1047-gd4b063ab 4ca399d9e65d d94f5c95fa8d 2024-07-03T15:47:33.367 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi148.xgpuld smithi148 running (5m) 16s ago 5m 13.2M - 17.2.7-1047-gd4b063ab 4ca399d9e65d fc70360a8de0 2024-07-03T15:47:33.367 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.smithi028.vtqrtz smithi028 *:8443,9283,8765 running (2m) 14s ago 12m 490M - 19.0.0-2550-gc00506dc 9d49405603e8 14d0363c5b77 2024-07-03T15:47:33.367 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.smithi148.rjzsar smithi148 *:8443,9283,8765 running (90s) 16s ago 10m 438M - 19.0.0-2550-gc00506dc 9d49405603e8 c0193aa8a5dd 2024-07-03T15:47:33.368 INFO:teuthology.orchestra.run.smithi028.stdout:mon.smithi028 smithi028 running (78s) 14s ago 12m 41.5M 2048M 19.0.0-2550-gc00506dc 9d49405603e8 5093f7bf47d7 2024-07-03T15:47:33.368 INFO:teuthology.orchestra.run.smithi028.stdout:mon.smithi148 smithi148 running (58s) 16s ago 10m 33.6M 2048M 19.0.0-2550-gc00506dc 9d49405603e8 d338d2c648da 2024-07-03T15:47:33.368 INFO:teuthology.orchestra.run.smithi028.stdout:node-exporter.smithi028 smithi028 *:9100 running (3m) 14s ago 11m 15.5M - 1.5.0 0da6a335fe13 91ca1f49de0e 2024-07-03T15:47:33.368 INFO:teuthology.orchestra.run.smithi028.stdout:node-exporter.smithi148 smithi148 *:9100 running (2m) 16s ago 10m 15.6M - 1.5.0 0da6a335fe13 1eaaf7b1b561 2024-07-03T15:47:33.368 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0 smithi028 running (9m) 14s ago 9m 256M 1558M 17.2.7-1047-gd4b063ab 4ca399d9e65d b18089a439ba 2024-07-03T15:47:33.368 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1 smithi028 running (9m) 14s ago 8m 255M 1558M 17.2.7-1047-gd4b063ab 4ca399d9e65d 87d415dde231 2024-07-03T15:47:33.368 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2 smithi028 running (8m) 14s ago 8m 219M 1558M 17.2.7-1047-gd4b063ab 4ca399d9e65d b2cbb312212d 2024-07-03T15:47:33.368 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3 smithi148 running (8m) 16s ago 8m 278M 2581M 17.2.7-1047-gd4b063ab 4ca399d9e65d 5be69a27df31 2024-07-03T15:47:33.368 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4 smithi148 running (7m) 16s ago 7m 250M 2581M 17.2.7-1047-gd4b063ab 4ca399d9e65d 54f7c26c77ca 2024-07-03T15:47:33.368 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5 smithi148 running (7m) 16s ago 7m 239M 2581M 17.2.7-1047-gd4b063ab 4ca399d9e65d d185c6905384 2024-07-03T15:47:33.368 INFO:teuthology.orchestra.run.smithi028.stdout:prometheus.smithi028 smithi028 *:9095 running (95s) 14s ago 10m 53.8M - 2.43.0 a07b618ecd1d 2b0eb2a95afd 2024-07-03T15:47:33.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.364+0000 7f23d80d0640 1 -- 172.21.15.28:0/1068843586 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f23bc06e1e0 msgr2=0x7f23bc070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:33.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.364+0000 7f23d80d0640 1 --2- 172.21.15.28:0/1068843586 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f23bc06e1e0 0x7f23bc070690 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f23c002a530 tx=0x7f23c0009da0 comp rx=0 tx=0).stop 2024-07-03T15:47:33.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.364+0000 7f23d80d0640 1 -- 172.21.15.28:0/1068843586 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f23d00fc330 msgr2=0x7f23d0071380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:33.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.364+0000 7f23d80d0640 1 --2- 172.21.15.28:0/1068843586 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f23d00fc330 0x7f23d0071380 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f23cc002a50 tx=0x7f23cc004600 comp rx=0 tx=0).stop 2024-07-03T15:47:33.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.364+0000 7f23d80d0640 1 -- 172.21.15.28:0/1068843586 shutdown_connections 2024-07-03T15:47:33.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.364+0000 7f23d80d0640 1 --2- 172.21.15.28:0/1068843586 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f23bc06e1e0 0x7f23bc070690 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:33.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.364+0000 7f23d80d0640 1 --2- 172.21.15.28:0/1068843586 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23d00fd600 0x7f23d006f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:33.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.364+0000 7f23d80d0640 1 --2- 172.21.15.28:0/1068843586 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f23d00fc330 0x7f23d0071380 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:33.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.364+0000 7f23d80d0640 1 -- 172.21.15.28:0/1068843586 >> 172.21.15.28:0/1068843586 conn(0x7f23d00f7af0 msgr2=0x7f23d00f91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:47:33.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.365+0000 7f23d80d0640 1 -- 172.21.15.28:0/1068843586 shutdown_connections 2024-07-03T15:47:33.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.365+0000 7f23d80d0640 1 -- 172.21.15.28:0/1068843586 wait complete. 2024-07-03T15:47:33.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.448+0000 7f71340e8640 1 -- 172.21.15.28:0/11478887 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f712c0fb0d0 msgr2=0x7f712c101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:33.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.448+0000 7f71340e8640 1 --2- 172.21.15.28:0/11478887 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f712c0fb0d0 0x7f712c101900 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f711c0048d0 tx=0x7f711c009da0 comp rx=0 tx=0).stop 2024-07-03T15:47:33.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.448+0000 7f71340e8640 1 -- 172.21.15.28:0/11478887 shutdown_connections 2024-07-03T15:47:33.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.448+0000 7f71340e8640 1 --2- 172.21.15.28:0/11478887 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f712c0fb0d0 0x7f712c101900 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:33.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.448+0000 7f71340e8640 1 --2- 172.21.15.28:0/11478887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f712c0fa760 0x7f712c0fab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:33.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.448+0000 7f71340e8640 1 -- 172.21.15.28:0/11478887 >> 172.21.15.28:0/11478887 conn(0x7f712c0f61a0 msgr2=0x7f712c0f85b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:47:33.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.448+0000 7f71340e8640 1 -- 172.21.15.28:0/11478887 shutdown_connections 2024-07-03T15:47:33.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.448+0000 7f71340e8640 1 -- 172.21.15.28:0/11478887 wait complete. 2024-07-03T15:47:33.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.448+0000 7f71340e8640 1 Processor -- start 2024-07-03T15:47:33.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.449+0000 7f71340e8640 1 -- start start 2024-07-03T15:47:33.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.449+0000 7f71340e8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f712c0fa760 0x7f712c0fd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:47:33.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.449+0000 7f71340e8640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f712c0fb0d0 0x7f712c0fb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:47:33.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.449+0000 7f71340e8640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712c0fbd80 con 0x7f712c0fb0d0 2024-07-03T15:47:33.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.449+0000 7f71340e8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712c0fbec0 con 0x7f712c0fa760 2024-07-03T15:47:33.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.449+0000 7f7131683640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f712c0fb0d0 0x7f712c0fb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:47:33.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.449+0000 7f7131683640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f712c0fb0d0 0x7f712c0fb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48430/0 (socket says 172.21.15.28:48430) 2024-07-03T15:47:33.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.449+0000 7f7131683640 1 -- 172.21.15.28:0/1262471469 learned_addr learned my addr 172.21.15.28:0/1262471469 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:47:33.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.449+0000 7f7131e84640 1 --2- 172.21.15.28:0/1262471469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f712c0fa760 0x7f712c0fd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:47:33.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.449+0000 7f7131683640 1 -- 172.21.15.28:0/1262471469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f712c0fa760 msgr2=0x7f712c0fd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:33.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.449+0000 7f7131683640 1 --2- 172.21.15.28:0/1262471469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f712c0fa760 0x7f712c0fd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:33.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.449+0000 7f7131683640 1 -- 172.21.15.28:0/1262471469 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f711c005040 con 0x7f712c0fb0d0 2024-07-03T15:47:33.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.449+0000 7f7131683640 1 --2- 172.21.15.28:0/1262471469 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f712c0fb0d0 0x7f712c0fb760 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f711c000c00 tx=0x7f711c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:47:33.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.450+0000 7f71227fc640 1 -- 172.21.15.28:0/1262471469 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f711c038070 con 0x7f712c0fb0d0 2024-07-03T15:47:33.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.450+0000 7f71340e8640 1 -- 172.21.15.28:0/1262471469 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f712c0fc140 con 0x7f712c0fb0d0 2024-07-03T15:47:33.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.450+0000 7f71227fc640 1 -- 172.21.15.28:0/1262471469 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f711c032970 con 0x7f712c0fb0d0 2024-07-03T15:47:33.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.450+0000 7f71227fc640 1 -- 172.21.15.28:0/1262471469 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f711c033950 con 0x7f712c0fb0d0 2024-07-03T15:47:33.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.450+0000 7f71340e8640 1 -- 172.21.15.28:0/1262471469 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f712c115090 con 0x7f712c0fb0d0 2024-07-03T15:47:33.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.451+0000 7f71340e8640 1 -- 172.21.15.28:0/1262471469 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f712c0f9e20 con 0x7f712c0fb0d0 2024-07-03T15:47:33.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.451+0000 7f71227fc640 1 -- 172.21.15.28:0/1262471469 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 36) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f711c030020 con 0x7f712c0fb0d0 2024-07-03T15:47:33.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.452+0000 7f71227fc640 1 --2- 172.21.15.28:0/1262471469 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f711806e1e0 0x7f7118070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:47:33.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.452+0000 7f71227fc640 1 -- 172.21.15.28:0/1262471469 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f711c0adcd0 con 0x7f712c0fb0d0 2024-07-03T15:47:33.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.452+0000 7f7131e84640 1 --2- 172.21.15.28:0/1262471469 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f711806e1e0 0x7f7118070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:47:33.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.453+0000 7f7131e84640 1 --2- 172.21.15.28:0/1262471469 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f711806e1e0 0x7f7118070690 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7128006fd0 tx=0x7f7128008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:47:33.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.457+0000 7f71227fc640 1 -- 172.21.15.28:0/1262471469 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f711c076f40 con 0x7f712c0fb0d0 2024-07-03T15:47:33.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:33 smithi028 ceph-mon[92858]: pgmap v40: 65 pgs: 9 stale+active+clean, 56 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-07-03T15:47:33.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:33 smithi028 ceph-mon[92858]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:47:33.623 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:33 smithi028 podman[101271]: 2024-07-03 15:47:33.443236594 +0000 UTC m=+0.407317827 container init 4454cecff8fcd7ef64517202475d69270ead15f9771e4e2f283fdb197351ff30 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240625, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-release-c00506d, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-07-03T15:47:33.623 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:33 smithi028 podman[101271]: 2024-07-03 15:47:33.447284766 +0000 UTC m=+0.411365983 container start 4454cecff8fcd7ef64517202475d69270ead15f9771e4e2f283fdb197351ff30 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-0, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-release-c00506d, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240625, maintainer=Guillaume Abrioux ) 2024-07-03T15:47:33.623 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:33 smithi028 ceph-osd[101325]: -- 172.21.15.28:0/1239847444 <== mon.0 v2:172.21.15.28:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5572d2b5c1e0 con 0x5572d2af4000 2024-07-03T15:47:33.623 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:33 smithi028 bash[101271]: 4454cecff8fcd7ef64517202475d69270ead15f9771e4e2f283fdb197351ff30 2024-07-03T15:47:33.623 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:33 smithi028 systemd[1]: Started Ceph osd.0 for a85e4be6-3951-11ef-bcab-c7b262605968. 2024-07-03T15:47:33.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:33 smithi148 ceph-mon[80490]: pgmap v40: 65 pgs: 9 stale+active+clean, 56 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-07-03T15:47:33.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:33 smithi148 ceph-mon[80490]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:47:33.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.739+0000 7f71340e8640 1 -- 172.21.15.28:0/1262471469 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f712c115220 con 0x7f712c0fb0d0 2024-07-03T15:47:33.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.739+0000 7f71227fc640 1 -- 172.21.15.28:0/1262471469 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+735 (secure 0 0 0) 0x7f711c07a600 con 0x7f712c0fb0d0 2024-07-03T15:47:33.744 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-07-03T15:47:33.744 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": { 2024-07-03T15:47:33.744 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 2 2024-07-03T15:47:33.744 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:47:33.744 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": { 2024-07-03T15:47:33.744 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 2 2024-07-03T15:47:33.744 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:47:33.744 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": { 2024-07-03T15:47:33.745 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 5 2024-07-03T15:47:33.745 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:47:33.745 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": { 2024-07-03T15:47:33.745 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 4 2024-07-03T15:47:33.745 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:47:33.745 INFO:teuthology.orchestra.run.smithi028.stdout: "overall": { 2024-07-03T15:47:33.745 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 9, 2024-07-03T15:47:33.745 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 4 2024-07-03T15:47:33.745 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-07-03T15:47:33.745 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-07-03T15:47:33.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.742+0000 7f71340e8640 1 -- 172.21.15.28:0/1262471469 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f711806e1e0 msgr2=0x7f7118070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:33.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.742+0000 7f71340e8640 1 --2- 172.21.15.28:0/1262471469 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f711806e1e0 0x7f7118070690 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7128006fd0 tx=0x7f7128008040 comp rx=0 tx=0).stop 2024-07-03T15:47:33.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.742+0000 7f71340e8640 1 -- 172.21.15.28:0/1262471469 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f712c0fb0d0 msgr2=0x7f712c0fb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:33.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.742+0000 7f71340e8640 1 --2- 172.21.15.28:0/1262471469 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f712c0fb0d0 0x7f712c0fb760 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f711c000c00 tx=0x7f711c009da0 comp rx=0 tx=0).stop 2024-07-03T15:47:33.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.742+0000 7f71340e8640 1 -- 172.21.15.28:0/1262471469 shutdown_connections 2024-07-03T15:47:33.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.742+0000 7f71340e8640 1 --2- 172.21.15.28:0/1262471469 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f711806e1e0 0x7f7118070690 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:33.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.742+0000 7f71340e8640 1 --2- 172.21.15.28:0/1262471469 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f712c0fb0d0 0x7f712c0fb760 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:33.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.742+0000 7f71340e8640 1 --2- 172.21.15.28:0/1262471469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f712c0fa760 0x7f712c0fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:33.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.742+0000 7f71340e8640 1 -- 172.21.15.28:0/1262471469 >> 172.21.15.28:0/1262471469 conn(0x7f712c0f61a0 msgr2=0x7f712c0ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:47:33.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.742+0000 7f71340e8640 1 -- 172.21.15.28:0/1262471469 shutdown_connections 2024-07-03T15:47:33.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.742+0000 7f71340e8640 1 -- 172.21.15.28:0/1262471469 wait complete. 2024-07-03T15:47:33.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.828+0000 7efdadfef640 1 -- 172.21.15.28:0/3737099290 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efda80fe310 msgr2=0x7efda80fa470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:33.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.828+0000 7efdadfef640 1 --2- 172.21.15.28:0/3737099290 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efda80fe310 0x7efda80fa470 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0048d0 tx=0x7efd9c009da0 comp rx=0 tx=0).stop 2024-07-03T15:47:33.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.828+0000 7efdadfef640 1 -- 172.21.15.28:0/3737099290 shutdown_connections 2024-07-03T15:47:33.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.828+0000 7efdadfef640 1 --2- 172.21.15.28:0/3737099290 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efda80fe310 0x7efda80fa470 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:33.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.828+0000 7efdadfef640 1 --2- 172.21.15.28:0/3737099290 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efda80fd9c0 0x7efda80fddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:33.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.828+0000 7efdadfef640 1 -- 172.21.15.28:0/3737099290 >> 172.21.15.28:0/3737099290 conn(0x7efda80f61c0 msgr2=0x7efda80f85f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:47:33.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.828+0000 7efdadfef640 1 -- 172.21.15.28:0/3737099290 shutdown_connections 2024-07-03T15:47:33.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.828+0000 7efdadfef640 1 -- 172.21.15.28:0/3737099290 wait complete. 2024-07-03T15:47:33.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.829+0000 7efdadfef640 1 Processor -- start 2024-07-03T15:47:33.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.829+0000 7efdadfef640 1 -- start start 2024-07-03T15:47:33.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.829+0000 7efdadfef640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efda80fd9c0 0x7efda81137a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:47:33.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.829+0000 7efdadfef640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efda8113ce0 0x7efda8114b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:47:33.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.830+0000 7efdadfef640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda81141a0 con 0x7efda8113ce0 2024-07-03T15:47:33.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.830+0000 7efdadfef640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda81142e0 con 0x7efda80fd9c0 2024-07-03T15:47:33.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.830+0000 7efda6ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efda8113ce0 0x7efda8114b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:47:33.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.830+0000 7efda6ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efda8113ce0 0x7efda8114b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48448/0 (socket says 172.21.15.28:48448) 2024-07-03T15:47:33.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.830+0000 7efda6ffd640 1 -- 172.21.15.28:0/1157980519 learned_addr learned my addr 172.21.15.28:0/1157980519 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:47:33.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.830+0000 7efda77fe640 1 --2- 172.21.15.28:0/1157980519 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efda80fd9c0 0x7efda81137a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:47:33.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.830+0000 7efda6ffd640 1 -- 172.21.15.28:0/1157980519 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efda80fd9c0 msgr2=0x7efda81137a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:33.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.830+0000 7efda6ffd640 1 --2- 172.21.15.28:0/1157980519 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efda80fd9c0 0x7efda81137a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:33.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.830+0000 7efda6ffd640 1 -- 172.21.15.28:0/1157980519 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd9c005040 con 0x7efda8113ce0 2024-07-03T15:47:33.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.830+0000 7efda77fe640 1 --2- 172.21.15.28:0/1157980519 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efda80fd9c0 0x7efda81137a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-03T15:47:33.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.830+0000 7efda6ffd640 1 --2- 172.21.15.28:0/1157980519 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efda8113ce0 0x7efda8114b00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0048a0 tx=0x7efd9c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:47:33.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.831+0000 7efd8ffff640 1 -- 172.21.15.28:0/1157980519 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd9c038070 con 0x7efda8113ce0 2024-07-03T15:47:33.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.831+0000 7efdadfef640 1 -- 172.21.15.28:0/1157980519 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efda81150a0 con 0x7efda8113ce0 2024-07-03T15:47:33.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.831+0000 7efd8ffff640 1 -- 172.21.15.28:0/1157980519 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efd9c009890 con 0x7efda8113ce0 2024-07-03T15:47:33.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.831+0000 7efd8ffff640 1 -- 172.21.15.28:0/1157980519 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd9c0337a0 con 0x7efda8113ce0 2024-07-03T15:47:33.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.831+0000 7efdadfef640 1 -- 172.21.15.28:0/1157980519 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efda81154f0 con 0x7efda8113ce0 2024-07-03T15:47:33.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.832+0000 7efdadfef640 1 -- 172.21.15.28:0/1157980519 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efda80f9e60 con 0x7efda8113ce0 2024-07-03T15:47:33.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.833+0000 7efd8ffff640 1 -- 172.21.15.28:0/1157980519 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 36) v1 ==== 92305+0+0 (secure 0 0 0) 0x7efd9c030020 con 0x7efda8113ce0 2024-07-03T15:47:33.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.833+0000 7efd8ffff640 1 --2- 172.21.15.28:0/1157980519 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7efd9006e1e0 0x7efd90070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:47:33.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.833+0000 7efd8ffff640 1 -- 172.21.15.28:0/1157980519 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6410+0+0 (secure 0 0 0) 0x7efd9c0ad900 con 0x7efda8113ce0 2024-07-03T15:47:33.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.834+0000 7efda77fe640 1 --2- 172.21.15.28:0/1157980519 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7efd9006e1e0 0x7efd90070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:47:33.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.834+0000 7efda77fe640 1 --2- 172.21.15.28:0/1157980519 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7efd9006e1e0 0x7efd90070690 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7efd98007fd0 tx=0x7efd98009e90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:47:33.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:33.838+0000 7efd8ffff640 1 -- 172.21.15.28:0/1157980519 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7efd9c076ad0 con 0x7efda8113ce0 2024-07-03T15:47:34.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.083+0000 7efdadfef640 1 -- 172.21.15.28:0/1157980519 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7efda804b530 con 0x7efda8113ce0 2024-07-03T15:47:34.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.084+0000 7efd8ffff640 1 -- 172.21.15.28:0/1157980519 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1908 (secure 0 0 0) 0x7efd9c07a190 con 0x7efda8113ce0 2024-07-03T15:47:34.089 INFO:teuthology.orchestra.run.smithi028.stdout:e17 2024-07-03T15:47:34.089 INFO:teuthology.orchestra.run.smithi028.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-03T15:47:34.089 INFO:teuthology.orchestra.run.smithi028.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:47:34.089 INFO:teuthology.orchestra.run.smithi028.stdout:legacy client fscid: 1 2024-07-03T15:47:34.089 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:47:34.089 INFO:teuthology.orchestra.run.smithi028.stdout:Filesystem 'cephfs' (1) 2024-07-03T15:47:34.089 INFO:teuthology.orchestra.run.smithi028.stdout:fs_name cephfs 2024-07-03T15:47:34.089 INFO:teuthology.orchestra.run.smithi028.stdout:epoch 16 2024-07-03T15:47:34.089 INFO:teuthology.orchestra.run.smithi028.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-03T15:47:34.089 INFO:teuthology.orchestra.run.smithi028.stdout:created 2024-07-03T15:41:40.656298+0000 2024-07-03T15:47:34.090 INFO:teuthology.orchestra.run.smithi028.stdout:modified 2024-07-03T15:41:56.454984+0000 2024-07-03T15:47:34.090 INFO:teuthology.orchestra.run.smithi028.stdout:tableserver 0 2024-07-03T15:47:34.090 INFO:teuthology.orchestra.run.smithi028.stdout:root 0 2024-07-03T15:47:34.090 INFO:teuthology.orchestra.run.smithi028.stdout:session_timeout 60 2024-07-03T15:47:34.090 INFO:teuthology.orchestra.run.smithi028.stdout:session_autoclose 300 2024-07-03T15:47:34.090 INFO:teuthology.orchestra.run.smithi028.stdout:max_file_size 1099511627776 2024-07-03T15:47:34.090 INFO:teuthology.orchestra.run.smithi028.stdout:max_xattr_size 65536 2024-07-03T15:47:34.090 INFO:teuthology.orchestra.run.smithi028.stdout:required_client_features {} 2024-07-03T15:47:34.090 INFO:teuthology.orchestra.run.smithi028.stdout:last_failure 0 2024-07-03T15:47:34.090 INFO:teuthology.orchestra.run.smithi028.stdout:last_failure_osd_epoch 0 2024-07-03T15:47:34.091 INFO:teuthology.orchestra.run.smithi028.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:47:34.091 INFO:teuthology.orchestra.run.smithi028.stdout:max_mds 1 2024-07-03T15:47:34.091 INFO:teuthology.orchestra.run.smithi028.stdout:in 0 2024-07-03T15:47:34.091 INFO:teuthology.orchestra.run.smithi028.stdout:up {0=14450} 2024-07-03T15:47:34.091 INFO:teuthology.orchestra.run.smithi028.stdout:failed 2024-07-03T15:47:34.091 INFO:teuthology.orchestra.run.smithi028.stdout:damaged 2024-07-03T15:47:34.091 INFO:teuthology.orchestra.run.smithi028.stdout:stopped 2024-07-03T15:47:34.091 INFO:teuthology.orchestra.run.smithi028.stdout:data_pools [3] 2024-07-03T15:47:34.091 INFO:teuthology.orchestra.run.smithi028.stdout:metadata_pool 2 2024-07-03T15:47:34.091 INFO:teuthology.orchestra.run.smithi028.stdout:inline_data disabled 2024-07-03T15:47:34.092 INFO:teuthology.orchestra.run.smithi028.stdout:balancer 2024-07-03T15:47:34.092 INFO:teuthology.orchestra.run.smithi028.stdout:bal_rank_mask -1 2024-07-03T15:47:34.092 INFO:teuthology.orchestra.run.smithi028.stdout:standby_count_wanted 1 2024-07-03T15:47:34.092 INFO:teuthology.orchestra.run.smithi028.stdout:qdb_cluster leader: 0 members: 2024-07-03T15:47:34.092 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi028.rhpvve{0:14450} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.28:6826/1613936042,v1:172.21.15.28:6827/1613936042] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:47:34.092 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi028.wmbxew{0:14460} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.28:6828/1730336821,v1:172.21.15.28:6829/1730336821] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:47:34.092 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:47:34.092 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:47:34.092 INFO:teuthology.orchestra.run.smithi028.stdout:Standby daemons: 2024-07-03T15:47:34.092 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:47:34.092 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi148.szomfl{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/4171231858,v1:172.21.15.148:6827/4171231858] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:47:34.092 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi148.xgpuld{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/2347536398,v1:172.21.15.148:6825/2347536398] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:47:34.093 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 17 2024-07-03T15:47:34.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.086+0000 7efdadfef640 1 -- 172.21.15.28:0/1157980519 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7efd9006e1e0 msgr2=0x7efd90070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:34.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.086+0000 7efdadfef640 1 --2- 172.21.15.28:0/1157980519 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7efd9006e1e0 0x7efd90070690 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7efd98007fd0 tx=0x7efd98009e90 comp rx=0 tx=0).stop 2024-07-03T15:47:34.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.086+0000 7efdadfef640 1 -- 172.21.15.28:0/1157980519 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efda8113ce0 msgr2=0x7efda8114b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:34.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.086+0000 7efdadfef640 1 --2- 172.21.15.28:0/1157980519 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efda8113ce0 0x7efda8114b00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0048a0 tx=0x7efd9c009da0 comp rx=0 tx=0).stop 2024-07-03T15:47:34.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.086+0000 7efdadfef640 1 -- 172.21.15.28:0/1157980519 shutdown_connections 2024-07-03T15:47:34.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.086+0000 7efdadfef640 1 --2- 172.21.15.28:0/1157980519 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7efd9006e1e0 0x7efd90070690 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:34.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.086+0000 7efdadfef640 1 --2- 172.21.15.28:0/1157980519 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efda8113ce0 0x7efda8114b00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:34.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.086+0000 7efdadfef640 1 --2- 172.21.15.28:0/1157980519 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efda80fd9c0 0x7efda81137a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:34.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.086+0000 7efdadfef640 1 -- 172.21.15.28:0/1157980519 >> 172.21.15.28:0/1157980519 conn(0x7efda80f61c0 msgr2=0x7efda80f7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:47:34.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.086+0000 7efdadfef640 1 -- 172.21.15.28:0/1157980519 shutdown_connections 2024-07-03T15:47:34.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.086+0000 7efdadfef640 1 -- 172.21.15.28:0/1157980519 wait complete. 2024-07-03T15:47:34.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.171+0000 7f1bc79d3640 1 -- 172.21.15.28:0/2166877049 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1bc0065a50 msgr2=0x7f1bc00fef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:34.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.171+0000 7f1bc79d3640 1 --2- 172.21.15.28:0/2166877049 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1bc0065a50 0x7f1bc00fef50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc0048d0 tx=0x7f1bbc009da0 comp rx=0 tx=0).stop 2024-07-03T15:47:34.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.171+0000 7f1bc79d3640 1 -- 172.21.15.28:0/2166877049 shutdown_connections 2024-07-03T15:47:34.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.171+0000 7f1bc79d3640 1 --2- 172.21.15.28:0/2166877049 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bc00ff490 0x7f1bc0101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:34.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.171+0000 7f1bc79d3640 1 --2- 172.21.15.28:0/2166877049 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1bc0065a50 0x7f1bc00fef50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:34.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.171+0000 7f1bc79d3640 1 -- 172.21.15.28:0/2166877049 >> 172.21.15.28:0/2166877049 conn(0x7f1bc00f6a20 msgr2=0x7f1bc00f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:47:34.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.171+0000 7f1bc79d3640 1 -- 172.21.15.28:0/2166877049 shutdown_connections 2024-07-03T15:47:34.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.171+0000 7f1bc79d3640 1 -- 172.21.15.28:0/2166877049 wait complete. 2024-07-03T15:47:34.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.172+0000 7f1bc79d3640 1 Processor -- start 2024-07-03T15:47:34.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.172+0000 7f1bc79d3640 1 -- start start 2024-07-03T15:47:34.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.172+0000 7f1bc79d3640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1bc0065a50 0x7f1bc010cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:47:34.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.172+0000 7f1bc79d3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bc00ff490 0x7f1bc010d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:47:34.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.172+0000 7f1bc79d3640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc010db40 con 0x7f1bc0065a50 2024-07-03T15:47:34.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.172+0000 7f1bc79d3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc010e360 con 0x7f1bc00ff490 2024-07-03T15:47:34.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.172+0000 7f1bc576f640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1bc0065a50 0x7f1bc010cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:47:34.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.172+0000 7f1bc576f640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1bc0065a50 0x7f1bc010cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48468/0 (socket says 172.21.15.28:48468) 2024-07-03T15:47:34.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.172+0000 7f1bc576f640 1 -- 172.21.15.28:0/928975598 learned_addr learned my addr 172.21.15.28:0/928975598 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:47:34.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.173+0000 7f1bc4f6e640 1 --2- 172.21.15.28:0/928975598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bc00ff490 0x7f1bc010d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:47:34.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.173+0000 7f1bc576f640 1 -- 172.21.15.28:0/928975598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bc00ff490 msgr2=0x7f1bc010d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:34.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.173+0000 7f1bc576f640 1 --2- 172.21.15.28:0/928975598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bc00ff490 0x7f1bc010d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:34.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.173+0000 7f1bc576f640 1 -- 172.21.15.28:0/928975598 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bbc005040 con 0x7f1bc0065a50 2024-07-03T15:47:34.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.173+0000 7f1bc576f640 1 --2- 172.21.15.28:0/928975598 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1bc0065a50 0x7f1bc010cfb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc000c00 tx=0x7f1bbc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:47:34.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.173+0000 7f1bb5ffb640 1 -- 172.21.15.28:0/928975598 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1bbc038070 con 0x7f1bc0065a50 2024-07-03T15:47:34.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.173+0000 7f1bc79d3640 1 -- 172.21.15.28:0/928975598 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bc010e590 con 0x7f1bc0065a50 2024-07-03T15:47:34.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.173+0000 7f1bb5ffb640 1 -- 172.21.15.28:0/928975598 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1bbc032970 con 0x7f1bc0065a50 2024-07-03T15:47:34.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.173+0000 7f1bb5ffb640 1 -- 172.21.15.28:0/928975598 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1bbc0338e0 con 0x7f1bc0065a50 2024-07-03T15:47:34.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.173+0000 7f1bc79d3640 1 -- 172.21.15.28:0/928975598 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bc010ead0 con 0x7f1bc0065a50 2024-07-03T15:47:34.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.174+0000 7f1bc79d3640 1 -- 172.21.15.28:0/928975598 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bc00fa6a0 con 0x7f1bc0065a50 2024-07-03T15:47:34.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.176+0000 7f1bb5ffb640 1 -- 172.21.15.28:0/928975598 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 36) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f1bbc030020 con 0x7f1bc0065a50 2024-07-03T15:47:34.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.176+0000 7f1bb5ffb640 1 --2- 172.21.15.28:0/928975598 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f1bac06e1e0 0x7f1bac070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:47:34.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.176+0000 7f1bb5ffb640 1 -- 172.21.15.28:0/928975598 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f1bbc0adb00 con 0x7f1bc0065a50 2024-07-03T15:47:34.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.176+0000 7f1bc4f6e640 1 --2- 172.21.15.28:0/928975598 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f1bac06e1e0 0x7f1bac070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:47:34.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.177+0000 7f1bc4f6e640 1 --2- 172.21.15.28:0/928975598 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f1bac06e1e0 0x7f1bac070690 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1bc010cd60 tx=0x7f1bb000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:47:34.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.181+0000 7f1bb5ffb640 1 -- 172.21.15.28:0/928975598 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f1bbc076e20 con 0x7f1bc0065a50 2024-07-03T15:47:34.289 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:33 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-0[101308]: 2024-07-03T15:47:33.991+0000 7fe93d13c640 -1 Falling back to public interface 2024-07-03T15:47:34.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.387+0000 7f1bc79d3640 1 -- 172.21.15.28:0/928975598 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1bc010ee00 con 0x7f1bac06e1e0 2024-07-03T15:47:34.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.389+0000 7f1bb5ffb640 1 -- 172.21.15.28:0/928975598 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f1bc010ee00 con 0x7f1bac06e1e0 2024-07-03T15:47:34.394 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-07-03T15:47:34.394 INFO:teuthology.orchestra.run.smithi028.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2", 2024-07-03T15:47:34.394 INFO:teuthology.orchestra.run.smithi028.stdout: "in_progress": true, 2024-07-03T15:47:34.394 INFO:teuthology.orchestra.run.smithi028.stdout: "which": "Upgrading all daemon types on all hosts", 2024-07-03T15:47:34.394 INFO:teuthology.orchestra.run.smithi028.stdout: "services_complete": [ 2024-07-03T15:47:34.394 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr", 2024-07-03T15:47:34.394 INFO:teuthology.orchestra.run.smithi028.stdout: "crash", 2024-07-03T15:47:34.394 INFO:teuthology.orchestra.run.smithi028.stdout: "mon" 2024-07-03T15:47:34.394 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-07-03T15:47:34.394 INFO:teuthology.orchestra.run.smithi028.stdout: "progress": "6/21 daemons upgraded", 2024-07-03T15:47:34.394 INFO:teuthology.orchestra.run.smithi028.stdout: "message": "Currently upgrading osd daemons", 2024-07-03T15:47:34.394 INFO:teuthology.orchestra.run.smithi028.stdout: "is_paused": false 2024-07-03T15:47:34.394 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-07-03T15:47:34.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.392+0000 7f1bc79d3640 1 -- 172.21.15.28:0/928975598 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f1bac06e1e0 msgr2=0x7f1bac070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:34.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.392+0000 7f1bc79d3640 1 --2- 172.21.15.28:0/928975598 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f1bac06e1e0 0x7f1bac070690 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1bc010cd60 tx=0x7f1bb000b040 comp rx=0 tx=0).stop 2024-07-03T15:47:34.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.392+0000 7f1bc79d3640 1 -- 172.21.15.28:0/928975598 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1bc0065a50 msgr2=0x7f1bc010cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:34.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.392+0000 7f1bc79d3640 1 --2- 172.21.15.28:0/928975598 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1bc0065a50 0x7f1bc010cfb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc000c00 tx=0x7f1bbc009da0 comp rx=0 tx=0).stop 2024-07-03T15:47:34.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.392+0000 7f1bc79d3640 1 -- 172.21.15.28:0/928975598 shutdown_connections 2024-07-03T15:47:34.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.392+0000 7f1bc79d3640 1 --2- 172.21.15.28:0/928975598 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f1bac06e1e0 0x7f1bac070690 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:34.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.392+0000 7f1bc79d3640 1 --2- 172.21.15.28:0/928975598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bc00ff490 0x7f1bc010d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:34.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.392+0000 7f1bc79d3640 1 --2- 172.21.15.28:0/928975598 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1bc0065a50 0x7f1bc010cfb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:34.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.392+0000 7f1bc79d3640 1 -- 172.21.15.28:0/928975598 >> 172.21.15.28:0/928975598 conn(0x7f1bc00f6a20 msgr2=0x7f1bc00f6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:47:34.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.392+0000 7f1bc79d3640 1 -- 172.21.15.28:0/928975598 shutdown_connections 2024-07-03T15:47:34.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.392+0000 7f1bc79d3640 1 -- 172.21.15.28:0/928975598 wait complete. 2024-07-03T15:47:34.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.475+0000 7f4a1ca02640 1 -- 172.21.15.28:0/830643380 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4a180ff360 msgr2=0x7f4a18101740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:34.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.475+0000 7f4a1ca02640 1 --2- 172.21.15.28:0/830643380 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4a180ff360 0x7f4a18101740 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4a000048d0 tx=0x7f4a00009da0 comp rx=0 tx=0).stop 2024-07-03T15:47:34.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.475+0000 7f4a1ca02640 1 -- 172.21.15.28:0/830643380 shutdown_connections 2024-07-03T15:47:34.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.475+0000 7f4a1ca02640 1 --2- 172.21.15.28:0/830643380 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4a180ff360 0x7f4a18101740 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:34.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.475+0000 7f4a1ca02640 1 --2- 172.21.15.28:0/830643380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a18065880 0x7f4a180fee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:34.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.475+0000 7f4a1ca02640 1 -- 172.21.15.28:0/830643380 >> 172.21.15.28:0/830643380 conn(0x7f4a180f6850 msgr2=0x7f4a180f8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:47:34.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.475+0000 7f4a1ca02640 1 -- 172.21.15.28:0/830643380 shutdown_connections 2024-07-03T15:47:34.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.475+0000 7f4a1ca02640 1 -- 172.21.15.28:0/830643380 wait complete. 2024-07-03T15:47:34.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.475+0000 7f4a1ca02640 1 Processor -- start 2024-07-03T15:47:34.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.475+0000 7f4a1ca02640 1 -- start start 2024-07-03T15:47:34.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.475+0000 7f4a1ca02640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a18065880 0x7f4a180fc830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:47:34.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.475+0000 7f4a1ca02640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4a180ff360 0x7f4a180fad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:47:34.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.476+0000 7f4a1ca02640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a180fb2b0 con 0x7f4a180ff360 2024-07-03T15:47:34.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.476+0000 7f4a1ca02640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a180fb3f0 con 0x7f4a18065880 2024-07-03T15:47:34.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.476+0000 7f4a0ffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4a180ff360 0x7f4a180fad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:47:34.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.476+0000 7f4a0ffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4a180ff360 0x7f4a180fad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48482/0 (socket says 172.21.15.28:48482) 2024-07-03T15:47:34.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.476+0000 7f4a0ffff640 1 -- 172.21.15.28:0/516339870 learned_addr learned my addr 172.21.15.28:0/516339870 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:47:34.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.476+0000 7f4a16d9d640 1 --2- 172.21.15.28:0/516339870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a18065880 0x7f4a180fc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:47:34.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.476+0000 7f4a0ffff640 1 -- 172.21.15.28:0/516339870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a18065880 msgr2=0x7f4a180fc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:34.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.476+0000 7f4a0ffff640 1 --2- 172.21.15.28:0/516339870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a18065880 0x7f4a180fc830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:34.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.476+0000 7f4a0ffff640 1 -- 172.21.15.28:0/516339870 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a00005040 con 0x7f4a180ff360 2024-07-03T15:47:34.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.476+0000 7f4a0ffff640 1 --2- 172.21.15.28:0/516339870 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4a180ff360 0x7f4a180fad20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4a00000c00 tx=0x7f4a00009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:47:34.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.476+0000 7f4a0f7fe640 1 -- 172.21.15.28:0/516339870 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a00038070 con 0x7f4a180ff360 2024-07-03T15:47:34.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.476+0000 7f4a0f7fe640 1 -- 172.21.15.28:0/516339870 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4a00032970 con 0x7f4a180ff360 2024-07-03T15:47:34.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.476+0000 7f4a1ca02640 1 -- 172.21.15.28:0/516339870 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a180fb670 con 0x7f4a180ff360 2024-07-03T15:47:34.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.476+0000 7f4a0f7fe640 1 -- 172.21.15.28:0/516339870 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a000338e0 con 0x7f4a180ff360 2024-07-03T15:47:34.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.477+0000 7f4a1ca02640 1 -- 172.21.15.28:0/516339870 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a18114e80 con 0x7f4a180ff360 2024-07-03T15:47:34.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.478+0000 7f4a1ca02640 1 -- 172.21.15.28:0/516339870 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a1805eb80 con 0x7f4a180ff360 2024-07-03T15:47:34.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.478+0000 7f4a0f7fe640 1 -- 172.21.15.28:0/516339870 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 36) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f4a00030020 con 0x7f4a180ff360 2024-07-03T15:47:34.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.478+0000 7f4a0f7fe640 1 --2- 172.21.15.28:0/516339870 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f4a0406e1d0 0x7f4a04070680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:47:34.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.479+0000 7f4a0f7fe640 1 -- 172.21.15.28:0/516339870 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f4a000adaa0 con 0x7f4a180ff360 2024-07-03T15:47:34.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.479+0000 7f4a16d9d640 1 --2- 172.21.15.28:0/516339870 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f4a0406e1d0 0x7f4a04070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:47:34.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.479+0000 7f4a16d9d640 1 --2- 172.21.15.28:0/516339870 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f4a0406e1d0 0x7f4a04070680 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4a08009fd0 tx=0x7f4a08009e70 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:47:34.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.483+0000 7f4a0f7fe640 1 -- 172.21.15.28:0/516339870 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f4a00076d90 con 0x7f4a180ff360 2024-07-03T15:47:34.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.765+0000 7f4a1ca02640 1 -- 172.21.15.28:0/516339870 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4a18104cf0 con 0x7f4a180ff360 2024-07-03T15:47:34.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.766+0000 7f4a0f7fe640 1 -- 172.21.15.28:0/516339870 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1216 (secure 0 0 0) 0x7f4a000414d0 con 0x7f4a180ff360 2024-07-03T15:47:34.770 INFO:teuthology.orchestra.run.smithi028.stdout:HEALTH_WARN 1 osds down; Degraded data redundancy: 45/219 objects degraded (20.548%), 17 pgs degraded 2024-07-03T15:47:34.770 INFO:teuthology.orchestra.run.smithi028.stdout:[WRN] OSD_DOWN: 1 osds down 2024-07-03T15:47:34.770 INFO:teuthology.orchestra.run.smithi028.stdout: osd.0 (root=default,host=smithi028) is down 2024-07-03T15:47:34.770 INFO:teuthology.orchestra.run.smithi028.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 45/219 objects degraded (20.548%), 17 pgs degraded 2024-07-03T15:47:34.770 INFO:teuthology.orchestra.run.smithi028.stdout: pg 1.0 is active+undersized+degraded, acting [3,1] 2024-07-03T15:47:34.770 INFO:teuthology.orchestra.run.smithi028.stdout: pg 2.0 is active+undersized+degraded, acting [3,1] 2024-07-03T15:47:34.771 INFO:teuthology.orchestra.run.smithi028.stdout: pg 2.1 is active+undersized+degraded, acting [2,1] 2024-07-03T15:47:34.771 INFO:teuthology.orchestra.run.smithi028.stdout: pg 2.2 is active+undersized+degraded, acting [5,1] 2024-07-03T15:47:34.771 INFO:teuthology.orchestra.run.smithi028.stdout: pg 2.4 is active+undersized+degraded, acting [1,4] 2024-07-03T15:47:34.771 INFO:teuthology.orchestra.run.smithi028.stdout: pg 2.5 is active+undersized+degraded, acting [3,4] 2024-07-03T15:47:34.771 INFO:teuthology.orchestra.run.smithi028.stdout: pg 2.9 is active+undersized+degraded, acting [1,4] 2024-07-03T15:47:34.771 INFO:teuthology.orchestra.run.smithi028.stdout: pg 2.c is active+undersized+degraded, acting [2,3] 2024-07-03T15:47:34.771 INFO:teuthology.orchestra.run.smithi028.stdout: pg 2.e is active+undersized+degraded, acting [2,3] 2024-07-03T15:47:34.771 INFO:teuthology.orchestra.run.smithi028.stdout: pg 2.f is active+undersized+degraded, acting [4,5] 2024-07-03T15:47:34.771 INFO:teuthology.orchestra.run.smithi028.stdout: pg 2.10 is active+undersized+degraded, acting [2,1] 2024-07-03T15:47:34.771 INFO:teuthology.orchestra.run.smithi028.stdout: pg 2.13 is active+undersized+degraded, acting [4,2] 2024-07-03T15:47:34.771 INFO:teuthology.orchestra.run.smithi028.stdout: pg 2.15 is active+undersized+degraded, acting [1,3] 2024-07-03T15:47:34.771 INFO:teuthology.orchestra.run.smithi028.stdout: pg 2.1b is active+undersized+degraded, acting [1,5] 2024-07-03T15:47:34.771 INFO:teuthology.orchestra.run.smithi028.stdout: pg 2.1d is active+undersized+degraded, acting [3,5] 2024-07-03T15:47:34.771 INFO:teuthology.orchestra.run.smithi028.stdout: pg 2.1e is active+undersized+degraded, acting [2,5] 2024-07-03T15:47:34.771 INFO:teuthology.orchestra.run.smithi028.stdout: pg 2.1f is active+undersized+degraded, acting [3,4] 2024-07-03T15:47:34.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.768+0000 7f4a1ca02640 1 -- 172.21.15.28:0/516339870 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f4a0406e1d0 msgr2=0x7f4a04070680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:34.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.768+0000 7f4a1ca02640 1 --2- 172.21.15.28:0/516339870 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f4a0406e1d0 0x7f4a04070680 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4a08009fd0 tx=0x7f4a08009e70 comp rx=0 tx=0).stop 2024-07-03T15:47:34.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.768+0000 7f4a1ca02640 1 -- 172.21.15.28:0/516339870 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4a180ff360 msgr2=0x7f4a180fad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:47:34.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.768+0000 7f4a1ca02640 1 --2- 172.21.15.28:0/516339870 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4a180ff360 0x7f4a180fad20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4a00000c00 tx=0x7f4a00009da0 comp rx=0 tx=0).stop 2024-07-03T15:47:34.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.768+0000 7f4a1ca02640 1 -- 172.21.15.28:0/516339870 shutdown_connections 2024-07-03T15:47:34.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.768+0000 7f4a1ca02640 1 --2- 172.21.15.28:0/516339870 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f4a0406e1d0 0x7f4a04070680 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:34.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.768+0000 7f4a1ca02640 1 --2- 172.21.15.28:0/516339870 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4a180ff360 0x7f4a180fad20 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:34.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.768+0000 7f4a1ca02640 1 --2- 172.21.15.28:0/516339870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a18065880 0x7f4a180fc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:47:34.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.768+0000 7f4a1ca02640 1 -- 172.21.15.28:0/516339870 >> 172.21.15.28:0/516339870 conn(0x7f4a180f6850 msgr2=0x7f4a180f6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:47:34.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.768+0000 7f4a1ca02640 1 -- 172.21.15.28:0/516339870 shutdown_connections 2024-07-03T15:47:34.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:47:34.768+0000 7f4a1ca02640 1 -- 172.21.15.28:0/516339870 wait complete. 2024-07-03T15:47:34.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:34 smithi028 ceph-mon[92858]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:47:34.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:34 smithi028 ceph-mon[92858]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:47:34.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:34 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:34.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:34 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:34.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:34 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:47:34.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:34 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/1262471469' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:47:34.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:34 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/1157980519' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-03T15:47:34.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:34 smithi148 ceph-mon[80490]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:47:34.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:34 smithi148 ceph-mon[80490]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:47:34.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:34 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:34.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:34 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:34.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:34 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:47:34.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:34 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/1262471469' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:47:34.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:34 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/1157980519' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-03T15:47:35.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:35 smithi028 ceph-mon[92858]: pgmap v41: 65 pgs: 17 active+undersized, 17 active+undersized+degraded, 31 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 45/219 objects degraded (20.548%) 2024-07-03T15:47:35.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:35 smithi028 ceph-mon[92858]: from='client.34210 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:47:35.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:35 smithi028 ceph-mon[92858]: Health check failed: Degraded data redundancy: 45/219 objects degraded (20.548%), 17 pgs degraded (PG_DEGRADED) 2024-07-03T15:47:35.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:35 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/516339870' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-03T15:47:35.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:35 smithi148 ceph-mon[80490]: pgmap v41: 65 pgs: 17 active+undersized, 17 active+undersized+degraded, 31 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 45/219 objects degraded (20.548%) 2024-07-03T15:47:35.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:35 smithi148 ceph-mon[80490]: from='client.34210 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:47:35.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:35 smithi148 ceph-mon[80490]: Health check failed: Degraded data redundancy: 45/219 objects degraded (20.548%), 17 pgs degraded (PG_DEGRADED) 2024-07-03T15:47:35.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:35 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/516339870' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-03T15:47:37.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:37 smithi028 ceph-mon[92858]: pgmap v42: 65 pgs: 17 active+undersized, 17 active+undersized+degraded, 31 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 639 B/s rd, 0 op/s; 45/219 objects degraded (20.548%) 2024-07-03T15:47:37.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:37 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:37.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:37 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:47:37.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:37 smithi148 ceph-mon[80490]: pgmap v42: 65 pgs: 17 active+undersized, 17 active+undersized+degraded, 31 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 639 B/s rd, 0 op/s; 45/219 objects degraded (20.548%) 2024-07-03T15:47:37.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:37 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:37.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:37 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:47:39.314 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:39 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-0[101308]: 2024-07-03T15:47:39.059+0000 7fe93d13c640 -1 osd.0 48 log_to_monitors true 2024-07-03T15:47:39.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:39 smithi028 ceph-mon[92858]: pgmap v43: 65 pgs: 17 active+undersized, 17 active+undersized+degraded, 31 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.0 KiB/s rd, 1 op/s; 45/219 objects degraded (20.548%) 2024-07-03T15:47:39.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:39 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:39.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:39 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:39.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:39 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:39.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:39 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:39.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:39 smithi028 ceph-mon[92858]: from='osd.0 [v2:172.21.15.28:6802/1756284953,v1:172.21.15.28:6803/1756284953]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-07-03T15:47:39.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:39 smithi148 ceph-mon[80490]: pgmap v43: 65 pgs: 17 active+undersized, 17 active+undersized+degraded, 31 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.0 KiB/s rd, 1 op/s; 45/219 objects degraded (20.548%) 2024-07-03T15:47:39.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:39 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:39.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:39 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:39.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:39 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:39.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:39 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:39.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:39 smithi148 ceph-mon[80490]: from='osd.0 [v2:172.21.15.28:6802/1756284953,v1:172.21.15.28:6803/1756284953]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-07-03T15:47:41.003 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:47:40 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-0[101308]: 2024-07-03T15:47:40.772+0000 7fe930ece640 -1 osd.0 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-03T15:47:41.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:41 smithi028 ceph-mon[92858]: from='osd.0 [v2:172.21.15.28:6802/1756284953,v1:172.21.15.28:6803/1756284953]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-07-03T15:47:41.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:41 smithi028 ceph-mon[92858]: osdmap e51: 6 total, 5 up, 6 in 2024-07-03T15:47:41.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:41 smithi028 ceph-mon[92858]: from='osd.0 [v2:172.21.15.28:6802/1756284953,v1:172.21.15.28:6803/1756284953]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi028", "root=default"]}]: dispatch 2024-07-03T15:47:41.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:41 smithi028 ceph-mon[92858]: pgmap v45: 65 pgs: 17 active+undersized, 17 active+undersized+degraded, 31 active+clean; 193 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 45/219 objects degraded (20.548%) 2024-07-03T15:47:41.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:41 smithi028 ceph-mon[92858]: from='osd.0 [v2:172.21.15.28:6802/1756284953,v1:172.21.15.28:6803/1756284953]' entity='osd.0' 2024-07-03T15:47:41.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:41 smithi148 ceph-mon[80490]: from='osd.0 [v2:172.21.15.28:6802/1756284953,v1:172.21.15.28:6803/1756284953]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-07-03T15:47:41.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:41 smithi148 ceph-mon[80490]: osdmap e51: 6 total, 5 up, 6 in 2024-07-03T15:47:41.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:41 smithi148 ceph-mon[80490]: from='osd.0 [v2:172.21.15.28:6802/1756284953,v1:172.21.15.28:6803/1756284953]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi028", "root=default"]}]: dispatch 2024-07-03T15:47:41.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:41 smithi148 ceph-mon[80490]: pgmap v45: 65 pgs: 17 active+undersized, 17 active+undersized+degraded, 31 active+clean; 193 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 45/219 objects degraded (20.548%) 2024-07-03T15:47:41.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:41 smithi148 ceph-mon[80490]: from='osd.0 [v2:172.21.15.28:6802/1756284953,v1:172.21.15.28:6803/1756284953]' entity='osd.0' 2024-07-03T15:47:42.261 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:42 smithi028 ceph-mon[92858]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-07-03T15:47:42.262 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:42 smithi028 ceph-mon[92858]: osd.0 [v2:172.21.15.28:6802/1756284953,v1:172.21.15.28:6803/1756284953] boot 2024-07-03T15:47:42.262 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:42 smithi028 ceph-mon[92858]: osdmap e52: 6 total, 6 up, 6 in 2024-07-03T15:47:42.262 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:42 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-03T15:47:42.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:42 smithi148 ceph-mon[80490]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-07-03T15:47:42.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:42 smithi148 ceph-mon[80490]: osd.0 [v2:172.21.15.28:6802/1756284953,v1:172.21.15.28:6803/1756284953] boot 2024-07-03T15:47:42.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:42 smithi148 ceph-mon[80490]: osdmap e52: 6 total, 6 up, 6 in 2024-07-03T15:47:42.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:42 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-03T15:47:43.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:43 smithi148 ceph-mon[80490]: pgmap v47: 65 pgs: 17 active+undersized, 17 active+undersized+degraded, 31 active+clean; 193 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 45/219 objects degraded (20.548%) 2024-07-03T15:47:43.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:43 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:43.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:43 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:43.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:43 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:47:43.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:43 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:47:43.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:43 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:43.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:43 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:47:43.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:43 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:47:43.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:43 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:47:43.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:43 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:47:43.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:43 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-07-03T15:47:43.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:43 smithi148 ceph-mon[80490]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-07-03T15:47:43.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:43 smithi148 ceph-mon[80490]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-07-03T15:47:43.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:43 smithi148 ceph-mon[80490]: osdmap e53: 6 total, 6 up, 6 in 2024-07-03T15:47:43.508 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:43 smithi028 ceph-mon[92858]: pgmap v47: 65 pgs: 17 active+undersized, 17 active+undersized+degraded, 31 active+clean; 193 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 45/219 objects degraded (20.548%) 2024-07-03T15:47:43.508 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:43 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:43.508 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:43 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:43.509 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:43 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:47:43.509 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:43 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:47:43.509 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:43 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:43.509 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:43 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:47:43.509 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:43 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:47:43.509 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:43 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:47:43.509 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:43 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:47:43.509 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:43 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-07-03T15:47:43.509 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:43 smithi028 ceph-mon[92858]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-07-03T15:47:43.509 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:43 smithi028 ceph-mon[92858]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-07-03T15:47:43.509 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:43 smithi028 ceph-mon[92858]: osdmap e53: 6 total, 6 up, 6 in 2024-07-03T15:47:44.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:44 smithi148 ceph-mon[80490]: Health check update: Degraded data redundancy: 6/219 objects degraded (2.740%), 2 pgs degraded (PG_DEGRADED) 2024-07-03T15:47:44.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:44 smithi028 ceph-mon[92858]: Health check update: Degraded data redundancy: 6/219 objects degraded (2.740%), 2 pgs degraded (PG_DEGRADED) 2024-07-03T15:47:45.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:45 smithi148 ceph-mon[80490]: pgmap v49: 65 pgs: 7 active+undersized, 2 active+undersized+degraded, 56 active+clean; 193 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 6/219 objects degraded (2.740%) 2024-07-03T15:47:45.596 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:45 smithi028 ceph-mon[92858]: pgmap v49: 65 pgs: 7 active+undersized, 2 active+undersized+degraded, 56 active+clean; 193 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 6/219 objects degraded (2.740%) 2024-07-03T15:47:46.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:46 smithi028 ceph-mon[92858]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 6/219 objects degraded (2.740%), 2 pgs degraded) 2024-07-03T15:47:46.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:46 smithi028 ceph-mon[92858]: Cluster is now healthy 2024-07-03T15:47:46.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:46 smithi148 ceph-mon[80490]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 6/219 objects degraded (2.740%), 2 pgs degraded) 2024-07-03T15:47:46.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:46 smithi148 ceph-mon[80490]: Cluster is now healthy 2024-07-03T15:47:47.524 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:47 smithi028 ceph-mon[92858]: pgmap v50: 65 pgs: 65 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 842 B/s rd, 1 op/s 2024-07-03T15:47:47.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:47 smithi148 ceph-mon[80490]: pgmap v50: 65 pgs: 65 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 842 B/s rd, 1 op/s 2024-07-03T15:47:49.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:49 smithi148 ceph-mon[80490]: pgmap v51: 65 pgs: 65 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:47:49.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:49 smithi028 ceph-mon[92858]: pgmap v51: 65 pgs: 65 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:47:51.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:51 smithi028 ceph-mon[92858]: pgmap v52: 65 pgs: 65 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:47:51.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:51 smithi148 ceph-mon[80490]: pgmap v52: 65 pgs: 65 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:47:53.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:53 smithi028 ceph-mon[92858]: pgmap v53: 65 pgs: 65 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-07-03T15:47:53.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:53 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:53.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:53 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:47:53.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:53 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:53.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:53 smithi148 ceph-mon[80490]: pgmap v53: 65 pgs: 65 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-07-03T15:47:53.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:53 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:53.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:53 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:47:53.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:53 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:55.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:55 smithi148 ceph-mon[80490]: pgmap v54: 65 pgs: 65 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.4 KiB/s rd, 271 B/s wr, 3 op/s 2024-07-03T15:47:55.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:55 smithi148 ceph-mon[80490]: mgrmap e37: smithi028.vtqrtz(active, since 92s), standbys: smithi148.rjzsar 2024-07-03T15:47:55.596 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:55 smithi028 ceph-mon[92858]: pgmap v54: 65 pgs: 65 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.4 KiB/s rd, 271 B/s wr, 3 op/s 2024-07-03T15:47:55.596 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:55 smithi028 ceph-mon[92858]: mgrmap e37: smithi028.vtqrtz(active, since 92s), standbys: smithi148.rjzsar 2024-07-03T15:47:57.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:57 smithi148 ceph-mon[80490]: pgmap v55: 65 pgs: 65 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-07-03T15:47:57.583 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:57 smithi028 ceph-mon[92858]: pgmap v55: 65 pgs: 65 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-07-03T15:47:58.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:58 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-07-03T15:47:58.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:58 smithi148 ceph-mon[80490]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-07-03T15:47:58.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:58 smithi148 ceph-mon[80490]: Upgrade: osd.1 is safe to restart 2024-07-03T15:47:58.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:58 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-07-03T15:47:58.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:58 smithi028 ceph-mon[92858]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-07-03T15:47:58.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:58 smithi028 ceph-mon[92858]: Upgrade: osd.1 is safe to restart 2024-07-03T15:47:59.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:59 smithi148 ceph-mon[80490]: pgmap v56: 65 pgs: 65 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-07-03T15:47:59.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:59 smithi148 ceph-mon[80490]: Upgrade: Updating osd.1 2024-07-03T15:47:59.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:59 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:47:59.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:59 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-07-03T15:47:59.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:59 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:47:59.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:47:59 smithi148 ceph-mon[80490]: Deploying daemon osd.1 on smithi028 2024-07-03T15:48:00.017 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:59 smithi028 ceph-mon[92858]: pgmap v56: 65 pgs: 65 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-07-03T15:48:00.017 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:59 smithi028 ceph-mon[92858]: Upgrade: Updating osd.1 2024-07-03T15:48:00.017 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:59 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:00.017 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:59 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-07-03T15:48:00.017 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:59 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:48:00.017 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:47:59 smithi028 ceph-mon[92858]: Deploying daemon osd.1 on smithi028 2024-07-03T15:48:01.981 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:01 smithi028 ceph-mon[92858]: pgmap v57: 65 pgs: 65 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-07-03T15:48:01.981 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:01 smithi028 systemd[1]: Stopping Ceph osd.1 for a85e4be6-3951-11ef-bcab-c7b262605968... 2024-07-03T15:48:01.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:01 smithi148 ceph-mon[80490]: pgmap v57: 65 pgs: 65 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-07-03T15:48:02.372 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:01 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-1[54714]: 2024-07-03T15:48:01.976+0000 7fe43bdc1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-03T15:48:02.373 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:01 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-1[54714]: 2024-07-03T15:48:01.976+0000 7fe43bdc1640 -1 osd.1 53 *** Got signal Terminated *** 2024-07-03T15:48:02.373 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:01 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-1[54714]: 2024-07-03T15:48:01.976+0000 7fe43bdc1640 -1 osd.1 53 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-03T15:48:02.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:02 smithi148 ceph-mon[80490]: osd.1 marked itself down and dead 2024-07-03T15:48:03.096 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:02 smithi028 ceph-mon[92858]: osd.1 marked itself down and dead 2024-07-03T15:48:03.372 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:03 smithi028 podman[106095]: 2024-07-03 15:48:03.093025153 +0000 UTC m=+1.229926605 container died 87d415dde2319ee8f7191adfa44a1f37200da7d1f15312e55b9458abb16d4649 (image=quay.ceph.io/ceph-ci/ceph@sha256:0b7651540cdb24c1b97466c79c298d54c7794c5f53d5d6b6c5d76a2d3b130d85, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-1, org.label-schema.schema-version=1.0, RELEASE=quincy-d4b063a, org.label-schema.build-date=20240625, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-07-03T15:48:03.700 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:03 smithi028 podman[106095]: 2024-07-03 15:48:03.450552418 +0000 UTC m=+1.587453872 container cleanup 87d415dde2319ee8f7191adfa44a1f37200da7d1f15312e55b9458abb16d4649 (image=quay.ceph.io/ceph-ci/ceph@sha256:0b7651540cdb24c1b97466c79c298d54c7794c5f53d5d6b6c5d76a2d3b130d85, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-1, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=quincy-d4b063a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240625, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.license=GPLv2) 2024-07-03T15:48:03.700 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:03 smithi028 bash[106095]: ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-1 2024-07-03T15:48:03.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:03 smithi148 ceph-mon[80490]: pgmap v58: 65 pgs: 65 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-07-03T15:48:03.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:03 smithi148 ceph-mon[80490]: Health check failed: 1 osds down (OSD_DOWN) 2024-07-03T15:48:03.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:03 smithi148 ceph-mon[80490]: osdmap e54: 6 total, 5 up, 6 in 2024-07-03T15:48:04.101 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:03 smithi028 ceph-mon[92858]: pgmap v58: 65 pgs: 65 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-07-03T15:48:04.102 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:03 smithi028 ceph-mon[92858]: Health check failed: 1 osds down (OSD_DOWN) 2024-07-03T15:48:04.102 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:03 smithi028 ceph-mon[92858]: osdmap e54: 6 total, 5 up, 6 in 2024-07-03T15:48:04.102 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:03 smithi028 podman[106112]: 2024-07-03 15:48:03.727148634 +0000 UTC m=+0.626383691 container remove 87d415dde2319ee8f7191adfa44a1f37200da7d1f15312e55b9458abb16d4649 (image=quay.ceph.io/ceph-ci/ceph@sha256:0b7651540cdb24c1b97466c79c298d54c7794c5f53d5d6b6c5d76a2d3b130d85, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240625, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, RELEASE=quincy-d4b063a, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-03T15:48:04.373 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:04 smithi028 podman[106199]: 2024-07-03 15:48:04.001024099 +0000 UTC m=+0.018402520 image pull 9d49405603e8445bc510362cdeb05f243619be76b0f3897dd4c152b00cffe45f quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2 2024-07-03T15:48:04.373 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:04 smithi028 podman[106199]: 2024-07-03 15:48:04.111575451 +0000 UTC m=+0.128953899 container create 55769d83aae6045f40cf3f4b431d0b92165d0b76160cdf797baa568a36e5ebfa (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-1-deactivate, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=squid-release-c00506d, org.label-schema.build-date=20240625, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2) 2024-07-03T15:48:04.373 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:04 smithi028 podman[106199]: 2024-07-03 15:48:04.322794658 +0000 UTC m=+0.340173080 container init 55769d83aae6045f40cf3f4b431d0b92165d0b76160cdf797baa568a36e5ebfa (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-1-deactivate, RELEASE=squid-release-c00506d, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240625) 2024-07-03T15:48:04.373 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:04 smithi028 podman[106199]: 2024-07-03 15:48:04.326245623 +0000 UTC m=+0.343624029 container start 55769d83aae6045f40cf3f4b431d0b92165d0b76160cdf797baa568a36e5ebfa (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, RELEASE=squid-release-c00506d, org.label-schema.build-date=20240625, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-03T15:48:04.708 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:04 smithi028 podman[106199]: 2024-07-03 15:48:04.406007691 +0000 UTC m=+0.423386094 container attach 55769d83aae6045f40cf3f4b431d0b92165d0b76160cdf797baa568a36e5ebfa (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-release-c00506d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240625, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-07-03T15:48:04.708 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:04 smithi028 podman[106199]: 2024-07-03 15:48:04.592368548 +0000 UTC m=+0.609746953 container died 55769d83aae6045f40cf3f4b431d0b92165d0b76160cdf797baa568a36e5ebfa (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240625, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, RELEASE=squid-release-c00506d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-07-03T15:48:04.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:04.855+0000 7f4cb75a6640 1 -- 172.21.15.28:0/15542410 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4cb00fd640 msgr2=0x7f4cb00fdab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:04.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:04.855+0000 7f4cb75a6640 1 --2- 172.21.15.28:0/15542410 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4cb00fd640 0x7f4cb00fdab0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4ca00048d0 tx=0x7f4ca0009da0 comp rx=0 tx=0).stop 2024-07-03T15:48:04.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:04.855+0000 7f4cb75a6640 1 -- 172.21.15.28:0/15542410 shutdown_connections 2024-07-03T15:48:04.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:04.855+0000 7f4cb75a6640 1 --2- 172.21.15.28:0/15542410 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4cb00fd640 0x7f4cb00fdab0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:04.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:04.855+0000 7f4cb75a6640 1 --2- 172.21.15.28:0/15542410 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cb00fc370 0x7f4cb00fc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:04.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:04.855+0000 7f4cb75a6640 1 -- 172.21.15.28:0/15542410 >> 172.21.15.28:0/15542410 conn(0x7f4cb00f7b50 msgr2=0x7f4cb00f9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:48:04.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:04.855+0000 7f4cb75a6640 1 -- 172.21.15.28:0/15542410 shutdown_connections 2024-07-03T15:48:04.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:04.855+0000 7f4cb75a6640 1 -- 172.21.15.28:0/15542410 wait complete. 2024-07-03T15:48:04.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:04.856+0000 7f4cb75a6640 1 Processor -- start 2024-07-03T15:48:04.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:04.856+0000 7f4cb75a6640 1 -- start start 2024-07-03T15:48:04.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:04.856+0000 7f4cb75a6640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4cb00fc370 0x7f4cb010cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:48:04.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:04.856+0000 7f4cb75a6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cb00fd640 0x7f4cb010d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:48:04.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:04.856+0000 7f4cb75a6640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb010e380 con 0x7f4cb00fc370 2024-07-03T15:48:04.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:04.856+0000 7f4cb75a6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb010e4f0 con 0x7f4cb00fd640 2024-07-03T15:48:04.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:04.856+0000 7f4cb5342640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4cb00fc370 0x7f4cb010cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:48:04.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:04.856+0000 7f4cb5342640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4cb00fc370 0x7f4cb010cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43460/0 (socket says 172.21.15.28:43460) 2024-07-03T15:48:04.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:04.856+0000 7f4cb5342640 1 -- 172.21.15.28:0/1917419688 learned_addr learned my addr 172.21.15.28:0/1917419688 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:48:04.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:04.856+0000 7f4cb4b41640 1 --2- 172.21.15.28:0/1917419688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cb00fd640 0x7f4cb010d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:48:04.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:04.856+0000 7f4cb5342640 1 -- 172.21.15.28:0/1917419688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cb00fd640 msgr2=0x7f4cb010d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:04.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:04.856+0000 7f4cb5342640 1 --2- 172.21.15.28:0/1917419688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cb00fd640 0x7f4cb010d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:04.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:04.856+0000 7f4cb5342640 1 -- 172.21.15.28:0/1917419688 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca0005040 con 0x7f4cb00fc370 2024-07-03T15:48:04.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:04.857+0000 7f4cb5342640 1 --2- 172.21.15.28:0/1917419688 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4cb00fc370 0x7f4cb010cfd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4cac00b790 tx=0x7f4cac0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:48:04.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:04.857+0000 7f4ca5ffb640 1 -- 172.21.15.28:0/1917419688 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cac00c700 con 0x7f4cb00fc370 2024-07-03T15:48:04.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:04.857+0000 7f4cb75a6640 1 -- 172.21.15.28:0/1917419688 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cb010e7d0 con 0x7f4cb00fc370 2024-07-03T15:48:04.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:04.857+0000 7f4ca5ffb640 1 -- 172.21.15.28:0/1917419688 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4cac00cd30 con 0x7f4cb00fc370 2024-07-03T15:48:04.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:04.857+0000 7f4ca5ffb640 1 -- 172.21.15.28:0/1917419688 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cac013370 con 0x7f4cb00fc370 2024-07-03T15:48:04.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:04.857+0000 7f4cb75a6640 1 -- 172.21.15.28:0/1917419688 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cb010ecf0 con 0x7f4cb00fc370 2024-07-03T15:48:04.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:04.859+0000 7f4cb75a6640 1 -- 172.21.15.28:0/1917419688 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cb00fb7d0 con 0x7f4cb00fc370 2024-07-03T15:48:04.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:04.859+0000 7f4ca5ffb640 1 -- 172.21.15.28:0/1917419688 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4cac012070 con 0x7f4cb00fc370 2024-07-03T15:48:04.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:04.860+0000 7f4ca5ffb640 1 --2- 172.21.15.28:0/1917419688 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f4c9c06e100 0x7f4c9c0705b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:48:04.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:04.860+0000 7f4ca5ffb640 1 -- 172.21.15.28:0/1917419688 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f4cac08d810 con 0x7f4cb00fc370 2024-07-03T15:48:04.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:04.860+0000 7f4cb4b41640 1 --2- 172.21.15.28:0/1917419688 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f4c9c06e100 0x7f4c9c0705b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:48:04.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:04.861+0000 7f4cb4b41640 1 --2- 172.21.15.28:0/1917419688 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f4c9c06e100 0x7f4c9c0705b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0009f80 tx=0x7f4ca0035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:48:04.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:04.864+0000 7f4ca5ffb640 1 -- 172.21.15.28:0/1917419688 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f4cac056af0 con 0x7f4cb00fc370 2024-07-03T15:48:04.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:04 smithi148 ceph-mon[80490]: osdmap e55: 6 total, 5 up, 6 in 2024-07-03T15:48:05.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.067+0000 7f4cb75a6640 1 -- 172.21.15.28:0/1917419688 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4cb010ef60 con 0x7f4c9c06e100 2024-07-03T15:48:05.071 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:04 smithi028 ceph-mon[92858]: osdmap e55: 6 total, 5 up, 6 in 2024-07-03T15:48:05.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.068+0000 7f4ca5ffb640 1 -- 172.21.15.28:0/1917419688 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f4cb010ef60 con 0x7f4c9c06e100 2024-07-03T15:48:05.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.070+0000 7f4cb75a6640 1 -- 172.21.15.28:0/1917419688 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f4c9c06e100 msgr2=0x7f4c9c0705b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:05.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.070+0000 7f4cb75a6640 1 --2- 172.21.15.28:0/1917419688 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f4c9c06e100 0x7f4c9c0705b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0009f80 tx=0x7f4ca0035040 comp rx=0 tx=0).stop 2024-07-03T15:48:05.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.070+0000 7f4cb75a6640 1 -- 172.21.15.28:0/1917419688 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4cb00fc370 msgr2=0x7f4cb010cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:05.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.070+0000 7f4cb75a6640 1 --2- 172.21.15.28:0/1917419688 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4cb00fc370 0x7f4cb010cfd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4cac00b790 tx=0x7f4cac0091d0 comp rx=0 tx=0).stop 2024-07-03T15:48:05.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.070+0000 7f4cb75a6640 1 -- 172.21.15.28:0/1917419688 shutdown_connections 2024-07-03T15:48:05.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.070+0000 7f4cb75a6640 1 --2- 172.21.15.28:0/1917419688 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f4c9c06e100 0x7f4c9c0705b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:05.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.070+0000 7f4cb75a6640 1 --2- 172.21.15.28:0/1917419688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cb00fd640 0x7f4cb010d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:05.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.070+0000 7f4cb75a6640 1 --2- 172.21.15.28:0/1917419688 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4cb00fc370 0x7f4cb010cfd0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:05.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.070+0000 7f4cb75a6640 1 -- 172.21.15.28:0/1917419688 >> 172.21.15.28:0/1917419688 conn(0x7f4cb00f7b50 msgr2=0x7f4cb00f9180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:48:05.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.070+0000 7f4cb75a6640 1 -- 172.21.15.28:0/1917419688 shutdown_connections 2024-07-03T15:48:05.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.070+0000 7f4cb75a6640 1 -- 172.21.15.28:0/1917419688 wait complete. 2024-07-03T15:48:05.083 INFO:teuthology.orchestra.run.smithi028.stdout:true 2024-07-03T15:48:05.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.154+0000 7f1030ae4640 1 -- 172.21.15.28:0/2927014765 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f102c0fc510 msgr2=0x7f102c0fc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:05.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.154+0000 7f1030ae4640 1 --2- 172.21.15.28:0/2927014765 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f102c0fc510 0x7f102c0fc980 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f10200048d0 tx=0x7f1020009da0 comp rx=0 tx=0).stop 2024-07-03T15:48:05.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.154+0000 7f1030ae4640 1 -- 172.21.15.28:0/2927014765 shutdown_connections 2024-07-03T15:48:05.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.154+0000 7f1030ae4640 1 --2- 172.21.15.28:0/2927014765 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f102c0fc510 0x7f102c0fc980 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:05.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.154+0000 7f1030ae4640 1 --2- 172.21.15.28:0/2927014765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f102c0fb240 0x7f102c0fb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:05.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.154+0000 7f1030ae4640 1 -- 172.21.15.28:0/2927014765 >> 172.21.15.28:0/2927014765 conn(0x7f102c0f6a20 msgr2=0x7f102c0f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:48:05.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.154+0000 7f1030ae4640 1 -- 172.21.15.28:0/2927014765 shutdown_connections 2024-07-03T15:48:05.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.154+0000 7f1030ae4640 1 -- 172.21.15.28:0/2927014765 wait complete. 2024-07-03T15:48:05.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.154+0000 7f1030ae4640 1 Processor -- start 2024-07-03T15:48:05.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.154+0000 7f1030ae4640 1 -- start start 2024-07-03T15:48:05.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.154+0000 7f1030ae4640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f102c0fb240 0x7f102c10cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:48:05.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.154+0000 7f1030ae4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f102c0fc510 0x7f102c10d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:48:05.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.154+0000 7f1030ae4640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102c10db50 con 0x7f102c0fb240 2024-07-03T15:48:05.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.154+0000 7f1030ae4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102c10e320 con 0x7f102c0fc510 2024-07-03T15:48:05.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.155+0000 7f102a59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f102c0fb240 0x7f102c10cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:48:05.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.155+0000 7f102a59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f102c0fb240 0x7f102c10cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43490/0 (socket says 172.21.15.28:43490) 2024-07-03T15:48:05.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.155+0000 7f102a59c640 1 -- 172.21.15.28:0/1191903201 learned_addr learned my addr 172.21.15.28:0/1191903201 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:48:05.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.155+0000 7f1029d9b640 1 --2- 172.21.15.28:0/1191903201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f102c0fc510 0x7f102c10d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:48:05.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.155+0000 7f102a59c640 1 -- 172.21.15.28:0/1191903201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f102c0fc510 msgr2=0x7f102c10d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:05.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.155+0000 7f102a59c640 1 --2- 172.21.15.28:0/1191903201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f102c0fc510 0x7f102c10d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:05.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.155+0000 7f102a59c640 1 -- 172.21.15.28:0/1191903201 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1020005040 con 0x7f102c0fb240 2024-07-03T15:48:05.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.155+0000 7f102a59c640 1 --2- 172.21.15.28:0/1191903201 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f102c0fb240 0x7f102c10cfc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f101c002a80 tx=0x7f101c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:48:05.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.155+0000 7f101affd640 1 -- 172.21.15.28:0/1191903201 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f101c007c30 con 0x7f102c0fb240 2024-07-03T15:48:05.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.155+0000 7f101affd640 1 -- 172.21.15.28:0/1191903201 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f101c00f040 con 0x7f102c0fb240 2024-07-03T15:48:05.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.155+0000 7f101affd640 1 -- 172.21.15.28:0/1191903201 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f101c00db70 con 0x7f102c0fb240 2024-07-03T15:48:05.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.155+0000 7f1030ae4640 1 -- 172.21.15.28:0/1191903201 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f102c10e5b0 con 0x7f102c0fb240 2024-07-03T15:48:05.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.155+0000 7f1030ae4640 1 -- 172.21.15.28:0/1191903201 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f102c10eaa0 con 0x7f102c0fb240 2024-07-03T15:48:05.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.156+0000 7f1030ae4640 1 -- 172.21.15.28:0/1191903201 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f102c1045a0 con 0x7f102c0fb240 2024-07-03T15:48:05.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.157+0000 7f101affd640 1 -- 172.21.15.28:0/1191903201 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f101c00dcd0 con 0x7f102c0fb240 2024-07-03T15:48:05.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.157+0000 7f101affd640 1 --2- 172.21.15.28:0/1191903201 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f101406e0b0 0x7f1014070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:48:05.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.157+0000 7f101affd640 1 -- 172.21.15.28:0/1191903201 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f101c08d3d0 con 0x7f102c0fb240 2024-07-03T15:48:05.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.157+0000 7f1029d9b640 1 --2- 172.21.15.28:0/1191903201 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f101406e0b0 0x7f1014070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:48:05.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.158+0000 7f1029d9b640 1 --2- 172.21.15.28:0/1191903201 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f101406e0b0 0x7f1014070560 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f10200043d0 tx=0x7f1020035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:48:05.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.162+0000 7f101affd640 1 -- 172.21.15.28:0/1191903201 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f101c0567e0 con 0x7f102c0fb240 2024-07-03T15:48:05.372 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:05 smithi028 podman[106222]: 2024-07-03 15:48:05.168580599 +0000 UTC m=+0.570278987 container remove 55769d83aae6045f40cf3f4b431d0b92165d0b76160cdf797baa568a36e5ebfa (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240625, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-release-c00506d, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0) 2024-07-03T15:48:05.372 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:05 smithi028 systemd[1]: ceph-a85e4be6-3951-11ef-bcab-c7b262605968@osd.1.service: Deactivated successfully. 2024-07-03T15:48:05.372 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:05 smithi028 systemd[1]: Stopped Ceph osd.1 for a85e4be6-3951-11ef-bcab-c7b262605968. 2024-07-03T15:48:05.372 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:05 smithi028 systemd[1]: ceph-a85e4be6-3951-11ef-bcab-c7b262605968@osd.1.service: Consumed 1min 5.860s CPU time. 2024-07-03T15:48:05.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.381+0000 7f1030ae4640 1 -- 172.21.15.28:0/1191903201 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f102c103f40 con 0x7f101406e0b0 2024-07-03T15:48:05.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.383+0000 7f101affd640 1 -- 172.21.15.28:0/1191903201 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f1020039040 con 0x7f101406e0b0 2024-07-03T15:48:05.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.385+0000 7f1030ae4640 1 -- 172.21.15.28:0/1191903201 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f101406e0b0 msgr2=0x7f1014070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:05.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.385+0000 7f1030ae4640 1 --2- 172.21.15.28:0/1191903201 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f101406e0b0 0x7f1014070560 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f10200043d0 tx=0x7f1020035040 comp rx=0 tx=0).stop 2024-07-03T15:48:05.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.385+0000 7f1030ae4640 1 -- 172.21.15.28:0/1191903201 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f102c0fb240 msgr2=0x7f102c10cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:05.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.385+0000 7f1030ae4640 1 --2- 172.21.15.28:0/1191903201 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f102c0fb240 0x7f102c10cfc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f101c002a80 tx=0x7f101c004600 comp rx=0 tx=0).stop 2024-07-03T15:48:05.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.385+0000 7f1030ae4640 1 -- 172.21.15.28:0/1191903201 shutdown_connections 2024-07-03T15:48:05.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.385+0000 7f1030ae4640 1 --2- 172.21.15.28:0/1191903201 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f101406e0b0 0x7f1014070560 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:05.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.385+0000 7f1030ae4640 1 --2- 172.21.15.28:0/1191903201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f102c0fc510 0x7f102c10d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:05.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.385+0000 7f1030ae4640 1 --2- 172.21.15.28:0/1191903201 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f102c0fb240 0x7f102c10cfc0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:05.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.385+0000 7f1030ae4640 1 -- 172.21.15.28:0/1191903201 >> 172.21.15.28:0/1191903201 conn(0x7f102c0f6a20 msgr2=0x7f102c065390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:48:05.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.385+0000 7f1030ae4640 1 -- 172.21.15.28:0/1191903201 shutdown_connections 2024-07-03T15:48:05.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.385+0000 7f1030ae4640 1 -- 172.21.15.28:0/1191903201 wait complete. 2024-07-03T15:48:05.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.474+0000 7feebda16640 1 -- 172.21.15.28:0/3733105857 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feeb80fbce0 msgr2=0x7feeb80fc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:05.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.474+0000 7feebda16640 1 --2- 172.21.15.28:0/3733105857 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feeb80fbce0 0x7feeb80fc150 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7feea00048d0 tx=0x7feea0009da0 comp rx=0 tx=0).stop 2024-07-03T15:48:05.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.474+0000 7feebda16640 1 -- 172.21.15.28:0/3733105857 shutdown_connections 2024-07-03T15:48:05.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.474+0000 7feebda16640 1 --2- 172.21.15.28:0/3733105857 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feeb80fbce0 0x7feeb80fc150 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:05.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.474+0000 7feebda16640 1 --2- 172.21.15.28:0/3733105857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feeb80faa10 0x7feeb80fae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:05.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.474+0000 7feebda16640 1 -- 172.21.15.28:0/3733105857 >> 172.21.15.28:0/3733105857 conn(0x7feeb80f61d0 msgr2=0x7feeb80f8600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:48:05.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.474+0000 7feebda16640 1 -- 172.21.15.28:0/3733105857 shutdown_connections 2024-07-03T15:48:05.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.474+0000 7feebda16640 1 -- 172.21.15.28:0/3733105857 wait complete. 2024-07-03T15:48:05.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.475+0000 7feebda16640 1 Processor -- start 2024-07-03T15:48:05.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.475+0000 7feebda16640 1 -- start start 2024-07-03T15:48:05.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.475+0000 7feebda16640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feeb80faa10 0x7feeb810bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:48:05.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.475+0000 7feebda16640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feeb80fbce0 0x7feeb810c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:48:05.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.475+0000 7feebda16640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb810cab0 con 0x7feeb80faa10 2024-07-03T15:48:05.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.475+0000 7feebda16640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb810d2d0 con 0x7feeb80fbce0 2024-07-03T15:48:05.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.475+0000 7feeb6ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feeb80faa10 0x7feeb810bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:48:05.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.475+0000 7feeb6ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feeb80faa10 0x7feeb810bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43516/0 (socket says 172.21.15.28:43516) 2024-07-03T15:48:05.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.475+0000 7feeb6ffd640 1 -- 172.21.15.28:0/3864153671 learned_addr learned my addr 172.21.15.28:0/3864153671 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:48:05.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.476+0000 7feeb67fc640 1 --2- 172.21.15.28:0/3864153671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feeb80fbce0 0x7feeb810c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:48:05.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.476+0000 7feeb6ffd640 1 -- 172.21.15.28:0/3864153671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feeb80fbce0 msgr2=0x7feeb810c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:05.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.476+0000 7feeb6ffd640 1 --2- 172.21.15.28:0/3864153671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feeb80fbce0 0x7feeb810c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:05.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.476+0000 7feeb6ffd640 1 -- 172.21.15.28:0/3864153671 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea0005040 con 0x7feeb80faa10 2024-07-03T15:48:05.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.476+0000 7feeb6ffd640 1 --2- 172.21.15.28:0/3864153671 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feeb80faa10 0x7feeb810bf20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7feea8002a50 tx=0x7feea8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:48:05.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.476+0000 7feeaf7fe640 1 -- 172.21.15.28:0/3864153671 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feea8007c30 con 0x7feeb80faa10 2024-07-03T15:48:05.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.476+0000 7feebda16640 1 -- 172.21.15.28:0/3864153671 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeb810d560 con 0x7feeb80faa10 2024-07-03T15:48:05.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.476+0000 7feeaf7fe640 1 -- 172.21.15.28:0/3864153671 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feea800f040 con 0x7feeb80faa10 2024-07-03T15:48:05.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.476+0000 7feeaf7fe640 1 -- 172.21.15.28:0/3864153671 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feea800dad0 con 0x7feeb80faa10 2024-07-03T15:48:05.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.476+0000 7feebda16640 1 -- 172.21.15.28:0/3864153671 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeb810db00 con 0x7feeb80faa10 2024-07-03T15:48:05.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.477+0000 7feebda16640 1 -- 172.21.15.28:0/3864153671 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeb80f9e70 con 0x7feeb80faa10 2024-07-03T15:48:05.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.478+0000 7feeaf7fe640 1 -- 172.21.15.28:0/3864153671 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7feea800dd40 con 0x7feeb80faa10 2024-07-03T15:48:05.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.479+0000 7feeaf7fe640 1 --2- 172.21.15.28:0/3864153671 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7feea406e0b0 0x7feea4070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:48:05.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.479+0000 7feeaf7fe640 1 -- 172.21.15.28:0/3864153671 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6410+0+0 (secure 0 0 0) 0x7feea808d330 con 0x7feeb80faa10 2024-07-03T15:48:05.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.479+0000 7feeb67fc640 1 --2- 172.21.15.28:0/3864153671 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7feea406e0b0 0x7feea4070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:48:05.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.479+0000 7feeb67fc640 1 --2- 172.21.15.28:0/3864153671 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7feea406e0b0 0x7feea4070560 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7feeb810bcd0 tx=0x7feea0035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:48:05.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.483+0000 7feeaf7fe640 1 -- 172.21.15.28:0/3864153671 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7feea8056760 con 0x7feeb80faa10 2024-07-03T15:48:05.644 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:05 smithi028 systemd[1]: Starting Ceph osd.1 for a85e4be6-3951-11ef-bcab-c7b262605968... 2024-07-03T15:48:05.644 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:05 smithi028 podman[106386]: 2024-07-03 15:48:05.619590464 +0000 UTC m=+0.094204405 container create b38e3c2bbdb43d52bf6b6c10e24295c09e3a1637af7bae47dd125e0f6536e65f (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-1-activate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_BRANCH=HEAD, RELEASE=squid-release-c00506d, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240625, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-03T15:48:05.644 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:05 smithi028 podman[106386]: 2024-07-03 15:48:05.543802653 +0000 UTC m=+0.018416606 image pull 9d49405603e8445bc510362cdeb05f243619be76b0f3897dd4c152b00cffe45f quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2 2024-07-03T15:48:05.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.686+0000 7feebda16640 1 -- 172.21.15.28:0/3864153671 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7feeb810de30 con 0x7feea406e0b0 2024-07-03T15:48:05.701 INFO:teuthology.orchestra.run.smithi028.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-03T15:48:05.701 INFO:teuthology.orchestra.run.smithi028.stdout:alertmanager.smithi028 smithi028 *:9093,9094 running (3m) 27s ago 11m 25.8M - 0.25.0 c8568f914cd2 1425315a0de9 2024-07-03T15:48:05.701 INFO:teuthology.orchestra.run.smithi028.stdout:crash.smithi028 smithi028 running (56s) 27s ago 11m 6895k - 19.0.0-2550-gc00506dc 9d49405603e8 c62c2f45cabe 2024-07-03T15:48:05.701 INFO:teuthology.orchestra.run.smithi028.stdout:crash.smithi148 smithi148 running (51s) 48s ago 10m 6895k - 19.0.0-2550-gc00506dc 9d49405603e8 b1318dcaacb6 2024-07-03T15:48:05.701 INFO:teuthology.orchestra.run.smithi028.stdout:grafana.smithi028 smithi028 *:3000 running (3m) 27s ago 11m 82.6M - 9.4.7 954c08fa6188 f9f17c92cfea 2024-07-03T15:48:05.701 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi028.rhpvve smithi028 running (6m) 27s ago 6m 152M - 17.2.7-1047-gd4b063ab 4ca399d9e65d 3ed5abd02cfb 2024-07-03T15:48:05.701 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi028.wmbxew smithi028 running (6m) 27s ago 6m 110M - 17.2.7-1047-gd4b063ab 4ca399d9e65d b3bf736998f9 2024-07-03T15:48:05.702 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi148.szomfl smithi148 running (6m) 48s ago 6m 15.7M - 17.2.7-1047-gd4b063ab 4ca399d9e65d d94f5c95fa8d 2024-07-03T15:48:05.702 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi148.xgpuld smithi148 running (6m) 48s ago 6m 13.2M - 17.2.7-1047-gd4b063ab 4ca399d9e65d fc70360a8de0 2024-07-03T15:48:05.702 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.smithi028.vtqrtz smithi028 *:8443,9283,8765 running (2m) 27s ago 13m 493M - 19.0.0-2550-gc00506dc 9d49405603e8 14d0363c5b77 2024-07-03T15:48:05.702 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.smithi148.rjzsar smithi148 *:8443,9283,8765 running (2m) 48s ago 10m 438M - 19.0.0-2550-gc00506dc 9d49405603e8 c0193aa8a5dd 2024-07-03T15:48:05.702 INFO:teuthology.orchestra.run.smithi028.stdout:mon.smithi028 smithi028 running (110s) 27s ago 13m 43.7M 2048M 19.0.0-2550-gc00506dc 9d49405603e8 5093f7bf47d7 2024-07-03T15:48:05.702 INFO:teuthology.orchestra.run.smithi028.stdout:mon.smithi148 smithi148 running (91s) 48s ago 10m 33.6M 2048M 19.0.0-2550-gc00506dc 9d49405603e8 d338d2c648da 2024-07-03T15:48:05.702 INFO:teuthology.orchestra.run.smithi028.stdout:node-exporter.smithi028 smithi028 *:9100 running (3m) 27s ago 11m 15.7M - 1.5.0 0da6a335fe13 91ca1f49de0e 2024-07-03T15:48:05.702 INFO:teuthology.orchestra.run.smithi028.stdout:node-exporter.smithi148 smithi148 *:9100 running (3m) 48s ago 10m 15.6M - 1.5.0 0da6a335fe13 1eaaf7b1b561 2024-07-03T15:48:05.702 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0 smithi028 running (32s) 27s ago 9m 12.1M 1558M 19.0.0-2550-gc00506dc 9d49405603e8 4454cecff8fc 2024-07-03T15:48:05.702 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1 smithi028 running (9m) 27s ago 9m 256M 1558M 17.2.7-1047-gd4b063ab 4ca399d9e65d 87d415dde231 2024-07-03T15:48:05.702 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2 smithi028 running (9m) 27s ago 9m 219M 1558M 17.2.7-1047-gd4b063ab 4ca399d9e65d b2cbb312212d 2024-07-03T15:48:05.702 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3 smithi148 running (8m) 48s ago 8m 278M 2581M 17.2.7-1047-gd4b063ab 4ca399d9e65d 5be69a27df31 2024-07-03T15:48:05.702 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4 smithi148 running (8m) 48s ago 8m 250M 2581M 17.2.7-1047-gd4b063ab 4ca399d9e65d 54f7c26c77ca 2024-07-03T15:48:05.703 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5 smithi148 running (7m) 48s ago 7m 239M 2581M 17.2.7-1047-gd4b063ab 4ca399d9e65d d185c6905384 2024-07-03T15:48:05.703 INFO:teuthology.orchestra.run.smithi028.stdout:prometheus.smithi028 smithi028 *:9095 running (2m) 27s ago 11m 54.0M - 2.43.0 a07b618ecd1d 2b0eb2a95afd 2024-07-03T15:48:05.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.697+0000 7feeaf7fe640 1 -- 172.21.15.28:0/3864153671 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7feeb810de30 con 0x7feea406e0b0 2024-07-03T15:48:05.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.699+0000 7feebda16640 1 -- 172.21.15.28:0/3864153671 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7feea406e0b0 msgr2=0x7feea4070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:05.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.699+0000 7feebda16640 1 --2- 172.21.15.28:0/3864153671 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7feea406e0b0 0x7feea4070560 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7feeb810bcd0 tx=0x7feea0035040 comp rx=0 tx=0).stop 2024-07-03T15:48:05.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.699+0000 7feebda16640 1 -- 172.21.15.28:0/3864153671 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feeb80faa10 msgr2=0x7feeb810bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:05.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.699+0000 7feebda16640 1 --2- 172.21.15.28:0/3864153671 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feeb80faa10 0x7feeb810bf20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7feea8002a50 tx=0x7feea8004600 comp rx=0 tx=0).stop 2024-07-03T15:48:05.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.699+0000 7feebda16640 1 -- 172.21.15.28:0/3864153671 shutdown_connections 2024-07-03T15:48:05.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.699+0000 7feebda16640 1 --2- 172.21.15.28:0/3864153671 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7feea406e0b0 0x7feea4070560 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:05.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.699+0000 7feebda16640 1 --2- 172.21.15.28:0/3864153671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feeb80fbce0 0x7feeb810c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:05.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.699+0000 7feebda16640 1 --2- 172.21.15.28:0/3864153671 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feeb80faa10 0x7feeb810bf20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:05.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.699+0000 7feebda16640 1 -- 172.21.15.28:0/3864153671 >> 172.21.15.28:0/3864153671 conn(0x7feeb80f61d0 msgr2=0x7feeb80f78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:48:05.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.699+0000 7feebda16640 1 -- 172.21.15.28:0/3864153671 shutdown_connections 2024-07-03T15:48:05.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.699+0000 7feebda16640 1 -- 172.21.15.28:0/3864153671 wait complete. 2024-07-03T15:48:05.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.783+0000 7fcc5a4e7640 1 -- 172.21.15.28:0/3894677566 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcc540fd600 msgr2=0x7fcc540fda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:05.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.783+0000 7fcc5a4e7640 1 --2- 172.21.15.28:0/3894677566 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcc540fd600 0x7fcc540fda70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c0048d0 tx=0x7fcc3c009da0 comp rx=0 tx=0).stop 2024-07-03T15:48:05.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.784+0000 7fcc5a4e7640 1 -- 172.21.15.28:0/3894677566 shutdown_connections 2024-07-03T15:48:05.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.784+0000 7fcc5a4e7640 1 --2- 172.21.15.28:0/3894677566 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcc540fd600 0x7fcc540fda70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:05.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.784+0000 7fcc5a4e7640 1 --2- 172.21.15.28:0/3894677566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc540fc330 0x7fcc540fc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:05.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.784+0000 7fcc5a4e7640 1 -- 172.21.15.28:0/3894677566 >> 172.21.15.28:0/3894677566 conn(0x7fcc540f7af0 msgr2=0x7fcc540f9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:48:05.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.784+0000 7fcc5a4e7640 1 -- 172.21.15.28:0/3894677566 shutdown_connections 2024-07-03T15:48:05.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.784+0000 7fcc5a4e7640 1 -- 172.21.15.28:0/3894677566 wait complete. 2024-07-03T15:48:05.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.784+0000 7fcc5a4e7640 1 Processor -- start 2024-07-03T15:48:05.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.784+0000 7fcc5a4e7640 1 -- start start 2024-07-03T15:48:05.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.784+0000 7fcc5a4e7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc540fc330 0x7fcc5410cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:48:05.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.784+0000 7fcc5a4e7640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcc540fd600 0x7fcc5410d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:48:05.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.784+0000 7fcc5a4e7640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5410dae0 con 0x7fcc540fd600 2024-07-03T15:48:05.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.784+0000 7fcc5a4e7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5410e300 con 0x7fcc540fc330 2024-07-03T15:48:05.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.784+0000 7fcc4bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcc540fd600 0x7fcc5410d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:48:05.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.785+0000 7fcc4bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcc540fd600 0x7fcc5410d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43532/0 (socket says 172.21.15.28:43532) 2024-07-03T15:48:05.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.785+0000 7fcc4bfff640 1 -- 172.21.15.28:0/1699817581 learned_addr learned my addr 172.21.15.28:0/1699817581 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:48:05.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.785+0000 7fcc53fff640 1 --2- 172.21.15.28:0/1699817581 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc540fc330 0x7fcc5410cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:48:05.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.785+0000 7fcc4bfff640 1 -- 172.21.15.28:0/1699817581 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc540fc330 msgr2=0x7fcc5410cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:05.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.785+0000 7fcc4bfff640 1 --2- 172.21.15.28:0/1699817581 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc540fc330 0x7fcc5410cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:05.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.785+0000 7fcc4bfff640 1 -- 172.21.15.28:0/1699817581 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc3c005040 con 0x7fcc540fd600 2024-07-03T15:48:05.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.785+0000 7fcc53fff640 1 --2- 172.21.15.28:0/1699817581 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc540fc330 0x7fcc5410cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-03T15:48:05.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.785+0000 7fcc4bfff640 1 --2- 172.21.15.28:0/1699817581 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcc540fd600 0x7fcc5410d490 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c009f80 tx=0x7fcc3c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:48:05.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.785+0000 7fcc517fa640 1 -- 172.21.15.28:0/1699817581 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc3c038070 con 0x7fcc540fd600 2024-07-03T15:48:05.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.785+0000 7fcc517fa640 1 -- 172.21.15.28:0/1699817581 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcc3c032970 con 0x7fcc540fd600 2024-07-03T15:48:05.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.785+0000 7fcc5a4e7640 1 -- 172.21.15.28:0/1699817581 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc5410e530 con 0x7fcc540fd600 2024-07-03T15:48:05.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.785+0000 7fcc517fa640 1 -- 172.21.15.28:0/1699817581 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc3c033860 con 0x7fcc540fd600 2024-07-03T15:48:05.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.785+0000 7fcc5a4e7640 1 -- 172.21.15.28:0/1699817581 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc5410ea70 con 0x7fcc540fd600 2024-07-03T15:48:05.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.786+0000 7fcc5a4e7640 1 -- 172.21.15.28:0/1699817581 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc540fb790 con 0x7fcc540fd600 2024-07-03T15:48:05.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.787+0000 7fcc517fa640 1 -- 172.21.15.28:0/1699817581 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fcc3c030020 con 0x7fcc540fd600 2024-07-03T15:48:05.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.787+0000 7fcc517fa640 1 --2- 172.21.15.28:0/1699817581 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fcc4006e0b0 0x7fcc40070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:48:05.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.788+0000 7fcc517fa640 1 -- 172.21.15.28:0/1699817581 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fcc3c0ad9a0 con 0x7fcc540fd600 2024-07-03T15:48:05.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.788+0000 7fcc53fff640 1 --2- 172.21.15.28:0/1699817581 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fcc4006e0b0 0x7fcc40070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:48:05.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.788+0000 7fcc53fff640 1 --2- 172.21.15.28:0/1699817581 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fcc4006e0b0 0x7fcc40070560 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcc44004d80 tx=0x7fcc4400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:48:05.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:05.792+0000 7fcc517fa640 1 -- 172.21.15.28:0/1699817581 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fcc3c076da0 con 0x7fcc540fd600 2024-07-03T15:48:05.926 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:05 smithi028 ceph-mon[92858]: pgmap v61: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-07-03T15:48:05.926 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:05 smithi028 ceph-mon[92858]: Health check failed: Reduced data availability: 9 pgs peering (PG_AVAILABILITY) 2024-07-03T15:48:05.927 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:05 smithi028 podman[106386]: 2024-07-03 15:48:05.831163235 +0000 UTC m=+0.305777188 container init b38e3c2bbdb43d52bf6b6c10e24295c09e3a1637af7bae47dd125e0f6536e65f (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-1-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240625, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=squid-release-c00506d, org.label-schema.license=GPLv2) 2024-07-03T15:48:05.927 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:05 smithi028 podman[106386]: 2024-07-03 15:48:05.836007408 +0000 UTC m=+0.310621356 container start b38e3c2bbdb43d52bf6b6c10e24295c09e3a1637af7bae47dd125e0f6536e65f (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-1-activate, org.label-schema.build-date=20240625, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-release-c00506d, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-07-03T15:48:05.927 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:05 smithi028 podman[106386]: 2024-07-03 15:48:05.922754034 +0000 UTC m=+0.397367982 container attach b38e3c2bbdb43d52bf6b6c10e24295c09e3a1637af7bae47dd125e0f6536e65f (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-1-activate, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-release-c00506d, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240625) 2024-07-03T15:48:05.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:05 smithi148 ceph-mon[80490]: pgmap v61: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-07-03T15:48:05.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:05 smithi148 ceph-mon[80490]: Health check failed: Reduced data availability: 9 pgs peering (PG_AVAILABILITY) 2024-07-03T15:48:06.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.053+0000 7fcc5a4e7640 1 -- 172.21.15.28:0/1699817581 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fcc5404b530 con 0x7fcc540fd600 2024-07-03T15:48:06.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.053+0000 7fcc517fa640 1 -- 172.21.15.28:0/1699817581 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+838 (secure 0 0 0) 0x7fcc3c07a460 con 0x7fcc540fd600 2024-07-03T15:48:06.058 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-07-03T15:48:06.058 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": { 2024-07-03T15:48:06.058 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 2 2024-07-03T15:48:06.058 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:48:06.058 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": { 2024-07-03T15:48:06.058 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 2 2024-07-03T15:48:06.058 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:48:06.058 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": { 2024-07-03T15:48:06.058 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 4, 2024-07-03T15:48:06.058 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 1 2024-07-03T15:48:06.058 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:48:06.058 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": { 2024-07-03T15:48:06.059 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 4 2024-07-03T15:48:06.059 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:48:06.059 INFO:teuthology.orchestra.run.smithi028.stdout: "overall": { 2024-07-03T15:48:06.059 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 8, 2024-07-03T15:48:06.059 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 5 2024-07-03T15:48:06.059 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-07-03T15:48:06.059 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-07-03T15:48:06.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.057+0000 7fcc5a4e7640 1 -- 172.21.15.28:0/1699817581 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fcc4006e0b0 msgr2=0x7fcc40070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:06.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.057+0000 7fcc5a4e7640 1 --2- 172.21.15.28:0/1699817581 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fcc4006e0b0 0x7fcc40070560 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcc44004d80 tx=0x7fcc4400b040 comp rx=0 tx=0).stop 2024-07-03T15:48:06.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.057+0000 7fcc5a4e7640 1 -- 172.21.15.28:0/1699817581 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcc540fd600 msgr2=0x7fcc5410d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:06.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.057+0000 7fcc5a4e7640 1 --2- 172.21.15.28:0/1699817581 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcc540fd600 0x7fcc5410d490 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c009f80 tx=0x7fcc3c009da0 comp rx=0 tx=0).stop 2024-07-03T15:48:06.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.057+0000 7fcc5a4e7640 1 -- 172.21.15.28:0/1699817581 shutdown_connections 2024-07-03T15:48:06.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.057+0000 7fcc5a4e7640 1 --2- 172.21.15.28:0/1699817581 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fcc4006e0b0 0x7fcc40070560 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:06.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.057+0000 7fcc5a4e7640 1 --2- 172.21.15.28:0/1699817581 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcc540fd600 0x7fcc5410d490 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:06.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.057+0000 7fcc5a4e7640 1 --2- 172.21.15.28:0/1699817581 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc540fc330 0x7fcc5410cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:06.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.057+0000 7fcc5a4e7640 1 -- 172.21.15.28:0/1699817581 >> 172.21.15.28:0/1699817581 conn(0x7fcc540f7af0 msgr2=0x7fcc540f91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:48:06.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.057+0000 7fcc5a4e7640 1 -- 172.21.15.28:0/1699817581 shutdown_connections 2024-07-03T15:48:06.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.057+0000 7fcc5a4e7640 1 -- 172.21.15.28:0/1699817581 wait complete. 2024-07-03T15:48:06.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.146+0000 7ff09413b640 1 -- 172.21.15.28:0/2803869832 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff08c0fd650 msgr2=0x7ff08c0fdac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:06.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.146+0000 7ff09413b640 1 --2- 172.21.15.28:0/2803869832 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff08c0fd650 0x7ff08c0fdac0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff080004810 tx=0x7ff080009da0 comp rx=0 tx=0).stop 2024-07-03T15:48:06.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.146+0000 7ff09413b640 1 -- 172.21.15.28:0/2803869832 shutdown_connections 2024-07-03T15:48:06.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.146+0000 7ff09413b640 1 --2- 172.21.15.28:0/2803869832 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff08c0fd650 0x7ff08c0fdac0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:06.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.146+0000 7ff09413b640 1 --2- 172.21.15.28:0/2803869832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff08c0fc380 0x7ff08c0fc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:06.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.146+0000 7ff09413b640 1 -- 172.21.15.28:0/2803869832 >> 172.21.15.28:0/2803869832 conn(0x7ff08c0f7b40 msgr2=0x7ff08c0f9f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:48:06.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.146+0000 7ff09413b640 1 -- 172.21.15.28:0/2803869832 shutdown_connections 2024-07-03T15:48:06.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.146+0000 7ff09413b640 1 -- 172.21.15.28:0/2803869832 wait complete. 2024-07-03T15:48:06.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.147+0000 7ff09413b640 1 Processor -- start 2024-07-03T15:48:06.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.147+0000 7ff09413b640 1 -- start start 2024-07-03T15:48:06.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.147+0000 7ff09413b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff08c0fc380 0x7ff08c071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:48:06.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.147+0000 7ff09413b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff08c0fd650 0x7ff08c06f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:48:06.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.147+0000 7ff09413b640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08c06fe00 con 0x7ff08c0fd650 2024-07-03T15:48:06.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.147+0000 7ff09413b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08c06ff40 con 0x7ff08c0fc380 2024-07-03T15:48:06.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.147+0000 7ff0916d6640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff08c0fd650 0x7ff08c06f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:48:06.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.147+0000 7ff0916d6640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff08c0fd650 0x7ff08c06f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43564/0 (socket says 172.21.15.28:43564) 2024-07-03T15:48:06.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.147+0000 7ff0916d6640 1 -- 172.21.15.28:0/1672615739 learned_addr learned my addr 172.21.15.28:0/1672615739 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:48:06.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.148+0000 7ff091ed7640 1 --2- 172.21.15.28:0/1672615739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff08c0fc380 0x7ff08c071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:48:06.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.148+0000 7ff0916d6640 1 -- 172.21.15.28:0/1672615739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff08c0fc380 msgr2=0x7ff08c071380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:06.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.148+0000 7ff0916d6640 1 --2- 172.21.15.28:0/1672615739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff08c0fc380 0x7ff08c071380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:06.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.148+0000 7ff0916d6640 1 -- 172.21.15.28:0/1672615739 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff080005040 con 0x7ff08c0fd650 2024-07-03T15:48:06.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.148+0000 7ff0916d6640 1 --2- 172.21.15.28:0/1672615739 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff08c0fd650 0x7ff08c06f870 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff080004870 tx=0x7ff080009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:48:06.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.148+0000 7ff07e7fc640 1 -- 172.21.15.28:0/1672615739 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff080038070 con 0x7ff08c0fd650 2024-07-03T15:48:06.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.148+0000 7ff09413b640 1 -- 172.21.15.28:0/1672615739 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff08c0701c0 con 0x7ff08c0fd650 2024-07-03T15:48:06.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.148+0000 7ff07e7fc640 1 -- 172.21.15.28:0/1672615739 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff080009240 con 0x7ff08c0fd650 2024-07-03T15:48:06.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.148+0000 7ff07e7fc640 1 -- 172.21.15.28:0/1672615739 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff08003d820 con 0x7ff08c0fd650 2024-07-03T15:48:06.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.148+0000 7ff09413b640 1 -- 172.21.15.28:0/1672615739 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff08c114f80 con 0x7ff08c0fd650 2024-07-03T15:48:06.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.149+0000 7ff09413b640 1 -- 172.21.15.28:0/1672615739 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff08c0fb7e0 con 0x7ff08c0fd650 2024-07-03T15:48:06.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.150+0000 7ff07e7fc640 1 -- 172.21.15.28:0/1672615739 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff080030020 con 0x7ff08c0fd650 2024-07-03T15:48:06.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.150+0000 7ff07e7fc640 1 --2- 172.21.15.28:0/1672615739 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7ff07806e0b0 0x7ff078070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:48:06.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.150+0000 7ff07e7fc640 1 -- 172.21.15.28:0/1672615739 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6410+0+0 (secure 0 0 0) 0x7ff0800ae190 con 0x7ff08c0fd650 2024-07-03T15:48:06.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.150+0000 7ff091ed7640 1 --2- 172.21.15.28:0/1672615739 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7ff07806e0b0 0x7ff078070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:48:06.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.151+0000 7ff091ed7640 1 --2- 172.21.15.28:0/1672615739 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7ff07806e0b0 0x7ff078070560 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff088004170 tx=0x7ff08800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:48:06.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.155+0000 7ff07e7fc640 1 -- 172.21.15.28:0/1672615739 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff080077510 con 0x7ff08c0fd650 2024-07-03T15:48:06.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.383+0000 7ff09413b640 1 -- 172.21.15.28:0/1672615739 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff08c04b530 con 0x7ff08c0fd650 2024-07-03T15:48:06.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.383+0000 7ff07e7fc640 1 -- 172.21.15.28:0/1672615739 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1908 (secure 0 0 0) 0x7ff08007abd0 con 0x7ff08c0fd650 2024-07-03T15:48:06.387 INFO:teuthology.orchestra.run.smithi028.stdout:e17 2024-07-03T15:48:06.387 INFO:teuthology.orchestra.run.smithi028.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-03T15:48:06.387 INFO:teuthology.orchestra.run.smithi028.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:48:06.387 INFO:teuthology.orchestra.run.smithi028.stdout:legacy client fscid: 1 2024-07-03T15:48:06.388 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:48:06.388 INFO:teuthology.orchestra.run.smithi028.stdout:Filesystem 'cephfs' (1) 2024-07-03T15:48:06.388 INFO:teuthology.orchestra.run.smithi028.stdout:fs_name cephfs 2024-07-03T15:48:06.388 INFO:teuthology.orchestra.run.smithi028.stdout:epoch 16 2024-07-03T15:48:06.388 INFO:teuthology.orchestra.run.smithi028.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-03T15:48:06.388 INFO:teuthology.orchestra.run.smithi028.stdout:created 2024-07-03T15:41:40.656298+0000 2024-07-03T15:48:06.388 INFO:teuthology.orchestra.run.smithi028.stdout:modified 2024-07-03T15:41:56.454984+0000 2024-07-03T15:48:06.388 INFO:teuthology.orchestra.run.smithi028.stdout:tableserver 0 2024-07-03T15:48:06.388 INFO:teuthology.orchestra.run.smithi028.stdout:root 0 2024-07-03T15:48:06.388 INFO:teuthology.orchestra.run.smithi028.stdout:session_timeout 60 2024-07-03T15:48:06.388 INFO:teuthology.orchestra.run.smithi028.stdout:session_autoclose 300 2024-07-03T15:48:06.389 INFO:teuthology.orchestra.run.smithi028.stdout:max_file_size 1099511627776 2024-07-03T15:48:06.389 INFO:teuthology.orchestra.run.smithi028.stdout:max_xattr_size 65536 2024-07-03T15:48:06.389 INFO:teuthology.orchestra.run.smithi028.stdout:required_client_features {} 2024-07-03T15:48:06.389 INFO:teuthology.orchestra.run.smithi028.stdout:last_failure 0 2024-07-03T15:48:06.389 INFO:teuthology.orchestra.run.smithi028.stdout:last_failure_osd_epoch 0 2024-07-03T15:48:06.389 INFO:teuthology.orchestra.run.smithi028.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:48:06.389 INFO:teuthology.orchestra.run.smithi028.stdout:max_mds 1 2024-07-03T15:48:06.389 INFO:teuthology.orchestra.run.smithi028.stdout:in 0 2024-07-03T15:48:06.389 INFO:teuthology.orchestra.run.smithi028.stdout:up {0=14450} 2024-07-03T15:48:06.389 INFO:teuthology.orchestra.run.smithi028.stdout:failed 2024-07-03T15:48:06.389 INFO:teuthology.orchestra.run.smithi028.stdout:damaged 2024-07-03T15:48:06.389 INFO:teuthology.orchestra.run.smithi028.stdout:stopped 2024-07-03T15:48:06.389 INFO:teuthology.orchestra.run.smithi028.stdout:data_pools [3] 2024-07-03T15:48:06.390 INFO:teuthology.orchestra.run.smithi028.stdout:metadata_pool 2 2024-07-03T15:48:06.390 INFO:teuthology.orchestra.run.smithi028.stdout:inline_data disabled 2024-07-03T15:48:06.390 INFO:teuthology.orchestra.run.smithi028.stdout:balancer 2024-07-03T15:48:06.390 INFO:teuthology.orchestra.run.smithi028.stdout:bal_rank_mask -1 2024-07-03T15:48:06.390 INFO:teuthology.orchestra.run.smithi028.stdout:standby_count_wanted 1 2024-07-03T15:48:06.390 INFO:teuthology.orchestra.run.smithi028.stdout:qdb_cluster leader: 0 members: 2024-07-03T15:48:06.390 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi028.rhpvve{0:14450} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.28:6826/1613936042,v1:172.21.15.28:6827/1613936042] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:48:06.390 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi028.wmbxew{0:14460} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.28:6828/1730336821,v1:172.21.15.28:6829/1730336821] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:48:06.390 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:48:06.390 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:48:06.390 INFO:teuthology.orchestra.run.smithi028.stdout:Standby daemons: 2024-07-03T15:48:06.390 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:48:06.391 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi148.szomfl{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/4171231858,v1:172.21.15.148:6827/4171231858] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:48:06.391 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi148.xgpuld{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/2347536398,v1:172.21.15.148:6825/2347536398] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:48:06.391 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 17 2024-07-03T15:48:06.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.385+0000 7ff09413b640 1 -- 172.21.15.28:0/1672615739 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7ff07806e0b0 msgr2=0x7ff078070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:06.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.385+0000 7ff09413b640 1 --2- 172.21.15.28:0/1672615739 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7ff07806e0b0 0x7ff078070560 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff088004170 tx=0x7ff08800b040 comp rx=0 tx=0).stop 2024-07-03T15:48:06.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.385+0000 7ff09413b640 1 -- 172.21.15.28:0/1672615739 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff08c0fd650 msgr2=0x7ff08c06f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:06.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.385+0000 7ff09413b640 1 --2- 172.21.15.28:0/1672615739 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff08c0fd650 0x7ff08c06f870 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff080004870 tx=0x7ff080009da0 comp rx=0 tx=0).stop 2024-07-03T15:48:06.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.385+0000 7ff09413b640 1 -- 172.21.15.28:0/1672615739 shutdown_connections 2024-07-03T15:48:06.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.385+0000 7ff09413b640 1 --2- 172.21.15.28:0/1672615739 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7ff07806e0b0 0x7ff078070560 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:06.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.385+0000 7ff09413b640 1 --2- 172.21.15.28:0/1672615739 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff08c0fd650 0x7ff08c06f870 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:06.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.385+0000 7ff09413b640 1 --2- 172.21.15.28:0/1672615739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff08c0fc380 0x7ff08c071380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:06.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.385+0000 7ff09413b640 1 -- 172.21.15.28:0/1672615739 >> 172.21.15.28:0/1672615739 conn(0x7ff08c0f7b40 msgr2=0x7ff08c0f91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:48:06.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.385+0000 7ff09413b640 1 -- 172.21.15.28:0/1672615739 shutdown_connections 2024-07-03T15:48:06.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.385+0000 7ff09413b640 1 -- 172.21.15.28:0/1672615739 wait complete. 2024-07-03T15:48:06.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.469+0000 7f5d9cf2f640 1 -- 172.21.15.28:0/863625390 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5d980fb0c0 msgr2=0x7f5d980fd940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:06.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.469+0000 7f5d9cf2f640 1 --2- 172.21.15.28:0/863625390 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5d980fb0c0 0x7f5d980fd940 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c0048d0 tx=0x7f5d8c009da0 comp rx=0 tx=0).stop 2024-07-03T15:48:06.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.469+0000 7f5d9cf2f640 1 -- 172.21.15.28:0/863625390 shutdown_connections 2024-07-03T15:48:06.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.469+0000 7f5d9cf2f640 1 --2- 172.21.15.28:0/863625390 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5d980fb0c0 0x7f5d980fd940 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:06.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.469+0000 7f5d9cf2f640 1 --2- 172.21.15.28:0/863625390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d980fa770 0x7f5d980fab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:06.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.469+0000 7f5d9cf2f640 1 -- 172.21.15.28:0/863625390 >> 172.21.15.28:0/863625390 conn(0x7f5d980f61b0 msgr2=0x7f5d980f85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:48:06.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.469+0000 7f5d9cf2f640 1 -- 172.21.15.28:0/863625390 shutdown_connections 2024-07-03T15:48:06.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.469+0000 7f5d9cf2f640 1 -- 172.21.15.28:0/863625390 wait complete. 2024-07-03T15:48:06.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.470+0000 7f5d9cf2f640 1 Processor -- start 2024-07-03T15:48:06.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.470+0000 7f5d9cf2f640 1 -- start start 2024-07-03T15:48:06.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.470+0000 7f5d9cf2f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d980fa770 0x7f5d9810d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:48:06.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.470+0000 7f5d9cf2f640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5d980fb0c0 0x7f5d9810d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:48:06.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.470+0000 7f5d9cf2f640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9810dd20 con 0x7f5d980fb0c0 2024-07-03T15:48:06.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.470+0000 7f5d9cf2f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9810e540 con 0x7f5d980fa770 2024-07-03T15:48:06.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.470+0000 7f5d95d9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5d980fb0c0 0x7f5d9810d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:48:06.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.470+0000 7f5d95d9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5d980fb0c0 0x7f5d9810d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43578/0 (socket says 172.21.15.28:43578) 2024-07-03T15:48:06.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.470+0000 7f5d95d9b640 1 -- 172.21.15.28:0/3720904963 learned_addr learned my addr 172.21.15.28:0/3720904963 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:48:06.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.470+0000 7f5d9659c640 1 --2- 172.21.15.28:0/3720904963 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d980fa770 0x7f5d9810d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:48:06.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.470+0000 7f5d95d9b640 1 -- 172.21.15.28:0/3720904963 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d980fa770 msgr2=0x7f5d9810d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:06.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.470+0000 7f5d95d9b640 1 --2- 172.21.15.28:0/3720904963 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d980fa770 0x7f5d9810d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:06.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.470+0000 7f5d95d9b640 1 -- 172.21.15.28:0/3720904963 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d8c005040 con 0x7f5d980fb0c0 2024-07-03T15:48:06.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.471+0000 7f5d95d9b640 1 --2- 172.21.15.28:0/3720904963 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5d980fb0c0 0x7f5d9810d6d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c000c00 tx=0x7f5d8c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:48:06.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.471+0000 7f5d86ffd640 1 -- 172.21.15.28:0/3720904963 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d8c038070 con 0x7f5d980fb0c0 2024-07-03T15:48:06.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.471+0000 7f5d86ffd640 1 -- 172.21.15.28:0/3720904963 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5d8c032970 con 0x7f5d980fb0c0 2024-07-03T15:48:06.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.471+0000 7f5d9cf2f640 1 -- 172.21.15.28:0/3720904963 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d9810e770 con 0x7f5d980fb0c0 2024-07-03T15:48:06.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.471+0000 7f5d86ffd640 1 -- 172.21.15.28:0/3720904963 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d8c033950 con 0x7f5d980fb0c0 2024-07-03T15:48:06.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.471+0000 7f5d9cf2f640 1 -- 172.21.15.28:0/3720904963 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d9810ecb0 con 0x7f5d980fb0c0 2024-07-03T15:48:06.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.472+0000 7f5d9cf2f640 1 -- 172.21.15.28:0/3720904963 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d980f9e50 con 0x7f5d980fb0c0 2024-07-03T15:48:06.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.472+0000 7f5d86ffd640 1 -- 172.21.15.28:0/3720904963 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5d8c030020 con 0x7f5d980fb0c0 2024-07-03T15:48:06.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.473+0000 7f5d86ffd640 1 --2- 172.21.15.28:0/3720904963 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f5d8006e0b0 0x7f5d80070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:48:06.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.473+0000 7f5d86ffd640 1 -- 172.21.15.28:0/3720904963 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f5d8c0adac0 con 0x7f5d980fb0c0 2024-07-03T15:48:06.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.473+0000 7f5d9659c640 1 --2- 172.21.15.28:0/3720904963 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f5d8006e0b0 0x7f5d80070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:48:06.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.473+0000 7f5d9659c640 1 --2- 172.21.15.28:0/3720904963 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f5d8006e0b0 0x7f5d80070560 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5d88007fd0 tx=0x7f5d88009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:48:06.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.477+0000 7f5d86ffd640 1 -- 172.21.15.28:0/3720904963 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f5d8c076ef0 con 0x7f5d980fb0c0 2024-07-03T15:48:06.623 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:06 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-1-activate[106403]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-07-03T15:48:06.623 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:06 smithi028 bash[106386]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-07-03T15:48:06.623 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:06 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-1-activate[106403]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-07-03T15:48:06.623 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:06 smithi028 bash[106386]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-07-03T15:48:06.623 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:06 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-1-activate[106403]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-07-03T15:48:06.623 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:06 smithi028 bash[106386]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-07-03T15:48:06.623 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:06 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-1-activate[106403]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-07-03T15:48:06.623 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:06 smithi028 bash[106386]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-07-03T15:48:06.624 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:06 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-1-activate[106403]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-07-03T15:48:06.624 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:06 smithi028 bash[106386]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-07-03T15:48:06.624 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:06 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-1-activate[106403]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-07-03T15:48:06.624 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:06 smithi028 bash[106386]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-07-03T15:48:06.624 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:06 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-1-activate[106403]: --> ceph-volume raw activate successful for osd ID: 1 2024-07-03T15:48:06.624 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:06 smithi028 bash[106386]: --> ceph-volume raw activate successful for osd ID: 1 2024-07-03T15:48:06.624 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:06 smithi028 podman[106386]: 2024-07-03 15:48:06.28489334 +0000 UTC m=+0.759507288 container died b38e3c2bbdb43d52bf6b6c10e24295c09e3a1637af7bae47dd125e0f6536e65f (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-1-activate, org.label-schema.build-date=20240625, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-release-c00506d, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True) 2024-07-03T15:48:06.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.685+0000 7f5d9cf2f640 1 -- 172.21.15.28:0/3720904963 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5d98105280 con 0x7f5d8006e0b0 2024-07-03T15:48:06.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.686+0000 7f5d86ffd640 1 -- 172.21.15.28:0/3720904963 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f5d98105280 con 0x7f5d8006e0b0 2024-07-03T15:48:06.690 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-07-03T15:48:06.691 INFO:teuthology.orchestra.run.smithi028.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2", 2024-07-03T15:48:06.691 INFO:teuthology.orchestra.run.smithi028.stdout: "in_progress": true, 2024-07-03T15:48:06.691 INFO:teuthology.orchestra.run.smithi028.stdout: "which": "Upgrading all daemon types on all hosts", 2024-07-03T15:48:06.691 INFO:teuthology.orchestra.run.smithi028.stdout: "services_complete": [ 2024-07-03T15:48:06.691 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr", 2024-07-03T15:48:06.691 INFO:teuthology.orchestra.run.smithi028.stdout: "crash", 2024-07-03T15:48:06.691 INFO:teuthology.orchestra.run.smithi028.stdout: "mon" 2024-07-03T15:48:06.691 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-07-03T15:48:06.691 INFO:teuthology.orchestra.run.smithi028.stdout: "progress": "7/21 daemons upgraded", 2024-07-03T15:48:06.691 INFO:teuthology.orchestra.run.smithi028.stdout: "message": "Currently upgrading osd daemons", 2024-07-03T15:48:06.691 INFO:teuthology.orchestra.run.smithi028.stdout: "is_paused": false 2024-07-03T15:48:06.691 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-07-03T15:48:06.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.688+0000 7f5d9cf2f640 1 -- 172.21.15.28:0/3720904963 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f5d8006e0b0 msgr2=0x7f5d80070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:06.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.688+0000 7f5d9cf2f640 1 --2- 172.21.15.28:0/3720904963 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f5d8006e0b0 0x7f5d80070560 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5d88007fd0 tx=0x7f5d88009040 comp rx=0 tx=0).stop 2024-07-03T15:48:06.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.688+0000 7f5d9cf2f640 1 -- 172.21.15.28:0/3720904963 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5d980fb0c0 msgr2=0x7f5d9810d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:06.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.688+0000 7f5d9cf2f640 1 --2- 172.21.15.28:0/3720904963 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5d980fb0c0 0x7f5d9810d6d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c000c00 tx=0x7f5d8c009da0 comp rx=0 tx=0).stop 2024-07-03T15:48:06.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.688+0000 7f5d9cf2f640 1 -- 172.21.15.28:0/3720904963 shutdown_connections 2024-07-03T15:48:06.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.689+0000 7f5d9cf2f640 1 --2- 172.21.15.28:0/3720904963 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f5d8006e0b0 0x7f5d80070560 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:06.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.689+0000 7f5d9cf2f640 1 --2- 172.21.15.28:0/3720904963 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5d980fb0c0 0x7f5d9810d6d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:06.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.689+0000 7f5d9cf2f640 1 --2- 172.21.15.28:0/3720904963 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d980fa770 0x7f5d9810d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:06.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.689+0000 7f5d9cf2f640 1 -- 172.21.15.28:0/3720904963 >> 172.21.15.28:0/3720904963 conn(0x7f5d980f61b0 msgr2=0x7f5d980f7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:48:06.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.689+0000 7f5d9cf2f640 1 -- 172.21.15.28:0/3720904963 shutdown_connections 2024-07-03T15:48:06.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.689+0000 7f5d9cf2f640 1 -- 172.21.15.28:0/3720904963 wait complete. 2024-07-03T15:48:06.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.772+0000 7fb109a1e640 1 -- 172.21.15.28:0/1555814034 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb1040fb280 msgr2=0x7fb1040fb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:06.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.772+0000 7fb109a1e640 1 --2- 172.21.15.28:0/1555814034 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb1040fb280 0x7fb1040fb690 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb0f40048d0 tx=0x7fb0f4009da0 comp rx=0 tx=0).stop 2024-07-03T15:48:06.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.772+0000 7fb109a1e640 1 -- 172.21.15.28:0/1555814034 shutdown_connections 2024-07-03T15:48:06.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.772+0000 7fb109a1e640 1 --2- 172.21.15.28:0/1555814034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1040fc550 0x7fb1040fc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:06.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.772+0000 7fb109a1e640 1 --2- 172.21.15.28:0/1555814034 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb1040fb280 0x7fb1040fb690 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:06.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.772+0000 7fb109a1e640 1 -- 172.21.15.28:0/1555814034 >> 172.21.15.28:0/1555814034 conn(0x7fb1040f6a40 msgr2=0x7fb1040f8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:48:06.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.772+0000 7fb109a1e640 1 -- 172.21.15.28:0/1555814034 shutdown_connections 2024-07-03T15:48:06.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.772+0000 7fb109a1e640 1 -- 172.21.15.28:0/1555814034 wait complete. 2024-07-03T15:48:06.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.773+0000 7fb109a1e640 1 Processor -- start 2024-07-03T15:48:06.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.773+0000 7fb109a1e640 1 -- start start 2024-07-03T15:48:06.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.773+0000 7fb109a1e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1040fb280 0x7fb10410fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:48:06.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.773+0000 7fb109a1e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb1040fc550 0x7fb1041121f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:48:06.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.773+0000 7fb109a1e640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1040665c0 con 0x7fb1040fc550 2024-07-03T15:48:06.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.773+0000 7fb109a1e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb104066730 con 0x7fb1040fb280 2024-07-03T15:48:06.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.773+0000 7fb1027fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb1040fc550 0x7fb1041121f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:48:06.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.773+0000 7fb1027fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb1040fc550 0x7fb1041121f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43594/0 (socket says 172.21.15.28:43594) 2024-07-03T15:48:06.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.773+0000 7fb1027fc640 1 -- 172.21.15.28:0/1938087773 learned_addr learned my addr 172.21.15.28:0/1938087773 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:48:06.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.773+0000 7fb102ffd640 1 --2- 172.21.15.28:0/1938087773 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1040fb280 0x7fb10410fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:48:06.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.773+0000 7fb1027fc640 1 -- 172.21.15.28:0/1938087773 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1040fb280 msgr2=0x7fb10410fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:06.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.773+0000 7fb1027fc640 1 --2- 172.21.15.28:0/1938087773 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1040fb280 0x7fb10410fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:06.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.773+0000 7fb1027fc640 1 -- 172.21.15.28:0/1938087773 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f4005040 con 0x7fb1040fc550 2024-07-03T15:48:06.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.774+0000 7fb1027fc640 1 --2- 172.21.15.28:0/1938087773 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb1040fc550 0x7fb1041121f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800c830 tx=0x7fb0f8004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:48:06.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.774+0000 7fb0eb7fe640 1 -- 172.21.15.28:0/1938087773 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0f800a7d0 con 0x7fb1040fc550 2024-07-03T15:48:06.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.774+0000 7fb0eb7fe640 1 -- 172.21.15.28:0/1938087773 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb0f800ae00 con 0x7fb1040fc550 2024-07-03T15:48:06.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.774+0000 7fb109a1e640 1 -- 172.21.15.28:0/1938087773 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1041127f0 con 0x7fb1040fc550 2024-07-03T15:48:06.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.774+0000 7fb0eb7fe640 1 -- 172.21.15.28:0/1938087773 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0f80113a0 con 0x7fb1040fc550 2024-07-03T15:48:06.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.774+0000 7fb109a1e640 1 -- 172.21.15.28:0/1938087773 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb104112d60 con 0x7fb1040fc550 2024-07-03T15:48:06.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.775+0000 7fb109a1e640 1 -- 172.21.15.28:0/1938087773 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1040fa6e0 con 0x7fb1040fc550 2024-07-03T15:48:06.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.776+0000 7fb0eb7fe640 1 -- 172.21.15.28:0/1938087773 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb0f8012400 con 0x7fb1040fc550 2024-07-03T15:48:06.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.776+0000 7fb0eb7fe640 1 --2- 172.21.15.28:0/1938087773 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fb0ec06e0b0 0x7fb0ec070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:48:06.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.776+0000 7fb0eb7fe640 1 -- 172.21.15.28:0/1938087773 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fb0f808db70 con 0x7fb1040fc550 2024-07-03T15:48:06.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.776+0000 7fb102ffd640 1 --2- 172.21.15.28:0/1938087773 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fb0ec06e0b0 0x7fb0ec070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:48:06.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.777+0000 7fb102ffd640 1 --2- 172.21.15.28:0/1938087773 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fb0ec06e0b0 0x7fb0ec070560 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4000c00 tx=0x7fb0f4035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:48:06.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:06.780+0000 7fb0eb7fe640 1 -- 172.21.15.28:0/1938087773 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fb0f8057010 con 0x7fb1040fc550 2024-07-03T15:48:06.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:06 smithi148 ceph-mon[80490]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:48:06.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:06 smithi148 ceph-mon[80490]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:48:06.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:06 smithi148 ceph-mon[80490]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:48:06.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:06 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/1699817581' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:48:06.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:06 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/1672615739' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-03T15:48:07.014 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:06 smithi028 ceph-mon[92858]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:48:07.014 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:06 smithi028 ceph-mon[92858]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:48:07.014 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:06 smithi028 ceph-mon[92858]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:48:07.014 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:06 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/1699817581' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:48:07.014 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:06 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/1672615739' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-03T15:48:07.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:07.045+0000 7fb109a1e640 1 -- 172.21.15.28:0/1938087773 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb10404b530 con 0x7fb1040fc550 2024-07-03T15:48:07.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:07.045+0000 7fb0eb7fe640 1 -- 172.21.15.28:0/1938087773 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1180 (secure 0 0 0) 0x7fb0f805a6d0 con 0x7fb1040fc550 2024-07-03T15:48:07.050 INFO:teuthology.orchestra.run.smithi028.stdout:HEALTH_WARN 1 osds down; Reduced data availability: 9 pgs peering; Degraded data redundancy: 2/219 objects degraded (0.913%), 2 pgs degraded 2024-07-03T15:48:07.050 INFO:teuthology.orchestra.run.smithi028.stdout:[WRN] OSD_DOWN: 1 osds down 2024-07-03T15:48:07.050 INFO:teuthology.orchestra.run.smithi028.stdout: osd.1 (root=default,host=smithi028) is down 2024-07-03T15:48:07.050 INFO:teuthology.orchestra.run.smithi028.stdout:[WRN] PG_AVAILABILITY: Reduced data availability: 9 pgs peering 2024-07-03T15:48:07.051 INFO:teuthology.orchestra.run.smithi028.stdout: pg 2.6 is stuck peering for 2m, current state peering, last acting [3,4] 2024-07-03T15:48:07.051 INFO:teuthology.orchestra.run.smithi028.stdout: pg 2.9 is stuck peering for 2m, current state peering, last acting [4,0] 2024-07-03T15:48:07.051 INFO:teuthology.orchestra.run.smithi028.stdout: pg 2.a is stuck peering for 2m, current state peering, last acting [4,3] 2024-07-03T15:48:07.051 INFO:teuthology.orchestra.run.smithi028.stdout: pg 2.d is stuck peering for 2m, current state peering, last acting [3,2] 2024-07-03T15:48:07.051 INFO:teuthology.orchestra.run.smithi028.stdout: pg 2.15 is stuck peering for 2m, current state peering, last acting [3,0] 2024-07-03T15:48:07.051 INFO:teuthology.orchestra.run.smithi028.stdout: pg 3.0 is stuck peering for 2m, current state peering, last acting [2,4] 2024-07-03T15:48:07.051 INFO:teuthology.orchestra.run.smithi028.stdout: pg 3.2 is stuck peering for 2m, current state peering, last acting [5,3] 2024-07-03T15:48:07.051 INFO:teuthology.orchestra.run.smithi028.stdout: pg 3.4 is stuck peering for 2m, current state peering, last acting [2,3] 2024-07-03T15:48:07.051 INFO:teuthology.orchestra.run.smithi028.stdout: pg 3.19 is stuck peering for 2m, current state peering, last acting [4,3] 2024-07-03T15:48:07.051 INFO:teuthology.orchestra.run.smithi028.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 2/219 objects degraded (0.913%), 2 pgs degraded 2024-07-03T15:48:07.051 INFO:teuthology.orchestra.run.smithi028.stdout: pg 2.4 is active+undersized+degraded, acting [0,4] 2024-07-03T15:48:07.051 INFO:teuthology.orchestra.run.smithi028.stdout: pg 2.1b is active+undersized+degraded, acting [0,5] 2024-07-03T15:48:07.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:07.048+0000 7fb109a1e640 1 -- 172.21.15.28:0/1938087773 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fb0ec06e0b0 msgr2=0x7fb0ec070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:07.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:07.048+0000 7fb109a1e640 1 --2- 172.21.15.28:0/1938087773 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fb0ec06e0b0 0x7fb0ec070560 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4000c00 tx=0x7fb0f4035040 comp rx=0 tx=0).stop 2024-07-03T15:48:07.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:07.048+0000 7fb109a1e640 1 -- 172.21.15.28:0/1938087773 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb1040fc550 msgr2=0x7fb1041121f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:07.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:07.048+0000 7fb109a1e640 1 --2- 172.21.15.28:0/1938087773 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb1040fc550 0x7fb1041121f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800c830 tx=0x7fb0f8004430 comp rx=0 tx=0).stop 2024-07-03T15:48:07.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:07.048+0000 7fb109a1e640 1 -- 172.21.15.28:0/1938087773 shutdown_connections 2024-07-03T15:48:07.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:07.048+0000 7fb109a1e640 1 --2- 172.21.15.28:0/1938087773 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fb0ec06e0b0 0x7fb0ec070560 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:07.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:07.048+0000 7fb109a1e640 1 --2- 172.21.15.28:0/1938087773 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb1040fc550 0x7fb1041121f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:07.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:07.048+0000 7fb109a1e640 1 --2- 172.21.15.28:0/1938087773 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1040fb280 0x7fb10410fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:07.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:07.048+0000 7fb109a1e640 1 -- 172.21.15.28:0/1938087773 >> 172.21.15.28:0/1938087773 conn(0x7fb1040f6a40 msgr2=0x7fb1040f8140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:48:07.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:07.048+0000 7fb109a1e640 1 -- 172.21.15.28:0/1938087773 shutdown_connections 2024-07-03T15:48:07.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:07.048+0000 7fb109a1e640 1 -- 172.21.15.28:0/1938087773 wait complete. 2024-07-03T15:48:07.363 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:07 smithi028 podman[106581]: 2024-07-03 15:48:07.010870182 +0000 UTC m=+0.717335744 container remove b38e3c2bbdb43d52bf6b6c10e24295c09e3a1637af7bae47dd125e0f6536e65f (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-1-activate, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240625, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-release-c00506d, GIT_CLEAN=True) 2024-07-03T15:48:07.363 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:07 smithi028 podman[106690]: 2024-07-03 15:48:07.134300981 +0000 UTC m=+0.018576452 image pull 9d49405603e8445bc510362cdeb05f243619be76b0f3897dd4c152b00cffe45f quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2 2024-07-03T15:48:07.363 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:07 smithi028 podman[106690]: 2024-07-03 15:48:07.253740218 +0000 UTC m=+0.138015680 container create af67f7088643df8759891f308a39b9da0d69befd1c684013a744d1d883f00ece (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-1, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240625, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-release-c00506d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-07-03T15:48:07.622 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:07 smithi028 podman[106690]: 2024-07-03 15:48:07.464712492 +0000 UTC m=+0.348987963 container init af67f7088643df8759891f308a39b9da0d69befd1c684013a744d1d883f00ece (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240625, ceph=True, RELEASE=squid-release-c00506d, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-07-03T15:48:07.622 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:07 smithi028 podman[106690]: 2024-07-03 15:48:07.468359689 +0000 UTC m=+0.352635163 container start af67f7088643df8759891f308a39b9da0d69befd1c684013a744d1d883f00ece (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-1, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-release-c00506d, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.build-date=20240625, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-03T15:48:07.622 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:07 smithi028 ceph-osd[106709]: -- 172.21.15.28:0/928500123 <== mon.0 v2:172.21.15.28:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5636316461e0 con 0x563630863c00 2024-07-03T15:48:07.622 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:07 smithi028 bash[106690]: af67f7088643df8759891f308a39b9da0d69befd1c684013a744d1d883f00ece 2024-07-03T15:48:07.623 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:07 smithi028 systemd[1]: Started Ceph osd.1 for a85e4be6-3951-11ef-bcab-c7b262605968. 2024-07-03T15:48:07.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:07 smithi148 ceph-mon[80490]: pgmap v62: 65 pgs: 3 active+undersized, 29 peering, 2 active+undersized+degraded, 31 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 639 B/s rd, 0 op/s; 2/219 objects degraded (0.913%) 2024-07-03T15:48:07.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:07 smithi148 ceph-mon[80490]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:48:07.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:07 smithi148 ceph-mon[80490]: Health check failed: Degraded data redundancy: 2/219 objects degraded (0.913%), 2 pgs degraded (PG_DEGRADED) 2024-07-03T15:48:07.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:07 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/1938087773' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-03T15:48:07.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:07 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:07.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:07 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:48:07.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:07 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:07.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:07 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:07.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:07 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:48:08.024 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:07 smithi028 ceph-mon[92858]: pgmap v62: 65 pgs: 3 active+undersized, 29 peering, 2 active+undersized+degraded, 31 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 639 B/s rd, 0 op/s; 2/219 objects degraded (0.913%) 2024-07-03T15:48:08.024 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:07 smithi028 ceph-mon[92858]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:48:08.024 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:07 smithi028 ceph-mon[92858]: Health check failed: Degraded data redundancy: 2/219 objects degraded (0.913%), 2 pgs degraded (PG_DEGRADED) 2024-07-03T15:48:08.024 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:07 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/1938087773' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-03T15:48:08.024 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:07 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:08.024 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:07 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:48:08.024 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:07 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:08.024 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:07 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:08.024 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:07 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:48:08.299 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:08 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-1[106705]: 2024-07-03T15:48:08.019+0000 7f16212d1640 -1 Falling back to public interface 2024-07-03T15:48:08.967 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:08 smithi028 ceph-mon[92858]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs peering) 2024-07-03T15:48:09.067 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:08 smithi148 ceph-mon[80490]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs peering) 2024-07-03T15:48:10.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:09 smithi028 ceph-mon[92858]: pgmap v63: 65 pgs: 21 active+undersized, 13 active+undersized+degraded, 31 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 33/219 objects degraded (15.068%) 2024-07-03T15:48:10.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:09 smithi148 ceph-mon[80490]: pgmap v63: 65 pgs: 21 active+undersized, 13 active+undersized+degraded, 31 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 33/219 objects degraded (15.068%) 2024-07-03T15:48:12.023 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:11 smithi028 ceph-mon[92858]: pgmap v64: 65 pgs: 21 active+undersized, 13 active+undersized+degraded, 31 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 33/219 objects degraded (15.068%) 2024-07-03T15:48:12.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:11 smithi148 ceph-mon[80490]: pgmap v64: 65 pgs: 21 active+undersized, 13 active+undersized+degraded, 31 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 33/219 objects degraded (15.068%) 2024-07-03T15:48:13.504 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:13 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-1[106705]: 2024-07-03T15:48:13.229+0000 7f16212d1640 -1 osd.1 53 log_to_monitors true 2024-07-03T15:48:13.505 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:13 smithi028 ceph-mon[92858]: pgmap v65: 65 pgs: 21 active+undersized, 13 active+undersized+degraded, 31 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s; 33/219 objects degraded (15.068%) 2024-07-03T15:48:13.505 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:13 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:13.505 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:13 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:13.505 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:13 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:13.505 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:13 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:13.505 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:13 smithi028 ceph-mon[92858]: from='osd.1 [v2:172.21.15.28:6810/662081515,v1:172.21.15.28:6811/662081515]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-07-03T15:48:13.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:13 smithi148 ceph-mon[80490]: pgmap v65: 65 pgs: 21 active+undersized, 13 active+undersized+degraded, 31 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s; 33/219 objects degraded (15.068%) 2024-07-03T15:48:13.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:13 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:13.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:13 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:13.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:13 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:13.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:13 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:13.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:13 smithi148 ceph-mon[80490]: from='osd.1 [v2:172.21.15.28:6810/662081515,v1:172.21.15.28:6811/662081515]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-07-03T15:48:14.372 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:48:14 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-1[106705]: 2024-07-03T15:48:14.096+0000 7f161906b640 -1 osd.1 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-03T15:48:15.092 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:15 smithi028 ceph-mon[92858]: pgmap v66: 65 pgs: 21 active+undersized, 13 active+undersized+degraded, 31 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 986 B/s rd, 1 op/s; 33/219 objects degraded (15.068%) 2024-07-03T15:48:15.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:15 smithi028 ceph-mon[92858]: from='osd.1 [v2:172.21.15.28:6810/662081515,v1:172.21.15.28:6811/662081515]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-07-03T15:48:15.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:15 smithi028 ceph-mon[92858]: osdmap e56: 6 total, 5 up, 6 in 2024-07-03T15:48:15.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:15 smithi028 ceph-mon[92858]: from='osd.1 [v2:172.21.15.28:6810/662081515,v1:172.21.15.28:6811/662081515]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi028", "root=default"]}]: dispatch 2024-07-03T15:48:15.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:15 smithi148 ceph-mon[80490]: pgmap v66: 65 pgs: 21 active+undersized, 13 active+undersized+degraded, 31 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 986 B/s rd, 1 op/s; 33/219 objects degraded (15.068%) 2024-07-03T15:48:15.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:15 smithi148 ceph-mon[80490]: from='osd.1 [v2:172.21.15.28:6810/662081515,v1:172.21.15.28:6811/662081515]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-07-03T15:48:15.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:15 smithi148 ceph-mon[80490]: osdmap e56: 6 total, 5 up, 6 in 2024-07-03T15:48:15.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:15 smithi148 ceph-mon[80490]: from='osd.1 [v2:172.21.15.28:6810/662081515,v1:172.21.15.28:6811/662081515]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi028", "root=default"]}]: dispatch 2024-07-03T15:48:16.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:16 smithi028 ceph-mon[92858]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-07-03T15:48:16.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:16 smithi028 ceph-mon[92858]: osd.1 [v2:172.21.15.28:6810/662081515,v1:172.21.15.28:6811/662081515] boot 2024-07-03T15:48:16.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:16 smithi028 ceph-mon[92858]: osdmap e57: 6 total, 6 up, 6 in 2024-07-03T15:48:16.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:16 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-03T15:48:16.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:16 smithi028 ceph-mon[92858]: Health check update: Degraded data redundancy: 33/219 objects degraded (15.068%), 13 pgs degraded (PG_DEGRADED) 2024-07-03T15:48:16.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:16 smithi028 ceph-mon[92858]: osdmap e58: 6 total, 6 up, 6 in 2024-07-03T15:48:16.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:16 smithi148 ceph-mon[80490]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-07-03T15:48:16.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:16 smithi148 ceph-mon[80490]: osd.1 [v2:172.21.15.28:6810/662081515,v1:172.21.15.28:6811/662081515] boot 2024-07-03T15:48:16.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:16 smithi148 ceph-mon[80490]: osdmap e57: 6 total, 6 up, 6 in 2024-07-03T15:48:16.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:16 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-03T15:48:16.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:16 smithi148 ceph-mon[80490]: Health check update: Degraded data redundancy: 33/219 objects degraded (15.068%), 13 pgs degraded (PG_DEGRADED) 2024-07-03T15:48:16.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:16 smithi148 ceph-mon[80490]: osdmap e58: 6 total, 6 up, 6 in 2024-07-03T15:48:17.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:17 smithi028 ceph-mon[92858]: pgmap v69: 65 pgs: 21 active+undersized, 13 active+undersized+degraded, 31 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 33/219 objects degraded (15.068%) 2024-07-03T15:48:17.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:17 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:17.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:17 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:17.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:17 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:48:17.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:17 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:48:17.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:17 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:17.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:17 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:48:17.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:17 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:48:17.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:17 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:48:17.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:17 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:48:17.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:17 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-07-03T15:48:17.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:17 smithi028 ceph-mon[92858]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-07-03T15:48:17.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:17 smithi028 ceph-mon[92858]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-07-03T15:48:17.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:17 smithi148 ceph-mon[80490]: pgmap v69: 65 pgs: 21 active+undersized, 13 active+undersized+degraded, 31 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 33/219 objects degraded (15.068%) 2024-07-03T15:48:17.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:17 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:17.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:17 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:17.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:17 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:48:17.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:17 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:48:17.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:17 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:17.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:17 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:48:17.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:17 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:48:17.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:17 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:48:17.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:17 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:48:17.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:17 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-07-03T15:48:17.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:17 smithi148 ceph-mon[80490]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-07-03T15:48:17.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:17 smithi148 ceph-mon[80490]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-07-03T15:48:19.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:19 smithi148 ceph-mon[80490]: pgmap v71: 65 pgs: 7 active+undersized, 7 active+undersized+degraded, 51 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s; 16/219 objects degraded (7.306%) 2024-07-03T15:48:19.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:19 smithi028 ceph-mon[92858]: pgmap v71: 65 pgs: 7 active+undersized, 7 active+undersized+degraded, 51 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s; 16/219 objects degraded (7.306%) 2024-07-03T15:48:20.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:20 smithi028 ceph-mon[92858]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 16/219 objects degraded (7.306%), 7 pgs degraded) 2024-07-03T15:48:20.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:20 smithi028 ceph-mon[92858]: Cluster is now healthy 2024-07-03T15:48:20.734 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:20 smithi148 ceph-mon[80490]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 16/219 objects degraded (7.306%), 7 pgs degraded) 2024-07-03T15:48:20.734 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:20 smithi148 ceph-mon[80490]: Cluster is now healthy 2024-07-03T15:48:21.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:21 smithi028 ceph-mon[92858]: pgmap v72: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 682 B/s rd, 1 op/s 2024-07-03T15:48:21.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:21 smithi148 ceph-mon[80490]: pgmap v72: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 682 B/s rd, 1 op/s 2024-07-03T15:48:23.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:23 smithi028 ceph-mon[92858]: pgmap v73: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 0 op/s 2024-07-03T15:48:23.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:23 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:23.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:23 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:48:23.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:23 smithi148 ceph-mon[80490]: pgmap v73: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 0 op/s 2024-07-03T15:48:23.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:23 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:23.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:23 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:48:25.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:25 smithi028 ceph-mon[92858]: pgmap v74: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1 KiB/s rd, 1 op/s 2024-07-03T15:48:25.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:25 smithi148 ceph-mon[80490]: pgmap v74: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1 KiB/s rd, 1 op/s 2024-07-03T15:48:27.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:27 smithi028 ceph-mon[92858]: pgmap v75: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 921 B/s rd, 1 op/s 2024-07-03T15:48:27.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:27 smithi148 ceph-mon[80490]: pgmap v75: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 921 B/s rd, 1 op/s 2024-07-03T15:48:28.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:28 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:28.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:28 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:29.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:29 smithi148 ceph-mon[80490]: pgmap v76: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:48:29.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:29 smithi028 ceph-mon[92858]: pgmap v76: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:48:31.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:31 smithi148 ceph-mon[80490]: pgmap v77: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:48:31.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:31 smithi028 ceph-mon[92858]: pgmap v77: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:48:32.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:32 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-07-03T15:48:32.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:32 smithi028 ceph-mon[92858]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-07-03T15:48:32.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:32 smithi028 ceph-mon[92858]: Upgrade: osd.2 is safe to restart 2024-07-03T15:48:32.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:32 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-07-03T15:48:32.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:32 smithi148 ceph-mon[80490]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-07-03T15:48:32.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:32 smithi148 ceph-mon[80490]: Upgrade: osd.2 is safe to restart 2024-07-03T15:48:34.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:33 smithi028 ceph-mon[92858]: pgmap v78: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:48:34.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:33 smithi028 ceph-mon[92858]: Upgrade: Updating osd.2 2024-07-03T15:48:34.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:33 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:34.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:33 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-07-03T15:48:34.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:33 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:48:34.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:33 smithi028 ceph-mon[92858]: Deploying daemon osd.2 on smithi028 2024-07-03T15:48:34.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:33 smithi148 ceph-mon[80490]: pgmap v78: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:48:34.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:33 smithi148 ceph-mon[80490]: Upgrade: Updating osd.2 2024-07-03T15:48:34.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:33 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:34.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:33 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-07-03T15:48:34.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:33 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:48:34.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:33 smithi148 ceph-mon[80490]: Deploying daemon osd.2 on smithi028 2024-07-03T15:48:36.043 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:35 smithi028 ceph-mon[92858]: pgmap v79: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:48:36.043 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:35 smithi028 systemd[1]: Stopping Ceph osd.2 for a85e4be6-3951-11ef-bcab-c7b262605968... 2024-07-03T15:48:36.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:35 smithi148 ceph-mon[80490]: pgmap v79: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:48:36.372 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:36 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-2[62448]: 2024-07-03T15:48:36.038+0000 7fe365710640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-03T15:48:36.372 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:36 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-2[62448]: 2024-07-03T15:48:36.038+0000 7fe365710640 -1 osd.2 58 *** Got signal Terminated *** 2024-07-03T15:48:36.372 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:36 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-2[62448]: 2024-07-03T15:48:36.038+0000 7fe365710640 -1 osd.2 58 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-03T15:48:37.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:36 smithi028 ceph-mon[92858]: osd.2 marked itself down and dead 2024-07-03T15:48:37.134 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:36 smithi148 ceph-mon[80490]: osd.2 marked itself down and dead 2024-07-03T15:48:37.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.147+0000 7f14b5b24640 1 -- 172.21.15.28:0/1414933528 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f14b00fe550 msgr2=0x7f14b00fe960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:37.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.147+0000 7f14b5b24640 1 --2- 172.21.15.28:0/1414933528 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f14b00fe550 0x7f14b00fe960 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f14a00048d0 tx=0x7f14a0009da0 comp rx=0 tx=0).stop 2024-07-03T15:48:37.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.147+0000 7f14b5b24640 1 -- 172.21.15.28:0/1414933528 shutdown_connections 2024-07-03T15:48:37.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.147+0000 7f14b5b24640 1 --2- 172.21.15.28:0/1414933528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14b00ff820 0x7f14b00ffc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:37.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.147+0000 7f14b5b24640 1 --2- 172.21.15.28:0/1414933528 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f14b00fe550 0x7f14b00fe960 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:37.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.147+0000 7f14b5b24640 1 -- 172.21.15.28:0/1414933528 >> 172.21.15.28:0/1414933528 conn(0x7f14b00f9e70 msgr2=0x7f14b00fc250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:48:37.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.147+0000 7f14b5b24640 1 -- 172.21.15.28:0/1414933528 shutdown_connections 2024-07-03T15:48:37.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.147+0000 7f14b5b24640 1 -- 172.21.15.28:0/1414933528 wait complete. 2024-07-03T15:48:37.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.148+0000 7f14b5b24640 1 Processor -- start 2024-07-03T15:48:37.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.148+0000 7f14b5b24640 1 -- start start 2024-07-03T15:48:37.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.148+0000 7f14b5b24640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14b00fe550 0x7f14b010f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:48:37.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.148+0000 7f14b5b24640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f14b00ff820 0x7f14b010f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:48:37.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.148+0000 7f14b5b24640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b010fcc0 con 0x7f14b00ff820 2024-07-03T15:48:37.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.148+0000 7f14b5b24640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b01104e0 con 0x7f14b00fe550 2024-07-03T15:48:37.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.149+0000 7f14a6ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f14b00ff820 0x7f14b010f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:48:37.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.149+0000 7f14a6ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f14b00ff820 0x7f14b010f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46450/0 (socket says 172.21.15.28:46450) 2024-07-03T15:48:37.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.149+0000 7f14a6ffd640 1 -- 172.21.15.28:0/1665079659 learned_addr learned my addr 172.21.15.28:0/1665079659 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:48:37.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.149+0000 7f14af7fe640 1 --2- 172.21.15.28:0/1665079659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14b00fe550 0x7f14b010f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:48:37.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.149+0000 7f14a6ffd640 1 -- 172.21.15.28:0/1665079659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14b00fe550 msgr2=0x7f14b010f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:37.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.149+0000 7f14a6ffd640 1 --2- 172.21.15.28:0/1665079659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14b00fe550 0x7f14b010f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:37.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.149+0000 7f14a6ffd640 1 -- 172.21.15.28:0/1665079659 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a0005040 con 0x7f14b00ff820 2024-07-03T15:48:37.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.149+0000 7f14a6ffd640 1 --2- 172.21.15.28:0/1665079659 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f14b00ff820 0x7f14b010f670 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1498002a50 tx=0x7f1498004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:48:37.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.149+0000 7f14acff9640 1 -- 172.21.15.28:0/1665079659 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1498007c30 con 0x7f14b00ff820 2024-07-03T15:48:37.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.149+0000 7f14acff9640 1 -- 172.21.15.28:0/1665079659 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f149800f040 con 0x7f14b00ff820 2024-07-03T15:48:37.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.149+0000 7f14b5b24640 1 -- 172.21.15.28:0/1665079659 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14b0110770 con 0x7f14b00ff820 2024-07-03T15:48:37.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.149+0000 7f14acff9640 1 -- 172.21.15.28:0/1665079659 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f149800db60 con 0x7f14b00ff820 2024-07-03T15:48:37.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.149+0000 7f14b5b24640 1 -- 172.21.15.28:0/1665079659 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14b0110d10 con 0x7f14b00ff820 2024-07-03T15:48:37.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.151+0000 7f14b5b24640 1 -- 172.21.15.28:0/1665079659 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14b00fd9b0 con 0x7f14b00ff820 2024-07-03T15:48:37.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.151+0000 7f14acff9640 1 -- 172.21.15.28:0/1665079659 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f149800dcc0 con 0x7f14b00ff820 2024-07-03T15:48:37.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.152+0000 7f14acff9640 1 --2- 172.21.15.28:0/1665079659 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f149c06e0b0 0x7f149c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:48:37.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.152+0000 7f14acff9640 1 -- 172.21.15.28:0/1665079659 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f149808da00 con 0x7f14b00ff820 2024-07-03T15:48:37.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.152+0000 7f14af7fe640 1 --2- 172.21.15.28:0/1665079659 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f149c06e0b0 0x7f149c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:48:37.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.153+0000 7f14af7fe640 1 --2- 172.21.15.28:0/1665079659 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f149c06e0b0 0x7f149c070560 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f14a00043d0 tx=0x7f14a0035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:48:37.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.157+0000 7f14acff9640 1 -- 172.21.15.28:0/1665079659 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f1498056d80 con 0x7f14b00ff820 2024-07-03T15:48:37.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.366+0000 7f14b5b24640 1 -- 172.21.15.28:0/1665079659 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f14b0111040 con 0x7f149c06e0b0 2024-07-03T15:48:37.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.367+0000 7f14acff9640 1 -- 172.21.15.28:0/1665079659 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f14b0111040 con 0x7f149c06e0b0 2024-07-03T15:48:37.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.369+0000 7f14b5b24640 1 -- 172.21.15.28:0/1665079659 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f149c06e0b0 msgr2=0x7f149c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:37.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.369+0000 7f14b5b24640 1 --2- 172.21.15.28:0/1665079659 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f149c06e0b0 0x7f149c070560 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f14a00043d0 tx=0x7f14a0035040 comp rx=0 tx=0).stop 2024-07-03T15:48:37.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.369+0000 7f14b5b24640 1 -- 172.21.15.28:0/1665079659 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f14b00ff820 msgr2=0x7f14b010f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:37.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.369+0000 7f14b5b24640 1 --2- 172.21.15.28:0/1665079659 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f14b00ff820 0x7f14b010f670 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1498002a50 tx=0x7f1498004600 comp rx=0 tx=0).stop 2024-07-03T15:48:37.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.369+0000 7f14b5b24640 1 -- 172.21.15.28:0/1665079659 shutdown_connections 2024-07-03T15:48:37.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.369+0000 7f14b5b24640 1 --2- 172.21.15.28:0/1665079659 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f149c06e0b0 0x7f149c070560 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:37.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.369+0000 7f14b5b24640 1 --2- 172.21.15.28:0/1665079659 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f14b00ff820 0x7f14b010f670 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:37.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.369+0000 7f14b5b24640 1 --2- 172.21.15.28:0/1665079659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14b00fe550 0x7f14b010f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:37.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.369+0000 7f14b5b24640 1 -- 172.21.15.28:0/1665079659 >> 172.21.15.28:0/1665079659 conn(0x7f14b00f9e70 msgr2=0x7f14b00fc520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:48:37.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.369+0000 7f14b5b24640 1 -- 172.21.15.28:0/1665079659 shutdown_connections 2024-07-03T15:48:37.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.369+0000 7f14b5b24640 1 -- 172.21.15.28:0/1665079659 wait complete. 2024-07-03T15:48:37.382 INFO:teuthology.orchestra.run.smithi028.stdout:true 2024-07-03T15:48:37.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.453+0000 7f3171445640 1 -- 172.21.15.28:0/3991334414 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f316c0ff550 msgr2=0x7f316c101930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:37.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.453+0000 7f3171445640 1 --2- 172.21.15.28:0/3991334414 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f316c0ff550 0x7f316c101930 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f31540048d0 tx=0x7f3154009da0 comp rx=0 tx=0).stop 2024-07-03T15:48:37.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.453+0000 7f3171445640 1 -- 172.21.15.28:0/3991334414 shutdown_connections 2024-07-03T15:48:37.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.453+0000 7f3171445640 1 --2- 172.21.15.28:0/3991334414 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f316c0ff550 0x7f316c101930 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:37.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.453+0000 7f3171445640 1 --2- 172.21.15.28:0/3991334414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f316c065990 0x7f316c0ff010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:37.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.453+0000 7f3171445640 1 -- 172.21.15.28:0/3991334414 >> 172.21.15.28:0/3991334414 conn(0x7f316c0f6a20 msgr2=0x7f316c0f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:48:37.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.453+0000 7f3171445640 1 -- 172.21.15.28:0/3991334414 shutdown_connections 2024-07-03T15:48:37.458 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:37 smithi028 podman[111563]: 2024-07-03 15:48:37.328967739 +0000 UTC m=+1.391345701 container died b2cbb312212d711566745e4bff155a8357fa823280c356f40594cca7b1c80074 (image=quay.ceph.io/ceph-ci/ceph@sha256:0b7651540cdb24c1b97466c79c298d54c7794c5f53d5d6b6c5d76a2d3b130d85, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.build-date=20240625, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=quincy-d4b063a, org.label-schema.schema-version=1.0) 2024-07-03T15:48:37.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.453+0000 7f3171445640 1 -- 172.21.15.28:0/3991334414 wait complete. 2024-07-03T15:48:37.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.454+0000 7f3171445640 1 Processor -- start 2024-07-03T15:48:37.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.454+0000 7f3171445640 1 -- start start 2024-07-03T15:48:37.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.454+0000 7f3171445640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f316c065990 0x7f316c10cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:48:37.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.454+0000 7f3171445640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f316c0ff550 0x7f316c10d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:48:37.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.454+0000 7f3171445640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316c10db10 con 0x7f316c0ff550 2024-07-03T15:48:37.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.454+0000 7f3171445640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316c10e330 con 0x7f316c065990 2024-07-03T15:48:37.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.454+0000 7f316a7fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f316c0ff550 0x7f316c10d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:48:37.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.454+0000 7f316a7fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f316c0ff550 0x7f316c10d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46470/0 (socket says 172.21.15.28:46470) 2024-07-03T15:48:37.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.454+0000 7f316a7fc640 1 -- 172.21.15.28:0/1742318501 learned_addr learned my addr 172.21.15.28:0/1742318501 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:48:37.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.454+0000 7f316affd640 1 --2- 172.21.15.28:0/1742318501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f316c065990 0x7f316c10cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:48:37.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.455+0000 7f316a7fc640 1 -- 172.21.15.28:0/1742318501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f316c065990 msgr2=0x7f316c10cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:37.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.455+0000 7f316a7fc640 1 --2- 172.21.15.28:0/1742318501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f316c065990 0x7f316c10cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:37.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.455+0000 7f316a7fc640 1 -- 172.21.15.28:0/1742318501 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3154005040 con 0x7f316c0ff550 2024-07-03T15:48:37.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.455+0000 7f316affd640 1 --2- 172.21.15.28:0/1742318501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f316c065990 0x7f316c10cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-03T15:48:37.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.455+0000 7f316a7fc640 1 --2- 172.21.15.28:0/1742318501 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f316c0ff550 0x7f316c10d4c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3154009f80 tx=0x7f3154009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:48:37.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.455+0000 7f31637fe640 1 -- 172.21.15.28:0/1742318501 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3154038070 con 0x7f316c0ff550 2024-07-03T15:48:37.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.455+0000 7f31637fe640 1 -- 172.21.15.28:0/1742318501 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3154032970 con 0x7f316c0ff550 2024-07-03T15:48:37.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.455+0000 7f3171445640 1 -- 172.21.15.28:0/1742318501 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f316c10e560 con 0x7f316c0ff550 2024-07-03T15:48:37.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.455+0000 7f31637fe640 1 -- 172.21.15.28:0/1742318501 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3154033860 con 0x7f316c0ff550 2024-07-03T15:48:37.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.455+0000 7f3171445640 1 -- 172.21.15.28:0/1742318501 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f316c10eaa0 con 0x7f316c0ff550 2024-07-03T15:48:37.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.456+0000 7f3171445640 1 -- 172.21.15.28:0/1742318501 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f316c0fa6a0 con 0x7f316c0ff550 2024-07-03T15:48:37.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.457+0000 7f31637fe640 1 -- 172.21.15.28:0/1742318501 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3154030020 con 0x7f316c0ff550 2024-07-03T15:48:37.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.457+0000 7f31637fe640 1 --2- 172.21.15.28:0/1742318501 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f315806e0b0 0x7f3158070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:48:37.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.457+0000 7f31637fe640 1 -- 172.21.15.28:0/1742318501 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f31540ad9c0 con 0x7f316c0ff550 2024-07-03T15:48:37.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.457+0000 7f316affd640 1 --2- 172.21.15.28:0/1742318501 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f315806e0b0 0x7f3158070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:48:37.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.458+0000 7f316affd640 1 --2- 172.21.15.28:0/1742318501 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f315806e0b0 0x7f3158070560 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f316c0656f0 tx=0x7f315c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:48:37.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.461+0000 7f31637fe640 1 -- 172.21.15.28:0/1742318501 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f3154076dc0 con 0x7f316c0ff550 2024-07-03T15:48:37.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.664+0000 7f3171445640 1 -- 172.21.15.28:0/1742318501 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f316c10edf0 con 0x7f315806e0b0 2024-07-03T15:48:37.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.665+0000 7f31637fe640 1 -- 172.21.15.28:0/1742318501 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f316c10edf0 con 0x7f315806e0b0 2024-07-03T15:48:37.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.667+0000 7f3171445640 1 -- 172.21.15.28:0/1742318501 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f315806e0b0 msgr2=0x7f3158070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:37.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.667+0000 7f3171445640 1 --2- 172.21.15.28:0/1742318501 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f315806e0b0 0x7f3158070560 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f316c0656f0 tx=0x7f315c00b040 comp rx=0 tx=0).stop 2024-07-03T15:48:37.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.667+0000 7f3171445640 1 -- 172.21.15.28:0/1742318501 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f316c0ff550 msgr2=0x7f316c10d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:37.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.667+0000 7f3171445640 1 --2- 172.21.15.28:0/1742318501 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f316c0ff550 0x7f316c10d4c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3154009f80 tx=0x7f3154009da0 comp rx=0 tx=0).stop 2024-07-03T15:48:37.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.667+0000 7f3171445640 1 -- 172.21.15.28:0/1742318501 shutdown_connections 2024-07-03T15:48:37.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.667+0000 7f3171445640 1 --2- 172.21.15.28:0/1742318501 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f315806e0b0 0x7f3158070560 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:37.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.667+0000 7f3171445640 1 --2- 172.21.15.28:0/1742318501 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f316c0ff550 0x7f316c10d4c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:37.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.667+0000 7f3171445640 1 --2- 172.21.15.28:0/1742318501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f316c065990 0x7f316c10cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:37.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.667+0000 7f3171445640 1 -- 172.21.15.28:0/1742318501 >> 172.21.15.28:0/1742318501 conn(0x7f316c0f6a20 msgr2=0x7f316c0f6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:48:37.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.668+0000 7f3171445640 1 -- 172.21.15.28:0/1742318501 shutdown_connections 2024-07-03T15:48:37.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.668+0000 7f3171445640 1 -- 172.21.15.28:0/1742318501 wait complete. 2024-07-03T15:48:37.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.752+0000 7f2073e39640 1 -- 172.21.15.28:0/132175631 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f206c0fc380 msgr2=0x7f206c0fc790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:37.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.752+0000 7f2073e39640 1 --2- 172.21.15.28:0/132175631 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f206c0fc380 0x7f206c0fc790 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2068004a10 tx=0x7f2068009d40 comp rx=0 tx=0).stop 2024-07-03T15:48:37.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.752+0000 7f2073e39640 1 -- 172.21.15.28:0/132175631 shutdown_connections 2024-07-03T15:48:37.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.752+0000 7f2073e39640 1 --2- 172.21.15.28:0/132175631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f206c0fd650 0x7f206c0fdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:37.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.752+0000 7f2073e39640 1 --2- 172.21.15.28:0/132175631 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f206c0fc380 0x7f206c0fc790 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:37.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.752+0000 7f2073e39640 1 -- 172.21.15.28:0/132175631 >> 172.21.15.28:0/132175631 conn(0x7f206c0f7b40 msgr2=0x7f206c0f9f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:48:37.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.752+0000 7f2073e39640 1 -- 172.21.15.28:0/132175631 shutdown_connections 2024-07-03T15:48:37.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.752+0000 7f2073e39640 1 -- 172.21.15.28:0/132175631 wait complete. 2024-07-03T15:48:37.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.753+0000 7f2073e39640 1 Processor -- start 2024-07-03T15:48:37.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.753+0000 7f2073e39640 1 -- start start 2024-07-03T15:48:37.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.753+0000 7f2073e39640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f206c0fc380 0x7f206c10f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:48:37.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.753+0000 7f2073e39640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f206c0fd650 0x7f206c10f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:48:37.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.753+0000 7f2073e39640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f206c10fce0 con 0x7f206c0fd650 2024-07-03T15:48:37.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.753+0000 7f2073e39640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f206c110500 con 0x7f206c0fc380 2024-07-03T15:48:37.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.753+0000 7f20713d4640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f206c0fd650 0x7f206c10f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:48:37.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.753+0000 7f20713d4640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f206c0fd650 0x7f206c10f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46490/0 (socket says 172.21.15.28:46490) 2024-07-03T15:48:37.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.753+0000 7f20713d4640 1 -- 172.21.15.28:0/961297276 learned_addr learned my addr 172.21.15.28:0/961297276 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:48:37.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.753+0000 7f2071bd5640 1 --2- 172.21.15.28:0/961297276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f206c0fc380 0x7f206c10f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:48:37.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.753+0000 7f20713d4640 1 -- 172.21.15.28:0/961297276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f206c0fc380 msgr2=0x7f206c10f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:37.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.753+0000 7f20713d4640 1 --2- 172.21.15.28:0/961297276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f206c0fc380 0x7f206c10f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:37.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.753+0000 7f20713d4640 1 -- 172.21.15.28:0/961297276 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20680045d0 con 0x7f206c0fd650 2024-07-03T15:48:37.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.753+0000 7f20713d4640 1 --2- 172.21.15.28:0/961297276 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f206c0fd650 0x7f206c10f690 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f205c00eb70 tx=0x7f205c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:48:37.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.754+0000 7f20627fc640 1 -- 172.21.15.28:0/961297276 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f205c007c30 con 0x7f206c0fd650 2024-07-03T15:48:37.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.754+0000 7f20627fc640 1 -- 172.21.15.28:0/961297276 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f205c00f040 con 0x7f206c0fd650 2024-07-03T15:48:37.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.754+0000 7f20627fc640 1 -- 172.21.15.28:0/961297276 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f205c009680 con 0x7f206c0fd650 2024-07-03T15:48:37.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.754+0000 7f2073e39640 1 -- 172.21.15.28:0/961297276 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f206c110730 con 0x7f206c0fd650 2024-07-03T15:48:37.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.754+0000 7f2073e39640 1 -- 172.21.15.28:0/961297276 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f206c110cd0 con 0x7f206c0fd650 2024-07-03T15:48:37.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.755+0000 7f2073e39640 1 -- 172.21.15.28:0/961297276 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f206c0fb7e0 con 0x7f206c0fd650 2024-07-03T15:48:37.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.755+0000 7f20627fc640 1 -- 172.21.15.28:0/961297276 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f205c00cd40 con 0x7f206c0fd650 2024-07-03T15:48:37.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.756+0000 7f20627fc640 1 --2- 172.21.15.28:0/961297276 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f205806e0b0 0x7f2058070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:48:37.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.756+0000 7f20627fc640 1 -- 172.21.15.28:0/961297276 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f205c08d750 con 0x7f206c0fd650 2024-07-03T15:48:37.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.756+0000 7f2071bd5640 1 --2- 172.21.15.28:0/961297276 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f205806e0b0 0x7f2058070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:48:37.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.756+0000 7f2071bd5640 1 --2- 172.21.15.28:0/961297276 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f205806e0b0 0x7f2058070560 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2068032410 tx=0x7f2068035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:48:37.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.761+0000 7f20627fc640 1 -- 172.21.15.28:0/961297276 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f205c056c00 con 0x7f206c0fd650 2024-07-03T15:48:37.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.977+0000 7f2073e39640 1 -- 172.21.15.28:0/961297276 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f206c105080 con 0x7f205806e0b0 2024-07-03T15:48:37.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.987+0000 7f20627fc640 1 -- 172.21.15.28:0/961297276 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f2068008be0 con 0x7f205806e0b0 2024-07-03T15:48:37.991 INFO:teuthology.orchestra.run.smithi028.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-03T15:48:37.991 INFO:teuthology.orchestra.run.smithi028.stdout:alertmanager.smithi028 smithi028 *:9093,9094 running (4m) 25s ago 12m 25.8M - 0.25.0 c8568f914cd2 1425315a0de9 2024-07-03T15:48:37.991 INFO:teuthology.orchestra.run.smithi028.stdout:crash.smithi028 smithi028 running (88s) 25s ago 12m 6895k - 19.0.0-2550-gc00506dc 9d49405603e8 c62c2f45cabe 2024-07-03T15:48:37.991 INFO:teuthology.orchestra.run.smithi028.stdout:crash.smithi148 smithi148 running (84s) 80s ago 11m 6895k - 19.0.0-2550-gc00506dc 9d49405603e8 b1318dcaacb6 2024-07-03T15:48:37.991 INFO:teuthology.orchestra.run.smithi028.stdout:grafana.smithi028 smithi028 *:3000 running (3m) 25s ago 11m 80.3M - 9.4.7 954c08fa6188 f9f17c92cfea 2024-07-03T15:48:37.991 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi028.rhpvve smithi028 running (6m) 25s ago 6m 152M - 17.2.7-1047-gd4b063ab 4ca399d9e65d 3ed5abd02cfb 2024-07-03T15:48:37.991 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi028.wmbxew smithi028 running (6m) 25s ago 6m 110M - 17.2.7-1047-gd4b063ab 4ca399d9e65d b3bf736998f9 2024-07-03T15:48:37.991 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi148.szomfl smithi148 running (6m) 80s ago 6m 15.7M - 17.2.7-1047-gd4b063ab 4ca399d9e65d d94f5c95fa8d 2024-07-03T15:48:37.991 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi148.xgpuld smithi148 running (6m) 80s ago 6m 13.2M - 17.2.7-1047-gd4b063ab 4ca399d9e65d fc70360a8de0 2024-07-03T15:48:37.992 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.smithi028.vtqrtz smithi028 *:8443,9283,8765 running (3m) 25s ago 13m 495M - 19.0.0-2550-gc00506dc 9d49405603e8 14d0363c5b77 2024-07-03T15:48:37.992 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.smithi148.rjzsar smithi148 *:8443,9283,8765 running (2m) 80s ago 11m 438M - 19.0.0-2550-gc00506dc 9d49405603e8 c0193aa8a5dd 2024-07-03T15:48:37.992 INFO:teuthology.orchestra.run.smithi028.stdout:mon.smithi028 smithi028 running (2m) 25s ago 13m 45.8M 2048M 19.0.0-2550-gc00506dc 9d49405603e8 5093f7bf47d7 2024-07-03T15:48:37.992 INFO:teuthology.orchestra.run.smithi028.stdout:mon.smithi148 smithi148 running (2m) 80s ago 11m 33.6M 2048M 19.0.0-2550-gc00506dc 9d49405603e8 d338d2c648da 2024-07-03T15:48:37.992 INFO:teuthology.orchestra.run.smithi028.stdout:node-exporter.smithi028 smithi028 *:9100 running (4m) 25s ago 12m 15.4M - 1.5.0 0da6a335fe13 91ca1f49de0e 2024-07-03T15:48:37.992 INFO:teuthology.orchestra.run.smithi028.stdout:node-exporter.smithi148 smithi148 *:9100 running (3m) 80s ago 11m 15.6M - 1.5.0 0da6a335fe13 1eaaf7b1b561 2024-07-03T15:48:37.992 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0 smithi028 running (64s) 25s ago 10m 101M 1558M 19.0.0-2550-gc00506dc 9d49405603e8 4454cecff8fc 2024-07-03T15:48:37.992 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1 smithi028 running (30s) 25s ago 10m 11.7M 1558M 19.0.0-2550-gc00506dc 9d49405603e8 af67f7088643 2024-07-03T15:48:37.992 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2 smithi028 running (9m) 25s ago 9m 217M 1558M 17.2.7-1047-gd4b063ab 4ca399d9e65d b2cbb312212d 2024-07-03T15:48:37.992 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3 smithi148 running (9m) 80s ago 9m 278M 2581M 17.2.7-1047-gd4b063ab 4ca399d9e65d 5be69a27df31 2024-07-03T15:48:37.992 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4 smithi148 running (8m) 80s ago 8m 250M 2581M 17.2.7-1047-gd4b063ab 4ca399d9e65d 54f7c26c77ca 2024-07-03T15:48:37.992 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5 smithi148 running (8m) 80s ago 8m 239M 2581M 17.2.7-1047-gd4b063ab 4ca399d9e65d d185c6905384 2024-07-03T15:48:37.993 INFO:teuthology.orchestra.run.smithi028.stdout:prometheus.smithi028 smithi028 *:9095 running (2m) 25s ago 11m 54.4M - 2.43.0 a07b618ecd1d 2b0eb2a95afd 2024-07-03T15:48:37.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.989+0000 7f2073e39640 1 -- 172.21.15.28:0/961297276 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f205806e0b0 msgr2=0x7f2058070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:37.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.989+0000 7f2073e39640 1 --2- 172.21.15.28:0/961297276 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f205806e0b0 0x7f2058070560 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2068032410 tx=0x7f2068035040 comp rx=0 tx=0).stop 2024-07-03T15:48:37.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.989+0000 7f2073e39640 1 -- 172.21.15.28:0/961297276 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f206c0fd650 msgr2=0x7f206c10f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:37.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.989+0000 7f2073e39640 1 --2- 172.21.15.28:0/961297276 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f206c0fd650 0x7f206c10f690 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f205c00eb70 tx=0x7f205c00b040 comp rx=0 tx=0).stop 2024-07-03T15:48:37.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.989+0000 7f2073e39640 1 -- 172.21.15.28:0/961297276 shutdown_connections 2024-07-03T15:48:37.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.989+0000 7f2073e39640 1 --2- 172.21.15.28:0/961297276 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f205806e0b0 0x7f2058070560 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:37.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.989+0000 7f2073e39640 1 --2- 172.21.15.28:0/961297276 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f206c0fd650 0x7f206c10f690 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:37.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.989+0000 7f2073e39640 1 --2- 172.21.15.28:0/961297276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f206c0fc380 0x7f206c10f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:37.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.989+0000 7f2073e39640 1 -- 172.21.15.28:0/961297276 >> 172.21.15.28:0/961297276 conn(0x7f206c0f7b40 msgr2=0x7f206c0f90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:48:37.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.989+0000 7f2073e39640 1 -- 172.21.15.28:0/961297276 shutdown_connections 2024-07-03T15:48:37.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:37.989+0000 7f2073e39640 1 -- 172.21.15.28:0/961297276 wait complete. 2024-07-03T15:48:38.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.085+0000 7fd5c88d3640 1 -- 172.21.15.28:0/486292151 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd5c40fb240 msgr2=0x7fd5c40fb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:38.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.085+0000 7fd5c88d3640 1 --2- 172.21.15.28:0/486292151 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd5c40fb240 0x7fd5c40fb650 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd5b40048d0 tx=0x7fd5b4009da0 comp rx=0 tx=0).stop 2024-07-03T15:48:38.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.085+0000 7fd5c88d3640 1 -- 172.21.15.28:0/486292151 shutdown_connections 2024-07-03T15:48:38.090 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:37 smithi028 ceph-mon[92858]: pgmap v80: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:48:38.090 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:37 smithi028 ceph-mon[92858]: Health check failed: 1 osds down (OSD_DOWN) 2024-07-03T15:48:38.090 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:37 smithi028 ceph-mon[92858]: osdmap e59: 6 total, 5 up, 6 in 2024-07-03T15:48:38.090 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:37 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:38.090 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:37 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:48:38.090 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:37 smithi028 podman[111563]: 2024-07-03 15:48:37.775062726 +0000 UTC m=+1.837440685 container cleanup b2cbb312212d711566745e4bff155a8357fa823280c356f40594cca7b1c80074 (image=quay.ceph.io/ceph-ci/ceph@sha256:0b7651540cdb24c1b97466c79c298d54c7794c5f53d5d6b6c5d76a2d3b130d85, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240625, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-d4b063a, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-03T15:48:38.091 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:37 smithi028 bash[111563]: ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-2 2024-07-03T15:48:38.091 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:37 smithi028 podman[111602]: 2024-07-03 15:48:37.965505852 +0000 UTC m=+0.632442280 container remove b2cbb312212d711566745e4bff155a8357fa823280c356f40594cca7b1c80074 (image=quay.ceph.io/ceph-ci/ceph@sha256:0b7651540cdb24c1b97466c79c298d54c7794c5f53d5d6b6c5d76a2d3b130d85, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240625, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=quincy-d4b063a, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux ) 2024-07-03T15:48:38.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.085+0000 7fd5c88d3640 1 --2- 172.21.15.28:0/486292151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5c40fc510 0x7fd5c40fc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:38.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.085+0000 7fd5c88d3640 1 --2- 172.21.15.28:0/486292151 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd5c40fb240 0x7fd5c40fb650 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:38.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.085+0000 7fd5c88d3640 1 -- 172.21.15.28:0/486292151 >> 172.21.15.28:0/486292151 conn(0x7fd5c40f6a20 msgr2=0x7fd5c40f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:48:38.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.086+0000 7fd5c88d3640 1 -- 172.21.15.28:0/486292151 shutdown_connections 2024-07-03T15:48:38.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.086+0000 7fd5c88d3640 1 -- 172.21.15.28:0/486292151 wait complete. 2024-07-03T15:48:38.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.086+0000 7fd5c88d3640 1 Processor -- start 2024-07-03T15:48:38.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.086+0000 7fd5c88d3640 1 -- start start 2024-07-03T15:48:38.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.086+0000 7fd5c88d3640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd5c40fb240 0x7fd5c410cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:48:38.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.086+0000 7fd5c88d3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5c40fc510 0x7fd5c410d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:48:38.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.086+0000 7fd5c88d3640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c410db20 con 0x7fd5c40fb240 2024-07-03T15:48:38.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.086+0000 7fd5c88d3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c410e2f0 con 0x7fd5c40fc510 2024-07-03T15:48:38.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.086+0000 7fd5c259c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd5c40fb240 0x7fd5c410cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:48:38.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.087+0000 7fd5c259c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd5c40fb240 0x7fd5c410cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46508/0 (socket says 172.21.15.28:46508) 2024-07-03T15:48:38.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.087+0000 7fd5c259c640 1 -- 172.21.15.28:0/2513532509 learned_addr learned my addr 172.21.15.28:0/2513532509 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:48:38.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.087+0000 7fd5bbfff640 1 --2- 172.21.15.28:0/2513532509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5c40fc510 0x7fd5c410d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:48:38.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.087+0000 7fd5c259c640 1 -- 172.21.15.28:0/2513532509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5c40fc510 msgr2=0x7fd5c410d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:38.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.087+0000 7fd5c259c640 1 --2- 172.21.15.28:0/2513532509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5c40fc510 0x7fd5c410d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:38.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.087+0000 7fd5c259c640 1 -- 172.21.15.28:0/2513532509 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b4005040 con 0x7fd5c40fb240 2024-07-03T15:48:38.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.087+0000 7fd5c259c640 1 --2- 172.21.15.28:0/2513532509 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd5c40fb240 0x7fd5c410cf90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4009f80 tx=0x7fd5b4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:48:38.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.087+0000 7fd5baffd640 1 -- 172.21.15.28:0/2513532509 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5b4038070 con 0x7fd5c40fb240 2024-07-03T15:48:38.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.087+0000 7fd5c88d3640 1 -- 172.21.15.28:0/2513532509 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5c410e520 con 0x7fd5c40fb240 2024-07-03T15:48:38.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.087+0000 7fd5baffd640 1 -- 172.21.15.28:0/2513532509 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd5b4032970 con 0x7fd5c40fb240 2024-07-03T15:48:38.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.087+0000 7fd5c88d3640 1 -- 172.21.15.28:0/2513532509 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5c410ea60 con 0x7fd5c40fb240 2024-07-03T15:48:38.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.087+0000 7fd5baffd640 1 -- 172.21.15.28:0/2513532509 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5b4033860 con 0x7fd5c40fb240 2024-07-03T15:48:38.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.089+0000 7fd5c88d3640 1 -- 172.21.15.28:0/2513532509 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5c40fa6a0 con 0x7fd5c40fb240 2024-07-03T15:48:38.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.089+0000 7fd5baffd640 1 -- 172.21.15.28:0/2513532509 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd5b4030020 con 0x7fd5c40fb240 2024-07-03T15:48:38.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.090+0000 7fd5baffd640 1 --2- 172.21.15.28:0/2513532509 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fd5b006e0b0 0x7fd5b0070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:48:38.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.090+0000 7fd5baffd640 1 -- 172.21.15.28:0/2513532509 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fd5b40ad8f0 con 0x7fd5c40fb240 2024-07-03T15:48:38.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.090+0000 7fd5bbfff640 1 --2- 172.21.15.28:0/2513532509 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fd5b006e0b0 0x7fd5b0070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:48:38.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.091+0000 7fd5bbfff640 1 --2- 172.21.15.28:0/2513532509 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fd5b006e0b0 0x7fd5b0070560 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd5c410cd40 tx=0x7fd5ac00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:48:38.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.098+0000 7fd5baffd640 1 -- 172.21.15.28:0/2513532509 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd5b4076da0 con 0x7fd5c40fb240 2024-07-03T15:48:38.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:37 smithi148 ceph-mon[80490]: pgmap v80: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:48:38.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:37 smithi148 ceph-mon[80490]: Health check failed: 1 osds down (OSD_DOWN) 2024-07-03T15:48:38.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:37 smithi148 ceph-mon[80490]: osdmap e59: 6 total, 5 up, 6 in 2024-07-03T15:48:38.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:37 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:38.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:37 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:48:38.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.401+0000 7fd5c88d3640 1 -- 172.21.15.28:0/2513532509 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd5c4103e30 con 0x7fd5c40fb240 2024-07-03T15:48:38.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.402+0000 7fd5baffd640 1 -- 172.21.15.28:0/2513532509 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+838 (secure 0 0 0) 0x7fd5b407a460 con 0x7fd5c40fb240 2024-07-03T15:48:38.406 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-07-03T15:48:38.406 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": { 2024-07-03T15:48:38.406 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 2 2024-07-03T15:48:38.406 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:48:38.406 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": { 2024-07-03T15:48:38.406 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 2 2024-07-03T15:48:38.406 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:48:38.407 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": { 2024-07-03T15:48:38.407 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 3, 2024-07-03T15:48:38.407 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 2 2024-07-03T15:48:38.407 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:48:38.407 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": { 2024-07-03T15:48:38.407 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 4 2024-07-03T15:48:38.407 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:48:38.407 INFO:teuthology.orchestra.run.smithi028.stdout: "overall": { 2024-07-03T15:48:38.407 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 7, 2024-07-03T15:48:38.407 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 6 2024-07-03T15:48:38.407 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-07-03T15:48:38.408 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-07-03T15:48:38.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.404+0000 7fd5c88d3640 1 -- 172.21.15.28:0/2513532509 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fd5b006e0b0 msgr2=0x7fd5b0070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:38.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.404+0000 7fd5c88d3640 1 --2- 172.21.15.28:0/2513532509 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fd5b006e0b0 0x7fd5b0070560 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd5c410cd40 tx=0x7fd5ac00b040 comp rx=0 tx=0).stop 2024-07-03T15:48:38.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.404+0000 7fd5c88d3640 1 -- 172.21.15.28:0/2513532509 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd5c40fb240 msgr2=0x7fd5c410cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:38.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.404+0000 7fd5c88d3640 1 --2- 172.21.15.28:0/2513532509 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd5c40fb240 0x7fd5c410cf90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4009f80 tx=0x7fd5b4009da0 comp rx=0 tx=0).stop 2024-07-03T15:48:38.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.404+0000 7fd5c88d3640 1 -- 172.21.15.28:0/2513532509 shutdown_connections 2024-07-03T15:48:38.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.404+0000 7fd5c88d3640 1 --2- 172.21.15.28:0/2513532509 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fd5b006e0b0 0x7fd5b0070560 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:38.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.404+0000 7fd5c88d3640 1 --2- 172.21.15.28:0/2513532509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5c40fc510 0x7fd5c410d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:38.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.404+0000 7fd5c88d3640 1 --2- 172.21.15.28:0/2513532509 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd5c40fb240 0x7fd5c410cf90 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:38.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.404+0000 7fd5c88d3640 1 -- 172.21.15.28:0/2513532509 >> 172.21.15.28:0/2513532509 conn(0x7fd5c40f6a20 msgr2=0x7fd5c4065330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:48:38.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.404+0000 7fd5c88d3640 1 -- 172.21.15.28:0/2513532509 shutdown_connections 2024-07-03T15:48:38.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.404+0000 7fd5c88d3640 1 -- 172.21.15.28:0/2513532509 wait complete. 2024-07-03T15:48:38.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.486+0000 7f5580325640 1 -- 172.21.15.28:0/1764574201 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f55780faa10 msgr2=0x7f55780fae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:38.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.486+0000 7f5580325640 1 --2- 172.21.15.28:0/1764574201 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f55780faa10 0x7f55780fae20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5568009c50 tx=0x7f5568004d10 comp rx=0 tx=0).stop 2024-07-03T15:48:38.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.486+0000 7f5580325640 1 -- 172.21.15.28:0/1764574201 shutdown_connections 2024-07-03T15:48:38.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.486+0000 7f5580325640 1 --2- 172.21.15.28:0/1764574201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55780fbce0 0x7f55780fc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:38.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.486+0000 7f5580325640 1 --2- 172.21.15.28:0/1764574201 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f55780faa10 0x7f55780fae20 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:38.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.486+0000 7f5580325640 1 -- 172.21.15.28:0/1764574201 >> 172.21.15.28:0/1764574201 conn(0x7f55780f61d0 msgr2=0x7f55780f8600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:48:38.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.486+0000 7f5580325640 1 -- 172.21.15.28:0/1764574201 shutdown_connections 2024-07-03T15:48:38.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.486+0000 7f5580325640 1 -- 172.21.15.28:0/1764574201 wait complete. 2024-07-03T15:48:38.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.486+0000 7f5580325640 1 Processor -- start 2024-07-03T15:48:38.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.487+0000 7f5580325640 1 -- start start 2024-07-03T15:48:38.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.487+0000 7f5580325640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f55780faa10 0x7f557810f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:48:38.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.487+0000 7f5580325640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55780fbce0 0x7f557810f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:48:38.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.487+0000 7f5580325640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5578110640 con 0x7f55780faa10 2024-07-03T15:48:38.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.487+0000 7f5580325640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55781107b0 con 0x7f55780fbce0 2024-07-03T15:48:38.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.487+0000 7f557e0c1640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f55780faa10 0x7f557810f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:48:38.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.487+0000 7f557e0c1640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f55780faa10 0x7f557810f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46520/0 (socket says 172.21.15.28:46520) 2024-07-03T15:48:38.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.487+0000 7f557e0c1640 1 -- 172.21.15.28:0/2946325616 learned_addr learned my addr 172.21.15.28:0/2946325616 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:48:38.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.487+0000 7f557d8c0640 1 --2- 172.21.15.28:0/2946325616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55780fbce0 0x7f557810f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:48:38.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.487+0000 7f557e0c1640 1 -- 172.21.15.28:0/2946325616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55780fbce0 msgr2=0x7f557810f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:38.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.487+0000 7f557e0c1640 1 --2- 172.21.15.28:0/2946325616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55780fbce0 0x7f557810f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:38.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.487+0000 7f557e0c1640 1 -- 172.21.15.28:0/2946325616 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55680098b0 con 0x7f55780faa10 2024-07-03T15:48:38.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.487+0000 7f557e0c1640 1 --2- 172.21.15.28:0/2946325616 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f55780faa10 0x7f557810f290 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5568009c20 tx=0x7f5568004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:48:38.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.488+0000 7f5576ffd640 1 -- 172.21.15.28:0/2946325616 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5568032730 con 0x7f55780faa10 2024-07-03T15:48:38.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.488+0000 7f5576ffd640 1 -- 172.21.15.28:0/2946325616 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5568032d60 con 0x7f55780faa10 2024-07-03T15:48:38.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.488+0000 7f5580325640 1 -- 172.21.15.28:0/2946325616 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5578110a30 con 0x7f55780faa10 2024-07-03T15:48:38.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.488+0000 7f5576ffd640 1 -- 172.21.15.28:0/2946325616 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f556803dd20 con 0x7f55780faa10 2024-07-03T15:48:38.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.488+0000 7f5580325640 1 -- 172.21.15.28:0/2946325616 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5578110f50 con 0x7f55780faa10 2024-07-03T15:48:38.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.489+0000 7f5580325640 1 -- 172.21.15.28:0/2946325616 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5578104680 con 0x7f55780faa10 2024-07-03T15:48:38.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.489+0000 7f5576ffd640 1 -- 172.21.15.28:0/2946325616 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f556802f030 con 0x7f55780faa10 2024-07-03T15:48:38.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.490+0000 7f5576ffd640 1 --2- 172.21.15.28:0/2946325616 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f556006e0b0 0x7f5560070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:48:38.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.490+0000 7f5576ffd640 1 -- 172.21.15.28:0/2946325616 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f55680af1c0 con 0x7f55780faa10 2024-07-03T15:48:38.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.490+0000 7f557d8c0640 1 --2- 172.21.15.28:0/2946325616 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f556006e0b0 0x7f5560070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:48:38.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.490+0000 7f557d8c0640 1 --2- 172.21.15.28:0/2946325616 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f556006e0b0 0x7f5560070560 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f557810f040 tx=0x7f556c008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:48:38.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.495+0000 7f5576ffd640 1 -- 172.21.15.28:0/2946325616 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f5568078670 con 0x7f55780faa10 2024-07-03T15:48:38.663 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:38 smithi028 podman[111752]: 2024-07-03 15:48:38.370177171 +0000 UTC m=+0.018219160 image pull 9d49405603e8445bc510362cdeb05f243619be76b0f3897dd4c152b00cffe45f quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2 2024-07-03T15:48:38.663 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:38 smithi028 podman[111752]: 2024-07-03 15:48:38.491613606 +0000 UTC m=+0.139655586 container create fe9650ca3e3aec0f5811bd8579b4e147011b6879f08778a08b098c3d5cc32149 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, RELEASE=squid-release-c00506d, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240625, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-03T15:48:38.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.741+0000 7f5580325640 1 -- 172.21.15.28:0/2946325616 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f557804b530 con 0x7f55780faa10 2024-07-03T15:48:38.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.741+0000 7f5576ffd640 1 -- 172.21.15.28:0/2946325616 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1908 (secure 0 0 0) 0x7f556807bd30 con 0x7f55780faa10 2024-07-03T15:48:38.746 INFO:teuthology.orchestra.run.smithi028.stdout:e17 2024-07-03T15:48:38.746 INFO:teuthology.orchestra.run.smithi028.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-03T15:48:38.746 INFO:teuthology.orchestra.run.smithi028.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:48:38.746 INFO:teuthology.orchestra.run.smithi028.stdout:legacy client fscid: 1 2024-07-03T15:48:38.746 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:48:38.746 INFO:teuthology.orchestra.run.smithi028.stdout:Filesystem 'cephfs' (1) 2024-07-03T15:48:38.747 INFO:teuthology.orchestra.run.smithi028.stdout:fs_name cephfs 2024-07-03T15:48:38.747 INFO:teuthology.orchestra.run.smithi028.stdout:epoch 16 2024-07-03T15:48:38.747 INFO:teuthology.orchestra.run.smithi028.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-03T15:48:38.747 INFO:teuthology.orchestra.run.smithi028.stdout:created 2024-07-03T15:41:40.656298+0000 2024-07-03T15:48:38.747 INFO:teuthology.orchestra.run.smithi028.stdout:modified 2024-07-03T15:41:56.454984+0000 2024-07-03T15:48:38.747 INFO:teuthology.orchestra.run.smithi028.stdout:tableserver 0 2024-07-03T15:48:38.747 INFO:teuthology.orchestra.run.smithi028.stdout:root 0 2024-07-03T15:48:38.747 INFO:teuthology.orchestra.run.smithi028.stdout:session_timeout 60 2024-07-03T15:48:38.747 INFO:teuthology.orchestra.run.smithi028.stdout:session_autoclose 300 2024-07-03T15:48:38.747 INFO:teuthology.orchestra.run.smithi028.stdout:max_file_size 1099511627776 2024-07-03T15:48:38.747 INFO:teuthology.orchestra.run.smithi028.stdout:max_xattr_size 65536 2024-07-03T15:48:38.747 INFO:teuthology.orchestra.run.smithi028.stdout:required_client_features {} 2024-07-03T15:48:38.748 INFO:teuthology.orchestra.run.smithi028.stdout:last_failure 0 2024-07-03T15:48:38.748 INFO:teuthology.orchestra.run.smithi028.stdout:last_failure_osd_epoch 0 2024-07-03T15:48:38.748 INFO:teuthology.orchestra.run.smithi028.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:48:38.748 INFO:teuthology.orchestra.run.smithi028.stdout:max_mds 1 2024-07-03T15:48:38.748 INFO:teuthology.orchestra.run.smithi028.stdout:in 0 2024-07-03T15:48:38.748 INFO:teuthology.orchestra.run.smithi028.stdout:up {0=14450} 2024-07-03T15:48:38.748 INFO:teuthology.orchestra.run.smithi028.stdout:failed 2024-07-03T15:48:38.748 INFO:teuthology.orchestra.run.smithi028.stdout:damaged 2024-07-03T15:48:38.748 INFO:teuthology.orchestra.run.smithi028.stdout:stopped 2024-07-03T15:48:38.748 INFO:teuthology.orchestra.run.smithi028.stdout:data_pools [3] 2024-07-03T15:48:38.748 INFO:teuthology.orchestra.run.smithi028.stdout:metadata_pool 2 2024-07-03T15:48:38.748 INFO:teuthology.orchestra.run.smithi028.stdout:inline_data disabled 2024-07-03T15:48:38.748 INFO:teuthology.orchestra.run.smithi028.stdout:balancer 2024-07-03T15:48:38.749 INFO:teuthology.orchestra.run.smithi028.stdout:bal_rank_mask -1 2024-07-03T15:48:38.749 INFO:teuthology.orchestra.run.smithi028.stdout:standby_count_wanted 1 2024-07-03T15:48:38.749 INFO:teuthology.orchestra.run.smithi028.stdout:qdb_cluster leader: 0 members: 2024-07-03T15:48:38.749 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi028.rhpvve{0:14450} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.28:6826/1613936042,v1:172.21.15.28:6827/1613936042] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:48:38.749 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi028.wmbxew{0:14460} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.28:6828/1730336821,v1:172.21.15.28:6829/1730336821] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:48:38.749 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:48:38.749 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:48:38.749 INFO:teuthology.orchestra.run.smithi028.stdout:Standby daemons: 2024-07-03T15:48:38.749 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:48:38.749 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi148.szomfl{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/4171231858,v1:172.21.15.148:6827/4171231858] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:48:38.749 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi148.xgpuld{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/2347536398,v1:172.21.15.148:6825/2347536398] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:48:38.750 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 17 2024-07-03T15:48:38.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.743+0000 7f5580325640 1 -- 172.21.15.28:0/2946325616 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f556006e0b0 msgr2=0x7f5560070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:38.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.743+0000 7f5580325640 1 --2- 172.21.15.28:0/2946325616 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f556006e0b0 0x7f5560070560 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f557810f040 tx=0x7f556c008040 comp rx=0 tx=0).stop 2024-07-03T15:48:38.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.743+0000 7f5580325640 1 -- 172.21.15.28:0/2946325616 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f55780faa10 msgr2=0x7f557810f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:38.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.743+0000 7f5580325640 1 --2- 172.21.15.28:0/2946325616 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f55780faa10 0x7f557810f290 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5568009c20 tx=0x7f5568004d10 comp rx=0 tx=0).stop 2024-07-03T15:48:38.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.744+0000 7f5580325640 1 -- 172.21.15.28:0/2946325616 shutdown_connections 2024-07-03T15:48:38.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.744+0000 7f5580325640 1 --2- 172.21.15.28:0/2946325616 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f556006e0b0 0x7f5560070560 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:38.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.744+0000 7f5580325640 1 --2- 172.21.15.28:0/2946325616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55780fbce0 0x7f557810f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:38.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.744+0000 7f5580325640 1 --2- 172.21.15.28:0/2946325616 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f55780faa10 0x7f557810f290 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:38.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.744+0000 7f5580325640 1 -- 172.21.15.28:0/2946325616 >> 172.21.15.28:0/2946325616 conn(0x7f55780f61d0 msgr2=0x7f55780f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:48:38.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.744+0000 7f5580325640 1 -- 172.21.15.28:0/2946325616 shutdown_connections 2024-07-03T15:48:38.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.744+0000 7f5580325640 1 -- 172.21.15.28:0/2946325616 wait complete. 2024-07-03T15:48:38.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.831+0000 7f31a6dd4640 1 -- 172.21.15.28:0/1418219893 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31a00fb0c0 msgr2=0x7f31a00fd940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:38.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.831+0000 7f31a6dd4640 1 --2- 172.21.15.28:0/1418219893 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31a00fb0c0 0x7f31a00fd940 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f318c0048d0 tx=0x7f318c009da0 comp rx=0 tx=0).stop 2024-07-03T15:48:38.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.831+0000 7f31a6dd4640 1 -- 172.21.15.28:0/1418219893 shutdown_connections 2024-07-03T15:48:38.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.831+0000 7f31a6dd4640 1 --2- 172.21.15.28:0/1418219893 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31a00fb0c0 0x7f31a00fd940 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:38.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.831+0000 7f31a6dd4640 1 --2- 172.21.15.28:0/1418219893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31a00fa770 0x7f31a00fab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:38.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.831+0000 7f31a6dd4640 1 -- 172.21.15.28:0/1418219893 >> 172.21.15.28:0/1418219893 conn(0x7f31a00f61b0 msgr2=0x7f31a00f85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:48:38.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.832+0000 7f31a6dd4640 1 -- 172.21.15.28:0/1418219893 shutdown_connections 2024-07-03T15:48:38.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.832+0000 7f31a6dd4640 1 -- 172.21.15.28:0/1418219893 wait complete. 2024-07-03T15:48:38.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.832+0000 7f31a6dd4640 1 Processor -- start 2024-07-03T15:48:38.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.832+0000 7f31a6dd4640 1 -- start start 2024-07-03T15:48:38.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.833+0000 7f31a6dd4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31a00fa770 0x7f31a010d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:48:38.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.833+0000 7f31a6dd4640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31a00fb0c0 0x7f31a010d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:48:38.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.833+0000 7f31a6dd4640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a010dd50 con 0x7f31a00fb0c0 2024-07-03T15:48:38.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.833+0000 7f31a6dd4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a010e570 con 0x7f31a00fa770 2024-07-03T15:48:38.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.833+0000 7f319ffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31a00fb0c0 0x7f31a010d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:48:38.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.833+0000 7f319ffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31a00fb0c0 0x7f31a010d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46528/0 (socket says 172.21.15.28:46528) 2024-07-03T15:48:38.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.833+0000 7f319ffff640 1 -- 172.21.15.28:0/3993440667 learned_addr learned my addr 172.21.15.28:0/3993440667 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:48:38.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.833+0000 7f31a4b70640 1 --2- 172.21.15.28:0/3993440667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31a00fa770 0x7f31a010d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:48:38.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.833+0000 7f319ffff640 1 -- 172.21.15.28:0/3993440667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31a00fa770 msgr2=0x7f31a010d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:38.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.833+0000 7f319ffff640 1 --2- 172.21.15.28:0/3993440667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31a00fa770 0x7f31a010d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:38.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.833+0000 7f319ffff640 1 -- 172.21.15.28:0/3993440667 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f318c005040 con 0x7f31a00fb0c0 2024-07-03T15:48:38.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.834+0000 7f319ffff640 1 --2- 172.21.15.28:0/3993440667 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31a00fb0c0 0x7f31a010d700 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f318c000c00 tx=0x7f318c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:48:38.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.834+0000 7f319d7fa640 1 -- 172.21.15.28:0/3993440667 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f318c038070 con 0x7f31a00fb0c0 2024-07-03T15:48:38.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.834+0000 7f319d7fa640 1 -- 172.21.15.28:0/3993440667 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f318c032970 con 0x7f31a00fb0c0 2024-07-03T15:48:38.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.834+0000 7f31a6dd4640 1 -- 172.21.15.28:0/3993440667 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31a010e7a0 con 0x7f31a00fb0c0 2024-07-03T15:48:38.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.834+0000 7f319d7fa640 1 -- 172.21.15.28:0/3993440667 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f318c0338e0 con 0x7f31a00fb0c0 2024-07-03T15:48:38.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.834+0000 7f31a6dd4640 1 -- 172.21.15.28:0/3993440667 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31a010ece0 con 0x7f31a00fb0c0 2024-07-03T15:48:38.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.835+0000 7f31a6dd4640 1 -- 172.21.15.28:0/3993440667 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31a00f9e50 con 0x7f31a00fb0c0 2024-07-03T15:48:38.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.836+0000 7f319d7fa640 1 -- 172.21.15.28:0/3993440667 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f318c030020 con 0x7f31a00fb0c0 2024-07-03T15:48:38.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.836+0000 7f319d7fa640 1 --2- 172.21.15.28:0/3993440667 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f319006e0b0 0x7f3190070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:48:38.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.836+0000 7f319d7fa640 1 -- 172.21.15.28:0/3993440667 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f318c0ad980 con 0x7f31a00fb0c0 2024-07-03T15:48:38.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.836+0000 7f31a4b70640 1 --2- 172.21.15.28:0/3993440667 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f319006e0b0 0x7f3190070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:48:38.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.837+0000 7f31a4b70640 1 --2- 172.21.15.28:0/3993440667 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f319006e0b0 0x7f3190070560 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3194007fd0 tx=0x7f3194009e90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:48:38.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:38.841+0000 7f319d7fa640 1 -- 172.21.15.28:0/3993440667 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f318c076d20 con 0x7f31a00fb0c0 2024-07-03T15:48:39.062 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:38 smithi028 podman[111752]: 2024-07-03 15:48:38.761529285 +0000 UTC m=+0.409571278 container init fe9650ca3e3aec0f5811bd8579b4e147011b6879f08778a08b098c3d5cc32149 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-release-c00506d, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240625, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-03T15:48:39.062 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:38 smithi028 podman[111752]: 2024-07-03 15:48:38.764947393 +0000 UTC m=+0.412989366 container start fe9650ca3e3aec0f5811bd8579b4e147011b6879f08778a08b098c3d5cc32149 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240625, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-release-c00506d) 2024-07-03T15:48:39.062 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:38 smithi028 podman[111752]: 2024-07-03 15:48:38.861674365 +0000 UTC m=+0.509716340 container attach fe9650ca3e3aec0f5811bd8579b4e147011b6879f08778a08b098c3d5cc32149 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240625, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-release-c00506d, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-03T15:48:39.062 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:38 smithi028 ceph-mon[92858]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:48:39.062 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:38 smithi028 ceph-mon[92858]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:48:39.062 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:38 smithi028 ceph-mon[92858]: osdmap e60: 6 total, 5 up, 6 in 2024-07-03T15:48:39.063 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:38 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/2513532509' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:48:39.063 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:38 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/2946325616' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-03T15:48:39.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.068+0000 7f31a6dd4640 1 -- 172.21.15.28:0/3993440667 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f31a01052a0 con 0x7f319006e0b0 2024-07-03T15:48:39.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.070+0000 7f319d7fa640 1 -- 172.21.15.28:0/3993440667 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f31a01052a0 con 0x7f319006e0b0 2024-07-03T15:48:39.074 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-07-03T15:48:39.074 INFO:teuthology.orchestra.run.smithi028.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2", 2024-07-03T15:48:39.074 INFO:teuthology.orchestra.run.smithi028.stdout: "in_progress": true, 2024-07-03T15:48:39.074 INFO:teuthology.orchestra.run.smithi028.stdout: "which": "Upgrading all daemon types on all hosts", 2024-07-03T15:48:39.074 INFO:teuthology.orchestra.run.smithi028.stdout: "services_complete": [ 2024-07-03T15:48:39.074 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr", 2024-07-03T15:48:39.074 INFO:teuthology.orchestra.run.smithi028.stdout: "crash", 2024-07-03T15:48:39.074 INFO:teuthology.orchestra.run.smithi028.stdout: "mon" 2024-07-03T15:48:39.074 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-07-03T15:48:39.075 INFO:teuthology.orchestra.run.smithi028.stdout: "progress": "8/21 daemons upgraded", 2024-07-03T15:48:39.075 INFO:teuthology.orchestra.run.smithi028.stdout: "message": "Currently upgrading osd daemons", 2024-07-03T15:48:39.075 INFO:teuthology.orchestra.run.smithi028.stdout: "is_paused": false 2024-07-03T15:48:39.075 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-07-03T15:48:39.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.072+0000 7f31a6dd4640 1 -- 172.21.15.28:0/3993440667 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f319006e0b0 msgr2=0x7f3190070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:39.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.072+0000 7f31a6dd4640 1 --2- 172.21.15.28:0/3993440667 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f319006e0b0 0x7f3190070560 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3194007fd0 tx=0x7f3194009e90 comp rx=0 tx=0).stop 2024-07-03T15:48:39.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.072+0000 7f31a6dd4640 1 -- 172.21.15.28:0/3993440667 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31a00fb0c0 msgr2=0x7f31a010d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:39.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.072+0000 7f31a6dd4640 1 --2- 172.21.15.28:0/3993440667 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31a00fb0c0 0x7f31a010d700 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f318c000c00 tx=0x7f318c009da0 comp rx=0 tx=0).stop 2024-07-03T15:48:39.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.072+0000 7f31a6dd4640 1 -- 172.21.15.28:0/3993440667 shutdown_connections 2024-07-03T15:48:39.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.072+0000 7f31a6dd4640 1 --2- 172.21.15.28:0/3993440667 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f319006e0b0 0x7f3190070560 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:39.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.072+0000 7f31a6dd4640 1 --2- 172.21.15.28:0/3993440667 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31a00fb0c0 0x7f31a010d700 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:39.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.072+0000 7f31a6dd4640 1 --2- 172.21.15.28:0/3993440667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31a00fa770 0x7f31a010d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:39.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.072+0000 7f31a6dd4640 1 -- 172.21.15.28:0/3993440667 >> 172.21.15.28:0/3993440667 conn(0x7f31a00f61b0 msgr2=0x7f31a00f78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:48:39.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.072+0000 7f31a6dd4640 1 -- 172.21.15.28:0/3993440667 shutdown_connections 2024-07-03T15:48:39.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.072+0000 7f31a6dd4640 1 -- 172.21.15.28:0/3993440667 wait complete. 2024-07-03T15:48:39.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.159+0000 7f2088e5e640 1 -- 172.21.15.28:0/2139258733 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2084071360 msgr2=0x7f208406f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:39.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.159+0000 7f2088e5e640 1 --2- 172.21.15.28:0/2139258733 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2084071360 0x7f208406f890 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f207c0048d0 tx=0x7f207c009da0 comp rx=0 tx=0).stop 2024-07-03T15:48:39.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.159+0000 7f2088e5e640 1 -- 172.21.15.28:0/2139258733 shutdown_connections 2024-07-03T15:48:39.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.159+0000 7f2088e5e640 1 --2- 172.21.15.28:0/2139258733 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f208406fdd0 0x7f2084070240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:39.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.159+0000 7f2088e5e640 1 --2- 172.21.15.28:0/2139258733 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2084071360 0x7f208406f890 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:39.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.159+0000 7f2088e5e640 1 -- 172.21.15.28:0/2139258733 >> 172.21.15.28:0/2139258733 conn(0x7f20840f7af0 msgr2=0x7f20840f9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:48:39.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.159+0000 7f2088e5e640 1 -- 172.21.15.28:0/2139258733 shutdown_connections 2024-07-03T15:48:39.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.159+0000 7f2088e5e640 1 -- 172.21.15.28:0/2139258733 wait complete. 2024-07-03T15:48:39.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.159+0000 7f2088e5e640 1 Processor -- start 2024-07-03T15:48:39.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.159+0000 7f2088e5e640 1 -- start start 2024-07-03T15:48:39.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.160+0000 7f2088e5e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f208406fdd0 0x7f208410f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:48:39.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.160+0000 7f2088e5e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2084071360 0x7f208410f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:48:39.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.160+0000 7f2088e5e640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f208410fd00 con 0x7f2084071360 2024-07-03T15:48:39.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.160+0000 7f2088e5e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2084110520 con 0x7f208406fdd0 2024-07-03T15:48:39.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.160+0000 7f2081d9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2084071360 0x7f208410f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:48:39.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.160+0000 7f2081d9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2084071360 0x7f208410f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46540/0 (socket says 172.21.15.28:46540) 2024-07-03T15:48:39.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.160+0000 7f2081d9b640 1 -- 172.21.15.28:0/2019037489 learned_addr learned my addr 172.21.15.28:0/2019037489 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:48:39.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.160+0000 7f208259c640 1 --2- 172.21.15.28:0/2019037489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f208406fdd0 0x7f208410f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:48:39.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.160+0000 7f2081d9b640 1 -- 172.21.15.28:0/2019037489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f208406fdd0 msgr2=0x7f208410f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:39.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.160+0000 7f2081d9b640 1 --2- 172.21.15.28:0/2019037489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f208406fdd0 0x7f208410f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:39.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.160+0000 7f2081d9b640 1 -- 172.21.15.28:0/2019037489 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f207c005040 con 0x7f2084071360 2024-07-03T15:48:39.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.160+0000 7f2081d9b640 1 --2- 172.21.15.28:0/2019037489 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2084071360 0x7f208410f6b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f206c002a50 tx=0x7f206c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:48:39.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.160+0000 7f207affd640 1 -- 172.21.15.28:0/2019037489 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f206c007c30 con 0x7f2084071360 2024-07-03T15:48:39.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.160+0000 7f2088e5e640 1 -- 172.21.15.28:0/2019037489 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20841107b0 con 0x7f2084071360 2024-07-03T15:48:39.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.160+0000 7f207affd640 1 -- 172.21.15.28:0/2019037489 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f206c00f040 con 0x7f2084071360 2024-07-03T15:48:39.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.160+0000 7f207affd640 1 -- 172.21.15.28:0/2019037489 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f206c00db50 con 0x7f2084071360 2024-07-03T15:48:39.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.161+0000 7f2088e5e640 1 -- 172.21.15.28:0/2019037489 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2084110d50 con 0x7f2084071360 2024-07-03T15:48:39.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.161+0000 7f2088e5e640 1 -- 172.21.15.28:0/2019037489 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20840fb790 con 0x7f2084071360 2024-07-03T15:48:39.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.162+0000 7f207affd640 1 -- 172.21.15.28:0/2019037489 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f206c01c400 con 0x7f2084071360 2024-07-03T15:48:39.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.162+0000 7f207affd640 1 --2- 172.21.15.28:0/2019037489 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f207006e0b0 0x7f2070070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:48:39.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.163+0000 7f207affd640 1 -- 172.21.15.28:0/2019037489 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f206c08d4c0 con 0x7f2084071360 2024-07-03T15:48:39.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.163+0000 7f208259c640 1 --2- 172.21.15.28:0/2019037489 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f207006e0b0 0x7f2070070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:48:39.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.163+0000 7f208259c640 1 --2- 172.21.15.28:0/2019037489 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f207006e0b0 0x7f2070070560 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f207c009f80 tx=0x7f207c035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:48:39.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.167+0000 7f207affd640 1 -- 172.21.15.28:0/2019037489 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f206c0567a0 con 0x7f2084071360 2024-07-03T15:48:39.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:38 smithi148 ceph-mon[80490]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:48:39.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:38 smithi148 ceph-mon[80490]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:48:39.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:38 smithi148 ceph-mon[80490]: osdmap e60: 6 total, 5 up, 6 in 2024-07-03T15:48:39.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:38 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/2513532509' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:48:39.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:38 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/2946325616' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-03T15:48:39.372 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:39 smithi028 podman[111752]: 2024-07-03 15:48:39.059036697 +0000 UTC m=+0.707078670 container died fe9650ca3e3aec0f5811bd8579b4e147011b6879f08778a08b098c3d5cc32149 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240625, GIT_BRANCH=HEAD, RELEASE=squid-release-c00506d, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True) 2024-07-03T15:48:39.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.429+0000 7f2088e5e640 1 -- 172.21.15.28:0/2019037489 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2084111010 con 0x7f2084071360 2024-07-03T15:48:39.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.430+0000 7f207affd640 1 -- 172.21.15.28:0/2019037489 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+545 (secure 0 0 0) 0x7f206c059e60 con 0x7f2084071360 2024-07-03T15:48:39.434 INFO:teuthology.orchestra.run.smithi028.stdout:HEALTH_WARN 1 osds down; Degraded data redundancy: 9/219 objects degraded (4.110%), 5 pgs degraded 2024-07-03T15:48:39.435 INFO:teuthology.orchestra.run.smithi028.stdout:[WRN] OSD_DOWN: 1 osds down 2024-07-03T15:48:39.435 INFO:teuthology.orchestra.run.smithi028.stdout: osd.2 (root=default,host=smithi028) is down 2024-07-03T15:48:39.435 INFO:teuthology.orchestra.run.smithi028.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 9/219 objects degraded (4.110%), 5 pgs degraded 2024-07-03T15:48:39.435 INFO:teuthology.orchestra.run.smithi028.stdout: pg 2.3 is active+undersized+degraded, acting [5,1] 2024-07-03T15:48:39.435 INFO:teuthology.orchestra.run.smithi028.stdout: pg 2.7 is active+undersized+degraded, acting [3,4] 2024-07-03T15:48:39.435 INFO:teuthology.orchestra.run.smithi028.stdout: pg 2.c is active+undersized+degraded, acting [3,0] 2024-07-03T15:48:39.435 INFO:teuthology.orchestra.run.smithi028.stdout: pg 2.16 is active+undersized+degraded, acting [5,3] 2024-07-03T15:48:39.435 INFO:teuthology.orchestra.run.smithi028.stdout: pg 2.1c is active+undersized+degraded, acting [4,5] 2024-07-03T15:48:39.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.432+0000 7f2088e5e640 1 -- 172.21.15.28:0/2019037489 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f207006e0b0 msgr2=0x7f2070070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:39.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.432+0000 7f2088e5e640 1 --2- 172.21.15.28:0/2019037489 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f207006e0b0 0x7f2070070560 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f207c009f80 tx=0x7f207c035040 comp rx=0 tx=0).stop 2024-07-03T15:48:39.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.432+0000 7f2088e5e640 1 -- 172.21.15.28:0/2019037489 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2084071360 msgr2=0x7f208410f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:48:39.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.432+0000 7f2088e5e640 1 --2- 172.21.15.28:0/2019037489 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2084071360 0x7f208410f6b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f206c002a50 tx=0x7f206c004600 comp rx=0 tx=0).stop 2024-07-03T15:48:39.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.432+0000 7f2088e5e640 1 -- 172.21.15.28:0/2019037489 shutdown_connections 2024-07-03T15:48:39.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.432+0000 7f2088e5e640 1 --2- 172.21.15.28:0/2019037489 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f207006e0b0 0x7f2070070560 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:39.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.432+0000 7f2088e5e640 1 --2- 172.21.15.28:0/2019037489 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2084071360 0x7f208410f6b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:39.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.432+0000 7f2088e5e640 1 --2- 172.21.15.28:0/2019037489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f208406fdd0 0x7f208410f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:48:39.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.432+0000 7f2088e5e640 1 -- 172.21.15.28:0/2019037489 >> 172.21.15.28:0/2019037489 conn(0x7f20840f7af0 msgr2=0x7f20840f91b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:48:39.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.432+0000 7f2088e5e640 1 -- 172.21.15.28:0/2019037489 shutdown_connections 2024-07-03T15:48:39.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:48:39.432+0000 7f2088e5e640 1 -- 172.21.15.28:0/2019037489 wait complete. 2024-07-03T15:48:40.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:39 smithi028 ceph-mon[92858]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:48:40.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:39 smithi028 ceph-mon[92858]: pgmap v83: 65 pgs: 7 active+undersized, 7 stale+active+clean, 5 active+undersized+degraded, 46 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 9/219 objects degraded (4.110%) 2024-07-03T15:48:40.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:39 smithi028 ceph-mon[92858]: Health check failed: Degraded data redundancy: 9/219 objects degraded (4.110%), 5 pgs degraded (PG_DEGRADED) 2024-07-03T15:48:40.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:39 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/2019037489' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-03T15:48:40.123 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:39 smithi028 podman[111819]: 2024-07-03 15:48:39.674568815 +0000 UTC m=+0.608752805 container remove fe9650ca3e3aec0f5811bd8579b4e147011b6879f08778a08b098c3d5cc32149 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.vendor=CentOS, RELEASE=squid-release-c00506d, ceph=True, org.label-schema.build-date=20240625) 2024-07-03T15:48:40.123 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:39 smithi028 systemd[1]: ceph-a85e4be6-3951-11ef-bcab-c7b262605968@osd.2.service: Deactivated successfully. 2024-07-03T15:48:40.123 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:39 smithi028 systemd[1]: Stopped Ceph osd.2 for a85e4be6-3951-11ef-bcab-c7b262605968. 2024-07-03T15:48:40.123 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:39 smithi028 systemd[1]: ceph-a85e4be6-3951-11ef-bcab-c7b262605968@osd.2.service: Consumed 54.497s CPU time. 2024-07-03T15:48:40.123 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:39 smithi028 systemd[1]: Starting Ceph osd.2 for a85e4be6-3951-11ef-bcab-c7b262605968... 2024-07-03T15:48:40.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:39 smithi148 ceph-mon[80490]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:48:40.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:39 smithi148 ceph-mon[80490]: pgmap v83: 65 pgs: 7 active+undersized, 7 stale+active+clean, 5 active+undersized+degraded, 46 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 9/219 objects degraded (4.110%) 2024-07-03T15:48:40.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:39 smithi148 ceph-mon[80490]: Health check failed: Degraded data redundancy: 9/219 objects degraded (4.110%), 5 pgs degraded (PG_DEGRADED) 2024-07-03T15:48:40.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:39 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/2019037489' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-03T15:48:40.424 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:40 smithi028 podman[111942]: 2024-07-03 15:48:40.042873319 +0000 UTC m=+0.017826111 image pull 9d49405603e8445bc510362cdeb05f243619be76b0f3897dd4c152b00cffe45f quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2 2024-07-03T15:48:40.424 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:40 smithi028 podman[111942]: 2024-07-03 15:48:40.184756073 +0000 UTC m=+0.159708871 container create 8ebcadfe74646432e93ef37cb805d86caf778187813e85bceef8b79e86258ae4 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-2-activate, ceph=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-release-c00506d, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240625, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-07-03T15:48:40.424 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:40 smithi028 podman[111942]: 2024-07-03 15:48:40.420991597 +0000 UTC m=+0.395944402 container init 8ebcadfe74646432e93ef37cb805d86caf778187813e85bceef8b79e86258ae4 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-2-activate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240625, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-release-c00506d, io.buildah.version=1.36.0, ceph=True) 2024-07-03T15:48:40.801 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:40 smithi028 podman[111942]: 2024-07-03 15:48:40.424602799 +0000 UTC m=+0.399555604 container start 8ebcadfe74646432e93ef37cb805d86caf778187813e85bceef8b79e86258ae4 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-2-activate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240625, RELEASE=squid-release-c00506d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=) 2024-07-03T15:48:40.801 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:40 smithi028 podman[111942]: 2024-07-03 15:48:40.504284784 +0000 UTC m=+0.479237582 container attach 8ebcadfe74646432e93ef37cb805d86caf778187813e85bceef8b79e86258ae4 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-2-activate, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240625, org.label-schema.license=GPLv2, RELEASE=squid-release-c00506d, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-07-03T15:48:41.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:40 smithi028 ceph-mon[92858]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:48:41.123 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:40 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-2-activate[111957]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-07-03T15:48:41.123 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:40 smithi028 bash[111942]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-07-03T15:48:41.123 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:40 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-2-activate[111957]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-07-03T15:48:41.123 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:40 smithi028 bash[111942]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-07-03T15:48:41.123 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:40 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-2-activate[111957]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-07-03T15:48:41.123 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:40 smithi028 bash[111942]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-07-03T15:48:41.123 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:40 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-2-activate[111957]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-07-03T15:48:41.123 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:40 smithi028 bash[111942]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-07-03T15:48:41.124 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:40 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-2-activate[111957]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-07-03T15:48:41.124 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:40 smithi028 bash[111942]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-07-03T15:48:41.124 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:40 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-2-activate[111957]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-07-03T15:48:41.124 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:40 smithi028 bash[111942]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-07-03T15:48:41.124 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:40 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-2-activate[111957]: --> ceph-volume raw activate successful for osd ID: 2 2024-07-03T15:48:41.124 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:40 smithi028 bash[111942]: --> ceph-volume raw activate successful for osd ID: 2 2024-07-03T15:48:41.124 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:40 smithi028 podman[111942]: 2024-07-03 15:48:40.860987948 +0000 UTC m=+0.835940740 container died 8ebcadfe74646432e93ef37cb805d86caf778187813e85bceef8b79e86258ae4 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-2-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240625, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, ceph=True, RELEASE=squid-release-c00506d, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-07-03T15:48:41.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:40 smithi148 ceph-mon[80490]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:48:41.802 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:41 smithi028 podman[112077]: 2024-07-03 15:48:41.483218574 +0000 UTC m=+0.614525708 container remove 8ebcadfe74646432e93ef37cb805d86caf778187813e85bceef8b79e86258ae4 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-2-activate, GIT_CLEAN=True, ceph=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240625, org.label-schema.vendor=CentOS, RELEASE=squid-release-c00506d, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-07-03T15:48:41.803 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:41 smithi028 podman[112137]: 2024-07-03 15:48:41.700871366 +0000 UTC m=+0.114566779 container create 5d160a327dc2c08da25282c52d408154d36dc13ce6ca73d0e36fe860691ede4d (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-release-c00506d, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, org.label-schema.build-date=20240625, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0) 2024-07-03T15:48:41.803 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:41 smithi028 podman[112137]: 2024-07-03 15:48:41.604214682 +0000 UTC m=+0.017910104 image pull 9d49405603e8445bc510362cdeb05f243619be76b0f3897dd4c152b00cffe45f quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2 2024-07-03T15:48:42.107 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:41 smithi028 ceph-mon[92858]: pgmap v84: 65 pgs: 16 active+undersized, 11 active+undersized+degraded, 38 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 32/219 objects degraded (14.612%) 2024-07-03T15:48:42.107 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:42 smithi028 podman[112137]: 2024-07-03 15:48:42.003704644 +0000 UTC m=+0.417400073 container init 5d160a327dc2c08da25282c52d408154d36dc13ce6ca73d0e36fe860691ede4d (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-2, RELEASE=squid-release-c00506d, org.label-schema.build-date=20240625, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-07-03T15:48:42.107 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:42 smithi028 podman[112137]: 2024-07-03 15:48:42.007019836 +0000 UTC m=+0.420715252 container start 5d160a327dc2c08da25282c52d408154d36dc13ce6ca73d0e36fe860691ede4d (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-2, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, RELEASE=squid-release-c00506d, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.build-date=20240625, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-03T15:48:42.107 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:42 smithi028 ceph-osd[112157]: -- 172.21.15.28:0/3698777236 <== mon.0 v2:172.21.15.28:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55772f9681e0 con 0x55772f966000 2024-07-03T15:48:42.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:41 smithi148 ceph-mon[80490]: pgmap v84: 65 pgs: 16 active+undersized, 11 active+undersized+degraded, 38 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 32/219 objects degraded (14.612%) 2024-07-03T15:48:42.372 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:42 smithi028 bash[112137]: 5d160a327dc2c08da25282c52d408154d36dc13ce6ca73d0e36fe860691ede4d 2024-07-03T15:48:42.372 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:42 smithi028 systemd[1]: Started Ceph osd.2 for a85e4be6-3951-11ef-bcab-c7b262605968. 2024-07-03T15:48:42.872 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:42 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-2[112153]: 2024-07-03T15:48:42.558+0000 7ff899728640 -1 Falling back to public interface 2024-07-03T15:48:43.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:43 smithi028 ceph-mon[92858]: pgmap v85: 65 pgs: 16 active+undersized, 11 active+undersized+degraded, 38 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 32/219 objects degraded (14.612%) 2024-07-03T15:48:43.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:43 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:43.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:43 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:43.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:43 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:48:43.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:43 smithi148 ceph-mon[80490]: pgmap v85: 65 pgs: 16 active+undersized, 11 active+undersized+degraded, 38 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 32/219 objects degraded (14.612%) 2024-07-03T15:48:43.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:43 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:43.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:43 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:43.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:43 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:48:45.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:45 smithi028 ceph-mon[92858]: pgmap v86: 65 pgs: 16 active+undersized, 11 active+undersized+degraded, 38 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 32/219 objects degraded (14.612%) 2024-07-03T15:48:45.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:45 smithi148 ceph-mon[80490]: pgmap v86: 65 pgs: 16 active+undersized, 11 active+undersized+degraded, 38 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 32/219 objects degraded (14.612%) 2024-07-03T15:48:46.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:46 smithi028 ceph-mon[92858]: Health check update: Degraded data redundancy: 32/219 objects degraded (14.612%), 11 pgs degraded (PG_DEGRADED) 2024-07-03T15:48:46.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:46 smithi148 ceph-mon[80490]: Health check update: Degraded data redundancy: 32/219 objects degraded (14.612%), 11 pgs degraded (PG_DEGRADED) 2024-07-03T15:48:47.418 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:47 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-2[112153]: 2024-07-03T15:48:47.267+0000 7ff899728640 -1 osd.2 58 log_to_monitors true 2024-07-03T15:48:47.418 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:47 smithi028 ceph-mon[92858]: pgmap v87: 65 pgs: 16 active+undersized, 11 active+undersized+degraded, 38 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 32/219 objects degraded (14.612%) 2024-07-03T15:48:47.418 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:47 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:47.418 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:47 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:47.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:47 smithi148 ceph-mon[80490]: pgmap v87: 65 pgs: 16 active+undersized, 11 active+undersized+degraded, 38 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 32/219 objects degraded (14.612%) 2024-07-03T15:48:47.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:47 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:47.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:47 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:48.431 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:48:48 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-2[112153]: 2024-07-03T15:48:48.181+0000 7ff8914c2640 -1 osd.2 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-03T15:48:48.431 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:48 smithi028 ceph-mon[92858]: from='osd.2 [v2:172.21.15.28:6818/2390177581,v1:172.21.15.28:6819/2390177581]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-07-03T15:48:48.432 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:48 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:48.432 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:48 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:48.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:48 smithi148 ceph-mon[80490]: from='osd.2 [v2:172.21.15.28:6818/2390177581,v1:172.21.15.28:6819/2390177581]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-07-03T15:48:48.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:48 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:48.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:48 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:49.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:49 smithi148 ceph-mon[80490]: pgmap v88: 65 pgs: 16 active+undersized, 11 active+undersized+degraded, 38 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 32/219 objects degraded (14.612%) 2024-07-03T15:48:49.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:49 smithi148 ceph-mon[80490]: from='osd.2 [v2:172.21.15.28:6818/2390177581,v1:172.21.15.28:6819/2390177581]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-07-03T15:48:49.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:49 smithi148 ceph-mon[80490]: osdmap e61: 6 total, 5 up, 6 in 2024-07-03T15:48:49.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:49 smithi148 ceph-mon[80490]: from='osd.2 [v2:172.21.15.28:6818/2390177581,v1:172.21.15.28:6819/2390177581]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi028", "root=default"]}]: dispatch 2024-07-03T15:48:49.569 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:49 smithi028 ceph-mon[92858]: pgmap v88: 65 pgs: 16 active+undersized, 11 active+undersized+degraded, 38 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 32/219 objects degraded (14.612%) 2024-07-03T15:48:49.570 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:49 smithi028 ceph-mon[92858]: from='osd.2 [v2:172.21.15.28:6818/2390177581,v1:172.21.15.28:6819/2390177581]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-07-03T15:48:49.570 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:49 smithi028 ceph-mon[92858]: osdmap e61: 6 total, 5 up, 6 in 2024-07-03T15:48:49.570 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:49 smithi028 ceph-mon[92858]: from='osd.2 [v2:172.21.15.28:6818/2390177581,v1:172.21.15.28:6819/2390177581]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi028", "root=default"]}]: dispatch 2024-07-03T15:48:50.447 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:50 smithi028 ceph-mon[92858]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-07-03T15:48:50.447 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:50 smithi028 ceph-mon[92858]: osd.2 [v2:172.21.15.28:6818/2390177581,v1:172.21.15.28:6819/2390177581] boot 2024-07-03T15:48:50.448 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:50 smithi028 ceph-mon[92858]: osdmap e62: 6 total, 6 up, 6 in 2024-07-03T15:48:50.448 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:50 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-03T15:48:50.448 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:50 smithi028 ceph-mon[92858]: osdmap e63: 6 total, 6 up, 6 in 2024-07-03T15:48:50.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:50 smithi148 ceph-mon[80490]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-07-03T15:48:50.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:50 smithi148 ceph-mon[80490]: osd.2 [v2:172.21.15.28:6818/2390177581,v1:172.21.15.28:6819/2390177581] boot 2024-07-03T15:48:50.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:50 smithi148 ceph-mon[80490]: osdmap e62: 6 total, 6 up, 6 in 2024-07-03T15:48:50.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:50 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-03T15:48:50.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:50 smithi148 ceph-mon[80490]: osdmap e63: 6 total, 6 up, 6 in 2024-07-03T15:48:51.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:51 smithi028 ceph-mon[92858]: pgmap v91: 65 pgs: 5 peering, 12 active+undersized, 10 active+undersized+degraded, 38 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 29/219 objects degraded (13.242%) 2024-07-03T15:48:51.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:51 smithi028 ceph-mon[92858]: Health check update: Degraded data redundancy: 29/219 objects degraded (13.242%), 10 pgs degraded (PG_DEGRADED) 2024-07-03T15:48:51.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:51 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:51.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:51 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:51.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:51 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:48:51.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:51 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:48:51.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:51 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:51.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:51 smithi148 ceph-mon[80490]: pgmap v91: 65 pgs: 5 peering, 12 active+undersized, 10 active+undersized+degraded, 38 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 29/219 objects degraded (13.242%) 2024-07-03T15:48:51.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:51 smithi148 ceph-mon[80490]: Health check update: Degraded data redundancy: 29/219 objects degraded (13.242%), 10 pgs degraded (PG_DEGRADED) 2024-07-03T15:48:51.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:51 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:51.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:51 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:51.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:51 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:48:51.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:51 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:48:51.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:51 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:52.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:52 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:48:52.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:52 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:48:52.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:52 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:48:52.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:52 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:48:52.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:52 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-07-03T15:48:52.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:52 smithi148 ceph-mon[80490]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-07-03T15:48:52.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:52 smithi148 ceph-mon[80490]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-07-03T15:48:52.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:52 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:52.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:52 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:48:52.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:52 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:48:52.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:52 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:48:52.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:52 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:48:52.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:52 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:48:52.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:52 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-07-03T15:48:52.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:52 smithi028 ceph-mon[92858]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-07-03T15:48:52.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:52 smithi028 ceph-mon[92858]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-07-03T15:48:52.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:52 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:48:52.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:52 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:48:53.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:53 smithi148 ceph-mon[80490]: pgmap v93: 65 pgs: 5 peering, 12 active+undersized, 10 active+undersized+degraded, 38 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 29/219 objects degraded (13.242%) 2024-07-03T15:48:53.509 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:53 smithi028 ceph-mon[92858]: pgmap v93: 65 pgs: 5 peering, 12 active+undersized, 10 active+undersized+degraded, 38 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 29/219 objects degraded (13.242%) 2024-07-03T15:48:55.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:55 smithi148 ceph-mon[80490]: pgmap v94: 65 pgs: 5 peering, 2 active+undersized, 1 active+undersized+degraded, 57 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 682 B/s rd, 1 op/s; 3/219 objects degraded (1.370%) 2024-07-03T15:48:55.595 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:55 smithi028 ceph-mon[92858]: pgmap v94: 65 pgs: 5 peering, 2 active+undersized, 1 active+undersized+degraded, 57 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 682 B/s rd, 1 op/s; 3/219 objects degraded (1.370%) 2024-07-03T15:48:56.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:56 smithi148 ceph-mon[80490]: Health check update: Degraded data redundancy: 3/219 objects degraded (1.370%), 1 pg degraded (PG_DEGRADED) 2024-07-03T15:48:56.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:56 smithi028 ceph-mon[92858]: Health check update: Degraded data redundancy: 3/219 objects degraded (1.370%), 1 pg degraded (PG_DEGRADED) 2024-07-03T15:48:57.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:57 smithi148 ceph-mon[80490]: pgmap v95: 65 pgs: 65 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 517 B/s rd, 0 op/s 2024-07-03T15:48:57.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:57 smithi148 ceph-mon[80490]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3/219 objects degraded (1.370%), 1 pg degraded) 2024-07-03T15:48:57.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:57 smithi148 ceph-mon[80490]: Cluster is now healthy 2024-07-03T15:48:57.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:57 smithi028 ceph-mon[92858]: pgmap v95: 65 pgs: 65 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 517 B/s rd, 0 op/s 2024-07-03T15:48:57.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:57 smithi028 ceph-mon[92858]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3/219 objects degraded (1.370%), 1 pg degraded) 2024-07-03T15:48:57.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:57 smithi028 ceph-mon[92858]: Cluster is now healthy 2024-07-03T15:48:59.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:48:59 smithi148 ceph-mon[80490]: pgmap v96: 65 pgs: 65 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-07-03T15:48:59.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:48:59 smithi028 ceph-mon[92858]: pgmap v96: 65 pgs: 65 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-07-03T15:49:01.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:01 smithi148 ceph-mon[80490]: pgmap v97: 65 pgs: 65 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 921 B/s rd, 1 op/s 2024-07-03T15:49:01.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:01 smithi028 ceph-mon[92858]: pgmap v97: 65 pgs: 65 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 921 B/s rd, 1 op/s 2024-07-03T15:49:03.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:03 smithi148 ceph-mon[80490]: pgmap v98: 65 pgs: 65 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 773 B/s rd, 1 op/s 2024-07-03T15:49:03.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:03 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:03.508 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:03 smithi028 ceph-mon[92858]: pgmap v98: 65 pgs: 65 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 773 B/s rd, 1 op/s 2024-07-03T15:49:03.509 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:03 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:05.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:05 smithi148 ceph-mon[80490]: pgmap v99: 65 pgs: 65 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:49:05.596 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:05 smithi028 ceph-mon[92858]: pgmap v99: 65 pgs: 65 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:49:07.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:07 smithi148 ceph-mon[80490]: pgmap v100: 65 pgs: 65 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:49:07.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:07 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-07-03T15:49:07.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:07 smithi148 ceph-mon[80490]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-07-03T15:49:07.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:07 smithi148 ceph-mon[80490]: Upgrade: osd.3 is safe to restart 2024-07-03T15:49:07.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:07 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:07.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:07 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:49:07.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:07 smithi028 ceph-mon[92858]: pgmap v100: 65 pgs: 65 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:49:07.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:07 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-07-03T15:49:07.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:07 smithi028 ceph-mon[92858]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-07-03T15:49:07.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:07 smithi028 ceph-mon[92858]: Upgrade: osd.3 is safe to restart 2024-07-03T15:49:07.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:07 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:07.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:07 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:49:08.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:08 smithi028 ceph-mon[92858]: Upgrade: Updating osd.3 2024-07-03T15:49:08.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:08 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:08.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:08 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-07-03T15:49:08.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:08 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:49:08.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:08 smithi028 ceph-mon[92858]: Deploying daemon osd.3 on smithi148 2024-07-03T15:49:08.946 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:08 smithi148 ceph-mon[80490]: Upgrade: Updating osd.3 2024-07-03T15:49:08.946 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:08 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:08.947 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:08 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-07-03T15:49:08.947 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:08 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:49:08.947 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:08 smithi148 ceph-mon[80490]: Deploying daemon osd.3 on smithi148 2024-07-03T15:49:09.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.527+0000 7fc47c99f640 1 -- 172.21.15.28:0/3403406019 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4780fe300 msgr2=0x7fc4780fa460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:09.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.527+0000 7fc47c99f640 1 --2- 172.21.15.28:0/3403406019 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4780fe300 0x7fc4780fa460 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc46c0048d0 tx=0x7fc46c009da0 comp rx=0 tx=0).stop 2024-07-03T15:49:09.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.528+0000 7fc47c99f640 1 -- 172.21.15.28:0/3403406019 shutdown_connections 2024-07-03T15:49:09.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.528+0000 7fc47c99f640 1 --2- 172.21.15.28:0/3403406019 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4780fe300 0x7fc4780fa460 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:09.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.528+0000 7fc47c99f640 1 --2- 172.21.15.28:0/3403406019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4780fd9b0 0x7fc4780fddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:09.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.528+0000 7fc47c99f640 1 -- 172.21.15.28:0/3403406019 >> 172.21.15.28:0/3403406019 conn(0x7fc4780f61b0 msgr2=0x7fc4780f85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:49:09.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.528+0000 7fc47c99f640 1 -- 172.21.15.28:0/3403406019 shutdown_connections 2024-07-03T15:49:09.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.528+0000 7fc47c99f640 1 -- 172.21.15.28:0/3403406019 wait complete. 2024-07-03T15:49:09.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.528+0000 7fc47c99f640 1 Processor -- start 2024-07-03T15:49:09.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.528+0000 7fc47c99f640 1 -- start start 2024-07-03T15:49:09.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.528+0000 7fc47c99f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4780fd9b0 0x7fc4780fd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:49:09.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.528+0000 7fc47c99f640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4780fe300 0x7fc4780fb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:49:09.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.528+0000 7fc47c99f640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4780fbc20 con 0x7fc4780fe300 2024-07-03T15:49:09.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.528+0000 7fc47c99f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4780fbd60 con 0x7fc4780fd9b0 2024-07-03T15:49:09.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.529+0000 7fc475d9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4780fe300 0x7fc4780fb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:49:09.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.529+0000 7fc47659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4780fd9b0 0x7fc4780fd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:49:09.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.529+0000 7fc475d9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4780fe300 0x7fc4780fb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58042/0 (socket says 172.21.15.28:58042) 2024-07-03T15:49:09.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.529+0000 7fc47659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4780fd9b0 0x7fc4780fd1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.28:33802/0 (socket says 172.21.15.28:33802) 2024-07-03T15:49:09.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.529+0000 7fc475d9b640 1 -- 172.21.15.28:0/2604748524 learned_addr learned my addr 172.21.15.28:0/2604748524 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:49:09.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.529+0000 7fc475d9b640 1 -- 172.21.15.28:0/2604748524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4780fd9b0 msgr2=0x7fc4780fd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:09.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.529+0000 7fc475d9b640 1 --2- 172.21.15.28:0/2604748524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4780fd9b0 0x7fc4780fd1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:09.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.529+0000 7fc475d9b640 1 -- 172.21.15.28:0/2604748524 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc46c005040 con 0x7fc4780fe300 2024-07-03T15:49:09.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.529+0000 7fc475d9b640 1 --2- 172.21.15.28:0/2604748524 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4780fe300 0x7fc4780fb690 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc46c009f80 tx=0x7fc46c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:49:09.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.529+0000 7fc466ffd640 1 -- 172.21.15.28:0/2604748524 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc46c038070 con 0x7fc4780fe300 2024-07-03T15:49:09.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.529+0000 7fc466ffd640 1 -- 172.21.15.28:0/2604748524 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc46c008ee0 con 0x7fc4780fe300 2024-07-03T15:49:09.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.529+0000 7fc47c99f640 1 -- 172.21.15.28:0/2604748524 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4780fbfe0 con 0x7fc4780fe300 2024-07-03T15:49:09.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.529+0000 7fc466ffd640 1 -- 172.21.15.28:0/2604748524 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc46c033780 con 0x7fc4780fe300 2024-07-03T15:49:09.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.530+0000 7fc47c99f640 1 -- 172.21.15.28:0/2604748524 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc478114fb0 con 0x7fc4780fe300 2024-07-03T15:49:09.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.531+0000 7fc47c99f640 1 -- 172.21.15.28:0/2604748524 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4780f9e50 con 0x7fc4780fe300 2024-07-03T15:49:09.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.531+0000 7fc466ffd640 1 -- 172.21.15.28:0/2604748524 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc46c030020 con 0x7fc4780fe300 2024-07-03T15:49:09.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.532+0000 7fc466ffd640 1 --2- 172.21.15.28:0/2604748524 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc46006e0b0 0x7fc460070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:49:09.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.532+0000 7fc466ffd640 1 -- 172.21.15.28:0/2604748524 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fc46c0ad7e0 con 0x7fc4780fe300 2024-07-03T15:49:09.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.532+0000 7fc47659c640 1 --2- 172.21.15.28:0/2604748524 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc46006e0b0 0x7fc460070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:49:09.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.532+0000 7fc47659c640 1 --2- 172.21.15.28:0/2604748524 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc46006e0b0 0x7fc460070560 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc468004fd0 tx=0x7fc46800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:49:09.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.537+0000 7fc466ffd640 1 -- 172.21.15.28:0/2604748524 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc46c076ca0 con 0x7fc4780fe300 2024-07-03T15:49:09.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.743+0000 7fc47c99f640 1 -- 172.21.15.28:0/2604748524 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc478101ac0 con 0x7fc46006e0b0 2024-07-03T15:49:09.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.744+0000 7fc466ffd640 1 -- 172.21.15.28:0/2604748524 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fc478101ac0 con 0x7fc46006e0b0 2024-07-03T15:49:09.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.746+0000 7fc47c99f640 1 -- 172.21.15.28:0/2604748524 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc46006e0b0 msgr2=0x7fc460070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:09.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.746+0000 7fc47c99f640 1 --2- 172.21.15.28:0/2604748524 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc46006e0b0 0x7fc460070560 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc468004fd0 tx=0x7fc46800b040 comp rx=0 tx=0).stop 2024-07-03T15:49:09.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.746+0000 7fc47c99f640 1 -- 172.21.15.28:0/2604748524 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4780fe300 msgr2=0x7fc4780fb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:09.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.746+0000 7fc47c99f640 1 --2- 172.21.15.28:0/2604748524 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4780fe300 0x7fc4780fb690 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc46c009f80 tx=0x7fc46c009da0 comp rx=0 tx=0).stop 2024-07-03T15:49:09.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.746+0000 7fc47c99f640 1 -- 172.21.15.28:0/2604748524 shutdown_connections 2024-07-03T15:49:09.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.746+0000 7fc47c99f640 1 --2- 172.21.15.28:0/2604748524 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc46006e0b0 0x7fc460070560 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:09.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.746+0000 7fc47c99f640 1 --2- 172.21.15.28:0/2604748524 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4780fe300 0x7fc4780fb690 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:09.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.746+0000 7fc47c99f640 1 --2- 172.21.15.28:0/2604748524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4780fd9b0 0x7fc4780fd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:09.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.746+0000 7fc47c99f640 1 -- 172.21.15.28:0/2604748524 >> 172.21.15.28:0/2604748524 conn(0x7fc4780f61b0 msgr2=0x7fc4780f78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:49:09.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.746+0000 7fc47c99f640 1 -- 172.21.15.28:0/2604748524 shutdown_connections 2024-07-03T15:49:09.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.746+0000 7fc47c99f640 1 -- 172.21.15.28:0/2604748524 wait complete. 2024-07-03T15:49:09.759 INFO:teuthology.orchestra.run.smithi028.stdout:true 2024-07-03T15:49:09.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.830+0000 7fc9c5c6f640 1 -- 172.21.15.28:0/1220978582 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9c00fc330 msgr2=0x7fc9c00fc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:09.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.830+0000 7fc9c5c6f640 1 --2- 172.21.15.28:0/1220978582 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9c00fc330 0x7fc9c00fc740 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0009c50 tx=0x7fc9b0004d10 comp rx=0 tx=0).stop 2024-07-03T15:49:09.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.831+0000 7fc9c5c6f640 1 -- 172.21.15.28:0/1220978582 shutdown_connections 2024-07-03T15:49:09.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.831+0000 7fc9c5c6f640 1 --2- 172.21.15.28:0/1220978582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9c00fd600 0x7fc9c00fda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:09.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.831+0000 7fc9c5c6f640 1 --2- 172.21.15.28:0/1220978582 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9c00fc330 0x7fc9c00fc740 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:09.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.831+0000 7fc9c5c6f640 1 -- 172.21.15.28:0/1220978582 >> 172.21.15.28:0/1220978582 conn(0x7fc9c00f7af0 msgr2=0x7fc9c00f9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:49:09.835 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:09 smithi028 ceph-mon[92858]: pgmap v101: 65 pgs: 65 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:49:09.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.831+0000 7fc9c5c6f640 1 -- 172.21.15.28:0/1220978582 shutdown_connections 2024-07-03T15:49:09.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.831+0000 7fc9c5c6f640 1 -- 172.21.15.28:0/1220978582 wait complete. 2024-07-03T15:49:09.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.831+0000 7fc9c5c6f640 1 Processor -- start 2024-07-03T15:49:09.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.831+0000 7fc9c5c6f640 1 -- start start 2024-07-03T15:49:09.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.832+0000 7fc9c5c6f640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9c00fc330 0x7fc9c0071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:49:09.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.832+0000 7fc9c5c6f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9c00fd600 0x7fc9c006f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:49:09.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.832+0000 7fc9c5c6f640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c006fe00 con 0x7fc9c00fc330 2024-07-03T15:49:09.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.832+0000 7fc9c5c6f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c006ff40 con 0x7fc9c00fd600 2024-07-03T15:49:09.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.832+0000 7fc9bf7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9c00fc330 0x7fc9c0071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:49:09.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.832+0000 7fc9bf7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9c00fc330 0x7fc9c0071380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37172/0 (socket says 172.21.15.28:37172) 2024-07-03T15:49:09.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.832+0000 7fc9bf7fe640 1 -- 172.21.15.28:0/717547642 learned_addr learned my addr 172.21.15.28:0/717547642 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:49:09.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.832+0000 7fc9beffd640 1 --2- 172.21.15.28:0/717547642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9c00fd600 0x7fc9c006f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:49:09.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.832+0000 7fc9bf7fe640 1 -- 172.21.15.28:0/717547642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9c00fd600 msgr2=0x7fc9c006f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:09.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.832+0000 7fc9bf7fe640 1 --2- 172.21.15.28:0/717547642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9c00fd600 0x7fc9c006f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:09.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.832+0000 7fc9bf7fe640 1 -- 172.21.15.28:0/717547642 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b00098b0 con 0x7fc9c00fc330 2024-07-03T15:49:09.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.832+0000 7fc9bf7fe640 1 --2- 172.21.15.28:0/717547642 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9c00fc330 0x7fc9c0071380 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc9b002a8b0 tx=0x7fc9b0004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:49:09.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.832+0000 7fc9a7fff640 1 -- 172.21.15.28:0/717547642 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9b0008de0 con 0x7fc9c00fc330 2024-07-03T15:49:09.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.832+0000 7fc9a7fff640 1 -- 172.21.15.28:0/717547642 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc9b0032c00 con 0x7fc9c00fc330 2024-07-03T15:49:09.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.833+0000 7fc9c5c6f640 1 -- 172.21.15.28:0/717547642 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9c00701c0 con 0x7fc9c00fc330 2024-07-03T15:49:09.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.833+0000 7fc9a7fff640 1 -- 172.21.15.28:0/717547642 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9b003dbb0 con 0x7fc9c00fc330 2024-07-03T15:49:09.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.833+0000 7fc9c5c6f640 1 -- 172.21.15.28:0/717547642 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9c00706e0 con 0x7fc9c00fc330 2024-07-03T15:49:09.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.834+0000 7fc9c5c6f640 1 -- 172.21.15.28:0/717547642 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9c00fb790 con 0x7fc9c00fc330 2024-07-03T15:49:09.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.834+0000 7fc9a7fff640 1 -- 172.21.15.28:0/717547642 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc9b002f030 con 0x7fc9c00fc330 2024-07-03T15:49:09.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.834+0000 7fc9a7fff640 1 --2- 172.21.15.28:0/717547642 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc9a806e0b0 0x7fc9a8070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:49:09.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.835+0000 7fc9a7fff640 1 -- 172.21.15.28:0/717547642 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fc9b00aef40 con 0x7fc9c00fc330 2024-07-03T15:49:09.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.835+0000 7fc9beffd640 1 --2- 172.21.15.28:0/717547642 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc9a806e0b0 0x7fc9a8070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:49:09.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.835+0000 7fc9beffd640 1 --2- 172.21.15.28:0/717547642 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc9a806e0b0 0x7fc9a8070560 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4009af0 tx=0x7fc9b400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:49:09.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:09.839+0000 7fc9a7fff640 1 -- 172.21.15.28:0/717547642 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc9b00e3850 con 0x7fc9c00fc330 2024-07-03T15:49:09.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:09 smithi148 ceph-mon[80490]: pgmap v101: 65 pgs: 65 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:49:10.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.041+0000 7fc9c5c6f640 1 -- 172.21.15.28:0/717547642 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc9c00706e0 con 0x7fc9a806e0b0 2024-07-03T15:49:10.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.042+0000 7fc9a7fff640 1 -- 172.21.15.28:0/717547642 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fc9c00706e0 con 0x7fc9a806e0b0 2024-07-03T15:49:10.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.044+0000 7fc9c5c6f640 1 -- 172.21.15.28:0/717547642 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc9a806e0b0 msgr2=0x7fc9a8070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:10.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.044+0000 7fc9c5c6f640 1 --2- 172.21.15.28:0/717547642 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc9a806e0b0 0x7fc9a8070560 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4009af0 tx=0x7fc9b400b040 comp rx=0 tx=0).stop 2024-07-03T15:49:10.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.044+0000 7fc9c5c6f640 1 -- 172.21.15.28:0/717547642 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9c00fc330 msgr2=0x7fc9c0071380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:10.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.044+0000 7fc9c5c6f640 1 --2- 172.21.15.28:0/717547642 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9c00fc330 0x7fc9c0071380 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc9b002a8b0 tx=0x7fc9b0004d10 comp rx=0 tx=0).stop 2024-07-03T15:49:10.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.044+0000 7fc9c5c6f640 1 -- 172.21.15.28:0/717547642 shutdown_connections 2024-07-03T15:49:10.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.044+0000 7fc9c5c6f640 1 --2- 172.21.15.28:0/717547642 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc9a806e0b0 0x7fc9a8070560 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:10.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.044+0000 7fc9c5c6f640 1 --2- 172.21.15.28:0/717547642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9c00fd600 0x7fc9c006f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:10.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.044+0000 7fc9c5c6f640 1 --2- 172.21.15.28:0/717547642 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9c00fc330 0x7fc9c0071380 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:10.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.044+0000 7fc9c5c6f640 1 -- 172.21.15.28:0/717547642 >> 172.21.15.28:0/717547642 conn(0x7fc9c00f7af0 msgr2=0x7fc9c00f91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:49:10.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.045+0000 7fc9c5c6f640 1 -- 172.21.15.28:0/717547642 shutdown_connections 2024-07-03T15:49:10.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.045+0000 7fc9c5c6f640 1 -- 172.21.15.28:0/717547642 wait complete. 2024-07-03T15:49:10.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.127+0000 7fda6c09b640 1 -- 172.21.15.28:0/3093790637 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda640fb240 msgr2=0x7fda640fb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:10.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.127+0000 7fda6c09b640 1 --2- 172.21.15.28:0/3093790637 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda640fb240 0x7fda640fb650 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fda54009c50 tx=0x7fda54004d10 comp rx=0 tx=0).stop 2024-07-03T15:49:10.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.128+0000 7fda6c09b640 1 -- 172.21.15.28:0/3093790637 shutdown_connections 2024-07-03T15:49:10.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.128+0000 7fda6c09b640 1 --2- 172.21.15.28:0/3093790637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda640fc510 0x7fda640fc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:10.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.128+0000 7fda6c09b640 1 --2- 172.21.15.28:0/3093790637 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda640fb240 0x7fda640fb650 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:10.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.128+0000 7fda6c09b640 1 -- 172.21.15.28:0/3093790637 >> 172.21.15.28:0/3093790637 conn(0x7fda640f6a20 msgr2=0x7fda640f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:49:10.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.128+0000 7fda6c09b640 1 -- 172.21.15.28:0/3093790637 shutdown_connections 2024-07-03T15:49:10.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.128+0000 7fda6c09b640 1 -- 172.21.15.28:0/3093790637 wait complete. 2024-07-03T15:49:10.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.128+0000 7fda6c09b640 1 Processor -- start 2024-07-03T15:49:10.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.128+0000 7fda6c09b640 1 -- start start 2024-07-03T15:49:10.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.129+0000 7fda6c09b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda640fb240 0x7fda6410fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:49:10.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.129+0000 7fda6c09b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda640fc510 0x7fda641121f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:49:10.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.129+0000 7fda6c09b640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda64065a70 con 0x7fda640fc510 2024-07-03T15:49:10.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.129+0000 7fda6c09b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda64065be0 con 0x7fda640fb240 2024-07-03T15:49:10.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.129+0000 7fda69636640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda640fc510 0x7fda641121f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:49:10.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.129+0000 7fda69636640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda640fc510 0x7fda641121f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37192/0 (socket says 172.21.15.28:37192) 2024-07-03T15:49:10.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.129+0000 7fda69e37640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda640fb240 0x7fda6410fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:49:10.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.129+0000 7fda69636640 1 -- 172.21.15.28:0/3833526386 learned_addr learned my addr 172.21.15.28:0/3833526386 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:49:10.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.129+0000 7fda69636640 1 -- 172.21.15.28:0/3833526386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda640fb240 msgr2=0x7fda6410fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:10.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.129+0000 7fda69636640 1 --2- 172.21.15.28:0/3833526386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda640fb240 0x7fda6410fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:10.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.129+0000 7fda69636640 1 -- 172.21.15.28:0/3833526386 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda540098b0 con 0x7fda640fc510 2024-07-03T15:49:10.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.129+0000 7fda69636640 1 --2- 172.21.15.28:0/3833526386 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda640fc510 0x7fda641121f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fda58002a50 tx=0x7fda58004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:49:10.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.129+0000 7fda627fc640 1 -- 172.21.15.28:0/3833526386 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda58007c30 con 0x7fda640fc510 2024-07-03T15:49:10.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.129+0000 7fda627fc640 1 -- 172.21.15.28:0/3833526386 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fda5800f040 con 0x7fda640fc510 2024-07-03T15:49:10.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.129+0000 7fda6c09b640 1 -- 172.21.15.28:0/3833526386 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda64112790 con 0x7fda640fc510 2024-07-03T15:49:10.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.129+0000 7fda627fc640 1 -- 172.21.15.28:0/3833526386 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda5800db50 con 0x7fda640fc510 2024-07-03T15:49:10.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.129+0000 7fda6c09b640 1 -- 172.21.15.28:0/3833526386 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda64112c80 con 0x7fda640fc510 2024-07-03T15:49:10.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.130+0000 7fda6c09b640 1 -- 172.21.15.28:0/3833526386 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda640fa6a0 con 0x7fda640fc510 2024-07-03T15:49:10.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.131+0000 7fda627fc640 1 -- 172.21.15.28:0/3833526386 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fda5801c400 con 0x7fda640fc510 2024-07-03T15:49:10.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.131+0000 7fda627fc640 1 --2- 172.21.15.28:0/3833526386 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fda4c06e0b0 0x7fda4c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:49:10.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.131+0000 7fda627fc640 1 -- 172.21.15.28:0/3833526386 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fda5808d370 con 0x7fda640fc510 2024-07-03T15:49:10.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.132+0000 7fda69e37640 1 --2- 172.21.15.28:0/3833526386 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fda4c06e0b0 0x7fda4c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:49:10.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.132+0000 7fda69e37640 1 --2- 172.21.15.28:0/3833526386 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fda4c06e0b0 0x7fda4c070560 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fda54000f80 tx=0x7fda54004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:49:10.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.136+0000 7fda627fc640 1 -- 172.21.15.28:0/3833526386 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fda580567a0 con 0x7fda640fc510 2024-07-03T15:49:10.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.335+0000 7fda6c09b640 1 -- 172.21.15.28:0/3833526386 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fda64100830 con 0x7fda4c06e0b0 2024-07-03T15:49:10.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.344+0000 7fda627fc640 1 -- 172.21.15.28:0/3833526386 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fda64100830 con 0x7fda4c06e0b0 2024-07-03T15:49:10.348 INFO:teuthology.orchestra.run.smithi028.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-03T15:49:10.348 INFO:teuthology.orchestra.run.smithi028.stdout:alertmanager.smithi028 smithi028 *:9093,9094 running (4m) 23s ago 12m 24.8M - 0.25.0 c8568f914cd2 1425315a0de9 2024-07-03T15:49:10.348 INFO:teuthology.orchestra.run.smithi028.stdout:crash.smithi028 smithi028 running (2m) 23s ago 12m 6895k - 19.0.0-2550-gc00506dc 9d49405603e8 c62c2f45cabe 2024-07-03T15:49:10.348 INFO:teuthology.orchestra.run.smithi028.stdout:crash.smithi148 smithi148 running (116s) 113s ago 11m 6895k - 19.0.0-2550-gc00506dc 9d49405603e8 b1318dcaacb6 2024-07-03T15:49:10.348 INFO:teuthology.orchestra.run.smithi028.stdout:grafana.smithi028 smithi028 *:3000 running (4m) 23s ago 12m 81.9M - 9.4.7 954c08fa6188 f9f17c92cfea 2024-07-03T15:49:10.348 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi028.rhpvve smithi028 running (7m) 23s ago 7m 152M - 17.2.7-1047-gd4b063ab 4ca399d9e65d 3ed5abd02cfb 2024-07-03T15:49:10.348 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi028.wmbxew smithi028 running (7m) 23s ago 7m 110M - 17.2.7-1047-gd4b063ab 4ca399d9e65d b3bf736998f9 2024-07-03T15:49:10.348 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi148.szomfl smithi148 running (7m) 113s ago 7m 15.7M - 17.2.7-1047-gd4b063ab 4ca399d9e65d d94f5c95fa8d 2024-07-03T15:49:10.348 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi148.xgpuld smithi148 running (7m) 113s ago 7m 13.2M - 17.2.7-1047-gd4b063ab 4ca399d9e65d fc70360a8de0 2024-07-03T15:49:10.349 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.smithi028.vtqrtz smithi028 *:8443,9283,8765 running (4m) 23s ago 14m 496M - 19.0.0-2550-gc00506dc 9d49405603e8 14d0363c5b77 2024-07-03T15:49:10.349 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.smithi148.rjzsar smithi148 *:8443,9283,8765 running (3m) 113s ago 11m 438M - 19.0.0-2550-gc00506dc 9d49405603e8 c0193aa8a5dd 2024-07-03T15:49:10.349 INFO:teuthology.orchestra.run.smithi028.stdout:mon.smithi028 smithi028 running (2m) 23s ago 14m 46.8M 2048M 19.0.0-2550-gc00506dc 9d49405603e8 5093f7bf47d7 2024-07-03T15:49:10.349 INFO:teuthology.orchestra.run.smithi028.stdout:mon.smithi148 smithi148 running (2m) 113s ago 11m 33.6M 2048M 19.0.0-2550-gc00506dc 9d49405603e8 d338d2c648da 2024-07-03T15:49:10.349 INFO:teuthology.orchestra.run.smithi028.stdout:node-exporter.smithi028 smithi028 *:9100 running (4m) 23s ago 12m 15.6M - 1.5.0 0da6a335fe13 91ca1f49de0e 2024-07-03T15:49:10.349 INFO:teuthology.orchestra.run.smithi028.stdout:node-exporter.smithi148 smithi148 *:9100 running (4m) 113s ago 11m 15.6M - 1.5.0 0da6a335fe13 1eaaf7b1b561 2024-07-03T15:49:10.349 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0 smithi028 running (97s) 23s ago 11m 101M 1558M 19.0.0-2550-gc00506dc 9d49405603e8 4454cecff8fc 2024-07-03T15:49:10.349 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1 smithi028 running (63s) 23s ago 10m 99.1M 1558M 19.0.0-2550-gc00506dc 9d49405603e8 af67f7088643 2024-07-03T15:49:10.349 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2 smithi028 running (28s) 23s ago 10m 12.1M 1558M 19.0.0-2550-gc00506dc 9d49405603e8 5d160a327dc2 2024-07-03T15:49:10.349 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3 smithi148 running (9m) 113s ago 9m 278M 2581M 17.2.7-1047-gd4b063ab 4ca399d9e65d 5be69a27df31 2024-07-03T15:49:10.349 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4 smithi148 running (9m) 113s ago 9m 250M 2581M 17.2.7-1047-gd4b063ab 4ca399d9e65d 54f7c26c77ca 2024-07-03T15:49:10.349 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5 smithi148 running (8m) 113s ago 8m 239M 2581M 17.2.7-1047-gd4b063ab 4ca399d9e65d d185c6905384 2024-07-03T15:49:10.349 INFO:teuthology.orchestra.run.smithi028.stdout:prometheus.smithi028 smithi028 *:9095 running (3m) 23s ago 12m 60.5M - 2.43.0 a07b618ecd1d 2b0eb2a95afd 2024-07-03T15:49:10.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.346+0000 7fda6c09b640 1 -- 172.21.15.28:0/3833526386 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fda4c06e0b0 msgr2=0x7fda4c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:10.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.346+0000 7fda6c09b640 1 --2- 172.21.15.28:0/3833526386 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fda4c06e0b0 0x7fda4c070560 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fda54000f80 tx=0x7fda54004d10 comp rx=0 tx=0).stop 2024-07-03T15:49:10.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.346+0000 7fda6c09b640 1 -- 172.21.15.28:0/3833526386 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda640fc510 msgr2=0x7fda641121f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:10.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.346+0000 7fda6c09b640 1 --2- 172.21.15.28:0/3833526386 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda640fc510 0x7fda641121f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fda58002a50 tx=0x7fda58004600 comp rx=0 tx=0).stop 2024-07-03T15:49:10.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.346+0000 7fda6c09b640 1 -- 172.21.15.28:0/3833526386 shutdown_connections 2024-07-03T15:49:10.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.347+0000 7fda6c09b640 1 --2- 172.21.15.28:0/3833526386 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fda4c06e0b0 0x7fda4c070560 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:10.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.347+0000 7fda6c09b640 1 --2- 172.21.15.28:0/3833526386 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda640fc510 0x7fda641121f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:10.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.347+0000 7fda6c09b640 1 --2- 172.21.15.28:0/3833526386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda640fb240 0x7fda6410fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:10.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.347+0000 7fda6c09b640 1 -- 172.21.15.28:0/3833526386 >> 172.21.15.28:0/3833526386 conn(0x7fda640f6a20 msgr2=0x7fda640f8110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:49:10.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.347+0000 7fda6c09b640 1 -- 172.21.15.28:0/3833526386 shutdown_connections 2024-07-03T15:49:10.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.347+0000 7fda6c09b640 1 -- 172.21.15.28:0/3833526386 wait complete. 2024-07-03T15:49:10.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.428+0000 7f28be2b2640 1 -- 172.21.15.28:0/3735833814 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28b81018d0 msgr2=0x7f28b8103cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:10.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.428+0000 7f28be2b2640 1 --2- 172.21.15.28:0/3735833814 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28b81018d0 0x7f28b8103cb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f28ac0048d0 tx=0x7f28ac009da0 comp rx=0 tx=0).stop 2024-07-03T15:49:10.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.428+0000 7f28be2b2640 1 -- 172.21.15.28:0/3735833814 shutdown_connections 2024-07-03T15:49:10.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.428+0000 7f28be2b2640 1 --2- 172.21.15.28:0/3735833814 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28b81018d0 0x7f28b8103cb0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:10.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.428+0000 7f28be2b2640 1 --2- 172.21.15.28:0/3735833814 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28b8066520 0x7f28b8101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:10.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.428+0000 7f28be2b2640 1 -- 172.21.15.28:0/3735833814 >> 172.21.15.28:0/3735833814 conn(0x7f28b80f6a20 msgr2=0x7f28b80f8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:49:10.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.428+0000 7f28be2b2640 1 -- 172.21.15.28:0/3735833814 shutdown_connections 2024-07-03T15:49:10.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.428+0000 7f28be2b2640 1 -- 172.21.15.28:0/3735833814 wait complete. 2024-07-03T15:49:10.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.429+0000 7f28be2b2640 1 Processor -- start 2024-07-03T15:49:10.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.429+0000 7f28be2b2640 1 -- start start 2024-07-03T15:49:10.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.429+0000 7f28be2b2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28b8066520 0x7f28b810f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:49:10.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.429+0000 7f28be2b2640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28b81018d0 0x7f28b810f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:49:10.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.429+0000 7f28be2b2640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b810fd70 con 0x7f28b81018d0 2024-07-03T15:49:10.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.429+0000 7f28be2b2640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b8110540 con 0x7f28b8066520 2024-07-03T15:49:10.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.429+0000 7f28b77fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28b81018d0 0x7f28b810f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:49:10.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.429+0000 7f28b77fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28b81018d0 0x7f28b810f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37200/0 (socket says 172.21.15.28:37200) 2024-07-03T15:49:10.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.429+0000 7f28b77fe640 1 -- 172.21.15.28:0/3590911662 learned_addr learned my addr 172.21.15.28:0/3590911662 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:49:10.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.429+0000 7f28b7fff640 1 --2- 172.21.15.28:0/3590911662 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28b8066520 0x7f28b810f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:49:10.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.430+0000 7f28b77fe640 1 -- 172.21.15.28:0/3590911662 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28b8066520 msgr2=0x7f28b810f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:10.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.430+0000 7f28b77fe640 1 --2- 172.21.15.28:0/3590911662 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28b8066520 0x7f28b810f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:10.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.430+0000 7f28b77fe640 1 -- 172.21.15.28:0/3590911662 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28ac005040 con 0x7f28b81018d0 2024-07-03T15:49:10.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.430+0000 7f28b77fe640 1 --2- 172.21.15.28:0/3590911662 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28b81018d0 0x7f28b810f720 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f28ac003080 tx=0x7f28ac009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:49:10.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.430+0000 7f28b4ff9640 1 -- 172.21.15.28:0/3590911662 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28ac038070 con 0x7f28b81018d0 2024-07-03T15:49:10.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.430+0000 7f28be2b2640 1 -- 172.21.15.28:0/3590911662 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28b8110770 con 0x7f28b81018d0 2024-07-03T15:49:10.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.430+0000 7f28b4ff9640 1 -- 172.21.15.28:0/3590911662 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f28ac032970 con 0x7f28b81018d0 2024-07-03T15:49:10.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.430+0000 7f28b4ff9640 1 -- 172.21.15.28:0/3590911662 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28ac033900 con 0x7f28b81018d0 2024-07-03T15:49:10.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.430+0000 7f28be2b2640 1 -- 172.21.15.28:0/3590911662 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28b8110cb0 con 0x7f28b81018d0 2024-07-03T15:49:10.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.431+0000 7f28be2b2640 1 -- 172.21.15.28:0/3590911662 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28b80fa6e0 con 0x7f28b81018d0 2024-07-03T15:49:10.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.432+0000 7f28b4ff9640 1 -- 172.21.15.28:0/3590911662 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f28ac030020 con 0x7f28b81018d0 2024-07-03T15:49:10.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.433+0000 7f28b4ff9640 1 --2- 172.21.15.28:0/3590911662 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f28a006e0b0 0x7f28a0070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:49:10.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.433+0000 7f28b4ff9640 1 -- 172.21.15.28:0/3590911662 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f28ac0adba0 con 0x7f28b81018d0 2024-07-03T15:49:10.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.433+0000 7f28b7fff640 1 --2- 172.21.15.28:0/3590911662 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f28a006e0b0 0x7f28a0070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:49:10.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.434+0000 7f28b7fff640 1 --2- 172.21.15.28:0/3590911662 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f28a006e0b0 0x7f28a0070560 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f28b8066280 tx=0x7f28a8009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:49:10.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.438+0000 7f28b4ff9640 1 -- 172.21.15.28:0/3590911662 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f28ac076fd0 con 0x7f28b81018d0 2024-07-03T15:49:10.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.695+0000 7f28be2b2640 1 -- 172.21.15.28:0/3590911662 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f28b804b530 con 0x7f28b81018d0 2024-07-03T15:49:10.699 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:10 smithi028 ceph-mon[92858]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:49:10.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.695+0000 7f28b4ff9640 1 -- 172.21.15.28:0/3590911662 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+838 (secure 0 0 0) 0x7f28ac07a690 con 0x7f28b81018d0 2024-07-03T15:49:10.700 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-07-03T15:49:10.700 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": { 2024-07-03T15:49:10.700 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 2 2024-07-03T15:49:10.700 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:49:10.700 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": { 2024-07-03T15:49:10.700 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 2 2024-07-03T15:49:10.700 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:49:10.700 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": { 2024-07-03T15:49:10.700 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 3, 2024-07-03T15:49:10.700 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 3 2024-07-03T15:49:10.701 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:49:10.701 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": { 2024-07-03T15:49:10.701 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 4 2024-07-03T15:49:10.701 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:49:10.701 INFO:teuthology.orchestra.run.smithi028.stdout: "overall": { 2024-07-03T15:49:10.701 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 7, 2024-07-03T15:49:10.701 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 7 2024-07-03T15:49:10.701 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-07-03T15:49:10.701 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-07-03T15:49:10.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.697+0000 7f28be2b2640 1 -- 172.21.15.28:0/3590911662 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f28a006e0b0 msgr2=0x7f28a0070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:10.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.697+0000 7f28be2b2640 1 --2- 172.21.15.28:0/3590911662 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f28a006e0b0 0x7f28a0070560 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f28b8066280 tx=0x7f28a8009040 comp rx=0 tx=0).stop 2024-07-03T15:49:10.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.697+0000 7f28be2b2640 1 -- 172.21.15.28:0/3590911662 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28b81018d0 msgr2=0x7f28b810f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:10.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.697+0000 7f28be2b2640 1 --2- 172.21.15.28:0/3590911662 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28b81018d0 0x7f28b810f720 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f28ac003080 tx=0x7f28ac009da0 comp rx=0 tx=0).stop 2024-07-03T15:49:10.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.698+0000 7f28be2b2640 1 -- 172.21.15.28:0/3590911662 shutdown_connections 2024-07-03T15:49:10.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.698+0000 7f28be2b2640 1 --2- 172.21.15.28:0/3590911662 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f28a006e0b0 0x7f28a0070560 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:10.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.698+0000 7f28be2b2640 1 --2- 172.21.15.28:0/3590911662 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28b81018d0 0x7f28b810f720 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:10.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.698+0000 7f28be2b2640 1 --2- 172.21.15.28:0/3590911662 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28b8066520 0x7f28b810f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:10.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.698+0000 7f28be2b2640 1 -- 172.21.15.28:0/3590911662 >> 172.21.15.28:0/3590911662 conn(0x7f28b80f6a20 msgr2=0x7f28b80f6e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:49:10.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.698+0000 7f28be2b2640 1 -- 172.21.15.28:0/3590911662 shutdown_connections 2024-07-03T15:49:10.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.698+0000 7f28be2b2640 1 -- 172.21.15.28:0/3590911662 wait complete. 2024-07-03T15:49:10.729 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:10 smithi148 ceph-mon[80490]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:49:10.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.780+0000 7fdd4a2f7640 1 -- 172.21.15.28:0/2199016658 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd440fc360 msgr2=0x7fdd440fc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:10.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.780+0000 7fdd4a2f7640 1 --2- 172.21.15.28:0/2199016658 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd440fc360 0x7fdd440fc770 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdd340048d0 tx=0x7fdd34009da0 comp rx=0 tx=0).stop 2024-07-03T15:49:10.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.780+0000 7fdd4a2f7640 1 -- 172.21.15.28:0/2199016658 shutdown_connections 2024-07-03T15:49:10.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.780+0000 7fdd4a2f7640 1 --2- 172.21.15.28:0/2199016658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd440fd630 0x7fdd440fdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:10.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.780+0000 7fdd4a2f7640 1 --2- 172.21.15.28:0/2199016658 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd440fc360 0x7fdd440fc770 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:10.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.780+0000 7fdd4a2f7640 1 -- 172.21.15.28:0/2199016658 >> 172.21.15.28:0/2199016658 conn(0x7fdd440f7b40 msgr2=0x7fdd440f9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:49:10.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.780+0000 7fdd4a2f7640 1 -- 172.21.15.28:0/2199016658 shutdown_connections 2024-07-03T15:49:10.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.780+0000 7fdd4a2f7640 1 -- 172.21.15.28:0/2199016658 wait complete. 2024-07-03T15:49:10.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.781+0000 7fdd4a2f7640 1 Processor -- start 2024-07-03T15:49:10.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.781+0000 7fdd4a2f7640 1 -- start start 2024-07-03T15:49:10.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.781+0000 7fdd4a2f7640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd440fc360 0x7fdd4410d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:49:10.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.781+0000 7fdd4a2f7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd440fd630 0x7fdd4410d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:49:10.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.781+0000 7fdd4a2f7640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4410e3b0 con 0x7fdd440fc360 2024-07-03T15:49:10.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.781+0000 7fdd4a2f7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4410e520 con 0x7fdd440fd630 2024-07-03T15:49:10.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.781+0000 7fdd43fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd440fc360 0x7fdd4410d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:49:10.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.781+0000 7fdd43fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd440fc360 0x7fdd4410d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37228/0 (socket says 172.21.15.28:37228) 2024-07-03T15:49:10.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.781+0000 7fdd43fff640 1 -- 172.21.15.28:0/3917615712 learned_addr learned my addr 172.21.15.28:0/3917615712 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:49:10.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.781+0000 7fdd437fe640 1 --2- 172.21.15.28:0/3917615712 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd440fd630 0x7fdd4410d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:49:10.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.781+0000 7fdd43fff640 1 -- 172.21.15.28:0/3917615712 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd440fd630 msgr2=0x7fdd4410d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:10.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.781+0000 7fdd43fff640 1 --2- 172.21.15.28:0/3917615712 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd440fd630 0x7fdd4410d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:10.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.782+0000 7fdd43fff640 1 -- 172.21.15.28:0/3917615712 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd34005040 con 0x7fdd440fc360 2024-07-03T15:49:10.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.782+0000 7fdd43fff640 1 --2- 172.21.15.28:0/3917615712 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd440fc360 0x7fdd4410d050 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdd34003080 tx=0x7fdd34009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:49:10.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.782+0000 7fdd40ff9640 1 -- 172.21.15.28:0/3917615712 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd34038070 con 0x7fdd440fc360 2024-07-03T15:49:10.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.782+0000 7fdd40ff9640 1 -- 172.21.15.28:0/3917615712 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd34032970 con 0x7fdd440fc360 2024-07-03T15:49:10.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.782+0000 7fdd4a2f7640 1 -- 172.21.15.28:0/3917615712 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd4410e7a0 con 0x7fdd440fc360 2024-07-03T15:49:10.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.782+0000 7fdd4a2f7640 1 -- 172.21.15.28:0/3917615712 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd4410ec60 con 0x7fdd440fc360 2024-07-03T15:49:10.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.782+0000 7fdd40ff9640 1 -- 172.21.15.28:0/3917615712 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd34033900 con 0x7fdd440fc360 2024-07-03T15:49:10.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.783+0000 7fdd4a2f7640 1 -- 172.21.15.28:0/3917615712 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd44105740 con 0x7fdd440fc360 2024-07-03T15:49:10.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.784+0000 7fdd40ff9640 1 -- 172.21.15.28:0/3917615712 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fdd34030020 con 0x7fdd440fc360 2024-07-03T15:49:10.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.784+0000 7fdd40ff9640 1 --2- 172.21.15.28:0/3917615712 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fdd2c06e0b0 0x7fdd2c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:49:10.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.784+0000 7fdd40ff9640 1 -- 172.21.15.28:0/3917615712 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fdd340adae0 con 0x7fdd440fc360 2024-07-03T15:49:10.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.784+0000 7fdd437fe640 1 --2- 172.21.15.28:0/3917615712 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fdd2c06e0b0 0x7fdd2c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:49:10.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.785+0000 7fdd437fe640 1 --2- 172.21.15.28:0/3917615712 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fdd2c06e0b0 0x7fdd2c070560 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdd4410ce00 tx=0x7fdd38009120 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:49:10.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:10.789+0000 7fdd40ff9640 1 -- 172.21.15.28:0/3917615712 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fdd34076f90 con 0x7fdd440fc360 2024-07-03T15:49:10.982 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:10 smithi148 systemd[1]: Stopping Ceph osd.3 for a85e4be6-3951-11ef-bcab-c7b262605968... 2024-07-03T15:49:11.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.011+0000 7fdd4a2f7640 1 -- 172.21.15.28:0/3917615712 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fdd440fb7c0 con 0x7fdd440fc360 2024-07-03T15:49:11.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.011+0000 7fdd40ff9640 1 -- 172.21.15.28:0/3917615712 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1908 (secure 0 0 0) 0x7fdd34041790 con 0x7fdd440fc360 2024-07-03T15:49:11.015 INFO:teuthology.orchestra.run.smithi028.stdout:e17 2024-07-03T15:49:11.015 INFO:teuthology.orchestra.run.smithi028.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-03T15:49:11.015 INFO:teuthology.orchestra.run.smithi028.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:49:11.015 INFO:teuthology.orchestra.run.smithi028.stdout:legacy client fscid: 1 2024-07-03T15:49:11.016 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:49:11.016 INFO:teuthology.orchestra.run.smithi028.stdout:Filesystem 'cephfs' (1) 2024-07-03T15:49:11.016 INFO:teuthology.orchestra.run.smithi028.stdout:fs_name cephfs 2024-07-03T15:49:11.016 INFO:teuthology.orchestra.run.smithi028.stdout:epoch 16 2024-07-03T15:49:11.016 INFO:teuthology.orchestra.run.smithi028.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-03T15:49:11.016 INFO:teuthology.orchestra.run.smithi028.stdout:created 2024-07-03T15:41:40.656298+0000 2024-07-03T15:49:11.016 INFO:teuthology.orchestra.run.smithi028.stdout:modified 2024-07-03T15:41:56.454984+0000 2024-07-03T15:49:11.016 INFO:teuthology.orchestra.run.smithi028.stdout:tableserver 0 2024-07-03T15:49:11.016 INFO:teuthology.orchestra.run.smithi028.stdout:root 0 2024-07-03T15:49:11.017 INFO:teuthology.orchestra.run.smithi028.stdout:session_timeout 60 2024-07-03T15:49:11.017 INFO:teuthology.orchestra.run.smithi028.stdout:session_autoclose 300 2024-07-03T15:49:11.017 INFO:teuthology.orchestra.run.smithi028.stdout:max_file_size 1099511627776 2024-07-03T15:49:11.017 INFO:teuthology.orchestra.run.smithi028.stdout:max_xattr_size 65536 2024-07-03T15:49:11.017 INFO:teuthology.orchestra.run.smithi028.stdout:required_client_features {} 2024-07-03T15:49:11.017 INFO:teuthology.orchestra.run.smithi028.stdout:last_failure 0 2024-07-03T15:49:11.017 INFO:teuthology.orchestra.run.smithi028.stdout:last_failure_osd_epoch 0 2024-07-03T15:49:11.017 INFO:teuthology.orchestra.run.smithi028.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:49:11.017 INFO:teuthology.orchestra.run.smithi028.stdout:max_mds 1 2024-07-03T15:49:11.017 INFO:teuthology.orchestra.run.smithi028.stdout:in 0 2024-07-03T15:49:11.017 INFO:teuthology.orchestra.run.smithi028.stdout:up {0=14450} 2024-07-03T15:49:11.017 INFO:teuthology.orchestra.run.smithi028.stdout:failed 2024-07-03T15:49:11.017 INFO:teuthology.orchestra.run.smithi028.stdout:damaged 2024-07-03T15:49:11.017 INFO:teuthology.orchestra.run.smithi028.stdout:stopped 2024-07-03T15:49:11.018 INFO:teuthology.orchestra.run.smithi028.stdout:data_pools [3] 2024-07-03T15:49:11.018 INFO:teuthology.orchestra.run.smithi028.stdout:metadata_pool 2 2024-07-03T15:49:11.018 INFO:teuthology.orchestra.run.smithi028.stdout:inline_data disabled 2024-07-03T15:49:11.018 INFO:teuthology.orchestra.run.smithi028.stdout:balancer 2024-07-03T15:49:11.018 INFO:teuthology.orchestra.run.smithi028.stdout:bal_rank_mask -1 2024-07-03T15:49:11.018 INFO:teuthology.orchestra.run.smithi028.stdout:standby_count_wanted 1 2024-07-03T15:49:11.018 INFO:teuthology.orchestra.run.smithi028.stdout:qdb_cluster leader: 0 members: 2024-07-03T15:49:11.018 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi028.rhpvve{0:14450} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.28:6826/1613936042,v1:172.21.15.28:6827/1613936042] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:49:11.018 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi028.wmbxew{0:14460} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.28:6828/1730336821,v1:172.21.15.28:6829/1730336821] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:49:11.018 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:49:11.018 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:49:11.018 INFO:teuthology.orchestra.run.smithi028.stdout:Standby daemons: 2024-07-03T15:49:11.018 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:49:11.018 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi148.szomfl{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/4171231858,v1:172.21.15.148:6827/4171231858] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:49:11.019 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi148.xgpuld{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/2347536398,v1:172.21.15.148:6825/2347536398] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:49:11.019 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 17 2024-07-03T15:49:11.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.014+0000 7fdd4a2f7640 1 -- 172.21.15.28:0/3917615712 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fdd2c06e0b0 msgr2=0x7fdd2c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:11.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.014+0000 7fdd4a2f7640 1 --2- 172.21.15.28:0/3917615712 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fdd2c06e0b0 0x7fdd2c070560 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdd4410ce00 tx=0x7fdd38009120 comp rx=0 tx=0).stop 2024-07-03T15:49:11.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.014+0000 7fdd4a2f7640 1 -- 172.21.15.28:0/3917615712 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd440fc360 msgr2=0x7fdd4410d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:11.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.014+0000 7fdd4a2f7640 1 --2- 172.21.15.28:0/3917615712 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd440fc360 0x7fdd4410d050 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdd34003080 tx=0x7fdd34009da0 comp rx=0 tx=0).stop 2024-07-03T15:49:11.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.014+0000 7fdd4a2f7640 1 -- 172.21.15.28:0/3917615712 shutdown_connections 2024-07-03T15:49:11.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.014+0000 7fdd4a2f7640 1 --2- 172.21.15.28:0/3917615712 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fdd2c06e0b0 0x7fdd2c070560 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:11.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.014+0000 7fdd4a2f7640 1 --2- 172.21.15.28:0/3917615712 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd440fd630 0x7fdd4410d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:11.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.014+0000 7fdd4a2f7640 1 --2- 172.21.15.28:0/3917615712 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd440fc360 0x7fdd4410d050 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:11.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.014+0000 7fdd4a2f7640 1 -- 172.21.15.28:0/3917615712 >> 172.21.15.28:0/3917615712 conn(0x7fdd440f7b40 msgr2=0x7fdd440f9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:49:11.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.014+0000 7fdd4a2f7640 1 -- 172.21.15.28:0/3917615712 shutdown_connections 2024-07-03T15:49:11.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.014+0000 7fdd4a2f7640 1 -- 172.21.15.28:0/3917615712 wait complete. 2024-07-03T15:49:11.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.097+0000 7fb630e97640 1 -- 172.21.15.28:0/868605997 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb62c0fc330 msgr2=0x7fb62c0fc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:11.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.097+0000 7fb630e97640 1 --2- 172.21.15.28:0/868605997 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb62c0fc330 0x7fb62c0fc740 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb61c0048d0 tx=0x7fb61c009da0 comp rx=0 tx=0).stop 2024-07-03T15:49:11.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.097+0000 7fb630e97640 1 -- 172.21.15.28:0/868605997 shutdown_connections 2024-07-03T15:49:11.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.097+0000 7fb630e97640 1 --2- 172.21.15.28:0/868605997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb62c0fd600 0x7fb62c0fda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:11.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.097+0000 7fb630e97640 1 --2- 172.21.15.28:0/868605997 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb62c0fc330 0x7fb62c0fc740 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:11.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.097+0000 7fb630e97640 1 -- 172.21.15.28:0/868605997 >> 172.21.15.28:0/868605997 conn(0x7fb62c0f7af0 msgr2=0x7fb62c0f9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:49:11.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.097+0000 7fb630e97640 1 -- 172.21.15.28:0/868605997 shutdown_connections 2024-07-03T15:49:11.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.097+0000 7fb630e97640 1 -- 172.21.15.28:0/868605997 wait complete. 2024-07-03T15:49:11.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.098+0000 7fb630e97640 1 Processor -- start 2024-07-03T15:49:11.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.098+0000 7fb630e97640 1 -- start start 2024-07-03T15:49:11.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.098+0000 7fb630e97640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb62c0fc330 0x7fb62c071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:49:11.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.098+0000 7fb630e97640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb62c0fd600 0x7fb62c06f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:49:11.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.098+0000 7fb630e97640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62c06fe00 con 0x7fb62c0fd600 2024-07-03T15:49:11.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.098+0000 7fb630e97640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62c06ff40 con 0x7fb62c0fc330 2024-07-03T15:49:11.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.098+0000 7fb623fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb62c0fd600 0x7fb62c06f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:49:11.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.098+0000 7fb623fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb62c0fd600 0x7fb62c06f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37250/0 (socket says 172.21.15.28:37250) 2024-07-03T15:49:11.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.098+0000 7fb623fff640 1 -- 172.21.15.28:0/3677592426 learned_addr learned my addr 172.21.15.28:0/3677592426 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:49:11.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.098+0000 7fb62a59c640 1 --2- 172.21.15.28:0/3677592426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb62c0fc330 0x7fb62c071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:49:11.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.098+0000 7fb623fff640 1 -- 172.21.15.28:0/3677592426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb62c0fc330 msgr2=0x7fb62c071380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:11.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.098+0000 7fb623fff640 1 --2- 172.21.15.28:0/3677592426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb62c0fc330 0x7fb62c071380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:11.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.098+0000 7fb623fff640 1 -- 172.21.15.28:0/3677592426 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb61c005040 con 0x7fb62c0fd600 2024-07-03T15:49:11.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.099+0000 7fb623fff640 1 --2- 172.21.15.28:0/3677592426 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb62c0fd600 0x7fb62c06f870 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb614002a50 tx=0x7fb614004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:49:11.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.099+0000 7fb622ffd640 1 -- 172.21.15.28:0/3677592426 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb614007c30 con 0x7fb62c0fd600 2024-07-03T15:49:11.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.099+0000 7fb622ffd640 1 -- 172.21.15.28:0/3677592426 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb61400f040 con 0x7fb62c0fd600 2024-07-03T15:49:11.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.099+0000 7fb630e97640 1 -- 172.21.15.28:0/3677592426 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb62c070220 con 0x7fb62c0fd600 2024-07-03T15:49:11.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.099+0000 7fb622ffd640 1 -- 172.21.15.28:0/3677592426 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb61400db60 con 0x7fb62c0fd600 2024-07-03T15:49:11.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.099+0000 7fb630e97640 1 -- 172.21.15.28:0/3677592426 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb62c114f90 con 0x7fb62c0fd600 2024-07-03T15:49:11.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.100+0000 7fb630e97640 1 -- 172.21.15.28:0/3677592426 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb62c04b530 con 0x7fb62c0fd600 2024-07-03T15:49:11.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.100+0000 7fb622ffd640 1 -- 172.21.15.28:0/3677592426 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb61400dcc0 con 0x7fb62c0fd600 2024-07-03T15:49:11.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.101+0000 7fb622ffd640 1 --2- 172.21.15.28:0/3677592426 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fb61806e0b0 0x7fb618070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:49:11.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.101+0000 7fb622ffd640 1 -- 172.21.15.28:0/3677592426 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fb61408d900 con 0x7fb62c0fd600 2024-07-03T15:49:11.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.101+0000 7fb62a59c640 1 --2- 172.21.15.28:0/3677592426 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fb61806e0b0 0x7fb618070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:49:11.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.102+0000 7fb62a59c640 1 --2- 172.21.15.28:0/3677592426 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fb61806e0b0 0x7fb618070560 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb61c000c00 tx=0x7fb61c035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:49:11.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.105+0000 7fb622ffd640 1 -- 172.21.15.28:0/3677592426 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fb614056d00 con 0x7fb62c0fd600 2024-07-03T15:49:11.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.307+0000 7fb630e97640 1 -- 172.21.15.28:0/3677592426 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb62c101930 con 0x7fb61806e0b0 2024-07-03T15:49:11.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.308+0000 7fb622ffd640 1 -- 172.21.15.28:0/3677592426 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fb62c101930 con 0x7fb61806e0b0 2024-07-03T15:49:11.312 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-07-03T15:49:11.312 INFO:teuthology.orchestra.run.smithi028.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2", 2024-07-03T15:49:11.312 INFO:teuthology.orchestra.run.smithi028.stdout: "in_progress": true, 2024-07-03T15:49:11.312 INFO:teuthology.orchestra.run.smithi028.stdout: "which": "Upgrading all daemon types on all hosts", 2024-07-03T15:49:11.313 INFO:teuthology.orchestra.run.smithi028.stdout: "services_complete": [ 2024-07-03T15:49:11.313 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr", 2024-07-03T15:49:11.313 INFO:teuthology.orchestra.run.smithi028.stdout: "crash", 2024-07-03T15:49:11.313 INFO:teuthology.orchestra.run.smithi028.stdout: "mon" 2024-07-03T15:49:11.313 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-07-03T15:49:11.313 INFO:teuthology.orchestra.run.smithi028.stdout: "progress": "9/21 daemons upgraded", 2024-07-03T15:49:11.313 INFO:teuthology.orchestra.run.smithi028.stdout: "message": "Currently upgrading osd daemons", 2024-07-03T15:49:11.313 INFO:teuthology.orchestra.run.smithi028.stdout: "is_paused": false 2024-07-03T15:49:11.313 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-07-03T15:49:11.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.310+0000 7fb630e97640 1 -- 172.21.15.28:0/3677592426 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fb61806e0b0 msgr2=0x7fb618070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:11.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.310+0000 7fb630e97640 1 --2- 172.21.15.28:0/3677592426 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fb61806e0b0 0x7fb618070560 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb61c000c00 tx=0x7fb61c035040 comp rx=0 tx=0).stop 2024-07-03T15:49:11.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.310+0000 7fb630e97640 1 -- 172.21.15.28:0/3677592426 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb62c0fd600 msgr2=0x7fb62c06f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:11.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.310+0000 7fb630e97640 1 --2- 172.21.15.28:0/3677592426 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb62c0fd600 0x7fb62c06f870 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb614002a50 tx=0x7fb614004600 comp rx=0 tx=0).stop 2024-07-03T15:49:11.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.310+0000 7fb630e97640 1 -- 172.21.15.28:0/3677592426 shutdown_connections 2024-07-03T15:49:11.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.310+0000 7fb630e97640 1 --2- 172.21.15.28:0/3677592426 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fb61806e0b0 0x7fb618070560 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:11.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.310+0000 7fb630e97640 1 --2- 172.21.15.28:0/3677592426 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb62c0fd600 0x7fb62c06f870 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:11.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.310+0000 7fb630e97640 1 --2- 172.21.15.28:0/3677592426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb62c0fc330 0x7fb62c071380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:11.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.310+0000 7fb630e97640 1 -- 172.21.15.28:0/3677592426 >> 172.21.15.28:0/3677592426 conn(0x7fb62c0f7af0 msgr2=0x7fb62c0f91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:49:11.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.311+0000 7fb630e97640 1 -- 172.21.15.28:0/3677592426 shutdown_connections 2024-07-03T15:49:11.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.311+0000 7fb630e97640 1 -- 172.21.15.28:0/3677592426 wait complete. 2024-07-03T15:49:11.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.393+0000 7f8c91a1a640 1 -- 172.21.15.28:0/3898292735 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8c8c0fc360 msgr2=0x7f8c8c0fc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:11.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.393+0000 7f8c91a1a640 1 --2- 172.21.15.28:0/3898292735 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8c8c0fc360 0x7f8c8c0fc770 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c0048d0 tx=0x7f8c7c009da0 comp rx=0 tx=0).stop 2024-07-03T15:49:11.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.393+0000 7f8c91a1a640 1 -- 172.21.15.28:0/3898292735 shutdown_connections 2024-07-03T15:49:11.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.393+0000 7f8c91a1a640 1 --2- 172.21.15.28:0/3898292735 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c8c0fd630 0x7f8c8c0fdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:11.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.393+0000 7f8c91a1a640 1 --2- 172.21.15.28:0/3898292735 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8c8c0fc360 0x7f8c8c0fc770 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:11.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.393+0000 7f8c91a1a640 1 -- 172.21.15.28:0/3898292735 >> 172.21.15.28:0/3898292735 conn(0x7f8c8c0f7b40 msgr2=0x7f8c8c0f9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:49:11.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.393+0000 7f8c91a1a640 1 -- 172.21.15.28:0/3898292735 shutdown_connections 2024-07-03T15:49:11.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.393+0000 7f8c91a1a640 1 -- 172.21.15.28:0/3898292735 wait complete. 2024-07-03T15:49:11.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.394+0000 7f8c91a1a640 1 Processor -- start 2024-07-03T15:49:11.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.394+0000 7f8c91a1a640 1 -- start start 2024-07-03T15:49:11.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.394+0000 7f8c91a1a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c8c0fc360 0x7f8c8c10cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:49:11.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.394+0000 7f8c91a1a640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8c8c0fd630 0x7f8c8c10d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:49:11.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.394+0000 7f8c91a1a640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8c10db60 con 0x7f8c8c0fd630 2024-07-03T15:49:11.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.394+0000 7f8c91a1a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8c10e330 con 0x7f8c8c0fc360 2024-07-03T15:49:11.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.394+0000 7f8c8a7fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8c8c0fd630 0x7f8c8c10d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:49:11.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.394+0000 7f8c8a7fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8c8c0fd630 0x7f8c8c10d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37264/0 (socket says 172.21.15.28:37264) 2024-07-03T15:49:11.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.394+0000 7f8c8a7fc640 1 -- 172.21.15.28:0/1612749541 learned_addr learned my addr 172.21.15.28:0/1612749541 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:49:11.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.394+0000 7f8c8affd640 1 --2- 172.21.15.28:0/1612749541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c8c0fc360 0x7f8c8c10cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:49:11.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.394+0000 7f8c8a7fc640 1 -- 172.21.15.28:0/1612749541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c8c0fc360 msgr2=0x7f8c8c10cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:11.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.394+0000 7f8c8a7fc640 1 --2- 172.21.15.28:0/1612749541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c8c0fc360 0x7f8c8c10cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:11.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.395+0000 7f8c8a7fc640 1 -- 172.21.15.28:0/1612749541 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c7c005040 con 0x7f8c8c0fd630 2024-07-03T15:49:11.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.395+0000 7f8c8a7fc640 1 --2- 172.21.15.28:0/1612749541 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8c8c0fd630 0x7f8c8c10d510 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8c80002a50 tx=0x7f8c80004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:49:11.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.395+0000 7f8c737fe640 1 -- 172.21.15.28:0/1612749541 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c80007c30 con 0x7f8c8c0fd630 2024-07-03T15:49:11.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.395+0000 7f8c737fe640 1 -- 172.21.15.28:0/1612749541 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8c8000f040 con 0x7f8c8c0fd630 2024-07-03T15:49:11.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.395+0000 7f8c91a1a640 1 -- 172.21.15.28:0/1612749541 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c8c10e5c0 con 0x7f8c8c0fd630 2024-07-03T15:49:11.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.395+0000 7f8c737fe640 1 -- 172.21.15.28:0/1612749541 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c8000db60 con 0x7f8c8c0fd630 2024-07-03T15:49:11.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.395+0000 7f8c91a1a640 1 -- 172.21.15.28:0/1612749541 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c8c10eb60 con 0x7f8c8c0fd630 2024-07-03T15:49:11.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.396+0000 7f8c91a1a640 1 -- 172.21.15.28:0/1612749541 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c8c0fb7c0 con 0x7f8c8c0fd630 2024-07-03T15:49:11.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.397+0000 7f8c737fe640 1 -- 172.21.15.28:0/1612749541 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8c8000dcc0 con 0x7f8c8c0fd630 2024-07-03T15:49:11.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.397+0000 7f8c737fe640 1 --2- 172.21.15.28:0/1612749541 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f8c7406e0b0 0x7f8c74070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:49:11.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.397+0000 7f8c737fe640 1 -- 172.21.15.28:0/1612749541 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f8c8008d900 con 0x7f8c8c0fd630 2024-07-03T15:49:11.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.397+0000 7f8c8affd640 1 --2- 172.21.15.28:0/1612749541 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f8c7406e0b0 0x7f8c74070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:49:11.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.398+0000 7f8c8affd640 1 --2- 172.21.15.28:0/1612749541 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f8c7406e0b0 0x7f8c74070560 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c0043d0 tx=0x7f8c7c035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:49:11.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.402+0000 7f8c737fe640 1 -- 172.21.15.28:0/1612749541 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f8c80056d00 con 0x7f8c8c0fd630 2024-07-03T15:49:11.482 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:10 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3[41393]: 2024-07-03T15:49:10.982+0000 7f4ada0a7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-03T15:49:11.482 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:10 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3[41393]: 2024-07-03T15:49:10.982+0000 7f4ada0a7640 -1 osd.3 63 *** Got signal Terminated *** 2024-07-03T15:49:11.482 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:10 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3[41393]: 2024-07-03T15:49:10.982+0000 7f4ada0a7640 -1 osd.3 63 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-03T15:49:11.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.666+0000 7f8c91a1a640 1 -- 172.21.15.28:0/1612749541 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f8c8c04b530 con 0x7f8c8c0fd630 2024-07-03T15:49:11.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.666+0000 7f8c737fe640 1 -- 172.21.15.28:0/1612749541 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+100 (secure 0 0 0) 0x7f8c8005a3c0 con 0x7f8c8c0fd630 2024-07-03T15:49:11.670 INFO:teuthology.orchestra.run.smithi028.stdout:HEALTH_WARN 1 osds down 2024-07-03T15:49:11.670 INFO:teuthology.orchestra.run.smithi028.stdout:[WRN] OSD_DOWN: 1 osds down 2024-07-03T15:49:11.670 INFO:teuthology.orchestra.run.smithi028.stdout: osd.3 (root=default,host=smithi148) is down 2024-07-03T15:49:11.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.668+0000 7f8c91a1a640 1 -- 172.21.15.28:0/1612749541 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f8c7406e0b0 msgr2=0x7f8c74070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:11.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.668+0000 7f8c91a1a640 1 --2- 172.21.15.28:0/1612749541 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f8c7406e0b0 0x7f8c74070560 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c0043d0 tx=0x7f8c7c035040 comp rx=0 tx=0).stop 2024-07-03T15:49:11.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.668+0000 7f8c91a1a640 1 -- 172.21.15.28:0/1612749541 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8c8c0fd630 msgr2=0x7f8c8c10d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:11.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.668+0000 7f8c91a1a640 1 --2- 172.21.15.28:0/1612749541 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8c8c0fd630 0x7f8c8c10d510 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8c80002a50 tx=0x7f8c80004600 comp rx=0 tx=0).stop 2024-07-03T15:49:11.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.668+0000 7f8c91a1a640 1 -- 172.21.15.28:0/1612749541 shutdown_connections 2024-07-03T15:49:11.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.668+0000 7f8c91a1a640 1 --2- 172.21.15.28:0/1612749541 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f8c7406e0b0 0x7f8c74070560 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:11.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.668+0000 7f8c91a1a640 1 --2- 172.21.15.28:0/1612749541 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8c8c0fd630 0x7f8c8c10d510 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:11.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.668+0000 7f8c91a1a640 1 --2- 172.21.15.28:0/1612749541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c8c0fc360 0x7f8c8c10cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:11.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.668+0000 7f8c91a1a640 1 -- 172.21.15.28:0/1612749541 >> 172.21.15.28:0/1612749541 conn(0x7f8c8c0f7b40 msgr2=0x7f8c8c0f9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:49:11.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.668+0000 7f8c91a1a640 1 -- 172.21.15.28:0/1612749541 shutdown_connections 2024-07-03T15:49:11.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:11.668+0000 7f8c91a1a640 1 -- 172.21.15.28:0/1612749541 wait complete. 2024-07-03T15:49:11.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:11 smithi028 ceph-mon[92858]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:49:11.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:11 smithi028 ceph-mon[92858]: pgmap v102: 65 pgs: 65 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:49:11.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:11 smithi028 ceph-mon[92858]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:49:11.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:11 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/3590911662' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:49:11.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:11 smithi028 ceph-mon[92858]: osd.3 marked itself down and dead 2024-07-03T15:49:11.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:11 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/3917615712' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-03T15:49:11.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:11 smithi148 ceph-mon[80490]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:49:11.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:11 smithi148 ceph-mon[80490]: pgmap v102: 65 pgs: 65 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:49:11.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:11 smithi148 ceph-mon[80490]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:49:11.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:11 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/3590911662' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:49:11.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:11 smithi148 ceph-mon[80490]: osd.3 marked itself down and dead 2024-07-03T15:49:11.943 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:11 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/3917615712' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-03T15:49:12.232 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:12 smithi148 podman[89521]: 2024-07-03 15:49:12.081146341 +0000 UTC m=+1.190755557 container died 5be69a27df3162366ba2e33dd35923fb80606ab8b7f9f94ece2e5b54c803f325 (image=quay.ceph.io/ceph-ci/ceph@sha256:0b7651540cdb24c1b97466c79c298d54c7794c5f53d5d6b6c5d76a2d3b130d85, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, ceph=True, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240625, GIT_BRANCH=HEAD, RELEASE=quincy-d4b063a, maintainer=Guillaume Abrioux ) 2024-07-03T15:49:12.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:12 smithi148 ceph-mon[80490]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:49:12.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:12 smithi148 ceph-mon[80490]: Health check failed: 1 osds down (OSD_DOWN) 2024-07-03T15:49:12.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:12 smithi148 ceph-mon[80490]: osdmap e64: 6 total, 5 up, 6 in 2024-07-03T15:49:12.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:12 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/1612749541' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-03T15:49:12.733 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:12 smithi148 podman[89521]: 2024-07-03 15:49:12.558319892 +0000 UTC m=+1.667929062 container cleanup 5be69a27df3162366ba2e33dd35923fb80606ab8b7f9f94ece2e5b54c803f325 (image=quay.ceph.io/ceph-ci/ceph@sha256:0b7651540cdb24c1b97466c79c298d54c7794c5f53d5d6b6c5d76a2d3b130d85, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3, RELEASE=quincy-d4b063a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240625, ceph=True, org.label-schema.vendor=CentOS) 2024-07-03T15:49:12.733 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:12 smithi148 bash[89521]: ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3 2024-07-03T15:49:12.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:12 smithi028 ceph-mon[92858]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:49:12.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:12 smithi028 ceph-mon[92858]: Health check failed: 1 osds down (OSD_DOWN) 2024-07-03T15:49:12.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:12 smithi028 ceph-mon[92858]: osdmap e64: 6 total, 5 up, 6 in 2024-07-03T15:49:12.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:12 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/1612749541' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-03T15:49:13.180 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:12 smithi148 podman[89547]: 2024-07-03 15:49:12.851669584 +0000 UTC m=+0.763839777 container remove 5be69a27df3162366ba2e33dd35923fb80606ab8b7f9f94ece2e5b54c803f325 (image=quay.ceph.io/ceph-ci/ceph@sha256:0b7651540cdb24c1b97466c79c298d54c7794c5f53d5d6b6c5d76a2d3b130d85, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=quincy-d4b063a, ceph=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240625, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-03T15:49:13.482 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:13 smithi148 podman[89635]: 2024-07-03 15:49:13.177437561 +0000 UTC m=+0.109337249 container create 0ece1985f40cefe3fb7e71bef36758370cdd962e4a0b0d910f947b2e7d8c2901 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240625, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-release-c00506d) 2024-07-03T15:49:13.483 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:13 smithi148 podman[89635]: 2024-07-03 15:49:13.085678219 +0000 UTC m=+0.017577913 image pull 9d49405603e8445bc510362cdeb05f243619be76b0f3897dd4c152b00cffe45f quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2 2024-07-03T15:49:13.483 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:13 smithi148 podman[89635]: 2024-07-03 15:49:13.405203834 +0000 UTC m=+0.337103523 container init 0ece1985f40cefe3fb7e71bef36758370cdd962e4a0b0d910f947b2e7d8c2901 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240625, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=squid-release-c00506d, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-03T15:49:13.483 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:13 smithi148 podman[89635]: 2024-07-03 15:49:13.40944141 +0000 UTC m=+0.341341099 container start 0ece1985f40cefe3fb7e71bef36758370cdd962e4a0b0d910f947b2e7d8c2901 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3-deactivate, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-release-c00506d, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240625) 2024-07-03T15:49:13.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:13 smithi028 ceph-mon[92858]: pgmap v104: 65 pgs: 16 stale+active+clean, 49 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-07-03T15:49:13.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:13 smithi028 ceph-mon[92858]: osdmap e65: 6 total, 5 up, 6 in 2024-07-03T15:49:13.961 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:13 smithi148 ceph-mon[80490]: pgmap v104: 65 pgs: 16 stale+active+clean, 49 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-07-03T15:49:13.961 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:13 smithi148 ceph-mon[80490]: osdmap e65: 6 total, 5 up, 6 in 2024-07-03T15:49:13.961 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:13 smithi148 podman[89635]: 2024-07-03 15:49:13.480047502 +0000 UTC m=+0.411947200 container attach 0ece1985f40cefe3fb7e71bef36758370cdd962e4a0b0d910f947b2e7d8c2901 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=squid-release-c00506d, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240625, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.36.0) 2024-07-03T15:49:13.961 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:13 smithi148 podman[89635]: 2024-07-03 15:49:13.668976014 +0000 UTC m=+0.600875713 container died 0ece1985f40cefe3fb7e71bef36758370cdd962e4a0b0d910f947b2e7d8c2901 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240625, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-release-c00506d, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-07-03T15:49:14.482 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:14 smithi148 podman[89658]: 2024-07-03 15:49:14.208493657 +0000 UTC m=+0.533740880 container remove 0ece1985f40cefe3fb7e71bef36758370cdd962e4a0b0d910f947b2e7d8c2901 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-release-c00506d, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240625, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-03T15:49:14.482 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:14 smithi148 systemd[1]: ceph-a85e4be6-3951-11ef-bcab-c7b262605968@osd.3.service: Deactivated successfully. 2024-07-03T15:49:14.482 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:14 smithi148 systemd[1]: Stopped Ceph osd.3 for a85e4be6-3951-11ef-bcab-c7b262605968. 2024-07-03T15:49:14.482 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:14 smithi148 systemd[1]: ceph-a85e4be6-3951-11ef-bcab-c7b262605968@osd.3.service: Consumed 1min 10.259s CPU time. 2024-07-03T15:49:14.483 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:14 smithi148 systemd[1]: Starting Ceph osd.3 for a85e4be6-3951-11ef-bcab-c7b262605968... 2024-07-03T15:49:14.956 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:14 smithi148 ceph-mon[80490]: Health check failed: Degraded data redundancy: 25/219 objects degraded (11.416%), 9 pgs degraded (PG_DEGRADED) 2024-07-03T15:49:14.956 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:14 smithi148 podman[89759]: 2024-07-03 15:49:14.569796672 +0000 UTC m=+0.017240257 image pull 9d49405603e8445bc510362cdeb05f243619be76b0f3897dd4c152b00cffe45f quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2 2024-07-03T15:49:14.956 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:14 smithi148 podman[89759]: 2024-07-03 15:49:14.709471092 +0000 UTC m=+0.156914670 container create 40c1ca800ed4206919e8830f9f80418eb38a2e53357746d70a67f224959e4b37 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3-activate, ceph=True, RELEASE=squid-release-c00506d, io.buildah.version=1.36.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240625, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-03T15:49:14.957 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:14 smithi148 podman[89759]: 2024-07-03 15:49:14.953495376 +0000 UTC m=+0.400938961 container init 40c1ca800ed4206919e8830f9f80418eb38a2e53357746d70a67f224959e4b37 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3-activate, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-release-c00506d, ceph=True, org.label-schema.build-date=20240625, org.label-schema.vendor=CentOS) 2024-07-03T15:49:15.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:14 smithi028 ceph-mon[92858]: Health check failed: Degraded data redundancy: 25/219 objects degraded (11.416%), 9 pgs degraded (PG_DEGRADED) 2024-07-03T15:49:15.232 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:14 smithi148 podman[89759]: 2024-07-03 15:49:14.956813328 +0000 UTC m=+0.404256914 container start 40c1ca800ed4206919e8830f9f80418eb38a2e53357746d70a67f224959e4b37 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3-activate, RELEASE=squid-release-c00506d, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240625, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.36.0) 2024-07-03T15:49:15.232 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:15 smithi148 podman[89759]: 2024-07-03 15:49:15.028523079 +0000 UTC m=+0.475966679 container attach 40c1ca800ed4206919e8830f9f80418eb38a2e53357746d70a67f224959e4b37 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-release-c00506d, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, org.label-schema.build-date=20240625, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True) 2024-07-03T15:49:15.629 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:15 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3-activate[89774]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-07-03T15:49:15.629 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:15 smithi148 bash[89759]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-07-03T15:49:15.629 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:15 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3-activate[89774]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-07-03T15:49:15.630 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:15 smithi148 bash[89759]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-07-03T15:49:15.630 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:15 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3-activate[89774]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-07-03T15:49:15.630 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:15 smithi148 bash[89759]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-07-03T15:49:15.630 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:15 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3-activate[89774]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-07-03T15:49:15.630 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:15 smithi148 bash[89759]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-07-03T15:49:15.630 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:15 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3-activate[89774]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-07-03T15:49:15.631 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:15 smithi148 bash[89759]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-07-03T15:49:15.631 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:15 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3-activate[89774]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-07-03T15:49:15.631 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:15 smithi148 bash[89759]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-07-03T15:49:15.631 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:15 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3-activate[89774]: --> ceph-volume raw activate successful for osd ID: 3 2024-07-03T15:49:15.631 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:15 smithi148 bash[89759]: --> ceph-volume raw activate successful for osd ID: 3 2024-07-03T15:49:15.631 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:15 smithi148 podman[89759]: 2024-07-03 15:49:15.376483502 +0000 UTC m=+0.823927078 container died 40c1ca800ed4206919e8830f9f80418eb38a2e53357746d70a67f224959e4b37 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3-activate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-release-c00506d, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240625, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True) 2024-07-03T15:49:15.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:15 smithi028 ceph-mon[92858]: pgmap v106: 65 pgs: 13 active+undersized, 5 stale+active+clean, 9 active+undersized+degraded, 38 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 25/219 objects degraded (11.416%) 2024-07-03T15:49:15.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:15 smithi148 ceph-mon[80490]: pgmap v106: 65 pgs: 13 active+undersized, 5 stale+active+clean, 9 active+undersized+degraded, 38 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 25/219 objects degraded (11.416%) 2024-07-03T15:49:15.982 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:15 smithi148 podman[89895]: 2024-07-03 15:49:15.865537525 +0000 UTC m=+0.483489171 container remove 40c1ca800ed4206919e8830f9f80418eb38a2e53357746d70a67f224959e4b37 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3-activate, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-release-c00506d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240625) 2024-07-03T15:49:16.380 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:16 smithi148 podman[89957]: 2024-07-03 15:49:15.979073394 +0000 UTC m=+0.017081777 image pull 9d49405603e8445bc510362cdeb05f243619be76b0f3897dd4c152b00cffe45f quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2 2024-07-03T15:49:16.380 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:16 smithi148 podman[89957]: 2024-07-03 15:49:16.158153204 +0000 UTC m=+0.196161571 container create 1d622eba4b3b59f5a22317e0a65a6343794549b0b33a510f362445651e06f686 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-release-c00506d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240625, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-07-03T15:49:16.380 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:16 smithi148 podman[89957]: 2024-07-03 15:49:16.377110061 +0000 UTC m=+0.415118427 container init 1d622eba4b3b59f5a22317e0a65a6343794549b0b33a510f362445651e06f686 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3, org.label-schema.license=GPLv2, RELEASE=squid-release-c00506d, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240625, io.buildah.version=1.36.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-03T15:49:16.732 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:16 smithi148 podman[89957]: 2024-07-03 15:49:16.380358796 +0000 UTC m=+0.418367163 container start 1d622eba4b3b59f5a22317e0a65a6343794549b0b33a510f362445651e06f686 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-release-c00506d, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240625, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-07-03T15:49:16.732 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:16 smithi148 ceph-osd[89977]: -- 172.21.15.148:0/3687570603 <== mon.1 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55eb7f0c21e0 con 0x55eb7e279c00 2024-07-03T15:49:16.732 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:16 smithi148 bash[89957]: 1d622eba4b3b59f5a22317e0a65a6343794549b0b33a510f362445651e06f686 2024-07-03T15:49:16.732 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:16 smithi148 systemd[1]: Started Ceph osd.3 for a85e4be6-3951-11ef-bcab-c7b262605968. 2024-07-03T15:49:17.232 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:16 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3[89973]: 2024-07-03T15:49:16.924+0000 7f6256911640 -1 Falling back to public interface 2024-07-03T15:49:17.854 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:17 smithi148 ceph-mon[80490]: pgmap v107: 65 pgs: 21 active+undersized, 17 active+undersized+degraded, 27 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46/219 objects degraded (21.005%) 2024-07-03T15:49:17.854 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:17 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:17.854 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:17 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:17.855 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:17 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:49:17.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:17 smithi028 ceph-mon[92858]: pgmap v107: 65 pgs: 21 active+undersized, 17 active+undersized+degraded, 27 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46/219 objects degraded (21.005%) 2024-07-03T15:49:17.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:17 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:17.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:17 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:17.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:17 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:49:19.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:19 smithi028 ceph-mon[92858]: pgmap v108: 65 pgs: 21 active+undersized, 17 active+undersized+degraded, 27 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46/219 objects degraded (21.005%) 2024-07-03T15:49:19.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:19 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:19.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:19 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:19.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:19 smithi148 ceph-mon[80490]: pgmap v108: 65 pgs: 21 active+undersized, 17 active+undersized+degraded, 27 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46/219 objects degraded (21.005%) 2024-07-03T15:49:19.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:19 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:19.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:19 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:21.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:20 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:21.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:20 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:21.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:20 smithi148 ceph-mon[80490]: pgmap v109: 65 pgs: 21 active+undersized, 17 active+undersized+degraded, 27 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46/219 objects degraded (21.005%) 2024-07-03T15:49:21.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:20 smithi148 ceph-mon[80490]: Health check update: Degraded data redundancy: 46/219 objects degraded (21.005%), 17 pgs degraded (PG_DEGRADED) 2024-07-03T15:49:21.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:20 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:21.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:20 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:21.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:20 smithi028 ceph-mon[92858]: pgmap v109: 65 pgs: 21 active+undersized, 17 active+undersized+degraded, 27 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46/219 objects degraded (21.005%) 2024-07-03T15:49:21.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:20 smithi028 ceph-mon[92858]: Health check update: Degraded data redundancy: 46/219 objects degraded (21.005%), 17 pgs degraded (PG_DEGRADED) 2024-07-03T15:49:22.732 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:22 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3[89973]: 2024-07-03T15:49:22.293+0000 7f6256911640 -1 osd.3 63 log_to_monitors true 2024-07-03T15:49:23.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:23 smithi028 ceph-mon[92858]: pgmap v110: 65 pgs: 21 active+undersized, 17 active+undersized+degraded, 27 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46/219 objects degraded (21.005%) 2024-07-03T15:49:23.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:23 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:23.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:23 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:49:23.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:23 smithi028 ceph-mon[92858]: from='osd.3 [v2:172.21.15.148:6800/660374222,v1:172.21.15.148:6801/660374222]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-07-03T15:49:23.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:23 smithi028 ceph-mon[92858]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-07-03T15:49:23.482 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:49:23 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3[89973]: 2024-07-03T15:49:23.170+0000 7f624e6ab640 -1 osd.3 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-03T15:49:23.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:23 smithi148 ceph-mon[80490]: pgmap v110: 65 pgs: 21 active+undersized, 17 active+undersized+degraded, 27 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46/219 objects degraded (21.005%) 2024-07-03T15:49:23.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:23 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:23.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:23 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:49:23.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:23 smithi148 ceph-mon[80490]: from='osd.3 [v2:172.21.15.148:6800/660374222,v1:172.21.15.148:6801/660374222]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-07-03T15:49:23.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:23 smithi148 ceph-mon[80490]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-07-03T15:49:24.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:24 smithi148 ceph-mon[80490]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-07-03T15:49:24.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:24 smithi148 ceph-mon[80490]: osdmap e66: 6 total, 5 up, 6 in 2024-07-03T15:49:24.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:24 smithi148 ceph-mon[80490]: from='osd.3 [v2:172.21.15.148:6800/660374222,v1:172.21.15.148:6801/660374222]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-07-03T15:49:24.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:24 smithi148 ceph-mon[80490]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-07-03T15:49:24.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:24 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:24.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:24 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:24.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:24 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:49:24.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:24 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:49:24.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:24 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:24.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:24 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:49:24.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:24 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:49:24.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:24 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:49:24.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:24 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:49:24.485 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:24 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-07-03T15:49:24.485 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:24 smithi148 ceph-mon[80490]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-07-03T15:49:24.485 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:24 smithi148 ceph-mon[80490]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-07-03T15:49:24.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:24 smithi028 ceph-mon[92858]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-07-03T15:49:24.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:24 smithi028 ceph-mon[92858]: osdmap e66: 6 total, 5 up, 6 in 2024-07-03T15:49:24.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:24 smithi028 ceph-mon[92858]: from='osd.3 [v2:172.21.15.148:6800/660374222,v1:172.21.15.148:6801/660374222]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-07-03T15:49:24.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:24 smithi028 ceph-mon[92858]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-07-03T15:49:24.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:24 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:24.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:24 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:24.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:24 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:49:24.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:24 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:49:24.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:24 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:24.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:24 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:49:24.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:24 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:49:24.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:24 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:49:24.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:24 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:49:24.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:24 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-07-03T15:49:24.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:24 smithi028 ceph-mon[92858]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-07-03T15:49:24.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:24 smithi028 ceph-mon[92858]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-07-03T15:49:25.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:25 smithi148 ceph-mon[80490]: pgmap v112: 65 pgs: 21 active+undersized, 17 active+undersized+degraded, 27 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 46/219 objects degraded (21.005%) 2024-07-03T15:49:25.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:25 smithi148 ceph-mon[80490]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-07-03T15:49:25.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:25 smithi148 ceph-mon[80490]: osd.3 [v2:172.21.15.148:6800/660374222,v1:172.21.15.148:6801/660374222] boot 2024-07-03T15:49:25.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:25 smithi148 ceph-mon[80490]: osdmap e67: 6 total, 6 up, 6 in 2024-07-03T15:49:25.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:25 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-03T15:49:25.595 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:25 smithi028 ceph-mon[92858]: pgmap v112: 65 pgs: 21 active+undersized, 17 active+undersized+degraded, 27 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 46/219 objects degraded (21.005%) 2024-07-03T15:49:25.595 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:25 smithi028 ceph-mon[92858]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-07-03T15:49:25.596 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:25 smithi028 ceph-mon[92858]: osd.3 [v2:172.21.15.148:6800/660374222,v1:172.21.15.148:6801/660374222] boot 2024-07-03T15:49:25.596 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:25 smithi028 ceph-mon[92858]: osdmap e67: 6 total, 6 up, 6 in 2024-07-03T15:49:25.596 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:25 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-03T15:49:26.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:26 smithi148 ceph-mon[80490]: osdmap e68: 6 total, 6 up, 6 in 2024-07-03T15:49:26.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:26 smithi028 ceph-mon[92858]: osdmap e68: 6 total, 6 up, 6 in 2024-07-03T15:49:27.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:27 smithi148 ceph-mon[80490]: pgmap v115: 65 pgs: 4 peering, 18 active+undersized, 16 active+undersized+degraded, 27 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 43/219 objects degraded (19.635%) 2024-07-03T15:49:27.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:27 smithi148 ceph-mon[80490]: Health check update: Degraded data redundancy: 43/219 objects degraded (19.635%), 16 pgs degraded (PG_DEGRADED) 2024-07-03T15:49:27.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:27 smithi028 ceph-mon[92858]: pgmap v115: 65 pgs: 4 peering, 18 active+undersized, 16 active+undersized+degraded, 27 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 43/219 objects degraded (19.635%) 2024-07-03T15:49:27.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:27 smithi028 ceph-mon[92858]: Health check update: Degraded data redundancy: 43/219 objects degraded (19.635%), 16 pgs degraded (PG_DEGRADED) 2024-07-03T15:49:29.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:29 smithi148 ceph-mon[80490]: pgmap v116: 65 pgs: 4 peering, 12 active+undersized, 14 active+undersized+degraded, 35 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s; 37/219 objects degraded (16.895%) 2024-07-03T15:49:29.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:29 smithi028 ceph-mon[92858]: pgmap v116: 65 pgs: 4 peering, 12 active+undersized, 14 active+undersized+degraded, 35 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s; 37/219 objects degraded (16.895%) 2024-07-03T15:49:30.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:30 smithi028 ceph-mon[92858]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 37/219 objects degraded (16.895%), 14 pgs degraded) 2024-07-03T15:49:30.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:30 smithi028 ceph-mon[92858]: Cluster is now healthy 2024-07-03T15:49:30.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:30 smithi148 ceph-mon[80490]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 37/219 objects degraded (16.895%), 14 pgs degraded) 2024-07-03T15:49:30.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:30 smithi148 ceph-mon[80490]: Cluster is now healthy 2024-07-03T15:49:31.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:31 smithi148 ceph-mon[80490]: pgmap v117: 65 pgs: 65 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 442 B/s rd, 1 op/s 2024-07-03T15:49:31.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:31 smithi028 ceph-mon[92858]: pgmap v117: 65 pgs: 65 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 442 B/s rd, 1 op/s 2024-07-03T15:49:33.508 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:33 smithi028 ceph-mon[92858]: pgmap v118: 65 pgs: 65 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 383 B/s rd, 0 op/s 2024-07-03T15:49:33.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:33 smithi148 ceph-mon[80490]: pgmap v118: 65 pgs: 65 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 383 B/s rd, 0 op/s 2024-07-03T15:49:35.595 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:35 smithi028 ceph-mon[92858]: pgmap v119: 65 pgs: 65 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 823 B/s rd, 1 op/s 2024-07-03T15:49:35.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:35 smithi148 ceph-mon[80490]: pgmap v119: 65 pgs: 65 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 823 B/s rd, 1 op/s 2024-07-03T15:49:37.534 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:37 smithi148 ceph-mon[80490]: pgmap v120: 65 pgs: 65 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 748 B/s rd, 1 op/s 2024-07-03T15:49:37.535 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:37 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:37.535 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:37 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:49:37.535 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:37 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:37.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:37 smithi028 ceph-mon[92858]: pgmap v120: 65 pgs: 65 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 748 B/s rd, 1 op/s 2024-07-03T15:49:37.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:37 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:37.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:37 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:49:37.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:37 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:39.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:39 smithi148 ceph-mon[80490]: pgmap v121: 65 pgs: 65 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 682 B/s rd, 1 op/s 2024-07-03T15:49:39.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:39 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-07-03T15:49:39.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:39 smithi148 ceph-mon[80490]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-07-03T15:49:39.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:39 smithi148 ceph-mon[80490]: Upgrade: osd.4 is safe to restart 2024-07-03T15:49:39.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:39 smithi028 ceph-mon[92858]: pgmap v121: 65 pgs: 65 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 682 B/s rd, 1 op/s 2024-07-03T15:49:39.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:39 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-07-03T15:49:39.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:39 smithi028 ceph-mon[92858]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-07-03T15:49:39.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:39 smithi028 ceph-mon[92858]: Upgrade: osd.4 is safe to restart 2024-07-03T15:49:41.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:40 smithi028 ceph-mon[92858]: Upgrade: Updating osd.4 2024-07-03T15:49:41.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:40 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:41.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:40 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-07-03T15:49:41.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:40 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:49:41.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:40 smithi028 ceph-mon[92858]: Deploying daemon osd.4 on smithi148 2024-07-03T15:49:41.178 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:40 smithi148 ceph-mon[80490]: Upgrade: Updating osd.4 2024-07-03T15:49:41.178 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:40 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:41.178 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:40 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-07-03T15:49:41.179 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:40 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:49:41.179 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:40 smithi148 ceph-mon[80490]: Deploying daemon osd.4 on smithi148 2024-07-03T15:49:41.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.763+0000 7f0b2ed80640 1 -- 172.21.15.28:0/2502511600 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b280ff820 msgr2=0x7f0b280ffc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:41.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.763+0000 7f0b2ed80640 1 --2- 172.21.15.28:0/2502511600 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b280ff820 0x7f0b280ffc90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c0048d0 tx=0x7f0b1c009da0 comp rx=0 tx=0).stop 2024-07-03T15:49:41.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.763+0000 7f0b2ed80640 1 -- 172.21.15.28:0/2502511600 shutdown_connections 2024-07-03T15:49:41.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.763+0000 7f0b2ed80640 1 --2- 172.21.15.28:0/2502511600 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b280ff820 0x7f0b280ffc90 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:41.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.763+0000 7f0b2ed80640 1 --2- 172.21.15.28:0/2502511600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b280fe550 0x7f0b280fe960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:41.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.763+0000 7f0b2ed80640 1 -- 172.21.15.28:0/2502511600 >> 172.21.15.28:0/2502511600 conn(0x7f0b280f9e70 msgr2=0x7f0b280fc250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:49:41.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.763+0000 7f0b2ed80640 1 -- 172.21.15.28:0/2502511600 shutdown_connections 2024-07-03T15:49:41.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.763+0000 7f0b2ed80640 1 -- 172.21.15.28:0/2502511600 wait complete. 2024-07-03T15:49:41.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.764+0000 7f0b2ed80640 1 Processor -- start 2024-07-03T15:49:41.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.764+0000 7f0b2ed80640 1 -- start start 2024-07-03T15:49:41.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.764+0000 7f0b2ed80640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b280fe550 0x7f0b2810f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:49:41.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.764+0000 7f0b2ed80640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b280ff820 0x7f0b2810f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:49:41.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.764+0000 7f0b2ed80640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2810fce0 con 0x7f0b280ff820 2024-07-03T15:49:41.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.764+0000 7f0b2ed80640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b28110500 con 0x7f0b280fe550 2024-07-03T15:49:41.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.764+0000 7f0b27fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b280ff820 0x7f0b2810f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:49:41.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.764+0000 7f0b2cb1c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b280fe550 0x7f0b2810f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:49:41.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.764+0000 7f0b27fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b280ff820 0x7f0b2810f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37392/0 (socket says 172.21.15.28:37392) 2024-07-03T15:49:41.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.764+0000 7f0b27fff640 1 -- 172.21.15.28:0/2655006201 learned_addr learned my addr 172.21.15.28:0/2655006201 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:49:41.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.765+0000 7f0b27fff640 1 -- 172.21.15.28:0/2655006201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b280fe550 msgr2=0x7f0b2810f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:41.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.765+0000 7f0b27fff640 1 --2- 172.21.15.28:0/2655006201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b280fe550 0x7f0b2810f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:41.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.765+0000 7f0b27fff640 1 -- 172.21.15.28:0/2655006201 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b1c005040 con 0x7f0b280ff820 2024-07-03T15:49:41.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.765+0000 7f0b27fff640 1 --2- 172.21.15.28:0/2655006201 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b280ff820 0x7f0b2810f690 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c0048a0 tx=0x7f0b1c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:49:41.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.765+0000 7f0b257fa640 1 -- 172.21.15.28:0/2655006201 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b1c038070 con 0x7f0b280ff820 2024-07-03T15:49:41.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.765+0000 7f0b257fa640 1 -- 172.21.15.28:0/2655006201 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0b1c0092b0 con 0x7f0b280ff820 2024-07-03T15:49:41.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.765+0000 7f0b2ed80640 1 -- 172.21.15.28:0/2655006201 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b28110730 con 0x7f0b280ff820 2024-07-03T15:49:41.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.765+0000 7f0b257fa640 1 -- 172.21.15.28:0/2655006201 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b1c033780 con 0x7f0b280ff820 2024-07-03T15:49:41.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.765+0000 7f0b2ed80640 1 -- 172.21.15.28:0/2655006201 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b28110c70 con 0x7f0b280ff820 2024-07-03T15:49:41.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.767+0000 7f0b2ed80640 1 -- 172.21.15.28:0/2655006201 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b280fd9b0 con 0x7f0b280ff820 2024-07-03T15:49:41.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.767+0000 7f0b257fa640 1 -- 172.21.15.28:0/2655006201 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0b1c030020 con 0x7f0b280ff820 2024-07-03T15:49:41.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.768+0000 7f0b257fa640 1 --2- 172.21.15.28:0/2655006201 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f0b1006e0b0 0x7f0b10070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:49:41.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.768+0000 7f0b257fa640 1 -- 172.21.15.28:0/2655006201 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f0b1c0ad810 con 0x7f0b280ff820 2024-07-03T15:49:41.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.768+0000 7f0b2cb1c640 1 --2- 172.21.15.28:0/2655006201 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f0b1006e0b0 0x7f0b10070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:49:41.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.768+0000 7f0b2cb1c640 1 --2- 172.21.15.28:0/2655006201 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f0b1006e0b0 0x7f0b10070560 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0b18009af0 tx=0x7f0b1800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:49:41.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.772+0000 7f0b257fa640 1 -- 172.21.15.28:0/2655006201 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0b1c076cc0 con 0x7f0b280ff820 2024-07-03T15:49:41.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.979+0000 7f0b2ed80640 1 -- 172.21.15.28:0/2655006201 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b28110f80 con 0x7f0b1006e0b0 2024-07-03T15:49:41.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.980+0000 7f0b257fa640 1 -- 172.21.15.28:0/2655006201 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f0b28110f80 con 0x7f0b1006e0b0 2024-07-03T15:49:41.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.982+0000 7f0b2ed80640 1 -- 172.21.15.28:0/2655006201 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f0b1006e0b0 msgr2=0x7f0b10070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:41.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.982+0000 7f0b2ed80640 1 --2- 172.21.15.28:0/2655006201 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f0b1006e0b0 0x7f0b10070560 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0b18009af0 tx=0x7f0b1800b040 comp rx=0 tx=0).stop 2024-07-03T15:49:41.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.982+0000 7f0b2ed80640 1 -- 172.21.15.28:0/2655006201 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b280ff820 msgr2=0x7f0b2810f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:41.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.982+0000 7f0b2ed80640 1 --2- 172.21.15.28:0/2655006201 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b280ff820 0x7f0b2810f690 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c0048a0 tx=0x7f0b1c009da0 comp rx=0 tx=0).stop 2024-07-03T15:49:41.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.982+0000 7f0b2ed80640 1 -- 172.21.15.28:0/2655006201 shutdown_connections 2024-07-03T15:49:41.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.982+0000 7f0b2ed80640 1 --2- 172.21.15.28:0/2655006201 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f0b1006e0b0 0x7f0b10070560 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:41.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.982+0000 7f0b2ed80640 1 --2- 172.21.15.28:0/2655006201 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b280ff820 0x7f0b2810f690 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:41.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.982+0000 7f0b2ed80640 1 --2- 172.21.15.28:0/2655006201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b280fe550 0x7f0b2810f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:41.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.982+0000 7f0b2ed80640 1 -- 172.21.15.28:0/2655006201 >> 172.21.15.28:0/2655006201 conn(0x7f0b280f9e70 msgr2=0x7f0b280fc520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:49:41.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.982+0000 7f0b2ed80640 1 -- 172.21.15.28:0/2655006201 shutdown_connections 2024-07-03T15:49:41.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:41.982+0000 7f0b2ed80640 1 -- 172.21.15.28:0/2655006201 wait complete. 2024-07-03T15:49:41.995 INFO:teuthology.orchestra.run.smithi028.stdout:true 2024-07-03T15:49:42.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.066+0000 7fc68d4f9640 1 -- 172.21.15.28:0/2954154914 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc6880fc370 msgr2=0x7fc6880fc780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:42.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.066+0000 7fc68d4f9640 1 --2- 172.21.15.28:0/2954154914 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc6880fc370 0x7fc6880fc780 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc6780048d0 tx=0x7fc678009da0 comp rx=0 tx=0).stop 2024-07-03T15:49:42.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.066+0000 7fc68d4f9640 1 -- 172.21.15.28:0/2954154914 shutdown_connections 2024-07-03T15:49:42.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.066+0000 7fc68d4f9640 1 --2- 172.21.15.28:0/2954154914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6880fd640 0x7fc6880fdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:42.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.066+0000 7fc68d4f9640 1 --2- 172.21.15.28:0/2954154914 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc6880fc370 0x7fc6880fc780 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:42.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.066+0000 7fc68d4f9640 1 -- 172.21.15.28:0/2954154914 >> 172.21.15.28:0/2954154914 conn(0x7fc6880f7b50 msgr2=0x7fc6880f9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:49:42.070 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:41 smithi028 ceph-mon[92858]: pgmap v122: 65 pgs: 65 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-07-03T15:49:42.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.066+0000 7fc68d4f9640 1 -- 172.21.15.28:0/2954154914 shutdown_connections 2024-07-03T15:49:42.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.066+0000 7fc68d4f9640 1 -- 172.21.15.28:0/2954154914 wait complete. 2024-07-03T15:49:42.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.067+0000 7fc68d4f9640 1 Processor -- start 2024-07-03T15:49:42.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.067+0000 7fc68d4f9640 1 -- start start 2024-07-03T15:49:42.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.067+0000 7fc68d4f9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc6880fc370 0x7fc688071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:49:42.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.067+0000 7fc68d4f9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6880fd640 0x7fc68806f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:49:42.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.067+0000 7fc68d4f9640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68806fe00 con 0x7fc6880fc370 2024-07-03T15:49:42.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.067+0000 7fc68d4f9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68806ff40 con 0x7fc6880fd640 2024-07-03T15:49:42.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.067+0000 7fc686ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc6880fc370 0x7fc688071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:49:42.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.067+0000 7fc686ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc6880fc370 0x7fc688071380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37414/0 (socket says 172.21.15.28:37414) 2024-07-03T15:49:42.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.067+0000 7fc686ffd640 1 -- 172.21.15.28:0/2256913949 learned_addr learned my addr 172.21.15.28:0/2256913949 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:49:42.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.067+0000 7fc6867fc640 1 --2- 172.21.15.28:0/2256913949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6880fd640 0x7fc68806f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:49:42.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.067+0000 7fc686ffd640 1 -- 172.21.15.28:0/2256913949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6880fd640 msgr2=0x7fc68806f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:42.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.067+0000 7fc686ffd640 1 --2- 172.21.15.28:0/2256913949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6880fd640 0x7fc68806f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:42.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.067+0000 7fc686ffd640 1 -- 172.21.15.28:0/2256913949 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc678005040 con 0x7fc6880fc370 2024-07-03T15:49:42.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.068+0000 7fc686ffd640 1 --2- 172.21.15.28:0/2256913949 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc6880fc370 0x7fc688071380 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc6780048a0 tx=0x7fc678009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:49:42.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.068+0000 7fc66f7fe640 1 -- 172.21.15.28:0/2256913949 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc678038070 con 0x7fc6880fc370 2024-07-03T15:49:42.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.068+0000 7fc66f7fe640 1 -- 172.21.15.28:0/2256913949 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc6780092b0 con 0x7fc6880fc370 2024-07-03T15:49:42.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.068+0000 7fc68d4f9640 1 -- 172.21.15.28:0/2256913949 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6880701c0 con 0x7fc6880fc370 2024-07-03T15:49:42.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.068+0000 7fc66f7fe640 1 -- 172.21.15.28:0/2256913949 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6780338a0 con 0x7fc6880fc370 2024-07-03T15:49:42.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.068+0000 7fc68d4f9640 1 -- 172.21.15.28:0/2256913949 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc688114fb0 con 0x7fc6880fc370 2024-07-03T15:49:42.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.069+0000 7fc68d4f9640 1 -- 172.21.15.28:0/2256913949 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6880fb7d0 con 0x7fc6880fc370 2024-07-03T15:49:42.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.069+0000 7fc66f7fe640 1 -- 172.21.15.28:0/2256913949 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc678030020 con 0x7fc6880fc370 2024-07-03T15:49:42.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.070+0000 7fc66f7fe640 1 --2- 172.21.15.28:0/2256913949 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc67006e0b0 0x7fc670070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:49:42.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.070+0000 7fc66f7fe640 1 -- 172.21.15.28:0/2256913949 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fc6780ad910 con 0x7fc6880fc370 2024-07-03T15:49:42.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.070+0000 7fc6867fc640 1 --2- 172.21.15.28:0/2256913949 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc67006e0b0 0x7fc670070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:49:42.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.070+0000 7fc6867fc640 1 --2- 172.21.15.28:0/2256913949 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc67006e0b0 0x7fc670070560 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc67c004ed0 tx=0x7fc67c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:49:42.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.074+0000 7fc66f7fe640 1 -- 172.21.15.28:0/2256913949 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc678076d40 con 0x7fc6880fc370 2024-07-03T15:49:42.161 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:41 smithi148 ceph-mon[80490]: pgmap v122: 65 pgs: 65 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-07-03T15:49:42.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.278+0000 7fc68d4f9640 1 -- 172.21.15.28:0/2256913949 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc688101960 con 0x7fc67006e0b0 2024-07-03T15:49:42.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.279+0000 7fc66f7fe640 1 -- 172.21.15.28:0/2256913949 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fc688101960 con 0x7fc67006e0b0 2024-07-03T15:49:42.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.281+0000 7fc68d4f9640 1 -- 172.21.15.28:0/2256913949 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc67006e0b0 msgr2=0x7fc670070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:42.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.281+0000 7fc68d4f9640 1 --2- 172.21.15.28:0/2256913949 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc67006e0b0 0x7fc670070560 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc67c004ed0 tx=0x7fc67c00b040 comp rx=0 tx=0).stop 2024-07-03T15:49:42.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.281+0000 7fc68d4f9640 1 -- 172.21.15.28:0/2256913949 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc6880fc370 msgr2=0x7fc688071380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:42.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.281+0000 7fc68d4f9640 1 --2- 172.21.15.28:0/2256913949 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc6880fc370 0x7fc688071380 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc6780048a0 tx=0x7fc678009da0 comp rx=0 tx=0).stop 2024-07-03T15:49:42.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.281+0000 7fc68d4f9640 1 -- 172.21.15.28:0/2256913949 shutdown_connections 2024-07-03T15:49:42.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.281+0000 7fc68d4f9640 1 --2- 172.21.15.28:0/2256913949 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc67006e0b0 0x7fc670070560 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:42.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.281+0000 7fc68d4f9640 1 --2- 172.21.15.28:0/2256913949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6880fd640 0x7fc68806f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:42.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.281+0000 7fc68d4f9640 1 --2- 172.21.15.28:0/2256913949 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc6880fc370 0x7fc688071380 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:42.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.281+0000 7fc68d4f9640 1 -- 172.21.15.28:0/2256913949 >> 172.21.15.28:0/2256913949 conn(0x7fc6880f7b50 msgr2=0x7fc6880f91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:49:42.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.281+0000 7fc68d4f9640 1 -- 172.21.15.28:0/2256913949 shutdown_connections 2024-07-03T15:49:42.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.281+0000 7fc68d4f9640 1 -- 172.21.15.28:0/2256913949 wait complete. 2024-07-03T15:49:42.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.363+0000 7fd6318f5640 1 -- 172.21.15.28:0/2452774950 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd62c0ff550 msgr2=0x7fd62c101930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:42.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.363+0000 7fd6318f5640 1 --2- 172.21.15.28:0/2452774950 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd62c0ff550 0x7fd62c101930 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd6200048d0 tx=0x7fd620009da0 comp rx=0 tx=0).stop 2024-07-03T15:49:42.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.363+0000 7fd6318f5640 1 -- 172.21.15.28:0/2452774950 shutdown_connections 2024-07-03T15:49:42.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.363+0000 7fd6318f5640 1 --2- 172.21.15.28:0/2452774950 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd62c0ff550 0x7fd62c101930 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:42.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.363+0000 7fd6318f5640 1 --2- 172.21.15.28:0/2452774950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd62c065990 0x7fd62c0ff010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:42.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.363+0000 7fd6318f5640 1 -- 172.21.15.28:0/2452774950 >> 172.21.15.28:0/2452774950 conn(0x7fd62c0f6a20 msgr2=0x7fd62c0f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:49:42.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.363+0000 7fd6318f5640 1 -- 172.21.15.28:0/2452774950 shutdown_connections 2024-07-03T15:49:42.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.363+0000 7fd6318f5640 1 -- 172.21.15.28:0/2452774950 wait complete. 2024-07-03T15:49:42.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.364+0000 7fd6318f5640 1 Processor -- start 2024-07-03T15:49:42.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.364+0000 7fd6318f5640 1 -- start start 2024-07-03T15:49:42.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.364+0000 7fd6318f5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd62c065990 0x7fd62c0fca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:49:42.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.364+0000 7fd6318f5640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd62c0ff550 0x7fd62c0faf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:49:42.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.364+0000 7fd6318f5640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62c0fb4a0 con 0x7fd62c0ff550 2024-07-03T15:49:42.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.364+0000 7fd6318f5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62c0fb5e0 con 0x7fd62c065990 2024-07-03T15:49:42.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.365+0000 7fd62a7fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd62c0ff550 0x7fd62c0faf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:49:42.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.365+0000 7fd62a7fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd62c0ff550 0x7fd62c0faf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37442/0 (socket says 172.21.15.28:37442) 2024-07-03T15:49:42.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.365+0000 7fd62a7fc640 1 -- 172.21.15.28:0/2217441548 learned_addr learned my addr 172.21.15.28:0/2217441548 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:49:42.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.365+0000 7fd62affd640 1 --2- 172.21.15.28:0/2217441548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd62c065990 0x7fd62c0fca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:49:42.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.365+0000 7fd62a7fc640 1 -- 172.21.15.28:0/2217441548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd62c065990 msgr2=0x7fd62c0fca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:42.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.365+0000 7fd62a7fc640 1 --2- 172.21.15.28:0/2217441548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd62c065990 0x7fd62c0fca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:42.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.365+0000 7fd62a7fc640 1 -- 172.21.15.28:0/2217441548 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd620005040 con 0x7fd62c0ff550 2024-07-03T15:49:42.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.365+0000 7fd62a7fc640 1 --2- 172.21.15.28:0/2217441548 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd62c0ff550 0x7fd62c0faf10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd620003080 tx=0x7fd620009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:49:42.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.365+0000 7fd6137fe640 1 -- 172.21.15.28:0/2217441548 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd620038070 con 0x7fd62c0ff550 2024-07-03T15:49:42.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.365+0000 7fd6137fe640 1 -- 172.21.15.28:0/2217441548 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd620032970 con 0x7fd62c0ff550 2024-07-03T15:49:42.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.365+0000 7fd6318f5640 1 -- 172.21.15.28:0/2217441548 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd62c0fb860 con 0x7fd62c0ff550 2024-07-03T15:49:42.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.365+0000 7fd6137fe640 1 -- 172.21.15.28:0/2217441548 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd620033900 con 0x7fd62c0ff550 2024-07-03T15:49:42.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.366+0000 7fd6318f5640 1 -- 172.21.15.28:0/2217441548 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd62c114fd0 con 0x7fd62c0ff550 2024-07-03T15:49:42.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.367+0000 7fd6318f5640 1 -- 172.21.15.28:0/2217441548 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd62c0fa6a0 con 0x7fd62c0ff550 2024-07-03T15:49:42.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.367+0000 7fd6137fe640 1 -- 172.21.15.28:0/2217441548 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd620030020 con 0x7fd62c0ff550 2024-07-03T15:49:42.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.367+0000 7fd6137fe640 1 --2- 172.21.15.28:0/2217441548 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fd61406e0b0 0x7fd614070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:49:42.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.368+0000 7fd6137fe640 1 -- 172.21.15.28:0/2217441548 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fd6200adb50 con 0x7fd62c0ff550 2024-07-03T15:49:42.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.368+0000 7fd62affd640 1 --2- 172.21.15.28:0/2217441548 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fd61406e0b0 0x7fd614070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:49:42.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.368+0000 7fd62affd640 1 --2- 172.21.15.28:0/2217441548 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fd61406e0b0 0x7fd614070560 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd61c0041d0 tx=0x7fd61c004330 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:49:42.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.372+0000 7fd6137fe640 1 -- 172.21.15.28:0/2217441548 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd620076f80 con 0x7fd62c0ff550 2024-07-03T15:49:42.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.565+0000 7fd6318f5640 1 -- 172.21.15.28:0/2217441548 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd62c0f8510 con 0x7fd61406e0b0 2024-07-03T15:49:42.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.575+0000 7fd6137fe640 1 -- 172.21.15.28:0/2217441548 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fd62c0f8510 con 0x7fd61406e0b0 2024-07-03T15:49:42.578 INFO:teuthology.orchestra.run.smithi028.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-03T15:49:42.579 INFO:teuthology.orchestra.run.smithi028.stdout:alertmanager.smithi028 smithi028 *:9093,9094 running (5m) 55s ago 13m 24.8M - 0.25.0 c8568f914cd2 1425315a0de9 2024-07-03T15:49:42.579 INFO:teuthology.orchestra.run.smithi028.stdout:crash.smithi028 smithi028 running (2m) 55s ago 13m 6895k - 19.0.0-2550-gc00506dc 9d49405603e8 c62c2f45cabe 2024-07-03T15:49:42.579 INFO:teuthology.orchestra.run.smithi028.stdout:crash.smithi148 smithi148 running (2m) 23s ago 12m 6895k - 19.0.0-2550-gc00506dc 9d49405603e8 b1318dcaacb6 2024-07-03T15:49:42.579 INFO:teuthology.orchestra.run.smithi028.stdout:grafana.smithi028 smithi028 *:3000 running (5m) 55s ago 12m 81.9M - 9.4.7 954c08fa6188 f9f17c92cfea 2024-07-03T15:49:42.579 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi028.rhpvve smithi028 running (7m) 55s ago 7m 152M - 17.2.7-1047-gd4b063ab 4ca399d9e65d 3ed5abd02cfb 2024-07-03T15:49:42.579 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi028.wmbxew smithi028 running (7m) 55s ago 7m 110M - 17.2.7-1047-gd4b063ab 4ca399d9e65d b3bf736998f9 2024-07-03T15:49:42.579 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi148.szomfl smithi148 running (7m) 23s ago 7m 16.3M - 17.2.7-1047-gd4b063ab 4ca399d9e65d d94f5c95fa8d 2024-07-03T15:49:42.579 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi148.xgpuld smithi148 running (7m) 23s ago 7m 13.7M - 17.2.7-1047-gd4b063ab 4ca399d9e65d fc70360a8de0 2024-07-03T15:49:42.579 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.smithi028.vtqrtz smithi028 *:8443,9283,8765 running (4m) 55s ago 14m 496M - 19.0.0-2550-gc00506dc 9d49405603e8 14d0363c5b77 2024-07-03T15:49:42.579 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.smithi148.rjzsar smithi148 *:8443,9283,8765 running (3m) 23s ago 12m 440M - 19.0.0-2550-gc00506dc 9d49405603e8 c0193aa8a5dd 2024-07-03T15:49:42.579 INFO:teuthology.orchestra.run.smithi028.stdout:mon.smithi028 smithi028 running (3m) 55s ago 14m 46.8M 2048M 19.0.0-2550-gc00506dc 9d49405603e8 5093f7bf47d7 2024-07-03T15:49:42.579 INFO:teuthology.orchestra.run.smithi028.stdout:mon.smithi148 smithi148 running (3m) 23s ago 12m 37.7M 2048M 19.0.0-2550-gc00506dc 9d49405603e8 d338d2c648da 2024-07-03T15:49:42.580 INFO:teuthology.orchestra.run.smithi028.stdout:node-exporter.smithi028 smithi028 *:9100 running (5m) 55s ago 13m 15.6M - 1.5.0 0da6a335fe13 91ca1f49de0e 2024-07-03T15:49:42.580 INFO:teuthology.orchestra.run.smithi028.stdout:node-exporter.smithi148 smithi148 *:9100 running (4m) 23s ago 12m 16.3M - 1.5.0 0da6a335fe13 1eaaf7b1b561 2024-07-03T15:49:42.580 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0 smithi028 running (2m) 55s ago 11m 101M 1558M 19.0.0-2550-gc00506dc 9d49405603e8 4454cecff8fc 2024-07-03T15:49:42.580 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1 smithi028 running (95s) 55s ago 11m 99.1M 1558M 19.0.0-2550-gc00506dc 9d49405603e8 af67f7088643 2024-07-03T15:49:42.580 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2 smithi028 running (60s) 55s ago 10m 12.1M 1558M 19.0.0-2550-gc00506dc 9d49405603e8 5d160a327dc2 2024-07-03T15:49:42.580 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3 smithi148 running (26s) 23s ago 10m 11.6M 2581M 19.0.0-2550-gc00506dc 9d49405603e8 1d622eba4b3b 2024-07-03T15:49:42.580 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4 smithi148 running (9m) 23s ago 9m 251M 2581M 17.2.7-1047-gd4b063ab 4ca399d9e65d 54f7c26c77ca 2024-07-03T15:49:42.580 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5 smithi148 running (9m) 23s ago 9m 238M 2581M 17.2.7-1047-gd4b063ab 4ca399d9e65d d185c6905384 2024-07-03T15:49:42.580 INFO:teuthology.orchestra.run.smithi028.stdout:prometheus.smithi028 smithi028 *:9095 running (3m) 55s ago 12m 60.5M - 2.43.0 a07b618ecd1d 2b0eb2a95afd 2024-07-03T15:49:42.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.577+0000 7fd6318f5640 1 -- 172.21.15.28:0/2217441548 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fd61406e0b0 msgr2=0x7fd614070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:42.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.577+0000 7fd6318f5640 1 --2- 172.21.15.28:0/2217441548 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fd61406e0b0 0x7fd614070560 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd61c0041d0 tx=0x7fd61c004330 comp rx=0 tx=0).stop 2024-07-03T15:49:42.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.577+0000 7fd6318f5640 1 -- 172.21.15.28:0/2217441548 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd62c0ff550 msgr2=0x7fd62c0faf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:42.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.577+0000 7fd6318f5640 1 --2- 172.21.15.28:0/2217441548 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd62c0ff550 0x7fd62c0faf10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd620003080 tx=0x7fd620009da0 comp rx=0 tx=0).stop 2024-07-03T15:49:42.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.577+0000 7fd6318f5640 1 -- 172.21.15.28:0/2217441548 shutdown_connections 2024-07-03T15:49:42.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.577+0000 7fd6318f5640 1 --2- 172.21.15.28:0/2217441548 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fd61406e0b0 0x7fd614070560 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:42.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.577+0000 7fd6318f5640 1 --2- 172.21.15.28:0/2217441548 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd62c0ff550 0x7fd62c0faf10 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:42.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.577+0000 7fd6318f5640 1 --2- 172.21.15.28:0/2217441548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd62c065990 0x7fd62c0fca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:42.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.577+0000 7fd6318f5640 1 -- 172.21.15.28:0/2217441548 >> 172.21.15.28:0/2217441548 conn(0x7fd62c0f6a20 msgr2=0x7fd62c0f6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:49:42.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.577+0000 7fd6318f5640 1 -- 172.21.15.28:0/2217441548 shutdown_connections 2024-07-03T15:49:42.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.577+0000 7fd6318f5640 1 -- 172.21.15.28:0/2217441548 wait complete. 2024-07-03T15:49:42.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.658+0000 7f2902eb0640 1 -- 172.21.15.28:0/1732559201 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28fc0fc360 msgr2=0x7f28fc0fc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:42.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.658+0000 7f2902eb0640 1 --2- 172.21.15.28:0/1732559201 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28fc0fc360 0x7f28fc0fc770 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f28f00048d0 tx=0x7f28f0009da0 comp rx=0 tx=0).stop 2024-07-03T15:49:42.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.658+0000 7f2902eb0640 1 -- 172.21.15.28:0/1732559201 shutdown_connections 2024-07-03T15:49:42.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.658+0000 7f2902eb0640 1 --2- 172.21.15.28:0/1732559201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28fc0fd630 0x7f28fc0fdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:42.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.658+0000 7f2902eb0640 1 --2- 172.21.15.28:0/1732559201 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28fc0fc360 0x7f28fc0fc770 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:42.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.658+0000 7f2902eb0640 1 -- 172.21.15.28:0/1732559201 >> 172.21.15.28:0/1732559201 conn(0x7f28fc0f7b40 msgr2=0x7f28fc0f9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:49:42.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.658+0000 7f2902eb0640 1 -- 172.21.15.28:0/1732559201 shutdown_connections 2024-07-03T15:49:42.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.658+0000 7f2902eb0640 1 -- 172.21.15.28:0/1732559201 wait complete. 2024-07-03T15:49:42.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.659+0000 7f2902eb0640 1 Processor -- start 2024-07-03T15:49:42.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.659+0000 7f2902eb0640 1 -- start start 2024-07-03T15:49:42.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.659+0000 7f2902eb0640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28fc0fc360 0x7f28fc10cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:49:42.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.659+0000 7f2902eb0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28fc0fd630 0x7f28fc10d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:49:42.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.659+0000 7f2902eb0640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28fc10db80 con 0x7f28fc0fc360 2024-07-03T15:49:42.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.659+0000 7f2902eb0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28fc10e350 con 0x7f28fc0fd630 2024-07-03T15:49:42.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.659+0000 7f2900c4c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28fc0fc360 0x7f28fc10cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:49:42.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.659+0000 7f2900c4c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28fc0fc360 0x7f28fc10cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37456/0 (socket says 172.21.15.28:37456) 2024-07-03T15:49:42.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.659+0000 7f2900c4c640 1 -- 172.21.15.28:0/1680113233 learned_addr learned my addr 172.21.15.28:0/1680113233 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:49:42.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.659+0000 7f28fbfff640 1 --2- 172.21.15.28:0/1680113233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28fc0fd630 0x7f28fc10d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:49:42.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.660+0000 7f2900c4c640 1 -- 172.21.15.28:0/1680113233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28fc0fd630 msgr2=0x7f28fc10d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:42.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.660+0000 7f2900c4c640 1 --2- 172.21.15.28:0/1680113233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28fc0fd630 0x7f28fc10d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:42.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.660+0000 7f2900c4c640 1 -- 172.21.15.28:0/1680113233 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f0005040 con 0x7f28fc0fc360 2024-07-03T15:49:42.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.660+0000 7f2900c4c640 1 --2- 172.21.15.28:0/1680113233 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28fc0fc360 0x7f28fc10cff0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f28f00048a0 tx=0x7f28f0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:49:42.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.660+0000 7f28f97fa640 1 -- 172.21.15.28:0/1680113233 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28f0038070 con 0x7f28fc0fc360 2024-07-03T15:49:42.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.660+0000 7f28f97fa640 1 -- 172.21.15.28:0/1680113233 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f28f00092b0 con 0x7f28fc0fc360 2024-07-03T15:49:42.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.660+0000 7f2902eb0640 1 -- 172.21.15.28:0/1680113233 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28fc10e580 con 0x7f28fc0fc360 2024-07-03T15:49:42.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.660+0000 7f28f97fa640 1 -- 172.21.15.28:0/1680113233 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28f0033780 con 0x7f28fc0fc360 2024-07-03T15:49:42.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.660+0000 7f2902eb0640 1 -- 172.21.15.28:0/1680113233 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28fc10eac0 con 0x7f28fc0fc360 2024-07-03T15:49:42.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.661+0000 7f2902eb0640 1 -- 172.21.15.28:0/1680113233 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28fc0fb7c0 con 0x7f28fc0fc360 2024-07-03T15:49:42.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.662+0000 7f28f97fa640 1 -- 172.21.15.28:0/1680113233 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f28f0030020 con 0x7f28fc0fc360 2024-07-03T15:49:42.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.662+0000 7f28f97fa640 1 --2- 172.21.15.28:0/1680113233 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f28ec06e0b0 0x7f28ec070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:49:42.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.662+0000 7f28f97fa640 1 -- 172.21.15.28:0/1680113233 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f28f00ad890 con 0x7f28fc0fc360 2024-07-03T15:49:42.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.662+0000 7f28fbfff640 1 --2- 172.21.15.28:0/1680113233 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f28ec06e0b0 0x7f28ec070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:49:42.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.663+0000 7f28fbfff640 1 --2- 172.21.15.28:0/1680113233 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f28ec06e0b0 0x7f28ec070560 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f28fc10cda0 tx=0x7f28e800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:49:42.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.667+0000 7f28f97fa640 1 -- 172.21.15.28:0/1680113233 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f28f0076cc0 con 0x7f28fc0fc360 2024-07-03T15:49:42.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.922+0000 7f2902eb0640 1 -- 172.21.15.28:0/1680113233 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f28fc10eda0 con 0x7f28fc0fc360 2024-07-03T15:49:42.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.923+0000 7f28f97fa640 1 -- 172.21.15.28:0/1680113233 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+838 (secure 0 0 0) 0x7f28f007a380 con 0x7f28fc0fc360 2024-07-03T15:49:42.927 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-07-03T15:49:42.927 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": { 2024-07-03T15:49:42.927 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 2 2024-07-03T15:49:42.927 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:49:42.927 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": { 2024-07-03T15:49:42.927 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 2 2024-07-03T15:49:42.927 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:49:42.927 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": { 2024-07-03T15:49:42.928 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 2, 2024-07-03T15:49:42.928 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 4 2024-07-03T15:49:42.928 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:49:42.928 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": { 2024-07-03T15:49:42.928 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 4 2024-07-03T15:49:42.928 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:49:42.928 INFO:teuthology.orchestra.run.smithi028.stdout: "overall": { 2024-07-03T15:49:42.928 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 6, 2024-07-03T15:49:42.928 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 8 2024-07-03T15:49:42.928 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-07-03T15:49:42.928 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-07-03T15:49:42.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.925+0000 7f2902eb0640 1 -- 172.21.15.28:0/1680113233 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f28ec06e0b0 msgr2=0x7f28ec070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:42.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.925+0000 7f2902eb0640 1 --2- 172.21.15.28:0/1680113233 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f28ec06e0b0 0x7f28ec070560 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f28fc10cda0 tx=0x7f28e800b040 comp rx=0 tx=0).stop 2024-07-03T15:49:42.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.925+0000 7f2902eb0640 1 -- 172.21.15.28:0/1680113233 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28fc0fc360 msgr2=0x7f28fc10cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:42.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.925+0000 7f2902eb0640 1 --2- 172.21.15.28:0/1680113233 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28fc0fc360 0x7f28fc10cff0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f28f00048a0 tx=0x7f28f0009da0 comp rx=0 tx=0).stop 2024-07-03T15:49:42.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.925+0000 7f2902eb0640 1 -- 172.21.15.28:0/1680113233 shutdown_connections 2024-07-03T15:49:42.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.925+0000 7f2902eb0640 1 --2- 172.21.15.28:0/1680113233 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f28ec06e0b0 0x7f28ec070560 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:42.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.925+0000 7f2902eb0640 1 --2- 172.21.15.28:0/1680113233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28fc0fd630 0x7f28fc10d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:42.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.925+0000 7f2902eb0640 1 --2- 172.21.15.28:0/1680113233 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28fc0fc360 0x7f28fc10cff0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:42.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.925+0000 7f2902eb0640 1 -- 172.21.15.28:0/1680113233 >> 172.21.15.28:0/1680113233 conn(0x7f28fc0f7b40 msgr2=0x7f28fc0f9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:49:42.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.925+0000 7f2902eb0640 1 -- 172.21.15.28:0/1680113233 shutdown_connections 2024-07-03T15:49:42.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:42.925+0000 7f2902eb0640 1 -- 172.21.15.28:0/1680113233 wait complete. 2024-07-03T15:49:43.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.007+0000 7f9ead523640 1 -- 172.21.15.28:0/2386442394 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9ea80fd600 msgr2=0x7f9ea80fda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:43.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.007+0000 7f9ead523640 1 --2- 172.21.15.28:0/2386442394 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9ea80fd600 0x7f9ea80fda70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9e900048d0 tx=0x7f9e90009da0 comp rx=0 tx=0).stop 2024-07-03T15:49:43.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.007+0000 7f9ead523640 1 -- 172.21.15.28:0/2386442394 shutdown_connections 2024-07-03T15:49:43.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.007+0000 7f9ead523640 1 --2- 172.21.15.28:0/2386442394 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9ea80fd600 0x7f9ea80fda70 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:43.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.007+0000 7f9ead523640 1 --2- 172.21.15.28:0/2386442394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ea80fc330 0x7f9ea80fc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:43.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.007+0000 7f9ead523640 1 -- 172.21.15.28:0/2386442394 >> 172.21.15.28:0/2386442394 conn(0x7f9ea80f7af0 msgr2=0x7f9ea80f9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:49:43.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.007+0000 7f9ead523640 1 -- 172.21.15.28:0/2386442394 shutdown_connections 2024-07-03T15:49:43.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.007+0000 7f9ead523640 1 -- 172.21.15.28:0/2386442394 wait complete. 2024-07-03T15:49:43.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.008+0000 7f9ead523640 1 Processor -- start 2024-07-03T15:49:43.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.008+0000 7f9ead523640 1 -- start start 2024-07-03T15:49:43.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.008+0000 7f9ead523640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ea80fc330 0x7f9ea8071330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:49:43.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.008+0000 7f9ead523640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9ea80fd600 0x7f9ea806f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:49:43.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.008+0000 7f9ead523640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea806fdb0 con 0x7f9ea80fd600 2024-07-03T15:49:43.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.008+0000 7f9ead523640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea806fef0 con 0x7f9ea80fc330 2024-07-03T15:49:43.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.008+0000 7f9ea67fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9ea80fd600 0x7f9ea806f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:49:43.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.009+0000 7f9ea67fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9ea80fd600 0x7f9ea806f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37478/0 (socket says 172.21.15.28:37478) 2024-07-03T15:49:43.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.009+0000 7f9ea67fc640 1 -- 172.21.15.28:0/944424481 learned_addr learned my addr 172.21.15.28:0/944424481 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:49:43.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.009+0000 7f9ea6ffd640 1 --2- 172.21.15.28:0/944424481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ea80fc330 0x7f9ea8071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:49:43.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.009+0000 7f9ea67fc640 1 -- 172.21.15.28:0/944424481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ea80fc330 msgr2=0x7f9ea8071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:43.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.009+0000 7f9ea67fc640 1 --2- 172.21.15.28:0/944424481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ea80fc330 0x7f9ea8071330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:43.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.009+0000 7f9ea67fc640 1 -- 172.21.15.28:0/944424481 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e90005040 con 0x7f9ea80fd600 2024-07-03T15:49:43.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.009+0000 7f9ea67fc640 1 --2- 172.21.15.28:0/944424481 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9ea80fd600 0x7f9ea806f870 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9e900048a0 tx=0x7f9e90009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:49:43.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.009+0000 7f9e9f7fe640 1 -- 172.21.15.28:0/944424481 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e90038070 con 0x7f9ea80fd600 2024-07-03T15:49:43.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.009+0000 7f9e9f7fe640 1 -- 172.21.15.28:0/944424481 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9e90032970 con 0x7f9ea80fd600 2024-07-03T15:49:43.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.009+0000 7f9ead523640 1 -- 172.21.15.28:0/944424481 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ea8070170 con 0x7f9ea80fd600 2024-07-03T15:49:43.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.009+0000 7f9e9f7fe640 1 -- 172.21.15.28:0/944424481 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e9003d9a0 con 0x7f9ea80fd600 2024-07-03T15:49:43.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.009+0000 7f9ead523640 1 -- 172.21.15.28:0/944424481 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea80706b0 con 0x7f9ea80fd600 2024-07-03T15:49:43.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.010+0000 7f9ead523640 1 -- 172.21.15.28:0/944424481 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ea80fb790 con 0x7f9ea80fd600 2024-07-03T15:49:43.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.011+0000 7f9e9f7fe640 1 -- 172.21.15.28:0/944424481 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9e90030020 con 0x7f9ea80fd600 2024-07-03T15:49:43.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.012+0000 7f9e9f7fe640 1 --2- 172.21.15.28:0/944424481 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f9e9406e0b0 0x7f9e94070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:49:43.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.012+0000 7f9ea6ffd640 1 --2- 172.21.15.28:0/944424481 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f9e9406e0b0 0x7f9e94070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:49:43.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.012+0000 7f9e9f7fe640 1 -- 172.21.15.28:0/944424481 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f9e900ae1a0 con 0x7f9ea80fd600 2024-07-03T15:49:43.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.012+0000 7f9ea6ffd640 1 --2- 172.21.15.28:0/944424481 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f9e9406e0b0 0x7f9e94070560 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9ea80710e0 tx=0x7f9e98009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:49:43.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.016+0000 7f9e9f7fe640 1 -- 172.21.15.28:0/944424481 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f9e90077660 con 0x7f9ea80fd600 2024-07-03T15:49:43.099 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:42 smithi148 systemd[1]: Stopping Ceph osd.4 for a85e4be6-3951-11ef-bcab-c7b262605968... 2024-07-03T15:49:43.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.238+0000 7f9ead523640 1 -- 172.21.15.28:0/944424481 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9ea804b530 con 0x7f9ea80fd600 2024-07-03T15:49:43.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.239+0000 7f9e9f7fe640 1 -- 172.21.15.28:0/944424481 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1908 (secure 0 0 0) 0x7f9e9007ad20 con 0x7f9ea80fd600 2024-07-03T15:49:43.243 INFO:teuthology.orchestra.run.smithi028.stdout:e17 2024-07-03T15:49:43.243 INFO:teuthology.orchestra.run.smithi028.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-03T15:49:43.243 INFO:teuthology.orchestra.run.smithi028.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:49:43.243 INFO:teuthology.orchestra.run.smithi028.stdout:legacy client fscid: 1 2024-07-03T15:49:43.243 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:49:43.243 INFO:teuthology.orchestra.run.smithi028.stdout:Filesystem 'cephfs' (1) 2024-07-03T15:49:43.243 INFO:teuthology.orchestra.run.smithi028.stdout:fs_name cephfs 2024-07-03T15:49:43.243 INFO:teuthology.orchestra.run.smithi028.stdout:epoch 16 2024-07-03T15:49:43.243 INFO:teuthology.orchestra.run.smithi028.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-03T15:49:43.243 INFO:teuthology.orchestra.run.smithi028.stdout:created 2024-07-03T15:41:40.656298+0000 2024-07-03T15:49:43.243 INFO:teuthology.orchestra.run.smithi028.stdout:modified 2024-07-03T15:41:56.454984+0000 2024-07-03T15:49:43.244 INFO:teuthology.orchestra.run.smithi028.stdout:tableserver 0 2024-07-03T15:49:43.244 INFO:teuthology.orchestra.run.smithi028.stdout:root 0 2024-07-03T15:49:43.244 INFO:teuthology.orchestra.run.smithi028.stdout:session_timeout 60 2024-07-03T15:49:43.244 INFO:teuthology.orchestra.run.smithi028.stdout:session_autoclose 300 2024-07-03T15:49:43.244 INFO:teuthology.orchestra.run.smithi028.stdout:max_file_size 1099511627776 2024-07-03T15:49:43.244 INFO:teuthology.orchestra.run.smithi028.stdout:max_xattr_size 65536 2024-07-03T15:49:43.244 INFO:teuthology.orchestra.run.smithi028.stdout:required_client_features {} 2024-07-03T15:49:43.244 INFO:teuthology.orchestra.run.smithi028.stdout:last_failure 0 2024-07-03T15:49:43.244 INFO:teuthology.orchestra.run.smithi028.stdout:last_failure_osd_epoch 0 2024-07-03T15:49:43.244 INFO:teuthology.orchestra.run.smithi028.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:49:43.244 INFO:teuthology.orchestra.run.smithi028.stdout:max_mds 1 2024-07-03T15:49:43.244 INFO:teuthology.orchestra.run.smithi028.stdout:in 0 2024-07-03T15:49:43.244 INFO:teuthology.orchestra.run.smithi028.stdout:up {0=14450} 2024-07-03T15:49:43.245 INFO:teuthology.orchestra.run.smithi028.stdout:failed 2024-07-03T15:49:43.245 INFO:teuthology.orchestra.run.smithi028.stdout:damaged 2024-07-03T15:49:43.245 INFO:teuthology.orchestra.run.smithi028.stdout:stopped 2024-07-03T15:49:43.245 INFO:teuthology.orchestra.run.smithi028.stdout:data_pools [3] 2024-07-03T15:49:43.245 INFO:teuthology.orchestra.run.smithi028.stdout:metadata_pool 2 2024-07-03T15:49:43.245 INFO:teuthology.orchestra.run.smithi028.stdout:inline_data disabled 2024-07-03T15:49:43.245 INFO:teuthology.orchestra.run.smithi028.stdout:balancer 2024-07-03T15:49:43.245 INFO:teuthology.orchestra.run.smithi028.stdout:bal_rank_mask -1 2024-07-03T15:49:43.245 INFO:teuthology.orchestra.run.smithi028.stdout:standby_count_wanted 1 2024-07-03T15:49:43.245 INFO:teuthology.orchestra.run.smithi028.stdout:qdb_cluster leader: 0 members: 2024-07-03T15:49:43.245 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi028.rhpvve{0:14450} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.28:6826/1613936042,v1:172.21.15.28:6827/1613936042] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:49:43.245 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi028.wmbxew{0:14460} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.28:6828/1730336821,v1:172.21.15.28:6829/1730336821] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:49:43.245 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:49:43.245 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:49:43.246 INFO:teuthology.orchestra.run.smithi028.stdout:Standby daemons: 2024-07-03T15:49:43.246 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:49:43.246 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi148.szomfl{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/4171231858,v1:172.21.15.148:6827/4171231858] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:49:43.246 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi148.xgpuld{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/2347536398,v1:172.21.15.148:6825/2347536398] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:49:43.247 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 17 2024-07-03T15:49:43.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.241+0000 7f9ead523640 1 -- 172.21.15.28:0/944424481 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f9e9406e0b0 msgr2=0x7f9e94070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:43.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.241+0000 7f9ead523640 1 --2- 172.21.15.28:0/944424481 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f9e9406e0b0 0x7f9e94070560 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9ea80710e0 tx=0x7f9e98009040 comp rx=0 tx=0).stop 2024-07-03T15:49:43.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.241+0000 7f9ead523640 1 -- 172.21.15.28:0/944424481 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9ea80fd600 msgr2=0x7f9ea806f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:43.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.241+0000 7f9ead523640 1 --2- 172.21.15.28:0/944424481 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9ea80fd600 0x7f9ea806f870 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9e900048a0 tx=0x7f9e90009da0 comp rx=0 tx=0).stop 2024-07-03T15:49:43.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.241+0000 7f9ead523640 1 -- 172.21.15.28:0/944424481 shutdown_connections 2024-07-03T15:49:43.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.241+0000 7f9ead523640 1 --2- 172.21.15.28:0/944424481 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f9e9406e0b0 0x7f9e94070560 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:43.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.241+0000 7f9ead523640 1 --2- 172.21.15.28:0/944424481 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9ea80fd600 0x7f9ea806f870 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:43.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.241+0000 7f9ead523640 1 --2- 172.21.15.28:0/944424481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ea80fc330 0x7f9ea8071330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:43.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.241+0000 7f9ead523640 1 -- 172.21.15.28:0/944424481 >> 172.21.15.28:0/944424481 conn(0x7f9ea80f7af0 msgr2=0x7f9ea80f91e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:49:43.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.241+0000 7f9ead523640 1 -- 172.21.15.28:0/944424481 shutdown_connections 2024-07-03T15:49:43.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.241+0000 7f9ead523640 1 -- 172.21.15.28:0/944424481 wait complete. 2024-07-03T15:49:43.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.325+0000 7fc80c511640 1 -- 172.21.15.28:0/1122657382 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc8040fd600 msgr2=0x7fc8040fda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:43.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.325+0000 7fc80c511640 1 --2- 172.21.15.28:0/1122657382 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc8040fd600 0x7fc8040fda70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc7f40048d0 tx=0x7fc7f4009da0 comp rx=0 tx=0).stop 2024-07-03T15:49:43.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.325+0000 7fc80c511640 1 -- 172.21.15.28:0/1122657382 shutdown_connections 2024-07-03T15:49:43.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.325+0000 7fc80c511640 1 --2- 172.21.15.28:0/1122657382 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc8040fd600 0x7fc8040fda70 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:43.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.325+0000 7fc80c511640 1 --2- 172.21.15.28:0/1122657382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8040fc330 0x7fc8040fc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:43.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.325+0000 7fc80c511640 1 -- 172.21.15.28:0/1122657382 >> 172.21.15.28:0/1122657382 conn(0x7fc8040f7af0 msgr2=0x7fc8040f9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:49:43.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.325+0000 7fc80c511640 1 -- 172.21.15.28:0/1122657382 shutdown_connections 2024-07-03T15:49:43.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.325+0000 7fc80c511640 1 -- 172.21.15.28:0/1122657382 wait complete. 2024-07-03T15:49:43.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.325+0000 7fc80c511640 1 Processor -- start 2024-07-03T15:49:43.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.325+0000 7fc80c511640 1 -- start start 2024-07-03T15:49:43.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.326+0000 7fc80c511640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8040fc330 0x7fc80410cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:49:43.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.326+0000 7fc80c511640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc8040fd600 0x7fc80410d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:49:43.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.326+0000 7fc80c511640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80410dae0 con 0x7fc8040fd600 2024-07-03T15:49:43.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.326+0000 7fc80c511640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80410e300 con 0x7fc8040fc330 2024-07-03T15:49:43.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.326+0000 7fc809aac640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc8040fd600 0x7fc80410d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:49:43.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.326+0000 7fc809aac640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc8040fd600 0x7fc80410d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37504/0 (socket says 172.21.15.28:37504) 2024-07-03T15:49:43.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.326+0000 7fc809aac640 1 -- 172.21.15.28:0/2758849211 learned_addr learned my addr 172.21.15.28:0/2758849211 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:49:43.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.326+0000 7fc80a2ad640 1 --2- 172.21.15.28:0/2758849211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8040fc330 0x7fc80410cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:49:43.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.326+0000 7fc809aac640 1 -- 172.21.15.28:0/2758849211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8040fc330 msgr2=0x7fc80410cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:43.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.326+0000 7fc809aac640 1 --2- 172.21.15.28:0/2758849211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8040fc330 0x7fc80410cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:43.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.326+0000 7fc809aac640 1 -- 172.21.15.28:0/2758849211 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f4005040 con 0x7fc8040fd600 2024-07-03T15:49:43.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.326+0000 7fc809aac640 1 --2- 172.21.15.28:0/2758849211 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc8040fd600 0x7fc80410d490 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4003080 tx=0x7fc7f4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:49:43.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.326+0000 7fc7faffd640 1 -- 172.21.15.28:0/2758849211 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7f4038070 con 0x7fc8040fd600 2024-07-03T15:49:43.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.326+0000 7fc7faffd640 1 -- 172.21.15.28:0/2758849211 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc7f4032970 con 0x7fc8040fd600 2024-07-03T15:49:43.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.326+0000 7fc80c511640 1 -- 172.21.15.28:0/2758849211 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc80410e530 con 0x7fc8040fd600 2024-07-03T15:49:43.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.326+0000 7fc7faffd640 1 -- 172.21.15.28:0/2758849211 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7f4033900 con 0x7fc8040fd600 2024-07-03T15:49:43.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.327+0000 7fc80c511640 1 -- 172.21.15.28:0/2758849211 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc80410ea70 con 0x7fc8040fd600 2024-07-03T15:49:43.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.327+0000 7fc80c511640 1 -- 172.21.15.28:0/2758849211 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8040fb790 con 0x7fc8040fd600 2024-07-03T15:49:43.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.328+0000 7fc7faffd640 1 -- 172.21.15.28:0/2758849211 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc7f4030020 con 0x7fc8040fd600 2024-07-03T15:49:43.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.328+0000 7fc7faffd640 1 --2- 172.21.15.28:0/2758849211 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc7f006e0b0 0x7fc7f0070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:49:43.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.329+0000 7fc7faffd640 1 -- 172.21.15.28:0/2758849211 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fc7f40adb40 con 0x7fc8040fd600 2024-07-03T15:49:43.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.329+0000 7fc80a2ad640 1 --2- 172.21.15.28:0/2758849211 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc7f006e0b0 0x7fc7f0070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:49:43.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.329+0000 7fc80a2ad640 1 --2- 172.21.15.28:0/2758849211 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc7f006e0b0 0x7fc7f0070560 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc8000041d0 tx=0x7fc800009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:49:43.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.333+0000 7fc7faffd640 1 -- 172.21.15.28:0/2758849211 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc7f4076f40 con 0x7fc8040fd600 2024-07-03T15:49:43.482 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:43 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4[47919]: 2024-07-03T15:49:43.093+0000 7ff1cbe16640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-03T15:49:43.483 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:43 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4[47919]: 2024-07-03T15:49:43.093+0000 7ff1cbe16640 -1 osd.4 68 *** Got signal Terminated *** 2024-07-03T15:49:43.483 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:43 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4[47919]: 2024-07-03T15:49:43.093+0000 7ff1cbe16640 -1 osd.4 68 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-03T15:49:43.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.536+0000 7fc80c511640 1 -- 172.21.15.28:0/2758849211 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc80410edf0 con 0x7fc7f006e0b0 2024-07-03T15:49:43.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.537+0000 7fc7faffd640 1 -- 172.21.15.28:0/2758849211 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fc80410edf0 con 0x7fc7f006e0b0 2024-07-03T15:49:43.541 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-07-03T15:49:43.541 INFO:teuthology.orchestra.run.smithi028.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2", 2024-07-03T15:49:43.541 INFO:teuthology.orchestra.run.smithi028.stdout: "in_progress": true, 2024-07-03T15:49:43.541 INFO:teuthology.orchestra.run.smithi028.stdout: "which": "Upgrading all daemon types on all hosts", 2024-07-03T15:49:43.541 INFO:teuthology.orchestra.run.smithi028.stdout: "services_complete": [ 2024-07-03T15:49:43.541 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr", 2024-07-03T15:49:43.541 INFO:teuthology.orchestra.run.smithi028.stdout: "crash", 2024-07-03T15:49:43.541 INFO:teuthology.orchestra.run.smithi028.stdout: "mon" 2024-07-03T15:49:43.541 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-07-03T15:49:43.541 INFO:teuthology.orchestra.run.smithi028.stdout: "progress": "10/21 daemons upgraded", 2024-07-03T15:49:43.541 INFO:teuthology.orchestra.run.smithi028.stdout: "message": "Currently upgrading osd daemons", 2024-07-03T15:49:43.542 INFO:teuthology.orchestra.run.smithi028.stdout: "is_paused": false 2024-07-03T15:49:43.542 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-07-03T15:49:43.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.540+0000 7fc80c511640 1 -- 172.21.15.28:0/2758849211 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc7f006e0b0 msgr2=0x7fc7f0070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:43.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.540+0000 7fc80c511640 1 --2- 172.21.15.28:0/2758849211 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc7f006e0b0 0x7fc7f0070560 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc8000041d0 tx=0x7fc800009040 comp rx=0 tx=0).stop 2024-07-03T15:49:43.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.540+0000 7fc80c511640 1 -- 172.21.15.28:0/2758849211 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc8040fd600 msgr2=0x7fc80410d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:43.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.540+0000 7fc80c511640 1 --2- 172.21.15.28:0/2758849211 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc8040fd600 0x7fc80410d490 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4003080 tx=0x7fc7f4009da0 comp rx=0 tx=0).stop 2024-07-03T15:49:43.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.540+0000 7fc80c511640 1 -- 172.21.15.28:0/2758849211 shutdown_connections 2024-07-03T15:49:43.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.540+0000 7fc80c511640 1 --2- 172.21.15.28:0/2758849211 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc7f006e0b0 0x7fc7f0070560 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:43.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.540+0000 7fc80c511640 1 --2- 172.21.15.28:0/2758849211 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc8040fd600 0x7fc80410d490 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:43.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.540+0000 7fc80c511640 1 --2- 172.21.15.28:0/2758849211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8040fc330 0x7fc80410cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:43.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.540+0000 7fc80c511640 1 -- 172.21.15.28:0/2758849211 >> 172.21.15.28:0/2758849211 conn(0x7fc8040f7af0 msgr2=0x7fc8040f91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:49:43.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.540+0000 7fc80c511640 1 -- 172.21.15.28:0/2758849211 shutdown_connections 2024-07-03T15:49:43.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.540+0000 7fc80c511640 1 -- 172.21.15.28:0/2758849211 wait complete. 2024-07-03T15:49:43.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.622+0000 7f704d20c640 1 -- 172.21.15.28:0/620639444 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70480fb290 msgr2=0x7f70480fb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:43.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.622+0000 7f704d20c640 1 --2- 172.21.15.28:0/620639444 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70480fb290 0x7f70480fb6a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f70380048d0 tx=0x7f7038009da0 comp rx=0 tx=0).stop 2024-07-03T15:49:43.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.622+0000 7f704d20c640 1 -- 172.21.15.28:0/620639444 shutdown_connections 2024-07-03T15:49:43.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.622+0000 7f704d20c640 1 --2- 172.21.15.28:0/620639444 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70480fc560 0x7f70480fc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:43.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.622+0000 7f704d20c640 1 --2- 172.21.15.28:0/620639444 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70480fb290 0x7f70480fb6a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:43.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.622+0000 7f704d20c640 1 -- 172.21.15.28:0/620639444 >> 172.21.15.28:0/620639444 conn(0x7f70480f6a50 msgr2=0x7f70480f8e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:49:43.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.622+0000 7f704d20c640 1 -- 172.21.15.28:0/620639444 shutdown_connections 2024-07-03T15:49:43.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.622+0000 7f704d20c640 1 -- 172.21.15.28:0/620639444 wait complete. 2024-07-03T15:49:43.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.622+0000 7f704d20c640 1 Processor -- start 2024-07-03T15:49:43.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.623+0000 7f704d20c640 1 -- start start 2024-07-03T15:49:43.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.623+0000 7f704d20c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70480fb290 0x7f704810f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:49:43.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.623+0000 7f704d20c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70480fc560 0x7f704810f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:49:43.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.623+0000 7f704d20c640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704810fd80 con 0x7f70480fb290 2024-07-03T15:49:43.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.623+0000 7f704d20c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70481105a0 con 0x7f70480fc560 2024-07-03T15:49:43.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.623+0000 7f7046d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70480fb290 0x7f704810f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:49:43.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.623+0000 7f7046d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70480fb290 0x7f704810f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37528/0 (socket says 172.21.15.28:37528) 2024-07-03T15:49:43.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.623+0000 7f7046d9d640 1 -- 172.21.15.28:0/488975785 learned_addr learned my addr 172.21.15.28:0/488975785 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:49:43.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.623+0000 7f703ffff640 1 --2- 172.21.15.28:0/488975785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70480fc560 0x7f704810f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:49:43.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.623+0000 7f7046d9d640 1 -- 172.21.15.28:0/488975785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70480fc560 msgr2=0x7f704810f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:43.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.623+0000 7f7046d9d640 1 --2- 172.21.15.28:0/488975785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70480fc560 0x7f704810f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:43.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.623+0000 7f7046d9d640 1 -- 172.21.15.28:0/488975785 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7038005040 con 0x7f70480fb290 2024-07-03T15:49:43.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.623+0000 7f7046d9d640 1 --2- 172.21.15.28:0/488975785 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70480fb290 0x7f704810f1f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7038009f80 tx=0x7f7038009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:49:43.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.623+0000 7f703f7fe640 1 -- 172.21.15.28:0/488975785 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7038038070 con 0x7f70480fb290 2024-07-03T15:49:43.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.624+0000 7f703f7fe640 1 -- 172.21.15.28:0/488975785 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7038008ee0 con 0x7f70480fb290 2024-07-03T15:49:43.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.624+0000 7f704d20c640 1 -- 172.21.15.28:0/488975785 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70481107d0 con 0x7f70480fb290 2024-07-03T15:49:43.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.624+0000 7f703f7fe640 1 -- 172.21.15.28:0/488975785 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7038033890 con 0x7f70480fb290 2024-07-03T15:49:43.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.624+0000 7f704d20c640 1 -- 172.21.15.28:0/488975785 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7048110d10 con 0x7f70480fb290 2024-07-03T15:49:43.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.625+0000 7f704d20c640 1 -- 172.21.15.28:0/488975785 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70480fa6f0 con 0x7f70480fb290 2024-07-03T15:49:43.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.625+0000 7f703f7fe640 1 -- 172.21.15.28:0/488975785 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7038030020 con 0x7f70480fb290 2024-07-03T15:49:43.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.625+0000 7f703f7fe640 1 --2- 172.21.15.28:0/488975785 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f703406e0b0 0x7f7034070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:49:43.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.626+0000 7f703f7fe640 1 -- 172.21.15.28:0/488975785 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f70380ad820 con 0x7f70480fb290 2024-07-03T15:49:43.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.626+0000 7f703ffff640 1 --2- 172.21.15.28:0/488975785 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f703406e0b0 0x7f7034070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:49:43.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.626+0000 7f703ffff640 1 --2- 172.21.15.28:0/488975785 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f703406e0b0 0x7f7034070560 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f704810efa0 tx=0x7f703000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:49:43.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.630+0000 7f703f7fe640 1 -- 172.21.15.28:0/488975785 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f7038076cd0 con 0x7f70480fb290 2024-07-03T15:49:43.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.894+0000 7f704d20c640 1 -- 172.21.15.28:0/488975785 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f704804b530 con 0x7f70480fb290 2024-07-03T15:49:43.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.894+0000 7f703f7fe640 1 -- 172.21.15.28:0/488975785 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+100 (secure 0 0 0) 0x7f703807a390 con 0x7f70480fb290 2024-07-03T15:49:43.898 INFO:teuthology.orchestra.run.smithi028.stdout:HEALTH_WARN 1 osds down 2024-07-03T15:49:43.898 INFO:teuthology.orchestra.run.smithi028.stdout:[WRN] OSD_DOWN: 1 osds down 2024-07-03T15:49:43.898 INFO:teuthology.orchestra.run.smithi028.stdout: osd.4 (root=default,host=smithi148) is down 2024-07-03T15:49:43.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.896+0000 7f704d20c640 1 -- 172.21.15.28:0/488975785 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f703406e0b0 msgr2=0x7f7034070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:43.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.896+0000 7f704d20c640 1 --2- 172.21.15.28:0/488975785 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f703406e0b0 0x7f7034070560 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f704810efa0 tx=0x7f703000b040 comp rx=0 tx=0).stop 2024-07-03T15:49:43.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.896+0000 7f704d20c640 1 -- 172.21.15.28:0/488975785 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70480fb290 msgr2=0x7f704810f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:49:43.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.896+0000 7f704d20c640 1 --2- 172.21.15.28:0/488975785 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70480fb290 0x7f704810f1f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7038009f80 tx=0x7f7038009da0 comp rx=0 tx=0).stop 2024-07-03T15:49:43.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.896+0000 7f704d20c640 1 -- 172.21.15.28:0/488975785 shutdown_connections 2024-07-03T15:49:43.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.896+0000 7f704d20c640 1 --2- 172.21.15.28:0/488975785 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f703406e0b0 0x7f7034070560 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:43.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.896+0000 7f704d20c640 1 --2- 172.21.15.28:0/488975785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70480fc560 0x7f704810f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:43.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.896+0000 7f704d20c640 1 --2- 172.21.15.28:0/488975785 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70480fb290 0x7f704810f1f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:49:43.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.896+0000 7f704d20c640 1 -- 172.21.15.28:0/488975785 >> 172.21.15.28:0/488975785 conn(0x7f70480f6a50 msgr2=0x7f7048065e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:49:43.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.896+0000 7f704d20c640 1 -- 172.21.15.28:0/488975785 shutdown_connections 2024-07-03T15:49:43.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:49:43.896+0000 7f704d20c640 1 -- 172.21.15.28:0/488975785 wait complete. 2024-07-03T15:49:44.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:43 smithi028 ceph-mon[92858]: from='client.34312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:49:44.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:43 smithi028 ceph-mon[92858]: pgmap v123: 65 pgs: 65 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:49:44.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:43 smithi028 ceph-mon[92858]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:49:44.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:43 smithi028 ceph-mon[92858]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:49:44.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:43 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/1680113233' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:49:44.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:43 smithi028 ceph-mon[92858]: osd.4 marked itself down and dead 2024-07-03T15:49:44.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:43 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/944424481' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-03T15:49:44.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:43 smithi148 ceph-mon[80490]: from='client.34312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:49:44.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:43 smithi148 ceph-mon[80490]: pgmap v123: 65 pgs: 65 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:49:44.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:43 smithi148 ceph-mon[80490]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:49:44.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:43 smithi148 ceph-mon[80490]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:49:44.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:43 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/1680113233' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:49:44.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:43 smithi148 ceph-mon[80490]: osd.4 marked itself down and dead 2024-07-03T15:49:44.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:43 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/944424481' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-03T15:49:44.691 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:44 smithi148 podman[94782]: 2024-07-03 15:49:44.415362877 +0000 UTC m=+1.442228625 container died 54f7c26c77ca0a79066754c83536f179afdaf94080c376b5cc3e0275e62b62a4 (image=quay.ceph.io/ceph-ci/ceph@sha256:0b7651540cdb24c1b97466c79c298d54c7794c5f53d5d6b6c5d76a2d3b130d85, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4, GIT_BRANCH=HEAD, org.label-schema.build-date=20240625, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=quincy-d4b063a, GIT_CLEAN=True, ceph=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-07-03T15:49:44.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:44 smithi148 ceph-mon[80490]: from='client.34332 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:49:44.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:44 smithi148 ceph-mon[80490]: Health check failed: 1 osds down (OSD_DOWN) 2024-07-03T15:49:44.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:44 smithi148 ceph-mon[80490]: osdmap e69: 6 total, 5 up, 6 in 2024-07-03T15:49:44.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:44 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/488975785' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-03T15:49:44.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:44 smithi148 ceph-mon[80490]: osdmap e70: 6 total, 5 up, 6 in 2024-07-03T15:49:44.983 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:44 smithi148 podman[94782]: 2024-07-03 15:49:44.778717407 +0000 UTC m=+1.805583150 container cleanup 54f7c26c77ca0a79066754c83536f179afdaf94080c376b5cc3e0275e62b62a4 (image=quay.ceph.io/ceph-ci/ceph@sha256:0b7651540cdb24c1b97466c79c298d54c7794c5f53d5d6b6c5d76a2d3b130d85, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240625, GIT_BRANCH=HEAD, RELEASE=quincy-d4b063a) 2024-07-03T15:49:44.983 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:44 smithi148 bash[94782]: ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4 2024-07-03T15:49:45.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:44 smithi028 ceph-mon[92858]: from='client.34332 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:49:45.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:44 smithi028 ceph-mon[92858]: Health check failed: 1 osds down (OSD_DOWN) 2024-07-03T15:49:45.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:44 smithi028 ceph-mon[92858]: osdmap e69: 6 total, 5 up, 6 in 2024-07-03T15:49:45.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:44 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/488975785' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-03T15:49:45.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:44 smithi028 ceph-mon[92858]: osdmap e70: 6 total, 5 up, 6 in 2024-07-03T15:49:45.417 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:45 smithi148 podman[94799]: 2024-07-03 15:49:45.080114204 +0000 UTC m=+0.657573246 container remove 54f7c26c77ca0a79066754c83536f179afdaf94080c376b5cc3e0275e62b62a4 (image=quay.ceph.io/ceph-ci/ceph@sha256:0b7651540cdb24c1b97466c79c298d54c7794c5f53d5d6b6c5d76a2d3b130d85, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.build-date=20240625, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=quincy-d4b063a, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-07-03T15:49:45.711 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:45 smithi148 podman[94890]: 2024-07-03 15:49:45.414447189 +0000 UTC m=+0.112578000 container create aff8fb9e3e4338a214888b0b7117a0fec444a76c592c34a80e370c714e6cbcd6 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4-deactivate, io.buildah.version=1.36.0, ceph=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-release-c00506d, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240625, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-07-03T15:49:45.711 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:45 smithi148 podman[94890]: 2024-07-03 15:49:45.31954485 +0000 UTC m=+0.017675624 image pull 9d49405603e8445bc510362cdeb05f243619be76b0f3897dd4c152b00cffe45f quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2 2024-07-03T15:49:45.711 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:45 smithi148 podman[94890]: 2024-07-03 15:49:45.633867042 +0000 UTC m=+0.331997820 container init aff8fb9e3e4338a214888b0b7117a0fec444a76c592c34a80e370c714e6cbcd6 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4-deactivate, ceph=True, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240625, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-release-c00506d, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-07-03T15:49:45.711 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:45 smithi148 podman[94890]: 2024-07-03 15:49:45.637303151 +0000 UTC m=+0.335433918 container start aff8fb9e3e4338a214888b0b7117a0fec444a76c592c34a80e370c714e6cbcd6 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240625, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=squid-release-c00506d, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-03T15:49:45.712 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:45 smithi148 podman[94890]: 2024-07-03 15:49:45.708328656 +0000 UTC m=+0.406459437 container attach aff8fb9e3e4338a214888b0b7117a0fec444a76c592c34a80e370c714e6cbcd6 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240625, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-release-c00506d, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-07-03T15:49:45.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:45 smithi148 ceph-mon[80490]: pgmap v125: 65 pgs: 8 peering, 5 stale+active+clean, 52 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-07-03T15:49:45.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:45 smithi148 ceph-mon[80490]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-07-03T15:49:45.983 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:45 smithi148 podman[94890]: 2024-07-03 15:49:45.893562343 +0000 UTC m=+0.591693109 container died aff8fb9e3e4338a214888b0b7117a0fec444a76c592c34a80e370c714e6cbcd6 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240625, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-release-c00506d, io.buildah.version=1.36.0, ceph=True, GIT_CLEAN=True) 2024-07-03T15:49:46.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:45 smithi028 ceph-mon[92858]: pgmap v125: 65 pgs: 8 peering, 5 stale+active+clean, 52 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-07-03T15:49:46.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:45 smithi028 ceph-mon[92858]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-07-03T15:49:46.933 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:46 smithi148 podman[94912]: 2024-07-03 15:49:46.495397594 +0000 UTC m=+0.592520760 container remove aff8fb9e3e4338a214888b0b7117a0fec444a76c592c34a80e370c714e6cbcd6 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-release-c00506d, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240625) 2024-07-03T15:49:46.933 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:46 smithi148 systemd[1]: ceph-a85e4be6-3951-11ef-bcab-c7b262605968@osd.4.service: Deactivated successfully. 2024-07-03T15:49:46.933 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:46 smithi148 systemd[1]: Stopped Ceph osd.4 for a85e4be6-3951-11ef-bcab-c7b262605968. 2024-07-03T15:49:46.933 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:46 smithi148 systemd[1]: ceph-a85e4be6-3951-11ef-bcab-c7b262605968@osd.4.service: Consumed 59.296s CPU time. 2024-07-03T15:49:46.933 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:46 smithi148 systemd[1]: Starting Ceph osd.4 for a85e4be6-3951-11ef-bcab-c7b262605968... 2024-07-03T15:49:47.202 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:46 smithi148 ceph-mon[80490]: pgmap v127: 65 pgs: 5 active+undersized, 8 peering, 3 stale+active+clean, 3 active+undersized+degraded, 46 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 8/219 objects degraded (3.653%) 2024-07-03T15:49:47.203 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:46 smithi148 ceph-mon[80490]: Health check failed: Degraded data redundancy: 8/219 objects degraded (3.653%), 3 pgs degraded (PG_DEGRADED) 2024-07-03T15:49:47.203 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:46 smithi148 podman[95010]: 2024-07-03 15:49:46.84139396 +0000 UTC m=+0.017304004 image pull 9d49405603e8445bc510362cdeb05f243619be76b0f3897dd4c152b00cffe45f quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2 2024-07-03T15:49:47.203 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:46 smithi148 podman[95010]: 2024-07-03 15:49:46.946430766 +0000 UTC m=+0.122340809 container create a072fd01d3d322b0733ecd2eb6e6057a793605c121c84daf592f4f964b0b87a7 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4-activate, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-release-c00506d, org.label-schema.license=GPLv2, org.label-schema.build-date=20240625, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-03T15:49:47.203 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:47 smithi148 podman[95010]: 2024-07-03 15:49:47.199362762 +0000 UTC m=+0.375272811 container init a072fd01d3d322b0733ecd2eb6e6057a793605c121c84daf592f4f964b0b87a7 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4-activate, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240625, org.label-schema.vendor=CentOS, RELEASE=squid-release-c00506d, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-07-03T15:49:47.273 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:46 smithi028 ceph-mon[92858]: pgmap v127: 65 pgs: 5 active+undersized, 8 peering, 3 stale+active+clean, 3 active+undersized+degraded, 46 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 8/219 objects degraded (3.653%) 2024-07-03T15:49:47.274 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:46 smithi028 ceph-mon[92858]: Health check failed: Degraded data redundancy: 8/219 objects degraded (3.653%), 3 pgs degraded (PG_DEGRADED) 2024-07-03T15:49:47.482 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:47 smithi148 podman[95010]: 2024-07-03 15:49:47.202980779 +0000 UTC m=+0.378890823 container start a072fd01d3d322b0733ecd2eb6e6057a793605c121c84daf592f4f964b0b87a7 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4-activate, RELEASE=squid-release-c00506d, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240625, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-03T15:49:47.482 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:47 smithi148 podman[95010]: 2024-07-03 15:49:47.290557343 +0000 UTC m=+0.466467391 container attach a072fd01d3d322b0733ecd2eb6e6057a793605c121c84daf592f4f964b0b87a7 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4-activate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240625, GIT_CLEAN=True, RELEASE=squid-release-c00506d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0) 2024-07-03T15:49:47.922 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:47 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4-activate[95026]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-07-03T15:49:47.922 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:47 smithi148 bash[95010]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-07-03T15:49:47.922 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:47 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4-activate[95026]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-07-03T15:49:47.922 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:47 smithi148 bash[95010]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-07-03T15:49:47.922 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:47 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4-activate[95026]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-07-03T15:49:47.922 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:47 smithi148 bash[95010]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-07-03T15:49:47.922 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:47 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4-activate[95026]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-07-03T15:49:47.922 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:47 smithi148 bash[95010]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-07-03T15:49:47.923 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:47 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4-activate[95026]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-07-03T15:49:47.923 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:47 smithi148 bash[95010]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-07-03T15:49:47.923 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:47 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4-activate[95026]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-07-03T15:49:47.923 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:47 smithi148 bash[95010]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-07-03T15:49:47.923 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:47 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4-activate[95026]: --> ceph-volume raw activate successful for osd ID: 4 2024-07-03T15:49:47.923 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:47 smithi148 bash[95010]: --> ceph-volume raw activate successful for osd ID: 4 2024-07-03T15:49:47.923 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:47 smithi148 podman[95010]: 2024-07-03 15:49:47.624035159 +0000 UTC m=+0.799945210 container died a072fd01d3d322b0733ecd2eb6e6057a793605c121c84daf592f4f964b0b87a7 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4-activate, org.label-schema.schema-version=1.0, RELEASE=squid-release-c00506d, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.build-date=20240625, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-07-03T15:49:48.482 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:48 smithi148 podman[95147]: 2024-07-03 15:49:48.194424251 +0000 UTC m=+0.564426957 container remove a072fd01d3d322b0733ecd2eb6e6057a793605c121c84daf592f4f964b0b87a7 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4-activate, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-release-c00506d, org.label-schema.build-date=20240625, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-03T15:49:48.482 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:48 smithi148 podman[95208]: 2024-07-03 15:49:48.306352981 +0000 UTC m=+0.017129097 image pull 9d49405603e8445bc510362cdeb05f243619be76b0f3897dd4c152b00cffe45f quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2 2024-07-03T15:49:48.482 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:48 smithi148 podman[95208]: 2024-07-03 15:49:48.403932284 +0000 UTC m=+0.114708439 container create 546343d0f31f6ddd8b4fe56c03a6832414698b9328b6824154bffa3d1af74df3 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240625, RELEASE=squid-release-c00506d) 2024-07-03T15:49:48.982 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:48 smithi148 podman[95208]: 2024-07-03 15:49:48.656337173 +0000 UTC m=+0.367113296 container init 546343d0f31f6ddd8b4fe56c03a6832414698b9328b6824154bffa3d1af74df3 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-release-c00506d, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240625, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-07-03T15:49:48.982 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:48 smithi148 podman[95208]: 2024-07-03 15:49:48.659545954 +0000 UTC m=+0.370322069 container start 546343d0f31f6ddd8b4fe56c03a6832414698b9328b6824154bffa3d1af74df3 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240625, RELEASE=squid-release-c00506d, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-07-03T15:49:48.983 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:48 smithi148 ceph-osd[95228]: -- 172.21.15.148:0/1291932008 <== mon.1 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d07f0041e0 con 0x55d07ef9c000 2024-07-03T15:49:48.983 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:48 smithi148 bash[95208]: 546343d0f31f6ddd8b4fe56c03a6832414698b9328b6824154bffa3d1af74df3 2024-07-03T15:49:48.983 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:48 smithi148 systemd[1]: Started Ceph osd.4 for a85e4be6-3951-11ef-bcab-c7b262605968. 2024-07-03T15:49:49.482 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:49 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4[95224]: 2024-07-03T15:49:49.204+0000 7fed7ce22640 -1 Falling back to public interface 2024-07-03T15:49:50.099 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:49 smithi148 ceph-mon[80490]: pgmap v128: 65 pgs: 8 active+undersized, 8 peering, 2 stale+active+clean, 5 active+undersized+degraded, 42 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 12/219 objects degraded (5.479%) 2024-07-03T15:49:50.099 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:50.099 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:50.099 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:49:50.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:49 smithi028 ceph-mon[92858]: pgmap v128: 65 pgs: 8 active+undersized, 8 peering, 2 stale+active+clean, 5 active+undersized+degraded, 42 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 12/219 objects degraded (5.479%) 2024-07-03T15:49:50.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:50.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:50.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:49:51.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:50 smithi028 ceph-mon[92858]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-07-03T15:49:51.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:50 smithi148 ceph-mon[80490]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-07-03T15:49:52.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:51 smithi028 ceph-mon[92858]: pgmap v129: 65 pgs: 20 active+undersized, 14 active+undersized+degraded, 31 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 895 B/s rd, 2 op/s; 34/219 objects degraded (15.525%) 2024-07-03T15:49:52.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:51 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:52.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:51 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:52.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:51 smithi148 ceph-mon[80490]: pgmap v129: 65 pgs: 20 active+undersized, 14 active+undersized+degraded, 31 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 895 B/s rd, 2 op/s; 34/219 objects degraded (15.525%) 2024-07-03T15:49:52.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:51 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:52.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:51 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:53.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:53 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:53.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:53 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:53.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:53 smithi028 ceph-mon[92858]: pgmap v130: 65 pgs: 20 active+undersized, 14 active+undersized+degraded, 31 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 248 B/s rd, 0 op/s; 34/219 objects degraded (15.525%) 2024-07-03T15:49:53.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:53 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:53.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:53 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:49:53.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:53 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:53.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:53 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:53.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:53 smithi148 ceph-mon[80490]: pgmap v130: 65 pgs: 20 active+undersized, 14 active+undersized+degraded, 31 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 248 B/s rd, 0 op/s; 34/219 objects degraded (15.525%) 2024-07-03T15:49:53.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:53 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:53.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:53 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:49:54.482 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:54 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4[95224]: 2024-07-03T15:49:54.300+0000 7fed7ce22640 -1 osd.4 68 log_to_monitors true 2024-07-03T15:49:55.309 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:49:55 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4[95224]: 2024-07-03T15:49:55.180+0000 7fed74bbc640 -1 osd.4 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-03T15:49:55.310 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:55 smithi148 ceph-mon[80490]: pgmap v131: 65 pgs: 20 active+undersized, 14 active+undersized+degraded, 31 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 716 B/s rd, 1 op/s; 34/219 objects degraded (15.525%) 2024-07-03T15:49:55.310 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:55 smithi148 ceph-mon[80490]: from='osd.4 [v2:172.21.15.148:6808/3538420843,v1:172.21.15.148:6809/3538420843]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-03T15:49:55.310 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:55 smithi148 ceph-mon[80490]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-03T15:49:55.478 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:55 smithi028 ceph-mon[92858]: pgmap v131: 65 pgs: 20 active+undersized, 14 active+undersized+degraded, 31 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 716 B/s rd, 1 op/s; 34/219 objects degraded (15.525%) 2024-07-03T15:49:55.478 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:55 smithi028 ceph-mon[92858]: from='osd.4 [v2:172.21.15.148:6808/3538420843,v1:172.21.15.148:6809/3538420843]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-03T15:49:55.478 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:55 smithi028 ceph-mon[92858]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-03T15:49:56.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:56 smithi148 ceph-mon[80490]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-07-03T15:49:56.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:56 smithi148 ceph-mon[80490]: osdmap e71: 6 total, 5 up, 6 in 2024-07-03T15:49:56.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:56 smithi148 ceph-mon[80490]: from='osd.4 [v2:172.21.15.148:6808/3538420843,v1:172.21.15.148:6809/3538420843]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-07-03T15:49:56.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:56 smithi148 ceph-mon[80490]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-07-03T15:49:56.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:56 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:56.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:56 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:56.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:56 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:49:56.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:56 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:49:56.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:56 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:56.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:56 smithi148 ceph-mon[80490]: Health check update: Degraded data redundancy: 34/219 objects degraded (15.525%), 14 pgs degraded (PG_DEGRADED) 2024-07-03T15:49:56.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:56 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:49:56.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:56 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:49:56.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:56 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:49:56.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:56 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:49:56.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:56 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-07-03T15:49:56.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:56 smithi148 ceph-mon[80490]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-07-03T15:49:56.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:56 smithi148 ceph-mon[80490]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-07-03T15:49:56.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:56 smithi028 ceph-mon[92858]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-07-03T15:49:56.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:56 smithi028 ceph-mon[92858]: osdmap e71: 6 total, 5 up, 6 in 2024-07-03T15:49:56.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:56 smithi028 ceph-mon[92858]: from='osd.4 [v2:172.21.15.148:6808/3538420843,v1:172.21.15.148:6809/3538420843]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-07-03T15:49:56.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:56 smithi028 ceph-mon[92858]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-07-03T15:49:56.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:56 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:56.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:56 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:56.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:56 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:49:56.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:56 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:49:56.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:56 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:49:56.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:56 smithi028 ceph-mon[92858]: Health check update: Degraded data redundancy: 34/219 objects degraded (15.525%), 14 pgs degraded (PG_DEGRADED) 2024-07-03T15:49:56.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:56 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:49:56.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:56 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:49:56.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:56 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:49:56.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:56 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:49:56.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:56 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-07-03T15:49:56.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:56 smithi028 ceph-mon[92858]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-07-03T15:49:56.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:56 smithi028 ceph-mon[92858]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-07-03T15:49:57.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:57 smithi148 ceph-mon[80490]: pgmap v133: 65 pgs: 20 active+undersized, 14 active+undersized+degraded, 31 active+clean; 193 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 716 B/s rd, 1 op/s; 34/219 objects degraded (15.525%) 2024-07-03T15:49:57.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:57 smithi148 ceph-mon[80490]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-07-03T15:49:57.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:57 smithi148 ceph-mon[80490]: osd.4 [v2:172.21.15.148:6808/3538420843,v1:172.21.15.148:6809/3538420843] boot 2024-07-03T15:49:57.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:57 smithi148 ceph-mon[80490]: osdmap e72: 6 total, 6 up, 6 in 2024-07-03T15:49:57.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:57 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-03T15:49:57.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:57 smithi148 ceph-mon[80490]: osdmap e73: 6 total, 6 up, 6 in 2024-07-03T15:49:57.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:57 smithi028 ceph-mon[92858]: pgmap v133: 65 pgs: 20 active+undersized, 14 active+undersized+degraded, 31 active+clean; 193 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 716 B/s rd, 1 op/s; 34/219 objects degraded (15.525%) 2024-07-03T15:49:57.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:57 smithi028 ceph-mon[92858]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-07-03T15:49:57.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:57 smithi028 ceph-mon[92858]: osd.4 [v2:172.21.15.148:6808/3538420843,v1:172.21.15.148:6809/3538420843] boot 2024-07-03T15:49:57.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:57 smithi028 ceph-mon[92858]: osdmap e72: 6 total, 6 up, 6 in 2024-07-03T15:49:57.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:57 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-03T15:49:57.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:57 smithi028 ceph-mon[92858]: osdmap e73: 6 total, 6 up, 6 in 2024-07-03T15:49:59.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:49:59 smithi148 ceph-mon[80490]: pgmap v136: 65 pgs: 17 active+undersized, 13 active+undersized+degraded, 35 active+clean; 193 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s; 31/219 objects degraded (14.155%) 2024-07-03T15:49:59.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:49:59 smithi028 ceph-mon[92858]: pgmap v136: 65 pgs: 17 active+undersized, 13 active+undersized+degraded, 35 active+clean; 193 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s; 31/219 objects degraded (14.155%) 2024-07-03T15:50:00.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:00 smithi148 ceph-mon[80490]: Health detail: HEALTH_WARN Degraded data redundancy: 31/219 objects degraded (14.155%), 13 pgs degraded 2024-07-03T15:50:00.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:00 smithi148 ceph-mon[80490]: [WRN] PG_DEGRADED: Degraded data redundancy: 31/219 objects degraded (14.155%), 13 pgs degraded 2024-07-03T15:50:00.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:00 smithi148 ceph-mon[80490]: pg 2.4 is active+undersized+degraded, acting [1,0] 2024-07-03T15:50:00.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:00 smithi148 ceph-mon[80490]: pg 2.5 is active+undersized+degraded, acting [3,0] 2024-07-03T15:50:00.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:00 smithi148 ceph-mon[80490]: pg 2.6 is active+undersized+degraded, acting [1,3] 2024-07-03T15:50:00.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:00 smithi148 ceph-mon[80490]: pg 2.7 is active+undersized+degraded, acting [3,2] 2024-07-03T15:50:00.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:00 smithi148 ceph-mon[80490]: pg 2.9 is active+undersized+degraded, acting [1,0] 2024-07-03T15:50:00.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:00 smithi148 ceph-mon[80490]: pg 2.a is active+undersized+degraded, acting [1,3] 2024-07-03T15:50:00.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:00 smithi148 ceph-mon[80490]: pg 2.b is active+undersized+degraded, acting [3,5] 2024-07-03T15:50:00.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:00 smithi148 ceph-mon[80490]: pg 2.f is active+undersized+degraded, acting [0,5] 2024-07-03T15:50:00.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:00 smithi148 ceph-mon[80490]: pg 2.13 is active+undersized+degraded, acting [0,2] 2024-07-03T15:50:00.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:00 smithi148 ceph-mon[80490]: pg 2.14 is active+undersized+degraded, acting [3,5] 2024-07-03T15:50:00.485 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:00 smithi148 ceph-mon[80490]: pg 2.1a is active+undersized+degraded, acting [3,5] 2024-07-03T15:50:00.485 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:00 smithi148 ceph-mon[80490]: pg 2.1c is active+undersized+degraded, acting [5,2] 2024-07-03T15:50:00.485 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:00 smithi148 ceph-mon[80490]: pg 2.1f is active+undersized+degraded, acting [0,3] 2024-07-03T15:50:00.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:00 smithi028 ceph-mon[92858]: Health detail: HEALTH_WARN Degraded data redundancy: 31/219 objects degraded (14.155%), 13 pgs degraded 2024-07-03T15:50:00.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:00 smithi028 ceph-mon[92858]: [WRN] PG_DEGRADED: Degraded data redundancy: 31/219 objects degraded (14.155%), 13 pgs degraded 2024-07-03T15:50:00.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:00 smithi028 ceph-mon[92858]: pg 2.4 is active+undersized+degraded, acting [1,0] 2024-07-03T15:50:00.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:00 smithi028 ceph-mon[92858]: pg 2.5 is active+undersized+degraded, acting [3,0] 2024-07-03T15:50:00.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:00 smithi028 ceph-mon[92858]: pg 2.6 is active+undersized+degraded, acting [1,3] 2024-07-03T15:50:00.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:00 smithi028 ceph-mon[92858]: pg 2.7 is active+undersized+degraded, acting [3,2] 2024-07-03T15:50:00.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:00 smithi028 ceph-mon[92858]: pg 2.9 is active+undersized+degraded, acting [1,0] 2024-07-03T15:50:00.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:00 smithi028 ceph-mon[92858]: pg 2.a is active+undersized+degraded, acting [1,3] 2024-07-03T15:50:00.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:00 smithi028 ceph-mon[92858]: pg 2.b is active+undersized+degraded, acting [3,5] 2024-07-03T15:50:00.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:00 smithi028 ceph-mon[92858]: pg 2.f is active+undersized+degraded, acting [0,5] 2024-07-03T15:50:00.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:00 smithi028 ceph-mon[92858]: pg 2.13 is active+undersized+degraded, acting [0,2] 2024-07-03T15:50:00.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:00 smithi028 ceph-mon[92858]: pg 2.14 is active+undersized+degraded, acting [3,5] 2024-07-03T15:50:00.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:00 smithi028 ceph-mon[92858]: pg 2.1a is active+undersized+degraded, acting [3,5] 2024-07-03T15:50:00.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:00 smithi028 ceph-mon[92858]: pg 2.1c is active+undersized+degraded, acting [5,2] 2024-07-03T15:50:00.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:00 smithi028 ceph-mon[92858]: pg 2.1f is active+undersized+degraded, acting [0,3] 2024-07-03T15:50:01.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:01 smithi148 ceph-mon[80490]: pgmap v137: 65 pgs: 4 active+undersized, 2 active+undersized+degraded, 59 active+clean; 193 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 1 op/s; 3/219 objects degraded (1.370%) 2024-07-03T15:50:01.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:01 smithi148 ceph-mon[80490]: Health check update: Degraded data redundancy: 3/219 objects degraded (1.370%), 2 pgs degraded (PG_DEGRADED) 2024-07-03T15:50:01.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:01 smithi028 ceph-mon[92858]: pgmap v137: 65 pgs: 4 active+undersized, 2 active+undersized+degraded, 59 active+clean; 193 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 1 op/s; 3/219 objects degraded (1.370%) 2024-07-03T15:50:01.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:01 smithi028 ceph-mon[92858]: Health check update: Degraded data redundancy: 3/219 objects degraded (1.370%), 2 pgs degraded (PG_DEGRADED) 2024-07-03T15:50:02.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:02 smithi028 ceph-mon[92858]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3/219 objects degraded (1.370%), 2 pgs degraded) 2024-07-03T15:50:02.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:02 smithi028 ceph-mon[92858]: Cluster is now healthy 2024-07-03T15:50:02.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:02 smithi148 ceph-mon[80490]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3/219 objects degraded (1.370%), 2 pgs degraded) 2024-07-03T15:50:02.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:02 smithi148 ceph-mon[80490]: Cluster is now healthy 2024-07-03T15:50:03.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:03 smithi028 ceph-mon[92858]: pgmap v138: 65 pgs: 65 active+clean; 193 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 442 B/s rd, 1 op/s 2024-07-03T15:50:03.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:03 smithi148 ceph-mon[80490]: pgmap v138: 65 pgs: 65 active+clean; 193 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 442 B/s rd, 1 op/s 2024-07-03T15:50:05.595 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:05 smithi028 ceph-mon[92858]: pgmap v139: 65 pgs: 65 active+clean; 193 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 2 op/s 2024-07-03T15:50:05.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:05 smithi148 ceph-mon[80490]: pgmap v139: 65 pgs: 65 active+clean; 193 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 2 op/s 2024-07-03T15:50:07.534 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:07 smithi148 ceph-mon[80490]: pgmap v140: 65 pgs: 65 active+clean; 193 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 823 B/s rd, 1 op/s 2024-07-03T15:50:07.534 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:07 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:07.534 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:07 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:50:07.534 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:07 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:07.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:07 smithi028 ceph-mon[92858]: pgmap v140: 65 pgs: 65 active+clean; 193 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 823 B/s rd, 1 op/s 2024-07-03T15:50:07.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:07 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:07.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:07 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:50:07.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:07 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:09.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:09 smithi028 ceph-mon[92858]: pgmap v141: 65 pgs: 65 active+clean; 193 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 748 B/s rd, 2 op/s 2024-07-03T15:50:09.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:09 smithi148 ceph-mon[80490]: pgmap v141: 65 pgs: 65 active+clean; 193 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 748 B/s rd, 2 op/s 2024-07-03T15:50:11.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:11 smithi148 ceph-mon[80490]: pgmap v142: 65 pgs: 65 active+clean; 193 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-07-03T15:50:11.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:11 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-07-03T15:50:11.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:11 smithi148 ceph-mon[80490]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-07-03T15:50:11.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:11 smithi148 ceph-mon[80490]: Upgrade: osd.5 is safe to restart 2024-07-03T15:50:11.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:11 smithi028 ceph-mon[92858]: pgmap v142: 65 pgs: 65 active+clean; 193 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-07-03T15:50:11.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:11 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-07-03T15:50:11.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:11 smithi028 ceph-mon[92858]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-07-03T15:50:11.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:11 smithi028 ceph-mon[92858]: Upgrade: osd.5 is safe to restart 2024-07-03T15:50:13.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:12 smithi028 ceph-mon[92858]: Upgrade: Updating osd.5 2024-07-03T15:50:13.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:12 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:13.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:12 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-07-03T15:50:13.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:12 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:50:13.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:12 smithi028 ceph-mon[92858]: Deploying daemon osd.5 on smithi148 2024-07-03T15:50:13.164 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:12 smithi148 ceph-mon[80490]: Upgrade: Updating osd.5 2024-07-03T15:50:13.164 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:12 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:13.164 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:12 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-07-03T15:50:13.164 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:12 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:50:13.164 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:12 smithi148 ceph-mon[80490]: Deploying daemon osd.5 on smithi148 2024-07-03T15:50:13.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:13 smithi148 ceph-mon[80490]: pgmap v143: 65 pgs: 65 active+clean; 193 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:50:13.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:13.992+0000 7f73931d0640 1 -- 172.21.15.28:0/2930183418 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f738c0664a0 msgr2=0x7f738c0ff020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:13.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:13.992+0000 7f73931d0640 1 --2- 172.21.15.28:0/2930183418 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f738c0664a0 0x7f738c0ff020 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f73740048d0 tx=0x7f7374009da0 comp rx=0 tx=0).stop 2024-07-03T15:50:13.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:13.992+0000 7f73931d0640 1 -- 172.21.15.28:0/2930183418 shutdown_connections 2024-07-03T15:50:13.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:13.992+0000 7f73931d0640 1 --2- 172.21.15.28:0/2930183418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f738c0ff560 0x7f738c101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:13.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:13.992+0000 7f73931d0640 1 --2- 172.21.15.28:0/2930183418 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f738c0664a0 0x7f738c0ff020 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:13.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:13.992+0000 7f73931d0640 1 -- 172.21.15.28:0/2930183418 >> 172.21.15.28:0/2930183418 conn(0x7f738c0f6a50 msgr2=0x7f738c0f8e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:50:13.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:13.992+0000 7f73931d0640 1 -- 172.21.15.28:0/2930183418 shutdown_connections 2024-07-03T15:50:13.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:13.992+0000 7f73931d0640 1 -- 172.21.15.28:0/2930183418 wait complete. 2024-07-03T15:50:13.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:13.993+0000 7f73931d0640 1 Processor -- start 2024-07-03T15:50:13.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:13.993+0000 7f73931d0640 1 -- start start 2024-07-03T15:50:13.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:13.993+0000 7f73931d0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f738c0664a0 0x7f738c0fca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:50:13.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:13.993+0000 7f73931d0640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f738c0ff560 0x7f738c0faf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:50:13.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:13.993+0000 7f73931d0640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738c0fb4b0 con 0x7f738c0ff560 2024-07-03T15:50:13.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:13.993+0000 7f73931d0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738c0fb5f0 con 0x7f738c0664a0 2024-07-03T15:50:13.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:13.993+0000 7f738bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f738c0ff560 0x7f738c0faf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:50:13.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:13.994+0000 7f738bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f738c0ff560 0x7f738c0faf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36542/0 (socket says 172.21.15.28:36542) 2024-07-03T15:50:13.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:13.994+0000 7f738bfff640 1 -- 172.21.15.28:0/2394018626 learned_addr learned my addr 172.21.15.28:0/2394018626 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:50:13.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:13.994+0000 7f7390f6c640 1 --2- 172.21.15.28:0/2394018626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f738c0664a0 0x7f738c0fca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:50:13.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:13.994+0000 7f738bfff640 1 -- 172.21.15.28:0/2394018626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f738c0664a0 msgr2=0x7f738c0fca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:13.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:13.994+0000 7f738bfff640 1 --2- 172.21.15.28:0/2394018626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f738c0664a0 0x7f738c0fca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:13.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:13.994+0000 7f738bfff640 1 -- 172.21.15.28:0/2394018626 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7374005040 con 0x7f738c0ff560 2024-07-03T15:50:14.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:13.994+0000 7f738bfff640 1 --2- 172.21.15.28:0/2394018626 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f738c0ff560 0x7f738c0faf20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f737c00dad0 tx=0x7f737c004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:50:14.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:13.994+0000 7f73897fa640 1 -- 172.21.15.28:0/2394018626 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f737c017070 con 0x7f738c0ff560 2024-07-03T15:50:14.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:13.994+0000 7f73897fa640 1 -- 172.21.15.28:0/2394018626 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f737c00f040 con 0x7f738c0ff560 2024-07-03T15:50:14.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:13.994+0000 7f73931d0640 1 -- 172.21.15.28:0/2394018626 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f738c0fb8d0 con 0x7f738c0ff560 2024-07-03T15:50:14.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:13.994+0000 7f73897fa640 1 -- 172.21.15.28:0/2394018626 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f737c0039d0 con 0x7f738c0ff560 2024-07-03T15:50:14.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:13.994+0000 7f73931d0640 1 -- 172.21.15.28:0/2394018626 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f738c1150c0 con 0x7f738c0ff560 2024-07-03T15:50:14.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:13.996+0000 7f73931d0640 1 -- 172.21.15.28:0/2394018626 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f738c0fa6f0 con 0x7f738c0ff560 2024-07-03T15:50:14.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:13.996+0000 7f73897fa640 1 -- 172.21.15.28:0/2394018626 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f737c003c40 con 0x7f738c0ff560 2024-07-03T15:50:14.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:13.996+0000 7f73897fa640 1 --2- 172.21.15.28:0/2394018626 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f737806e0b0 0x7f7378070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:50:14.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:13.997+0000 7f73897fa640 1 -- 172.21.15.28:0/2394018626 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f737c08dac0 con 0x7f738c0ff560 2024-07-03T15:50:14.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:13.997+0000 7f7390f6c640 1 --2- 172.21.15.28:0/2394018626 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f737806e0b0 0x7f7378070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:50:14.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:13.997+0000 7f7390f6c640 1 --2- 172.21.15.28:0/2394018626 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f737806e0b0 0x7f7378070560 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7374000c00 tx=0x7f7374035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:50:14.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.001+0000 7f73897fa640 1 -- 172.21.15.28:0/2394018626 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f737c056f00 con 0x7f738c0ff560 2024-07-03T15:50:14.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:13 smithi028 ceph-mon[92858]: pgmap v143: 65 pgs: 65 active+clean; 193 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:50:14.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.205+0000 7f73931d0640 1 -- 172.21.15.28:0/2394018626 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f738c0f8590 con 0x7f737806e0b0 2024-07-03T15:50:14.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.206+0000 7f73897fa640 1 -- 172.21.15.28:0/2394018626 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f738c0f8590 con 0x7f737806e0b0 2024-07-03T15:50:14.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.208+0000 7f73931d0640 1 -- 172.21.15.28:0/2394018626 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f737806e0b0 msgr2=0x7f7378070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:14.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.208+0000 7f73931d0640 1 --2- 172.21.15.28:0/2394018626 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f737806e0b0 0x7f7378070560 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7374000c00 tx=0x7f7374035040 comp rx=0 tx=0).stop 2024-07-03T15:50:14.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.208+0000 7f73931d0640 1 -- 172.21.15.28:0/2394018626 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f738c0ff560 msgr2=0x7f738c0faf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:14.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.208+0000 7f73931d0640 1 --2- 172.21.15.28:0/2394018626 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f738c0ff560 0x7f738c0faf20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f737c00dad0 tx=0x7f737c004e40 comp rx=0 tx=0).stop 2024-07-03T15:50:14.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.209+0000 7f73931d0640 1 -- 172.21.15.28:0/2394018626 shutdown_connections 2024-07-03T15:50:14.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.209+0000 7f73931d0640 1 --2- 172.21.15.28:0/2394018626 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f737806e0b0 0x7f7378070560 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:14.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.209+0000 7f73931d0640 1 --2- 172.21.15.28:0/2394018626 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f738c0ff560 0x7f738c0faf20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:14.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.209+0000 7f73931d0640 1 --2- 172.21.15.28:0/2394018626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f738c0664a0 0x7f738c0fca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:14.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.209+0000 7f73931d0640 1 -- 172.21.15.28:0/2394018626 >> 172.21.15.28:0/2394018626 conn(0x7f738c0f6a50 msgr2=0x7f738c0f6e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:50:14.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.209+0000 7f73931d0640 1 -- 172.21.15.28:0/2394018626 shutdown_connections 2024-07-03T15:50:14.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.209+0000 7f73931d0640 1 -- 172.21.15.28:0/2394018626 wait complete. 2024-07-03T15:50:14.221 INFO:teuthology.orchestra.run.smithi028.stdout:true 2024-07-03T15:50:14.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.292+0000 7f540b8c8640 1 -- 172.21.15.28:0/298033275 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54040fd640 msgr2=0x7f54040fdab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:14.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.292+0000 7f540b8c8640 1 --2- 172.21.15.28:0/298033275 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54040fd640 0x7f54040fdab0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f54000048d0 tx=0x7f5400009da0 comp rx=0 tx=0).stop 2024-07-03T15:50:14.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.292+0000 7f540b8c8640 1 -- 172.21.15.28:0/298033275 shutdown_connections 2024-07-03T15:50:14.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.292+0000 7f540b8c8640 1 --2- 172.21.15.28:0/298033275 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54040fd640 0x7f54040fdab0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:14.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.292+0000 7f540b8c8640 1 --2- 172.21.15.28:0/298033275 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54040fc370 0x7f54040fc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:14.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.292+0000 7f540b8c8640 1 -- 172.21.15.28:0/298033275 >> 172.21.15.28:0/298033275 conn(0x7f54040f7b50 msgr2=0x7f54040f9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:50:14.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.292+0000 7f540b8c8640 1 -- 172.21.15.28:0/298033275 shutdown_connections 2024-07-03T15:50:14.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.292+0000 7f540b8c8640 1 -- 172.21.15.28:0/298033275 wait complete. 2024-07-03T15:50:14.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.293+0000 7f540b8c8640 1 Processor -- start 2024-07-03T15:50:14.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.293+0000 7f540b8c8640 1 -- start start 2024-07-03T15:50:14.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.293+0000 7f540b8c8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54040fc370 0x7f540410cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:50:14.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.293+0000 7f540b8c8640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54040fd640 0x7f540410d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:50:14.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.293+0000 7f540b8c8640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540410daa0 con 0x7f54040fd640 2024-07-03T15:50:14.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.293+0000 7f540b8c8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540410e2c0 con 0x7f54040fc370 2024-07-03T15:50:14.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.293+0000 7f5408e63640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54040fd640 0x7f540410d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:50:14.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.293+0000 7f5408e63640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54040fd640 0x7f540410d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36564/0 (socket says 172.21.15.28:36564) 2024-07-03T15:50:14.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.293+0000 7f5408e63640 1 -- 172.21.15.28:0/2727315611 learned_addr learned my addr 172.21.15.28:0/2727315611 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:50:14.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.293+0000 7f5409664640 1 --2- 172.21.15.28:0/2727315611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54040fc370 0x7f540410cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:50:14.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.293+0000 7f5408e63640 1 -- 172.21.15.28:0/2727315611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54040fc370 msgr2=0x7f540410cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:14.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.293+0000 7f5408e63640 1 --2- 172.21.15.28:0/2727315611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54040fc370 0x7f540410cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:14.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.294+0000 7f5408e63640 1 -- 172.21.15.28:0/2727315611 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5400005040 con 0x7f54040fd640 2024-07-03T15:50:14.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.294+0000 7f5409664640 1 --2- 172.21.15.28:0/2727315611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54040fc370 0x7f540410cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-03T15:50:14.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.294+0000 7f5408e63640 1 --2- 172.21.15.28:0/2727315611 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54040fd640 0x7f540410d450 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5400003080 tx=0x7f5400009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:50:14.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.294+0000 7f53f9ffb640 1 -- 172.21.15.28:0/2727315611 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5400038070 con 0x7f54040fd640 2024-07-03T15:50:14.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.294+0000 7f53f9ffb640 1 -- 172.21.15.28:0/2727315611 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5400032970 con 0x7f54040fd640 2024-07-03T15:50:14.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.294+0000 7f540b8c8640 1 -- 172.21.15.28:0/2727315611 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f540410e4f0 con 0x7f54040fd640 2024-07-03T15:50:14.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.294+0000 7f53f9ffb640 1 -- 172.21.15.28:0/2727315611 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5400033900 con 0x7f54040fd640 2024-07-03T15:50:14.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.294+0000 7f540b8c8640 1 -- 172.21.15.28:0/2727315611 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f540410ea30 con 0x7f54040fd640 2024-07-03T15:50:14.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.295+0000 7f540b8c8640 1 -- 172.21.15.28:0/2727315611 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54040fb7d0 con 0x7f54040fd640 2024-07-03T15:50:14.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.296+0000 7f53f9ffb640 1 -- 172.21.15.28:0/2727315611 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5400030020 con 0x7f54040fd640 2024-07-03T15:50:14.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.296+0000 7f53f9ffb640 1 --2- 172.21.15.28:0/2727315611 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f53f006e0b0 0x7f53f0070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:50:14.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.296+0000 7f53f9ffb640 1 -- 172.21.15.28:0/2727315611 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f54000adb60 con 0x7f54040fd640 2024-07-03T15:50:14.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.296+0000 7f5409664640 1 --2- 172.21.15.28:0/2727315611 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f53f006e0b0 0x7f53f0070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:50:14.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.297+0000 7f5409664640 1 --2- 172.21.15.28:0/2727315611 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f53f006e0b0 0x7f53f0070560 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f53f40041d0 tx=0x7f53f4009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:50:14.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.300+0000 7f53f9ffb640 1 -- 172.21.15.28:0/2727315611 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f5400076f60 con 0x7f54040fd640 2024-07-03T15:50:14.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.510+0000 7f540b8c8640 1 -- 172.21.15.28:0/2727315611 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f540410edd0 con 0x7f53f006e0b0 2024-07-03T15:50:14.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.511+0000 7f53f9ffb640 1 -- 172.21.15.28:0/2727315611 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f540410edd0 con 0x7f53f006e0b0 2024-07-03T15:50:14.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.514+0000 7f540b8c8640 1 -- 172.21.15.28:0/2727315611 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f53f006e0b0 msgr2=0x7f53f0070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:14.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.514+0000 7f540b8c8640 1 --2- 172.21.15.28:0/2727315611 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f53f006e0b0 0x7f53f0070560 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f53f40041d0 tx=0x7f53f4009040 comp rx=0 tx=0).stop 2024-07-03T15:50:14.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.514+0000 7f540b8c8640 1 -- 172.21.15.28:0/2727315611 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54040fd640 msgr2=0x7f540410d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:14.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.514+0000 7f540b8c8640 1 --2- 172.21.15.28:0/2727315611 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54040fd640 0x7f540410d450 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5400003080 tx=0x7f5400009da0 comp rx=0 tx=0).stop 2024-07-03T15:50:14.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.514+0000 7f540b8c8640 1 -- 172.21.15.28:0/2727315611 shutdown_connections 2024-07-03T15:50:14.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.514+0000 7f540b8c8640 1 --2- 172.21.15.28:0/2727315611 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f53f006e0b0 0x7f53f0070560 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:14.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.514+0000 7f540b8c8640 1 --2- 172.21.15.28:0/2727315611 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54040fd640 0x7f540410d450 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:14.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.514+0000 7f540b8c8640 1 --2- 172.21.15.28:0/2727315611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54040fc370 0x7f540410cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:14.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.514+0000 7f540b8c8640 1 -- 172.21.15.28:0/2727315611 >> 172.21.15.28:0/2727315611 conn(0x7f54040f7b50 msgr2=0x7f54040f91a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:50:14.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.514+0000 7f540b8c8640 1 -- 172.21.15.28:0/2727315611 shutdown_connections 2024-07-03T15:50:14.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.514+0000 7f540b8c8640 1 -- 172.21.15.28:0/2727315611 wait complete. 2024-07-03T15:50:14.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.596+0000 7fe5ba7ca640 1 -- 172.21.15.28:0/3284535738 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5b40fd600 msgr2=0x7fe5b40fda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:14.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.596+0000 7fe5ba7ca640 1 --2- 172.21.15.28:0/3284535738 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5b40fd600 0x7fe5b40fda70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe5a80048d0 tx=0x7fe5a8009da0 comp rx=0 tx=0).stop 2024-07-03T15:50:14.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.596+0000 7fe5ba7ca640 1 -- 172.21.15.28:0/3284535738 shutdown_connections 2024-07-03T15:50:14.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.596+0000 7fe5ba7ca640 1 --2- 172.21.15.28:0/3284535738 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5b40fd600 0x7fe5b40fda70 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:14.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.596+0000 7fe5ba7ca640 1 --2- 172.21.15.28:0/3284535738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5b40fc330 0x7fe5b40fc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:14.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.596+0000 7fe5ba7ca640 1 -- 172.21.15.28:0/3284535738 >> 172.21.15.28:0/3284535738 conn(0x7fe5b40f7af0 msgr2=0x7fe5b40f9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:50:14.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.596+0000 7fe5ba7ca640 1 -- 172.21.15.28:0/3284535738 shutdown_connections 2024-07-03T15:50:14.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.596+0000 7fe5ba7ca640 1 -- 172.21.15.28:0/3284535738 wait complete. 2024-07-03T15:50:14.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.596+0000 7fe5ba7ca640 1 Processor -- start 2024-07-03T15:50:14.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.596+0000 7fe5ba7ca640 1 -- start start 2024-07-03T15:50:14.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.597+0000 7fe5ba7ca640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5b40fc330 0x7fe5b410f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:50:14.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.597+0000 7fe5ba7ca640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5b40fd600 0x7fe5b410f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:50:14.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.597+0000 7fe5ba7ca640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b410fcf0 con 0x7fe5b40fd600 2024-07-03T15:50:14.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.597+0000 7fe5ba7ca640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b4110510 con 0x7fe5b40fc330 2024-07-03T15:50:14.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.597+0000 7fe5b37fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5b40fd600 0x7fe5b410f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:50:14.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.597+0000 7fe5b37fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5b40fd600 0x7fe5b410f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36584/0 (socket says 172.21.15.28:36584) 2024-07-03T15:50:14.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.597+0000 7fe5b37fe640 1 -- 172.21.15.28:0/2707832362 learned_addr learned my addr 172.21.15.28:0/2707832362 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:50:14.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.597+0000 7fe5b3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5b40fc330 0x7fe5b410f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:50:14.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.597+0000 7fe5b37fe640 1 -- 172.21.15.28:0/2707832362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5b40fc330 msgr2=0x7fe5b410f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:14.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.597+0000 7fe5b37fe640 1 --2- 172.21.15.28:0/2707832362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5b40fc330 0x7fe5b410f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:14.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.597+0000 7fe5b37fe640 1 -- 172.21.15.28:0/2707832362 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a8005040 con 0x7fe5b40fd600 2024-07-03T15:50:14.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.597+0000 7fe5b37fe640 1 --2- 172.21.15.28:0/2707832362 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5b40fd600 0x7fe5b410f6a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe5a8009f80 tx=0x7fe5a8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:50:14.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.597+0000 7fe5b0ff9640 1 -- 172.21.15.28:0/2707832362 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5a8038070 con 0x7fe5b40fd600 2024-07-03T15:50:14.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.597+0000 7fe5b0ff9640 1 -- 172.21.15.28:0/2707832362 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe5a8032970 con 0x7fe5b40fd600 2024-07-03T15:50:14.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.597+0000 7fe5ba7ca640 1 -- 172.21.15.28:0/2707832362 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5b4110740 con 0x7fe5b40fd600 2024-07-03T15:50:14.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.598+0000 7fe5b0ff9640 1 -- 172.21.15.28:0/2707832362 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5a8033860 con 0x7fe5b40fd600 2024-07-03T15:50:14.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.598+0000 7fe5ba7ca640 1 -- 172.21.15.28:0/2707832362 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5b4110c80 con 0x7fe5b40fd600 2024-07-03T15:50:14.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.599+0000 7fe5ba7ca640 1 -- 172.21.15.28:0/2707832362 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5b40fb790 con 0x7fe5b40fd600 2024-07-03T15:50:14.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.599+0000 7fe5b0ff9640 1 -- 172.21.15.28:0/2707832362 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe5a8030020 con 0x7fe5b40fd600 2024-07-03T15:50:14.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.600+0000 7fe5b0ff9640 1 --2- 172.21.15.28:0/2707832362 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fe59c06e0b0 0x7fe59c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:50:14.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.600+0000 7fe5b0ff9640 1 -- 172.21.15.28:0/2707832362 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fe5a80ada00 con 0x7fe5b40fd600 2024-07-03T15:50:14.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.600+0000 7fe5b3fff640 1 --2- 172.21.15.28:0/2707832362 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fe59c06e0b0 0x7fe59c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:50:14.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.600+0000 7fe5b3fff640 1 --2- 172.21.15.28:0/2707832362 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fe59c06e0b0 0x7fe59c070560 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4004ed0 tx=0x7fe5a400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:50:14.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.604+0000 7fe5b0ff9640 1 -- 172.21.15.28:0/2707832362 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fe5a8076e30 con 0x7fe5b40fd600 2024-07-03T15:50:14.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.798+0000 7fe5ba7ca640 1 -- 172.21.15.28:0/2707832362 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe5b40008d0 con 0x7fe59c06e0b0 2024-07-03T15:50:14.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.807+0000 7fe5b0ff9640 1 -- 172.21.15.28:0/2707832362 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fe5b40008d0 con 0x7fe59c06e0b0 2024-07-03T15:50:14.811 INFO:teuthology.orchestra.run.smithi028.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-03T15:50:14.811 INFO:teuthology.orchestra.run.smithi028.stdout:alertmanager.smithi028 smithi028 *:9093,9094 running (5m) 87s ago 13m 24.8M - 0.25.0 c8568f914cd2 1425315a0de9 2024-07-03T15:50:14.811 INFO:teuthology.orchestra.run.smithi028.stdout:crash.smithi028 smithi028 running (3m) 87s ago 14m 6895k - 19.0.0-2550-gc00506dc 9d49405603e8 c62c2f45cabe 2024-07-03T15:50:14.811 INFO:teuthology.orchestra.run.smithi028.stdout:crash.smithi148 smithi148 running (3m) 23s ago 13m 6895k - 19.0.0-2550-gc00506dc 9d49405603e8 b1318dcaacb6 2024-07-03T15:50:14.811 INFO:teuthology.orchestra.run.smithi028.stdout:grafana.smithi028 smithi028 *:3000 running (5m) 87s ago 13m 81.9M - 9.4.7 954c08fa6188 f9f17c92cfea 2024-07-03T15:50:14.811 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi028.rhpvve smithi028 running (8m) 87s ago 8m 152M - 17.2.7-1047-gd4b063ab 4ca399d9e65d 3ed5abd02cfb 2024-07-03T15:50:14.811 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi028.wmbxew smithi028 running (8m) 87s ago 8m 110M - 17.2.7-1047-gd4b063ab 4ca399d9e65d b3bf736998f9 2024-07-03T15:50:14.812 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi148.szomfl smithi148 running (8m) 23s ago 8m 16.4M - 17.2.7-1047-gd4b063ab 4ca399d9e65d d94f5c95fa8d 2024-07-03T15:50:14.812 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi148.xgpuld smithi148 running (8m) 23s ago 8m 13.8M - 17.2.7-1047-gd4b063ab 4ca399d9e65d fc70360a8de0 2024-07-03T15:50:14.812 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.smithi028.vtqrtz smithi028 *:8443,9283,8765 running (5m) 87s ago 15m 496M - 19.0.0-2550-gc00506dc 9d49405603e8 14d0363c5b77 2024-07-03T15:50:14.812 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.smithi148.rjzsar smithi148 *:8443,9283,8765 running (4m) 23s ago 12m 440M - 19.0.0-2550-gc00506dc 9d49405603e8 c0193aa8a5dd 2024-07-03T15:50:14.812 INFO:teuthology.orchestra.run.smithi028.stdout:mon.smithi028 smithi028 running (3m) 87s ago 15m 46.8M 2048M 19.0.0-2550-gc00506dc 9d49405603e8 5093f7bf47d7 2024-07-03T15:50:14.812 INFO:teuthology.orchestra.run.smithi028.stdout:mon.smithi148 smithi148 running (3m) 23s ago 12m 38.5M 2048M 19.0.0-2550-gc00506dc 9d49405603e8 d338d2c648da 2024-07-03T15:50:14.812 INFO:teuthology.orchestra.run.smithi028.stdout:node-exporter.smithi028 smithi028 *:9100 running (5m) 87s ago 13m 15.6M - 1.5.0 0da6a335fe13 91ca1f49de0e 2024-07-03T15:50:14.812 INFO:teuthology.orchestra.run.smithi028.stdout:node-exporter.smithi148 smithi148 *:9100 running (5m) 23s ago 12m 16.7M - 1.5.0 0da6a335fe13 1eaaf7b1b561 2024-07-03T15:50:14.812 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0 smithi028 running (2m) 87s ago 12m 101M 1558M 19.0.0-2550-gc00506dc 9d49405603e8 4454cecff8fc 2024-07-03T15:50:14.812 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1 smithi028 running (2m) 87s ago 11m 99.1M 1558M 19.0.0-2550-gc00506dc 9d49405603e8 af67f7088643 2024-07-03T15:50:14.812 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2 smithi028 running (93s) 87s ago 11m 12.1M 1558M 19.0.0-2550-gc00506dc 9d49405603e8 5d160a327dc2 2024-07-03T15:50:14.812 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3 smithi148 running (58s) 23s ago 10m 104M 2581M 19.0.0-2550-gc00506dc 9d49405603e8 1d622eba4b3b 2024-07-03T15:50:14.812 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4 smithi148 running (26s) 23s ago 10m 14.3M 2581M 19.0.0-2550-gc00506dc 9d49405603e8 546343d0f31f 2024-07-03T15:50:14.813 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5 smithi148 running (10m) 23s ago 10m 238M 2581M 17.2.7-1047-gd4b063ab 4ca399d9e65d d185c6905384 2024-07-03T15:50:14.813 INFO:teuthology.orchestra.run.smithi028.stdout:prometheus.smithi028 smithi028 *:9095 running (4m) 87s ago 13m 60.5M - 2.43.0 a07b618ecd1d 2b0eb2a95afd 2024-07-03T15:50:14.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.810+0000 7fe5ba7ca640 1 -- 172.21.15.28:0/2707832362 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fe59c06e0b0 msgr2=0x7fe59c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:14.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.810+0000 7fe5ba7ca640 1 --2- 172.21.15.28:0/2707832362 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fe59c06e0b0 0x7fe59c070560 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4004ed0 tx=0x7fe5a400b040 comp rx=0 tx=0).stop 2024-07-03T15:50:14.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.810+0000 7fe5ba7ca640 1 -- 172.21.15.28:0/2707832362 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5b40fd600 msgr2=0x7fe5b410f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:14.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.810+0000 7fe5ba7ca640 1 --2- 172.21.15.28:0/2707832362 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5b40fd600 0x7fe5b410f6a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe5a8009f80 tx=0x7fe5a8009da0 comp rx=0 tx=0).stop 2024-07-03T15:50:14.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.810+0000 7fe5ba7ca640 1 -- 172.21.15.28:0/2707832362 shutdown_connections 2024-07-03T15:50:14.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.810+0000 7fe5ba7ca640 1 --2- 172.21.15.28:0/2707832362 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fe59c06e0b0 0x7fe59c070560 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:14.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.810+0000 7fe5ba7ca640 1 --2- 172.21.15.28:0/2707832362 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5b40fd600 0x7fe5b410f6a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:14.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.810+0000 7fe5ba7ca640 1 --2- 172.21.15.28:0/2707832362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5b40fc330 0x7fe5b410f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:14.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.810+0000 7fe5ba7ca640 1 -- 172.21.15.28:0/2707832362 >> 172.21.15.28:0/2707832362 conn(0x7fe5b40f7af0 msgr2=0x7fe5b40f9120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:50:14.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.810+0000 7fe5ba7ca640 1 -- 172.21.15.28:0/2707832362 shutdown_connections 2024-07-03T15:50:14.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.810+0000 7fe5ba7ca640 1 -- 172.21.15.28:0/2707832362 wait complete. 2024-07-03T15:50:14.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.891+0000 7f1a76c94640 1 -- 172.21.15.28:0/3149678357 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1a700fd9d0 msgr2=0x7f1a700fdde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:14.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.891+0000 7f1a76c94640 1 --2- 172.21.15.28:0/3149678357 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1a700fd9d0 0x7f1a700fdde0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1a600048d0 tx=0x7f1a60009da0 comp rx=0 tx=0).stop 2024-07-03T15:50:14.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.891+0000 7f1a76c94640 1 -- 172.21.15.28:0/3149678357 shutdown_connections 2024-07-03T15:50:14.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.891+0000 7f1a76c94640 1 --2- 172.21.15.28:0/3149678357 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a700fe320 0x7f1a700fa480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:14.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.891+0000 7f1a76c94640 1 --2- 172.21.15.28:0/3149678357 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1a700fd9d0 0x7f1a700fdde0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:14.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.892+0000 7f1a76c94640 1 -- 172.21.15.28:0/3149678357 >> 172.21.15.28:0/3149678357 conn(0x7f1a700f61d0 msgr2=0x7f1a700f8600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:50:14.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.892+0000 7f1a76c94640 1 -- 172.21.15.28:0/3149678357 shutdown_connections 2024-07-03T15:50:14.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.892+0000 7f1a76c94640 1 -- 172.21.15.28:0/3149678357 wait complete. 2024-07-03T15:50:14.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.892+0000 7f1a76c94640 1 Processor -- start 2024-07-03T15:50:14.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.892+0000 7f1a76c94640 1 -- start start 2024-07-03T15:50:14.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.892+0000 7f1a76c94640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a700fd9d0 0x7f1a700fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:50:14.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.892+0000 7f1a76c94640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1a700fe320 0x7f1a700fb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:50:14.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.892+0000 7f1a76c94640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a700fbc40 con 0x7f1a700fe320 2024-07-03T15:50:14.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.892+0000 7f1a76c94640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a700fbd80 con 0x7f1a700fd9d0 2024-07-03T15:50:14.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.893+0000 7f1a6ffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1a700fe320 0x7f1a700fb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:50:14.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.893+0000 7f1a6ffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1a700fe320 0x7f1a700fb6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36606/0 (socket says 172.21.15.28:36606) 2024-07-03T15:50:14.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.893+0000 7f1a6ffff640 1 -- 172.21.15.28:0/372015856 learned_addr learned my addr 172.21.15.28:0/372015856 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:50:14.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.893+0000 7f1a74a30640 1 --2- 172.21.15.28:0/372015856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a700fd9d0 0x7f1a700fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:50:14.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.893+0000 7f1a6ffff640 1 -- 172.21.15.28:0/372015856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a700fd9d0 msgr2=0x7f1a700fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:14.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.893+0000 7f1a6ffff640 1 --2- 172.21.15.28:0/372015856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a700fd9d0 0x7f1a700fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:14.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.893+0000 7f1a6ffff640 1 -- 172.21.15.28:0/372015856 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a60005040 con 0x7f1a700fe320 2024-07-03T15:50:14.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.893+0000 7f1a6ffff640 1 --2- 172.21.15.28:0/372015856 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1a700fe320 0x7f1a700fb6b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1a64002840 tx=0x7f1a64004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:50:14.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.893+0000 7f1a6d7fa640 1 -- 172.21.15.28:0/372015856 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a64007c30 con 0x7f1a700fe320 2024-07-03T15:50:14.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.893+0000 7f1a6d7fa640 1 -- 172.21.15.28:0/372015856 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1a6400f040 con 0x7f1a700fe320 2024-07-03T15:50:14.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.893+0000 7f1a76c94640 1 -- 172.21.15.28:0/372015856 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a700fc060 con 0x7f1a700fe320 2024-07-03T15:50:14.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.893+0000 7f1a6d7fa640 1 -- 172.21.15.28:0/372015856 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a6400da30 con 0x7f1a700fe320 2024-07-03T15:50:14.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.893+0000 7f1a76c94640 1 -- 172.21.15.28:0/372015856 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a70115050 con 0x7f1a700fe320 2024-07-03T15:50:14.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.894+0000 7f1a76c94640 1 -- 172.21.15.28:0/372015856 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a700f9e70 con 0x7f1a700fe320 2024-07-03T15:50:14.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.895+0000 7f1a6d7fa640 1 -- 172.21.15.28:0/372015856 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1a640096a0 con 0x7f1a700fe320 2024-07-03T15:50:14.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.895+0000 7f1a6d7fa640 1 --2- 172.21.15.28:0/372015856 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f1a5806e0b0 0x7f1a58070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:50:14.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.895+0000 7f1a6d7fa640 1 -- 172.21.15.28:0/372015856 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f1a6408d9b0 con 0x7f1a700fe320 2024-07-03T15:50:14.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.896+0000 7f1a74a30640 1 --2- 172.21.15.28:0/372015856 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f1a5806e0b0 0x7f1a58070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:50:14.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.896+0000 7f1a74a30640 1 --2- 172.21.15.28:0/372015856 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f1a5806e0b0 0x7f1a58070560 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1a600043d0 tx=0x7f1a60035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:50:14.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:14.900+0000 7f1a6d7fa640 1 -- 172.21.15.28:0/372015856 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f1a64056cd0 con 0x7f1a700fe320 2024-07-03T15:50:14.982 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:14 smithi148 systemd[1]: Stopping Ceph osd.5 for a85e4be6-3951-11ef-bcab-c7b262605968... 2024-07-03T15:50:15.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.155+0000 7f1a76c94640 1 -- 172.21.15.28:0/372015856 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1a7004b530 con 0x7f1a700fe320 2024-07-03T15:50:15.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.156+0000 7f1a6d7fa640 1 -- 172.21.15.28:0/372015856 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+838 (secure 0 0 0) 0x7f1a6405a390 con 0x7f1a700fe320 2024-07-03T15:50:15.160 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-07-03T15:50:15.160 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": { 2024-07-03T15:50:15.160 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 2 2024-07-03T15:50:15.160 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:50:15.160 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": { 2024-07-03T15:50:15.160 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 2 2024-07-03T15:50:15.160 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:50:15.160 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": { 2024-07-03T15:50:15.160 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 1, 2024-07-03T15:50:15.161 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 5 2024-07-03T15:50:15.161 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:50:15.161 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": { 2024-07-03T15:50:15.161 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 4 2024-07-03T15:50:15.161 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:50:15.161 INFO:teuthology.orchestra.run.smithi028.stdout: "overall": { 2024-07-03T15:50:15.161 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 5, 2024-07-03T15:50:15.161 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 9 2024-07-03T15:50:15.161 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-07-03T15:50:15.161 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-07-03T15:50:15.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.158+0000 7f1a76c94640 1 -- 172.21.15.28:0/372015856 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f1a5806e0b0 msgr2=0x7f1a58070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:15.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.158+0000 7f1a76c94640 1 --2- 172.21.15.28:0/372015856 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f1a5806e0b0 0x7f1a58070560 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1a600043d0 tx=0x7f1a60035040 comp rx=0 tx=0).stop 2024-07-03T15:50:15.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.158+0000 7f1a76c94640 1 -- 172.21.15.28:0/372015856 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1a700fe320 msgr2=0x7f1a700fb6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:15.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.158+0000 7f1a76c94640 1 --2- 172.21.15.28:0/372015856 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1a700fe320 0x7f1a700fb6b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1a64002840 tx=0x7f1a64004570 comp rx=0 tx=0).stop 2024-07-03T15:50:15.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.158+0000 7f1a76c94640 1 -- 172.21.15.28:0/372015856 shutdown_connections 2024-07-03T15:50:15.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.158+0000 7f1a76c94640 1 --2- 172.21.15.28:0/372015856 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f1a5806e0b0 0x7f1a58070560 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:15.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.158+0000 7f1a76c94640 1 --2- 172.21.15.28:0/372015856 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1a700fe320 0x7f1a700fb6b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:15.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.158+0000 7f1a76c94640 1 --2- 172.21.15.28:0/372015856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a700fd9d0 0x7f1a700fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:15.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.158+0000 7f1a76c94640 1 -- 172.21.15.28:0/372015856 >> 172.21.15.28:0/372015856 conn(0x7f1a700f61d0 msgr2=0x7f1a700f78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:50:15.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.158+0000 7f1a76c94640 1 -- 172.21.15.28:0/372015856 shutdown_connections 2024-07-03T15:50:15.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.158+0000 7f1a76c94640 1 -- 172.21.15.28:0/372015856 wait complete. 2024-07-03T15:50:15.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.241+0000 7f9a30b16640 1 -- 172.21.15.28:0/4200731962 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a2c0fb280 msgr2=0x7f9a2c0fb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:15.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.241+0000 7f9a30b16640 1 --2- 172.21.15.28:0/4200731962 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a2c0fb280 0x7f9a2c0fb690 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c0048d0 tx=0x7f9a1c009da0 comp rx=0 tx=0).stop 2024-07-03T15:50:15.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.241+0000 7f9a30b16640 1 -- 172.21.15.28:0/4200731962 shutdown_connections 2024-07-03T15:50:15.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.241+0000 7f9a30b16640 1 --2- 172.21.15.28:0/4200731962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a2c0fc550 0x7f9a2c0fc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:15.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.241+0000 7f9a30b16640 1 --2- 172.21.15.28:0/4200731962 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a2c0fb280 0x7f9a2c0fb690 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:15.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.241+0000 7f9a30b16640 1 -- 172.21.15.28:0/4200731962 >> 172.21.15.28:0/4200731962 conn(0x7f9a2c0f6a20 msgr2=0x7f9a2c0f8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:50:15.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.241+0000 7f9a30b16640 1 -- 172.21.15.28:0/4200731962 shutdown_connections 2024-07-03T15:50:15.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.241+0000 7f9a30b16640 1 -- 172.21.15.28:0/4200731962 wait complete. 2024-07-03T15:50:15.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.242+0000 7f9a30b16640 1 Processor -- start 2024-07-03T15:50:15.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.242+0000 7f9a30b16640 1 -- start start 2024-07-03T15:50:15.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.242+0000 7f9a30b16640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a2c0fb280 0x7f9a2c10cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:50:15.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.242+0000 7f9a30b16640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a2c0fc550 0x7f9a2c10d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:50:15.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.242+0000 7f9a30b16640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2c10db20 con 0x7f9a2c0fb280 2024-07-03T15:50:15.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.242+0000 7f9a30b16640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2c10e340 con 0x7f9a2c0fc550 2024-07-03T15:50:15.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.242+0000 7f9a2a59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a2c0fb280 0x7f9a2c10cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:50:15.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.242+0000 7f9a2a59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a2c0fb280 0x7f9a2c10cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36616/0 (socket says 172.21.15.28:36616) 2024-07-03T15:50:15.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.242+0000 7f9a2a59c640 1 -- 172.21.15.28:0/623469332 learned_addr learned my addr 172.21.15.28:0/623469332 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:50:15.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.242+0000 7f9a23fff640 1 --2- 172.21.15.28:0/623469332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a2c0fc550 0x7f9a2c10d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:50:15.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.242+0000 7f9a2a59c640 1 -- 172.21.15.28:0/623469332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a2c0fc550 msgr2=0x7f9a2c10d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:15.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.242+0000 7f9a2a59c640 1 --2- 172.21.15.28:0/623469332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a2c0fc550 0x7f9a2c10d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:15.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.242+0000 7f9a2a59c640 1 -- 172.21.15.28:0/623469332 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a1c005040 con 0x7f9a2c0fb280 2024-07-03T15:50:15.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.243+0000 7f9a2a59c640 1 --2- 172.21.15.28:0/623469332 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a2c0fb280 0x7f9a2c10cf90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c009f80 tx=0x7f9a1c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:50:15.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.243+0000 7f9a22ffd640 1 -- 172.21.15.28:0/623469332 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a1c038070 con 0x7f9a2c0fb280 2024-07-03T15:50:15.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.243+0000 7f9a30b16640 1 -- 172.21.15.28:0/623469332 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a2c10e570 con 0x7f9a2c0fb280 2024-07-03T15:50:15.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.243+0000 7f9a22ffd640 1 -- 172.21.15.28:0/623469332 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9a1c032970 con 0x7f9a2c0fb280 2024-07-03T15:50:15.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.243+0000 7f9a22ffd640 1 -- 172.21.15.28:0/623469332 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a1c033860 con 0x7f9a2c0fb280 2024-07-03T15:50:15.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.243+0000 7f9a30b16640 1 -- 172.21.15.28:0/623469332 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a2c10eab0 con 0x7f9a2c0fb280 2024-07-03T15:50:15.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.244+0000 7f9a30b16640 1 -- 172.21.15.28:0/623469332 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a2c0fa6e0 con 0x7f9a2c0fb280 2024-07-03T15:50:15.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.244+0000 7f9a22ffd640 1 -- 172.21.15.28:0/623469332 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9a1c030020 con 0x7f9a2c0fb280 2024-07-03T15:50:15.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.245+0000 7f9a22ffd640 1 --2- 172.21.15.28:0/623469332 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f9a1806e0b0 0x7f9a18070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:50:15.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.245+0000 7f9a22ffd640 1 -- 172.21.15.28:0/623469332 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f9a1c0ad9a0 con 0x7f9a2c0fb280 2024-07-03T15:50:15.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.245+0000 7f9a23fff640 1 --2- 172.21.15.28:0/623469332 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f9a1806e0b0 0x7f9a18070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:50:15.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.245+0000 7f9a23fff640 1 --2- 172.21.15.28:0/623469332 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f9a1806e0b0 0x7f9a18070560 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9a14009fd0 tx=0x7f9a14009e50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:50:15.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.249+0000 7f9a22ffd640 1 -- 172.21.15.28:0/623469332 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f9a1c076da0 con 0x7f9a2c0fb280 2024-07-03T15:50:15.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.472+0000 7f9a30b16640 1 -- 172.21.15.28:0/623469332 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9a2c103e70 con 0x7f9a2c0fb280 2024-07-03T15:50:15.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.472+0000 7f9a22ffd640 1 -- 172.21.15.28:0/623469332 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1908 (secure 0 0 0) 0x7f9a1c041790 con 0x7f9a2c0fb280 2024-07-03T15:50:15.476 INFO:teuthology.orchestra.run.smithi028.stdout:e17 2024-07-03T15:50:15.476 INFO:teuthology.orchestra.run.smithi028.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-03T15:50:15.477 INFO:teuthology.orchestra.run.smithi028.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:50:15.477 INFO:teuthology.orchestra.run.smithi028.stdout:legacy client fscid: 1 2024-07-03T15:50:15.477 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:50:15.477 INFO:teuthology.orchestra.run.smithi028.stdout:Filesystem 'cephfs' (1) 2024-07-03T15:50:15.477 INFO:teuthology.orchestra.run.smithi028.stdout:fs_name cephfs 2024-07-03T15:50:15.477 INFO:teuthology.orchestra.run.smithi028.stdout:epoch 16 2024-07-03T15:50:15.477 INFO:teuthology.orchestra.run.smithi028.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-03T15:50:15.477 INFO:teuthology.orchestra.run.smithi028.stdout:created 2024-07-03T15:41:40.656298+0000 2024-07-03T15:50:15.477 INFO:teuthology.orchestra.run.smithi028.stdout:modified 2024-07-03T15:41:56.454984+0000 2024-07-03T15:50:15.477 INFO:teuthology.orchestra.run.smithi028.stdout:tableserver 0 2024-07-03T15:50:15.477 INFO:teuthology.orchestra.run.smithi028.stdout:root 0 2024-07-03T15:50:15.477 INFO:teuthology.orchestra.run.smithi028.stdout:session_timeout 60 2024-07-03T15:50:15.477 INFO:teuthology.orchestra.run.smithi028.stdout:session_autoclose 300 2024-07-03T15:50:15.478 INFO:teuthology.orchestra.run.smithi028.stdout:max_file_size 1099511627776 2024-07-03T15:50:15.478 INFO:teuthology.orchestra.run.smithi028.stdout:max_xattr_size 65536 2024-07-03T15:50:15.478 INFO:teuthology.orchestra.run.smithi028.stdout:required_client_features {} 2024-07-03T15:50:15.478 INFO:teuthology.orchestra.run.smithi028.stdout:last_failure 0 2024-07-03T15:50:15.478 INFO:teuthology.orchestra.run.smithi028.stdout:last_failure_osd_epoch 0 2024-07-03T15:50:15.478 INFO:teuthology.orchestra.run.smithi028.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:50:15.478 INFO:teuthology.orchestra.run.smithi028.stdout:max_mds 1 2024-07-03T15:50:15.478 INFO:teuthology.orchestra.run.smithi028.stdout:in 0 2024-07-03T15:50:15.478 INFO:teuthology.orchestra.run.smithi028.stdout:up {0=14450} 2024-07-03T15:50:15.478 INFO:teuthology.orchestra.run.smithi028.stdout:failed 2024-07-03T15:50:15.478 INFO:teuthology.orchestra.run.smithi028.stdout:damaged 2024-07-03T15:50:15.478 INFO:teuthology.orchestra.run.smithi028.stdout:stopped 2024-07-03T15:50:15.479 INFO:teuthology.orchestra.run.smithi028.stdout:data_pools [3] 2024-07-03T15:50:15.479 INFO:teuthology.orchestra.run.smithi028.stdout:metadata_pool 2 2024-07-03T15:50:15.479 INFO:teuthology.orchestra.run.smithi028.stdout:inline_data disabled 2024-07-03T15:50:15.479 INFO:teuthology.orchestra.run.smithi028.stdout:balancer 2024-07-03T15:50:15.479 INFO:teuthology.orchestra.run.smithi028.stdout:bal_rank_mask -1 2024-07-03T15:50:15.479 INFO:teuthology.orchestra.run.smithi028.stdout:standby_count_wanted 1 2024-07-03T15:50:15.479 INFO:teuthology.orchestra.run.smithi028.stdout:qdb_cluster leader: 0 members: 2024-07-03T15:50:15.479 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi028.rhpvve{0:14450} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.28:6826/1613936042,v1:172.21.15.28:6827/1613936042] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:50:15.479 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi028.wmbxew{0:14460} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.28:6828/1730336821,v1:172.21.15.28:6829/1730336821] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:50:15.479 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:50:15.479 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:50:15.479 INFO:teuthology.orchestra.run.smithi028.stdout:Standby daemons: 2024-07-03T15:50:15.479 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:50:15.479 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi148.szomfl{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/4171231858,v1:172.21.15.148:6827/4171231858] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:50:15.480 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi148.xgpuld{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/2347536398,v1:172.21.15.148:6825/2347536398] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:50:15.480 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 17 2024-07-03T15:50:15.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.474+0000 7f9a30b16640 1 -- 172.21.15.28:0/623469332 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f9a1806e0b0 msgr2=0x7f9a18070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:15.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.474+0000 7f9a30b16640 1 --2- 172.21.15.28:0/623469332 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f9a1806e0b0 0x7f9a18070560 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9a14009fd0 tx=0x7f9a14009e50 comp rx=0 tx=0).stop 2024-07-03T15:50:15.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.474+0000 7f9a30b16640 1 -- 172.21.15.28:0/623469332 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a2c0fb280 msgr2=0x7f9a2c10cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:15.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.474+0000 7f9a30b16640 1 --2- 172.21.15.28:0/623469332 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a2c0fb280 0x7f9a2c10cf90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c009f80 tx=0x7f9a1c009da0 comp rx=0 tx=0).stop 2024-07-03T15:50:15.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.474+0000 7f9a30b16640 1 -- 172.21.15.28:0/623469332 shutdown_connections 2024-07-03T15:50:15.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.474+0000 7f9a30b16640 1 --2- 172.21.15.28:0/623469332 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f9a1806e0b0 0x7f9a18070560 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:15.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.474+0000 7f9a30b16640 1 --2- 172.21.15.28:0/623469332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a2c0fc550 0x7f9a2c10d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:15.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.474+0000 7f9a30b16640 1 --2- 172.21.15.28:0/623469332 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a2c0fb280 0x7f9a2c10cf90 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:15.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.474+0000 7f9a30b16640 1 -- 172.21.15.28:0/623469332 >> 172.21.15.28:0/623469332 conn(0x7f9a2c0f6a20 msgr2=0x7f9a2c066120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:50:15.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.475+0000 7f9a30b16640 1 -- 172.21.15.28:0/623469332 shutdown_connections 2024-07-03T15:50:15.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.475+0000 7f9a30b16640 1 -- 172.21.15.28:0/623469332 wait complete. 2024-07-03T15:50:15.482 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:15 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5[54485]: 2024-07-03T15:50:15.022+0000 7fc3a6f4b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-03T15:50:15.482 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:15 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5[54485]: 2024-07-03T15:50:15.022+0000 7fc3a6f4b640 -1 osd.5 73 *** Got signal Terminated *** 2024-07-03T15:50:15.482 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:15 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5[54485]: 2024-07-03T15:50:15.022+0000 7fc3a6f4b640 -1 osd.5 73 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-03T15:50:15.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.558+0000 7fd367dd5640 1 -- 172.21.15.28:0/3190739138 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd3600fc560 msgr2=0x7fd3600fc9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:15.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.558+0000 7fd367dd5640 1 --2- 172.21.15.28:0/3190739138 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd3600fc560 0x7fd3600fc9d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd35c0048d0 tx=0x7fd35c009da0 comp rx=0 tx=0).stop 2024-07-03T15:50:15.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.558+0000 7fd367dd5640 1 -- 172.21.15.28:0/3190739138 shutdown_connections 2024-07-03T15:50:15.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.558+0000 7fd367dd5640 1 --2- 172.21.15.28:0/3190739138 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd3600fc560 0x7fd3600fc9d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:15.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.558+0000 7fd367dd5640 1 --2- 172.21.15.28:0/3190739138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3600fb290 0x7fd3600fb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:15.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.558+0000 7fd367dd5640 1 -- 172.21.15.28:0/3190739138 >> 172.21.15.28:0/3190739138 conn(0x7fd3600f6a50 msgr2=0x7fd3600f8e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:50:15.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.558+0000 7fd367dd5640 1 -- 172.21.15.28:0/3190739138 shutdown_connections 2024-07-03T15:50:15.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.558+0000 7fd367dd5640 1 -- 172.21.15.28:0/3190739138 wait complete. 2024-07-03T15:50:15.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.559+0000 7fd367dd5640 1 Processor -- start 2024-07-03T15:50:15.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.559+0000 7fd367dd5640 1 -- start start 2024-07-03T15:50:15.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.559+0000 7fd367dd5640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd3600fb290 0x7fd36010f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:50:15.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.559+0000 7fd367dd5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3600fc560 0x7fd36010f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:50:15.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.559+0000 7fd367dd5640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd36010fda0 con 0x7fd3600fb290 2024-07-03T15:50:15.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.559+0000 7fd367dd5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3601105c0 con 0x7fd3600fc560 2024-07-03T15:50:15.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.559+0000 7fd365b71640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd3600fb290 0x7fd36010f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:50:15.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.559+0000 7fd365b71640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd3600fb290 0x7fd36010f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36634/0 (socket says 172.21.15.28:36634) 2024-07-03T15:50:15.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.559+0000 7fd365b71640 1 -- 172.21.15.28:0/2307928071 learned_addr learned my addr 172.21.15.28:0/2307928071 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:50:15.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.559+0000 7fd365370640 1 --2- 172.21.15.28:0/2307928071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3600fc560 0x7fd36010f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:50:15.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.560+0000 7fd365b71640 1 -- 172.21.15.28:0/2307928071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3600fc560 msgr2=0x7fd36010f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:15.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.560+0000 7fd365b71640 1 --2- 172.21.15.28:0/2307928071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3600fc560 0x7fd36010f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:15.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.560+0000 7fd365b71640 1 -- 172.21.15.28:0/2307928071 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd35c005040 con 0x7fd3600fb290 2024-07-03T15:50:15.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.560+0000 7fd365b71640 1 --2- 172.21.15.28:0/2307928071 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd3600fb290 0x7fd36010f210 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd354002a50 tx=0x7fd354004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:50:15.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.560+0000 7fd3527fc640 1 -- 172.21.15.28:0/2307928071 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd354007c30 con 0x7fd3600fb290 2024-07-03T15:50:15.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.560+0000 7fd3527fc640 1 -- 172.21.15.28:0/2307928071 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd35400f040 con 0x7fd3600fb290 2024-07-03T15:50:15.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.560+0000 7fd367dd5640 1 -- 172.21.15.28:0/2307928071 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd360110850 con 0x7fd3600fb290 2024-07-03T15:50:15.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.560+0000 7fd3527fc640 1 -- 172.21.15.28:0/2307928071 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd35400dad0 con 0x7fd3600fb290 2024-07-03T15:50:15.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.560+0000 7fd367dd5640 1 -- 172.21.15.28:0/2307928071 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd360110dc0 con 0x7fd3600fb290 2024-07-03T15:50:15.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.561+0000 7fd367dd5640 1 -- 172.21.15.28:0/2307928071 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd360105710 con 0x7fd3600fb290 2024-07-03T15:50:15.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.561+0000 7fd3527fc640 1 -- 172.21.15.28:0/2307928071 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd35400dc70 con 0x7fd3600fb290 2024-07-03T15:50:15.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.562+0000 7fd3527fc640 1 --2- 172.21.15.28:0/2307928071 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fd34c06e0b0 0x7fd34c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:50:15.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.562+0000 7fd3527fc640 1 -- 172.21.15.28:0/2307928071 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fd35408d270 con 0x7fd3600fb290 2024-07-03T15:50:15.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.562+0000 7fd365370640 1 --2- 172.21.15.28:0/2307928071 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fd34c06e0b0 0x7fd34c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:50:15.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.563+0000 7fd365370640 1 --2- 172.21.15.28:0/2307928071 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fd34c06e0b0 0x7fd34c070560 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd35c0043d0 tx=0x7fd35c035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:50:15.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.567+0000 7fd3527fc640 1 -- 172.21.15.28:0/2307928071 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd354056720 con 0x7fd3600fb290 2024-07-03T15:50:15.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.769+0000 7fd367dd5640 1 -- 172.21.15.28:0/2307928071 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd3600008d0 con 0x7fd34c06e0b0 2024-07-03T15:50:15.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.770+0000 7fd3527fc640 1 -- 172.21.15.28:0/2307928071 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fd3600008d0 con 0x7fd34c06e0b0 2024-07-03T15:50:15.773 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-07-03T15:50:15.774 INFO:teuthology.orchestra.run.smithi028.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2", 2024-07-03T15:50:15.774 INFO:teuthology.orchestra.run.smithi028.stdout: "in_progress": true, 2024-07-03T15:50:15.774 INFO:teuthology.orchestra.run.smithi028.stdout: "which": "Upgrading all daemon types on all hosts", 2024-07-03T15:50:15.774 INFO:teuthology.orchestra.run.smithi028.stdout: "services_complete": [ 2024-07-03T15:50:15.774 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr", 2024-07-03T15:50:15.774 INFO:teuthology.orchestra.run.smithi028.stdout: "crash", 2024-07-03T15:50:15.774 INFO:teuthology.orchestra.run.smithi028.stdout: "mon" 2024-07-03T15:50:15.774 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-07-03T15:50:15.774 INFO:teuthology.orchestra.run.smithi028.stdout: "progress": "11/21 daemons upgraded", 2024-07-03T15:50:15.774 INFO:teuthology.orchestra.run.smithi028.stdout: "message": "Currently upgrading osd daemons", 2024-07-03T15:50:15.774 INFO:teuthology.orchestra.run.smithi028.stdout: "is_paused": false 2024-07-03T15:50:15.774 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-07-03T15:50:15.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.772+0000 7fd367dd5640 1 -- 172.21.15.28:0/2307928071 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fd34c06e0b0 msgr2=0x7fd34c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:15.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.772+0000 7fd367dd5640 1 --2- 172.21.15.28:0/2307928071 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fd34c06e0b0 0x7fd34c070560 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd35c0043d0 tx=0x7fd35c035040 comp rx=0 tx=0).stop 2024-07-03T15:50:15.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.772+0000 7fd367dd5640 1 -- 172.21.15.28:0/2307928071 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd3600fb290 msgr2=0x7fd36010f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:15.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.772+0000 7fd367dd5640 1 --2- 172.21.15.28:0/2307928071 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd3600fb290 0x7fd36010f210 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd354002a50 tx=0x7fd354004600 comp rx=0 tx=0).stop 2024-07-03T15:50:15.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.772+0000 7fd367dd5640 1 -- 172.21.15.28:0/2307928071 shutdown_connections 2024-07-03T15:50:15.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.772+0000 7fd367dd5640 1 --2- 172.21.15.28:0/2307928071 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fd34c06e0b0 0x7fd34c070560 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:15.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.772+0000 7fd367dd5640 1 --2- 172.21.15.28:0/2307928071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3600fc560 0x7fd36010f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:15.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.772+0000 7fd367dd5640 1 --2- 172.21.15.28:0/2307928071 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd3600fb290 0x7fd36010f210 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:15.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.772+0000 7fd367dd5640 1 -- 172.21.15.28:0/2307928071 >> 172.21.15.28:0/2307928071 conn(0x7fd3600f6a50 msgr2=0x7fd3600f80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:50:15.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.773+0000 7fd367dd5640 1 -- 172.21.15.28:0/2307928071 shutdown_connections 2024-07-03T15:50:15.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.773+0000 7fd367dd5640 1 -- 172.21.15.28:0/2307928071 wait complete. 2024-07-03T15:50:15.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.860+0000 7f78e4dbe640 1 -- 172.21.15.28:0/664613338 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78e00fb240 msgr2=0x7f78e00fb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:15.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.860+0000 7f78e4dbe640 1 --2- 172.21.15.28:0/664613338 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78e00fb240 0x7f78e00fb650 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f78d00048d0 tx=0x7f78d0009da0 comp rx=0 tx=0).stop 2024-07-03T15:50:15.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.860+0000 7f78e4dbe640 1 -- 172.21.15.28:0/664613338 shutdown_connections 2024-07-03T15:50:15.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.860+0000 7f78e4dbe640 1 --2- 172.21.15.28:0/664613338 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78e00fc510 0x7f78e00fc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:15.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.860+0000 7f78e4dbe640 1 --2- 172.21.15.28:0/664613338 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78e00fb240 0x7f78e00fb650 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:15.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.860+0000 7f78e4dbe640 1 -- 172.21.15.28:0/664613338 >> 172.21.15.28:0/664613338 conn(0x7f78e00f6a20 msgr2=0x7f78e00f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:50:15.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.860+0000 7f78e4dbe640 1 -- 172.21.15.28:0/664613338 shutdown_connections 2024-07-03T15:50:15.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.860+0000 7f78e4dbe640 1 -- 172.21.15.28:0/664613338 wait complete. 2024-07-03T15:50:15.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.860+0000 7f78e4dbe640 1 Processor -- start 2024-07-03T15:50:15.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.860+0000 7f78e4dbe640 1 -- start start 2024-07-03T15:50:15.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.860+0000 7f78e4dbe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78e00fb240 0x7f78e010cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:50:15.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.860+0000 7f78e4dbe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78e00fc510 0x7f78e010d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:50:15.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.860+0000 7f78e4dbe640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e010db00 con 0x7f78e00fc510 2024-07-03T15:50:15.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.861+0000 7f78e4dbe640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e010e320 con 0x7f78e00fb240 2024-07-03T15:50:15.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.861+0000 7f78ddd9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78e00fc510 0x7f78e010d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:50:15.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.861+0000 7f78ddd9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78e00fc510 0x7f78e010d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36650/0 (socket says 172.21.15.28:36650) 2024-07-03T15:50:15.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.861+0000 7f78ddd9b640 1 -- 172.21.15.28:0/4044561859 learned_addr learned my addr 172.21.15.28:0/4044561859 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:50:15.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.861+0000 7f78de59c640 1 --2- 172.21.15.28:0/4044561859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78e00fb240 0x7f78e010cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:50:15.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.861+0000 7f78ddd9b640 1 -- 172.21.15.28:0/4044561859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78e00fb240 msgr2=0x7f78e010cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:15.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.861+0000 7f78ddd9b640 1 --2- 172.21.15.28:0/4044561859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78e00fb240 0x7f78e010cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:15.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.861+0000 7f78ddd9b640 1 -- 172.21.15.28:0/4044561859 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d0005040 con 0x7f78e00fc510 2024-07-03T15:50:15.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.861+0000 7f78ddd9b640 1 --2- 172.21.15.28:0/4044561859 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78e00fc510 0x7f78e010d4b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f78d4002a50 tx=0x7f78d4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:50:15.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.861+0000 7f78ceffd640 1 -- 172.21.15.28:0/4044561859 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78d4007c30 con 0x7f78e00fc510 2024-07-03T15:50:15.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.861+0000 7f78e4dbe640 1 -- 172.21.15.28:0/4044561859 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78e010e5b0 con 0x7f78e00fc510 2024-07-03T15:50:15.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.861+0000 7f78ceffd640 1 -- 172.21.15.28:0/4044561859 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f78d400f040 con 0x7f78e00fc510 2024-07-03T15:50:15.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.861+0000 7f78ceffd640 1 -- 172.21.15.28:0/4044561859 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78d400dad0 con 0x7f78e00fc510 2024-07-03T15:50:15.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.861+0000 7f78e4dbe640 1 -- 172.21.15.28:0/4044561859 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78e010eb50 con 0x7f78e00fc510 2024-07-03T15:50:15.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.863+0000 7f78e4dbe640 1 -- 172.21.15.28:0/4044561859 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78e00fa6a0 con 0x7f78e00fc510 2024-07-03T15:50:15.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.863+0000 7f78ceffd640 1 -- 172.21.15.28:0/4044561859 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f78d400dc70 con 0x7f78e00fc510 2024-07-03T15:50:15.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.863+0000 7f78ceffd640 1 --2- 172.21.15.28:0/4044561859 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f78c806e0b0 0x7f78c8070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:50:15.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.864+0000 7f78ceffd640 1 -- 172.21.15.28:0/4044561859 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f78d408d220 con 0x7f78e00fc510 2024-07-03T15:50:15.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.864+0000 7f78de59c640 1 --2- 172.21.15.28:0/4044561859 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f78c806e0b0 0x7f78c8070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:50:15.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.864+0000 7f78de59c640 1 --2- 172.21.15.28:0/4044561859 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f78c806e0b0 0x7f78c8070560 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f78d00043d0 tx=0x7f78d0035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:50:15.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:15.868+0000 7f78ceffd640 1 -- 172.21.15.28:0/4044561859 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f78d40566d0 con 0x7f78e00fc510 2024-07-03T15:50:16.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:15 smithi028 ceph-mon[92858]: pgmap v144: 65 pgs: 65 active+clean; 193 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:50:16.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:15 smithi028 ceph-mon[92858]: from='client.34344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:50:16.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:15 smithi028 ceph-mon[92858]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:50:16.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:15 smithi028 ceph-mon[92858]: from='client.34352 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:50:16.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:15 smithi028 ceph-mon[92858]: osd.5 marked itself down and dead 2024-07-03T15:50:16.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:15 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/372015856' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:50:16.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:15 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/623469332' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-03T15:50:16.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:16.123+0000 7f78e4dbe640 1 -- 172.21.15.28:0/4044561859 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f78e0103eb0 con 0x7f78e00fc510 2024-07-03T15:50:16.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:16.124+0000 7f78ceffd640 1 -- 172.21.15.28:0/4044561859 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+337 (secure 0 0 0) 0x7f78d4059d90 con 0x7f78e00fc510 2024-07-03T15:50:16.127 INFO:teuthology.orchestra.run.smithi028.stdout:HEALTH_WARN 1 osds down; all OSDs are running squid or later but require_osd_release < squid 2024-07-03T15:50:16.127 INFO:teuthology.orchestra.run.smithi028.stdout:[WRN] OSD_DOWN: 1 osds down 2024-07-03T15:50:16.127 INFO:teuthology.orchestra.run.smithi028.stdout: osd.5 (root=default,host=smithi148) is down 2024-07-03T15:50:16.128 INFO:teuthology.orchestra.run.smithi028.stdout:[WRN] OSD_UPGRADE_FINISHED: all OSDs are running squid or later but require_osd_release < squid 2024-07-03T15:50:16.128 INFO:teuthology.orchestra.run.smithi028.stdout: all OSDs are running squid or later but require_osd_release < squid 2024-07-03T15:50:16.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:16.126+0000 7f78e4dbe640 1 -- 172.21.15.28:0/4044561859 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f78c806e0b0 msgr2=0x7f78c8070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:16.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:16.126+0000 7f78e4dbe640 1 --2- 172.21.15.28:0/4044561859 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f78c806e0b0 0x7f78c8070560 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f78d00043d0 tx=0x7f78d0035040 comp rx=0 tx=0).stop 2024-07-03T15:50:16.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:16.126+0000 7f78e4dbe640 1 -- 172.21.15.28:0/4044561859 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78e00fc510 msgr2=0x7f78e010d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:16.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:16.126+0000 7f78e4dbe640 1 --2- 172.21.15.28:0/4044561859 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78e00fc510 0x7f78e010d4b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f78d4002a50 tx=0x7f78d4004600 comp rx=0 tx=0).stop 2024-07-03T15:50:16.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:16.126+0000 7f78e4dbe640 1 -- 172.21.15.28:0/4044561859 shutdown_connections 2024-07-03T15:50:16.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:16.126+0000 7f78e4dbe640 1 --2- 172.21.15.28:0/4044561859 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f78c806e0b0 0x7f78c8070560 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:16.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:16.126+0000 7f78e4dbe640 1 --2- 172.21.15.28:0/4044561859 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78e00fc510 0x7f78e010d4b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:16.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:16.126+0000 7f78e4dbe640 1 --2- 172.21.15.28:0/4044561859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78e00fb240 0x7f78e010cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:16.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:16.126+0000 7f78e4dbe640 1 -- 172.21.15.28:0/4044561859 >> 172.21.15.28:0/4044561859 conn(0x7f78e00f6a20 msgr2=0x7f78e0065300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:50:16.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:16.126+0000 7f78e4dbe640 1 -- 172.21.15.28:0/4044561859 shutdown_connections 2024-07-03T15:50:16.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:16.126+0000 7f78e4dbe640 1 -- 172.21.15.28:0/4044561859 wait complete. 2024-07-03T15:50:16.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:15 smithi148 ceph-mon[80490]: pgmap v144: 65 pgs: 65 active+clean; 193 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:50:16.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:15 smithi148 ceph-mon[80490]: from='client.34344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:50:16.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:15 smithi148 ceph-mon[80490]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:50:16.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:15 smithi148 ceph-mon[80490]: from='client.34352 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:50:16.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:15 smithi148 ceph-mon[80490]: osd.5 marked itself down and dead 2024-07-03T15:50:16.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:15 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/372015856' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:50:16.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:15 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/623469332' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-03T15:50:16.601 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:16 smithi148 podman[99583]: 2024-07-03 15:50:16.292130421 +0000 UTC m=+1.349877988 container died d185c6905384f8dc64d57b1826b8bf89183f851f0956507b97f6808aa7e689d9 (image=quay.ceph.io/ceph-ci/ceph@sha256:0b7651540cdb24c1b97466c79c298d54c7794c5f53d5d6b6c5d76a2d3b130d85, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=quincy-d4b063a, org.label-schema.build-date=20240625, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-07-03T15:50:16.982 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:16 smithi148 podman[99583]: 2024-07-03 15:50:16.69756768 +0000 UTC m=+1.755315247 container cleanup d185c6905384f8dc64d57b1826b8bf89183f851f0956507b97f6808aa7e689d9 (image=quay.ceph.io/ceph-ci/ceph@sha256:0b7651540cdb24c1b97466c79c298d54c7794c5f53d5d6b6c5d76a2d3b130d85, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240625, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=quincy-d4b063a, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-07-03T15:50:16.982 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:16 smithi148 bash[99583]: ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5 2024-07-03T15:50:16.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:16 smithi148 ceph-mon[80490]: from='client.34364 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:50:16.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:16 smithi148 ceph-mon[80490]: Health check failed: 1 osds down (OSD_DOWN) 2024-07-03T15:50:16.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:16 smithi148 ceph-mon[80490]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-07-03T15:50:16.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:16 smithi148 ceph-mon[80490]: osdmap e74: 6 total, 5 up, 6 in 2024-07-03T15:50:16.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:16 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/4044561859' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-03T15:50:17.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:16 smithi028 ceph-mon[92858]: from='client.34364 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:50:17.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:16 smithi028 ceph-mon[92858]: Health check failed: 1 osds down (OSD_DOWN) 2024-07-03T15:50:17.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:16 smithi028 ceph-mon[92858]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-07-03T15:50:17.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:16 smithi028 ceph-mon[92858]: osdmap e74: 6 total, 5 up, 6 in 2024-07-03T15:50:17.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:16 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/4044561859' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-03T15:50:17.325 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:16 smithi148 podman[99600]: 2024-07-03 15:50:16.981918348 +0000 UTC m=+0.683412571 container remove d185c6905384f8dc64d57b1826b8bf89183f851f0956507b97f6808aa7e689d9 (image=quay.ceph.io/ceph-ci/ceph@sha256:0b7651540cdb24c1b97466c79c298d54c7794c5f53d5d6b6c5d76a2d3b130d85, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-d4b063a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.36.0, org.label-schema.build-date=20240625, GIT_CLEAN=True) 2024-07-03T15:50:17.647 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:17 smithi148 podman[99685]: 2024-07-03 15:50:17.225177752 +0000 UTC m=+0.017351420 image pull 9d49405603e8445bc510362cdeb05f243619be76b0f3897dd4c152b00cffe45f quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2 2024-07-03T15:50:17.647 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:17 smithi148 podman[99685]: 2024-07-03 15:50:17.332885215 +0000 UTC m=+0.125058879 container create ebd3dfce0adf0863f42a067b629c609a2d535e679bf6613d2ee7820709db6d3d (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-release-c00506d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240625) 2024-07-03T15:50:17.647 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:17 smithi148 podman[99685]: 2024-07-03 15:50:17.569437057 +0000 UTC m=+0.361610724 container init ebd3dfce0adf0863f42a067b629c609a2d535e679bf6613d2ee7820709db6d3d (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240625, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=squid-release-c00506d, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-07-03T15:50:17.647 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:17 smithi148 podman[99685]: 2024-07-03 15:50:17.572810326 +0000 UTC m=+0.364983990 container start ebd3dfce0adf0863f42a067b629c609a2d535e679bf6613d2ee7820709db6d3d (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240625, RELEASE=squid-release-c00506d, GIT_CLEAN=True) 2024-07-03T15:50:17.647 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:17 smithi148 podman[99685]: 2024-07-03 15:50:17.644089535 +0000 UTC m=+0.436263198 container attach ebd3dfce0adf0863f42a067b629c609a2d535e679bf6613d2ee7820709db6d3d (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240625, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=squid-release-c00506d, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-07-03T15:50:17.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:17 smithi148 ceph-mon[80490]: pgmap v146: 65 pgs: 13 stale+active+clean, 52 active+clean; 193 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-07-03T15:50:17.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:17 smithi148 ceph-mon[80490]: osdmap e75: 6 total, 5 up, 6 in 2024-07-03T15:50:17.983 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:17 smithi148 podman[99685]: 2024-07-03 15:50:17.832949588 +0000 UTC m=+0.625123250 container died ebd3dfce0adf0863f42a067b629c609a2d535e679bf6613d2ee7820709db6d3d (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240625, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-release-c00506d, org.label-schema.schema-version=1.0) 2024-07-03T15:50:18.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:17 smithi028 ceph-mon[92858]: pgmap v146: 65 pgs: 13 stale+active+clean, 52 active+clean; 193 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-07-03T15:50:18.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:17 smithi028 ceph-mon[92858]: osdmap e75: 6 total, 5 up, 6 in 2024-07-03T15:50:18.732 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:18 smithi148 podman[99708]: 2024-07-03 15:50:18.38104839 +0000 UTC m=+0.542462545 container remove ebd3dfce0adf0863f42a067b629c609a2d535e679bf6613d2ee7820709db6d3d (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5-deactivate, RELEASE=squid-release-c00506d, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240625, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-07-03T15:50:18.732 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:18 smithi148 systemd[1]: ceph-a85e4be6-3951-11ef-bcab-c7b262605968@osd.5.service: Deactivated successfully. 2024-07-03T15:50:18.732 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:18 smithi148 systemd[1]: Stopped Ceph osd.5 for a85e4be6-3951-11ef-bcab-c7b262605968. 2024-07-03T15:50:18.733 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:18 smithi148 systemd[1]: ceph-a85e4be6-3951-11ef-bcab-c7b262605968@osd.5.service: Consumed 1min 1.208s CPU time. 2024-07-03T15:50:18.733 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:18 smithi148 systemd[1]: Starting Ceph osd.5 for a85e4be6-3951-11ef-bcab-c7b262605968... 2024-07-03T15:50:19.079 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:18 smithi148 podman[99808]: 2024-07-03 15:50:18.708440073 +0000 UTC m=+0.017393412 image pull 9d49405603e8445bc510362cdeb05f243619be76b0f3897dd4c152b00cffe45f quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2 2024-07-03T15:50:19.079 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:18 smithi148 podman[99808]: 2024-07-03 15:50:18.83197905 +0000 UTC m=+0.140932376 container create 41e6c12be84d373e53d6f0dde0558e3f58804a8c9e12a676837fffee90c1a7dc (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5-activate, org.label-schema.vendor=CentOS, RELEASE=squid-release-c00506d, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240625, GIT_CLEAN=True) 2024-07-03T15:50:19.079 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:19 smithi148 podman[99808]: 2024-07-03 15:50:19.076576147 +0000 UTC m=+0.385529480 container init 41e6c12be84d373e53d6f0dde0558e3f58804a8c9e12a676837fffee90c1a7dc (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5-activate, ceph=True, org.label-schema.build-date=20240625, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, RELEASE=squid-release-c00506d, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-03T15:50:19.450 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:19 smithi148 podman[99808]: 2024-07-03 15:50:19.079869864 +0000 UTC m=+0.388823190 container start 41e6c12be84d373e53d6f0dde0558e3f58804a8c9e12a676837fffee90c1a7dc (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5-activate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240625, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-release-c00506d, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-07-03T15:50:19.451 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:19 smithi148 podman[99808]: 2024-07-03 15:50:19.168035339 +0000 UTC m=+0.476988669 container attach 41e6c12be84d373e53d6f0dde0558e3f58804a8c9e12a676837fffee90c1a7dc (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5-activate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240625, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-release-c00506d, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-07-03T15:50:19.732 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:19 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5-activate[99823]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-07-03T15:50:19.733 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:19 smithi148 bash[99808]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-07-03T15:50:19.733 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:19 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5-activate[99823]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-07-03T15:50:19.733 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:19 smithi148 bash[99808]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-07-03T15:50:19.733 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:19 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5-activate[99823]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-07-03T15:50:19.733 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:19 smithi148 bash[99808]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-07-03T15:50:19.733 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:19 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5-activate[99823]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-07-03T15:50:19.733 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:19 smithi148 bash[99808]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-07-03T15:50:19.733 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:19 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5-activate[99823]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-07-03T15:50:19.733 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:19 smithi148 bash[99808]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-07-03T15:50:19.733 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:19 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5-activate[99823]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-07-03T15:50:19.733 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:19 smithi148 bash[99808]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-07-03T15:50:19.733 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:19 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5-activate[99823]: --> ceph-volume raw activate successful for osd ID: 5 2024-07-03T15:50:19.734 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:19 smithi148 bash[99808]: --> ceph-volume raw activate successful for osd ID: 5 2024-07-03T15:50:19.734 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:19 smithi148 podman[99808]: 2024-07-03 15:50:19.501080257 +0000 UTC m=+0.810033583 container died 41e6c12be84d373e53d6f0dde0558e3f58804a8c9e12a676837fffee90c1a7dc (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5-activate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240625, CEPH_POINT_RELEASE=, RELEASE=squid-release-c00506d, ceph=True, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-07-03T15:50:20.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:19 smithi028 ceph-mon[92858]: pgmap v148: 65 pgs: 13 stale+active+clean, 52 active+clean; 193 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-07-03T15:50:20.141 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:19 smithi148 ceph-mon[80490]: pgmap v148: 65 pgs: 13 stale+active+clean, 52 active+clean; 193 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-07-03T15:50:20.444 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:20 smithi148 podman[99948]: 2024-07-03 15:50:20.138152856 +0000 UTC m=+0.631304856 container remove 41e6c12be84d373e53d6f0dde0558e3f58804a8c9e12a676837fffee90c1a7dc (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5-activate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240625, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-release-c00506d, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True) 2024-07-03T15:50:20.444 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:20 smithi148 podman[100008]: 2024-07-03 15:50:20.347390127 +0000 UTC m=+0.109866684 container create a819fcc3b5696d337d6bfb3999e88dc58da6d2c415f1a7636f84873cbbcd9f3f (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=squid-release-c00506d, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240625, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-07-03T15:50:20.444 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:20 smithi148 podman[100008]: 2024-07-03 15:50:20.254800955 +0000 UTC m=+0.017277520 image pull 9d49405603e8445bc510362cdeb05f243619be76b0f3897dd4c152b00cffe45f quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2 2024-07-03T15:50:20.732 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:20 smithi148 podman[100008]: 2024-07-03 15:50:20.541899857 +0000 UTC m=+0.304376433 container init a819fcc3b5696d337d6bfb3999e88dc58da6d2c415f1a7636f84873cbbcd9f3f (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5, RELEASE=squid-release-c00506d, org.label-schema.build-date=20240625, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-07-03T15:50:20.732 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:20 smithi148 podman[100008]: 2024-07-03 15:50:20.545443758 +0000 UTC m=+0.307920318 container start a819fcc3b5696d337d6bfb3999e88dc58da6d2c415f1a7636f84873cbbcd9f3f (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240625, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-release-c00506d, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True) 2024-07-03T15:50:20.732 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:20 smithi148 ceph-osd[100027]: -- 172.21.15.148:0/373963881 <== mon.1 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5586f18a21e0 con 0x5586f0a59c00 2024-07-03T15:50:20.732 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:20 smithi148 bash[100008]: a819fcc3b5696d337d6bfb3999e88dc58da6d2c415f1a7636f84873cbbcd9f3f 2024-07-03T15:50:20.732 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:20 smithi148 systemd[1]: Started Ceph osd.5 for a85e4be6-3951-11ef-bcab-c7b262605968. 2024-07-03T15:50:21.464 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:21 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5[100022]: 2024-07-03T15:50:21.091+0000 7f5b606f6640 -1 Falling back to public interface 2024-07-03T15:50:21.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:21 smithi148 ceph-mon[80490]: pgmap v149: 65 pgs: 12 active+undersized, 4 stale+active+clean, 9 active+undersized+degraded, 40 active+clean; 193 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 23/219 objects degraded (10.502%) 2024-07-03T15:50:21.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:21 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:21.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:21 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:21.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:21 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:50:21.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:21 smithi148 ceph-mon[80490]: Health check failed: Degraded data redundancy: 23/219 objects degraded (10.502%), 9 pgs degraded (PG_DEGRADED) 2024-07-03T15:50:22.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:21 smithi028 ceph-mon[92858]: pgmap v149: 65 pgs: 12 active+undersized, 4 stale+active+clean, 9 active+undersized+degraded, 40 active+clean; 193 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 23/219 objects degraded (10.502%) 2024-07-03T15:50:22.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:21 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:22.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:21 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:22.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:21 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:50:22.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:21 smithi028 ceph-mon[92858]: Health check failed: Degraded data redundancy: 23/219 objects degraded (10.502%), 9 pgs degraded (PG_DEGRADED) 2024-07-03T15:50:23.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:23 smithi148 ceph-mon[80490]: pgmap v150: 65 pgs: 16 active+undersized, 12 active+undersized+degraded, 37 active+clean; 193 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 29/219 objects degraded (13.242%) 2024-07-03T15:50:23.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:23 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:23.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:23 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:50:23.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:23 smithi028 ceph-mon[92858]: pgmap v150: 65 pgs: 16 active+undersized, 12 active+undersized+degraded, 37 active+clean; 193 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 29/219 objects degraded (13.242%) 2024-07-03T15:50:23.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:23 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:23.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:23 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:50:24.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:24 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:24.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:24 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:24.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:24 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:24.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:24 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:24.666 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:24 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:24.667 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:24 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:24.667 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:24 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:24.667 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:24 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:25.395 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:25 smithi148 ceph-mon[80490]: pgmap v151: 65 pgs: 16 active+undersized, 12 active+undersized+degraded, 37 active+clean; 193 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 29/219 objects degraded (13.242%) 2024-07-03T15:50:25.595 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:25 smithi028 ceph-mon[92858]: pgmap v151: 65 pgs: 16 active+undersized, 12 active+undersized+degraded, 37 active+clean; 193 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 29/219 objects degraded (13.242%) 2024-07-03T15:50:25.973 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:25 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5[100022]: 2024-07-03T15:50:25.743+0000 7f5b606f6640 -1 osd.5 73 log_to_monitors true 2024-07-03T15:50:26.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:26 smithi028 ceph-mon[92858]: from='osd.5 [v2:172.21.15.148:6816/2178157137,v1:172.21.15.148:6817/2178157137]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-07-03T15:50:26.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:26 smithi028 ceph-mon[92858]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-07-03T15:50:26.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:26 smithi148 ceph-mon[80490]: from='osd.5 [v2:172.21.15.148:6816/2178157137,v1:172.21.15.148:6817/2178157137]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-07-03T15:50:26.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:26 smithi148 ceph-mon[80490]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-07-03T15:50:27.482 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:50:27 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5[100022]: 2024-07-03T15:50:27.197+0000 7f5b54488640 -1 osd.5 73 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-03T15:50:27.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:27 smithi148 ceph-mon[80490]: pgmap v152: 65 pgs: 16 active+undersized, 12 active+undersized+degraded, 37 active+clean; 193 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 29/219 objects degraded (13.242%) 2024-07-03T15:50:27.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:27 smithi148 ceph-mon[80490]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-07-03T15:50:27.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:27 smithi148 ceph-mon[80490]: osdmap e76: 6 total, 5 up, 6 in 2024-07-03T15:50:27.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:27 smithi148 ceph-mon[80490]: from='osd.5 [v2:172.21.15.148:6816/2178157137,v1:172.21.15.148:6817/2178157137]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-07-03T15:50:27.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:27 smithi148 ceph-mon[80490]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-07-03T15:50:27.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:27 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:27.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:27 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:27.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:27 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:50:27.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:27 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:50:27.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:27 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:27.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:27 smithi148 ceph-mon[80490]: from='osd.5 ' entity='osd.5' 2024-07-03T15:50:27.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:27 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:50:27.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:27 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:50:27.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:27 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:50:27.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:27 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:50:27.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:27 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:50:27.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:27 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:27.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:27 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-07-03T15:50:27.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:27 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-07-03T15:50:27.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:27 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-07-03T15:50:27.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:27 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-07-03T15:50:27.485 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:27 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-07-03T15:50:27.485 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:27 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-07-03T15:50:27.485 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:27 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-07-03T15:50:27.485 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:27 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-07-03T15:50:27.485 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:27 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-07-03T15:50:27.485 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:27 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-07-03T15:50:27.485 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:27 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-07-03T15:50:27.485 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:27 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-07-03T15:50:27.485 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:27 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-07-03T15:50:27.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:27 smithi028 ceph-mon[92858]: pgmap v152: 65 pgs: 16 active+undersized, 12 active+undersized+degraded, 37 active+clean; 193 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 29/219 objects degraded (13.242%) 2024-07-03T15:50:27.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:27 smithi028 ceph-mon[92858]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-07-03T15:50:27.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:27 smithi028 ceph-mon[92858]: osdmap e76: 6 total, 5 up, 6 in 2024-07-03T15:50:27.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:27 smithi028 ceph-mon[92858]: from='osd.5 [v2:172.21.15.148:6816/2178157137,v1:172.21.15.148:6817/2178157137]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-07-03T15:50:27.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:27 smithi028 ceph-mon[92858]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-07-03T15:50:27.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:27 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:27.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:27 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:27.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:27 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:50:27.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:27 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:50:27.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:27 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:27.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:27 smithi028 ceph-mon[92858]: from='osd.5 ' entity='osd.5' 2024-07-03T15:50:27.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:27 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:50:27.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:27 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:50:27.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:27 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:50:27.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:27 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:50:27.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:27 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:50:27.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:27 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:27.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:27 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-07-03T15:50:27.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:27 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-07-03T15:50:27.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:27 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-07-03T15:50:27.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:27 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-07-03T15:50:27.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:27 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-07-03T15:50:27.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:27 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-07-03T15:50:27.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:27 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-07-03T15:50:27.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:27 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-07-03T15:50:27.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:27 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-07-03T15:50:27.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:27 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-07-03T15:50:27.624 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:27 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-07-03T15:50:27.625 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:27 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-07-03T15:50:27.625 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:27 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-07-03T15:50:28.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:28 smithi028 ceph-mon[92858]: Upgrade: Setting container_image for all osd 2024-07-03T15:50:28.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:28 smithi028 ceph-mon[92858]: Upgrade: Setting require_osd_release to 19 squid 2024-07-03T15:50:28.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:28 smithi028 ceph-mon[92858]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-07-03T15:50:28.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:28 smithi028 ceph-mon[92858]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-07-03T15:50:28.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:28 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-07-03T15:50:28.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:28 smithi028 ceph-mon[92858]: osd.5 [v2:172.21.15.148:6816/2178157137,v1:172.21.15.148:6817/2178157137] boot 2024-07-03T15:50:28.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:28 smithi028 ceph-mon[92858]: osdmap e77: 6 total, 6 up, 6 in 2024-07-03T15:50:28.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:28 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-03T15:50:28.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:28 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:28.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:28 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-07-03T15:50:28.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:28 smithi148 ceph-mon[80490]: Upgrade: Setting container_image for all osd 2024-07-03T15:50:28.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:28 smithi148 ceph-mon[80490]: Upgrade: Setting require_osd_release to 19 squid 2024-07-03T15:50:28.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:28 smithi148 ceph-mon[80490]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-07-03T15:50:28.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:28 smithi148 ceph-mon[80490]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-07-03T15:50:28.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:28 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-07-03T15:50:28.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:28 smithi148 ceph-mon[80490]: osd.5 [v2:172.21.15.148:6816/2178157137,v1:172.21.15.148:6817/2178157137] boot 2024-07-03T15:50:28.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:28 smithi148 ceph-mon[80490]: osdmap e77: 6 total, 6 up, 6 in 2024-07-03T15:50:28.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:28 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-03T15:50:28.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:28 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:28.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:28 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-07-03T15:50:29.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:29 smithi028 ceph-mon[92858]: pgmap v154: 65 pgs: 16 active+undersized, 12 active+undersized+degraded, 37 active+clean; 193 MiB data, 787 MiB used, 536 GiB / 536 GiB avail; 29/219 objects degraded (13.242%) 2024-07-03T15:50:29.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:29 smithi028 ceph-mon[92858]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-07-03T15:50:29.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:29 smithi028 ceph-mon[92858]: osdmap e78: 6 total, 6 up, 6 in 2024-07-03T15:50:29.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:29 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-07-03T15:50:29.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:29 smithi028 ceph-mon[92858]: fsmap cephfs:1 {0=cephfs.smithi028.rhpvve=up:active} 2 up:standby 2024-07-03T15:50:29.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:29 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:50:29.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:29 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:50:29.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:29 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:50:29.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:29 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:29.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:29 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:50:29.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:29 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:50:29.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:29 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:50:29.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:29 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:50:29.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:29 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:50:29.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:29 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi028.rhpvve"]}]: dispatch 2024-07-03T15:50:29.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:29 smithi148 ceph-mon[80490]: pgmap v154: 65 pgs: 16 active+undersized, 12 active+undersized+degraded, 37 active+clean; 193 MiB data, 787 MiB used, 536 GiB / 536 GiB avail; 29/219 objects degraded (13.242%) 2024-07-03T15:50:29.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:29 smithi148 ceph-mon[80490]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-07-03T15:50:29.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:29 smithi148 ceph-mon[80490]: osdmap e78: 6 total, 6 up, 6 in 2024-07-03T15:50:29.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:29 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-07-03T15:50:29.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:29 smithi148 ceph-mon[80490]: fsmap cephfs:1 {0=cephfs.smithi028.rhpvve=up:active} 2 up:standby 2024-07-03T15:50:29.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:29 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:50:29.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:29 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:50:29.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:29 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:50:29.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:29 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:29.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:29 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:50:29.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:29 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:50:29.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:29 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:50:29.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:29 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:50:29.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:29 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:50:29.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:29 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi028.rhpvve"]}]: dispatch 2024-07-03T15:50:30.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:30 smithi028 ceph-mon[92858]: Upgrade: It appears safe to stop mds.cephfs.smithi028.rhpvve 2024-07-03T15:50:30.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:30 smithi028 ceph-mon[92858]: mds.? [v2:172.21.15.28:6828/1224242160,v1:172.21.15.28:6829/1224242160] up:boot 2024-07-03T15:50:30.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:30 smithi028 ceph-mon[92858]: fsmap cephfs:1 {0=cephfs.smithi028.rhpvve=up:active} 3 up:standby 2024-07-03T15:50:30.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:30 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi028.wmbxew"}]: dispatch 2024-07-03T15:50:30.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:30 smithi148 ceph-mon[80490]: Upgrade: It appears safe to stop mds.cephfs.smithi028.rhpvve 2024-07-03T15:50:30.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:30 smithi148 ceph-mon[80490]: mds.? [v2:172.21.15.28:6828/1224242160,v1:172.21.15.28:6829/1224242160] up:boot 2024-07-03T15:50:30.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:30 smithi148 ceph-mon[80490]: fsmap cephfs:1 {0=cephfs.smithi028.rhpvve=up:active} 3 up:standby 2024-07-03T15:50:30.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:30 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi028.wmbxew"}]: dispatch 2024-07-03T15:50:31.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:31 smithi028 ceph-mon[92858]: pgmap v157: 65 pgs: 14 active+undersized, 11 active+undersized+degraded, 40 active+clean; 193 MiB data, 787 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 28/219 objects degraded (12.785%) 2024-07-03T15:50:31.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:31 smithi028 ceph-mon[92858]: Health check update: Degraded data redundancy: 28/219 objects degraded (12.785%), 11 pgs degraded (PG_DEGRADED) 2024-07-03T15:50:31.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:31 smithi028 ceph-mon[92858]: Upgrade: Updating mds.cephfs.smithi028.rhpvve 2024-07-03T15:50:31.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:31 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:31.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:31 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi028.rhpvve", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-03T15:50:31.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:31 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:50:31.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:31 smithi028 ceph-mon[92858]: Deploying daemon mds.cephfs.smithi028.rhpvve on smithi028 2024-07-03T15:50:31.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:31 smithi148 ceph-mon[80490]: pgmap v157: 65 pgs: 14 active+undersized, 11 active+undersized+degraded, 40 active+clean; 193 MiB data, 787 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 28/219 objects degraded (12.785%) 2024-07-03T15:50:31.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:31 smithi148 ceph-mon[80490]: Health check update: Degraded data redundancy: 28/219 objects degraded (12.785%), 11 pgs degraded (PG_DEGRADED) 2024-07-03T15:50:31.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:31 smithi148 ceph-mon[80490]: Upgrade: Updating mds.cephfs.smithi028.rhpvve 2024-07-03T15:50:31.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:31 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:31.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:31 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi028.rhpvve", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-03T15:50:31.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:31 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:50:31.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:31 smithi148 ceph-mon[80490]: Deploying daemon mds.cephfs.smithi028.rhpvve on smithi028 2024-07-03T15:50:32.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:32 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-mon-smithi028[92854]: 2024-07-03T15:50:32.503+0000 7f3b8b97a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-03T15:50:33.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:33 smithi028 ceph-mon[92858]: pgmap v158: 65 pgs: 4 active+undersized, 5 active+undersized+degraded, 56 active+clean; 193 MiB data, 791 MiB used, 536 GiB / 536 GiB avail; 16/219 objects degraded (7.306%) 2024-07-03T15:50:33.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:33 smithi028 ceph-mon[92858]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-07-03T15:50:33.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:33 smithi028 ceph-mon[92858]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-03T15:50:33.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:33 smithi028 ceph-mon[92858]: osdmap e79: 6 total, 6 up, 6 in 2024-07-03T15:50:33.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:33 smithi028 ceph-mon[92858]: Standby daemon mds.cephfs.smithi148.szomfl assigned to filesystem cephfs as rank 0 2024-07-03T15:50:33.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:33 smithi028 ceph-mon[92858]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-07-03T15:50:33.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:33 smithi028 ceph-mon[92858]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-07-03T15:50:33.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:33 smithi028 ceph-mon[92858]: fsmap cephfs:1/1 {0=cephfs.smithi148.szomfl=up:replay} 2 up:standby 2024-07-03T15:50:33.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:33 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi028.rhpvve"}]: dispatch 2024-07-03T15:50:33.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:33 smithi148 ceph-mon[80490]: pgmap v158: 65 pgs: 4 active+undersized, 5 active+undersized+degraded, 56 active+clean; 193 MiB data, 791 MiB used, 536 GiB / 536 GiB avail; 16/219 objects degraded (7.306%) 2024-07-03T15:50:33.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:33 smithi148 ceph-mon[80490]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-07-03T15:50:33.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:33 smithi148 ceph-mon[80490]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-03T15:50:33.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:33 smithi148 ceph-mon[80490]: osdmap e79: 6 total, 6 up, 6 in 2024-07-03T15:50:33.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:33 smithi148 ceph-mon[80490]: Standby daemon mds.cephfs.smithi148.szomfl assigned to filesystem cephfs as rank 0 2024-07-03T15:50:33.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:33 smithi148 ceph-mon[80490]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-07-03T15:50:33.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:33 smithi148 ceph-mon[80490]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-07-03T15:50:33.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:33 smithi148 ceph-mon[80490]: fsmap cephfs:1/1 {0=cephfs.smithi148.szomfl=up:replay} 2 up:standby 2024-07-03T15:50:33.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:33 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi028.rhpvve"}]: dispatch 2024-07-03T15:50:34.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:34 smithi148 ceph-mon[80490]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 16/219 objects degraded (7.306%), 5 pgs degraded) 2024-07-03T15:50:34.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:34 smithi028 ceph-mon[92858]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 16/219 objects degraded (7.306%), 5 pgs degraded) 2024-07-03T15:50:35.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:35 smithi148 ceph-mon[80490]: pgmap v160: 65 pgs: 65 active+clean; 193 MiB data, 791 MiB used, 536 GiB / 536 GiB avail; 9.3 MiB/s rd, 3 op/s 2024-07-03T15:50:35.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:35 smithi028 ceph-mon[92858]: pgmap v160: 65 pgs: 65 active+clean; 193 MiB data, 791 MiB used, 536 GiB / 536 GiB avail; 9.3 MiB/s rd, 3 op/s 2024-07-03T15:50:37.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:37 smithi148 ceph-mon[80490]: pgmap v161: 65 pgs: 65 active+clean; 193 MiB data, 791 MiB used, 536 GiB / 536 GiB avail; 7.6 MiB/s rd, 1 op/s 2024-07-03T15:50:37.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:37 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:37.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:37 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:50:37.734 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:37 smithi028 ceph-mon[92858]: pgmap v161: 65 pgs: 65 active+clean; 193 MiB data, 791 MiB used, 536 GiB / 536 GiB avail; 7.6 MiB/s rd, 1 op/s 2024-07-03T15:50:37.735 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:37 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:37.735 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:37 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:50:39.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:39 smithi028 ceph-mon[92858]: pgmap v162: 65 pgs: 65 active+clean; 193 MiB data, 791 MiB used, 536 GiB / 536 GiB avail; 9.0 MiB/s rd, 2 op/s 2024-07-03T15:50:39.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:39 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:39.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:39 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:39.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:39 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:50:39.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:39 smithi028 ceph-mon[92858]: mds.? [v2:172.21.15.28:6826/3127260162,v1:172.21.15.28:6827/3127260162] up:boot 2024-07-03T15:50:39.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:39 smithi028 ceph-mon[92858]: fsmap cephfs:1/1 {0=cephfs.smithi148.szomfl=up:replay} 3 up:standby 2024-07-03T15:50:39.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:39 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi028.rhpvve"}]: dispatch 2024-07-03T15:50:39.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:39 smithi148 ceph-mon[80490]: pgmap v162: 65 pgs: 65 active+clean; 193 MiB data, 791 MiB used, 536 GiB / 536 GiB avail; 9.0 MiB/s rd, 2 op/s 2024-07-03T15:50:39.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:39 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:39.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:39 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:39.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:39 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:50:39.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:39 smithi148 ceph-mon[80490]: mds.? [v2:172.21.15.28:6826/3127260162,v1:172.21.15.28:6827/3127260162] up:boot 2024-07-03T15:50:39.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:39 smithi148 ceph-mon[80490]: fsmap cephfs:1/1 {0=cephfs.smithi148.szomfl=up:replay} 3 up:standby 2024-07-03T15:50:39.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:39 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi028.rhpvve"}]: dispatch 2024-07-03T15:50:41.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:41 smithi028 ceph-mon[92858]: pgmap v163: 65 pgs: 65 active+clean; 193 MiB data, 791 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 4 op/s 2024-07-03T15:50:41.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:41 smithi028 ceph-mon[92858]: reconnect by client.14514 v1:172.21.15.148:0/4158297337 after 0 2024-07-03T15:50:41.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:41 smithi028 ceph-mon[92858]: reconnect by client.24301 v1:172.21.15.28:0/1387201540 after 0 2024-07-03T15:50:41.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:41 smithi028 ceph-mon[92858]: mds.? [v2:172.21.15.148:6826/4171231858,v1:172.21.15.148:6827/4171231858] up:reconnect 2024-07-03T15:50:41.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:41 smithi028 ceph-mon[92858]: fsmap cephfs:1/1 {0=cephfs.smithi148.szomfl=up:reconnect} 3 up:standby 2024-07-03T15:50:41.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:41 smithi148 ceph-mon[80490]: pgmap v163: 65 pgs: 65 active+clean; 193 MiB data, 791 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 4 op/s 2024-07-03T15:50:41.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:41 smithi148 ceph-mon[80490]: reconnect by client.14514 v1:172.21.15.148:0/4158297337 after 0 2024-07-03T15:50:41.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:41 smithi148 ceph-mon[80490]: reconnect by client.24301 v1:172.21.15.28:0/1387201540 after 0 2024-07-03T15:50:41.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:41 smithi148 ceph-mon[80490]: mds.? [v2:172.21.15.148:6826/4171231858,v1:172.21.15.148:6827/4171231858] up:reconnect 2024-07-03T15:50:41.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:41 smithi148 ceph-mon[80490]: fsmap cephfs:1/1 {0=cephfs.smithi148.szomfl=up:reconnect} 3 up:standby 2024-07-03T15:50:42.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:42 smithi028 ceph-mon[92858]: mds.? [v2:172.21.15.148:6826/4171231858,v1:172.21.15.148:6827/4171231858] up:rejoin 2024-07-03T15:50:42.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:42 smithi028 ceph-mon[92858]: fsmap cephfs:1/1 {0=cephfs.smithi148.szomfl=up:rejoin} 3 up:standby 2024-07-03T15:50:42.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:42 smithi028 ceph-mon[92858]: daemon mds.cephfs.smithi148.szomfl is now active in filesystem cephfs as rank 0 2024-07-03T15:50:42.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:42 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:42.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:42 smithi148 ceph-mon[80490]: mds.? [v2:172.21.15.148:6826/4171231858,v1:172.21.15.148:6827/4171231858] up:rejoin 2024-07-03T15:50:42.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:42 smithi148 ceph-mon[80490]: fsmap cephfs:1/1 {0=cephfs.smithi148.szomfl=up:rejoin} 3 up:standby 2024-07-03T15:50:42.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:42 smithi148 ceph-mon[80490]: daemon mds.cephfs.smithi148.szomfl is now active in filesystem cephfs as rank 0 2024-07-03T15:50:42.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:42 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:43.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:43 smithi028 ceph-mon[92858]: pgmap v164: 65 pgs: 65 active+clean; 193 MiB data, 791 MiB used, 536 GiB / 536 GiB avail; 19 MiB/s rd, 102 B/s wr, 6 op/s 2024-07-03T15:50:43.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:43 smithi028 ceph-mon[92858]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-07-03T15:50:43.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:43 smithi028 ceph-mon[92858]: Cluster is now healthy 2024-07-03T15:50:43.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:43 smithi028 ceph-mon[92858]: mds.? [v2:172.21.15.148:6826/4171231858,v1:172.21.15.148:6827/4171231858] up:active 2024-07-03T15:50:43.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:43 smithi028 ceph-mon[92858]: fsmap cephfs:1 {0=cephfs.smithi148.szomfl=up:active} 3 up:standby 2024-07-03T15:50:43.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:43 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:43.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:43 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:43.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:43 smithi148 ceph-mon[80490]: pgmap v164: 65 pgs: 65 active+clean; 193 MiB data, 791 MiB used, 536 GiB / 536 GiB avail; 19 MiB/s rd, 102 B/s wr, 6 op/s 2024-07-03T15:50:43.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:43 smithi148 ceph-mon[80490]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-07-03T15:50:43.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:43 smithi148 ceph-mon[80490]: Cluster is now healthy 2024-07-03T15:50:43.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:43 smithi148 ceph-mon[80490]: mds.? [v2:172.21.15.148:6826/4171231858,v1:172.21.15.148:6827/4171231858] up:active 2024-07-03T15:50:43.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:43 smithi148 ceph-mon[80490]: fsmap cephfs:1 {0=cephfs.smithi148.szomfl=up:active} 3 up:standby 2024-07-03T15:50:43.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:43 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:43.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:43 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:45.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:44 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:45.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:44 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:45.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:44 smithi148 ceph-mon[80490]: pgmap v165: 65 pgs: 65 active+clean; 193 MiB data, 791 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 440 B/s wr, 8 op/s 2024-07-03T15:50:45.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:44 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:45.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:44 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:45.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:44 smithi028 ceph-mon[92858]: pgmap v165: 65 pgs: 65 active+clean; 193 MiB data, 791 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 440 B/s wr, 8 op/s 2024-07-03T15:50:46.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.211+0000 7fed31f76640 1 -- 172.21.15.28:0/1432422717 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed2c0fd630 msgr2=0x7fed2c0fdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:46.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.211+0000 7fed31f76640 1 --2- 172.21.15.28:0/1432422717 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed2c0fd630 0x7fed2c0fdaa0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fed200048d0 tx=0x7fed20009da0 comp rx=0 tx=0).stop 2024-07-03T15:50:46.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.212+0000 7fed31f76640 1 -- 172.21.15.28:0/1432422717 shutdown_connections 2024-07-03T15:50:46.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.212+0000 7fed31f76640 1 --2- 172.21.15.28:0/1432422717 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed2c0fd630 0x7fed2c0fdaa0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:46.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.212+0000 7fed31f76640 1 --2- 172.21.15.28:0/1432422717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed2c0fc360 0x7fed2c0fc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:46.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.212+0000 7fed31f76640 1 -- 172.21.15.28:0/1432422717 >> 172.21.15.28:0/1432422717 conn(0x7fed2c0f7b40 msgr2=0x7fed2c0f9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:50:46.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.212+0000 7fed31f76640 1 -- 172.21.15.28:0/1432422717 shutdown_connections 2024-07-03T15:50:46.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.212+0000 7fed31f76640 1 -- 172.21.15.28:0/1432422717 wait complete. 2024-07-03T15:50:46.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.212+0000 7fed31f76640 1 Processor -- start 2024-07-03T15:50:46.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.212+0000 7fed31f76640 1 -- start start 2024-07-03T15:50:46.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.213+0000 7fed31f76640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed2c0fc360 0x7fed2c10cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:50:46.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.213+0000 7fed31f76640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed2c0fd630 0x7fed2c10d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:50:46.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.213+0000 7fed31f76640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c10e390 con 0x7fed2c0fc360 2024-07-03T15:50:46.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.213+0000 7fed31f76640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c10e500 con 0x7fed2c0fd630 2024-07-03T15:50:46.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.213+0000 7fed2b7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed2c0fc360 0x7fed2c10cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:50:46.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.213+0000 7fed2b7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed2c0fc360 0x7fed2c10cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59448/0 (socket says 172.21.15.28:59448) 2024-07-03T15:50:46.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.213+0000 7fed2b7fe640 1 -- 172.21.15.28:0/2334441987 learned_addr learned my addr 172.21.15.28:0/2334441987 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:50:46.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.213+0000 7fed2affd640 1 --2- 172.21.15.28:0/2334441987 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed2c0fd630 0x7fed2c10d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:50:46.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.213+0000 7fed2b7fe640 1 -- 172.21.15.28:0/2334441987 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed2c0fd630 msgr2=0x7fed2c10d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:46.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.213+0000 7fed2b7fe640 1 --2- 172.21.15.28:0/2334441987 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed2c0fd630 0x7fed2c10d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:46.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.213+0000 7fed2b7fe640 1 -- 172.21.15.28:0/2334441987 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed20005040 con 0x7fed2c0fc360 2024-07-03T15:50:46.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.213+0000 7fed2affd640 1 --2- 172.21.15.28:0/2334441987 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed2c0fd630 0x7fed2c10d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-03T15:50:46.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.214+0000 7fed2b7fe640 1 --2- 172.21.15.28:0/2334441987 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed2c0fc360 0x7fed2c10cfe0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fed1c00b790 tx=0x7fed1c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:50:46.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.214+0000 7fed13fff640 1 -- 172.21.15.28:0/2334441987 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed1c00c700 con 0x7fed2c0fc360 2024-07-03T15:50:46.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.214+0000 7fed31f76640 1 -- 172.21.15.28:0/2334441987 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed2c10e7e0 con 0x7fed2c0fc360 2024-07-03T15:50:46.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.214+0000 7fed13fff640 1 -- 172.21.15.28:0/2334441987 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed1c00cd30 con 0x7fed2c0fc360 2024-07-03T15:50:46.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.214+0000 7fed13fff640 1 -- 172.21.15.28:0/2334441987 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed1c013370 con 0x7fed2c0fc360 2024-07-03T15:50:46.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.214+0000 7fed31f76640 1 -- 172.21.15.28:0/2334441987 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed2c10ed00 con 0x7fed2c0fc360 2024-07-03T15:50:46.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.215+0000 7fed31f76640 1 -- 172.21.15.28:0/2334441987 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed18005240 con 0x7fed2c0fc360 2024-07-03T15:50:46.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.216+0000 7fed13fff640 1 -- 172.21.15.28:0/2334441987 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fed1c012070 con 0x7fed2c0fc360 2024-07-03T15:50:46.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.216+0000 7fed13fff640 1 --2- 172.21.15.28:0/2334441987 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fed1406e0b0 0x7fed14070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:50:46.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.216+0000 7fed13fff640 1 -- 172.21.15.28:0/2334441987 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fed1c08d410 con 0x7fed2c0fc360 2024-07-03T15:50:46.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.216+0000 7fed2affd640 1 --2- 172.21.15.28:0/2334441987 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fed1406e0b0 0x7fed14070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:50:46.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.217+0000 7fed2affd640 1 --2- 172.21.15.28:0/2334441987 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fed1406e0b0 0x7fed14070560 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fed20000c00 tx=0x7fed20035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:50:46.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.221+0000 7fed13fff640 1 -- 172.21.15.28:0/2334441987 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fed1c0567c0 con 0x7fed2c0fc360 2024-07-03T15:50:46.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.425+0000 7fed31f76640 1 -- 172.21.15.28:0/2334441987 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fed18000bf0 con 0x7fed1406e0b0 2024-07-03T15:50:46.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.426+0000 7fed13fff640 1 -- 172.21.15.28:0/2334441987 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fed18000bf0 con 0x7fed1406e0b0 2024-07-03T15:50:46.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.428+0000 7fed31f76640 1 -- 172.21.15.28:0/2334441987 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fed1406e0b0 msgr2=0x7fed14070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:46.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.428+0000 7fed31f76640 1 --2- 172.21.15.28:0/2334441987 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fed1406e0b0 0x7fed14070560 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fed20000c00 tx=0x7fed20035040 comp rx=0 tx=0).stop 2024-07-03T15:50:46.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.428+0000 7fed31f76640 1 -- 172.21.15.28:0/2334441987 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed2c0fc360 msgr2=0x7fed2c10cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:46.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.428+0000 7fed31f76640 1 --2- 172.21.15.28:0/2334441987 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed2c0fc360 0x7fed2c10cfe0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fed1c00b790 tx=0x7fed1c0091d0 comp rx=0 tx=0).stop 2024-07-03T15:50:46.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.428+0000 7fed31f76640 1 -- 172.21.15.28:0/2334441987 shutdown_connections 2024-07-03T15:50:46.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.429+0000 7fed31f76640 1 --2- 172.21.15.28:0/2334441987 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fed1406e0b0 0x7fed14070560 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:46.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.429+0000 7fed31f76640 1 --2- 172.21.15.28:0/2334441987 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed2c0fd630 0x7fed2c10d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:46.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.429+0000 7fed31f76640 1 --2- 172.21.15.28:0/2334441987 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed2c0fc360 0x7fed2c10cfe0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:46.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.429+0000 7fed31f76640 1 -- 172.21.15.28:0/2334441987 >> 172.21.15.28:0/2334441987 conn(0x7fed2c0f7b40 msgr2=0x7fed2c0f9230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:50:46.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.429+0000 7fed31f76640 1 -- 172.21.15.28:0/2334441987 shutdown_connections 2024-07-03T15:50:46.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.429+0000 7fed31f76640 1 -- 172.21.15.28:0/2334441987 wait complete. 2024-07-03T15:50:46.441 INFO:teuthology.orchestra.run.smithi028.stdout:true 2024-07-03T15:50:46.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.513+0000 7f02f4e2d640 1 -- 172.21.15.28:0/3185138240 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02f00659d0 msgr2=0x7f02f00fefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:46.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.513+0000 7f02f4e2d640 1 --2- 172.21.15.28:0/3185138240 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02f00659d0 0x7f02f00fefd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f02e00048d0 tx=0x7f02e0009da0 comp rx=0 tx=0).stop 2024-07-03T15:50:46.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.513+0000 7f02f4e2d640 1 -- 172.21.15.28:0/3185138240 shutdown_connections 2024-07-03T15:50:46.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.513+0000 7f02f4e2d640 1 --2- 172.21.15.28:0/3185138240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02f00ff510 0x7f02f01018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:46.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.513+0000 7f02f4e2d640 1 --2- 172.21.15.28:0/3185138240 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02f00659d0 0x7f02f00fefd0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:46.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.513+0000 7f02f4e2d640 1 -- 172.21.15.28:0/3185138240 >> 172.21.15.28:0/3185138240 conn(0x7f02f00f6a20 msgr2=0x7f02f00f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:50:46.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.513+0000 7f02f4e2d640 1 -- 172.21.15.28:0/3185138240 shutdown_connections 2024-07-03T15:50:46.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.513+0000 7f02f4e2d640 1 -- 172.21.15.28:0/3185138240 wait complete. 2024-07-03T15:50:46.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.514+0000 7f02f4e2d640 1 Processor -- start 2024-07-03T15:50:46.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.514+0000 7f02f4e2d640 1 -- start start 2024-07-03T15:50:46.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.515+0000 7f02f4e2d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02f00659d0 0x7f02f010cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:50:46.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.515+0000 7f02f4e2d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02f00ff510 0x7f02f010d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:50:46.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.515+0000 7f02f4e2d640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f010db30 con 0x7f02f00ff510 2024-07-03T15:50:46.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.515+0000 7f02f4e2d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f010e350 con 0x7f02f00659d0 2024-07-03T15:50:46.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.515+0000 7f02e7fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02f00ff510 0x7f02f010d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:50:46.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.515+0000 7f02e7fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02f00ff510 0x7f02f010d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59456/0 (socket says 172.21.15.28:59456) 2024-07-03T15:50:46.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.515+0000 7f02e7fff640 1 -- 172.21.15.28:0/1952385333 learned_addr learned my addr 172.21.15.28:0/1952385333 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:50:46.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.515+0000 7f02ee59c640 1 --2- 172.21.15.28:0/1952385333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02f00659d0 0x7f02f010cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:50:46.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.515+0000 7f02e7fff640 1 -- 172.21.15.28:0/1952385333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02f00659d0 msgr2=0x7f02f010cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:46.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.515+0000 7f02e7fff640 1 --2- 172.21.15.28:0/1952385333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02f00659d0 0x7f02f010cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:46.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.515+0000 7f02e7fff640 1 -- 172.21.15.28:0/1952385333 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e0005040 con 0x7f02f00ff510 2024-07-03T15:50:46.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.515+0000 7f02e7fff640 1 --2- 172.21.15.28:0/1952385333 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02f00ff510 0x7f02f010d4e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f02d8002870 tx=0x7f02d8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:50:46.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.516+0000 7f02e6ffd640 1 -- 172.21.15.28:0/1952385333 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02d8007c30 con 0x7f02f00ff510 2024-07-03T15:50:46.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.516+0000 7f02e6ffd640 1 -- 172.21.15.28:0/1952385333 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f02d800f040 con 0x7f02f00ff510 2024-07-03T15:50:46.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.516+0000 7f02f4e2d640 1 -- 172.21.15.28:0/1952385333 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02f010e5e0 con 0x7f02f00ff510 2024-07-03T15:50:46.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.516+0000 7f02e6ffd640 1 -- 172.21.15.28:0/1952385333 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02d800da90 con 0x7f02f00ff510 2024-07-03T15:50:46.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.516+0000 7f02f4e2d640 1 -- 172.21.15.28:0/1952385333 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02f010eb80 con 0x7f02f00ff510 2024-07-03T15:50:46.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.517+0000 7f02f4e2d640 1 -- 172.21.15.28:0/1952385333 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02f00fa6a0 con 0x7f02f00ff510 2024-07-03T15:50:46.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.518+0000 7f02e6ffd640 1 -- 172.21.15.28:0/1952385333 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f02d800dd00 con 0x7f02f00ff510 2024-07-03T15:50:46.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.518+0000 7f02e6ffd640 1 --2- 172.21.15.28:0/1952385333 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f02dc06e0b0 0x7f02dc070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:50:46.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.519+0000 7f02e6ffd640 1 -- 172.21.15.28:0/1952385333 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f02d808d1a0 con 0x7f02f00ff510 2024-07-03T15:50:46.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.519+0000 7f02ee59c640 1 --2- 172.21.15.28:0/1952385333 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f02dc06e0b0 0x7f02dc070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:50:46.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.519+0000 7f02ee59c640 1 --2- 172.21.15.28:0/1952385333 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f02dc06e0b0 0x7f02dc070560 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f02e0000c00 tx=0x7f02e0035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:50:46.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.524+0000 7f02e6ffd640 1 -- 172.21.15.28:0/1952385333 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f02d8056600 con 0x7f02f00ff510 2024-07-03T15:50:46.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.740+0000 7f02f4e2d640 1 -- 172.21.15.28:0/1952385333 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f02f010ef00 con 0x7f02dc06e0b0 2024-07-03T15:50:46.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.742+0000 7f02e6ffd640 1 -- 172.21.15.28:0/1952385333 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f02f010ef00 con 0x7f02dc06e0b0 2024-07-03T15:50:46.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.744+0000 7f02f4e2d640 1 -- 172.21.15.28:0/1952385333 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f02dc06e0b0 msgr2=0x7f02dc070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:46.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.744+0000 7f02f4e2d640 1 --2- 172.21.15.28:0/1952385333 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f02dc06e0b0 0x7f02dc070560 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f02e0000c00 tx=0x7f02e0035040 comp rx=0 tx=0).stop 2024-07-03T15:50:46.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.744+0000 7f02f4e2d640 1 -- 172.21.15.28:0/1952385333 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02f00ff510 msgr2=0x7f02f010d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:46.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.744+0000 7f02f4e2d640 1 --2- 172.21.15.28:0/1952385333 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02f00ff510 0x7f02f010d4e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f02d8002870 tx=0x7f02d8004600 comp rx=0 tx=0).stop 2024-07-03T15:50:46.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.744+0000 7f02f4e2d640 1 -- 172.21.15.28:0/1952385333 shutdown_connections 2024-07-03T15:50:46.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.744+0000 7f02f4e2d640 1 --2- 172.21.15.28:0/1952385333 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f02dc06e0b0 0x7f02dc070560 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:46.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.744+0000 7f02f4e2d640 1 --2- 172.21.15.28:0/1952385333 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02f00ff510 0x7f02f010d4e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:46.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.744+0000 7f02f4e2d640 1 --2- 172.21.15.28:0/1952385333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02f00659d0 0x7f02f010cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:46.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.744+0000 7f02f4e2d640 1 -- 172.21.15.28:0/1952385333 >> 172.21.15.28:0/1952385333 conn(0x7f02f00f6a20 msgr2=0x7f02f00f6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:50:46.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.744+0000 7f02f4e2d640 1 -- 172.21.15.28:0/1952385333 shutdown_connections 2024-07-03T15:50:46.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.744+0000 7f02f4e2d640 1 -- 172.21.15.28:0/1952385333 wait complete. 2024-07-03T15:50:46.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.825+0000 7fb232584640 1 -- 172.21.15.28:0/3271223005 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb22c0fb240 msgr2=0x7fb22c0fb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:46.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.825+0000 7fb232584640 1 --2- 172.21.15.28:0/3271223005 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb22c0fb240 0x7fb22c0fb650 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb21c0048d0 tx=0x7fb21c009da0 comp rx=0 tx=0).stop 2024-07-03T15:50:46.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.826+0000 7fb232584640 1 -- 172.21.15.28:0/3271223005 shutdown_connections 2024-07-03T15:50:46.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.826+0000 7fb232584640 1 --2- 172.21.15.28:0/3271223005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb22c0fc510 0x7fb22c0fc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:46.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.826+0000 7fb232584640 1 --2- 172.21.15.28:0/3271223005 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb22c0fb240 0x7fb22c0fb650 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:46.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.826+0000 7fb232584640 1 -- 172.21.15.28:0/3271223005 >> 172.21.15.28:0/3271223005 conn(0x7fb22c0f6a20 msgr2=0x7fb22c0f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:50:46.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.826+0000 7fb232584640 1 -- 172.21.15.28:0/3271223005 shutdown_connections 2024-07-03T15:50:46.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.826+0000 7fb232584640 1 -- 172.21.15.28:0/3271223005 wait complete. 2024-07-03T15:50:46.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.826+0000 7fb232584640 1 Processor -- start 2024-07-03T15:50:46.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.826+0000 7fb232584640 1 -- start start 2024-07-03T15:50:46.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.827+0000 7fb232584640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb22c0fb240 0x7fb22c10f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:50:46.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.827+0000 7fb232584640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb22c0fc510 0x7fb22c10f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:50:46.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.827+0000 7fb232584640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb22c1105c0 con 0x7fb22c0fc510 2024-07-03T15:50:46.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.827+0000 7fb232584640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb22c110730 con 0x7fb22c0fb240 2024-07-03T15:50:46.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.827+0000 7fb22b7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb22c0fc510 0x7fb22c10f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:50:46.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.827+0000 7fb22b7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb22c0fc510 0x7fb22c10f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59470/0 (socket says 172.21.15.28:59470) 2024-07-03T15:50:46.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.827+0000 7fb22b7fe640 1 -- 172.21.15.28:0/2857763784 learned_addr learned my addr 172.21.15.28:0/2857763784 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:50:46.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.827+0000 7fb22bfff640 1 --2- 172.21.15.28:0/2857763784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb22c0fb240 0x7fb22c10f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:50:46.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.827+0000 7fb22b7fe640 1 -- 172.21.15.28:0/2857763784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb22c0fb240 msgr2=0x7fb22c10f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:46.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.827+0000 7fb22b7fe640 1 --2- 172.21.15.28:0/2857763784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb22c0fb240 0x7fb22c10f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:46.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.827+0000 7fb22b7fe640 1 -- 172.21.15.28:0/2857763784 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb21c005040 con 0x7fb22c0fc510 2024-07-03T15:50:46.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.827+0000 7fb22b7fe640 1 --2- 172.21.15.28:0/2857763784 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb22c0fc510 0x7fb22c10f7a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb22000b790 tx=0x7fb2200091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:50:46.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.827+0000 7fb228ff9640 1 -- 172.21.15.28:0/2857763784 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb22000c700 con 0x7fb22c0fc510 2024-07-03T15:50:46.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.828+0000 7fb228ff9640 1 -- 172.21.15.28:0/2857763784 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb22000cd30 con 0x7fb22c0fc510 2024-07-03T15:50:46.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.828+0000 7fb232584640 1 -- 172.21.15.28:0/2857763784 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb22c110a10 con 0x7fb22c0fc510 2024-07-03T15:50:46.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.828+0000 7fb228ff9640 1 -- 172.21.15.28:0/2857763784 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb220013370 con 0x7fb22c0fc510 2024-07-03T15:50:46.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.828+0000 7fb232584640 1 -- 172.21.15.28:0/2857763784 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb22c110f30 con 0x7fb22c0fc510 2024-07-03T15:50:46.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.829+0000 7fb232584640 1 -- 172.21.15.28:0/2857763784 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb22c0fa6a0 con 0x7fb22c0fc510 2024-07-03T15:50:46.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.829+0000 7fb228ff9640 1 -- 172.21.15.28:0/2857763784 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb220012070 con 0x7fb22c0fc510 2024-07-03T15:50:46.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.830+0000 7fb228ff9640 1 --2- 172.21.15.28:0/2857763784 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fb21406e0b0 0x7fb214070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:50:46.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.830+0000 7fb228ff9640 1 -- 172.21.15.28:0/2857763784 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb22008d4d0 con 0x7fb22c0fc510 2024-07-03T15:50:46.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.830+0000 7fb22bfff640 1 --2- 172.21.15.28:0/2857763784 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fb21406e0b0 0x7fb214070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:50:46.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.830+0000 7fb22bfff640 1 --2- 172.21.15.28:0/2857763784 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fb21406e0b0 0x7fb214070560 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb21c000c00 tx=0x7fb21c035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:50:46.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:46.834+0000 7fb228ff9640 1 -- 172.21.15.28:0/2857763784 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fb220056800 con 0x7fb22c0fc510 2024-07-03T15:50:47.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.031+0000 7fb232584640 1 -- 172.21.15.28:0/2857763784 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb22c103e10 con 0x7fb21406e0b0 2024-07-03T15:50:47.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.040+0000 7fb228ff9640 1 -- 172.21.15.28:0/2857763784 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fb22c103e10 con 0x7fb21406e0b0 2024-07-03T15:50:47.044 INFO:teuthology.orchestra.run.smithi028.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-03T15:50:47.044 INFO:teuthology.orchestra.run.smithi028.stdout:alertmanager.smithi028 smithi028 *:9093,9094 running (6m) 3s ago 14m 24.6M - 0.25.0 c8568f914cd2 1425315a0de9 2024-07-03T15:50:47.044 INFO:teuthology.orchestra.run.smithi028.stdout:crash.smithi028 smithi028 running (3m) 3s ago 14m 6895k - 19.0.0-2550-gc00506dc 9d49405603e8 c62c2f45cabe 2024-07-03T15:50:47.044 INFO:teuthology.orchestra.run.smithi028.stdout:crash.smithi148 smithi148 running (3m) 23s ago 13m 6895k - 19.0.0-2550-gc00506dc 9d49405603e8 b1318dcaacb6 2024-07-03T15:50:47.044 INFO:teuthology.orchestra.run.smithi028.stdout:grafana.smithi028 smithi028 *:3000 running (6m) 3s ago 14m 82.1M - 9.4.7 954c08fa6188 f9f17c92cfea 2024-07-03T15:50:47.044 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi028.rhpvve smithi028 running (9s) 3s ago 9m 13.0M - 19.0.0-2550-gc00506dc 9d49405603e8 5d528ea2a2f0 2024-07-03T15:50:47.044 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi028.wmbxew smithi028 running (8m) 3s ago 8m 10.8M - 17.2.7-1047-gd4b063ab 4ca399d9e65d b3bf736998f9 2024-07-03T15:50:47.045 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi148.szomfl smithi148 running (8m) 23s ago 8m 16.5M - 17.2.7-1047-gd4b063ab 4ca399d9e65d d94f5c95fa8d 2024-07-03T15:50:47.045 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi148.xgpuld smithi148 running (8m) 23s ago 8m 14.0M - 17.2.7-1047-gd4b063ab 4ca399d9e65d fc70360a8de0 2024-07-03T15:50:47.045 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.smithi028.vtqrtz smithi028 *:8443,9283,8765 running (5m) 3s ago 15m 503M - 19.0.0-2550-gc00506dc 9d49405603e8 14d0363c5b77 2024-07-03T15:50:47.045 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.smithi148.rjzsar smithi148 *:8443,9283,8765 running (4m) 23s ago 13m 441M - 19.0.0-2550-gc00506dc 9d49405603e8 c0193aa8a5dd 2024-07-03T15:50:47.045 INFO:teuthology.orchestra.run.smithi028.stdout:mon.smithi028 smithi028 running (4m) 3s ago 15m 51.5M 2048M 19.0.0-2550-gc00506dc 9d49405603e8 5093f7bf47d7 2024-07-03T15:50:47.045 INFO:teuthology.orchestra.run.smithi028.stdout:mon.smithi148 smithi148 running (4m) 23s ago 13m 39.7M 2048M 19.0.0-2550-gc00506dc 9d49405603e8 d338d2c648da 2024-07-03T15:50:47.045 INFO:teuthology.orchestra.run.smithi028.stdout:node-exporter.smithi028 smithi028 *:9100 running (6m) 3s ago 14m 15.6M - 1.5.0 0da6a335fe13 91ca1f49de0e 2024-07-03T15:50:47.045 INFO:teuthology.orchestra.run.smithi028.stdout:node-exporter.smithi148 smithi148 *:9100 running (5m) 23s ago 13m 16.2M - 1.5.0 0da6a335fe13 1eaaf7b1b561 2024-07-03T15:50:47.045 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0 smithi028 running (3m) 3s ago 12m 109M 1558M 19.0.0-2550-gc00506dc 9d49405603e8 4454cecff8fc 2024-07-03T15:50:47.045 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1 smithi028 running (2m) 3s ago 12m 113M 1558M 19.0.0-2550-gc00506dc 9d49405603e8 af67f7088643 2024-07-03T15:50:47.045 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2 smithi028 running (2m) 3s ago 11m 97.0M 1558M 19.0.0-2550-gc00506dc 9d49405603e8 5d160a327dc2 2024-07-03T15:50:47.045 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3 smithi148 running (91s) 23s ago 11m 106M 2581M 19.0.0-2550-gc00506dc 9d49405603e8 1d622eba4b3b 2024-07-03T15:50:47.046 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4 smithi148 running (58s) 23s ago 10m 102M 2581M 19.0.0-2550-gc00506dc 9d49405603e8 546343d0f31f 2024-07-03T15:50:47.046 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5 smithi148 running (26s) 23s ago 10m 11.7M 2581M 19.0.0-2550-gc00506dc 9d49405603e8 a819fcc3b569 2024-07-03T15:50:47.046 INFO:teuthology.orchestra.run.smithi028.stdout:prometheus.smithi028 smithi028 *:9095 running (4m) 3s ago 13m 62.6M - 2.43.0 a07b618ecd1d 2b0eb2a95afd 2024-07-03T15:50:47.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.042+0000 7fb232584640 1 -- 172.21.15.28:0/2857763784 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fb21406e0b0 msgr2=0x7fb214070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:47.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.042+0000 7fb232584640 1 --2- 172.21.15.28:0/2857763784 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fb21406e0b0 0x7fb214070560 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb21c000c00 tx=0x7fb21c035040 comp rx=0 tx=0).stop 2024-07-03T15:50:47.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.042+0000 7fb232584640 1 -- 172.21.15.28:0/2857763784 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb22c0fc510 msgr2=0x7fb22c10f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:47.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.042+0000 7fb232584640 1 --2- 172.21.15.28:0/2857763784 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb22c0fc510 0x7fb22c10f7a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb22000b790 tx=0x7fb2200091d0 comp rx=0 tx=0).stop 2024-07-03T15:50:47.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.042+0000 7fb232584640 1 -- 172.21.15.28:0/2857763784 shutdown_connections 2024-07-03T15:50:47.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.042+0000 7fb232584640 1 --2- 172.21.15.28:0/2857763784 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fb21406e0b0 0x7fb214070560 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:47.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.042+0000 7fb232584640 1 --2- 172.21.15.28:0/2857763784 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb22c0fc510 0x7fb22c10f7a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:47.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.042+0000 7fb232584640 1 --2- 172.21.15.28:0/2857763784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb22c0fb240 0x7fb22c10f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:47.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.042+0000 7fb232584640 1 -- 172.21.15.28:0/2857763784 >> 172.21.15.28:0/2857763784 conn(0x7fb22c0f6a20 msgr2=0x7fb22c065380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:50:47.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.042+0000 7fb232584640 1 -- 172.21.15.28:0/2857763784 shutdown_connections 2024-07-03T15:50:47.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.042+0000 7fb232584640 1 -- 172.21.15.28:0/2857763784 wait complete. 2024-07-03T15:50:47.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.124+0000 7f138156b640 1 -- 172.21.15.28:0/215109805 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f137c0fb250 msgr2=0x7f137c0fb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:47.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.124+0000 7f138156b640 1 --2- 172.21.15.28:0/215109805 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f137c0fb250 0x7f137c0fb660 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f136c0048d0 tx=0x7f136c009da0 comp rx=0 tx=0).stop 2024-07-03T15:50:47.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.124+0000 7f138156b640 1 -- 172.21.15.28:0/215109805 shutdown_connections 2024-07-03T15:50:47.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.124+0000 7f138156b640 1 --2- 172.21.15.28:0/215109805 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f137c0fc520 0x7f137c0fc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:47.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.124+0000 7f138156b640 1 --2- 172.21.15.28:0/215109805 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f137c0fb250 0x7f137c0fb660 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:47.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.124+0000 7f138156b640 1 -- 172.21.15.28:0/215109805 >> 172.21.15.28:0/215109805 conn(0x7f137c0f6a10 msgr2=0x7f137c0f8e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:50:47.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.124+0000 7f138156b640 1 -- 172.21.15.28:0/215109805 shutdown_connections 2024-07-03T15:50:47.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.124+0000 7f138156b640 1 -- 172.21.15.28:0/215109805 wait complete. 2024-07-03T15:50:47.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.125+0000 7f138156b640 1 Processor -- start 2024-07-03T15:50:47.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.125+0000 7f138156b640 1 -- start start 2024-07-03T15:50:47.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.125+0000 7f138156b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f137c0fb250 0x7f137c10f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:50:47.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.125+0000 7f138156b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f137c0fc520 0x7f137c10f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:50:47.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.125+0000 7f138156b640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137c10fd20 con 0x7f137c0fc520 2024-07-03T15:50:47.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.125+0000 7f138156b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137c110540 con 0x7f137c0fb250 2024-07-03T15:50:47.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.125+0000 7f137a7fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f137c0fc520 0x7f137c10f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:50:47.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.125+0000 7f137a7fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f137c0fc520 0x7f137c10f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59486/0 (socket says 172.21.15.28:59486) 2024-07-03T15:50:47.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.125+0000 7f137a7fc640 1 -- 172.21.15.28:0/387096689 learned_addr learned my addr 172.21.15.28:0/387096689 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:50:47.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.125+0000 7f137affd640 1 --2- 172.21.15.28:0/387096689 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f137c0fb250 0x7f137c10f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:50:47.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.125+0000 7f137a7fc640 1 -- 172.21.15.28:0/387096689 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f137c0fb250 msgr2=0x7f137c10f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:47.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.125+0000 7f137a7fc640 1 --2- 172.21.15.28:0/387096689 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f137c0fb250 0x7f137c10f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:47.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.125+0000 7f137a7fc640 1 -- 172.21.15.28:0/387096689 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f136c005040 con 0x7f137c0fc520 2024-07-03T15:50:47.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.126+0000 7f137a7fc640 1 --2- 172.21.15.28:0/387096689 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f137c0fc520 0x7f137c10f6d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1370002a50 tx=0x7f1370004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:50:47.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.126+0000 7f13637fe640 1 -- 172.21.15.28:0/387096689 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1370007c30 con 0x7f137c0fc520 2024-07-03T15:50:47.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.126+0000 7f13637fe640 1 -- 172.21.15.28:0/387096689 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f137000f040 con 0x7f137c0fc520 2024-07-03T15:50:47.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.126+0000 7f138156b640 1 -- 172.21.15.28:0/387096689 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f137c1107d0 con 0x7f137c0fc520 2024-07-03T15:50:47.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.126+0000 7f13637fe640 1 -- 172.21.15.28:0/387096689 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f137000db50 con 0x7f137c0fc520 2024-07-03T15:50:47.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.126+0000 7f138156b640 1 -- 172.21.15.28:0/387096689 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f137c110d70 con 0x7f137c0fc520 2024-07-03T15:50:47.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.127+0000 7f138156b640 1 -- 172.21.15.28:0/387096689 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f137c0fa6b0 con 0x7f137c0fc520 2024-07-03T15:50:47.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.127+0000 7f13637fe640 1 -- 172.21.15.28:0/387096689 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f137001c400 con 0x7f137c0fc520 2024-07-03T15:50:47.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.128+0000 7f13637fe640 1 --2- 172.21.15.28:0/387096689 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f136406e0b0 0x7f1364070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:50:47.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.128+0000 7f13637fe640 1 -- 172.21.15.28:0/387096689 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f137008d470 con 0x7f137c0fc520 2024-07-03T15:50:47.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.128+0000 7f137affd640 1 --2- 172.21.15.28:0/387096689 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f136406e0b0 0x7f1364070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:50:47.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.128+0000 7f137affd640 1 --2- 172.21.15.28:0/387096689 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f136406e0b0 0x7f1364070560 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f136c02a530 tx=0x7f136c009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:50:47.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.132+0000 7f13637fe640 1 -- 172.21.15.28:0/387096689 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f13700567a0 con 0x7f137c0fc520 2024-07-03T15:50:47.374 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:47 smithi028 ceph-mon[92858]: pgmap v166: 65 pgs: 65 active+clean; 193 MiB data, 791 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 426 B/s wr, 6 op/s 2024-07-03T15:50:47.374 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:47 smithi028 ceph-mon[92858]: from='client.34380 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:50:47.374 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:47 smithi028 ceph-mon[92858]: from='client.34384 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:50:47.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.392+0000 7f138156b640 1 -- 172.21.15.28:0/387096689 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f137c103f00 con 0x7f137c0fc520 2024-07-03T15:50:47.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.393+0000 7f13637fe640 1 -- 172.21.15.28:0/387096689 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+834 (secure 0 0 0) 0x7f1370059e60 con 0x7f137c0fc520 2024-07-03T15:50:47.397 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-07-03T15:50:47.397 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": { 2024-07-03T15:50:47.397 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 2 2024-07-03T15:50:47.397 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:50:47.397 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": { 2024-07-03T15:50:47.397 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 2 2024-07-03T15:50:47.397 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:50:47.398 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": { 2024-07-03T15:50:47.398 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 6 2024-07-03T15:50:47.398 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:50:47.398 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": { 2024-07-03T15:50:47.398 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 3, 2024-07-03T15:50:47.398 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 1 2024-07-03T15:50:47.398 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:50:47.398 INFO:teuthology.orchestra.run.smithi028.stdout: "overall": { 2024-07-03T15:50:47.398 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 3, 2024-07-03T15:50:47.398 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 11 2024-07-03T15:50:47.398 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-07-03T15:50:47.398 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-07-03T15:50:47.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.395+0000 7f138156b640 1 -- 172.21.15.28:0/387096689 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f136406e0b0 msgr2=0x7f1364070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:47.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.395+0000 7f138156b640 1 --2- 172.21.15.28:0/387096689 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f136406e0b0 0x7f1364070560 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f136c02a530 tx=0x7f136c009da0 comp rx=0 tx=0).stop 2024-07-03T15:50:47.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.395+0000 7f138156b640 1 -- 172.21.15.28:0/387096689 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f137c0fc520 msgr2=0x7f137c10f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:47.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.395+0000 7f138156b640 1 --2- 172.21.15.28:0/387096689 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f137c0fc520 0x7f137c10f6d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1370002a50 tx=0x7f1370004600 comp rx=0 tx=0).stop 2024-07-03T15:50:47.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.395+0000 7f138156b640 1 -- 172.21.15.28:0/387096689 shutdown_connections 2024-07-03T15:50:47.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.395+0000 7f138156b640 1 --2- 172.21.15.28:0/387096689 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f136406e0b0 0x7f1364070560 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:47.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.395+0000 7f138156b640 1 --2- 172.21.15.28:0/387096689 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f137c0fc520 0x7f137c10f6d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:47.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.395+0000 7f138156b640 1 --2- 172.21.15.28:0/387096689 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f137c0fb250 0x7f137c10f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:47.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.395+0000 7f138156b640 1 -- 172.21.15.28:0/387096689 >> 172.21.15.28:0/387096689 conn(0x7f137c0f6a10 msgr2=0x7f137c0f80e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:50:47.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.395+0000 7f138156b640 1 -- 172.21.15.28:0/387096689 shutdown_connections 2024-07-03T15:50:47.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.395+0000 7f138156b640 1 -- 172.21.15.28:0/387096689 wait complete. 2024-07-03T15:50:47.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:47 smithi148 ceph-mon[80490]: pgmap v166: 65 pgs: 65 active+clean; 193 MiB data, 791 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 426 B/s wr, 6 op/s 2024-07-03T15:50:47.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:47 smithi148 ceph-mon[80490]: from='client.34380 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:50:47.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:47 smithi148 ceph-mon[80490]: from='client.34384 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:50:47.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.482+0000 7fa4503d5640 1 -- 172.21.15.28:0/3833501487 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4480ff4d0 msgr2=0x7fa4481018b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:47.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.482+0000 7fa4503d5640 1 --2- 172.21.15.28:0/3833501487 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4480ff4d0 0x7fa4481018b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa4380048d0 tx=0x7fa438009da0 comp rx=0 tx=0).stop 2024-07-03T15:50:47.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.482+0000 7fa4503d5640 1 -- 172.21.15.28:0/3833501487 shutdown_connections 2024-07-03T15:50:47.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.482+0000 7fa4503d5640 1 --2- 172.21.15.28:0/3833501487 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4480ff4d0 0x7fa4481018b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:47.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.482+0000 7fa4503d5640 1 --2- 172.21.15.28:0/3833501487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa448066520 0x7fa4480fef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:47.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.482+0000 7fa4503d5640 1 -- 172.21.15.28:0/3833501487 >> 172.21.15.28:0/3833501487 conn(0x7fa4480f6a20 msgr2=0x7fa4480f8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:50:47.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.482+0000 7fa4503d5640 1 -- 172.21.15.28:0/3833501487 shutdown_connections 2024-07-03T15:50:47.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.482+0000 7fa4503d5640 1 -- 172.21.15.28:0/3833501487 wait complete. 2024-07-03T15:50:47.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.483+0000 7fa4503d5640 1 Processor -- start 2024-07-03T15:50:47.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.483+0000 7fa4503d5640 1 -- start start 2024-07-03T15:50:47.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.483+0000 7fa4503d5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa448066520 0x7fa44810cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:50:47.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.483+0000 7fa4503d5640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4480ff4d0 0x7fa44810d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:50:47.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.483+0000 7fa4503d5640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44810db20 con 0x7fa4480ff4d0 2024-07-03T15:50:47.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.483+0000 7fa4503d5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44810e2f0 con 0x7fa448066520 2024-07-03T15:50:47.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.483+0000 7fa44d970640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4480ff4d0 0x7fa44810d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:50:47.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.483+0000 7fa44d970640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4480ff4d0 0x7fa44810d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59494/0 (socket says 172.21.15.28:59494) 2024-07-03T15:50:47.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.483+0000 7fa44d970640 1 -- 172.21.15.28:0/1902013713 learned_addr learned my addr 172.21.15.28:0/1902013713 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:50:47.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.483+0000 7fa44e171640 1 --2- 172.21.15.28:0/1902013713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa448066520 0x7fa44810cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:50:47.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.484+0000 7fa44d970640 1 -- 172.21.15.28:0/1902013713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa448066520 msgr2=0x7fa44810cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:47.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.484+0000 7fa44d970640 1 --2- 172.21.15.28:0/1902013713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa448066520 0x7fa44810cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:47.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.484+0000 7fa44d970640 1 -- 172.21.15.28:0/1902013713 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa438005040 con 0x7fa4480ff4d0 2024-07-03T15:50:47.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.484+0000 7fa44d970640 1 --2- 172.21.15.28:0/1902013713 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4480ff4d0 0x7fa44810d4d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa438009f80 tx=0x7fa438009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:50:47.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.484+0000 7fa43effd640 1 -- 172.21.15.28:0/1902013713 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa438038070 con 0x7fa4480ff4d0 2024-07-03T15:50:47.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.484+0000 7fa4503d5640 1 -- 172.21.15.28:0/1902013713 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa44810e520 con 0x7fa4480ff4d0 2024-07-03T15:50:47.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.484+0000 7fa43effd640 1 -- 172.21.15.28:0/1902013713 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa438032970 con 0x7fa4480ff4d0 2024-07-03T15:50:47.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.484+0000 7fa43effd640 1 -- 172.21.15.28:0/1902013713 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa438033860 con 0x7fa4480ff4d0 2024-07-03T15:50:47.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.484+0000 7fa4503d5640 1 -- 172.21.15.28:0/1902013713 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa44810ea60 con 0x7fa4480ff4d0 2024-07-03T15:50:47.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.485+0000 7fa4503d5640 1 -- 172.21.15.28:0/1902013713 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4480fa6e0 con 0x7fa4480ff4d0 2024-07-03T15:50:47.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.486+0000 7fa43effd640 1 -- 172.21.15.28:0/1902013713 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa438030020 con 0x7fa4480ff4d0 2024-07-03T15:50:47.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.486+0000 7fa43effd640 1 --2- 172.21.15.28:0/1902013713 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fa43406e0b0 0x7fa434070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:50:47.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.486+0000 7fa43effd640 1 -- 172.21.15.28:0/1902013713 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa4380adaf0 con 0x7fa4480ff4d0 2024-07-03T15:50:47.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.486+0000 7fa44e171640 1 --2- 172.21.15.28:0/1902013713 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fa43406e0b0 0x7fa434070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:50:47.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.487+0000 7fa44e171640 1 --2- 172.21.15.28:0/1902013713 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fa43406e0b0 0x7fa434070560 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa444009fd0 tx=0x7fa444009e50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:50:47.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.491+0000 7fa43effd640 1 -- 172.21.15.28:0/1902013713 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fa438076e20 con 0x7fa4480ff4d0 2024-07-03T15:50:47.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.739+0000 7fa4503d5640 1 -- 172.21.15.28:0/1902013713 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa44804b530 con 0x7fa4480ff4d0 2024-07-03T15:50:47.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.740+0000 7fa43effd640 1 -- 172.21.15.28:0/1902013713 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) v1 ==== 76+0+1894 (secure 0 0 0) 0x7fa43807a4e0 con 0x7fa4480ff4d0 2024-07-03T15:50:47.743 INFO:teuthology.orchestra.run.smithi028.stdout:e25 2024-07-03T15:50:47.744 INFO:teuthology.orchestra.run.smithi028.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-03T15:50:47.744 INFO:teuthology.orchestra.run.smithi028.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:50:47.744 INFO:teuthology.orchestra.run.smithi028.stdout:legacy client fscid: 1 2024-07-03T15:50:47.744 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:50:47.744 INFO:teuthology.orchestra.run.smithi028.stdout:Filesystem 'cephfs' (1) 2024-07-03T15:50:47.744 INFO:teuthology.orchestra.run.smithi028.stdout:fs_name cephfs 2024-07-03T15:50:47.744 INFO:teuthology.orchestra.run.smithi028.stdout:epoch 25 2024-07-03T15:50:47.744 INFO:teuthology.orchestra.run.smithi028.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-07-03T15:50:47.744 INFO:teuthology.orchestra.run.smithi028.stdout:created 2024-07-03T15:41:40.656298+0000 2024-07-03T15:50:47.744 INFO:teuthology.orchestra.run.smithi028.stdout:modified 2024-07-03T15:50:42.345494+0000 2024-07-03T15:50:47.744 INFO:teuthology.orchestra.run.smithi028.stdout:tableserver 0 2024-07-03T15:50:47.745 INFO:teuthology.orchestra.run.smithi028.stdout:root 0 2024-07-03T15:50:47.745 INFO:teuthology.orchestra.run.smithi028.stdout:session_timeout 60 2024-07-03T15:50:47.745 INFO:teuthology.orchestra.run.smithi028.stdout:session_autoclose 300 2024-07-03T15:50:47.745 INFO:teuthology.orchestra.run.smithi028.stdout:max_file_size 1099511627776 2024-07-03T15:50:47.745 INFO:teuthology.orchestra.run.smithi028.stdout:max_xattr_size 65536 2024-07-03T15:50:47.745 INFO:teuthology.orchestra.run.smithi028.stdout:required_client_features {} 2024-07-03T15:50:47.745 INFO:teuthology.orchestra.run.smithi028.stdout:last_failure 0 2024-07-03T15:50:47.745 INFO:teuthology.orchestra.run.smithi028.stdout:last_failure_osd_epoch 79 2024-07-03T15:50:47.745 INFO:teuthology.orchestra.run.smithi028.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:50:47.745 INFO:teuthology.orchestra.run.smithi028.stdout:max_mds 1 2024-07-03T15:50:47.745 INFO:teuthology.orchestra.run.smithi028.stdout:in 0 2024-07-03T15:50:47.745 INFO:teuthology.orchestra.run.smithi028.stdout:up {0=24277} 2024-07-03T15:50:47.745 INFO:teuthology.orchestra.run.smithi028.stdout:failed 2024-07-03T15:50:47.745 INFO:teuthology.orchestra.run.smithi028.stdout:damaged 2024-07-03T15:50:47.746 INFO:teuthology.orchestra.run.smithi028.stdout:stopped 2024-07-03T15:50:47.746 INFO:teuthology.orchestra.run.smithi028.stdout:data_pools [3] 2024-07-03T15:50:47.746 INFO:teuthology.orchestra.run.smithi028.stdout:metadata_pool 2 2024-07-03T15:50:47.746 INFO:teuthology.orchestra.run.smithi028.stdout:inline_data disabled 2024-07-03T15:50:47.746 INFO:teuthology.orchestra.run.smithi028.stdout:balancer 2024-07-03T15:50:47.746 INFO:teuthology.orchestra.run.smithi028.stdout:bal_rank_mask -1 2024-07-03T15:50:47.746 INFO:teuthology.orchestra.run.smithi028.stdout:standby_count_wanted 1 2024-07-03T15:50:47.746 INFO:teuthology.orchestra.run.smithi028.stdout:qdb_cluster leader: 24277 members: 24277 2024-07-03T15:50:47.746 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi148.szomfl{0:24277} state up:active seq 135 join_fscid=1 addr [v2:172.21.15.148:6826/4171231858,v1:172.21.15.148:6827/4171231858] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:50:47.746 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:50:47.746 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:50:47.746 INFO:teuthology.orchestra.run.smithi028.stdout:Standby daemons: 2024-07-03T15:50:47.746 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:50:47.747 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi148.xgpuld{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/2347536398,v1:172.21.15.148:6825/2347536398] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:50:47.747 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi028.wmbxew{-1:34372} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.28:6828/1224242160,v1:172.21.15.28:6829/1224242160] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:50:47.747 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi028.rhpvve{-1:34376} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.28:6826/3127260162,v1:172.21.15.28:6827/3127260162] compat {c=[1],r=[1],i=[1fff]}] 2024-07-03T15:50:47.747 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 25 2024-07-03T15:50:47.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.742+0000 7fa4503d5640 1 -- 172.21.15.28:0/1902013713 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fa43406e0b0 msgr2=0x7fa434070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:47.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.742+0000 7fa4503d5640 1 --2- 172.21.15.28:0/1902013713 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fa43406e0b0 0x7fa434070560 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa444009fd0 tx=0x7fa444009e50 comp rx=0 tx=0).stop 2024-07-03T15:50:47.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.742+0000 7fa4503d5640 1 -- 172.21.15.28:0/1902013713 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4480ff4d0 msgr2=0x7fa44810d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:47.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.742+0000 7fa4503d5640 1 --2- 172.21.15.28:0/1902013713 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4480ff4d0 0x7fa44810d4d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa438009f80 tx=0x7fa438009da0 comp rx=0 tx=0).stop 2024-07-03T15:50:47.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.742+0000 7fa4503d5640 1 -- 172.21.15.28:0/1902013713 shutdown_connections 2024-07-03T15:50:47.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.742+0000 7fa4503d5640 1 --2- 172.21.15.28:0/1902013713 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fa43406e0b0 0x7fa434070560 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:47.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.742+0000 7fa4503d5640 1 --2- 172.21.15.28:0/1902013713 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4480ff4d0 0x7fa44810d4d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:47.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.742+0000 7fa4503d5640 1 --2- 172.21.15.28:0/1902013713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa448066520 0x7fa44810cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:47.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.742+0000 7fa4503d5640 1 -- 172.21.15.28:0/1902013713 >> 172.21.15.28:0/1902013713 conn(0x7fa4480f6a20 msgr2=0x7fa4480f6e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:50:47.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.742+0000 7fa4503d5640 1 -- 172.21.15.28:0/1902013713 shutdown_connections 2024-07-03T15:50:47.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.742+0000 7fa4503d5640 1 -- 172.21.15.28:0/1902013713 wait complete. 2024-07-03T15:50:47.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.831+0000 7f5f9efc8640 1 -- 172.21.15.28:0/3076407033 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5f980fd620 msgr2=0x7f5f980fda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:47.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.831+0000 7f5f9efc8640 1 --2- 172.21.15.28:0/3076407033 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5f980fd620 0x7f5f980fda90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5f840048d0 tx=0x7f5f84009da0 comp rx=0 tx=0).stop 2024-07-03T15:50:47.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.831+0000 7f5f9efc8640 1 -- 172.21.15.28:0/3076407033 shutdown_connections 2024-07-03T15:50:47.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.831+0000 7f5f9efc8640 1 --2- 172.21.15.28:0/3076407033 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5f980fd620 0x7f5f980fda90 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:47.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.831+0000 7f5f9efc8640 1 --2- 172.21.15.28:0/3076407033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f980fc350 0x7f5f980fc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:47.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.831+0000 7f5f9efc8640 1 -- 172.21.15.28:0/3076407033 >> 172.21.15.28:0/3076407033 conn(0x7f5f980f7af0 msgr2=0x7f5f980f9f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:50:47.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.831+0000 7f5f9efc8640 1 -- 172.21.15.28:0/3076407033 shutdown_connections 2024-07-03T15:50:47.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.831+0000 7f5f9efc8640 1 -- 172.21.15.28:0/3076407033 wait complete. 2024-07-03T15:50:47.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.831+0000 7f5f9efc8640 1 Processor -- start 2024-07-03T15:50:47.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.831+0000 7f5f9efc8640 1 -- start start 2024-07-03T15:50:47.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.832+0000 7f5f9efc8640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5f980fc350 0x7f5f98071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:50:47.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.832+0000 7f5f9efc8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f980fd620 0x7f5f9806f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:50:47.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.832+0000 7f5f9efc8640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f98110be0 con 0x7f5f980fc350 2024-07-03T15:50:47.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.832+0000 7f5f9efc8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f98110d20 con 0x7f5f980fd620 2024-07-03T15:50:47.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.832+0000 7f5f9cd64640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5f980fc350 0x7f5f98071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:50:47.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.832+0000 7f5f9cd64640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5f980fc350 0x7f5f98071380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59514/0 (socket says 172.21.15.28:59514) 2024-07-03T15:50:47.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.832+0000 7f5f9cd64640 1 -- 172.21.15.28:0/2375579164 learned_addr learned my addr 172.21.15.28:0/2375579164 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:50:47.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.832+0000 7f5f97fff640 1 --2- 172.21.15.28:0/2375579164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f980fd620 0x7f5f9806f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:50:47.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.832+0000 7f5f9cd64640 1 -- 172.21.15.28:0/2375579164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f980fd620 msgr2=0x7f5f9806f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:47.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.832+0000 7f5f9cd64640 1 --2- 172.21.15.28:0/2375579164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f980fd620 0x7f5f9806f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:47.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.832+0000 7f5f9cd64640 1 -- 172.21.15.28:0/2375579164 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f84005040 con 0x7f5f980fc350 2024-07-03T15:50:47.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.832+0000 7f5f97fff640 1 --2- 172.21.15.28:0/2375579164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f980fd620 0x7f5f9806f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-03T15:50:47.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.833+0000 7f5f9cd64640 1 --2- 172.21.15.28:0/2375579164 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5f980fc350 0x7f5f98071380 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c002a50 tx=0x7f5f8c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:50:47.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.833+0000 7f5f957fa640 1 -- 172.21.15.28:0/2375579164 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f8c007c30 con 0x7f5f980fc350 2024-07-03T15:50:47.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.833+0000 7f5f957fa640 1 -- 172.21.15.28:0/2375579164 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5f8c00f040 con 0x7f5f980fc350 2024-07-03T15:50:47.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.833+0000 7f5f9efc8640 1 -- 172.21.15.28:0/2375579164 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f9806fe70 con 0x7f5f980fc350 2024-07-03T15:50:47.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.833+0000 7f5f957fa640 1 -- 172.21.15.28:0/2375579164 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f8c00db50 con 0x7f5f980fc350 2024-07-03T15:50:47.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.833+0000 7f5f9efc8640 1 -- 172.21.15.28:0/2375579164 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f98070410 con 0x7f5f980fc350 2024-07-03T15:50:47.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.834+0000 7f5f9efc8640 1 -- 172.21.15.28:0/2375579164 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f980fb7b0 con 0x7f5f980fc350 2024-07-03T15:50:47.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.835+0000 7f5f957fa640 1 -- 172.21.15.28:0/2375579164 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5f8c00dcf0 con 0x7f5f980fc350 2024-07-03T15:50:47.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.835+0000 7f5f957fa640 1 --2- 172.21.15.28:0/2375579164 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f5f8806e0b0 0x7f5f88070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:50:47.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.835+0000 7f5f957fa640 1 -- 172.21.15.28:0/2375579164 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f5f8c08d3d0 con 0x7f5f980fc350 2024-07-03T15:50:47.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.836+0000 7f5f97fff640 1 --2- 172.21.15.28:0/2375579164 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f5f8806e0b0 0x7f5f88070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:50:47.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.836+0000 7f5f97fff640 1 --2- 172.21.15.28:0/2375579164 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f5f8806e0b0 0x7f5f88070560 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5f840047f0 tx=0x7f5f84009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:50:47.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:47.841+0000 7f5f957fa640 1 -- 172.21.15.28:0/2375579164 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f5f8c056780 con 0x7f5f980fc350 2024-07-03T15:50:48.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.043+0000 7f5f9efc8640 1 -- 172.21.15.28:0/2375579164 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5f98101940 con 0x7f5f8806e0b0 2024-07-03T15:50:48.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.044+0000 7f5f957fa640 1 -- 172.21.15.28:0/2375579164 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f5f98101940 con 0x7f5f8806e0b0 2024-07-03T15:50:48.048 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-07-03T15:50:48.048 INFO:teuthology.orchestra.run.smithi028.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2", 2024-07-03T15:50:48.048 INFO:teuthology.orchestra.run.smithi028.stdout: "in_progress": true, 2024-07-03T15:50:48.049 INFO:teuthology.orchestra.run.smithi028.stdout: "which": "Upgrading all daemon types on all hosts", 2024-07-03T15:50:48.049 INFO:teuthology.orchestra.run.smithi028.stdout: "services_complete": [ 2024-07-03T15:50:48.049 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr", 2024-07-03T15:50:48.049 INFO:teuthology.orchestra.run.smithi028.stdout: "crash", 2024-07-03T15:50:48.049 INFO:teuthology.orchestra.run.smithi028.stdout: "mon", 2024-07-03T15:50:48.049 INFO:teuthology.orchestra.run.smithi028.stdout: "osd" 2024-07-03T15:50:48.049 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-07-03T15:50:48.049 INFO:teuthology.orchestra.run.smithi028.stdout: "progress": "13/21 daemons upgraded", 2024-07-03T15:50:48.049 INFO:teuthology.orchestra.run.smithi028.stdout: "message": "Currently upgrading mds daemons", 2024-07-03T15:50:48.049 INFO:teuthology.orchestra.run.smithi028.stdout: "is_paused": false 2024-07-03T15:50:48.049 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-07-03T15:50:48.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.046+0000 7f5f9efc8640 1 -- 172.21.15.28:0/2375579164 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f5f8806e0b0 msgr2=0x7f5f88070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:48.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.046+0000 7f5f9efc8640 1 --2- 172.21.15.28:0/2375579164 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f5f8806e0b0 0x7f5f88070560 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5f840047f0 tx=0x7f5f84009da0 comp rx=0 tx=0).stop 2024-07-03T15:50:48.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.046+0000 7f5f9efc8640 1 -- 172.21.15.28:0/2375579164 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5f980fc350 msgr2=0x7f5f98071380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:48.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.046+0000 7f5f9efc8640 1 --2- 172.21.15.28:0/2375579164 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5f980fc350 0x7f5f98071380 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c002a50 tx=0x7f5f8c004600 comp rx=0 tx=0).stop 2024-07-03T15:50:48.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.047+0000 7f5f9efc8640 1 -- 172.21.15.28:0/2375579164 shutdown_connections 2024-07-03T15:50:48.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.047+0000 7f5f9efc8640 1 --2- 172.21.15.28:0/2375579164 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f5f8806e0b0 0x7f5f88070560 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:48.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.047+0000 7f5f9efc8640 1 --2- 172.21.15.28:0/2375579164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f980fd620 0x7f5f9806f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:48.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.047+0000 7f5f9efc8640 1 --2- 172.21.15.28:0/2375579164 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5f980fc350 0x7f5f98071380 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:48.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.047+0000 7f5f9efc8640 1 -- 172.21.15.28:0/2375579164 >> 172.21.15.28:0/2375579164 conn(0x7f5f980f7af0 msgr2=0x7f5f980f9200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:50:48.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.047+0000 7f5f9efc8640 1 -- 172.21.15.28:0/2375579164 shutdown_connections 2024-07-03T15:50:48.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.047+0000 7f5f9efc8640 1 -- 172.21.15.28:0/2375579164 wait complete. 2024-07-03T15:50:48.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.140+0000 7f3ced694640 1 -- 172.21.15.28:0/3097183183 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ce80fc530 msgr2=0x7f3ce80fc9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:48.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.140+0000 7f3ced694640 1 --2- 172.21.15.28:0/3097183183 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ce80fc530 0x7f3ce80fc9a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc0048d0 tx=0x7f3cdc009da0 comp rx=0 tx=0).stop 2024-07-03T15:50:48.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.140+0000 7f3ced694640 1 -- 172.21.15.28:0/3097183183 shutdown_connections 2024-07-03T15:50:48.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.140+0000 7f3ced694640 1 --2- 172.21.15.28:0/3097183183 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ce80fc530 0x7f3ce80fc9a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:48.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.140+0000 7f3ced694640 1 --2- 172.21.15.28:0/3097183183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ce80fb260 0x7f3ce80fb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:48.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.140+0000 7f3ced694640 1 -- 172.21.15.28:0/3097183183 >> 172.21.15.28:0/3097183183 conn(0x7f3ce80f6a20 msgr2=0x7f3ce80f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:50:48.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.140+0000 7f3ced694640 1 -- 172.21.15.28:0/3097183183 shutdown_connections 2024-07-03T15:50:48.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.140+0000 7f3ced694640 1 -- 172.21.15.28:0/3097183183 wait complete. 2024-07-03T15:50:48.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.141+0000 7f3ced694640 1 Processor -- start 2024-07-03T15:50:48.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.141+0000 7f3ced694640 1 -- start start 2024-07-03T15:50:48.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.141+0000 7f3ced694640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ce80fb260 0x7f3ce810bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:50:48.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.141+0000 7f3ced694640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ce80fc530 0x7f3ce810c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:50:48.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.141+0000 7f3ced694640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce810ca50 con 0x7f3ce80fc530 2024-07-03T15:50:48.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.141+0000 7f3ced694640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce810d270 con 0x7f3ce80fb260 2024-07-03T15:50:48.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.141+0000 7f3ce67fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ce80fc530 0x7f3ce810c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:50:48.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.141+0000 7f3ce67fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ce80fc530 0x7f3ce810c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59526/0 (socket says 172.21.15.28:59526) 2024-07-03T15:50:48.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.141+0000 7f3ce67fc640 1 -- 172.21.15.28:0/1810137548 learned_addr learned my addr 172.21.15.28:0/1810137548 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:50:48.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.142+0000 7f3ce6ffd640 1 --2- 172.21.15.28:0/1810137548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ce80fb260 0x7f3ce810bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:50:48.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.142+0000 7f3ce67fc640 1 -- 172.21.15.28:0/1810137548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ce80fb260 msgr2=0x7f3ce810bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:48.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.142+0000 7f3ce67fc640 1 --2- 172.21.15.28:0/1810137548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ce80fb260 0x7f3ce810bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:48.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.142+0000 7f3ce67fc640 1 -- 172.21.15.28:0/1810137548 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cdc005040 con 0x7f3ce80fc530 2024-07-03T15:50:48.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.142+0000 7f3ce67fc640 1 --2- 172.21.15.28:0/1810137548 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ce80fc530 0x7f3ce810c400 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc0048a0 tx=0x7f3cdc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:50:48.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.142+0000 7f3ccf7fe640 1 -- 172.21.15.28:0/1810137548 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cdc038070 con 0x7f3ce80fc530 2024-07-03T15:50:48.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.142+0000 7f3ccf7fe640 1 -- 172.21.15.28:0/1810137548 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3cdc0092b0 con 0x7f3ce80fc530 2024-07-03T15:50:48.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.142+0000 7f3ced694640 1 -- 172.21.15.28:0/1810137548 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ce810d4a0 con 0x7f3ce80fc530 2024-07-03T15:50:48.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.142+0000 7f3ccf7fe640 1 -- 172.21.15.28:0/1810137548 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cdc0337a0 con 0x7f3ce80fc530 2024-07-03T15:50:48.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.142+0000 7f3ced694640 1 -- 172.21.15.28:0/1810137548 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce810d9e0 con 0x7f3ce80fc530 2024-07-03T15:50:48.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.143+0000 7f3ced694640 1 -- 172.21.15.28:0/1810137548 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ce80fa6c0 con 0x7f3ce80fc530 2024-07-03T15:50:48.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.144+0000 7f3ccf7fe640 1 -- 172.21.15.28:0/1810137548 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3cdc030020 con 0x7f3ce80fc530 2024-07-03T15:50:48.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.144+0000 7f3ccf7fe640 1 --2- 172.21.15.28:0/1810137548 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f3cd006e0b0 0x7f3cd0070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:50:48.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.144+0000 7f3ccf7fe640 1 -- 172.21.15.28:0/1810137548 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f3cdc0ad9d0 con 0x7f3ce80fc530 2024-07-03T15:50:48.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.144+0000 7f3ce6ffd640 1 --2- 172.21.15.28:0/1810137548 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f3cd006e0b0 0x7f3cd0070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:50:48.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.145+0000 7f3ce6ffd640 1 --2- 172.21.15.28:0/1810137548 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f3cd006e0b0 0x7f3cd0070560 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8004ed0 tx=0x7f3cd800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:50:48.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.149+0000 7f3ccf7fe640 1 -- 172.21.15.28:0/1810137548 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f3cdc076db0 con 0x7f3ce80fc530 2024-07-03T15:50:48.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:48 smithi028 ceph-mon[92858]: from='client.34388 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:50:48.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:48 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/387096689' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:50:48.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:48 smithi028 ceph-mon[92858]: Detected new or changed devices on smithi028 2024-07-03T15:50:48.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:48 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:48.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:48 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:48.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:48 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:50:48.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:48 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:50:48.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:48 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:48.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:48 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:50:48.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:48 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:50:48.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:48 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:50:48.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:48 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:50:48.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:48 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:50:48.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:48 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi028.wmbxew"]}]: dispatch 2024-07-03T15:50:48.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:48 smithi028 ceph-mon[92858]: Upgrade: It appears safe to stop mds.cephfs.smithi028.wmbxew 2024-07-03T15:50:48.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:48 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/1902013713' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-03T15:50:48.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.410+0000 7f3ced694640 1 -- 172.21.15.28:0/1810137548 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3ce810dca0 con 0x7f3ce80fc530 2024-07-03T15:50:48.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.410+0000 7f3ccf7fe640 1 -- 172.21.15.28:0/1810137548 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f3cdc07a470 con 0x7f3ce80fc530 2024-07-03T15:50:48.414 INFO:teuthology.orchestra.run.smithi028.stdout:HEALTH_OK 2024-07-03T15:50:48.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.412+0000 7f3ced694640 1 -- 172.21.15.28:0/1810137548 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f3cd006e0b0 msgr2=0x7f3cd0070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:48.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.412+0000 7f3ced694640 1 --2- 172.21.15.28:0/1810137548 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f3cd006e0b0 0x7f3cd0070560 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8004ed0 tx=0x7f3cd800b040 comp rx=0 tx=0).stop 2024-07-03T15:50:48.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.412+0000 7f3ced694640 1 -- 172.21.15.28:0/1810137548 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ce80fc530 msgr2=0x7f3ce810c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:50:48.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.412+0000 7f3ced694640 1 --2- 172.21.15.28:0/1810137548 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ce80fc530 0x7f3ce810c400 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc0048a0 tx=0x7f3cdc009da0 comp rx=0 tx=0).stop 2024-07-03T15:50:48.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.412+0000 7f3ced694640 1 -- 172.21.15.28:0/1810137548 shutdown_connections 2024-07-03T15:50:48.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.412+0000 7f3ced694640 1 --2- 172.21.15.28:0/1810137548 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f3cd006e0b0 0x7f3cd0070560 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:48.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.412+0000 7f3ced694640 1 --2- 172.21.15.28:0/1810137548 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ce80fc530 0x7f3ce810c400 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:48.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.412+0000 7f3ced694640 1 --2- 172.21.15.28:0/1810137548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ce80fb260 0x7f3ce810bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:50:48.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.412+0000 7f3ced694640 1 -- 172.21.15.28:0/1810137548 >> 172.21.15.28:0/1810137548 conn(0x7f3ce80f6a20 msgr2=0x7f3ce80f8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:50:48.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.412+0000 7f3ced694640 1 -- 172.21.15.28:0/1810137548 shutdown_connections 2024-07-03T15:50:48.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:50:48.413+0000 7f3ced694640 1 -- 172.21.15.28:0/1810137548 wait complete. 2024-07-03T15:50:48.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:48 smithi148 ceph-mon[80490]: from='client.34388 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:50:48.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:48 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/387096689' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:50:48.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:48 smithi148 ceph-mon[80490]: Detected new or changed devices on smithi028 2024-07-03T15:50:48.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:48 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:48.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:48 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:48.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:48 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:50:48.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:48 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:50:48.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:48 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:48.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:48 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:50:48.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:48 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:50:48.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:48 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:50:48.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:48 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:50:48.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:48 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:50:48.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:48 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi028.wmbxew"]}]: dispatch 2024-07-03T15:50:48.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:48 smithi148 ceph-mon[80490]: Upgrade: It appears safe to stop mds.cephfs.smithi028.wmbxew 2024-07-03T15:50:48.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:48 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/1902013713' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-03T15:50:49.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:49 smithi028 ceph-mon[92858]: from='client.34400 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:50:49.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:49 smithi028 ceph-mon[92858]: pgmap v167: 65 pgs: 65 active+clean; 193 MiB data, 791 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-07-03T15:50:49.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:49 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/1810137548' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-03T15:50:49.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:49.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi028.wmbxew", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-03T15:50:49.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:50:49.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:49 smithi148 ceph-mon[80490]: from='client.34400 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:50:49.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:49 smithi148 ceph-mon[80490]: pgmap v167: 65 pgs: 65 active+clean; 193 MiB data, 791 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-07-03T15:50:49.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:49 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/1810137548' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-03T15:50:49.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:49.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi028.wmbxew", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-03T15:50:49.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:50:50.332 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:50 smithi028 ceph-mon[92858]: Upgrade: Updating mds.cephfs.smithi028.wmbxew 2024-07-03T15:50:50.332 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:50 smithi028 ceph-mon[92858]: Deploying daemon mds.cephfs.smithi028.wmbxew on smithi028 2024-07-03T15:50:50.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:50 smithi148 ceph-mon[80490]: Upgrade: Updating mds.cephfs.smithi028.wmbxew 2024-07-03T15:50:50.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:50 smithi148 ceph-mon[80490]: Deploying daemon mds.cephfs.smithi028.wmbxew on smithi028 2024-07-03T15:50:51.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:51 smithi028 ceph-mon[92858]: pgmap v168: 65 pgs: 65 active+clean; 193 MiB data, 791 MiB used, 536 GiB / 536 GiB avail; 8.5 MiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-07-03T15:50:51.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:51 smithi148 ceph-mon[80490]: pgmap v168: 65 pgs: 65 active+clean; 193 MiB data, 791 MiB used, 536 GiB / 536 GiB avail; 8.5 MiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-07-03T15:50:52.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:52 smithi148 ceph-mon[80490]: osdmap e80: 6 total, 6 up, 6 in 2024-07-03T15:50:52.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:52 smithi148 ceph-mon[80490]: fsmap cephfs:1 {0=cephfs.smithi148.szomfl=up:active} 2 up:standby 2024-07-03T15:50:52.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:52 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:52.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:52 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:50:52.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:52 smithi028 ceph-mon[92858]: osdmap e80: 6 total, 6 up, 6 in 2024-07-03T15:50:52.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:52 smithi028 ceph-mon[92858]: fsmap cephfs:1 {0=cephfs.smithi148.szomfl=up:active} 2 up:standby 2024-07-03T15:50:52.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:52 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:52.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:52 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:50:53.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:53 smithi148 ceph-mon[80490]: pgmap v170: 65 pgs: 65 active+clean; 193 MiB data, 791 MiB used, 536 GiB / 536 GiB avail; 23 KiB/s rd, 5.6 KiB/s wr, 6 op/s 2024-07-03T15:50:53.562 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:53 smithi028 ceph-mon[92858]: pgmap v170: 65 pgs: 65 active+clean; 193 MiB data, 791 MiB used, 536 GiB / 536 GiB avail; 23 KiB/s rd, 5.6 KiB/s wr, 6 op/s 2024-07-03T15:50:56.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:55 smithi148 ceph-mon[80490]: pgmap v171: 65 pgs: 65 active+clean; 193 MiB data, 791 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s rd, 5.2 KiB/s wr, 3 op/s 2024-07-03T15:50:56.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:55 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:56.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:55 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:56.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:55 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:50:56.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:55 smithi148 ceph-mon[80490]: mds.? [v2:172.21.15.28:6828/2908130327,v1:172.21.15.28:6829/2908130327] up:boot 2024-07-03T15:50:56.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:55 smithi148 ceph-mon[80490]: fsmap cephfs:1 {0=cephfs.smithi148.szomfl=up:active} 3 up:standby 2024-07-03T15:50:56.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:55 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi028.wmbxew"}]: dispatch 2024-07-03T15:50:56.264 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:55 smithi028 ceph-mon[92858]: pgmap v171: 65 pgs: 65 active+clean; 193 MiB data, 791 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s rd, 5.2 KiB/s wr, 3 op/s 2024-07-03T15:50:56.264 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:55 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:56.264 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:55 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:50:56.264 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:55 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:50:56.264 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:55 smithi028 ceph-mon[92858]: mds.? [v2:172.21.15.28:6828/2908130327,v1:172.21.15.28:6829/2908130327] up:boot 2024-07-03T15:50:56.264 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:55 smithi028 ceph-mon[92858]: fsmap cephfs:1 {0=cephfs.smithi148.szomfl=up:active} 3 up:standby 2024-07-03T15:50:56.265 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:55 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi028.wmbxew"}]: dispatch 2024-07-03T15:50:58.162 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:57 smithi028 ceph-mon[92858]: pgmap v172: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 5.2 KiB/s wr, 2 op/s 2024-07-03T15:50:58.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:57 smithi148 ceph-mon[80490]: pgmap v172: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 5.2 KiB/s wr, 2 op/s 2024-07-03T15:50:59.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:58 smithi148 ceph-mon[80490]: pgmap v173: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail 2024-07-03T15:50:59.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:58 smithi028 ceph-mon[92858]: pgmap v173: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail 2024-07-03T15:51:00.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:59 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:00.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:59 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:00.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:59 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:00.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:50:59 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:00.237 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:59 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:00.238 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:59 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:00.238 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:59 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:00.238 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:50:59 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:01.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:00 smithi148 ceph-mon[80490]: pgmap v174: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail 2024-07-03T15:51:01.264 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:00 smithi028 ceph-mon[92858]: pgmap v174: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail 2024-07-03T15:51:03.447 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:03 smithi148 ceph-mon[80490]: pgmap v175: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail 2024-07-03T15:51:03.447 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:03 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:03.447 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:03 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:03.447 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:03 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:51:03.447 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:03 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:51:03.447 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:03 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:03.447 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:03 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:51:03.447 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:03 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:03.447 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:03 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:03.447 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:03 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:03.448 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:03 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:03.448 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:03 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi148.xgpuld"]}]: dispatch 2024-07-03T15:51:03.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:03 smithi028 ceph-mon[92858]: pgmap v175: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail 2024-07-03T15:51:03.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:03 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:03.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:03 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:03.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:03 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:51:03.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:03 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:51:03.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:03 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:03.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:03 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:51:03.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:03 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:03.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:03 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:03.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:03 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:03.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:03 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:03.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:03 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi148.xgpuld"]}]: dispatch 2024-07-03T15:51:04.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:04 smithi148 ceph-mon[80490]: Upgrade: It appears safe to stop mds.cephfs.smithi148.xgpuld 2024-07-03T15:51:04.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:04 smithi028 ceph-mon[92858]: Upgrade: It appears safe to stop mds.cephfs.smithi148.xgpuld 2024-07-03T15:51:05.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:05 smithi148 ceph-mon[80490]: pgmap v176: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail 2024-07-03T15:51:05.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:05 smithi148 ceph-mon[80490]: Upgrade: Updating mds.cephfs.smithi148.xgpuld 2024-07-03T15:51:05.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:05 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:05.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:05 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.xgpuld", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-03T15:51:05.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:05 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:51:05.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:05 smithi148 ceph-mon[80490]: Deploying daemon mds.cephfs.smithi148.xgpuld on smithi148 2024-07-03T15:51:05.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:05 smithi028 ceph-mon[92858]: pgmap v176: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail 2024-07-03T15:51:05.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:05 smithi028 ceph-mon[92858]: Upgrade: Updating mds.cephfs.smithi148.xgpuld 2024-07-03T15:51:05.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:05 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:05.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:05 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.xgpuld", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-03T15:51:05.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:05 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:51:05.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:05 smithi028 ceph-mon[92858]: Deploying daemon mds.cephfs.smithi148.xgpuld on smithi148 2024-07-03T15:51:07.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:07 smithi148 ceph-mon[80490]: pgmap v177: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail 2024-07-03T15:51:07.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:07 smithi148 ceph-mon[80490]: osdmap e81: 6 total, 6 up, 6 in 2024-07-03T15:51:07.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:07 smithi148 ceph-mon[80490]: fsmap cephfs:1 {0=cephfs.smithi148.szomfl=up:active} 2 up:standby 2024-07-03T15:51:07.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:07 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:51:07.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:07 smithi028 ceph-mon[92858]: pgmap v177: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail 2024-07-03T15:51:07.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:07 smithi028 ceph-mon[92858]: osdmap e81: 6 total, 6 up, 6 in 2024-07-03T15:51:07.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:07 smithi028 ceph-mon[92858]: fsmap cephfs:1 {0=cephfs.smithi148.szomfl=up:active} 2 up:standby 2024-07-03T15:51:07.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:07 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:51:08.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:08 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.xgpuld"}]: dispatch 2024-07-03T15:51:08.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:08 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.xgpuld"}]: dispatch 2024-07-03T15:51:09.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:09 smithi148 ceph-mon[80490]: pgmap v179: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail 2024-07-03T15:51:09.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:09 smithi028 ceph-mon[92858]: pgmap v179: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail 2024-07-03T15:51:11.714 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:11 smithi148 ceph-mon[80490]: pgmap v180: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail 2024-07-03T15:51:11.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:11 smithi028 ceph-mon[92858]: pgmap v180: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail 2024-07-03T15:51:13.289 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:13 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:13.289 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:13 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:13.289 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:13 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:51:13.289 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:13 smithi148 ceph-mon[80490]: pgmap v181: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail 2024-07-03T15:51:13.290 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:13 smithi148 ceph-mon[80490]: mds.? [v2:172.21.15.148:6824/1480077002,v1:172.21.15.148:6825/1480077002] up:boot 2024-07-03T15:51:13.290 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:13 smithi148 ceph-mon[80490]: fsmap cephfs:1 {0=cephfs.smithi148.szomfl=up:active} 3 up:standby 2024-07-03T15:51:13.290 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:13 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.xgpuld"}]: dispatch 2024-07-03T15:51:13.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:13 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:13.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:13 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:13.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:13 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:51:13.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:13 smithi028 ceph-mon[92858]: pgmap v181: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail 2024-07-03T15:51:13.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:13 smithi028 ceph-mon[92858]: mds.? [v2:172.21.15.148:6824/1480077002,v1:172.21.15.148:6825/1480077002] up:boot 2024-07-03T15:51:13.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:13 smithi028 ceph-mon[92858]: fsmap cephfs:1 {0=cephfs.smithi148.szomfl=up:active} 3 up:standby 2024-07-03T15:51:13.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:13 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.xgpuld"}]: dispatch 2024-07-03T15:51:15.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:15 smithi028 ceph-mon[92858]: pgmap v182: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail 2024-07-03T15:51:15.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:15 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:15.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:15 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:15.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:15 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:15.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:15 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:15.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:15 smithi148 ceph-mon[80490]: pgmap v182: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail 2024-07-03T15:51:15.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:15 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:15.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:15 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:15.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:15 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:15.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:15 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:17.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:17 smithi148 ceph-mon[80490]: pgmap v183: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail 2024-07-03T15:51:18.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:17 smithi028 ceph-mon[92858]: pgmap v183: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail 2024-07-03T15:51:18.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.501+0000 7fc2ab099640 1 -- 172.21.15.28:0/97996221 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc2a40fc330 msgr2=0x7fc2a40fc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:18.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.501+0000 7fc2ab099640 1 --2- 172.21.15.28:0/97996221 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc2a40fc330 0x7fc2a40fc740 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc2980048d0 tx=0x7fc298009da0 comp rx=0 tx=0).stop 2024-07-03T15:51:18.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.501+0000 7fc2ab099640 1 -- 172.21.15.28:0/97996221 shutdown_connections 2024-07-03T15:51:18.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.501+0000 7fc2ab099640 1 --2- 172.21.15.28:0/97996221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2a40fd600 0x7fc2a40fda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:18.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.501+0000 7fc2ab099640 1 --2- 172.21.15.28:0/97996221 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc2a40fc330 0x7fc2a40fc740 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:18.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.501+0000 7fc2ab099640 1 -- 172.21.15.28:0/97996221 >> 172.21.15.28:0/97996221 conn(0x7fc2a40f7af0 msgr2=0x7fc2a40f9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:51:18.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.501+0000 7fc2ab099640 1 -- 172.21.15.28:0/97996221 shutdown_connections 2024-07-03T15:51:18.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.501+0000 7fc2ab099640 1 -- 172.21.15.28:0/97996221 wait complete. 2024-07-03T15:51:18.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.502+0000 7fc2ab099640 1 Processor -- start 2024-07-03T15:51:18.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.502+0000 7fc2ab099640 1 -- start start 2024-07-03T15:51:18.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.502+0000 7fc2ab099640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2a40fc330 0x7fc2a410cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:51:18.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.502+0000 7fc2ab099640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc2a40fd600 0x7fc2a410d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:51:18.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.502+0000 7fc2ab099640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a410daf0 con 0x7fc2a40fd600 2024-07-03T15:51:18.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.502+0000 7fc2ab099640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a410e310 con 0x7fc2a40fc330 2024-07-03T15:51:18.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.503+0000 7fc2a3fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc2a40fd600 0x7fc2a410d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:51:18.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.503+0000 7fc2a8e35640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2a40fc330 0x7fc2a410cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:51:18.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.503+0000 7fc2a3fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc2a40fd600 0x7fc2a410d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49590/0 (socket says 172.21.15.28:49590) 2024-07-03T15:51:18.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.503+0000 7fc2a3fff640 1 -- 172.21.15.28:0/3503383436 learned_addr learned my addr 172.21.15.28:0/3503383436 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:51:18.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.503+0000 7fc2a8e35640 1 -- 172.21.15.28:0/3503383436 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc2a40fd600 msgr2=0x7fc2a410d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:18.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.503+0000 7fc2a8e35640 1 --2- 172.21.15.28:0/3503383436 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc2a40fd600 0x7fc2a410d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:18.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.503+0000 7fc2a8e35640 1 -- 172.21.15.28:0/3503383436 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc298005040 con 0x7fc2a40fc330 2024-07-03T15:51:18.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.503+0000 7fc2a3fff640 1 --2- 172.21.15.28:0/3503383436 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc2a40fd600 0x7fc2a410d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-03T15:51:18.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.503+0000 7fc2a8e35640 1 --2- 172.21.15.28:0/3503383436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2a40fc330 0x7fc2a410cf60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc2980048a0 tx=0x7fc298009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:51:18.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.503+0000 7fc2a17fa640 1 -- 172.21.15.28:0/3503383436 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc298038070 con 0x7fc2a40fc330 2024-07-03T15:51:18.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.503+0000 7fc2a17fa640 1 -- 172.21.15.28:0/3503383436 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc298032970 con 0x7fc2a40fc330 2024-07-03T15:51:18.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.504+0000 7fc2a17fa640 1 -- 172.21.15.28:0/3503383436 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2980338d0 con 0x7fc2a40fc330 2024-07-03T15:51:18.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.504+0000 7fc2ab099640 1 -- 172.21.15.28:0/3503383436 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2a410e540 con 0x7fc2a40fc330 2024-07-03T15:51:18.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.504+0000 7fc2ab099640 1 -- 172.21.15.28:0/3503383436 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a410ea50 con 0x7fc2a40fc330 2024-07-03T15:51:18.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.505+0000 7fc2ab099640 1 -- 172.21.15.28:0/3503383436 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2a41056b0 con 0x7fc2a40fc330 2024-07-03T15:51:18.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.505+0000 7fc2a17fa640 1 -- 172.21.15.28:0/3503383436 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc298030020 con 0x7fc2a40fc330 2024-07-03T15:51:18.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.506+0000 7fc2a17fa640 1 --2- 172.21.15.28:0/3503383436 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc29406e0b0 0x7fc294070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:51:18.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.506+0000 7fc2a17fa640 1 -- 172.21.15.28:0/3503383436 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fc2980ada80 con 0x7fc2a40fc330 2024-07-03T15:51:18.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.506+0000 7fc2a3fff640 1 --2- 172.21.15.28:0/3503383436 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc29406e0b0 0x7fc294070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:51:18.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.506+0000 7fc2a3fff640 1 --2- 172.21.15.28:0/3503383436 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc29406e0b0 0x7fc294070560 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc290004fd0 tx=0x7fc2900043c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:51:18.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.510+0000 7fc2a17fa640 1 -- 172.21.15.28:0/3503383436 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc298076e60 con 0x7fc2a40fc330 2024-07-03T15:51:18.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.720+0000 7fc2ab099640 1 -- 172.21.15.28:0/3503383436 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc2a410ed30 con 0x7fc29406e0b0 2024-07-03T15:51:18.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.721+0000 7fc2a17fa640 1 -- 172.21.15.28:0/3503383436 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fc2a410ed30 con 0x7fc29406e0b0 2024-07-03T15:51:18.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.724+0000 7fc2ab099640 1 -- 172.21.15.28:0/3503383436 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc29406e0b0 msgr2=0x7fc294070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:18.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.724+0000 7fc2ab099640 1 --2- 172.21.15.28:0/3503383436 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc29406e0b0 0x7fc294070560 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc290004fd0 tx=0x7fc2900043c0 comp rx=0 tx=0).stop 2024-07-03T15:51:18.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.724+0000 7fc2ab099640 1 -- 172.21.15.28:0/3503383436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2a40fc330 msgr2=0x7fc2a410cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:18.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.724+0000 7fc2ab099640 1 --2- 172.21.15.28:0/3503383436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2a40fc330 0x7fc2a410cf60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc2980048a0 tx=0x7fc298009da0 comp rx=0 tx=0).stop 2024-07-03T15:51:18.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.724+0000 7fc2ab099640 1 -- 172.21.15.28:0/3503383436 shutdown_connections 2024-07-03T15:51:18.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.724+0000 7fc2ab099640 1 --2- 172.21.15.28:0/3503383436 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc29406e0b0 0x7fc294070560 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:18.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.724+0000 7fc2ab099640 1 --2- 172.21.15.28:0/3503383436 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc2a40fd600 0x7fc2a410d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:18.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.724+0000 7fc2ab099640 1 --2- 172.21.15.28:0/3503383436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2a40fc330 0x7fc2a410cf60 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:18.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.724+0000 7fc2ab099640 1 -- 172.21.15.28:0/3503383436 >> 172.21.15.28:0/3503383436 conn(0x7fc2a40f7af0 msgr2=0x7fc2a40f91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:51:18.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.724+0000 7fc2ab099640 1 -- 172.21.15.28:0/3503383436 shutdown_connections 2024-07-03T15:51:18.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.724+0000 7fc2ab099640 1 -- 172.21.15.28:0/3503383436 wait complete. 2024-07-03T15:51:18.736 INFO:teuthology.orchestra.run.smithi028.stdout:true 2024-07-03T15:51:18.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.809+0000 7fe4fa095640 1 -- 172.21.15.28:0/3735095222 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4f40fc330 msgr2=0x7fe4f40fc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:18.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.809+0000 7fe4fa095640 1 --2- 172.21.15.28:0/3735095222 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4f40fc330 0x7fe4f40fc740 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc0048d0 tx=0x7fe4dc009da0 comp rx=0 tx=0).stop 2024-07-03T15:51:18.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.809+0000 7fe4fa095640 1 -- 172.21.15.28:0/3735095222 shutdown_connections 2024-07-03T15:51:18.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.809+0000 7fe4fa095640 1 --2- 172.21.15.28:0/3735095222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4f40fd600 0x7fe4f40fda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:18.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.809+0000 7fe4fa095640 1 --2- 172.21.15.28:0/3735095222 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4f40fc330 0x7fe4f40fc740 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:18.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.809+0000 7fe4fa095640 1 -- 172.21.15.28:0/3735095222 >> 172.21.15.28:0/3735095222 conn(0x7fe4f40f7af0 msgr2=0x7fe4f40f9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:51:18.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.809+0000 7fe4fa095640 1 -- 172.21.15.28:0/3735095222 shutdown_connections 2024-07-03T15:51:18.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.809+0000 7fe4fa095640 1 -- 172.21.15.28:0/3735095222 wait complete. 2024-07-03T15:51:18.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.809+0000 7fe4fa095640 1 Processor -- start 2024-07-03T15:51:18.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.810+0000 7fe4fa095640 1 -- start start 2024-07-03T15:51:18.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.810+0000 7fe4fa095640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4f40fc330 0x7fe4f410cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:51:18.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.810+0000 7fe4fa095640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4f40fd600 0x7fe4f410d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:51:18.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.810+0000 7fe4fa095640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f410dad0 con 0x7fe4f40fc330 2024-07-03T15:51:18.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.810+0000 7fe4fa095640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f410e2f0 con 0x7fe4f40fd600 2024-07-03T15:51:18.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.810+0000 7fe4f37fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4f40fc330 0x7fe4f410cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:51:18.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.810+0000 7fe4f37fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4f40fc330 0x7fe4f410cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49608/0 (socket says 172.21.15.28:49608) 2024-07-03T15:51:18.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.810+0000 7fe4f37fe640 1 -- 172.21.15.28:0/675081526 learned_addr learned my addr 172.21.15.28:0/675081526 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:51:18.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.810+0000 7fe4f2ffd640 1 --2- 172.21.15.28:0/675081526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4f40fd600 0x7fe4f410d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:51:18.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.810+0000 7fe4f2ffd640 1 -- 172.21.15.28:0/675081526 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4f40fc330 msgr2=0x7fe4f410cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:18.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.810+0000 7fe4f2ffd640 1 --2- 172.21.15.28:0/675081526 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4f40fc330 0x7fe4f410cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:18.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.810+0000 7fe4f2ffd640 1 -- 172.21.15.28:0/675081526 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4dc005040 con 0x7fe4f40fd600 2024-07-03T15:51:18.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.810+0000 7fe4f37fe640 1 --2- 172.21.15.28:0/675081526 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4f40fc330 0x7fe4f410cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-03T15:51:18.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.811+0000 7fe4f2ffd640 1 --2- 172.21.15.28:0/675081526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4f40fd600 0x7fe4f410d480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe4e40027a0 tx=0x7fe4e4009f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:51:18.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.811+0000 7fe4ebfff640 1 -- 172.21.15.28:0/675081526 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4e4002d30 con 0x7fe4f40fd600 2024-07-03T15:51:18.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.811+0000 7fe4fa095640 1 -- 172.21.15.28:0/675081526 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4f410e580 con 0x7fe4f40fd600 2024-07-03T15:51:18.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.811+0000 7fe4fa095640 1 -- 172.21.15.28:0/675081526 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4f410eb20 con 0x7fe4f40fd600 2024-07-03T15:51:18.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.811+0000 7fe4ebfff640 1 -- 172.21.15.28:0/675081526 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe4e400f040 con 0x7fe4f40fd600 2024-07-03T15:51:18.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.811+0000 7fe4ebfff640 1 -- 172.21.15.28:0/675081526 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4e4009470 con 0x7fe4f40fd600 2024-07-03T15:51:18.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.812+0000 7fe4fa095640 1 -- 172.21.15.28:0/675081526 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4f40fb790 con 0x7fe4f40fd600 2024-07-03T15:51:18.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.812+0000 7fe4ebfff640 1 -- 172.21.15.28:0/675081526 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe4e40095d0 con 0x7fe4f40fd600 2024-07-03T15:51:18.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.813+0000 7fe4ebfff640 1 --2- 172.21.15.28:0/675081526 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fe4e006e0b0 0x7fe4e0070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:51:18.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.813+0000 7fe4ebfff640 1 -- 172.21.15.28:0/675081526 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe4e408da60 con 0x7fe4f40fd600 2024-07-03T15:51:18.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.813+0000 7fe4f37fe640 1 --2- 172.21.15.28:0/675081526 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fe4e006e0b0 0x7fe4e0070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:51:18.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.814+0000 7fe4f37fe640 1 --2- 172.21.15.28:0/675081526 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fe4e006e0b0 0x7fe4e0070560 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc004540 tx=0x7fe4dc035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:51:18.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:18.818+0000 7fe4ebfff640 1 -- 172.21.15.28:0/675081526 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fe4e4056d10 con 0x7fe4f40fd600 2024-07-03T15:51:19.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.024+0000 7fe4fa095640 1 -- 172.21.15.28:0/675081526 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe4f410ef20 con 0x7fe4e006e0b0 2024-07-03T15:51:19.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.026+0000 7fe4ebfff640 1 -- 172.21.15.28:0/675081526 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe4f410ef20 con 0x7fe4e006e0b0 2024-07-03T15:51:19.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.028+0000 7fe4fa095640 1 -- 172.21.15.28:0/675081526 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fe4e006e0b0 msgr2=0x7fe4e0070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:19.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.028+0000 7fe4fa095640 1 --2- 172.21.15.28:0/675081526 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fe4e006e0b0 0x7fe4e0070560 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc004540 tx=0x7fe4dc035040 comp rx=0 tx=0).stop 2024-07-03T15:51:19.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.028+0000 7fe4fa095640 1 -- 172.21.15.28:0/675081526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4f40fd600 msgr2=0x7fe4f410d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:19.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.028+0000 7fe4fa095640 1 --2- 172.21.15.28:0/675081526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4f40fd600 0x7fe4f410d480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe4e40027a0 tx=0x7fe4e4009f30 comp rx=0 tx=0).stop 2024-07-03T15:51:19.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.028+0000 7fe4fa095640 1 -- 172.21.15.28:0/675081526 shutdown_connections 2024-07-03T15:51:19.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.028+0000 7fe4fa095640 1 --2- 172.21.15.28:0/675081526 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fe4e006e0b0 0x7fe4e0070560 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:19.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.028+0000 7fe4fa095640 1 --2- 172.21.15.28:0/675081526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4f40fd600 0x7fe4f410d480 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:19.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.028+0000 7fe4fa095640 1 --2- 172.21.15.28:0/675081526 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4f40fc330 0x7fe4f410cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:19.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.028+0000 7fe4fa095640 1 -- 172.21.15.28:0/675081526 >> 172.21.15.28:0/675081526 conn(0x7fe4f40f7af0 msgr2=0x7fe4f40f91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:51:19.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.028+0000 7fe4fa095640 1 -- 172.21.15.28:0/675081526 shutdown_connections 2024-07-03T15:51:19.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.028+0000 7fe4fa095640 1 -- 172.21.15.28:0/675081526 wait complete. 2024-07-03T15:51:19.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.111+0000 7f7a3a070640 1 -- 172.21.15.28:0/953058227 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7a340659f0 msgr2=0x7f7a341013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:19.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.111+0000 7f7a3a070640 1 --2- 172.21.15.28:0/953058227 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7a340659f0 0x7f7a341013b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7a240048d0 tx=0x7f7a24009da0 comp rx=0 tx=0).stop 2024-07-03T15:51:19.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.111+0000 7f7a3a070640 1 -- 172.21.15.28:0/953058227 shutdown_connections 2024-07-03T15:51:19.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.111+0000 7f7a3a070640 1 --2- 172.21.15.28:0/953058227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a341018f0 0x7f7a34103cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:19.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.111+0000 7f7a3a070640 1 --2- 172.21.15.28:0/953058227 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7a340659f0 0x7f7a341013b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:19.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.111+0000 7f7a3a070640 1 -- 172.21.15.28:0/953058227 >> 172.21.15.28:0/953058227 conn(0x7f7a340f6a20 msgr2=0x7f7a340f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:51:19.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.112+0000 7f7a3a070640 1 -- 172.21.15.28:0/953058227 shutdown_connections 2024-07-03T15:51:19.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.112+0000 7f7a3a070640 1 -- 172.21.15.28:0/953058227 wait complete. 2024-07-03T15:51:19.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.112+0000 7f7a3a070640 1 Processor -- start 2024-07-03T15:51:19.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.112+0000 7f7a3a070640 1 -- start start 2024-07-03T15:51:19.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.113+0000 7f7a3a070640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7a340659f0 0x7f7a3410f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:51:19.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.113+0000 7f7a3a070640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a341018f0 0x7f7a3410f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:51:19.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.113+0000 7f7a3a070640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3410fd10 con 0x7f7a340659f0 2024-07-03T15:51:19.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.113+0000 7f7a3a070640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a34110530 con 0x7f7a341018f0 2024-07-03T15:51:19.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.113+0000 7f7a337fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7a340659f0 0x7f7a3410f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:51:19.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.113+0000 7f7a337fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7a340659f0 0x7f7a3410f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49632/0 (socket says 172.21.15.28:49632) 2024-07-03T15:51:19.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.113+0000 7f7a337fe640 1 -- 172.21.15.28:0/1895184849 learned_addr learned my addr 172.21.15.28:0/1895184849 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:51:19.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.113+0000 7f7a32ffd640 1 --2- 172.21.15.28:0/1895184849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a341018f0 0x7f7a3410f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:51:19.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.113+0000 7f7a32ffd640 1 -- 172.21.15.28:0/1895184849 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7a340659f0 msgr2=0x7f7a3410f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:19.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.113+0000 7f7a32ffd640 1 --2- 172.21.15.28:0/1895184849 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7a340659f0 0x7f7a3410f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:19.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.113+0000 7f7a32ffd640 1 -- 172.21.15.28:0/1895184849 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a24005040 con 0x7f7a341018f0 2024-07-03T15:51:19.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.113+0000 7f7a337fe640 1 --2- 172.21.15.28:0/1895184849 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7a340659f0 0x7f7a3410f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-03T15:51:19.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.113+0000 7f7a32ffd640 1 --2- 172.21.15.28:0/1895184849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a341018f0 0x7f7a3410f6c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7a2800e910 tx=0x7f7a2800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:51:19.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.113+0000 7f7a1bfff640 1 -- 172.21.15.28:0/1895184849 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a2800eea0 con 0x7f7a341018f0 2024-07-03T15:51:19.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.114+0000 7f7a3a070640 1 -- 172.21.15.28:0/1895184849 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a34110760 con 0x7f7a341018f0 2024-07-03T15:51:19.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.114+0000 7f7a1bfff640 1 -- 172.21.15.28:0/1895184849 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7a2800f040 con 0x7f7a341018f0 2024-07-03T15:51:19.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.114+0000 7f7a1bfff640 1 -- 172.21.15.28:0/1895184849 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a280094a0 con 0x7f7a341018f0 2024-07-03T15:51:19.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.114+0000 7f7a3a070640 1 -- 172.21.15.28:0/1895184849 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a34110d00 con 0x7f7a341018f0 2024-07-03T15:51:19.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.115+0000 7f7a3a070640 1 -- 172.21.15.28:0/1895184849 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a340fa6a0 con 0x7f7a341018f0 2024-07-03T15:51:19.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.115+0000 7f7a1bfff640 1 -- 172.21.15.28:0/1895184849 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7a28009710 con 0x7f7a341018f0 2024-07-03T15:51:19.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.115+0000 7f7a1bfff640 1 --2- 172.21.15.28:0/1895184849 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f7a1c06e0b0 0x7f7a1c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:51:19.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.116+0000 7f7a1bfff640 1 -- 172.21.15.28:0/1895184849 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f7a2808d750 con 0x7f7a341018f0 2024-07-03T15:51:19.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.116+0000 7f7a337fe640 1 --2- 172.21.15.28:0/1895184849 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f7a1c06e0b0 0x7f7a1c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:51:19.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.116+0000 7f7a337fe640 1 --2- 172.21.15.28:0/1895184849 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f7a1c06e0b0 0x7f7a1c070560 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7a240043d0 tx=0x7f7a24035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:51:19.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.120+0000 7f7a1bfff640 1 -- 172.21.15.28:0/1895184849 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f7a28056ab0 con 0x7f7a341018f0 2024-07-03T15:51:19.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.314+0000 7f7a3a070640 1 -- 172.21.15.28:0/1895184849 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a34111030 con 0x7f7a1c06e0b0 2024-07-03T15:51:19.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.325+0000 7f7a1bfff640 1 -- 172.21.15.28:0/1895184849 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f7a34111030 con 0x7f7a1c06e0b0 2024-07-03T15:51:19.328 INFO:teuthology.orchestra.run.smithi028.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-03T15:51:19.329 INFO:teuthology.orchestra.run.smithi028.stdout:alertmanager.smithi028 smithi028 *:9093,9094 running (6m) 20s ago 14m 24.7M - 0.25.0 c8568f914cd2 1425315a0de9 2024-07-03T15:51:19.329 INFO:teuthology.orchestra.run.smithi028.stdout:crash.smithi028 smithi028 running (4m) 20s ago 15m 6895k - 19.0.0-2550-gc00506dc 9d49405603e8 c62c2f45cabe 2024-07-03T15:51:19.329 INFO:teuthology.orchestra.run.smithi028.stdout:crash.smithi148 smithi148 running (4m) 4s ago 14m 6895k - 19.0.0-2550-gc00506dc 9d49405603e8 b1318dcaacb6 2024-07-03T15:51:19.329 INFO:teuthology.orchestra.run.smithi028.stdout:grafana.smithi028 smithi028 *:3000 running (6m) 20s ago 14m 82.2M - 9.4.7 954c08fa6188 f9f17c92cfea 2024-07-03T15:51:19.329 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi028.rhpvve smithi028 running (41s) 20s ago 9m 13.1M - 19.0.0-2550-gc00506dc 9d49405603e8 5d528ea2a2f0 2024-07-03T15:51:19.329 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi028.wmbxew smithi028 running (24s) 20s ago 9m 11.4M - 19.0.0-2550-gc00506dc 9d49405603e8 ebdbfa592ac8 2024-07-03T15:51:19.329 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi148.szomfl smithi148 running (9m) 4s ago 9m 80.1M - 17.2.7-1047-gd4b063ab 4ca399d9e65d d94f5c95fa8d 2024-07-03T15:51:19.329 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi148.xgpuld smithi148 running (7s) 4s ago 9m 13.2M - 19.0.0-2550-gc00506dc 9d49405603e8 f24ab04507dc 2024-07-03T15:51:19.329 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.smithi028.vtqrtz smithi028 *:8443,9283,8765 running (6m) 20s ago 16m 503M - 19.0.0-2550-gc00506dc 9d49405603e8 14d0363c5b77 2024-07-03T15:51:19.329 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.smithi148.rjzsar smithi148 *:8443,9283,8765 running (5m) 4s ago 13m 441M - 19.0.0-2550-gc00506dc 9d49405603e8 c0193aa8a5dd 2024-07-03T15:51:19.329 INFO:teuthology.orchestra.run.smithi028.stdout:mon.smithi028 smithi028 running (5m) 20s ago 16m 52.0M 2048M 19.0.0-2550-gc00506dc 9d49405603e8 5093f7bf47d7 2024-07-03T15:51:19.329 INFO:teuthology.orchestra.run.smithi028.stdout:mon.smithi148 smithi148 running (4m) 4s ago 13m 37.1M 2048M 19.0.0-2550-gc00506dc 9d49405603e8 d338d2c648da 2024-07-03T15:51:19.330 INFO:teuthology.orchestra.run.smithi028.stdout:node-exporter.smithi028 smithi028 *:9100 running (6m) 20s ago 15m 15.0M - 1.5.0 0da6a335fe13 91ca1f49de0e 2024-07-03T15:51:19.330 INFO:teuthology.orchestra.run.smithi028.stdout:node-exporter.smithi148 smithi148 *:9100 running (6m) 4s ago 14m 16.6M - 1.5.0 0da6a335fe13 1eaaf7b1b561 2024-07-03T15:51:19.330 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0 smithi028 running (3m) 20s ago 13m 109M 1558M 19.0.0-2550-gc00506dc 9d49405603e8 4454cecff8fc 2024-07-03T15:51:19.330 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1 smithi028 running (3m) 20s ago 12m 114M 1558M 19.0.0-2550-gc00506dc 9d49405603e8 af67f7088643 2024-07-03T15:51:19.330 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2 smithi028 running (2m) 20s ago 12m 96.8M 1558M 19.0.0-2550-gc00506dc 9d49405603e8 5d160a327dc2 2024-07-03T15:51:19.330 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3 smithi148 running (2m) 4s ago 11m 118M 2581M 19.0.0-2550-gc00506dc 9d49405603e8 1d622eba4b3b 2024-07-03T15:51:19.330 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4 smithi148 running (91s) 4s ago 11m 105M 2581M 19.0.0-2550-gc00506dc 9d49405603e8 546343d0f31f 2024-07-03T15:51:19.330 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5 smithi148 running (59s) 4s ago 11m 103M 2581M 19.0.0-2550-gc00506dc 9d49405603e8 a819fcc3b569 2024-07-03T15:51:19.330 INFO:teuthology.orchestra.run.smithi028.stdout:prometheus.smithi028 smithi028 *:9095 running (5m) 20s ago 14m 62.6M - 2.43.0 a07b618ecd1d 2b0eb2a95afd 2024-07-03T15:51:19.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.328+0000 7f7a3a070640 1 -- 172.21.15.28:0/1895184849 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f7a1c06e0b0 msgr2=0x7f7a1c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:19.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.328+0000 7f7a3a070640 1 --2- 172.21.15.28:0/1895184849 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f7a1c06e0b0 0x7f7a1c070560 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7a240043d0 tx=0x7f7a24035040 comp rx=0 tx=0).stop 2024-07-03T15:51:19.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.328+0000 7f7a3a070640 1 -- 172.21.15.28:0/1895184849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a341018f0 msgr2=0x7f7a3410f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:19.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.328+0000 7f7a3a070640 1 --2- 172.21.15.28:0/1895184849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a341018f0 0x7f7a3410f6c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7a2800e910 tx=0x7f7a2800b040 comp rx=0 tx=0).stop 2024-07-03T15:51:19.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.328+0000 7f7a3a070640 1 -- 172.21.15.28:0/1895184849 shutdown_connections 2024-07-03T15:51:19.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.328+0000 7f7a3a070640 1 --2- 172.21.15.28:0/1895184849 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f7a1c06e0b0 0x7f7a1c070560 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:19.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.328+0000 7f7a3a070640 1 --2- 172.21.15.28:0/1895184849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a341018f0 0x7f7a3410f6c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:19.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.328+0000 7f7a3a070640 1 --2- 172.21.15.28:0/1895184849 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7a340659f0 0x7f7a3410f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:19.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.328+0000 7f7a3a070640 1 -- 172.21.15.28:0/1895184849 >> 172.21.15.28:0/1895184849 conn(0x7f7a340f6a20 msgr2=0x7f7a340f6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:51:19.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.328+0000 7f7a3a070640 1 -- 172.21.15.28:0/1895184849 shutdown_connections 2024-07-03T15:51:19.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.328+0000 7f7a3a070640 1 -- 172.21.15.28:0/1895184849 wait complete. 2024-07-03T15:51:19.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.410+0000 7fc5a136c640 1 -- 172.21.15.28:0/1281592364 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc59c0fe300 msgr2=0x7fc59c0fa460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:19.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.410+0000 7fc5a136c640 1 --2- 172.21.15.28:0/1281592364 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc59c0fe300 0x7fc59c0fa460 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc5900048d0 tx=0x7fc590009da0 comp rx=0 tx=0).stop 2024-07-03T15:51:19.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.411+0000 7fc5a136c640 1 -- 172.21.15.28:0/1281592364 shutdown_connections 2024-07-03T15:51:19.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.411+0000 7fc5a136c640 1 --2- 172.21.15.28:0/1281592364 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc59c0fe300 0x7fc59c0fa460 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:19.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.411+0000 7fc5a136c640 1 --2- 172.21.15.28:0/1281592364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc59c0fd9b0 0x7fc59c0fddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:19.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.411+0000 7fc5a136c640 1 -- 172.21.15.28:0/1281592364 >> 172.21.15.28:0/1281592364 conn(0x7fc59c0f61b0 msgr2=0x7fc59c0f85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:51:19.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.411+0000 7fc5a136c640 1 -- 172.21.15.28:0/1281592364 shutdown_connections 2024-07-03T15:51:19.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.411+0000 7fc5a136c640 1 -- 172.21.15.28:0/1281592364 wait complete. 2024-07-03T15:51:19.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.411+0000 7fc5a136c640 1 Processor -- start 2024-07-03T15:51:19.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.411+0000 7fc5a136c640 1 -- start start 2024-07-03T15:51:19.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.412+0000 7fc5a136c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc59c0fd9b0 0x7fc59c10cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:51:19.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.412+0000 7fc5a136c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc59c0fe300 0x7fc59c10d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:51:19.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.412+0000 7fc5a136c640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc59c10dad0 con 0x7fc59c0fd9b0 2024-07-03T15:51:19.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.412+0000 7fc5a136c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc59c10e2f0 con 0x7fc59c0fe300 2024-07-03T15:51:19.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.412+0000 7fc59affd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc59c0fd9b0 0x7fc59c10cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:51:19.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.412+0000 7fc59affd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc59c0fd9b0 0x7fc59c10cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49648/0 (socket says 172.21.15.28:49648) 2024-07-03T15:51:19.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.412+0000 7fc59affd640 1 -- 172.21.15.28:0/3461107257 learned_addr learned my addr 172.21.15.28:0/3461107257 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:51:19.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.412+0000 7fc59a7fc640 1 --2- 172.21.15.28:0/3461107257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc59c0fe300 0x7fc59c10d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:51:19.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.412+0000 7fc59affd640 1 -- 172.21.15.28:0/3461107257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc59c0fe300 msgr2=0x7fc59c10d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:19.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.412+0000 7fc59affd640 1 --2- 172.21.15.28:0/3461107257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc59c0fe300 0x7fc59c10d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:19.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.412+0000 7fc59affd640 1 -- 172.21.15.28:0/3461107257 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc590005040 con 0x7fc59c0fd9b0 2024-07-03T15:51:19.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.412+0000 7fc59affd640 1 --2- 172.21.15.28:0/3461107257 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc59c0fd9b0 0x7fc59c10cf40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc58c002a50 tx=0x7fc58c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:51:19.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.413+0000 7fc5837fe640 1 -- 172.21.15.28:0/3461107257 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc58c007c30 con 0x7fc59c0fd9b0 2024-07-03T15:51:19.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.413+0000 7fc5837fe640 1 -- 172.21.15.28:0/3461107257 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc58c00f040 con 0x7fc59c0fd9b0 2024-07-03T15:51:19.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.413+0000 7fc5a136c640 1 -- 172.21.15.28:0/3461107257 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc59c10e580 con 0x7fc59c0fd9b0 2024-07-03T15:51:19.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.413+0000 7fc5837fe640 1 -- 172.21.15.28:0/3461107257 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc58c00db60 con 0x7fc59c0fd9b0 2024-07-03T15:51:19.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.413+0000 7fc5a136c640 1 -- 172.21.15.28:0/3461107257 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc59c10eaf0 con 0x7fc59c0fd9b0 2024-07-03T15:51:19.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.414+0000 7fc5a136c640 1 -- 172.21.15.28:0/3461107257 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc59c1056e0 con 0x7fc59c0fd9b0 2024-07-03T15:51:19.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.414+0000 7fc5837fe640 1 -- 172.21.15.28:0/3461107257 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc58c00dcc0 con 0x7fc59c0fd9b0 2024-07-03T15:51:19.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.415+0000 7fc5837fe640 1 --2- 172.21.15.28:0/3461107257 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc58406e0b0 0x7fc584070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:51:19.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.415+0000 7fc5837fe640 1 -- 172.21.15.28:0/3461107257 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fc58c08da40 con 0x7fc59c0fd9b0 2024-07-03T15:51:19.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.415+0000 7fc59a7fc640 1 --2- 172.21.15.28:0/3461107257 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc58406e0b0 0x7fc584070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:51:19.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.415+0000 7fc59a7fc640 1 --2- 172.21.15.28:0/3461107257 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc58406e0b0 0x7fc584070560 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc590000c00 tx=0x7fc590035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:51:19.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.419+0000 7fc5837fe640 1 -- 172.21.15.28:0/3461107257 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc58c056d70 con 0x7fc59c0fd9b0 2024-07-03T15:51:19.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:19 smithi148 ceph-mon[80490]: pgmap v184: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail 2024-07-03T15:51:19.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:19 smithi148 ceph-mon[80490]: Detected new or changed devices on smithi148 2024-07-03T15:51:19.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:19 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:19.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:19 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:19.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:19 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:51:19.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:19 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:51:19.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:19 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:19.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:19 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:51:19.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:19 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:19.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:19 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:19.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:19 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:19.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:19 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:19.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:19 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi148.szomfl"]}]: dispatch 2024-07-03T15:51:19.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:19 smithi148 ceph-mon[80490]: Upgrade: It appears safe to stop mds.cephfs.smithi148.szomfl 2024-07-03T15:51:19.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:19 smithi148 ceph-mon[80490]: from='client.44303 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:51:19.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:19 smithi028 ceph-mon[92858]: pgmap v184: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail 2024-07-03T15:51:19.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:19 smithi028 ceph-mon[92858]: Detected new or changed devices on smithi148 2024-07-03T15:51:19.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:19 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:19.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:19 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:19.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:19 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:51:19.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:19 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:51:19.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:19 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:19.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:19 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:51:19.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:19 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:19.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:19 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:19.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:19 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:19.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:19 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:19.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:19 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi148.szomfl"]}]: dispatch 2024-07-03T15:51:19.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:19 smithi028 ceph-mon[92858]: Upgrade: It appears safe to stop mds.cephfs.smithi148.szomfl 2024-07-03T15:51:19.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:19 smithi028 ceph-mon[92858]: from='client.44303 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:51:19.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.680+0000 7fc5a136c640 1 -- 172.21.15.28:0/3461107257 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc59c10edd0 con 0x7fc59c0fd9b0 2024-07-03T15:51:19.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.681+0000 7fc5837fe640 1 -- 172.21.15.28:0/3461107257 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+834 (secure 0 0 0) 0x7fc58c05a430 con 0x7fc59c0fd9b0 2024-07-03T15:51:19.685 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-07-03T15:51:19.685 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": { 2024-07-03T15:51:19.685 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 2 2024-07-03T15:51:19.685 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:51:19.685 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": { 2024-07-03T15:51:19.685 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 2 2024-07-03T15:51:19.685 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:51:19.685 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": { 2024-07-03T15:51:19.685 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 6 2024-07-03T15:51:19.685 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:51:19.685 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": { 2024-07-03T15:51:19.686 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 1, 2024-07-03T15:51:19.686 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 3 2024-07-03T15:51:19.686 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:51:19.686 INFO:teuthology.orchestra.run.smithi028.stdout: "overall": { 2024-07-03T15:51:19.686 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 17.2.7-1047-gd4b063ab (d4b063aba337ced2097ed32abf475d4c9292d8bc) quincy (stable)": 1, 2024-07-03T15:51:19.686 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 13 2024-07-03T15:51:19.686 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-07-03T15:51:19.686 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-07-03T15:51:19.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.683+0000 7fc5a136c640 1 -- 172.21.15.28:0/3461107257 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc58406e0b0 msgr2=0x7fc584070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:19.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.683+0000 7fc5a136c640 1 --2- 172.21.15.28:0/3461107257 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc58406e0b0 0x7fc584070560 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc590000c00 tx=0x7fc590035040 comp rx=0 tx=0).stop 2024-07-03T15:51:19.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.683+0000 7fc5a136c640 1 -- 172.21.15.28:0/3461107257 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc59c0fd9b0 msgr2=0x7fc59c10cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:19.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.683+0000 7fc5a136c640 1 --2- 172.21.15.28:0/3461107257 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc59c0fd9b0 0x7fc59c10cf40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc58c002a50 tx=0x7fc58c004600 comp rx=0 tx=0).stop 2024-07-03T15:51:19.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.683+0000 7fc5a136c640 1 -- 172.21.15.28:0/3461107257 shutdown_connections 2024-07-03T15:51:19.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.683+0000 7fc5a136c640 1 --2- 172.21.15.28:0/3461107257 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc58406e0b0 0x7fc584070560 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:19.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.683+0000 7fc5a136c640 1 --2- 172.21.15.28:0/3461107257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc59c0fe300 0x7fc59c10d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:19.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.683+0000 7fc5a136c640 1 --2- 172.21.15.28:0/3461107257 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc59c0fd9b0 0x7fc59c10cf40 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:19.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.683+0000 7fc5a136c640 1 -- 172.21.15.28:0/3461107257 >> 172.21.15.28:0/3461107257 conn(0x7fc59c0f61b0 msgr2=0x7fc59c0f7830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:51:19.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.683+0000 7fc5a136c640 1 -- 172.21.15.28:0/3461107257 shutdown_connections 2024-07-03T15:51:19.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.683+0000 7fc5a136c640 1 -- 172.21.15.28:0/3461107257 wait complete. 2024-07-03T15:51:19.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.765+0000 7fc28c354640 1 -- 172.21.15.28:0/3796613901 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc2840fd630 msgr2=0x7fc2840fdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:19.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.765+0000 7fc28c354640 1 --2- 172.21.15.28:0/3796613901 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc2840fd630 0x7fc2840fdaa0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc2800048d0 tx=0x7fc280009da0 comp rx=0 tx=0).stop 2024-07-03T15:51:19.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.765+0000 7fc28c354640 1 -- 172.21.15.28:0/3796613901 shutdown_connections 2024-07-03T15:51:19.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.765+0000 7fc28c354640 1 --2- 172.21.15.28:0/3796613901 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc2840fd630 0x7fc2840fdaa0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:19.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.765+0000 7fc28c354640 1 --2- 172.21.15.28:0/3796613901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2840fc360 0x7fc2840fc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:19.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.765+0000 7fc28c354640 1 -- 172.21.15.28:0/3796613901 >> 172.21.15.28:0/3796613901 conn(0x7fc2840f7b40 msgr2=0x7fc2840f9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:51:19.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.765+0000 7fc28c354640 1 -- 172.21.15.28:0/3796613901 shutdown_connections 2024-07-03T15:51:19.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.765+0000 7fc28c354640 1 -- 172.21.15.28:0/3796613901 wait complete. 2024-07-03T15:51:19.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.765+0000 7fc28c354640 1 Processor -- start 2024-07-03T15:51:19.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.766+0000 7fc28c354640 1 -- start start 2024-07-03T15:51:19.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.766+0000 7fc28c354640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc2840fc360 0x7fc28410cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:51:19.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.766+0000 7fc28c354640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2840fd630 0x7fc28410d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:51:19.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.766+0000 7fc28c354640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28410e390 con 0x7fc2840fc360 2024-07-03T15:51:19.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.766+0000 7fc28c354640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28410e500 con 0x7fc2840fd630 2024-07-03T15:51:19.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.766+0000 7fc28a0f0640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc2840fc360 0x7fc28410cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:51:19.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.766+0000 7fc28a0f0640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc2840fc360 0x7fc28410cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49666/0 (socket says 172.21.15.28:49666) 2024-07-03T15:51:19.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.766+0000 7fc28a0f0640 1 -- 172.21.15.28:0/1213114309 learned_addr learned my addr 172.21.15.28:0/1213114309 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:51:19.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.766+0000 7fc2898ef640 1 --2- 172.21.15.28:0/1213114309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2840fd630 0x7fc28410d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:51:19.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.766+0000 7fc28a0f0640 1 -- 172.21.15.28:0/1213114309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2840fd630 msgr2=0x7fc28410d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:19.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.766+0000 7fc28a0f0640 1 --2- 172.21.15.28:0/1213114309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2840fd630 0x7fc28410d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:19.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.766+0000 7fc28a0f0640 1 -- 172.21.15.28:0/1213114309 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc280005040 con 0x7fc2840fc360 2024-07-03T15:51:19.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.766+0000 7fc28a0f0640 1 --2- 172.21.15.28:0/1213114309 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc2840fc360 0x7fc28410cfe0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc27800b790 tx=0x7fc2780091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:51:19.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.766+0000 7fc276ffd640 1 -- 172.21.15.28:0/1213114309 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc27800c700 con 0x7fc2840fc360 2024-07-03T15:51:19.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.766+0000 7fc276ffd640 1 -- 172.21.15.28:0/1213114309 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc27800cd30 con 0x7fc2840fc360 2024-07-03T15:51:19.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.767+0000 7fc28c354640 1 -- 172.21.15.28:0/1213114309 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc28410e7e0 con 0x7fc2840fc360 2024-07-03T15:51:19.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.767+0000 7fc276ffd640 1 -- 172.21.15.28:0/1213114309 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc278013370 con 0x7fc2840fc360 2024-07-03T15:51:19.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.767+0000 7fc28c354640 1 -- 172.21.15.28:0/1213114309 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc28410ed00 con 0x7fc2840fc360 2024-07-03T15:51:19.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.768+0000 7fc28c354640 1 -- 172.21.15.28:0/1213114309 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc284105740 con 0x7fc2840fc360 2024-07-03T15:51:19.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.768+0000 7fc276ffd640 1 -- 172.21.15.28:0/1213114309 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc278012070 con 0x7fc2840fc360 2024-07-03T15:51:19.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.769+0000 7fc276ffd640 1 --2- 172.21.15.28:0/1213114309 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc27006e0b0 0x7fc270070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:51:19.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.769+0000 7fc276ffd640 1 -- 172.21.15.28:0/1213114309 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fc27808d490 con 0x7fc2840fc360 2024-07-03T15:51:19.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.769+0000 7fc2898ef640 1 --2- 172.21.15.28:0/1213114309 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc27006e0b0 0x7fc270070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:51:19.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.769+0000 7fc2898ef640 1 --2- 172.21.15.28:0/1213114309 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc27006e0b0 0x7fc270070560 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc28002a530 tx=0x7fc280009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:51:19.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:19.773+0000 7fc276ffd640 1 -- 172.21.15.28:0/1213114309 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc278056840 con 0x7fc2840fc360 2024-07-03T15:51:20.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.012+0000 7fc28c354640 1 -- 172.21.15.28:0/1213114309 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc28404b530 con 0x7fc2840fc360 2024-07-03T15:51:20.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.012+0000 7fc276ffd640 1 -- 172.21.15.28:0/1213114309 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1896 (secure 0 0 0) 0x7fc278059f00 con 0x7fc2840fc360 2024-07-03T15:51:20.016 INFO:teuthology.orchestra.run.smithi028.stdout:e29 2024-07-03T15:51:20.016 INFO:teuthology.orchestra.run.smithi028.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-03T15:51:20.016 INFO:teuthology.orchestra.run.smithi028.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:51:20.017 INFO:teuthology.orchestra.run.smithi028.stdout:legacy client fscid: 1 2024-07-03T15:51:20.017 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:51:20.017 INFO:teuthology.orchestra.run.smithi028.stdout:Filesystem 'cephfs' (1) 2024-07-03T15:51:20.017 INFO:teuthology.orchestra.run.smithi028.stdout:fs_name cephfs 2024-07-03T15:51:20.017 INFO:teuthology.orchestra.run.smithi028.stdout:epoch 25 2024-07-03T15:51:20.017 INFO:teuthology.orchestra.run.smithi028.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-07-03T15:51:20.017 INFO:teuthology.orchestra.run.smithi028.stdout:created 2024-07-03T15:41:40.656298+0000 2024-07-03T15:51:20.017 INFO:teuthology.orchestra.run.smithi028.stdout:modified 2024-07-03T15:50:42.345494+0000 2024-07-03T15:51:20.017 INFO:teuthology.orchestra.run.smithi028.stdout:tableserver 0 2024-07-03T15:51:20.017 INFO:teuthology.orchestra.run.smithi028.stdout:root 0 2024-07-03T15:51:20.017 INFO:teuthology.orchestra.run.smithi028.stdout:session_timeout 60 2024-07-03T15:51:20.017 INFO:teuthology.orchestra.run.smithi028.stdout:session_autoclose 300 2024-07-03T15:51:20.018 INFO:teuthology.orchestra.run.smithi028.stdout:max_file_size 1099511627776 2024-07-03T15:51:20.018 INFO:teuthology.orchestra.run.smithi028.stdout:max_xattr_size 65536 2024-07-03T15:51:20.018 INFO:teuthology.orchestra.run.smithi028.stdout:required_client_features {} 2024-07-03T15:51:20.018 INFO:teuthology.orchestra.run.smithi028.stdout:last_failure 0 2024-07-03T15:51:20.018 INFO:teuthology.orchestra.run.smithi028.stdout:last_failure_osd_epoch 79 2024-07-03T15:51:20.018 INFO:teuthology.orchestra.run.smithi028.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:51:20.018 INFO:teuthology.orchestra.run.smithi028.stdout:max_mds 1 2024-07-03T15:51:20.018 INFO:teuthology.orchestra.run.smithi028.stdout:in 0 2024-07-03T15:51:20.018 INFO:teuthology.orchestra.run.smithi028.stdout:up {0=24277} 2024-07-03T15:51:20.018 INFO:teuthology.orchestra.run.smithi028.stdout:failed 2024-07-03T15:51:20.018 INFO:teuthology.orchestra.run.smithi028.stdout:damaged 2024-07-03T15:51:20.018 INFO:teuthology.orchestra.run.smithi028.stdout:stopped 2024-07-03T15:51:20.018 INFO:teuthology.orchestra.run.smithi028.stdout:data_pools [3] 2024-07-03T15:51:20.019 INFO:teuthology.orchestra.run.smithi028.stdout:metadata_pool 2 2024-07-03T15:51:20.019 INFO:teuthology.orchestra.run.smithi028.stdout:inline_data disabled 2024-07-03T15:51:20.019 INFO:teuthology.orchestra.run.smithi028.stdout:balancer 2024-07-03T15:51:20.019 INFO:teuthology.orchestra.run.smithi028.stdout:bal_rank_mask -1 2024-07-03T15:51:20.019 INFO:teuthology.orchestra.run.smithi028.stdout:standby_count_wanted 1 2024-07-03T15:51:20.019 INFO:teuthology.orchestra.run.smithi028.stdout:qdb_cluster leader: 24277 members: 24277 2024-07-03T15:51:20.019 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi148.szomfl{0:24277} state up:active seq 135 join_fscid=1 addr [v2:172.21.15.148:6826/4171231858,v1:172.21.15.148:6827/4171231858] compat {c=[1],r=[1],i=[7ff]}] 2024-07-03T15:51:20.019 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:51:20.019 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:51:20.019 INFO:teuthology.orchestra.run.smithi028.stdout:Standby daemons: 2024-07-03T15:51:20.019 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:51:20.019 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi028.rhpvve{-1:34376} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.28:6826/3127260162,v1:172.21.15.28:6827/3127260162] compat {c=[1],r=[1],i=[1fff]}] 2024-07-03T15:51:20.019 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi028.wmbxew{-1:34408} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.28:6828/2908130327,v1:172.21.15.28:6829/2908130327] compat {c=[1],r=[1],i=[1fff]}] 2024-07-03T15:51:20.020 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi148.xgpuld{-1:44301} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/1480077002,v1:172.21.15.148:6825/1480077002] compat {c=[1],r=[1],i=[1fff]}] 2024-07-03T15:51:20.021 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 29 2024-07-03T15:51:20.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.015+0000 7fc28c354640 1 -- 172.21.15.28:0/1213114309 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc27006e0b0 msgr2=0x7fc270070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:20.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.015+0000 7fc28c354640 1 --2- 172.21.15.28:0/1213114309 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc27006e0b0 0x7fc270070560 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc28002a530 tx=0x7fc280009da0 comp rx=0 tx=0).stop 2024-07-03T15:51:20.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.015+0000 7fc28c354640 1 -- 172.21.15.28:0/1213114309 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc2840fc360 msgr2=0x7fc28410cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:20.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.015+0000 7fc28c354640 1 --2- 172.21.15.28:0/1213114309 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc2840fc360 0x7fc28410cfe0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc27800b790 tx=0x7fc2780091d0 comp rx=0 tx=0).stop 2024-07-03T15:51:20.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.015+0000 7fc28c354640 1 -- 172.21.15.28:0/1213114309 shutdown_connections 2024-07-03T15:51:20.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.015+0000 7fc28c354640 1 --2- 172.21.15.28:0/1213114309 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc27006e0b0 0x7fc270070560 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:20.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.015+0000 7fc28c354640 1 --2- 172.21.15.28:0/1213114309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2840fd630 0x7fc28410d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:20.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.015+0000 7fc28c354640 1 --2- 172.21.15.28:0/1213114309 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc2840fc360 0x7fc28410cfe0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:20.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.015+0000 7fc28c354640 1 -- 172.21.15.28:0/1213114309 >> 172.21.15.28:0/1213114309 conn(0x7fc2840f7b40 msgr2=0x7fc2840f9230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:51:20.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.016+0000 7fc28c354640 1 -- 172.21.15.28:0/1213114309 shutdown_connections 2024-07-03T15:51:20.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.016+0000 7fc28c354640 1 -- 172.21.15.28:0/1213114309 wait complete. 2024-07-03T15:51:20.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.099+0000 7f1e03383640 1 -- 172.21.15.28:0/1276236325 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1dfc0ff830 msgr2=0x7f1dfc0ffca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:20.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.099+0000 7f1e03383640 1 --2- 172.21.15.28:0/1276236325 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1dfc0ff830 0x7f1dfc0ffca0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1dec0048d0 tx=0x7f1dec009da0 comp rx=0 tx=0).stop 2024-07-03T15:51:20.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.099+0000 7f1e03383640 1 -- 172.21.15.28:0/1276236325 shutdown_connections 2024-07-03T15:51:20.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.099+0000 7f1e03383640 1 --2- 172.21.15.28:0/1276236325 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1dfc0ff830 0x7f1dfc0ffca0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:20.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.099+0000 7f1e03383640 1 --2- 172.21.15.28:0/1276236325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dfc0fe560 0x7f1dfc0fe970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:20.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.099+0000 7f1e03383640 1 -- 172.21.15.28:0/1276236325 >> 172.21.15.28:0/1276236325 conn(0x7f1dfc0fa010 msgr2=0x7f1dfc0fc3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:51:20.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.100+0000 7f1e03383640 1 -- 172.21.15.28:0/1276236325 shutdown_connections 2024-07-03T15:51:20.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.100+0000 7f1e03383640 1 -- 172.21.15.28:0/1276236325 wait complete. 2024-07-03T15:51:20.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.100+0000 7f1e03383640 1 Processor -- start 2024-07-03T15:51:20.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.100+0000 7f1e03383640 1 -- start start 2024-07-03T15:51:20.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.101+0000 7f1e03383640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1dfc0fe560 0x7f1dfc10f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:51:20.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.101+0000 7f1e03383640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dfc0ff830 0x7f1dfc10f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:51:20.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.101+0000 7f1e03383640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc10fcf0 con 0x7f1dfc0fe560 2024-07-03T15:51:20.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.101+0000 7f1e03383640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc110510 con 0x7f1dfc0ff830 2024-07-03T15:51:20.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.101+0000 7f1e0111f640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1dfc0fe560 0x7f1dfc10f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:51:20.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.101+0000 7f1e0111f640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1dfc0fe560 0x7f1dfc10f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44756/0 (socket says 172.21.15.28:44756) 2024-07-03T15:51:20.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.101+0000 7f1e0111f640 1 -- 172.21.15.28:0/2131542705 learned_addr learned my addr 172.21.15.28:0/2131542705 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:51:20.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.101+0000 7f1e0091e640 1 --2- 172.21.15.28:0/2131542705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dfc0ff830 0x7f1dfc10f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:51:20.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.101+0000 7f1e0111f640 1 -- 172.21.15.28:0/2131542705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dfc0ff830 msgr2=0x7f1dfc10f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:20.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.101+0000 7f1e0111f640 1 --2- 172.21.15.28:0/2131542705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dfc0ff830 0x7f1dfc10f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:20.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.101+0000 7f1e0111f640 1 -- 172.21.15.28:0/2131542705 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dec005040 con 0x7f1dfc0fe560 2024-07-03T15:51:20.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.101+0000 7f1e0111f640 1 --2- 172.21.15.28:0/2131542705 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1dfc0fe560 0x7f1dfc10f160 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1df8002a50 tx=0x7f1df8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:51:20.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.101+0000 7f1df1ffb640 1 -- 172.21.15.28:0/2131542705 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1df8007c30 con 0x7f1dfc0fe560 2024-07-03T15:51:20.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.101+0000 7f1df1ffb640 1 -- 172.21.15.28:0/2131542705 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1df800f040 con 0x7f1dfc0fe560 2024-07-03T15:51:20.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.101+0000 7f1e03383640 1 -- 172.21.15.28:0/2131542705 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dfc1107a0 con 0x7f1dfc0fe560 2024-07-03T15:51:20.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.101+0000 7f1df1ffb640 1 -- 172.21.15.28:0/2131542705 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1df800db60 con 0x7f1dfc0fe560 2024-07-03T15:51:20.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.102+0000 7f1e03383640 1 -- 172.21.15.28:0/2131542705 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dfc110d10 con 0x7f1dfc0fe560 2024-07-03T15:51:20.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.103+0000 7f1e03383640 1 -- 172.21.15.28:0/2131542705 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dfc1078d0 con 0x7f1dfc0fe560 2024-07-03T15:51:20.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.103+0000 7f1df1ffb640 1 -- 172.21.15.28:0/2131542705 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1df800dcc0 con 0x7f1dfc0fe560 2024-07-03T15:51:20.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.104+0000 7f1df1ffb640 1 --2- 172.21.15.28:0/2131542705 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f1de806e0b0 0x7f1de8070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:51:20.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.104+0000 7f1df1ffb640 1 -- 172.21.15.28:0/2131542705 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f1df808da50 con 0x7f1dfc0fe560 2024-07-03T15:51:20.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.104+0000 7f1e0091e640 1 --2- 172.21.15.28:0/2131542705 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f1de806e0b0 0x7f1de8070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:51:20.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.104+0000 7f1e0091e640 1 --2- 172.21.15.28:0/2131542705 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f1de806e0b0 0x7f1de8070560 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1dec0043d0 tx=0x7f1dec035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:51:20.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.108+0000 7f1df1ffb640 1 -- 172.21.15.28:0/2131542705 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f1df8056d80 con 0x7f1dfc0fe560 2024-07-03T15:51:20.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.310+0000 7f1e03383640 1 -- 172.21.15.28:0/2131542705 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1dfc111040 con 0x7f1de806e0b0 2024-07-03T15:51:20.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.312+0000 7f1df1ffb640 1 -- 172.21.15.28:0/2131542705 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f1dfc111040 con 0x7f1de806e0b0 2024-07-03T15:51:20.315 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-07-03T15:51:20.315 INFO:teuthology.orchestra.run.smithi028.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2", 2024-07-03T15:51:20.315 INFO:teuthology.orchestra.run.smithi028.stdout: "in_progress": true, 2024-07-03T15:51:20.315 INFO:teuthology.orchestra.run.smithi028.stdout: "which": "Upgrading all daemon types on all hosts", 2024-07-03T15:51:20.315 INFO:teuthology.orchestra.run.smithi028.stdout: "services_complete": [ 2024-07-03T15:51:20.316 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr", 2024-07-03T15:51:20.316 INFO:teuthology.orchestra.run.smithi028.stdout: "crash", 2024-07-03T15:51:20.316 INFO:teuthology.orchestra.run.smithi028.stdout: "mon", 2024-07-03T15:51:20.316 INFO:teuthology.orchestra.run.smithi028.stdout: "osd" 2024-07-03T15:51:20.316 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-07-03T15:51:20.316 INFO:teuthology.orchestra.run.smithi028.stdout: "progress": "15/21 daemons upgraded", 2024-07-03T15:51:20.316 INFO:teuthology.orchestra.run.smithi028.stdout: "message": "Currently upgrading mds daemons", 2024-07-03T15:51:20.316 INFO:teuthology.orchestra.run.smithi028.stdout: "is_paused": false 2024-07-03T15:51:20.316 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-07-03T15:51:20.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.314+0000 7f1e03383640 1 -- 172.21.15.28:0/2131542705 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f1de806e0b0 msgr2=0x7f1de8070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:20.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.314+0000 7f1e03383640 1 --2- 172.21.15.28:0/2131542705 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f1de806e0b0 0x7f1de8070560 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1dec0043d0 tx=0x7f1dec035040 comp rx=0 tx=0).stop 2024-07-03T15:51:20.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.314+0000 7f1e03383640 1 -- 172.21.15.28:0/2131542705 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1dfc0fe560 msgr2=0x7f1dfc10f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:20.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.314+0000 7f1e03383640 1 --2- 172.21.15.28:0/2131542705 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1dfc0fe560 0x7f1dfc10f160 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1df8002a50 tx=0x7f1df8004600 comp rx=0 tx=0).stop 2024-07-03T15:51:20.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.314+0000 7f1e03383640 1 -- 172.21.15.28:0/2131542705 shutdown_connections 2024-07-03T15:51:20.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.314+0000 7f1e03383640 1 --2- 172.21.15.28:0/2131542705 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f1de806e0b0 0x7f1de8070560 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:20.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.314+0000 7f1e03383640 1 --2- 172.21.15.28:0/2131542705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dfc0ff830 0x7f1dfc10f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:20.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.314+0000 7f1e03383640 1 --2- 172.21.15.28:0/2131542705 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1dfc0fe560 0x7f1dfc10f160 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:20.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.314+0000 7f1e03383640 1 -- 172.21.15.28:0/2131542705 >> 172.21.15.28:0/2131542705 conn(0x7f1dfc0fa010 msgr2=0x7f1dfc0fc640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:51:20.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.314+0000 7f1e03383640 1 -- 172.21.15.28:0/2131542705 shutdown_connections 2024-07-03T15:51:20.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.314+0000 7f1e03383640 1 -- 172.21.15.28:0/2131542705 wait complete. 2024-07-03T15:51:20.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.396+0000 7f7ebf5fa640 1 -- 172.21.15.28:0/2066371451 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7eb80fd9b0 msgr2=0x7f7eb80fddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:20.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.396+0000 7f7ebf5fa640 1 --2- 172.21.15.28:0/2066371451 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7eb80fd9b0 0x7f7eb80fddc0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7eb40048d0 tx=0x7f7eb4009da0 comp rx=0 tx=0).stop 2024-07-03T15:51:20.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.396+0000 7f7ebf5fa640 1 -- 172.21.15.28:0/2066371451 shutdown_connections 2024-07-03T15:51:20.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.396+0000 7f7ebf5fa640 1 --2- 172.21.15.28:0/2066371451 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7eb80fe300 0x7f7eb80fa460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:20.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.396+0000 7f7ebf5fa640 1 --2- 172.21.15.28:0/2066371451 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7eb80fd9b0 0x7f7eb80fddc0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:20.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.396+0000 7f7ebf5fa640 1 -- 172.21.15.28:0/2066371451 >> 172.21.15.28:0/2066371451 conn(0x7f7eb80f61b0 msgr2=0x7f7eb80f85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:51:20.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.397+0000 7f7ebf5fa640 1 -- 172.21.15.28:0/2066371451 shutdown_connections 2024-07-03T15:51:20.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.397+0000 7f7ebf5fa640 1 -- 172.21.15.28:0/2066371451 wait complete. 2024-07-03T15:51:20.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.397+0000 7f7ebf5fa640 1 Processor -- start 2024-07-03T15:51:20.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.397+0000 7f7ebf5fa640 1 -- start start 2024-07-03T15:51:20.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.397+0000 7f7ebf5fa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7eb80fd9b0 0x7f7eb80fd150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:51:20.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.397+0000 7f7ebf5fa640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7eb80fe300 0x7f7eb80fb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:51:20.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.397+0000 7f7ebf5fa640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb80fbbd0 con 0x7f7eb80fe300 2024-07-03T15:51:20.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.397+0000 7f7ebf5fa640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb80fbd10 con 0x7f7eb80fd9b0 2024-07-03T15:51:20.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.398+0000 7f7ebcb95640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7eb80fe300 0x7f7eb80fb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:51:20.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.398+0000 7f7ebcb95640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7eb80fe300 0x7f7eb80fb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44782/0 (socket says 172.21.15.28:44782) 2024-07-03T15:51:20.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.398+0000 7f7ebcb95640 1 -- 172.21.15.28:0/1446101492 learned_addr learned my addr 172.21.15.28:0/1446101492 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:51:20.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.398+0000 7f7ebd396640 1 --2- 172.21.15.28:0/1446101492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7eb80fd9b0 0x7f7eb80fd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:51:20.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.398+0000 7f7ebcb95640 1 -- 172.21.15.28:0/1446101492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7eb80fd9b0 msgr2=0x7f7eb80fd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:20.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.398+0000 7f7ebcb95640 1 --2- 172.21.15.28:0/1446101492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7eb80fd9b0 0x7f7eb80fd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:20.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.398+0000 7f7ebcb95640 1 -- 172.21.15.28:0/1446101492 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb4005040 con 0x7f7eb80fe300 2024-07-03T15:51:20.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.398+0000 7f7ebcb95640 1 --2- 172.21.15.28:0/1446101492 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7eb80fe300 0x7f7eb80fb690 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8002870 tx=0x7f7ea8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:51:20.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.399+0000 7f7eadffb640 1 -- 172.21.15.28:0/1446101492 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ea8007c30 con 0x7f7eb80fe300 2024-07-03T15:51:20.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.399+0000 7f7eadffb640 1 -- 172.21.15.28:0/1446101492 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7ea800f040 con 0x7f7eb80fe300 2024-07-03T15:51:20.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.399+0000 7f7ebf5fa640 1 -- 172.21.15.28:0/1446101492 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eb80fbff0 con 0x7f7eb80fe300 2024-07-03T15:51:20.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.399+0000 7f7eadffb640 1 -- 172.21.15.28:0/1446101492 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ea800da90 con 0x7f7eb80fe300 2024-07-03T15:51:20.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.399+0000 7f7ebf5fa640 1 -- 172.21.15.28:0/1446101492 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eb8115080 con 0x7f7eb80fe300 2024-07-03T15:51:20.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.400+0000 7f7ebf5fa640 1 -- 172.21.15.28:0/1446101492 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eb8105720 con 0x7f7eb80fe300 2024-07-03T15:51:20.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.401+0000 7f7eadffb640 1 -- 172.21.15.28:0/1446101492 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7ea800dd50 con 0x7f7eb80fe300 2024-07-03T15:51:20.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.401+0000 7f7eadffb640 1 --2- 172.21.15.28:0/1446101492 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f7ea406e0b0 0x7f7ea4070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:51:20.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.401+0000 7f7eadffb640 1 -- 172.21.15.28:0/1446101492 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f7ea808d2d0 con 0x7f7eb80fe300 2024-07-03T15:51:20.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.401+0000 7f7ebd396640 1 --2- 172.21.15.28:0/1446101492 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f7ea406e0b0 0x7f7ea4070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:51:20.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.402+0000 7f7ebd396640 1 --2- 172.21.15.28:0/1446101492 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f7ea406e0b0 0x7f7ea4070560 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7eb40043d0 tx=0x7f7eb4035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:51:20.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.406+0000 7f7eadffb640 1 -- 172.21.15.28:0/1446101492 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f7ea80566b0 con 0x7f7eb80fe300 2024-07-03T15:51:20.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:20 smithi028 ceph-mon[92858]: from='client.44307 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:51:20.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:20 smithi028 ceph-mon[92858]: from='client.44311 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:51:20.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:20 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/3461107257' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:20.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:20 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:20.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:20 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.szomfl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-03T15:51:20.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:20 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:51:20.623 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:20 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/1213114309' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-03T15:51:20.642 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:20 smithi148 ceph-mon[80490]: from='client.44307 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:51:20.643 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:20 smithi148 ceph-mon[80490]: from='client.44311 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:51:20.643 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:20 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/3461107257' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:20.643 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:20 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:20.643 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:20 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.szomfl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-03T15:51:20.643 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:20 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:51:20.643 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:20 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/1213114309' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-03T15:51:20.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.661+0000 7f7ebf5fa640 1 -- 172.21.15.28:0/1446101492 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7eb80f9e50 con 0x7f7eb80fe300 2024-07-03T15:51:20.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.661+0000 7f7eadffb640 1 -- 172.21.15.28:0/1446101492 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f7ea8059d70 con 0x7f7eb80fe300 2024-07-03T15:51:20.665 INFO:teuthology.orchestra.run.smithi028.stdout:HEALTH_OK 2024-07-03T15:51:20.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.663+0000 7f7ebf5fa640 1 -- 172.21.15.28:0/1446101492 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f7ea406e0b0 msgr2=0x7f7ea4070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:20.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.663+0000 7f7ebf5fa640 1 --2- 172.21.15.28:0/1446101492 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f7ea406e0b0 0x7f7ea4070560 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7eb40043d0 tx=0x7f7eb4035040 comp rx=0 tx=0).stop 2024-07-03T15:51:20.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.663+0000 7f7ebf5fa640 1 -- 172.21.15.28:0/1446101492 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7eb80fe300 msgr2=0x7f7eb80fb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:20.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.663+0000 7f7ebf5fa640 1 --2- 172.21.15.28:0/1446101492 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7eb80fe300 0x7f7eb80fb690 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8002870 tx=0x7f7ea8004600 comp rx=0 tx=0).stop 2024-07-03T15:51:20.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.663+0000 7f7ebf5fa640 1 -- 172.21.15.28:0/1446101492 shutdown_connections 2024-07-03T15:51:20.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.663+0000 7f7ebf5fa640 1 --2- 172.21.15.28:0/1446101492 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f7ea406e0b0 0x7f7ea4070560 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:20.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.663+0000 7f7ebf5fa640 1 --2- 172.21.15.28:0/1446101492 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7eb80fe300 0x7f7eb80fb690 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:20.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.663+0000 7f7ebf5fa640 1 --2- 172.21.15.28:0/1446101492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7eb80fd9b0 0x7f7eb80fd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:20.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.663+0000 7f7ebf5fa640 1 -- 172.21.15.28:0/1446101492 >> 172.21.15.28:0/1446101492 conn(0x7f7eb80f61b0 msgr2=0x7f7eb80f7ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:51:20.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.663+0000 7f7ebf5fa640 1 -- 172.21.15.28:0/1446101492 shutdown_connections 2024-07-03T15:51:20.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:20.663+0000 7f7ebf5fa640 1 -- 172.21.15.28:0/1446101492 wait complete. 2024-07-03T15:51:21.521 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:21 smithi148 ceph-mon[80490]: Upgrade: Updating mds.cephfs.smithi148.szomfl 2024-07-03T15:51:21.521 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:21 smithi148 ceph-mon[80490]: Deploying daemon mds.cephfs.smithi148.szomfl on smithi148 2024-07-03T15:51:21.521 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:21 smithi148 ceph-mon[80490]: pgmap v185: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail 2024-07-03T15:51:21.521 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:21 smithi148 ceph-mon[80490]: from='client.34436 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:51:21.521 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:21 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/1446101492' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-03T15:51:21.784 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:21 smithi028 ceph-mon[92858]: Upgrade: Updating mds.cephfs.smithi148.szomfl 2024-07-03T15:51:21.784 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:21 smithi028 ceph-mon[92858]: Deploying daemon mds.cephfs.smithi148.szomfl on smithi148 2024-07-03T15:51:21.784 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:21 smithi028 ceph-mon[92858]: pgmap v185: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail 2024-07-03T15:51:21.784 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:21 smithi028 ceph-mon[92858]: from='client.34436 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:51:21.784 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:21 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/1446101492' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-03T15:51:22.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:21 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-mon-smithi028[92854]: 2024-07-03T15:51:21.780+0000 7f3b8b97a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-03T15:51:22.232 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:51:21 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3[89973]: 2024-07-03T15:51:21.801+0000 7f623466e640 -1 bdev(0x55eb7e312e00 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 2 2024-07-03T15:51:22.232 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:51:21 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3[89973]: 2024-07-03T15:51:21.801+0000 7f6239678640 -1 bdev(0x55eb7e312e00 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 5 2024-07-03T15:51:22.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:22 smithi148 ceph-mon[80490]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-07-03T15:51:22.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:22 smithi148 ceph-mon[80490]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-03T15:51:22.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:22 smithi148 ceph-mon[80490]: osdmap e82: 6 total, 6 up, 6 in 2024-07-03T15:51:22.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:22 smithi148 ceph-mon[80490]: Standby daemon mds.cephfs.smithi028.rhpvve assigned to filesystem cephfs as rank 0 2024-07-03T15:51:22.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:22 smithi148 ceph-mon[80490]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-07-03T15:51:22.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:22 smithi148 ceph-mon[80490]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-07-03T15:51:22.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:22 smithi148 ceph-mon[80490]: fsmap cephfs:1/1 {0=cephfs.smithi028.rhpvve=up:replay} 2 up:standby 2024-07-03T15:51:22.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:22 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:22.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:22 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:51:22.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:22 smithi028 ceph-mon[92858]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-07-03T15:51:22.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:22 smithi028 ceph-mon[92858]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-03T15:51:22.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:22 smithi028 ceph-mon[92858]: osdmap e82: 6 total, 6 up, 6 in 2024-07-03T15:51:22.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:22 smithi028 ceph-mon[92858]: Standby daemon mds.cephfs.smithi028.rhpvve assigned to filesystem cephfs as rank 0 2024-07-03T15:51:22.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:22 smithi028 ceph-mon[92858]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-07-03T15:51:22.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:22 smithi028 ceph-mon[92858]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-07-03T15:51:22.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:22 smithi028 ceph-mon[92858]: fsmap cephfs:1/1 {0=cephfs.smithi028.rhpvve=up:replay} 2 up:standby 2024-07-03T15:51:22.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:22 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:22.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:22 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:51:23.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:23 smithi148 ceph-mon[80490]: pgmap v187: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail 2024-07-03T15:51:23.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:23 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.szomfl"}]: dispatch 2024-07-03T15:51:23.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:23 smithi028 ceph-mon[92858]: pgmap v187: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail 2024-07-03T15:51:23.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:23 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.szomfl"}]: dispatch 2024-07-03T15:51:25.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:25 smithi148 ceph-mon[80490]: pgmap v188: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 6.8 MiB/s rd, 2 op/s 2024-07-03T15:51:25.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:25 smithi028 ceph-mon[92858]: pgmap v188: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 6.8 MiB/s rd, 2 op/s 2024-07-03T15:51:27.717 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:27 smithi148 ceph-mon[80490]: pgmap v189: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 7.2 MiB/s rd, 2 op/s 2024-07-03T15:51:27.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:27 smithi028 ceph-mon[92858]: pgmap v189: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 7.2 MiB/s rd, 2 op/s 2024-07-03T15:51:29.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:29 smithi148 ceph-mon[80490]: pgmap v190: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 10 MiB/s rd, 3 op/s 2024-07-03T15:51:29.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:29 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:29.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:29 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:29.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:29 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:51:29.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:29 smithi028 ceph-mon[92858]: pgmap v190: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 10 MiB/s rd, 3 op/s 2024-07-03T15:51:29.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:29 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:29.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:29 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:29.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:29 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:51:30.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:30 smithi148 ceph-mon[80490]: reconnect by client.24301 v1:172.21.15.28:0/1387201540 after 0 2024-07-03T15:51:30.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:30 smithi148 ceph-mon[80490]: reconnect by client.14514 v1:172.21.15.148:0/4158297337 after 0 2024-07-03T15:51:30.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:30 smithi148 ceph-mon[80490]: mds.? [v2:172.21.15.28:6826/3127260162,v1:172.21.15.28:6827/3127260162] up:reconnect 2024-07-03T15:51:30.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:30 smithi148 ceph-mon[80490]: mds.? [v2:172.21.15.148:6826/3922163302,v1:172.21.15.148:6827/3922163302] up:boot 2024-07-03T15:51:30.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:30 smithi148 ceph-mon[80490]: fsmap cephfs:1/1 {0=cephfs.smithi028.rhpvve=up:reconnect} 3 up:standby 2024-07-03T15:51:30.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:30 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.szomfl"}]: dispatch 2024-07-03T15:51:30.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:30 smithi028 ceph-mon[92858]: reconnect by client.24301 v1:172.21.15.28:0/1387201540 after 0 2024-07-03T15:51:30.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:30 smithi028 ceph-mon[92858]: reconnect by client.14514 v1:172.21.15.148:0/4158297337 after 0 2024-07-03T15:51:30.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:30 smithi028 ceph-mon[92858]: mds.? [v2:172.21.15.28:6826/3127260162,v1:172.21.15.28:6827/3127260162] up:reconnect 2024-07-03T15:51:30.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:30 smithi028 ceph-mon[92858]: mds.? [v2:172.21.15.148:6826/3922163302,v1:172.21.15.148:6827/3922163302] up:boot 2024-07-03T15:51:30.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:30 smithi028 ceph-mon[92858]: fsmap cephfs:1/1 {0=cephfs.smithi028.rhpvve=up:reconnect} 3 up:standby 2024-07-03T15:51:30.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:30 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.szomfl"}]: dispatch 2024-07-03T15:51:31.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:31 smithi148 ceph-mon[80490]: pgmap v191: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-07-03T15:51:31.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:31 smithi148 ceph-mon[80490]: mds.? [v2:172.21.15.28:6826/3127260162,v1:172.21.15.28:6827/3127260162] up:rejoin 2024-07-03T15:51:31.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:31 smithi148 ceph-mon[80490]: fsmap cephfs:1/1 {0=cephfs.smithi028.rhpvve=up:rejoin} 3 up:standby 2024-07-03T15:51:31.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:31 smithi148 ceph-mon[80490]: daemon mds.cephfs.smithi028.rhpvve is now active in filesystem cephfs as rank 0 2024-07-03T15:51:31.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:31 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:31.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:31 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:31.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:31 smithi028 ceph-mon[92858]: pgmap v191: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-07-03T15:51:31.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:31 smithi028 ceph-mon[92858]: mds.? [v2:172.21.15.28:6826/3127260162,v1:172.21.15.28:6827/3127260162] up:rejoin 2024-07-03T15:51:31.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:31 smithi028 ceph-mon[92858]: fsmap cephfs:1/1 {0=cephfs.smithi028.rhpvve=up:rejoin} 3 up:standby 2024-07-03T15:51:31.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:31 smithi028 ceph-mon[92858]: daemon mds.cephfs.smithi028.rhpvve is now active in filesystem cephfs as rank 0 2024-07-03T15:51:31.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:31 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:31.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:31 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:32.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:32 smithi148 ceph-mon[80490]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-07-03T15:51:32.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:32 smithi148 ceph-mon[80490]: Cluster is now healthy 2024-07-03T15:51:32.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:32 smithi148 ceph-mon[80490]: mds.? [v2:172.21.15.28:6826/3127260162,v1:172.21.15.28:6827/3127260162] up:active 2024-07-03T15:51:32.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:32 smithi148 ceph-mon[80490]: fsmap cephfs:1 {0=cephfs.smithi028.rhpvve=up:active} 3 up:standby 2024-07-03T15:51:32.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:32 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:32.733 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:32 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:32.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:32 smithi028 ceph-mon[92858]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-07-03T15:51:32.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:32 smithi028 ceph-mon[92858]: Cluster is now healthy 2024-07-03T15:51:32.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:32 smithi028 ceph-mon[92858]: mds.? [v2:172.21.15.28:6826/3127260162,v1:172.21.15.28:6827/3127260162] up:active 2024-07-03T15:51:32.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:32 smithi028 ceph-mon[92858]: fsmap cephfs:1 {0=cephfs.smithi028.rhpvve=up:active} 3 up:standby 2024-07-03T15:51:32.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:32 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:32.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:32 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:33.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:33 smithi148 ceph-mon[80490]: pgmap v192: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 98 B/s wr, 5 op/s 2024-07-03T15:51:34.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:33 smithi028 ceph-mon[92858]: pgmap v192: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 98 B/s wr, 5 op/s 2024-07-03T15:51:35.732 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:51:35 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3[89973]: 2024-07-03T15:51:35.446+0000 7f623466e640 -1 bdev(0x55eb7e312e00 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 6 2024-07-03T15:51:35.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:35 smithi028 ceph-mon[92858]: pgmap v193: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 170 B/s wr, 7 op/s 2024-07-03T15:51:35.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:35 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:35.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:35 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:35.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:35 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:51:35.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:35 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:51:35.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:35 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:35.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:35 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:51:35.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:35 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:35.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:35 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:35.873 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:35 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:35.874 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:35 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:35.874 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:35 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:35.874 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:35 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:35.874 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:35 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi028.rhpvve"}]: dispatch 2024-07-03T15:51:35.874 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:35 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi028.rhpvve"}]': finished 2024-07-03T15:51:35.874 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:35 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi028.wmbxew"}]: dispatch 2024-07-03T15:51:35.874 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:35 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi028.wmbxew"}]': finished 2024-07-03T15:51:35.874 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:35 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi148.szomfl"}]: dispatch 2024-07-03T15:51:35.874 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:35 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi148.szomfl"}]': finished 2024-07-03T15:51:35.874 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:35 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi148.xgpuld"}]: dispatch 2024-07-03T15:51:35.874 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:35 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi148.xgpuld"}]': finished 2024-07-03T15:51:35.874 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:35 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-07-03T15:51:35.875 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:35 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-07-03T15:51:35.875 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:35 smithi028 ceph-mon[92858]: fsmap cephfs:1 {0=cephfs.smithi028.rhpvve=up:active} 1 up:standby-replay 2 up:standby 2024-07-03T15:51:35.875 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:35 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:35.875 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:35 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:35.875 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:35 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:35.875 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:35 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:35.875 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:35 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:35.875 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:35 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:35.875 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:35 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:35.875 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:35 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:35.875 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:35 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:35.875 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:35 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:35.875 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:35 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:35.876 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:35 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:35.876 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:35 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:35.876 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:35 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:36.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:35 smithi148 ceph-mon[80490]: pgmap v193: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 170 B/s wr, 7 op/s 2024-07-03T15:51:36.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:35 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:36.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:35 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:36.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:35 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:51:36.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:35 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:51:36.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:35 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:36.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:35 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:51:36.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:35 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:36.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:35 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:36.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:35 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:36.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:35 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:36.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:35 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:36.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:35 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:36.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:35 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi028.rhpvve"}]: dispatch 2024-07-03T15:51:36.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:35 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi028.rhpvve"}]': finished 2024-07-03T15:51:36.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:35 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi028.wmbxew"}]: dispatch 2024-07-03T15:51:36.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:35 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi028.wmbxew"}]': finished 2024-07-03T15:51:36.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:35 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi148.szomfl"}]: dispatch 2024-07-03T15:51:36.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:35 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi148.szomfl"}]': finished 2024-07-03T15:51:36.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:35 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi148.xgpuld"}]: dispatch 2024-07-03T15:51:36.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:35 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi148.xgpuld"}]': finished 2024-07-03T15:51:36.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:35 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-07-03T15:51:36.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:35 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-07-03T15:51:36.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:35 smithi148 ceph-mon[80490]: fsmap cephfs:1 {0=cephfs.smithi028.rhpvve=up:active} 1 up:standby-replay 2 up:standby 2024-07-03T15:51:36.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:35 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:36.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:35 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:36.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:35 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:36.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:35 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:36.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:35 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:36.235 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:35 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:36.235 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:35 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:36.235 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:35 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:36.235 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:35 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:36.235 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:35 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:36.235 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:35 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:36.235 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:35 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:36.235 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:35 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:36.235 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:35 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:37.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:36 smithi028 ceph-mon[92858]: Upgrade: Setting container_image for all mds 2024-07-03T15:51:37.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:36 smithi028 ceph-mon[92858]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-07-03T15:51:37.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:36 smithi028 ceph-mon[92858]: Upgrade: Setting container_image for all rgw 2024-07-03T15:51:37.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:36 smithi028 ceph-mon[92858]: Upgrade: Setting container_image for all rbd-mirror 2024-07-03T15:51:37.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:36 smithi028 ceph-mon[92858]: Upgrade: Setting container_image for all ceph-exporter 2024-07-03T15:51:37.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:36 smithi028 ceph-mon[92858]: Upgrade: Setting container_image for all iscsi 2024-07-03T15:51:37.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:36 smithi028 ceph-mon[92858]: Upgrade: Setting container_image for all nfs 2024-07-03T15:51:37.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:36 smithi028 ceph-mon[92858]: Upgrade: Setting container_image for all nvmeof 2024-07-03T15:51:37.135 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:36 smithi148 ceph-mon[80490]: Upgrade: Setting container_image for all mds 2024-07-03T15:51:37.135 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:36 smithi148 ceph-mon[80490]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-07-03T15:51:37.135 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:36 smithi148 ceph-mon[80490]: Upgrade: Setting container_image for all rgw 2024-07-03T15:51:37.135 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:36 smithi148 ceph-mon[80490]: Upgrade: Setting container_image for all rbd-mirror 2024-07-03T15:51:37.135 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:36 smithi148 ceph-mon[80490]: Upgrade: Setting container_image for all ceph-exporter 2024-07-03T15:51:37.136 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:36 smithi148 ceph-mon[80490]: Upgrade: Setting container_image for all iscsi 2024-07-03T15:51:37.136 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:36 smithi148 ceph-mon[80490]: Upgrade: Setting container_image for all nfs 2024-07-03T15:51:37.136 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:36 smithi148 ceph-mon[80490]: Upgrade: Setting container_image for all nvmeof 2024-07-03T15:51:38.049 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:37 smithi028 ceph-mon[92858]: pgmap v194: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 10 MiB/s rd, 170 B/s wr, 6 op/s 2024-07-03T15:51:38.050 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:37 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:38.050 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:37 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:51:38.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:37 smithi148 ceph-mon[80490]: pgmap v194: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 10 MiB/s rd, 170 B/s wr, 6 op/s 2024-07-03T15:51:38.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:37 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:38.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:37 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:51:39.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:38 smithi028 ceph-mon[92858]: Upgrade: Updating node-exporter.smithi028 (1/2) 2024-07-03T15:51:39.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:38 smithi028 ceph-mon[92858]: Deploying daemon node-exporter.smithi028 on smithi028 2024-07-03T15:51:39.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:38 smithi148 ceph-mon[80490]: Upgrade: Updating node-exporter.smithi028 (1/2) 2024-07-03T15:51:39.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:38 smithi148 ceph-mon[80490]: Deploying daemon node-exporter.smithi028 on smithi028 2024-07-03T15:51:40.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:39 smithi028 ceph-mon[92858]: pgmap v195: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 4.4 KiB/s wr, 8 op/s 2024-07-03T15:51:40.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:39 smithi148 ceph-mon[80490]: pgmap v195: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 4.4 KiB/s wr, 8 op/s 2024-07-03T15:51:42.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:41 smithi028 ceph-mon[92858]: pgmap v196: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 4.4 KiB/s wr, 9 op/s 2024-07-03T15:51:42.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:41 smithi148 ceph-mon[80490]: pgmap v196: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 4.4 KiB/s wr, 9 op/s 2024-07-03T15:51:43.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:43 smithi028 ceph-mon[92858]: pgmap v197: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 4.4 KiB/s wr, 8 op/s 2024-07-03T15:51:43.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:43 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:43.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:43 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:43.916 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:43 smithi148 ceph-mon[80490]: pgmap v197: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 4.4 KiB/s wr, 8 op/s 2024-07-03T15:51:43.916 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:43 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:43.916 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:43 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:45.572 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:45 smithi148 ceph-mon[80490]: Upgrade: Updating node-exporter.smithi148 (2/2) 2024-07-03T15:51:45.572 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:45 smithi148 ceph-mon[80490]: Deploying daemon node-exporter.smithi148 on smithi148 2024-07-03T15:51:45.572 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:45 smithi148 ceph-mon[80490]: pgmap v198: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 4.3 KiB/s wr, 9 op/s 2024-07-03T15:51:45.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:45 smithi028 ceph-mon[92858]: Upgrade: Updating node-exporter.smithi148 (2/2) 2024-07-03T15:51:45.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:45 smithi028 ceph-mon[92858]: Deploying daemon node-exporter.smithi148 on smithi148 2024-07-03T15:51:45.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:45 smithi028 ceph-mon[92858]: pgmap v198: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 4.3 KiB/s wr, 9 op/s 2024-07-03T15:51:47.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:47 smithi028 ceph-mon[92858]: pgmap v199: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-07-03T15:51:47.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:47 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:47.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:47 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:47.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:47 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:51:47.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:47 smithi148 ceph-mon[80490]: pgmap v199: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-07-03T15:51:47.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:47 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:47.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:47 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:47.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:47 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:51:49.689 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:49 smithi148 ceph-mon[80490]: pgmap v200: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-07-03T15:51:49.689 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:49.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:49.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:49.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:49.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:49 smithi028 ceph-mon[92858]: pgmap v200: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-07-03T15:51:49.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:49.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:49.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:49.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:50.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.759+0000 7ffa927a0640 1 -- 172.21.15.28:0/3575669072 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa8c0fc370 msgr2=0x7ffa8c0fc780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:50.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.759+0000 7ffa927a0640 1 --2- 172.21.15.28:0/3575669072 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa8c0fc370 0x7ffa8c0fc780 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c0048d0 tx=0x7ffa7c009da0 comp rx=0 tx=0).stop 2024-07-03T15:51:50.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.759+0000 7ffa927a0640 1 -- 172.21.15.28:0/3575669072 shutdown_connections 2024-07-03T15:51:50.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.759+0000 7ffa927a0640 1 --2- 172.21.15.28:0/3575669072 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa8c0fd640 0x7ffa8c0fdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:50.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.759+0000 7ffa927a0640 1 --2- 172.21.15.28:0/3575669072 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa8c0fc370 0x7ffa8c0fc780 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:50.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.759+0000 7ffa927a0640 1 -- 172.21.15.28:0/3575669072 >> 172.21.15.28:0/3575669072 conn(0x7ffa8c0f7b50 msgr2=0x7ffa8c0f9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:51:50.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.759+0000 7ffa927a0640 1 -- 172.21.15.28:0/3575669072 shutdown_connections 2024-07-03T15:51:50.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.759+0000 7ffa927a0640 1 -- 172.21.15.28:0/3575669072 wait complete. 2024-07-03T15:51:50.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.760+0000 7ffa927a0640 1 Processor -- start 2024-07-03T15:51:50.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.760+0000 7ffa927a0640 1 -- start start 2024-07-03T15:51:50.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.760+0000 7ffa927a0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa8c0fc370 0x7ffa8c071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:51:50.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.760+0000 7ffa927a0640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa8c0fd640 0x7ffa8c06f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:51:50.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.760+0000 7ffa927a0640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8c06fe00 con 0x7ffa8c0fd640 2024-07-03T15:51:50.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.760+0000 7ffa927a0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8c06ff40 con 0x7ffa8c0fc370 2024-07-03T15:51:50.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.760+0000 7ffa8b7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa8c0fd640 0x7ffa8c06f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:51:50.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.760+0000 7ffa8b7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa8c0fd640 0x7ffa8c06f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:40030/0 (socket says 172.21.15.28:40030) 2024-07-03T15:51:50.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.760+0000 7ffa8b7fe640 1 -- 172.21.15.28:0/2028426077 learned_addr learned my addr 172.21.15.28:0/2028426077 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:51:50.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.760+0000 7ffa8bfff640 1 --2- 172.21.15.28:0/2028426077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa8c0fc370 0x7ffa8c071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:51:50.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.761+0000 7ffa8b7fe640 1 -- 172.21.15.28:0/2028426077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa8c0fc370 msgr2=0x7ffa8c071380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:50.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.761+0000 7ffa8b7fe640 1 --2- 172.21.15.28:0/2028426077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa8c0fc370 0x7ffa8c071380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:50.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.761+0000 7ffa8b7fe640 1 -- 172.21.15.28:0/2028426077 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa80005040 con 0x7ffa8c0fd640 2024-07-03T15:51:50.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.761+0000 7ffa8b7fe640 1 --2- 172.21.15.28:0/2028426077 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa8c0fd640 0x7ffa8c06f870 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ffa80009700 tx=0x7ffa80009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:51:50.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.761+0000 7ffa88ff9640 1 -- 172.21.15.28:0/2028426077 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa8000e8b0 con 0x7ffa8c0fd640 2024-07-03T15:51:50.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.761+0000 7ffa927a0640 1 -- 172.21.15.28:0/2028426077 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa7c005040 con 0x7ffa8c0fd640 2024-07-03T15:51:50.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.761+0000 7ffa88ff9640 1 -- 172.21.15.28:0/2028426077 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffa8000f040 con 0x7ffa8c0fd640 2024-07-03T15:51:50.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.761+0000 7ffa88ff9640 1 -- 172.21.15.28:0/2028426077 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa80013e50 con 0x7ffa8c0fd640 2024-07-03T15:51:50.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.761+0000 7ffa927a0640 1 -- 172.21.15.28:0/2028426077 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa8c0705a0 con 0x7ffa8c0fd640 2024-07-03T15:51:50.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.762+0000 7ffa927a0640 1 -- 172.21.15.28:0/2028426077 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa8c1056d0 con 0x7ffa8c0fd640 2024-07-03T15:51:50.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.763+0000 7ffa88ff9640 1 -- 172.21.15.28:0/2028426077 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ffa8000ea10 con 0x7ffa8c0fd640 2024-07-03T15:51:50.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.763+0000 7ffa88ff9640 1 --2- 172.21.15.28:0/2028426077 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7ffa7406e0b0 0x7ffa74070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:51:50.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.764+0000 7ffa88ff9640 1 -- 172.21.15.28:0/2028426077 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7ffa8008dc50 con 0x7ffa8c0fd640 2024-07-03T15:51:50.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.764+0000 7ffa8bfff640 1 --2- 172.21.15.28:0/2028426077 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7ffa7406e0b0 0x7ffa74070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:51:50.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.764+0000 7ffa8bfff640 1 --2- 172.21.15.28:0/2028426077 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7ffa7406e0b0 0x7ffa74070560 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c009f80 tx=0x7ffa7c035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:51:50.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.768+0000 7ffa88ff9640 1 -- 172.21.15.28:0/2028426077 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ffa80056e80 con 0x7ffa8c0fd640 2024-07-03T15:51:50.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.971+0000 7ffa927a0640 1 -- 172.21.15.28:0/2028426077 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ffa8c101960 con 0x7ffa7406e0b0 2024-07-03T15:51:50.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.972+0000 7ffa88ff9640 1 -- 172.21.15.28:0/2028426077 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7ffa8c101960 con 0x7ffa7406e0b0 2024-07-03T15:51:50.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.974+0000 7ffa927a0640 1 -- 172.21.15.28:0/2028426077 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7ffa7406e0b0 msgr2=0x7ffa74070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:50.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.974+0000 7ffa927a0640 1 --2- 172.21.15.28:0/2028426077 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7ffa7406e0b0 0x7ffa74070560 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c009f80 tx=0x7ffa7c035040 comp rx=0 tx=0).stop 2024-07-03T15:51:50.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.974+0000 7ffa927a0640 1 -- 172.21.15.28:0/2028426077 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa8c0fd640 msgr2=0x7ffa8c06f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:50.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.974+0000 7ffa927a0640 1 --2- 172.21.15.28:0/2028426077 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa8c0fd640 0x7ffa8c06f870 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ffa80009700 tx=0x7ffa80009da0 comp rx=0 tx=0).stop 2024-07-03T15:51:50.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.974+0000 7ffa927a0640 1 -- 172.21.15.28:0/2028426077 shutdown_connections 2024-07-03T15:51:50.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.974+0000 7ffa927a0640 1 --2- 172.21.15.28:0/2028426077 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7ffa7406e0b0 0x7ffa74070560 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:50.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.974+0000 7ffa927a0640 1 --2- 172.21.15.28:0/2028426077 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa8c0fd640 0x7ffa8c06f870 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:50.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.974+0000 7ffa927a0640 1 --2- 172.21.15.28:0/2028426077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa8c0fc370 0x7ffa8c071380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:50.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.974+0000 7ffa927a0640 1 -- 172.21.15.28:0/2028426077 >> 172.21.15.28:0/2028426077 conn(0x7ffa8c0f7b50 msgr2=0x7ffa8c0f91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:51:50.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.974+0000 7ffa927a0640 1 -- 172.21.15.28:0/2028426077 shutdown_connections 2024-07-03T15:51:50.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:50.974+0000 7ffa927a0640 1 -- 172.21.15.28:0/2028426077 wait complete. 2024-07-03T15:51:50.986 INFO:teuthology.orchestra.run.smithi028.stdout:true 2024-07-03T15:51:51.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.057+0000 7f19d8e4e640 1 -- 172.21.15.28:0/1040850538 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19d40fb280 msgr2=0x7f19d40fb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:51.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.057+0000 7f19d8e4e640 1 --2- 172.21.15.28:0/1040850538 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19d40fb280 0x7f19d40fb690 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f19c40048d0 tx=0x7f19c4009da0 comp rx=0 tx=0).stop 2024-07-03T15:51:51.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.057+0000 7f19d8e4e640 1 -- 172.21.15.28:0/1040850538 shutdown_connections 2024-07-03T15:51:51.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.057+0000 7f19d8e4e640 1 --2- 172.21.15.28:0/1040850538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19d40fc550 0x7f19d40fc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:51.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.057+0000 7f19d8e4e640 1 --2- 172.21.15.28:0/1040850538 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19d40fb280 0x7f19d40fb690 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:51.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.057+0000 7f19d8e4e640 1 -- 172.21.15.28:0/1040850538 >> 172.21.15.28:0/1040850538 conn(0x7f19d40f6a20 msgr2=0x7f19d40f8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:51:51.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.057+0000 7f19d8e4e640 1 -- 172.21.15.28:0/1040850538 shutdown_connections 2024-07-03T15:51:51.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.057+0000 7f19d8e4e640 1 -- 172.21.15.28:0/1040850538 wait complete. 2024-07-03T15:51:51.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.057+0000 7f19d8e4e640 1 Processor -- start 2024-07-03T15:51:51.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.057+0000 7f19d8e4e640 1 -- start start 2024-07-03T15:51:51.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.058+0000 7f19d8e4e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19d40fb280 0x7f19d410f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:51:51.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.058+0000 7f19d8e4e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19d40fc550 0x7f19d410f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:51:51.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.058+0000 7f19d8e4e640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d410fd70 con 0x7f19d40fb280 2024-07-03T15:51:51.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.058+0000 7f19d8e4e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d4110540 con 0x7f19d40fc550 2024-07-03T15:51:51.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.058+0000 7f19d259c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19d40fb280 0x7f19d410f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:51:51.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.058+0000 7f19d259c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19d40fb280 0x7f19d410f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:40050/0 (socket says 172.21.15.28:40050) 2024-07-03T15:51:51.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.058+0000 7f19d259c640 1 -- 172.21.15.28:0/1216142447 learned_addr learned my addr 172.21.15.28:0/1216142447 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:51:51.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.058+0000 7f19c9d9b640 1 --2- 172.21.15.28:0/1216142447 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19d40fc550 0x7f19d410f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:51:51.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.058+0000 7f19d259c640 1 -- 172.21.15.28:0/1216142447 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19d40fc550 msgr2=0x7f19d410f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:51.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.058+0000 7f19d259c640 1 --2- 172.21.15.28:0/1216142447 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19d40fc550 0x7f19d410f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:51.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.058+0000 7f19d259c640 1 -- 172.21.15.28:0/1216142447 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c4005040 con 0x7f19d40fb280 2024-07-03T15:51:51.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.058+0000 7f19d259c640 1 --2- 172.21.15.28:0/1216142447 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19d40fb280 0x7f19d410f1e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f19c40048a0 tx=0x7f19c4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:51:51.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.059+0000 7f19cb7fe640 1 -- 172.21.15.28:0/1216142447 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19c4038070 con 0x7f19d40fb280 2024-07-03T15:51:51.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.059+0000 7f19d8e4e640 1 -- 172.21.15.28:0/1216142447 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19d4110770 con 0x7f19d40fb280 2024-07-03T15:51:51.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.059+0000 7f19cb7fe640 1 -- 172.21.15.28:0/1216142447 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f19c40092b0 con 0x7f19d40fb280 2024-07-03T15:51:51.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.059+0000 7f19cb7fe640 1 -- 172.21.15.28:0/1216142447 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19c40338a0 con 0x7f19d40fb280 2024-07-03T15:51:51.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.059+0000 7f19d8e4e640 1 -- 172.21.15.28:0/1216142447 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19d4110cb0 con 0x7f19d40fb280 2024-07-03T15:51:51.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.060+0000 7f19d8e4e640 1 -- 172.21.15.28:0/1216142447 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19d40fa6e0 con 0x7f19d40fb280 2024-07-03T15:51:51.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.060+0000 7f19cb7fe640 1 -- 172.21.15.28:0/1216142447 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f19c4030020 con 0x7f19d40fb280 2024-07-03T15:51:51.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.061+0000 7f19cb7fe640 1 --2- 172.21.15.28:0/1216142447 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f19c006e0b0 0x7f19c0070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:51:51.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.061+0000 7f19cb7fe640 1 -- 172.21.15.28:0/1216142447 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f19c40ad9c0 con 0x7f19d40fb280 2024-07-03T15:51:51.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.061+0000 7f19c9d9b640 1 --2- 172.21.15.28:0/1216142447 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f19c006e0b0 0x7f19c0070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:51:51.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.061+0000 7f19c9d9b640 1 --2- 172.21.15.28:0/1216142447 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f19c006e0b0 0x7f19c0070560 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f19d410ef90 tx=0x7f19bc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:51:51.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.065+0000 7f19cb7fe640 1 -- 172.21.15.28:0/1216142447 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f19c4076d40 con 0x7f19d40fb280 2024-07-03T15:51:51.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.267+0000 7f19d8e4e640 1 -- 172.21.15.28:0/1216142447 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f19d4103f50 con 0x7f19c006e0b0 2024-07-03T15:51:51.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.268+0000 7f19cb7fe640 1 -- 172.21.15.28:0/1216142447 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f19d4103f50 con 0x7f19c006e0b0 2024-07-03T15:51:51.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.270+0000 7f19d8e4e640 1 -- 172.21.15.28:0/1216142447 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f19c006e0b0 msgr2=0x7f19c0070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:51.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.270+0000 7f19d8e4e640 1 --2- 172.21.15.28:0/1216142447 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f19c006e0b0 0x7f19c0070560 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f19d410ef90 tx=0x7f19bc00b040 comp rx=0 tx=0).stop 2024-07-03T15:51:51.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.270+0000 7f19d8e4e640 1 -- 172.21.15.28:0/1216142447 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19d40fb280 msgr2=0x7f19d410f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:51.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.270+0000 7f19d8e4e640 1 --2- 172.21.15.28:0/1216142447 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19d40fb280 0x7f19d410f1e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f19c40048a0 tx=0x7f19c4009da0 comp rx=0 tx=0).stop 2024-07-03T15:51:51.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.270+0000 7f19d8e4e640 1 -- 172.21.15.28:0/1216142447 shutdown_connections 2024-07-03T15:51:51.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.270+0000 7f19d8e4e640 1 --2- 172.21.15.28:0/1216142447 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f19c006e0b0 0x7f19c0070560 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:51.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.270+0000 7f19d8e4e640 1 --2- 172.21.15.28:0/1216142447 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19d40fc550 0x7f19d410f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:51.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.270+0000 7f19d8e4e640 1 --2- 172.21.15.28:0/1216142447 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19d40fb280 0x7f19d410f1e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:51.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.270+0000 7f19d8e4e640 1 -- 172.21.15.28:0/1216142447 >> 172.21.15.28:0/1216142447 conn(0x7f19d40f6a20 msgr2=0x7f19d4065db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:51:51.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.270+0000 7f19d8e4e640 1 -- 172.21.15.28:0/1216142447 shutdown_connections 2024-07-03T15:51:51.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.270+0000 7f19d8e4e640 1 -- 172.21.15.28:0/1216142447 wait complete. 2024-07-03T15:51:51.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.352+0000 7f2eca12d640 1 -- 172.21.15.28:0/3718861780 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ec40fd600 msgr2=0x7f2ec40fda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:51.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.352+0000 7f2eca12d640 1 --2- 172.21.15.28:0/3718861780 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ec40fd600 0x7f2ec40fda70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2eac0048d0 tx=0x7f2eac009da0 comp rx=0 tx=0).stop 2024-07-03T15:51:51.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.352+0000 7f2eca12d640 1 -- 172.21.15.28:0/3718861780 shutdown_connections 2024-07-03T15:51:51.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.352+0000 7f2eca12d640 1 --2- 172.21.15.28:0/3718861780 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ec40fd600 0x7f2ec40fda70 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:51.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.352+0000 7f2eca12d640 1 --2- 172.21.15.28:0/3718861780 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ec40fc330 0x7f2ec40fc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:51.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.352+0000 7f2eca12d640 1 -- 172.21.15.28:0/3718861780 >> 172.21.15.28:0/3718861780 conn(0x7f2ec40f7af0 msgr2=0x7f2ec40f9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:51:51.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.352+0000 7f2eca12d640 1 -- 172.21.15.28:0/3718861780 shutdown_connections 2024-07-03T15:51:51.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.352+0000 7f2eca12d640 1 -- 172.21.15.28:0/3718861780 wait complete. 2024-07-03T15:51:51.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.352+0000 7f2eca12d640 1 Processor -- start 2024-07-03T15:51:51.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.353+0000 7f2eca12d640 1 -- start start 2024-07-03T15:51:51.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.353+0000 7f2eca12d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ec40fc330 0x7f2ec410cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:51:51.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.353+0000 7f2eca12d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ec40fd600 0x7f2ec410d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:51:51.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.353+0000 7f2eca12d640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec410dae0 con 0x7f2ec40fd600 2024-07-03T15:51:51.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.353+0000 7f2eca12d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec410e300 con 0x7f2ec40fc330 2024-07-03T15:51:51.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.353+0000 7f2ebbfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ec40fd600 0x7f2ec410d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:51:51.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.353+0000 7f2ebbfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ec40fd600 0x7f2ec410d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:40058/0 (socket says 172.21.15.28:40058) 2024-07-03T15:51:51.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.353+0000 7f2ebbfff640 1 -- 172.21.15.28:0/3120881049 learned_addr learned my addr 172.21.15.28:0/3120881049 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:51:51.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.353+0000 7f2ec37fe640 1 --2- 172.21.15.28:0/3120881049 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ec40fc330 0x7f2ec410cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:51:51.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.353+0000 7f2ebbfff640 1 -- 172.21.15.28:0/3120881049 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ec40fc330 msgr2=0x7f2ec410cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:51.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.353+0000 7f2ebbfff640 1 --2- 172.21.15.28:0/3120881049 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ec40fc330 0x7f2ec410cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:51.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.353+0000 7f2ebbfff640 1 -- 172.21.15.28:0/3120881049 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eac005040 con 0x7f2ec40fd600 2024-07-03T15:51:51.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.353+0000 7f2ebbfff640 1 --2- 172.21.15.28:0/3120881049 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ec40fd600 0x7f2ec410d490 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2eac0048a0 tx=0x7f2eac009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:51:51.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.354+0000 7f2ec0ff9640 1 -- 172.21.15.28:0/3120881049 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2eac038070 con 0x7f2ec40fd600 2024-07-03T15:51:51.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.354+0000 7f2ec0ff9640 1 -- 172.21.15.28:0/3120881049 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2eac0092b0 con 0x7f2ec40fd600 2024-07-03T15:51:51.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.354+0000 7f2eca12d640 1 -- 172.21.15.28:0/3120881049 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ec410e530 con 0x7f2ec40fd600 2024-07-03T15:51:51.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.354+0000 7f2ec0ff9640 1 -- 172.21.15.28:0/3120881049 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2eac0338a0 con 0x7f2ec40fd600 2024-07-03T15:51:51.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.354+0000 7f2eca12d640 1 -- 172.21.15.28:0/3120881049 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ec410ea70 con 0x7f2ec40fd600 2024-07-03T15:51:51.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.355+0000 7f2eca12d640 1 -- 172.21.15.28:0/3120881049 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ec40fb790 con 0x7f2ec40fd600 2024-07-03T15:51:51.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.355+0000 7f2ec0ff9640 1 -- 172.21.15.28:0/3120881049 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2eac030020 con 0x7f2ec40fd600 2024-07-03T15:51:51.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.356+0000 7f2ec0ff9640 1 --2- 172.21.15.28:0/3120881049 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f2eb006e0b0 0x7f2eb0070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:51:51.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.356+0000 7f2ec0ff9640 1 -- 172.21.15.28:0/3120881049 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f2eac0ad9c0 con 0x7f2ec40fd600 2024-07-03T15:51:51.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.356+0000 7f2ec37fe640 1 --2- 172.21.15.28:0/3120881049 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f2eb006e0b0 0x7f2eb0070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:51:51.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.356+0000 7f2ec37fe640 1 --2- 172.21.15.28:0/3120881049 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f2eb006e0b0 0x7f2eb0070560 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4004ed0 tx=0x7f2eb400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:51:51.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.360+0000 7f2ec0ff9640 1 -- 172.21.15.28:0/3120881049 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f2eac076d40 con 0x7f2ec40fd600 2024-07-03T15:51:51.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.566+0000 7f2eca12d640 1 -- 172.21.15.28:0/3120881049 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ec410edc0 con 0x7f2eb006e0b0 2024-07-03T15:51:51.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.575+0000 7f2ec0ff9640 1 -- 172.21.15.28:0/3120881049 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f2ec410edc0 con 0x7f2eb006e0b0 2024-07-03T15:51:51.579 INFO:teuthology.orchestra.run.smithi028.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-03T15:51:51.579 INFO:teuthology.orchestra.run.smithi028.stdout:alertmanager.smithi028 smithi028 *:9093,9094 running (7m) 52s ago 15m 24.7M - 0.25.0 c8568f914cd2 1425315a0de9 2024-07-03T15:51:51.579 INFO:teuthology.orchestra.run.smithi028.stdout:crash.smithi028 smithi028 running (4m) 52s ago 15m 6895k - 19.0.0-2550-gc00506dc 9d49405603e8 c62c2f45cabe 2024-07-03T15:51:51.579 INFO:teuthology.orchestra.run.smithi028.stdout:crash.smithi148 smithi148 running (4m) 3s ago 14m 6895k - 19.0.0-2550-gc00506dc 9d49405603e8 b1318dcaacb6 2024-07-03T15:51:51.579 INFO:teuthology.orchestra.run.smithi028.stdout:grafana.smithi028 smithi028 *:3000 running (7m) 52s ago 15m 82.2M - 9.4.7 954c08fa6188 f9f17c92cfea 2024-07-03T15:51:51.580 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi028.rhpvve smithi028 running (73s) 52s ago 10m 13.1M - 19.0.0-2550-gc00506dc 9d49405603e8 5d528ea2a2f0 2024-07-03T15:51:51.580 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi028.wmbxew smithi028 running (57s) 52s ago 10m 11.4M - 19.0.0-2550-gc00506dc 9d49405603e8 ebdbfa592ac8 2024-07-03T15:51:51.580 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi148.szomfl smithi148 running (22s) 3s ago 9m 13.0M - 19.0.0-2550-gc00506dc 9d49405603e8 27650f9fb39f 2024-07-03T15:51:51.580 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi148.xgpuld smithi148 running (40s) 3s ago 10m 14.6M - 19.0.0-2550-gc00506dc 9d49405603e8 f24ab04507dc 2024-07-03T15:51:51.580 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.smithi028.vtqrtz smithi028 *:8443,9283,8765 running (6m) 52s ago 16m 503M - 19.0.0-2550-gc00506dc 9d49405603e8 14d0363c5b77 2024-07-03T15:51:51.580 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.smithi148.rjzsar smithi148 *:8443,9283,8765 running (5m) 3s ago 14m 441M - 19.0.0-2550-gc00506dc 9d49405603e8 c0193aa8a5dd 2024-07-03T15:51:51.580 INFO:teuthology.orchestra.run.smithi028.stdout:mon.smithi028 smithi028 running (5m) 52s ago 16m 52.0M 2048M 19.0.0-2550-gc00506dc 9d49405603e8 5093f7bf47d7 2024-07-03T15:51:51.580 INFO:teuthology.orchestra.run.smithi028.stdout:mon.smithi148 smithi148 running (5m) 3s ago 14m 38.2M 2048M 19.0.0-2550-gc00506dc 9d49405603e8 d338d2c648da 2024-07-03T15:51:51.580 INFO:teuthology.orchestra.run.smithi028.stdout:node-exporter.smithi028 smithi028 *:9100 starting - - - - 2024-07-03T15:51:51.580 INFO:teuthology.orchestra.run.smithi028.stdout:node-exporter.smithi148 smithi148 *:9100 running (5s) 3s ago 14m 3577k - 1.5.0 0da6a335fe13 edc710a27078 2024-07-03T15:51:51.580 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0 smithi028 running (4m) 52s ago 13m 109M 1558M 19.0.0-2550-gc00506dc 9d49405603e8 4454cecff8fc 2024-07-03T15:51:51.580 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1 smithi028 running (3m) 52s ago 13m 114M 1558M 19.0.0-2550-gc00506dc 9d49405603e8 af67f7088643 2024-07-03T15:51:51.580 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2 smithi028 running (3m) 52s ago 12m 96.8M 1558M 19.0.0-2550-gc00506dc 9d49405603e8 5d160a327dc2 2024-07-03T15:51:51.581 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3 smithi148 running (2m) 3s ago 12m 120M 2581M 19.0.0-2550-gc00506dc 9d49405603e8 1d622eba4b3b 2024-07-03T15:51:51.581 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4 smithi148 running (2m) 3s ago 12m 106M 2581M 19.0.0-2550-gc00506dc 9d49405603e8 546343d0f31f 2024-07-03T15:51:51.581 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5 smithi148 running (91s) 3s ago 11m 106M 2581M 19.0.0-2550-gc00506dc 9d49405603e8 a819fcc3b569 2024-07-03T15:51:51.581 INFO:teuthology.orchestra.run.smithi028.stdout:prometheus.smithi028 smithi028 *:9095 running (5m) 52s ago 14m 62.6M - 2.43.0 a07b618ecd1d 2b0eb2a95afd 2024-07-03T15:51:51.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.577+0000 7f2eca12d640 1 -- 172.21.15.28:0/3120881049 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f2eb006e0b0 msgr2=0x7f2eb0070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:51.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.577+0000 7f2eca12d640 1 --2- 172.21.15.28:0/3120881049 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f2eb006e0b0 0x7f2eb0070560 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4004ed0 tx=0x7f2eb400b040 comp rx=0 tx=0).stop 2024-07-03T15:51:51.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.577+0000 7f2eca12d640 1 -- 172.21.15.28:0/3120881049 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ec40fd600 msgr2=0x7f2ec410d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:51.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.577+0000 7f2eca12d640 1 --2- 172.21.15.28:0/3120881049 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ec40fd600 0x7f2ec410d490 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2eac0048a0 tx=0x7f2eac009da0 comp rx=0 tx=0).stop 2024-07-03T15:51:51.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.577+0000 7f2eca12d640 1 -- 172.21.15.28:0/3120881049 shutdown_connections 2024-07-03T15:51:51.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.577+0000 7f2eca12d640 1 --2- 172.21.15.28:0/3120881049 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f2eb006e0b0 0x7f2eb0070560 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:51.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.577+0000 7f2eca12d640 1 --2- 172.21.15.28:0/3120881049 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ec40fd600 0x7f2ec410d490 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:51.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.577+0000 7f2eca12d640 1 --2- 172.21.15.28:0/3120881049 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ec40fc330 0x7f2ec410cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:51.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.577+0000 7f2eca12d640 1 -- 172.21.15.28:0/3120881049 >> 172.21.15.28:0/3120881049 conn(0x7f2ec40f7af0 msgr2=0x7f2ec40f91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:51:51.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.578+0000 7f2eca12d640 1 -- 172.21.15.28:0/3120881049 shutdown_connections 2024-07-03T15:51:51.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.578+0000 7f2eca12d640 1 -- 172.21.15.28:0/3120881049 wait complete. 2024-07-03T15:51:51.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:51 smithi028 ceph-mon[92858]: pgmap v201: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 5 op/s 2024-07-03T15:51:51.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.659+0000 7f6e6543e640 1 -- 172.21.15.28:0/213628115 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6e600fc360 msgr2=0x7f6e600fc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:51.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.659+0000 7f6e6543e640 1 --2- 172.21.15.28:0/213628115 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6e600fc360 0x7f6e600fc770 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6e500048d0 tx=0x7f6e50009da0 comp rx=0 tx=0).stop 2024-07-03T15:51:51.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.659+0000 7f6e6543e640 1 -- 172.21.15.28:0/213628115 shutdown_connections 2024-07-03T15:51:51.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.659+0000 7f6e6543e640 1 --2- 172.21.15.28:0/213628115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e600fd630 0x7f6e600fdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:51.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.659+0000 7f6e6543e640 1 --2- 172.21.15.28:0/213628115 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6e600fc360 0x7f6e600fc770 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:51.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.659+0000 7f6e6543e640 1 -- 172.21.15.28:0/213628115 >> 172.21.15.28:0/213628115 conn(0x7f6e600f7b40 msgr2=0x7f6e600f9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:51:51.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.659+0000 7f6e6543e640 1 -- 172.21.15.28:0/213628115 shutdown_connections 2024-07-03T15:51:51.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.659+0000 7f6e6543e640 1 -- 172.21.15.28:0/213628115 wait complete. 2024-07-03T15:51:51.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.660+0000 7f6e6543e640 1 Processor -- start 2024-07-03T15:51:51.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.660+0000 7f6e6543e640 1 -- start start 2024-07-03T15:51:51.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.660+0000 7f6e6543e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6e600fc360 0x7f6e6010cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:51:51.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.660+0000 7f6e6543e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e600fd630 0x7f6e6010d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:51:51.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.660+0000 7f6e6543e640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6010db80 con 0x7f6e600fc360 2024-07-03T15:51:51.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.660+0000 7f6e6543e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6010e350 con 0x7f6e600fd630 2024-07-03T15:51:51.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.660+0000 7f6e5effd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6e600fc360 0x7f6e6010cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:51:51.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.660+0000 7f6e5effd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6e600fc360 0x7f6e6010cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:40076/0 (socket says 172.21.15.28:40076) 2024-07-03T15:51:51.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.660+0000 7f6e5effd640 1 -- 172.21.15.28:0/4099304248 learned_addr learned my addr 172.21.15.28:0/4099304248 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:51:51.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.660+0000 7f6e5e7fc640 1 --2- 172.21.15.28:0/4099304248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e600fd630 0x7f6e6010d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:51:51.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.660+0000 7f6e5effd640 1 -- 172.21.15.28:0/4099304248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e600fd630 msgr2=0x7f6e6010d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:51.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.660+0000 7f6e5effd640 1 --2- 172.21.15.28:0/4099304248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e600fd630 0x7f6e6010d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:51.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.660+0000 7f6e5effd640 1 -- 172.21.15.28:0/4099304248 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e50005040 con 0x7f6e600fc360 2024-07-03T15:51:51.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.660+0000 7f6e5effd640 1 --2- 172.21.15.28:0/4099304248 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6e600fc360 0x7f6e6010cff0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6e500048a0 tx=0x7f6e50009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:51:51.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.661+0000 7f6e477fe640 1 -- 172.21.15.28:0/4099304248 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e50038070 con 0x7f6e600fc360 2024-07-03T15:51:51.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.661+0000 7f6e6543e640 1 -- 172.21.15.28:0/4099304248 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e6010e580 con 0x7f6e600fc360 2024-07-03T15:51:51.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.661+0000 7f6e477fe640 1 -- 172.21.15.28:0/4099304248 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6e500092b0 con 0x7f6e600fc360 2024-07-03T15:51:51.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.661+0000 7f6e477fe640 1 -- 172.21.15.28:0/4099304248 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e50033780 con 0x7f6e600fc360 2024-07-03T15:51:51.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.661+0000 7f6e6543e640 1 -- 172.21.15.28:0/4099304248 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e6010eac0 con 0x7f6e600fc360 2024-07-03T15:51:51.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.662+0000 7f6e6543e640 1 -- 172.21.15.28:0/4099304248 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e600fb7c0 con 0x7f6e600fc360 2024-07-03T15:51:51.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.662+0000 7f6e477fe640 1 -- 172.21.15.28:0/4099304248 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6e50030020 con 0x7f6e600fc360 2024-07-03T15:51:51.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.663+0000 7f6e477fe640 1 --2- 172.21.15.28:0/4099304248 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f6e4806e0b0 0x7f6e48070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:51:51.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.663+0000 7f6e5e7fc640 1 --2- 172.21.15.28:0/4099304248 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f6e4806e0b0 0x7f6e48070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:51:51.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.663+0000 7f6e477fe640 1 -- 172.21.15.28:0/4099304248 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f6e500ad940 con 0x7f6e600fc360 2024-07-03T15:51:51.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.664+0000 7f6e5e7fc640 1 --2- 172.21.15.28:0/4099304248 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f6e4806e0b0 0x7f6e48070560 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6e6010cda0 tx=0x7f6e5400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:51:51.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.668+0000 7f6e477fe640 1 -- 172.21.15.28:0/4099304248 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f6e50076cc0 con 0x7f6e600fc360 2024-07-03T15:51:51.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:51 smithi148 ceph-mon[80490]: pgmap v201: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 5 op/s 2024-07-03T15:51:51.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.927+0000 7f6e6543e640 1 -- 172.21.15.28:0/4099304248 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6e6010ed80 con 0x7f6e600fc360 2024-07-03T15:51:51.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.928+0000 7f6e477fe640 1 -- 172.21.15.28:0/4099304248 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7f6e5007a380 con 0x7f6e600fc360 2024-07-03T15:51:51.931 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-07-03T15:51:51.931 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": { 2024-07-03T15:51:51.931 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 2 2024-07-03T15:51:51.931 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:51:51.932 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": { 2024-07-03T15:51:51.932 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 2 2024-07-03T15:51:51.932 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:51:51.932 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": { 2024-07-03T15:51:51.932 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 6 2024-07-03T15:51:51.932 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:51:51.932 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": { 2024-07-03T15:51:51.932 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 4 2024-07-03T15:51:51.932 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:51:51.932 INFO:teuthology.orchestra.run.smithi028.stdout: "overall": { 2024-07-03T15:51:51.932 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 14 2024-07-03T15:51:51.932 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-07-03T15:51:51.932 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-07-03T15:51:51.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.930+0000 7f6e6543e640 1 -- 172.21.15.28:0/4099304248 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f6e4806e0b0 msgr2=0x7f6e48070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:51.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.930+0000 7f6e6543e640 1 --2- 172.21.15.28:0/4099304248 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f6e4806e0b0 0x7f6e48070560 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6e6010cda0 tx=0x7f6e5400b040 comp rx=0 tx=0).stop 2024-07-03T15:51:51.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.930+0000 7f6e6543e640 1 -- 172.21.15.28:0/4099304248 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6e600fc360 msgr2=0x7f6e6010cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:51.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.930+0000 7f6e6543e640 1 --2- 172.21.15.28:0/4099304248 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6e600fc360 0x7f6e6010cff0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6e500048a0 tx=0x7f6e50009da0 comp rx=0 tx=0).stop 2024-07-03T15:51:51.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.930+0000 7f6e6543e640 1 -- 172.21.15.28:0/4099304248 shutdown_connections 2024-07-03T15:51:51.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.930+0000 7f6e6543e640 1 --2- 172.21.15.28:0/4099304248 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f6e4806e0b0 0x7f6e48070560 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:51.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.930+0000 7f6e6543e640 1 --2- 172.21.15.28:0/4099304248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e600fd630 0x7f6e6010d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:51.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.930+0000 7f6e6543e640 1 --2- 172.21.15.28:0/4099304248 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6e600fc360 0x7f6e6010cff0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:51.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.930+0000 7f6e6543e640 1 -- 172.21.15.28:0/4099304248 >> 172.21.15.28:0/4099304248 conn(0x7f6e600f7b40 msgr2=0x7f6e600f9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:51:51.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.930+0000 7f6e6543e640 1 -- 172.21.15.28:0/4099304248 shutdown_connections 2024-07-03T15:51:51.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:51.930+0000 7f6e6543e640 1 -- 172.21.15.28:0/4099304248 wait complete. 2024-07-03T15:51:52.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.012+0000 7f2a8985b640 1 -- 172.21.15.28:0/943399711 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a84107f70 msgr2=0x7f2a841083e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:52.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.012+0000 7f2a8985b640 1 --2- 172.21.15.28:0/943399711 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a84107f70 0x7f2a841083e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2a780048d0 tx=0x7f2a78009da0 comp rx=0 tx=0).stop 2024-07-03T15:51:52.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.012+0000 7f2a8985b640 1 -- 172.21.15.28:0/943399711 shutdown_connections 2024-07-03T15:51:52.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.012+0000 7f2a8985b640 1 --2- 172.21.15.28:0/943399711 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a84107f70 0x7f2a841083e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:52.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.012+0000 7f2a8985b640 1 --2- 172.21.15.28:0/943399711 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a84106de0 0x7f2a841071f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:52.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.012+0000 7f2a8985b640 1 -- 172.21.15.28:0/943399711 >> 172.21.15.28:0/943399711 conn(0x7f2a8406d740 msgr2=0x7f2a8406db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:51:52.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.012+0000 7f2a8985b640 1 -- 172.21.15.28:0/943399711 shutdown_connections 2024-07-03T15:51:52.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.012+0000 7f2a8985b640 1 -- 172.21.15.28:0/943399711 wait complete. 2024-07-03T15:51:52.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.012+0000 7f2a8985b640 1 Processor -- start 2024-07-03T15:51:52.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.013+0000 7f2a8985b640 1 -- start start 2024-07-03T15:51:52.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.013+0000 7f2a8985b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a84106de0 0x7f2a84117a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:51:52.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.013+0000 7f2a8985b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a84107f70 0x7f2a84117f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:51:52.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.013+0000 7f2a8985b640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a84118db0 con 0x7f2a84107f70 2024-07-03T15:51:52.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.013+0000 7f2a8985b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a84118f20 con 0x7f2a84106de0 2024-07-03T15:51:52.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.013+0000 7f2a827fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a84107f70 0x7f2a84117f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:51:52.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.013+0000 7f2a827fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a84107f70 0x7f2a84117f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:40096/0 (socket says 172.21.15.28:40096) 2024-07-03T15:51:52.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.013+0000 7f2a827fc640 1 -- 172.21.15.28:0/3792428309 learned_addr learned my addr 172.21.15.28:0/3792428309 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:51:52.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.013+0000 7f2a82ffd640 1 --2- 172.21.15.28:0/3792428309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a84106de0 0x7f2a84117a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:51:52.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.013+0000 7f2a827fc640 1 -- 172.21.15.28:0/3792428309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a84106de0 msgr2=0x7f2a84117a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:52.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.013+0000 7f2a827fc640 1 --2- 172.21.15.28:0/3792428309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a84106de0 0x7f2a84117a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:52.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.013+0000 7f2a827fc640 1 -- 172.21.15.28:0/3792428309 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a78005040 con 0x7f2a84107f70 2024-07-03T15:51:52.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.013+0000 7f2a82ffd640 1 --2- 172.21.15.28:0/3792428309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a84106de0 0x7f2a84117a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-03T15:51:52.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.013+0000 7f2a827fc640 1 --2- 172.21.15.28:0/3792428309 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a84107f70 0x7f2a84117f90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2a78003080 tx=0x7f2a78009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:51:52.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.014+0000 7f2a6b7fe640 1 -- 172.21.15.28:0/3792428309 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a78038070 con 0x7f2a84107f70 2024-07-03T15:51:52.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.014+0000 7f2a6b7fe640 1 -- 172.21.15.28:0/3792428309 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a78032970 con 0x7f2a84107f70 2024-07-03T15:51:52.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.014+0000 7f2a8985b640 1 -- 172.21.15.28:0/3792428309 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a841191a0 con 0x7f2a84107f70 2024-07-03T15:51:52.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.014+0000 7f2a6b7fe640 1 -- 172.21.15.28:0/3792428309 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a78033900 con 0x7f2a84107f70 2024-07-03T15:51:52.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.014+0000 7f2a8985b640 1 -- 172.21.15.28:0/3792428309 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a84119660 con 0x7f2a84107f70 2024-07-03T15:51:52.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.015+0000 7f2a8985b640 1 -- 172.21.15.28:0/3792428309 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a8406f3b0 con 0x7f2a84107f70 2024-07-03T15:51:52.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.015+0000 7f2a6b7fe640 1 -- 172.21.15.28:0/3792428309 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2a78030020 con 0x7f2a84107f70 2024-07-03T15:51:52.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.016+0000 7f2a6b7fe640 1 --2- 172.21.15.28:0/3792428309 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f2a6c06e0b0 0x7f2a6c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:51:52.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.016+0000 7f2a6b7fe640 1 -- 172.21.15.28:0/3792428309 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f2a780adc80 con 0x7f2a84107f70 2024-07-03T15:51:52.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.016+0000 7f2a82ffd640 1 --2- 172.21.15.28:0/3792428309 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f2a6c06e0b0 0x7f2a6c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:51:52.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.016+0000 7f2a82ffd640 1 --2- 172.21.15.28:0/3792428309 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f2a6c06e0b0 0x7f2a6c070560 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2a74006fd0 tx=0x7f2a74008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:51:52.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.020+0000 7f2a6b7fe640 1 -- 172.21.15.28:0/3792428309 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f2a78076f50 con 0x7f2a84107f70 2024-07-03T15:51:52.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.247+0000 7f2a8985b640 1 -- 172.21.15.28:0/3792428309 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2a8404b530 con 0x7f2a84107f70 2024-07-03T15:51:52.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.247+0000 7f2a6b7fe640 1 -- 172.21.15.28:0/3792428309 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1967 (secure 0 0 0) 0x7f2a7807a610 con 0x7f2a84107f70 2024-07-03T15:51:52.251 INFO:teuthology.orchestra.run.smithi028.stdout:e35 2024-07-03T15:51:52.251 INFO:teuthology.orchestra.run.smithi028.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-03T15:51:52.251 INFO:teuthology.orchestra.run.smithi028.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:51:52.251 INFO:teuthology.orchestra.run.smithi028.stdout:legacy client fscid: 1 2024-07-03T15:51:52.251 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:51:52.251 INFO:teuthology.orchestra.run.smithi028.stdout:Filesystem 'cephfs' (1) 2024-07-03T15:51:52.252 INFO:teuthology.orchestra.run.smithi028.stdout:fs_name cephfs 2024-07-03T15:51:52.252 INFO:teuthology.orchestra.run.smithi028.stdout:epoch 35 2024-07-03T15:51:52.252 INFO:teuthology.orchestra.run.smithi028.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-03T15:51:52.252 INFO:teuthology.orchestra.run.smithi028.stdout:created 2024-07-03T15:41:40.656298+0000 2024-07-03T15:51:52.252 INFO:teuthology.orchestra.run.smithi028.stdout:modified 2024-07-03T15:51:35.429732+0000 2024-07-03T15:51:52.252 INFO:teuthology.orchestra.run.smithi028.stdout:tableserver 0 2024-07-03T15:51:52.252 INFO:teuthology.orchestra.run.smithi028.stdout:root 0 2024-07-03T15:51:52.252 INFO:teuthology.orchestra.run.smithi028.stdout:session_timeout 60 2024-07-03T15:51:52.253 INFO:teuthology.orchestra.run.smithi028.stdout:session_autoclose 300 2024-07-03T15:51:52.253 INFO:teuthology.orchestra.run.smithi028.stdout:max_file_size 1099511627776 2024-07-03T15:51:52.253 INFO:teuthology.orchestra.run.smithi028.stdout:max_xattr_size 65536 2024-07-03T15:51:52.253 INFO:teuthology.orchestra.run.smithi028.stdout:required_client_features {} 2024-07-03T15:51:52.253 INFO:teuthology.orchestra.run.smithi028.stdout:last_failure 0 2024-07-03T15:51:52.253 INFO:teuthology.orchestra.run.smithi028.stdout:last_failure_osd_epoch 82 2024-07-03T15:51:52.253 INFO:teuthology.orchestra.run.smithi028.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-07-03T15:51:52.253 INFO:teuthology.orchestra.run.smithi028.stdout:max_mds 1 2024-07-03T15:51:52.254 INFO:teuthology.orchestra.run.smithi028.stdout:in 0 2024-07-03T15:51:52.254 INFO:teuthology.orchestra.run.smithi028.stdout:up {0=34376} 2024-07-03T15:51:52.254 INFO:teuthology.orchestra.run.smithi028.stdout:failed 2024-07-03T15:51:52.254 INFO:teuthology.orchestra.run.smithi028.stdout:damaged 2024-07-03T15:51:52.255 INFO:teuthology.orchestra.run.smithi028.stdout:stopped 2024-07-03T15:51:52.255 INFO:teuthology.orchestra.run.smithi028.stdout:data_pools [3] 2024-07-03T15:51:52.255 INFO:teuthology.orchestra.run.smithi028.stdout:metadata_pool 2 2024-07-03T15:51:52.255 INFO:teuthology.orchestra.run.smithi028.stdout:inline_data disabled 2024-07-03T15:51:52.255 INFO:teuthology.orchestra.run.smithi028.stdout:balancer 2024-07-03T15:51:52.256 INFO:teuthology.orchestra.run.smithi028.stdout:bal_rank_mask -1 2024-07-03T15:51:52.256 INFO:teuthology.orchestra.run.smithi028.stdout:standby_count_wanted 1 2024-07-03T15:51:52.256 INFO:teuthology.orchestra.run.smithi028.stdout:qdb_cluster leader: 34376 members: 34376 2024-07-03T15:51:52.256 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi028.rhpvve{0:34376} state up:active seq 16 join_fscid=1 addr [v2:172.21.15.28:6826/3127260162,v1:172.21.15.28:6827/3127260162] compat {c=[1],r=[1],i=[1fff]}] 2024-07-03T15:51:52.256 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi028.wmbxew{0:34408} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.28:6828/2908130327,v1:172.21.15.28:6829/2908130327] compat {c=[1],r=[1],i=[1fff]}] 2024-07-03T15:51:52.256 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:51:52.256 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:51:52.256 INFO:teuthology.orchestra.run.smithi028.stdout:Standby daemons: 2024-07-03T15:51:52.256 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:51:52.256 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi148.xgpuld{-1:44301} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/1480077002,v1:172.21.15.148:6825/1480077002] compat {c=[1],r=[1],i=[1fff]}] 2024-07-03T15:51:52.256 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi148.szomfl{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6826/3922163302,v1:172.21.15.148:6827/3922163302] compat {c=[1],r=[1],i=[1fff]}] 2024-07-03T15:51:52.257 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 35 2024-07-03T15:51:52.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.249+0000 7f2a8985b640 1 -- 172.21.15.28:0/3792428309 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f2a6c06e0b0 msgr2=0x7f2a6c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:52.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.249+0000 7f2a8985b640 1 --2- 172.21.15.28:0/3792428309 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f2a6c06e0b0 0x7f2a6c070560 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2a74006fd0 tx=0x7f2a74008040 comp rx=0 tx=0).stop 2024-07-03T15:51:52.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.249+0000 7f2a8985b640 1 -- 172.21.15.28:0/3792428309 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a84107f70 msgr2=0x7f2a84117f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:52.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.249+0000 7f2a8985b640 1 --2- 172.21.15.28:0/3792428309 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a84107f70 0x7f2a84117f90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2a78003080 tx=0x7f2a78009da0 comp rx=0 tx=0).stop 2024-07-03T15:51:52.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.249+0000 7f2a8985b640 1 -- 172.21.15.28:0/3792428309 shutdown_connections 2024-07-03T15:51:52.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.249+0000 7f2a8985b640 1 --2- 172.21.15.28:0/3792428309 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f2a6c06e0b0 0x7f2a6c070560 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:52.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.249+0000 7f2a8985b640 1 --2- 172.21.15.28:0/3792428309 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a84107f70 0x7f2a84117f90 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:52.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.249+0000 7f2a8985b640 1 --2- 172.21.15.28:0/3792428309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a84106de0 0x7f2a84117a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:52.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.249+0000 7f2a8985b640 1 -- 172.21.15.28:0/3792428309 >> 172.21.15.28:0/3792428309 conn(0x7f2a8406d740 msgr2=0x7f2a84105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:51:52.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.250+0000 7f2a8985b640 1 -- 172.21.15.28:0/3792428309 shutdown_connections 2024-07-03T15:51:52.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.250+0000 7f2a8985b640 1 -- 172.21.15.28:0/3792428309 wait complete. 2024-07-03T15:51:52.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.333+0000 7f4cbe7c7640 1 -- 172.21.15.28:0/815232052 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4cb80fb260 msgr2=0x7f4cb80fb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:52.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.333+0000 7f4cbe7c7640 1 --2- 172.21.15.28:0/815232052 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4cb80fb260 0x7f4cb80fb670 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4ca80048d0 tx=0x7f4ca8009da0 comp rx=0 tx=0).stop 2024-07-03T15:51:52.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.333+0000 7f4cbe7c7640 1 -- 172.21.15.28:0/815232052 shutdown_connections 2024-07-03T15:51:52.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.333+0000 7f4cbe7c7640 1 --2- 172.21.15.28:0/815232052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cb80fc530 0x7f4cb80fc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:52.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.333+0000 7f4cbe7c7640 1 --2- 172.21.15.28:0/815232052 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4cb80fb260 0x7f4cb80fb670 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:52.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.333+0000 7f4cbe7c7640 1 -- 172.21.15.28:0/815232052 >> 172.21.15.28:0/815232052 conn(0x7f4cb80f6a20 msgr2=0x7f4cb80f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:51:52.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.333+0000 7f4cbe7c7640 1 -- 172.21.15.28:0/815232052 shutdown_connections 2024-07-03T15:51:52.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.333+0000 7f4cbe7c7640 1 -- 172.21.15.28:0/815232052 wait complete. 2024-07-03T15:51:52.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.334+0000 7f4cbe7c7640 1 Processor -- start 2024-07-03T15:51:52.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.334+0000 7f4cbe7c7640 1 -- start start 2024-07-03T15:51:52.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.334+0000 7f4cbe7c7640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4cb80fb260 0x7f4cb810fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:51:52.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.334+0000 7f4cbe7c7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cb80fc530 0x7f4cb8112230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:51:52.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.334+0000 7f4cbe7c7640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb8066560 con 0x7f4cb80fb260 2024-07-03T15:51:52.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.334+0000 7f4cbe7c7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb80666d0 con 0x7f4cb80fc530 2024-07-03T15:51:52.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.334+0000 7f4cb7fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4cb80fb260 0x7f4cb810fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:51:52.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.334+0000 7f4cb7fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4cb80fb260 0x7f4cb810fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:40112/0 (socket says 172.21.15.28:40112) 2024-07-03T15:51:52.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.334+0000 7f4cb7fff640 1 -- 172.21.15.28:0/55707515 learned_addr learned my addr 172.21.15.28:0/55707515 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:51:52.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.334+0000 7f4cb77fe640 1 --2- 172.21.15.28:0/55707515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cb80fc530 0x7f4cb8112230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:51:52.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.335+0000 7f4cb7fff640 1 -- 172.21.15.28:0/55707515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cb80fc530 msgr2=0x7f4cb8112230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:52.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.335+0000 7f4cb7fff640 1 --2- 172.21.15.28:0/55707515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cb80fc530 0x7f4cb8112230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:52.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.335+0000 7f4cb7fff640 1 -- 172.21.15.28:0/55707515 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca8005040 con 0x7f4cb80fb260 2024-07-03T15:51:52.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.335+0000 7f4cb7fff640 1 --2- 172.21.15.28:0/55707515 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4cb80fb260 0x7f4cb810fce0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4ca8009f80 tx=0x7f4ca8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:51:52.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.335+0000 7f4cb4ff9640 1 -- 172.21.15.28:0/55707515 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ca8038070 con 0x7f4cb80fb260 2024-07-03T15:51:52.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.335+0000 7f4cb4ff9640 1 -- 172.21.15.28:0/55707515 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ca8008ee0 con 0x7f4cb80fb260 2024-07-03T15:51:52.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.335+0000 7f4cbe7c7640 1 -- 172.21.15.28:0/55707515 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cb8112770 con 0x7f4cb80fb260 2024-07-03T15:51:52.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.335+0000 7f4cb4ff9640 1 -- 172.21.15.28:0/55707515 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ca8033890 con 0x7f4cb80fb260 2024-07-03T15:51:52.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.335+0000 7f4cbe7c7640 1 -- 172.21.15.28:0/55707515 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cb8112cb0 con 0x7f4cb80fb260 2024-07-03T15:51:52.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.336+0000 7f4cbe7c7640 1 -- 172.21.15.28:0/55707515 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cb80fa6c0 con 0x7f4cb80fb260 2024-07-03T15:51:52.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.337+0000 7f4cb4ff9640 1 -- 172.21.15.28:0/55707515 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4ca8030020 con 0x7f4cb80fb260 2024-07-03T15:51:52.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.337+0000 7f4cb4ff9640 1 --2- 172.21.15.28:0/55707515 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f4ca006e0b0 0x7f4ca0070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:51:52.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.337+0000 7f4cb4ff9640 1 -- 172.21.15.28:0/55707515 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f4ca80ada50 con 0x7f4cb80fb260 2024-07-03T15:51:52.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.337+0000 7f4cb77fe640 1 --2- 172.21.15.28:0/55707515 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f4ca006e0b0 0x7f4ca0070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:51:52.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.338+0000 7f4cb77fe640 1 --2- 172.21.15.28:0/55707515 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f4ca006e0b0 0x7f4ca0070560 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4cac007fd0 tx=0x7f4cac009e90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:51:52.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.341+0000 7f4cb4ff9640 1 -- 172.21.15.28:0/55707515 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f4ca8076d20 con 0x7f4cb80fb260 2024-07-03T15:51:52.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:52 smithi148 ceph-mon[80490]: from='client.34446 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:51:52.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:52 smithi148 ceph-mon[80490]: from='client.34450 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:51:52.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:52 smithi148 ceph-mon[80490]: from='client.34454 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:51:52.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:52 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/4099304248' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:52.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:52 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:51:52.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:52 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:52.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:52 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:52.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:52 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/3792428309' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-03T15:51:52.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.543+0000 7f4cbe7c7640 1 -- 172.21.15.28:0/55707515 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4cb8100840 con 0x7f4ca006e0b0 2024-07-03T15:51:52.547 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:52 smithi028 ceph-mon[92858]: from='client.34446 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:51:52.547 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:52 smithi028 ceph-mon[92858]: from='client.34450 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:51:52.547 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:52 smithi028 ceph-mon[92858]: from='client.34454 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:51:52.547 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:52 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/4099304248' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:52.548 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:52 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:51:52.548 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:52 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:52.548 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:52 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:52.548 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:52 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/3792428309' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-03T15:51:52.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.545+0000 7f4cb4ff9640 1 -- 172.21.15.28:0/55707515 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f4cb8100840 con 0x7f4ca006e0b0 2024-07-03T15:51:52.548 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-07-03T15:51:52.549 INFO:teuthology.orchestra.run.smithi028.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2", 2024-07-03T15:51:52.549 INFO:teuthology.orchestra.run.smithi028.stdout: "in_progress": true, 2024-07-03T15:51:52.549 INFO:teuthology.orchestra.run.smithi028.stdout: "which": "Upgrading all daemon types on all hosts", 2024-07-03T15:51:52.549 INFO:teuthology.orchestra.run.smithi028.stdout: "services_complete": [ 2024-07-03T15:51:52.549 INFO:teuthology.orchestra.run.smithi028.stdout: "mds", 2024-07-03T15:51:52.549 INFO:teuthology.orchestra.run.smithi028.stdout: "mon", 2024-07-03T15:51:52.549 INFO:teuthology.orchestra.run.smithi028.stdout: "osd", 2024-07-03T15:51:52.549 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr", 2024-07-03T15:51:52.549 INFO:teuthology.orchestra.run.smithi028.stdout: "crash" 2024-07-03T15:51:52.549 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-07-03T15:51:52.549 INFO:teuthology.orchestra.run.smithi028.stdout: "progress": "16/21 daemons upgraded", 2024-07-03T15:51:52.549 INFO:teuthology.orchestra.run.smithi028.stdout: "message": "Currently upgrading node-exporter daemons", 2024-07-03T15:51:52.549 INFO:teuthology.orchestra.run.smithi028.stdout: "is_paused": false 2024-07-03T15:51:52.549 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-07-03T15:51:52.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.546+0000 7f4cbe7c7640 1 -- 172.21.15.28:0/55707515 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f4ca006e0b0 msgr2=0x7f4ca0070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:52.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.546+0000 7f4cbe7c7640 1 --2- 172.21.15.28:0/55707515 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f4ca006e0b0 0x7f4ca0070560 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4cac007fd0 tx=0x7f4cac009e90 comp rx=0 tx=0).stop 2024-07-03T15:51:52.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.546+0000 7f4cbe7c7640 1 -- 172.21.15.28:0/55707515 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4cb80fb260 msgr2=0x7f4cb810fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:52.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.546+0000 7f4cbe7c7640 1 --2- 172.21.15.28:0/55707515 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4cb80fb260 0x7f4cb810fce0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4ca8009f80 tx=0x7f4ca8009da0 comp rx=0 tx=0).stop 2024-07-03T15:51:52.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.547+0000 7f4cbe7c7640 1 -- 172.21.15.28:0/55707515 shutdown_connections 2024-07-03T15:51:52.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.547+0000 7f4cbe7c7640 1 --2- 172.21.15.28:0/55707515 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f4ca006e0b0 0x7f4ca0070560 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:52.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.547+0000 7f4cbe7c7640 1 --2- 172.21.15.28:0/55707515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cb80fc530 0x7f4cb8112230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:52.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.547+0000 7f4cbe7c7640 1 --2- 172.21.15.28:0/55707515 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4cb80fb260 0x7f4cb810fce0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:52.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.547+0000 7f4cbe7c7640 1 -- 172.21.15.28:0/55707515 >> 172.21.15.28:0/55707515 conn(0x7f4cb80f6a20 msgr2=0x7f4cb80f8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:51:52.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.547+0000 7f4cbe7c7640 1 -- 172.21.15.28:0/55707515 shutdown_connections 2024-07-03T15:51:52.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.547+0000 7f4cbe7c7640 1 -- 172.21.15.28:0/55707515 wait complete. 2024-07-03T15:51:52.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.629+0000 7ff087e6d640 1 -- 172.21.15.28:0/97170424 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff0800fc370 msgr2=0x7ff0800fc780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:52.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.629+0000 7ff087e6d640 1 --2- 172.21.15.28:0/97170424 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff0800fc370 0x7ff0800fc780 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff07c0048d0 tx=0x7ff07c009da0 comp rx=0 tx=0).stop 2024-07-03T15:51:52.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.629+0000 7ff087e6d640 1 -- 172.21.15.28:0/97170424 shutdown_connections 2024-07-03T15:51:52.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.629+0000 7ff087e6d640 1 --2- 172.21.15.28:0/97170424 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0800fd640 0x7ff0800fdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:52.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.629+0000 7ff087e6d640 1 --2- 172.21.15.28:0/97170424 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff0800fc370 0x7ff0800fc780 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:52.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.629+0000 7ff087e6d640 1 -- 172.21.15.28:0/97170424 >> 172.21.15.28:0/97170424 conn(0x7ff0800f7b50 msgr2=0x7ff0800f9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:51:52.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.629+0000 7ff087e6d640 1 -- 172.21.15.28:0/97170424 shutdown_connections 2024-07-03T15:51:52.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.629+0000 7ff087e6d640 1 -- 172.21.15.28:0/97170424 wait complete. 2024-07-03T15:51:52.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.630+0000 7ff087e6d640 1 Processor -- start 2024-07-03T15:51:52.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.630+0000 7ff087e6d640 1 -- start start 2024-07-03T15:51:52.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.630+0000 7ff087e6d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0800fc370 0x7ff080071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:51:52.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.630+0000 7ff087e6d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff0800fd640 0x7ff08006f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:51:52.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.630+0000 7ff087e6d640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08006fe00 con 0x7ff0800fd640 2024-07-03T15:51:52.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.630+0000 7ff087e6d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08006ff40 con 0x7ff0800fc370 2024-07-03T15:51:52.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.630+0000 7ff085408640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff0800fd640 0x7ff08006f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:51:52.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.630+0000 7ff085408640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff0800fd640 0x7ff08006f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:40130/0 (socket says 172.21.15.28:40130) 2024-07-03T15:51:52.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.630+0000 7ff085408640 1 -- 172.21.15.28:0/49687079 learned_addr learned my addr 172.21.15.28:0/49687079 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:51:52.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.630+0000 7ff085c09640 1 --2- 172.21.15.28:0/49687079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0800fc370 0x7ff080071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:51:52.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.630+0000 7ff085408640 1 -- 172.21.15.28:0/49687079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0800fc370 msgr2=0x7ff080071380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:52.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.630+0000 7ff085408640 1 --2- 172.21.15.28:0/49687079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0800fc370 0x7ff080071380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:52.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.630+0000 7ff085408640 1 -- 172.21.15.28:0/49687079 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff07c005040 con 0x7ff0800fd640 2024-07-03T15:51:52.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.630+0000 7ff085408640 1 --2- 172.21.15.28:0/49687079 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff0800fd640 0x7ff08006f870 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff070002a50 tx=0x7ff070004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:51:52.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.631+0000 7ff0767fc640 1 -- 172.21.15.28:0/49687079 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff070007c30 con 0x7ff0800fd640 2024-07-03T15:51:52.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.631+0000 7ff0767fc640 1 -- 172.21.15.28:0/49687079 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff07000f040 con 0x7ff0800fd640 2024-07-03T15:51:52.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.631+0000 7ff087e6d640 1 -- 172.21.15.28:0/49687079 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff080070220 con 0x7ff0800fd640 2024-07-03T15:51:52.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.631+0000 7ff0767fc640 1 -- 172.21.15.28:0/49687079 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff07000dad0 con 0x7ff0800fd640 2024-07-03T15:51:52.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.631+0000 7ff087e6d640 1 -- 172.21.15.28:0/49687079 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff080114fc0 con 0x7ff0800fd640 2024-07-03T15:51:52.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.632+0000 7ff087e6d640 1 -- 172.21.15.28:0/49687079 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0800fb7d0 con 0x7ff0800fd640 2024-07-03T15:51:52.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.632+0000 7ff0767fc640 1 -- 172.21.15.28:0/49687079 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff07000dd40 con 0x7ff0800fd640 2024-07-03T15:51:52.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.633+0000 7ff0767fc640 1 --2- 172.21.15.28:0/49687079 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7ff06c06e0b0 0x7ff06c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:51:52.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.633+0000 7ff0767fc640 1 -- 172.21.15.28:0/49687079 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7ff07008d3e0 con 0x7ff0800fd640 2024-07-03T15:51:52.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.633+0000 7ff085c09640 1 --2- 172.21.15.28:0/49687079 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7ff06c06e0b0 0x7ff06c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:51:52.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.633+0000 7ff085c09640 1 --2- 172.21.15.28:0/49687079 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7ff06c06e0b0 0x7ff06c070560 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff07c0043d0 tx=0x7ff07c035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:51:52.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.637+0000 7ff0767fc640 1 -- 172.21.15.28:0/49687079 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff070056760 con 0x7ff0800fd640 2024-07-03T15:51:52.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.894+0000 7ff087e6d640 1 -- 172.21.15.28:0/49687079 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff08004b530 con 0x7ff0800fd640 2024-07-03T15:51:52.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.894+0000 7ff0767fc640 1 -- 172.21.15.28:0/49687079 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7ff070059e20 con 0x7ff0800fd640 2024-07-03T15:51:52.898 INFO:teuthology.orchestra.run.smithi028.stdout:HEALTH_OK 2024-07-03T15:51:52.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.896+0000 7ff087e6d640 1 -- 172.21.15.28:0/49687079 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7ff06c06e0b0 msgr2=0x7ff06c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:52.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.896+0000 7ff087e6d640 1 --2- 172.21.15.28:0/49687079 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7ff06c06e0b0 0x7ff06c070560 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff07c0043d0 tx=0x7ff07c035040 comp rx=0 tx=0).stop 2024-07-03T15:51:52.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.896+0000 7ff087e6d640 1 -- 172.21.15.28:0/49687079 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff0800fd640 msgr2=0x7ff08006f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:51:52.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.896+0000 7ff087e6d640 1 --2- 172.21.15.28:0/49687079 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff0800fd640 0x7ff08006f870 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff070002a50 tx=0x7ff070004600 comp rx=0 tx=0).stop 2024-07-03T15:51:52.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.896+0000 7ff087e6d640 1 -- 172.21.15.28:0/49687079 shutdown_connections 2024-07-03T15:51:52.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.896+0000 7ff087e6d640 1 --2- 172.21.15.28:0/49687079 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7ff06c06e0b0 0x7ff06c070560 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:52.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.896+0000 7ff087e6d640 1 --2- 172.21.15.28:0/49687079 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff0800fd640 0x7ff08006f870 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:52.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.896+0000 7ff087e6d640 1 --2- 172.21.15.28:0/49687079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0800fc370 0x7ff080071380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:51:52.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.896+0000 7ff087e6d640 1 -- 172.21.15.28:0/49687079 >> 172.21.15.28:0/49687079 conn(0x7ff0800f7b50 msgr2=0x7ff0800f9180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:51:52.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.897+0000 7ff087e6d640 1 -- 172.21.15.28:0/49687079 shutdown_connections 2024-07-03T15:51:52.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:51:52.897+0000 7ff087e6d640 1 -- 172.21.15.28:0/49687079 wait complete. 2024-07-03T15:51:53.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:53 smithi148 ceph-mon[80490]: pgmap v202: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 6.5 MiB/s rd, 2 op/s 2024-07-03T15:51:53.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:53 smithi148 ceph-mon[80490]: from='client.34466 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:51:53.732 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:53 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/49687079' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-03T15:51:53.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:53 smithi028 ceph-mon[92858]: pgmap v202: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 6.5 MiB/s rd, 2 op/s 2024-07-03T15:51:53.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:53 smithi028 ceph-mon[92858]: from='client.34466 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:51:53.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:53 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/49687079' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-03T15:51:55.438 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:55 smithi028 ceph-mon[92858]: pgmap v203: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 4.3 MiB/s rd, 2 op/s 2024-07-03T15:51:55.438 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:55 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:55.438 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:55 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:55.438 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:55 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:55.439 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:55 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:55.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:55 smithi148 ceph-mon[80490]: pgmap v203: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 4.3 MiB/s rd, 2 op/s 2024-07-03T15:51:55.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:55 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:55.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:55 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:55.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:55 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:55.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:55 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:57.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:57 smithi148 ceph-mon[80490]: pgmap v204: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 342 KiB/s rd, 1 op/s 2024-07-03T15:51:57.622 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:57 smithi028 ceph-mon[92858]: pgmap v204: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 342 KiB/s rd, 1 op/s 2024-07-03T15:51:59.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:59 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:59.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:59 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:59.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:59 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:51:59.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:59 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:51:59.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:59 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:59.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:59 smithi028 ceph-mon[92858]: pgmap v205: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:51:59.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:59 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:51:59.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:59 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:59.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:59 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:59.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:59 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:59.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:59 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:59.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:59 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:59.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:59 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:59.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:59 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:59.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:59 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:59.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:59 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:59.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:59 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:59.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:59 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:59.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:59 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:59.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:51:59 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:59.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:59 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:59.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:59 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:59.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:59 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:51:59.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:59 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:51:59.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:59 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:51:59.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:59 smithi148 ceph-mon[80490]: pgmap v205: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:51:59.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:59 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:51:59.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:59 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:59.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:59 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:59.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:59 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:59.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:59 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:59.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:59 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:59.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:59 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:59.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:59 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:59.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:59 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:59.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:59 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:59.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:59 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:59.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:59 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:59.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:59 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:51:59.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:51:59 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:00.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:00 smithi028 ceph-mon[92858]: Upgrade: Updating prometheus.smithi028 2024-07-03T15:52:00.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:00 smithi028 ceph-mon[92858]: Deploying daemon prometheus.smithi028 on smithi028 2024-07-03T15:52:00.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:00 smithi148 ceph-mon[80490]: Upgrade: Updating prometheus.smithi028 2024-07-03T15:52:00.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:00 smithi148 ceph-mon[80490]: Deploying daemon prometheus.smithi028 on smithi028 2024-07-03T15:52:01.257 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:01 smithi028 ceph-mon[92858]: pgmap v206: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:52:01.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:01 smithi148 ceph-mon[80490]: pgmap v206: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:52:03.221 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:03 smithi028 ceph-mon[92858]: pgmap v207: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:52:03.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:03 smithi148 ceph-mon[80490]: pgmap v207: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:52:06.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:05 smithi148 ceph-mon[80490]: pgmap v208: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:52:06.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:05 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:06.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:05 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:06.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:05 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:52:06.282 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:05 smithi028 ceph-mon[92858]: pgmap v208: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:52:06.282 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:05 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:06.282 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:05 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:06.282 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:05 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:52:08.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:07 smithi028 ceph-mon[92858]: pgmap v209: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:52:08.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:07 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:52:08.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:07 smithi148 ceph-mon[80490]: pgmap v209: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:52:08.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:07 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:52:10.180 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:09 smithi028 ceph-mon[92858]: pgmap v210: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:52:10.180 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:09 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:10.180 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:09 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:10.180 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:09 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:10.181 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:09 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:10.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:09 smithi148 ceph-mon[80490]: pgmap v210: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:52:10.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:09 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:10.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:09 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:10.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:09 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:10.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:09 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:12.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:11 smithi148 ceph-mon[80490]: pgmap v211: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:52:12.272 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:11 smithi028 ceph-mon[92858]: pgmap v211: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:52:14.050 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:13 smithi028 ceph-mon[92858]: pgmap v212: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:52:14.050 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:13 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:14.050 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:13 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:14.050 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:13 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:52:14.050 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:13 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:52:14.050 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:13 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:14.050 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:13 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-07-03T15:52:14.051 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:13 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:52:14.051 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:13 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:14.051 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:13 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:14.051 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:13 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:14.051 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:13 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:14.051 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:13 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:14.051 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:13 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:14.051 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:13 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:14.051 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:13 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:14.051 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:13 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:14.051 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:13 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:14.051 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:13 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:14.052 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:13 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:14.052 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:13 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:14.052 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:13 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:14.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:13 smithi148 ceph-mon[80490]: pgmap v212: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:52:14.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:13 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:14.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:13 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:14.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:13 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:52:14.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:13 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:52:14.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:13 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:14.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:13 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-07-03T15:52:14.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:13 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:52:14.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:13 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:14.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:13 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:14.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:13 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:14.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:13 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:14.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:13 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:14.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:13 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:14.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:13 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:14.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:13 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:14.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:13 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:14.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:13 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:14.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:13 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:14.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:13 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:14.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:13 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:14.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:13 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:15.054 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:14 smithi028 ceph-mon[92858]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-07-03T15:52:15.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:14 smithi148 ceph-mon[80490]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-07-03T15:52:16.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:15 smithi028 ceph-mon[92858]: pgmap v213: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:52:16.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:15 smithi028 ceph-mon[92858]: Upgrade: Updating alertmanager.smithi028 2024-07-03T15:52:16.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:15 smithi028 ceph-mon[92858]: Deploying daemon alertmanager.smithi028 on smithi028 2024-07-03T15:52:16.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:15 smithi148 ceph-mon[80490]: pgmap v213: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:52:16.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:15 smithi148 ceph-mon[80490]: Upgrade: Updating alertmanager.smithi028 2024-07-03T15:52:16.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:15 smithi148 ceph-mon[80490]: Deploying daemon alertmanager.smithi028 on smithi028 2024-07-03T15:52:17.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:16 smithi148 ceph-mon[80490]: pgmap v214: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:52:17.326 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:16 smithi028 ceph-mon[92858]: pgmap v214: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:52:19.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:19 smithi028 ceph-mon[92858]: pgmap v215: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:52:19.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:19 smithi148 ceph-mon[80490]: pgmap v215: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:52:21.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:20 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:21.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:20 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:21.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:20 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:52:21.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:20 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:21.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:20 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:21.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:20 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:52:22.186 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:21 smithi028 ceph-mon[92858]: pgmap v216: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:52:22.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:21 smithi148 ceph-mon[80490]: pgmap v216: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:52:22.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:22.981+0000 7f2870013640 1 -- 172.21.15.28:0/2768696142 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28680fc370 msgr2=0x7f28680fc780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:52:22.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:22.981+0000 7f2870013640 1 --2- 172.21.15.28:0/2768696142 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28680fc370 0x7f28680fc780 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f28640048d0 tx=0x7f2864009da0 comp rx=0 tx=0).stop 2024-07-03T15:52:22.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:22.981+0000 7f2870013640 1 -- 172.21.15.28:0/2768696142 shutdown_connections 2024-07-03T15:52:22.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:22.981+0000 7f2870013640 1 --2- 172.21.15.28:0/2768696142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28680fd640 0x7f28680fdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:22.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:22.981+0000 7f2870013640 1 --2- 172.21.15.28:0/2768696142 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28680fc370 0x7f28680fc780 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:22.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:22.981+0000 7f2870013640 1 -- 172.21.15.28:0/2768696142 >> 172.21.15.28:0/2768696142 conn(0x7f28680f7b50 msgr2=0x7f28680f9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:52:22.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:22.981+0000 7f2870013640 1 -- 172.21.15.28:0/2768696142 shutdown_connections 2024-07-03T15:52:22.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:22.981+0000 7f2870013640 1 -- 172.21.15.28:0/2768696142 wait complete. 2024-07-03T15:52:22.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:22.982+0000 7f2870013640 1 Processor -- start 2024-07-03T15:52:22.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:22.982+0000 7f2870013640 1 -- start start 2024-07-03T15:52:22.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:22.982+0000 7f2870013640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28680fc370 0x7f286810cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:52:22.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:22.982+0000 7f2870013640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28680fd640 0x7f286810d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:52:22.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:22.982+0000 7f2870013640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f286810e310 con 0x7f28680fd640 2024-07-03T15:52:22.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:22.982+0000 7f2870013640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f286810e480 con 0x7f28680fc370 2024-07-03T15:52:22.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:22.982+0000 7f286d5ae640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28680fd640 0x7f286810d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:52:22.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:22.982+0000 7f286d5ae640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28680fd640 0x7f286810d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59826/0 (socket says 172.21.15.28:59826) 2024-07-03T15:52:22.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:22.982+0000 7f286d5ae640 1 -- 172.21.15.28:0/2595812249 learned_addr learned my addr 172.21.15.28:0/2595812249 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:52:22.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:22.982+0000 7f286ddaf640 1 --2- 172.21.15.28:0/2595812249 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28680fc370 0x7f286810cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:52:22.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:22.982+0000 7f286d5ae640 1 -- 172.21.15.28:0/2595812249 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28680fc370 msgr2=0x7f286810cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:52:22.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:22.982+0000 7f286d5ae640 1 --2- 172.21.15.28:0/2595812249 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28680fc370 0x7f286810cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:22.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:22.982+0000 7f286d5ae640 1 -- 172.21.15.28:0/2595812249 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2864005040 con 0x7f28680fd640 2024-07-03T15:52:22.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:22.983+0000 7f286d5ae640 1 --2- 172.21.15.28:0/2595812249 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28680fd640 0x7f286810d510 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f285800b790 tx=0x7f28580091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:52:22.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:22.983+0000 7f285e7fc640 1 -- 172.21.15.28:0/2595812249 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f285800c700 con 0x7f28680fd640 2024-07-03T15:52:22.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:22.983+0000 7f285e7fc640 1 -- 172.21.15.28:0/2595812249 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f285800cd30 con 0x7f28680fd640 2024-07-03T15:52:22.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:22.983+0000 7f285e7fc640 1 -- 172.21.15.28:0/2595812249 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2858013370 con 0x7f28680fd640 2024-07-03T15:52:22.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:22.983+0000 7f2870013640 1 -- 172.21.15.28:0/2595812249 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f286810e760 con 0x7f28680fd640 2024-07-03T15:52:22.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:22.983+0000 7f2870013640 1 -- 172.21.15.28:0/2595812249 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f286810ec30 con 0x7f28680fd640 2024-07-03T15:52:22.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:22.984+0000 7f2870013640 1 -- 172.21.15.28:0/2595812249 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28680fb7d0 con 0x7f28680fd640 2024-07-03T15:52:22.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:22.985+0000 7f285e7fc640 1 -- 172.21.15.28:0/2595812249 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2858012070 con 0x7f28680fd640 2024-07-03T15:52:22.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:22.985+0000 7f285e7fc640 1 --2- 172.21.15.28:0/2595812249 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f285406e0b0 0x7f2854070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:52:22.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:22.985+0000 7f285e7fc640 1 -- 172.21.15.28:0/2595812249 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f285808d530 con 0x7f28680fd640 2024-07-03T15:52:22.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:22.985+0000 7f286ddaf640 1 --2- 172.21.15.28:0/2595812249 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f285406e0b0 0x7f2854070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:52:22.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:22.986+0000 7f286ddaf640 1 --2- 172.21.15.28:0/2595812249 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f285406e0b0 0x7f2854070560 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2864000c00 tx=0x7f2864035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:52:22.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:22.990+0000 7f285e7fc640 1 -- 172.21.15.28:0/2595812249 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f2858056800 con 0x7f28680fd640 2024-07-03T15:52:23.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.209+0000 7f2870013640 1 -- 172.21.15.28:0/2595812249 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f286810ef00 con 0x7f285406e0b0 2024-07-03T15:52:23.214 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:22 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:52:23.214 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:22 smithi028 ceph-mon[92858]: pgmap v217: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:52:23.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.210+0000 7f285e7fc640 1 -- 172.21.15.28:0/2595812249 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+440 (secure 0 0 0) 0x7f286810ef00 con 0x7f285406e0b0 2024-07-03T15:52:23.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.212+0000 7f2870013640 1 -- 172.21.15.28:0/2595812249 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f285406e0b0 msgr2=0x7f2854070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:52:23.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.212+0000 7f2870013640 1 --2- 172.21.15.28:0/2595812249 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f285406e0b0 0x7f2854070560 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2864000c00 tx=0x7f2864035040 comp rx=0 tx=0).stop 2024-07-03T15:52:23.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.212+0000 7f2870013640 1 -- 172.21.15.28:0/2595812249 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28680fd640 msgr2=0x7f286810d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:52:23.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.212+0000 7f2870013640 1 --2- 172.21.15.28:0/2595812249 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28680fd640 0x7f286810d510 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f285800b790 tx=0x7f28580091d0 comp rx=0 tx=0).stop 2024-07-03T15:52:23.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.212+0000 7f2870013640 1 -- 172.21.15.28:0/2595812249 shutdown_connections 2024-07-03T15:52:23.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.212+0000 7f2870013640 1 --2- 172.21.15.28:0/2595812249 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f285406e0b0 0x7f2854070560 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:23.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.212+0000 7f2870013640 1 --2- 172.21.15.28:0/2595812249 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28680fd640 0x7f286810d510 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:23.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.212+0000 7f2870013640 1 --2- 172.21.15.28:0/2595812249 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28680fc370 0x7f286810cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:23.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.212+0000 7f2870013640 1 -- 172.21.15.28:0/2595812249 >> 172.21.15.28:0/2595812249 conn(0x7f28680f7b50 msgr2=0x7f28680f9180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:52:23.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.212+0000 7f2870013640 1 -- 172.21.15.28:0/2595812249 shutdown_connections 2024-07-03T15:52:23.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.212+0000 7f2870013640 1 -- 172.21.15.28:0/2595812249 wait complete. 2024-07-03T15:52:23.225 INFO:teuthology.orchestra.run.smithi028.stdout:true 2024-07-03T15:52:23.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:22 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:52:23.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:22 smithi148 ceph-mon[80490]: pgmap v217: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:52:23.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.295+0000 7f7bee870640 1 -- 172.21.15.28:0/3858583072 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7be80fd640 msgr2=0x7f7be80fdab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:52:23.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.295+0000 7f7bee870640 1 --2- 172.21.15.28:0/3858583072 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7be80fd640 0x7f7be80fdab0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7bd00048d0 tx=0x7f7bd0009da0 comp rx=0 tx=0).stop 2024-07-03T15:52:23.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.295+0000 7f7bee870640 1 -- 172.21.15.28:0/3858583072 shutdown_connections 2024-07-03T15:52:23.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.295+0000 7f7bee870640 1 --2- 172.21.15.28:0/3858583072 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7be80fd640 0x7f7be80fdab0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:23.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.295+0000 7f7bee870640 1 --2- 172.21.15.28:0/3858583072 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7be80fc370 0x7f7be80fc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:23.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.295+0000 7f7bee870640 1 -- 172.21.15.28:0/3858583072 >> 172.21.15.28:0/3858583072 conn(0x7f7be80f7b50 msgr2=0x7f7be80f9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:52:23.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.295+0000 7f7bee870640 1 -- 172.21.15.28:0/3858583072 shutdown_connections 2024-07-03T15:52:23.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.295+0000 7f7bee870640 1 -- 172.21.15.28:0/3858583072 wait complete. 2024-07-03T15:52:23.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.296+0000 7f7bee870640 1 Processor -- start 2024-07-03T15:52:23.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.296+0000 7f7bee870640 1 -- start start 2024-07-03T15:52:23.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.296+0000 7f7bee870640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7be80fc370 0x7f7be810cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:52:23.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.296+0000 7f7bee870640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7be80fd640 0x7f7be810d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:52:23.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.296+0000 7f7bee870640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be810daf0 con 0x7f7be80fd640 2024-07-03T15:52:23.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.296+0000 7f7bee870640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be810e2a0 con 0x7f7be80fc370 2024-07-03T15:52:23.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.296+0000 7f7be77fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7be80fd640 0x7f7be810d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:52:23.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.296+0000 7f7be77fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7be80fd640 0x7f7be810d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59838/0 (socket says 172.21.15.28:59838) 2024-07-03T15:52:23.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.296+0000 7f7be77fe640 1 -- 172.21.15.28:0/2673994979 learned_addr learned my addr 172.21.15.28:0/2673994979 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:52:23.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.296+0000 7f7be7fff640 1 --2- 172.21.15.28:0/2673994979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7be80fc370 0x7f7be810cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:52:23.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.297+0000 7f7be77fe640 1 -- 172.21.15.28:0/2673994979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7be80fc370 msgr2=0x7f7be810cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:52:23.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.297+0000 7f7be77fe640 1 --2- 172.21.15.28:0/2673994979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7be80fc370 0x7f7be810cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:23.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.297+0000 7f7be77fe640 1 -- 172.21.15.28:0/2673994979 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bd0005040 con 0x7f7be80fd640 2024-07-03T15:52:23.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.297+0000 7f7be77fe640 1 --2- 172.21.15.28:0/2673994979 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7be80fd640 0x7f7be810d4a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7bd00048a0 tx=0x7f7bd0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:52:23.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.297+0000 7f7be4ff9640 1 -- 172.21.15.28:0/2673994979 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bd0038070 con 0x7f7be80fd640 2024-07-03T15:52:23.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.297+0000 7f7be4ff9640 1 -- 172.21.15.28:0/2673994979 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7bd00092b0 con 0x7f7be80fd640 2024-07-03T15:52:23.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.297+0000 7f7bee870640 1 -- 172.21.15.28:0/2673994979 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7be810e4d0 con 0x7f7be80fd640 2024-07-03T15:52:23.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.297+0000 7f7be4ff9640 1 -- 172.21.15.28:0/2673994979 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bd00338a0 con 0x7f7be80fd640 2024-07-03T15:52:23.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.297+0000 7f7bee870640 1 -- 172.21.15.28:0/2673994979 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7be810ea10 con 0x7f7be80fd640 2024-07-03T15:52:23.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.298+0000 7f7bee870640 1 -- 172.21.15.28:0/2673994979 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7be80fb7d0 con 0x7f7be80fd640 2024-07-03T15:52:23.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.299+0000 7f7be4ff9640 1 -- 172.21.15.28:0/2673994979 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7bd0030020 con 0x7f7be80fd640 2024-07-03T15:52:23.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.299+0000 7f7be4ff9640 1 --2- 172.21.15.28:0/2673994979 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f7bd406e0b0 0x7f7bd4070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:52:23.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.299+0000 7f7be4ff9640 1 -- 172.21.15.28:0/2673994979 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f7bd00ad920 con 0x7f7be80fd640 2024-07-03T15:52:23.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.299+0000 7f7be7fff640 1 --2- 172.21.15.28:0/2673994979 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f7bd406e0b0 0x7f7bd4070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:52:23.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.300+0000 7f7be7fff640 1 --2- 172.21.15.28:0/2673994979 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f7bd406e0b0 0x7f7bd4070560 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8004ed0 tx=0x7f7bd800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:52:23.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.304+0000 7f7be4ff9640 1 -- 172.21.15.28:0/2673994979 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f7bd0076d20 con 0x7f7be80fd640 2024-07-03T15:52:23.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.521+0000 7f7bee870640 1 -- 172.21.15.28:0/2673994979 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7be810edb0 con 0x7f7bd406e0b0 2024-07-03T15:52:23.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.523+0000 7f7be4ff9640 1 -- 172.21.15.28:0/2673994979 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+440 (secure 0 0 0) 0x7f7be810edb0 con 0x7f7bd406e0b0 2024-07-03T15:52:23.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.525+0000 7f7bee870640 1 -- 172.21.15.28:0/2673994979 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f7bd406e0b0 msgr2=0x7f7bd4070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:52:23.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.525+0000 7f7bee870640 1 --2- 172.21.15.28:0/2673994979 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f7bd406e0b0 0x7f7bd4070560 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8004ed0 tx=0x7f7bd800b040 comp rx=0 tx=0).stop 2024-07-03T15:52:23.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.525+0000 7f7bee870640 1 -- 172.21.15.28:0/2673994979 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7be80fd640 msgr2=0x7f7be810d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:52:23.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.525+0000 7f7bee870640 1 --2- 172.21.15.28:0/2673994979 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7be80fd640 0x7f7be810d4a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7bd00048a0 tx=0x7f7bd0009da0 comp rx=0 tx=0).stop 2024-07-03T15:52:23.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.525+0000 7f7bee870640 1 -- 172.21.15.28:0/2673994979 shutdown_connections 2024-07-03T15:52:23.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.525+0000 7f7bee870640 1 --2- 172.21.15.28:0/2673994979 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f7bd406e0b0 0x7f7bd4070560 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:23.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.525+0000 7f7bee870640 1 --2- 172.21.15.28:0/2673994979 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7be80fd640 0x7f7be810d4a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:23.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.525+0000 7f7bee870640 1 --2- 172.21.15.28:0/2673994979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7be80fc370 0x7f7be810cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:23.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.525+0000 7f7bee870640 1 -- 172.21.15.28:0/2673994979 >> 172.21.15.28:0/2673994979 conn(0x7f7be80f7b50 msgr2=0x7f7be80f91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:52:23.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.525+0000 7f7bee870640 1 -- 172.21.15.28:0/2673994979 shutdown_connections 2024-07-03T15:52:23.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.525+0000 7f7bee870640 1 -- 172.21.15.28:0/2673994979 wait complete. 2024-07-03T15:52:23.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.607+0000 7fa3b3a6f640 1 -- 172.21.15.28:0/194973768 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3ac0fc330 msgr2=0x7fa3ac0fc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:52:23.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.607+0000 7fa3b3a6f640 1 --2- 172.21.15.28:0/194973768 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3ac0fc330 0x7fa3ac0fc740 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa3a80048d0 tx=0x7fa3a8009da0 comp rx=0 tx=0).stop 2024-07-03T15:52:23.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.607+0000 7fa3b3a6f640 1 -- 172.21.15.28:0/194973768 shutdown_connections 2024-07-03T15:52:23.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.607+0000 7fa3b3a6f640 1 --2- 172.21.15.28:0/194973768 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3ac0fd600 0x7fa3ac0fda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:23.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.607+0000 7fa3b3a6f640 1 --2- 172.21.15.28:0/194973768 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3ac0fc330 0x7fa3ac0fc740 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:23.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.607+0000 7fa3b3a6f640 1 -- 172.21.15.28:0/194973768 >> 172.21.15.28:0/194973768 conn(0x7fa3ac0f7af0 msgr2=0x7fa3ac0f9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:52:23.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.608+0000 7fa3b3a6f640 1 -- 172.21.15.28:0/194973768 shutdown_connections 2024-07-03T15:52:23.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.608+0000 7fa3b3a6f640 1 -- 172.21.15.28:0/194973768 wait complete. 2024-07-03T15:52:23.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.608+0000 7fa3b3a6f640 1 Processor -- start 2024-07-03T15:52:23.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.608+0000 7fa3b3a6f640 1 -- start start 2024-07-03T15:52:23.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.608+0000 7fa3b3a6f640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3ac0fc330 0x7fa3ac10cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:52:23.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.608+0000 7fa3b3a6f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3ac0fd600 0x7fa3ac10d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:52:23.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.608+0000 7fa3b3a6f640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac10dad0 con 0x7fa3ac0fc330 2024-07-03T15:52:23.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.608+0000 7fa3b3a6f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac10e2f0 con 0x7fa3ac0fd600 2024-07-03T15:52:23.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.608+0000 7fa3b180b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3ac0fc330 0x7fa3ac10cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:52:23.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.608+0000 7fa3b180b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3ac0fc330 0x7fa3ac10cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59858/0 (socket says 172.21.15.28:59858) 2024-07-03T15:52:23.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.608+0000 7fa3b180b640 1 -- 172.21.15.28:0/2762651566 learned_addr learned my addr 172.21.15.28:0/2762651566 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:52:23.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.609+0000 7fa3b100a640 1 --2- 172.21.15.28:0/2762651566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3ac0fd600 0x7fa3ac10d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:52:23.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.609+0000 7fa3b180b640 1 -- 172.21.15.28:0/2762651566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3ac0fd600 msgr2=0x7fa3ac10d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:52:23.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.609+0000 7fa3b180b640 1 --2- 172.21.15.28:0/2762651566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3ac0fd600 0x7fa3ac10d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:23.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.609+0000 7fa3b180b640 1 -- 172.21.15.28:0/2762651566 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3a8005040 con 0x7fa3ac0fc330 2024-07-03T15:52:23.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.609+0000 7fa3b180b640 1 --2- 172.21.15.28:0/2762651566 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3ac0fc330 0x7fa3ac10cf40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa3a80048a0 tx=0x7fa3a8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:52:23.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.609+0000 7fa3a27fc640 1 -- 172.21.15.28:0/2762651566 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3a8038070 con 0x7fa3ac0fc330 2024-07-03T15:52:23.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.609+0000 7fa3a27fc640 1 -- 172.21.15.28:0/2762651566 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa3a8032970 con 0x7fa3ac0fc330 2024-07-03T15:52:23.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.609+0000 7fa3b3a6f640 1 -- 172.21.15.28:0/2762651566 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3ac10e520 con 0x7fa3ac0fc330 2024-07-03T15:52:23.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.609+0000 7fa3a27fc640 1 -- 172.21.15.28:0/2762651566 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3a803d9a0 con 0x7fa3ac0fc330 2024-07-03T15:52:23.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.609+0000 7fa3b3a6f640 1 -- 172.21.15.28:0/2762651566 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3ac10ea60 con 0x7fa3ac0fc330 2024-07-03T15:52:23.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.610+0000 7fa3b3a6f640 1 -- 172.21.15.28:0/2762651566 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3ac0fb790 con 0x7fa3ac0fc330 2024-07-03T15:52:23.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.611+0000 7fa3a27fc640 1 -- 172.21.15.28:0/2762651566 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa3a8030020 con 0x7fa3ac0fc330 2024-07-03T15:52:23.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.611+0000 7fa3a27fc640 1 --2- 172.21.15.28:0/2762651566 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fa39806e0b0 0x7fa398070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:52:23.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.611+0000 7fa3a27fc640 1 -- 172.21.15.28:0/2762651566 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fa3a80ae3a0 con 0x7fa3ac0fc330 2024-07-03T15:52:23.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.611+0000 7fa3b100a640 1 --2- 172.21.15.28:0/2762651566 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fa39806e0b0 0x7fa398070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:52:23.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.612+0000 7fa3b100a640 1 --2- 172.21.15.28:0/2762651566 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fa39806e0b0 0x7fa398070560 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac10ccf0 tx=0x7fa39c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:52:23.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.616+0000 7fa3a27fc640 1 -- 172.21.15.28:0/2762651566 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fa3a8077690 con 0x7fa3ac0fc330 2024-07-03T15:52:23.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.811+0000 7fa3b3a6f640 1 -- 172.21.15.28:0/2762651566 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa3ac10ed90 con 0x7fa39806e0b0 2024-07-03T15:52:23.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.822+0000 7fa3a27fc640 1 -- 172.21.15.28:0/2762651566 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fa3ac10ed90 con 0x7fa39806e0b0 2024-07-03T15:52:23.826 INFO:teuthology.orchestra.run.smithi028.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-03T15:52:23.826 INFO:teuthology.orchestra.run.smithi028.stdout:alertmanager.smithi028 smithi028 *:9093,9094 starting - - - - 2024-07-03T15:52:23.826 INFO:teuthology.orchestra.run.smithi028.stdout:crash.smithi028 smithi028 running (5m) 14s ago 16m 6895k - 19.0.0-2550-gc00506dc 9d49405603e8 c62c2f45cabe 2024-07-03T15:52:23.826 INFO:teuthology.orchestra.run.smithi028.stdout:crash.smithi148 smithi148 running (5m) 35s ago 15m 6895k - 19.0.0-2550-gc00506dc 9d49405603e8 b1318dcaacb6 2024-07-03T15:52:23.827 INFO:teuthology.orchestra.run.smithi028.stdout:grafana.smithi028 smithi028 *:3000 running (7m) 14s ago 15m 84.3M - 9.4.7 954c08fa6188 f9f17c92cfea 2024-07-03T15:52:23.827 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi028.rhpvve smithi028 running (105s) 14s ago 10m 84.4M - 19.0.0-2550-gc00506dc 9d49405603e8 5d528ea2a2f0 2024-07-03T15:52:23.827 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi028.wmbxew smithi028 running (89s) 14s ago 10m 82.0M - 19.0.0-2550-gc00506dc 9d49405603e8 ebdbfa592ac8 2024-07-03T15:52:23.827 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi148.szomfl smithi148 running (55s) 35s ago 10m 13.0M - 19.0.0-2550-gc00506dc 9d49405603e8 27650f9fb39f 2024-07-03T15:52:23.827 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi148.xgpuld smithi148 running (72s) 35s ago 10m 14.6M - 19.0.0-2550-gc00506dc 9d49405603e8 f24ab04507dc 2024-07-03T15:52:23.827 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.smithi028.vtqrtz smithi028 *:8443,9283,8765 running (7m) 14s ago 17m 508M - 19.0.0-2550-gc00506dc 9d49405603e8 14d0363c5b77 2024-07-03T15:52:23.827 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.smithi148.rjzsar smithi148 *:8443,9283,8765 running (6m) 35s ago 15m 441M - 19.0.0-2550-gc00506dc 9d49405603e8 c0193aa8a5dd 2024-07-03T15:52:23.827 INFO:teuthology.orchestra.run.smithi028.stdout:mon.smithi028 smithi028 running (6m) 14s ago 17m 52.5M 2048M 19.0.0-2550-gc00506dc 9d49405603e8 5093f7bf47d7 2024-07-03T15:52:23.827 INFO:teuthology.orchestra.run.smithi028.stdout:mon.smithi148 smithi148 running (5m) 35s ago 14m 38.2M 2048M 19.0.0-2550-gc00506dc 9d49405603e8 d338d2c648da 2024-07-03T15:52:23.827 INFO:teuthology.orchestra.run.smithi028.stdout:node-exporter.smithi028 smithi028 *:9100 running (42s) 14s ago 16m 8396k - 1.5.0 0da6a335fe13 125dcba51db4 2024-07-03T15:52:23.827 INFO:teuthology.orchestra.run.smithi028.stdout:node-exporter.smithi148 smithi148 *:9100 running (37s) 35s ago 15m 3577k - 1.5.0 0da6a335fe13 edc710a27078 2024-07-03T15:52:23.827 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0 smithi028 running (4m) 14s ago 14m 111M 1558M 19.0.0-2550-gc00506dc 9d49405603e8 4454cecff8fc 2024-07-03T15:52:23.827 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1 smithi028 running (4m) 14s ago 13m 117M 1558M 19.0.0-2550-gc00506dc 9d49405603e8 af67f7088643 2024-07-03T15:52:23.827 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2 smithi028 running (3m) 14s ago 13m 100M 1558M 19.0.0-2550-gc00506dc 9d49405603e8 5d160a327dc2 2024-07-03T15:52:23.827 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3 smithi148 running (3m) 35s ago 13m 120M 2581M 19.0.0-2550-gc00506dc 9d49405603e8 1d622eba4b3b 2024-07-03T15:52:23.828 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4 smithi148 running (2m) 35s ago 12m 106M 2581M 19.0.0-2550-gc00506dc 9d49405603e8 546343d0f31f 2024-07-03T15:52:23.828 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5 smithi148 running (2m) 35s ago 12m 106M 2581M 19.0.0-2550-gc00506dc 9d49405603e8 a819fcc3b569 2024-07-03T15:52:23.828 INFO:teuthology.orchestra.run.smithi028.stdout:prometheus.smithi028 smithi028 *:9095 running (19s) 14s ago 15m 41.8M - 2.43.0 a07b618ecd1d 406cff811726 2024-07-03T15:52:23.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.824+0000 7fa3b3a6f640 1 -- 172.21.15.28:0/2762651566 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fa39806e0b0 msgr2=0x7fa398070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:52:23.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.824+0000 7fa3b3a6f640 1 --2- 172.21.15.28:0/2762651566 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fa39806e0b0 0x7fa398070560 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac10ccf0 tx=0x7fa39c009040 comp rx=0 tx=0).stop 2024-07-03T15:52:23.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.824+0000 7fa3b3a6f640 1 -- 172.21.15.28:0/2762651566 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3ac0fc330 msgr2=0x7fa3ac10cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:52:23.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.824+0000 7fa3b3a6f640 1 --2- 172.21.15.28:0/2762651566 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3ac0fc330 0x7fa3ac10cf40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa3a80048a0 tx=0x7fa3a8009da0 comp rx=0 tx=0).stop 2024-07-03T15:52:23.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.824+0000 7fa3b3a6f640 1 -- 172.21.15.28:0/2762651566 shutdown_connections 2024-07-03T15:52:23.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.824+0000 7fa3b3a6f640 1 --2- 172.21.15.28:0/2762651566 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fa39806e0b0 0x7fa398070560 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:23.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.824+0000 7fa3b3a6f640 1 --2- 172.21.15.28:0/2762651566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3ac0fd600 0x7fa3ac10d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:23.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.824+0000 7fa3b3a6f640 1 --2- 172.21.15.28:0/2762651566 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3ac0fc330 0x7fa3ac10cf40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:23.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.824+0000 7fa3b3a6f640 1 -- 172.21.15.28:0/2762651566 >> 172.21.15.28:0/2762651566 conn(0x7fa3ac0f7af0 msgr2=0x7fa3ac0f91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:52:23.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.824+0000 7fa3b3a6f640 1 -- 172.21.15.28:0/2762651566 shutdown_connections 2024-07-03T15:52:23.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.824+0000 7fa3b3a6f640 1 -- 172.21.15.28:0/2762651566 wait complete. 2024-07-03T15:52:23.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.910+0000 7f136f1f8640 1 -- 172.21.15.28:0/2335836793 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13680fc510 msgr2=0x7f13680fc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:52:23.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.910+0000 7f136f1f8640 1 --2- 172.21.15.28:0/2335836793 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13680fc510 0x7f13680fc980 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f13540048d0 tx=0x7f1354009da0 comp rx=0 tx=0).stop 2024-07-03T15:52:23.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.910+0000 7f136f1f8640 1 -- 172.21.15.28:0/2335836793 shutdown_connections 2024-07-03T15:52:23.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.910+0000 7f136f1f8640 1 --2- 172.21.15.28:0/2335836793 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13680fc510 0x7f13680fc980 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:23.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.910+0000 7f136f1f8640 1 --2- 172.21.15.28:0/2335836793 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13680fb240 0x7f13680fb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:23.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.910+0000 7f136f1f8640 1 -- 172.21.15.28:0/2335836793 >> 172.21.15.28:0/2335836793 conn(0x7f13680f6a20 msgr2=0x7f13680f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:52:23.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.910+0000 7f136f1f8640 1 -- 172.21.15.28:0/2335836793 shutdown_connections 2024-07-03T15:52:23.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.910+0000 7f136f1f8640 1 -- 172.21.15.28:0/2335836793 wait complete. 2024-07-03T15:52:23.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.911+0000 7f136f1f8640 1 Processor -- start 2024-07-03T15:52:23.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.911+0000 7f136f1f8640 1 -- start start 2024-07-03T15:52:23.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.911+0000 7f136f1f8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13680fb240 0x7f13680ff310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:52:23.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.911+0000 7f136f1f8640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13680fc510 0x7f13680fd850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:52:23.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.911+0000 7f136f1f8640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13680fdd90 con 0x7f13680fc510 2024-07-03T15:52:23.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.911+0000 7f136f1f8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13680fded0 con 0x7f13680fb240 2024-07-03T15:52:23.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.911+0000 7f1367fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13680fc510 0x7f13680fd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:52:23.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.911+0000 7f1367fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13680fc510 0x7f13680fd850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59874/0 (socket says 172.21.15.28:59874) 2024-07-03T15:52:23.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.911+0000 7f1367fff640 1 -- 172.21.15.28:0/554307608 learned_addr learned my addr 172.21.15.28:0/554307608 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:52:23.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.912+0000 7f136cf94640 1 --2- 172.21.15.28:0/554307608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13680fb240 0x7f13680ff310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:52:23.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.912+0000 7f1367fff640 1 -- 172.21.15.28:0/554307608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13680fb240 msgr2=0x7f13680ff310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:52:23.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.912+0000 7f1367fff640 1 --2- 172.21.15.28:0/554307608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13680fb240 0x7f13680ff310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:23.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.912+0000 7f1367fff640 1 -- 172.21.15.28:0/554307608 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1354005040 con 0x7f13680fc510 2024-07-03T15:52:23.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.912+0000 7f136cf94640 1 --2- 172.21.15.28:0/554307608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13680fb240 0x7f13680ff310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-03T15:52:23.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.912+0000 7f1367fff640 1 --2- 172.21.15.28:0/554307608 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13680fc510 0x7f13680fd850 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1354000c00 tx=0x7f1354009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:52:23.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.912+0000 7f13657fa640 1 -- 172.21.15.28:0/554307608 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1354038070 con 0x7f13680fc510 2024-07-03T15:52:23.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.912+0000 7f136f1f8640 1 -- 172.21.15.28:0/554307608 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13680fe150 con 0x7f13680fc510 2024-07-03T15:52:23.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.912+0000 7f13657fa640 1 -- 172.21.15.28:0/554307608 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1354032b50 con 0x7f13680fc510 2024-07-03T15:52:23.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.912+0000 7f13657fa640 1 -- 172.21.15.28:0/554307608 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1354033b50 con 0x7f13680fc510 2024-07-03T15:52:23.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.912+0000 7f136f1f8640 1 -- 172.21.15.28:0/554307608 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13680fe610 con 0x7f13680fc510 2024-07-03T15:52:23.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.913+0000 7f136f1f8640 1 -- 172.21.15.28:0/554307608 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13680fa6a0 con 0x7f13680fc510 2024-07-03T15:52:23.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.914+0000 7f13657fa640 1 -- 172.21.15.28:0/554307608 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1354030020 con 0x7f13680fc510 2024-07-03T15:52:23.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.915+0000 7f13657fa640 1 --2- 172.21.15.28:0/554307608 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f135806e0b0 0x7f1358070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:52:23.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.915+0000 7f13657fa640 1 -- 172.21.15.28:0/554307608 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f13540adda0 con 0x7f13680fc510 2024-07-03T15:52:23.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.915+0000 7f136cf94640 1 --2- 172.21.15.28:0/554307608 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f135806e0b0 0x7f1358070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:52:23.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.915+0000 7f136cf94640 1 --2- 172.21.15.28:0/554307608 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f135806e0b0 0x7f1358070560 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f135c009fd0 tx=0x7f135c009e90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:52:23.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:23.919+0000 7f13657fa640 1 -- 172.21.15.28:0/554307608 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f1354077050 con 0x7f13680fc510 2024-07-03T15:52:24.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:23 smithi028 ceph-mon[92858]: from='client.34474 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:52:24.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:23 smithi028 ceph-mon[92858]: from='client.34478 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:52:24.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:23 smithi028 ceph-mon[92858]: from='client.34482 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:52:24.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.193+0000 7f136f1f8640 1 -- 172.21.15.28:0/554307608 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f136804b530 con 0x7f13680fc510 2024-07-03T15:52:24.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.194+0000 7f13657fa640 1 -- 172.21.15.28:0/554307608 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7f135407a710 con 0x7f13680fc510 2024-07-03T15:52:24.199 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-07-03T15:52:24.199 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": { 2024-07-03T15:52:24.199 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 2 2024-07-03T15:52:24.199 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:52:24.199 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": { 2024-07-03T15:52:24.199 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 2 2024-07-03T15:52:24.199 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:52:24.199 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": { 2024-07-03T15:52:24.199 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 6 2024-07-03T15:52:24.199 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:52:24.199 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": { 2024-07-03T15:52:24.199 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 4 2024-07-03T15:52:24.200 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:52:24.200 INFO:teuthology.orchestra.run.smithi028.stdout: "overall": { 2024-07-03T15:52:24.200 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 14 2024-07-03T15:52:24.200 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-07-03T15:52:24.200 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-07-03T15:52:24.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.196+0000 7f136f1f8640 1 -- 172.21.15.28:0/554307608 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f135806e0b0 msgr2=0x7f1358070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:52:24.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.196+0000 7f136f1f8640 1 --2- 172.21.15.28:0/554307608 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f135806e0b0 0x7f1358070560 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f135c009fd0 tx=0x7f135c009e90 comp rx=0 tx=0).stop 2024-07-03T15:52:24.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.196+0000 7f136f1f8640 1 -- 172.21.15.28:0/554307608 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13680fc510 msgr2=0x7f13680fd850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:52:24.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.196+0000 7f136f1f8640 1 --2- 172.21.15.28:0/554307608 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13680fc510 0x7f13680fd850 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1354000c00 tx=0x7f1354009da0 comp rx=0 tx=0).stop 2024-07-03T15:52:24.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.196+0000 7f136f1f8640 1 -- 172.21.15.28:0/554307608 shutdown_connections 2024-07-03T15:52:24.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.196+0000 7f136f1f8640 1 --2- 172.21.15.28:0/554307608 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f135806e0b0 0x7f1358070560 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:24.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.196+0000 7f136f1f8640 1 --2- 172.21.15.28:0/554307608 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13680fc510 0x7f13680fd850 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:24.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.196+0000 7f136f1f8640 1 --2- 172.21.15.28:0/554307608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13680fb240 0x7f13680ff310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:24.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.196+0000 7f136f1f8640 1 -- 172.21.15.28:0/554307608 >> 172.21.15.28:0/554307608 conn(0x7f13680f6a20 msgr2=0x7f13680651d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:52:24.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.196+0000 7f136f1f8640 1 -- 172.21.15.28:0/554307608 shutdown_connections 2024-07-03T15:52:24.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.196+0000 7f136f1f8640 1 -- 172.21.15.28:0/554307608 wait complete. 2024-07-03T15:52:24.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:23 smithi148 ceph-mon[80490]: from='client.34474 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:52:24.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:23 smithi148 ceph-mon[80490]: from='client.34478 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:52:24.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:23 smithi148 ceph-mon[80490]: from='client.34482 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:52:24.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.279+0000 7fcc8a812640 1 -- 172.21.15.28:0/1888605351 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcc840fd600 msgr2=0x7fcc840fda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:52:24.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.279+0000 7fcc8a812640 1 --2- 172.21.15.28:0/1888605351 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcc840fd600 0x7fcc840fda70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcc780048d0 tx=0x7fcc78009da0 comp rx=0 tx=0).stop 2024-07-03T15:52:24.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.279+0000 7fcc8a812640 1 -- 172.21.15.28:0/1888605351 shutdown_connections 2024-07-03T15:52:24.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.279+0000 7fcc8a812640 1 --2- 172.21.15.28:0/1888605351 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcc840fd600 0x7fcc840fda70 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:24.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.279+0000 7fcc8a812640 1 --2- 172.21.15.28:0/1888605351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc840fc330 0x7fcc840fc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:24.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.279+0000 7fcc8a812640 1 -- 172.21.15.28:0/1888605351 >> 172.21.15.28:0/1888605351 conn(0x7fcc840f7af0 msgr2=0x7fcc840f9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:52:24.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.280+0000 7fcc8a812640 1 -- 172.21.15.28:0/1888605351 shutdown_connections 2024-07-03T15:52:24.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.280+0000 7fcc8a812640 1 -- 172.21.15.28:0/1888605351 wait complete. 2024-07-03T15:52:24.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.280+0000 7fcc8a812640 1 Processor -- start 2024-07-03T15:52:24.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.280+0000 7fcc8a812640 1 -- start start 2024-07-03T15:52:24.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.280+0000 7fcc8a812640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcc840fc330 0x7fcc84071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:52:24.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.280+0000 7fcc8a812640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc840fd600 0x7fcc8406f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:52:24.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.280+0000 7fcc8a812640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8406fe00 con 0x7fcc840fc330 2024-07-03T15:52:24.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.280+0000 7fcc8a812640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8406ff40 con 0x7fcc840fd600 2024-07-03T15:52:24.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.281+0000 7fcc83fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcc840fc330 0x7fcc84071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:52:24.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.281+0000 7fcc83fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcc840fc330 0x7fcc84071380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59888/0 (socket says 172.21.15.28:59888) 2024-07-03T15:52:24.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.281+0000 7fcc83fff640 1 -- 172.21.15.28:0/2594132096 learned_addr learned my addr 172.21.15.28:0/2594132096 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:52:24.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.281+0000 7fcc837fe640 1 --2- 172.21.15.28:0/2594132096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc840fd600 0x7fcc8406f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:52:24.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.281+0000 7fcc83fff640 1 -- 172.21.15.28:0/2594132096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc840fd600 msgr2=0x7fcc8406f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:52:24.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.281+0000 7fcc83fff640 1 --2- 172.21.15.28:0/2594132096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc840fd600 0x7fcc8406f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:24.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.281+0000 7fcc83fff640 1 -- 172.21.15.28:0/2594132096 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc78005040 con 0x7fcc840fc330 2024-07-03T15:52:24.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.281+0000 7fcc83fff640 1 --2- 172.21.15.28:0/2594132096 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcc840fc330 0x7fcc84071380 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcc74002a50 tx=0x7fcc74004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:52:24.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.281+0000 7fcc80ff9640 1 -- 172.21.15.28:0/2594132096 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc74007c30 con 0x7fcc840fc330 2024-07-03T15:52:24.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.281+0000 7fcc80ff9640 1 -- 172.21.15.28:0/2594132096 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcc7400f040 con 0x7fcc840fc330 2024-07-03T15:52:24.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.281+0000 7fcc80ff9640 1 -- 172.21.15.28:0/2594132096 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc7400dad0 con 0x7fcc840fc330 2024-07-03T15:52:24.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.281+0000 7fcc8a812640 1 -- 172.21.15.28:0/2594132096 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc84070220 con 0x7fcc840fc330 2024-07-03T15:52:24.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.281+0000 7fcc8a812640 1 -- 172.21.15.28:0/2594132096 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc84114fa0 con 0x7fcc840fc330 2024-07-03T15:52:24.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.283+0000 7fcc8a812640 1 -- 172.21.15.28:0/2594132096 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc8404b530 con 0x7fcc840fc330 2024-07-03T15:52:24.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.283+0000 7fcc80ff9640 1 -- 172.21.15.28:0/2594132096 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fcc7400dc70 con 0x7fcc840fc330 2024-07-03T15:52:24.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.284+0000 7fcc80ff9640 1 --2- 172.21.15.28:0/2594132096 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fcc6c06e0b0 0x7fcc6c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:52:24.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.284+0000 7fcc80ff9640 1 -- 172.21.15.28:0/2594132096 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fcc7408d320 con 0x7fcc840fc330 2024-07-03T15:52:24.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.284+0000 7fcc837fe640 1 --2- 172.21.15.28:0/2594132096 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fcc6c06e0b0 0x7fcc6c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:52:24.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.284+0000 7fcc837fe640 1 --2- 172.21.15.28:0/2594132096 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fcc6c06e0b0 0x7fcc6c070560 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcc780043d0 tx=0x7fcc78035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:52:24.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.288+0000 7fcc80ff9640 1 -- 172.21.15.28:0/2594132096 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fcc74056720 con 0x7fcc840fc330 2024-07-03T15:52:24.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.520+0000 7fcc8a812640 1 -- 172.21.15.28:0/2594132096 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fcc84115250 con 0x7fcc840fc330 2024-07-03T15:52:24.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.521+0000 7fcc80ff9640 1 -- 172.21.15.28:0/2594132096 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1967 (secure 0 0 0) 0x7fcc74059de0 con 0x7fcc840fc330 2024-07-03T15:52:24.526 INFO:teuthology.orchestra.run.smithi028.stdout:e35 2024-07-03T15:52:24.526 INFO:teuthology.orchestra.run.smithi028.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-03T15:52:24.526 INFO:teuthology.orchestra.run.smithi028.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:52:24.526 INFO:teuthology.orchestra.run.smithi028.stdout:legacy client fscid: 1 2024-07-03T15:52:24.526 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:52:24.526 INFO:teuthology.orchestra.run.smithi028.stdout:Filesystem 'cephfs' (1) 2024-07-03T15:52:24.526 INFO:teuthology.orchestra.run.smithi028.stdout:fs_name cephfs 2024-07-03T15:52:24.526 INFO:teuthology.orchestra.run.smithi028.stdout:epoch 35 2024-07-03T15:52:24.526 INFO:teuthology.orchestra.run.smithi028.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-03T15:52:24.526 INFO:teuthology.orchestra.run.smithi028.stdout:created 2024-07-03T15:41:40.656298+0000 2024-07-03T15:52:24.526 INFO:teuthology.orchestra.run.smithi028.stdout:modified 2024-07-03T15:51:35.429732+0000 2024-07-03T15:52:24.527 INFO:teuthology.orchestra.run.smithi028.stdout:tableserver 0 2024-07-03T15:52:24.527 INFO:teuthology.orchestra.run.smithi028.stdout:root 0 2024-07-03T15:52:24.527 INFO:teuthology.orchestra.run.smithi028.stdout:session_timeout 60 2024-07-03T15:52:24.527 INFO:teuthology.orchestra.run.smithi028.stdout:session_autoclose 300 2024-07-03T15:52:24.527 INFO:teuthology.orchestra.run.smithi028.stdout:max_file_size 1099511627776 2024-07-03T15:52:24.527 INFO:teuthology.orchestra.run.smithi028.stdout:max_xattr_size 65536 2024-07-03T15:52:24.527 INFO:teuthology.orchestra.run.smithi028.stdout:required_client_features {} 2024-07-03T15:52:24.527 INFO:teuthology.orchestra.run.smithi028.stdout:last_failure 0 2024-07-03T15:52:24.527 INFO:teuthology.orchestra.run.smithi028.stdout:last_failure_osd_epoch 82 2024-07-03T15:52:24.527 INFO:teuthology.orchestra.run.smithi028.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-07-03T15:52:24.527 INFO:teuthology.orchestra.run.smithi028.stdout:max_mds 1 2024-07-03T15:52:24.527 INFO:teuthology.orchestra.run.smithi028.stdout:in 0 2024-07-03T15:52:24.527 INFO:teuthology.orchestra.run.smithi028.stdout:up {0=34376} 2024-07-03T15:52:24.527 INFO:teuthology.orchestra.run.smithi028.stdout:failed 2024-07-03T15:52:24.528 INFO:teuthology.orchestra.run.smithi028.stdout:damaged 2024-07-03T15:52:24.528 INFO:teuthology.orchestra.run.smithi028.stdout:stopped 2024-07-03T15:52:24.528 INFO:teuthology.orchestra.run.smithi028.stdout:data_pools [3] 2024-07-03T15:52:24.528 INFO:teuthology.orchestra.run.smithi028.stdout:metadata_pool 2 2024-07-03T15:52:24.528 INFO:teuthology.orchestra.run.smithi028.stdout:inline_data disabled 2024-07-03T15:52:24.528 INFO:teuthology.orchestra.run.smithi028.stdout:balancer 2024-07-03T15:52:24.528 INFO:teuthology.orchestra.run.smithi028.stdout:bal_rank_mask -1 2024-07-03T15:52:24.528 INFO:teuthology.orchestra.run.smithi028.stdout:standby_count_wanted 1 2024-07-03T15:52:24.528 INFO:teuthology.orchestra.run.smithi028.stdout:qdb_cluster leader: 34376 members: 34376 2024-07-03T15:52:24.528 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi028.rhpvve{0:34376} state up:active seq 16 join_fscid=1 addr [v2:172.21.15.28:6826/3127260162,v1:172.21.15.28:6827/3127260162] compat {c=[1],r=[1],i=[1fff]}] 2024-07-03T15:52:24.528 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi028.wmbxew{0:34408} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.28:6828/2908130327,v1:172.21.15.28:6829/2908130327] compat {c=[1],r=[1],i=[1fff]}] 2024-07-03T15:52:24.528 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:52:24.528 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:52:24.528 INFO:teuthology.orchestra.run.smithi028.stdout:Standby daemons: 2024-07-03T15:52:24.529 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:52:24.529 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi148.xgpuld{-1:44301} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/1480077002,v1:172.21.15.148:6825/1480077002] compat {c=[1],r=[1],i=[1fff]}] 2024-07-03T15:52:24.529 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi148.szomfl{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6826/3922163302,v1:172.21.15.148:6827/3922163302] compat {c=[1],r=[1],i=[1fff]}] 2024-07-03T15:52:24.529 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 35 2024-07-03T15:52:24.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.523+0000 7fcc8a812640 1 -- 172.21.15.28:0/2594132096 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fcc6c06e0b0 msgr2=0x7fcc6c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:52:24.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.523+0000 7fcc8a812640 1 --2- 172.21.15.28:0/2594132096 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fcc6c06e0b0 0x7fcc6c070560 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcc780043d0 tx=0x7fcc78035040 comp rx=0 tx=0).stop 2024-07-03T15:52:24.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.523+0000 7fcc8a812640 1 -- 172.21.15.28:0/2594132096 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcc840fc330 msgr2=0x7fcc84071380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:52:24.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.523+0000 7fcc8a812640 1 --2- 172.21.15.28:0/2594132096 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcc840fc330 0x7fcc84071380 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcc74002a50 tx=0x7fcc74004600 comp rx=0 tx=0).stop 2024-07-03T15:52:24.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.523+0000 7fcc8a812640 1 -- 172.21.15.28:0/2594132096 shutdown_connections 2024-07-03T15:52:24.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.523+0000 7fcc8a812640 1 --2- 172.21.15.28:0/2594132096 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fcc6c06e0b0 0x7fcc6c070560 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:24.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.523+0000 7fcc8a812640 1 --2- 172.21.15.28:0/2594132096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc840fd600 0x7fcc8406f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:24.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.523+0000 7fcc8a812640 1 --2- 172.21.15.28:0/2594132096 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcc840fc330 0x7fcc84071380 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:24.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.523+0000 7fcc8a812640 1 -- 172.21.15.28:0/2594132096 >> 172.21.15.28:0/2594132096 conn(0x7fcc840f7af0 msgr2=0x7fcc840f91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:52:24.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.523+0000 7fcc8a812640 1 -- 172.21.15.28:0/2594132096 shutdown_connections 2024-07-03T15:52:24.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.523+0000 7fcc8a812640 1 -- 172.21.15.28:0/2594132096 wait complete. 2024-07-03T15:52:24.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.609+0000 7f3d286e1640 1 -- 172.21.15.28:0/3389527891 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d200fb240 msgr2=0x7f3d200fb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:52:24.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.609+0000 7f3d286e1640 1 --2- 172.21.15.28:0/3389527891 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d200fb240 0x7f3d200fb650 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3d100048d0 tx=0x7f3d10009da0 comp rx=0 tx=0).stop 2024-07-03T15:52:24.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.609+0000 7f3d286e1640 1 -- 172.21.15.28:0/3389527891 shutdown_connections 2024-07-03T15:52:24.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.609+0000 7f3d286e1640 1 --2- 172.21.15.28:0/3389527891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d200fc510 0x7f3d200fc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:24.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.609+0000 7f3d286e1640 1 --2- 172.21.15.28:0/3389527891 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d200fb240 0x7f3d200fb650 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:24.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.609+0000 7f3d286e1640 1 -- 172.21.15.28:0/3389527891 >> 172.21.15.28:0/3389527891 conn(0x7f3d200f6a20 msgr2=0x7f3d200f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:52:24.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.609+0000 7f3d286e1640 1 -- 172.21.15.28:0/3389527891 shutdown_connections 2024-07-03T15:52:24.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.609+0000 7f3d286e1640 1 -- 172.21.15.28:0/3389527891 wait complete. 2024-07-03T15:52:24.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.609+0000 7f3d286e1640 1 Processor -- start 2024-07-03T15:52:24.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.609+0000 7f3d286e1640 1 -- start start 2024-07-03T15:52:24.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.609+0000 7f3d286e1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d200fb240 0x7f3d200ff360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:52:24.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.609+0000 7f3d286e1640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d200fc510 0x7f3d200fd850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:52:24.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.609+0000 7f3d286e1640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d200fdde0 con 0x7f3d200fc510 2024-07-03T15:52:24.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.609+0000 7f3d286e1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d200fdf20 con 0x7f3d200fb240 2024-07-03T15:52:24.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.610+0000 7f3d25c7c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d200fc510 0x7f3d200fd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:52:24.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.610+0000 7f3d25c7c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d200fc510 0x7f3d200fd850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59916/0 (socket says 172.21.15.28:59916) 2024-07-03T15:52:24.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.610+0000 7f3d25c7c640 1 -- 172.21.15.28:0/168408487 learned_addr learned my addr 172.21.15.28:0/168408487 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:52:24.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.610+0000 7f3d2647d640 1 --2- 172.21.15.28:0/168408487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d200fb240 0x7f3d200ff360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:52:24.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.610+0000 7f3d25c7c640 1 -- 172.21.15.28:0/168408487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d200fb240 msgr2=0x7f3d200ff360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:52:24.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.610+0000 7f3d25c7c640 1 --2- 172.21.15.28:0/168408487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d200fb240 0x7f3d200ff360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:24.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.610+0000 7f3d25c7c640 1 -- 172.21.15.28:0/168408487 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d10005040 con 0x7f3d200fc510 2024-07-03T15:52:24.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.610+0000 7f3d25c7c640 1 --2- 172.21.15.28:0/168408487 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d200fc510 0x7f3d200fd850 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c00eb70 tx=0x7f3d1c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:52:24.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.610+0000 7f3d16ffd640 1 -- 172.21.15.28:0/168408487 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d1c007c30 con 0x7f3d200fc510 2024-07-03T15:52:24.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.610+0000 7f3d16ffd640 1 -- 172.21.15.28:0/168408487 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3d1c00f040 con 0x7f3d200fc510 2024-07-03T15:52:24.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.610+0000 7f3d286e1640 1 -- 172.21.15.28:0/168408487 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d200fe1a0 con 0x7f3d200fc510 2024-07-03T15:52:24.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.610+0000 7f3d16ffd640 1 -- 172.21.15.28:0/168408487 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d1c00cad0 con 0x7f3d200fc510 2024-07-03T15:52:24.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.610+0000 7f3d286e1640 1 -- 172.21.15.28:0/168408487 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d20114fc0 con 0x7f3d200fc510 2024-07-03T15:52:24.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.612+0000 7f3d286e1640 1 -- 172.21.15.28:0/168408487 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d200fa6a0 con 0x7f3d200fc510 2024-07-03T15:52:24.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.612+0000 7f3d16ffd640 1 -- 172.21.15.28:0/168408487 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3d1c00cc30 con 0x7f3d200fc510 2024-07-03T15:52:24.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.612+0000 7f3d16ffd640 1 --2- 172.21.15.28:0/168408487 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f3d0c06e0b0 0x7f3d0c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:52:24.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.613+0000 7f3d16ffd640 1 -- 172.21.15.28:0/168408487 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f3d1c08d860 con 0x7f3d200fc510 2024-07-03T15:52:24.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.613+0000 7f3d2647d640 1 --2- 172.21.15.28:0/168408487 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f3d0c06e0b0 0x7f3d0c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:52:24.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.613+0000 7f3d2647d640 1 --2- 172.21.15.28:0/168408487 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f3d0c06e0b0 0x7f3d0c070560 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3d10007ac0 tx=0x7f3d10035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:52:24.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.617+0000 7f3d16ffd640 1 -- 172.21.15.28:0/168408487 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f3d1c056bb0 con 0x7f3d200fc510 2024-07-03T15:52:24.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.829+0000 7f3d286e1640 1 -- 172.21.15.28:0/168408487 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d20103ef0 con 0x7f3d0c06e0b0 2024-07-03T15:52:24.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.831+0000 7f3d16ffd640 1 -- 172.21.15.28:0/168408487 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+440 (secure 0 0 0) 0x7f3d20103ef0 con 0x7f3d0c06e0b0 2024-07-03T15:52:24.835 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-07-03T15:52:24.835 INFO:teuthology.orchestra.run.smithi028.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2", 2024-07-03T15:52:24.836 INFO:teuthology.orchestra.run.smithi028.stdout: "in_progress": true, 2024-07-03T15:52:24.836 INFO:teuthology.orchestra.run.smithi028.stdout: "which": "Upgrading all daemon types on all hosts", 2024-07-03T15:52:24.836 INFO:teuthology.orchestra.run.smithi028.stdout: "services_complete": [ 2024-07-03T15:52:24.836 INFO:teuthology.orchestra.run.smithi028.stdout: "mds", 2024-07-03T15:52:24.836 INFO:teuthology.orchestra.run.smithi028.stdout: "mon", 2024-07-03T15:52:24.836 INFO:teuthology.orchestra.run.smithi028.stdout: "osd", 2024-07-03T15:52:24.836 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr", 2024-07-03T15:52:24.836 INFO:teuthology.orchestra.run.smithi028.stdout: "crash" 2024-07-03T15:52:24.836 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-07-03T15:52:24.836 INFO:teuthology.orchestra.run.smithi028.stdout: "progress": "16/21 daemons upgraded", 2024-07-03T15:52:24.836 INFO:teuthology.orchestra.run.smithi028.stdout: "message": "Currently upgrading alertmanager daemons", 2024-07-03T15:52:24.836 INFO:teuthology.orchestra.run.smithi028.stdout: "is_paused": false 2024-07-03T15:52:24.836 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-07-03T15:52:24.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.833+0000 7f3d286e1640 1 -- 172.21.15.28:0/168408487 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f3d0c06e0b0 msgr2=0x7f3d0c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:52:24.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.833+0000 7f3d286e1640 1 --2- 172.21.15.28:0/168408487 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f3d0c06e0b0 0x7f3d0c070560 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3d10007ac0 tx=0x7f3d10035040 comp rx=0 tx=0).stop 2024-07-03T15:52:24.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.833+0000 7f3d286e1640 1 -- 172.21.15.28:0/168408487 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d200fc510 msgr2=0x7f3d200fd850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:52:24.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.833+0000 7f3d286e1640 1 --2- 172.21.15.28:0/168408487 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d200fc510 0x7f3d200fd850 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c00eb70 tx=0x7f3d1c00b040 comp rx=0 tx=0).stop 2024-07-03T15:52:24.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.833+0000 7f3d286e1640 1 -- 172.21.15.28:0/168408487 shutdown_connections 2024-07-03T15:52:24.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.833+0000 7f3d286e1640 1 --2- 172.21.15.28:0/168408487 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f3d0c06e0b0 0x7f3d0c070560 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:24.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.833+0000 7f3d286e1640 1 --2- 172.21.15.28:0/168408487 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d200fc510 0x7f3d200fd850 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:24.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.833+0000 7f3d286e1640 1 --2- 172.21.15.28:0/168408487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d200fb240 0x7f3d200ff360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:24.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.833+0000 7f3d286e1640 1 -- 172.21.15.28:0/168408487 >> 172.21.15.28:0/168408487 conn(0x7f3d200f6a20 msgr2=0x7f3d20065300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:52:24.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.833+0000 7f3d286e1640 1 -- 172.21.15.28:0/168408487 shutdown_connections 2024-07-03T15:52:24.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.833+0000 7f3d286e1640 1 -- 172.21.15.28:0/168408487 wait complete. 2024-07-03T15:52:24.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.916+0000 7fdafe9ec640 1 -- 172.21.15.28:0/3216940138 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdaf80fd600 msgr2=0x7fdaf80fda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:52:24.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.916+0000 7fdafe9ec640 1 --2- 172.21.15.28:0/3216940138 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdaf80fd600 0x7fdaf80fda70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdaec0048d0 tx=0x7fdaec009da0 comp rx=0 tx=0).stop 2024-07-03T15:52:24.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.916+0000 7fdafe9ec640 1 -- 172.21.15.28:0/3216940138 shutdown_connections 2024-07-03T15:52:24.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.916+0000 7fdafe9ec640 1 --2- 172.21.15.28:0/3216940138 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdaf80fd600 0x7fdaf80fda70 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:24.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.916+0000 7fdafe9ec640 1 --2- 172.21.15.28:0/3216940138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdaf80fc330 0x7fdaf80fc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:24.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.916+0000 7fdafe9ec640 1 -- 172.21.15.28:0/3216940138 >> 172.21.15.28:0/3216940138 conn(0x7fdaf80f7af0 msgr2=0x7fdaf80f9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:52:24.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.917+0000 7fdafe9ec640 1 -- 172.21.15.28:0/3216940138 shutdown_connections 2024-07-03T15:52:24.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.917+0000 7fdafe9ec640 1 -- 172.21.15.28:0/3216940138 wait complete. 2024-07-03T15:52:24.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.917+0000 7fdafe9ec640 1 Processor -- start 2024-07-03T15:52:24.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.917+0000 7fdafe9ec640 1 -- start start 2024-07-03T15:52:24.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.917+0000 7fdafe9ec640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdaf80fc330 0x7fdaf810cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:52:24.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.918+0000 7fdafe9ec640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdaf80fd600 0x7fdaf810d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:52:24.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.918+0000 7fdafe9ec640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf810db30 con 0x7fdaf80fc330 2024-07-03T15:52:24.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.918+0000 7fdafe9ec640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf810e300 con 0x7fdaf80fd600 2024-07-03T15:52:24.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.918+0000 7fdaf7fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdaf80fc330 0x7fdaf810cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:52:24.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.918+0000 7fdaf7fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdaf80fc330 0x7fdaf810cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59928/0 (socket says 172.21.15.28:59928) 2024-07-03T15:52:24.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.918+0000 7fdaf7fff640 1 -- 172.21.15.28:0/2456648517 learned_addr learned my addr 172.21.15.28:0/2456648517 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:52:24.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.918+0000 7fdaf77fe640 1 --2- 172.21.15.28:0/2456648517 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdaf80fd600 0x7fdaf810d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:52:24.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.918+0000 7fdaf7fff640 1 -- 172.21.15.28:0/2456648517 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdaf80fd600 msgr2=0x7fdaf810d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:52:24.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.918+0000 7fdaf7fff640 1 --2- 172.21.15.28:0/2456648517 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdaf80fd600 0x7fdaf810d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:24.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.918+0000 7fdaf7fff640 1 -- 172.21.15.28:0/2456648517 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaec005040 con 0x7fdaf80fc330 2024-07-03T15:52:24.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.918+0000 7fdaf7fff640 1 --2- 172.21.15.28:0/2456648517 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdaf80fc330 0x7fdaf810cfa0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdae8002a80 tx=0x7fdae8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:52:24.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.918+0000 7fdaf4ff9640 1 -- 172.21.15.28:0/2456648517 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdae8007c30 con 0x7fdaf80fc330 2024-07-03T15:52:24.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.918+0000 7fdafe9ec640 1 -- 172.21.15.28:0/2456648517 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaf810e590 con 0x7fdaf80fc330 2024-07-03T15:52:24.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.919+0000 7fdaf4ff9640 1 -- 172.21.15.28:0/2456648517 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdae800f040 con 0x7fdaf80fc330 2024-07-03T15:52:24.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.919+0000 7fdaf4ff9640 1 -- 172.21.15.28:0/2456648517 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdae800db70 con 0x7fdaf80fc330 2024-07-03T15:52:24.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.919+0000 7fdafe9ec640 1 -- 172.21.15.28:0/2456648517 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaf810ea80 con 0x7fdaf80fc330 2024-07-03T15:52:24.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.920+0000 7fdafe9ec640 1 -- 172.21.15.28:0/2456648517 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaf8105690 con 0x7fdaf80fc330 2024-07-03T15:52:24.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.920+0000 7fdaf4ff9640 1 -- 172.21.15.28:0/2456648517 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fdae800dcf0 con 0x7fdaf80fc330 2024-07-03T15:52:24.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.921+0000 7fdaf4ff9640 1 --2- 172.21.15.28:0/2456648517 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fdae006e0b0 0x7fdae0070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:52:24.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.921+0000 7fdaf4ff9640 1 -- 172.21.15.28:0/2456648517 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fdae808d3b0 con 0x7fdaf80fc330 2024-07-03T15:52:24.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.921+0000 7fdaf77fe640 1 --2- 172.21.15.28:0/2456648517 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fdae006e0b0 0x7fdae0070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:52:24.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.921+0000 7fdaf77fe640 1 --2- 172.21.15.28:0/2456648517 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fdae006e0b0 0x7fdae0070560 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdaec0043d0 tx=0x7fdaec035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:52:24.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:24.925+0000 7fdaf4ff9640 1 -- 172.21.15.28:0/2456648517 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fdae80567b0 con 0x7fdaf80fc330 2024-07-03T15:52:25.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:25.193+0000 7fdafe9ec640 1 -- 172.21.15.28:0/2456648517 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fdaf80fb790 con 0x7fdaf80fc330 2024-07-03T15:52:25.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:25.193+0000 7fdaf4ff9640 1 -- 172.21.15.28:0/2456648517 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fdae8059e70 con 0x7fdaf80fc330 2024-07-03T15:52:25.198 INFO:teuthology.orchestra.run.smithi028.stdout:HEALTH_OK 2024-07-03T15:52:25.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:25.195+0000 7fdafe9ec640 1 -- 172.21.15.28:0/2456648517 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fdae006e0b0 msgr2=0x7fdae0070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:52:25.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:25.195+0000 7fdafe9ec640 1 --2- 172.21.15.28:0/2456648517 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fdae006e0b0 0x7fdae0070560 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdaec0043d0 tx=0x7fdaec035040 comp rx=0 tx=0).stop 2024-07-03T15:52:25.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:25.195+0000 7fdafe9ec640 1 -- 172.21.15.28:0/2456648517 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdaf80fc330 msgr2=0x7fdaf810cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:52:25.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:25.195+0000 7fdafe9ec640 1 --2- 172.21.15.28:0/2456648517 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdaf80fc330 0x7fdaf810cfa0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdae8002a80 tx=0x7fdae8004600 comp rx=0 tx=0).stop 2024-07-03T15:52:25.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:25.195+0000 7fdafe9ec640 1 -- 172.21.15.28:0/2456648517 shutdown_connections 2024-07-03T15:52:25.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:25.195+0000 7fdafe9ec640 1 --2- 172.21.15.28:0/2456648517 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fdae006e0b0 0x7fdae0070560 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:25.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:25.195+0000 7fdafe9ec640 1 --2- 172.21.15.28:0/2456648517 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdaf80fd600 0x7fdaf810d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:25.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:25.195+0000 7fdafe9ec640 1 --2- 172.21.15.28:0/2456648517 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdaf80fc330 0x7fdaf810cfa0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:25.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:25.195+0000 7fdafe9ec640 1 -- 172.21.15.28:0/2456648517 >> 172.21.15.28:0/2456648517 conn(0x7fdaf80f7af0 msgr2=0x7fdaf80f91e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:52:25.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:25.195+0000 7fdafe9ec640 1 -- 172.21.15.28:0/2456648517 shutdown_connections 2024-07-03T15:52:25.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:25.195+0000 7fdafe9ec640 1 -- 172.21.15.28:0/2456648517 wait complete. 2024-07-03T15:52:25.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:25 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:25.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:25 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:25.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:25 smithi028 ceph-mon[92858]: pgmap v218: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:52:25.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:25 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/554307608' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:25.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:25 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/2594132096' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-03T15:52:25.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:25 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:25.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:25 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:25.373 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:25 smithi028 ceph-mon[92858]: from='client.34494 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:52:25.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:25 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:25.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:25 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:25.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:25 smithi148 ceph-mon[80490]: pgmap v218: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:52:25.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:25 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/554307608' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:25.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:25 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/2594132096' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-03T15:52:25.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:25 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:25.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:25 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:25.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:25 smithi148 ceph-mon[80490]: from='client.34494 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:52:26.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:26 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/2456648517' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-03T15:52:26.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:26 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/2456648517' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-03T15:52:27.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:27 smithi028 ceph-mon[92858]: pgmap v219: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:52:27.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:27 smithi148 ceph-mon[80490]: pgmap v219: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:52:29.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:29 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:29.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:29 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:29.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:29 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:52:29.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:29 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:52:29.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:29 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:29.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:29 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-07-03T15:52:29.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:29 smithi028 ceph-mon[92858]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-07-03T15:52:29.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:29 smithi028 ceph-mon[92858]: pgmap v220: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:52:29.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:29 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:52:29.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:29 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:29.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:29 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:29.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:29 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:29.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:29 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:29.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:29 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:29.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:29 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:29.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:29 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:29.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:29 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:29.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:29 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:29.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:29 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:29.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:29 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:29.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:29 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:29.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:29 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:29.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:29 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:29.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:29 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:29.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:29 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:29.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:29 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:29.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:29 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:52:29.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:29 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:52:29.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:29 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:29.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:29 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-07-03T15:52:29.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:29 smithi148 ceph-mon[80490]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-07-03T15:52:29.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:29 smithi148 ceph-mon[80490]: pgmap v220: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:52:29.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:29 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:52:29.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:29 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:29.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:29 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:29.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:29 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:29.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:29 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:29.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:29 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:29.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:29 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:29.483 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:29 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:29.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:29 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:29.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:29 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:29.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:29 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:29.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:29 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:29.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:29 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:29.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:29 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:29.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:29 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:29.484 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:29 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:30.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:30 smithi028 ceph-mon[92858]: Upgrade: Updating grafana.smithi028 2024-07-03T15:52:30.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:30 smithi028 ceph-mon[92858]: Deploying daemon grafana.smithi028 on smithi028 2024-07-03T15:52:30.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:30 smithi148 ceph-mon[80490]: Upgrade: Updating grafana.smithi028 2024-07-03T15:52:30.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:30 smithi148 ceph-mon[80490]: Deploying daemon grafana.smithi028 on smithi028 2024-07-03T15:52:31.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:31 smithi028 ceph-mon[92858]: pgmap v221: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:52:31.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:31 smithi148 ceph-mon[80490]: pgmap v221: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:52:33.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:33 smithi148 ceph-mon[80490]: pgmap v222: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:52:33.621 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:33 smithi028 ceph-mon[92858]: pgmap v222: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:52:35.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:35 smithi148 ceph-mon[80490]: pgmap v223: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:52:35.594 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:35 smithi028 ceph-mon[92858]: pgmap v223: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:52:37.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:37 smithi148 ceph-mon[80490]: pgmap v224: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:52:37.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:37 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:52:37.486 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:37 smithi028 ceph-mon[92858]: pgmap v224: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:52:37.487 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:37 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:52:39.307 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:39 smithi028 ceph-mon[92858]: pgmap v225: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:52:39.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:39 smithi148 ceph-mon[80490]: pgmap v225: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:52:41.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:41 smithi148 ceph-mon[80490]: pgmap v226: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:52:41.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:41 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:41.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:41 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:41.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:41 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:52:42.109 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:41 smithi028 ceph-mon[92858]: pgmap v226: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:52:42.109 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:41 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:42.109 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:41 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:42.109 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:41 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:52:43.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:43 smithi028 ceph-mon[92858]: pgmap v227: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:52:44.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:43 smithi148 ceph-mon[80490]: pgmap v227: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:52:45.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:45 smithi028 ceph-mon[92858]: pgmap v228: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:52:45.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:45 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:45.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:45 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:45.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:45 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:45.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:45 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:46.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:45 smithi148 ceph-mon[80490]: pgmap v228: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:52:46.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:45 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:46.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:45 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:46.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:45 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:46.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:45 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:48.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:47 smithi028 ceph-mon[92858]: pgmap v229: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:52:48.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:47 smithi148 ceph-mon[80490]: pgmap v229: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:52:50.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: pgmap v230: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:52:50.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:50.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:50.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:52:50.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:52:50.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:50.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-07-03T15:52:50.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:52:50.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:50.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:50.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:50.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:50.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:50.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:50.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:50.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:50.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:50.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:50.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:50.124 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:50.125 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:50.125 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:50.125 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:50.125 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:50.125 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:50.125 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:50.125 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:50.125 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-07-03T15:52:50.125 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-07-03T15:52:50.125 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-07-03T15:52:50.125 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-07-03T15:52:50.125 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-07-03T15:52:50.126 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-07-03T15:52:50.126 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-07-03T15:52:50.126 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-07-03T15:52:50.126 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-07-03T15:52:50.126 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-07-03T15:52:50.126 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-07-03T15:52:50.126 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-07-03T15:52:50.126 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-07-03T15:52:50.126 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-07-03T15:52:50.126 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-07-03T15:52:50.126 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-07-03T15:52:50.126 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-07-03T15:52:50.127 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-07-03T15:52:50.127 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-07-03T15:52:50.127 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-07-03T15:52:50.127 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-07-03T15:52:50.127 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-07-03T15:52:50.127 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-07-03T15:52:50.127 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-07-03T15:52:50.127 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-07-03T15:52:50.127 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-07-03T15:52:50.127 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-07-03T15:52:50.127 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-07-03T15:52:50.127 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-07-03T15:52:50.128 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-07-03T15:52:50.128 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-07-03T15:52:50.128 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:49 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:52:50.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: pgmap v230: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:52:50.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:50.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:50.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:52:50.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:52:50.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:50.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-07-03T15:52:50.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:52:50.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:50.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:50.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:50.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:50.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:50.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:50.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:50.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:50.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:50.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:50.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:50.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:50.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:50.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:50.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:50.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:50.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:50.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:52:50.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:50.235 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-07-03T15:52:50.235 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-07-03T15:52:50.235 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-07-03T15:52:50.235 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-07-03T15:52:50.235 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-07-03T15:52:50.235 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-07-03T15:52:50.235 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-07-03T15:52:50.235 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-07-03T15:52:50.235 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-07-03T15:52:50.235 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-07-03T15:52:50.235 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-07-03T15:52:50.235 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-07-03T15:52:50.235 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-07-03T15:52:50.235 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-07-03T15:52:50.236 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-07-03T15:52:50.236 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-07-03T15:52:50.236 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-07-03T15:52:50.236 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-07-03T15:52:50.236 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-07-03T15:52:50.236 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-07-03T15:52:50.236 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-07-03T15:52:50.236 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-07-03T15:52:50.236 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-07-03T15:52:50.236 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-07-03T15:52:50.236 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-07-03T15:52:50.236 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-07-03T15:52:50.236 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-07-03T15:52:50.237 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-07-03T15:52:50.237 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-07-03T15:52:50.237 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-07-03T15:52:50.237 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-07-03T15:52:50.237 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:49 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:52:50.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:50 smithi148 ceph-mon[80490]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-07-03T15:52:50.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:50 smithi148 ceph-mon[80490]: Upgrade: Finalizing container_image settings 2024-07-03T15:52:50.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:50 smithi148 ceph-mon[80490]: Upgrade: Complete! 2024-07-03T15:52:50.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:50 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:52:50.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:50 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:52:50.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:50 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:50.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:50 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:52:50.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:50 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:52:50.982 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:50 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:52:50.983 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:50 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:51.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:50 smithi028 ceph-mon[92858]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-07-03T15:52:51.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:50 smithi028 ceph-mon[92858]: Upgrade: Finalizing container_image settings 2024-07-03T15:52:51.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:50 smithi028 ceph-mon[92858]: Upgrade: Complete! 2024-07-03T15:52:51.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:50 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:52:51.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:50 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:52:51.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:50 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:51.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:50 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:52:51.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:50 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:52:51.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:50 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:52:51.123 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:50 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:52.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:51 smithi028 ceph-mon[92858]: pgmap v231: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:52:52.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:51 smithi148 ceph-mon[80490]: pgmap v231: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:52:53.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:52 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:52:53.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:52 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:53.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:52 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:52:53.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:52 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:52:54.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:53 smithi028 ceph-mon[92858]: pgmap v232: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:52:54.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:53 smithi148 ceph-mon[80490]: pgmap v232: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:52:55.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.292+0000 7f3fadd6f640 1 -- 172.21.15.28:0/2819730591 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fa80fc510 msgr2=0x7f3fa80fc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:52:55.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.292+0000 7f3fadd6f640 1 --2- 172.21.15.28:0/2819730591 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fa80fc510 0x7f3fa80fc980 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c0048d0 tx=0x7f3f9c009da0 comp rx=0 tx=0).stop 2024-07-03T15:52:55.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.292+0000 7f3fadd6f640 1 -- 172.21.15.28:0/2819730591 shutdown_connections 2024-07-03T15:52:55.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.292+0000 7f3fadd6f640 1 --2- 172.21.15.28:0/2819730591 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fa80fc510 0x7f3fa80fc980 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:55.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.292+0000 7f3fadd6f640 1 --2- 172.21.15.28:0/2819730591 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fa80fb240 0x7f3fa80fb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:55.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.292+0000 7f3fadd6f640 1 -- 172.21.15.28:0/2819730591 >> 172.21.15.28:0/2819730591 conn(0x7f3fa80f6a20 msgr2=0x7f3fa80f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:52:55.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.292+0000 7f3fadd6f640 1 -- 172.21.15.28:0/2819730591 shutdown_connections 2024-07-03T15:52:55.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.292+0000 7f3fadd6f640 1 -- 172.21.15.28:0/2819730591 wait complete. 2024-07-03T15:52:55.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.293+0000 7f3fadd6f640 1 Processor -- start 2024-07-03T15:52:55.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.293+0000 7f3fadd6f640 1 -- start start 2024-07-03T15:52:55.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.293+0000 7f3fadd6f640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fa80fb240 0x7f3fa810cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:52:55.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.293+0000 7f3fadd6f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fa80fc510 0x7f3fa810d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:52:55.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.293+0000 7f3fadd6f640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa810db50 con 0x7f3fa80fb240 2024-07-03T15:52:55.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.293+0000 7f3fadd6f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa810e320 con 0x7f3fa80fc510 2024-07-03T15:52:55.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.293+0000 7f3fa77fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fa80fb240 0x7f3fa810cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:52:55.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.293+0000 7f3fa77fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fa80fb240 0x7f3fa810cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35384/0 (socket says 172.21.15.28:35384) 2024-07-03T15:52:55.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.293+0000 7f3fa77fe640 1 -- 172.21.15.28:0/2442272913 learned_addr learned my addr 172.21.15.28:0/2442272913 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:52:55.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.294+0000 7f3fa6ffd640 1 --2- 172.21.15.28:0/2442272913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fa80fc510 0x7f3fa810d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:52:55.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.294+0000 7f3fa77fe640 1 -- 172.21.15.28:0/2442272913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fa80fc510 msgr2=0x7f3fa810d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:52:55.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.294+0000 7f3fa77fe640 1 --2- 172.21.15.28:0/2442272913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fa80fc510 0x7f3fa810d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:55.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.294+0000 7f3fa77fe640 1 -- 172.21.15.28:0/2442272913 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f9c005040 con 0x7f3fa80fb240 2024-07-03T15:52:55.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.294+0000 7f3fa77fe640 1 --2- 172.21.15.28:0/2442272913 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fa80fb240 0x7f3fa810cfc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3f9800db80 tx=0x7f3f98004f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:52:55.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.294+0000 7f3f8ffff640 1 -- 172.21.15.28:0/2442272913 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f9800b8d0 con 0x7f3fa80fb240 2024-07-03T15:52:55.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.294+0000 7f3fadd6f640 1 -- 172.21.15.28:0/2442272913 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fa810e5b0 con 0x7f3fa80fb240 2024-07-03T15:52:55.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.294+0000 7f3f8ffff640 1 -- 172.21.15.28:0/2442272913 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3f9800f040 con 0x7f3fa80fb240 2024-07-03T15:52:55.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.294+0000 7f3f8ffff640 1 -- 172.21.15.28:0/2442272913 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f98012390 con 0x7f3fa80fb240 2024-07-03T15:52:55.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.294+0000 7f3fadd6f640 1 -- 172.21.15.28:0/2442272913 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fa810eaa0 con 0x7f3fa80fb240 2024-07-03T15:52:55.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.295+0000 7f3fadd6f640 1 -- 172.21.15.28:0/2442272913 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fa81045a0 con 0x7f3fa80fb240 2024-07-03T15:52:55.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.296+0000 7f3f8ffff640 1 -- 172.21.15.28:0/2442272913 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3f9800ba30 con 0x7f3fa80fb240 2024-07-03T15:52:55.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.296+0000 7f3f8ffff640 1 --2- 172.21.15.28:0/2442272913 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f3f9006e0b0 0x7f3f90070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:52:55.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.297+0000 7f3f8ffff640 1 -- 172.21.15.28:0/2442272913 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f3f9808dad0 con 0x7f3fa80fb240 2024-07-03T15:52:55.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.297+0000 7f3fa6ffd640 1 --2- 172.21.15.28:0/2442272913 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f3f9006e0b0 0x7f3f90070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:52:55.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.297+0000 7f3fa6ffd640 1 --2- 172.21.15.28:0/2442272913 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f3f9006e0b0 0x7f3f90070560 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c0043d0 tx=0x7f3f9c035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:52:55.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.301+0000 7f3f8ffff640 1 -- 172.21.15.28:0/2442272913 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f3f98056da0 con 0x7f3fa80fb240 2024-07-03T15:52:55.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.504+0000 7f3fadd6f640 1 -- 172.21.15.28:0/2442272913 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3fa8103f00 con 0x7f3f9006e0b0 2024-07-03T15:52:55.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.505+0000 7f3f8ffff640 1 -- 172.21.15.28:0/2442272913 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f3fa8103f00 con 0x7f3f9006e0b0 2024-07-03T15:52:55.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.508+0000 7f3fadd6f640 1 -- 172.21.15.28:0/2442272913 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f3f9006e0b0 msgr2=0x7f3f90070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:52:55.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.508+0000 7f3fadd6f640 1 --2- 172.21.15.28:0/2442272913 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f3f9006e0b0 0x7f3f90070560 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c0043d0 tx=0x7f3f9c035040 comp rx=0 tx=0).stop 2024-07-03T15:52:55.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.508+0000 7f3fadd6f640 1 -- 172.21.15.28:0/2442272913 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fa80fb240 msgr2=0x7f3fa810cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:52:55.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.508+0000 7f3fadd6f640 1 --2- 172.21.15.28:0/2442272913 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fa80fb240 0x7f3fa810cfc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3f9800db80 tx=0x7f3f98004f00 comp rx=0 tx=0).stop 2024-07-03T15:52:55.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.508+0000 7f3fadd6f640 1 -- 172.21.15.28:0/2442272913 shutdown_connections 2024-07-03T15:52:55.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.508+0000 7f3fadd6f640 1 --2- 172.21.15.28:0/2442272913 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f3f9006e0b0 0x7f3f90070560 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:55.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.508+0000 7f3fadd6f640 1 --2- 172.21.15.28:0/2442272913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fa80fc510 0x7f3fa810d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:55.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.508+0000 7f3fadd6f640 1 --2- 172.21.15.28:0/2442272913 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fa80fb240 0x7f3fa810cfc0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:55.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.508+0000 7f3fadd6f640 1 -- 172.21.15.28:0/2442272913 >> 172.21.15.28:0/2442272913 conn(0x7f3fa80f6a20 msgr2=0x7f3fa8065390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:52:55.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.508+0000 7f3fadd6f640 1 -- 172.21.15.28:0/2442272913 shutdown_connections 2024-07-03T15:52:55.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:55.508+0000 7f3fadd6f640 1 -- 172.21.15.28:0/2442272913 wait complete. 2024-07-03T15:52:55.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:55 smithi028 ceph-mon[92858]: pgmap v233: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:52:56.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:55 smithi148 ceph-mon[80490]: pgmap v233: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:52:56.608 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -e sha1=c00506dc10003928f8cb40769168571f611cbd67 -- bash -c 'ceph orch ps' 2024-07-03T15:52:56.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:56 smithi028 ceph-mon[92858]: from='client.34502 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:52:56.927 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:52:57.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:56 smithi148 ceph-mon[80490]: from='client.34502 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:52:58.001 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:57 smithi028 ceph-mon[92858]: pgmap v234: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:52:58.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:57 smithi148 ceph-mon[80490]: pgmap v234: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:52:58.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.507+0000 7f488bccb640 1 -- 172.21.15.28:0/2286149561 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4884102d30 msgr2=0x7f48840faa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:52:58.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.507+0000 7f488bccb640 1 --2- 172.21.15.28:0/2286149561 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4884102d30 0x7f48840faa70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f48780048d0 tx=0x7f4878009da0 comp rx=0 tx=0).stop 2024-07-03T15:52:58.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.507+0000 7f488bccb640 1 -- 172.21.15.28:0/2286149561 shutdown_connections 2024-07-03T15:52:58.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.507+0000 7f488bccb640 1 --2- 172.21.15.28:0/2286149561 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4884102d30 0x7f48840faa70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:58.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.507+0000 7f488bccb640 1 --2- 172.21.15.28:0/2286149561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4884102420 0x7f48841027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:58.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.507+0000 7f488bccb640 1 -- 172.21.15.28:0/2286149561 >> 172.21.15.28:0/2286149561 conn(0x7f4884070a90 msgr2=0x7f4884070e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:52:58.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.507+0000 7f488bccb640 1 -- 172.21.15.28:0/2286149561 shutdown_connections 2024-07-03T15:52:58.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.507+0000 7f488bccb640 1 -- 172.21.15.28:0/2286149561 wait complete. 2024-07-03T15:52:58.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.508+0000 7f488bccb640 1 Processor -- start 2024-07-03T15:52:58.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.508+0000 7f488bccb640 1 -- start start 2024-07-03T15:52:58.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.508+0000 7f488bccb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4884102420 0x7f48840fbc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:52:58.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.508+0000 7f488bccb640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4884102d30 0x7f48840fc190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:52:58.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.508+0000 7f488bccb640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48840fc7b0 con 0x7f4884102d30 2024-07-03T15:52:58.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.508+0000 7f488bccb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48840fc920 con 0x7f4884102420 2024-07-03T15:52:58.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.508+0000 7f4889266640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4884102d30 0x7f48840fc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:52:58.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.508+0000 7f4889266640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4884102d30 0x7f48840fc190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35396/0 (socket says 172.21.15.28:35396) 2024-07-03T15:52:58.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.508+0000 7f4889266640 1 -- 172.21.15.28:0/149004336 learned_addr learned my addr 172.21.15.28:0/149004336 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:52:58.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.508+0000 7f4889a67640 1 --2- 172.21.15.28:0/149004336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4884102420 0x7f48840fbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:52:58.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.508+0000 7f4889266640 1 -- 172.21.15.28:0/149004336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4884102420 msgr2=0x7f48840fbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:52:58.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.508+0000 7f4889266640 1 --2- 172.21.15.28:0/149004336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4884102420 0x7f48840fbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:58.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.508+0000 7f4889266640 1 -- 172.21.15.28:0/149004336 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4878005040 con 0x7f4884102d30 2024-07-03T15:52:58.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.509+0000 7f4889266640 1 --2- 172.21.15.28:0/149004336 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4884102d30 0x7f48840fc190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4878032430 tx=0x7f4878009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:52:58.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.509+0000 7f48827fc640 1 -- 172.21.15.28:0/149004336 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4878038070 con 0x7f4884102d30 2024-07-03T15:52:58.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.509+0000 7f48827fc640 1 -- 172.21.15.28:0/149004336 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4878032d60 con 0x7f4884102d30 2024-07-03T15:52:58.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.509+0000 7f488bccb640 1 -- 172.21.15.28:0/149004336 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48841194f0 con 0x7f4884102d30 2024-07-03T15:52:58.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.509+0000 7f48827fc640 1 -- 172.21.15.28:0/149004336 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f487803de10 con 0x7f4884102d30 2024-07-03T15:52:58.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.509+0000 7f488bccb640 1 -- 172.21.15.28:0/149004336 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4884119930 con 0x7f4884102d30 2024-07-03T15:52:58.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.510+0000 7f488bccb640 1 -- 172.21.15.28:0/149004336 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4884065610 con 0x7f4884102d30 2024-07-03T15:52:58.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.510+0000 7f48827fc640 1 -- 172.21.15.28:0/149004336 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4878030020 con 0x7f4884102d30 2024-07-03T15:52:58.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.511+0000 7f48827fc640 1 --2- 172.21.15.28:0/149004336 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f486c06e0b0 0x7f486c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:52:58.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.511+0000 7f4889a67640 1 --2- 172.21.15.28:0/149004336 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f486c06e0b0 0x7f486c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:52:58.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.511+0000 7f48827fc640 1 -- 172.21.15.28:0/149004336 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f48780ae6b0 con 0x7f4884102d30 2024-07-03T15:52:58.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.512+0000 7f4889a67640 1 --2- 172.21.15.28:0/149004336 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f486c06e0b0 0x7f486c070560 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4874001250 tx=0x7f4874009620 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:52:58.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.516+0000 7f48827fc640 1 -- 172.21.15.28:0/149004336 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f4878077980 con 0x7f4884102d30 2024-07-03T15:52:58.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.712+0000 7f488bccb640 1 -- 172.21.15.28:0/149004336 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f48841095e0 con 0x7f486c06e0b0 2024-07-03T15:52:58.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.721+0000 7f48827fc640 1 -- 172.21.15.28:0/149004336 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f48841095e0 con 0x7f486c06e0b0 2024-07-03T15:52:58.726 INFO:teuthology.orchestra.run.smithi028.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-03T15:52:58.726 INFO:teuthology.orchestra.run.smithi028.stdout:alertmanager.smithi028 smithi028 *:9093,9094 running (39s) 13s ago 16m 16.5M - 0.25.0 c8568f914cd2 f997825b72e3 2024-07-03T15:52:58.726 INFO:teuthology.orchestra.run.smithi028.stdout:crash.smithi028 smithi028 running (5m) 13s ago 16m 6895k - 19.0.0-2550-gc00506dc 9d49405603e8 c62c2f45cabe 2024-07-03T15:52:58.726 INFO:teuthology.orchestra.run.smithi028.stdout:crash.smithi148 smithi148 running (5m) 70s ago 15m 6895k - 19.0.0-2550-gc00506dc 9d49405603e8 b1318dcaacb6 2024-07-03T15:52:58.727 INFO:teuthology.orchestra.run.smithi028.stdout:grafana.smithi028 smithi028 *:3000 running (18s) 13s ago 16m 44.8M - 9.4.12 2bacad6d85d8 c7e25b78cdeb 2024-07-03T15:52:58.727 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi028.rhpvve smithi028 running (2m) 13s ago 11m 84.3M - 19.0.0-2550-gc00506dc 9d49405603e8 5d528ea2a2f0 2024-07-03T15:52:58.727 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi028.wmbxew smithi028 running (2m) 13s ago 11m 82.0M - 19.0.0-2550-gc00506dc 9d49405603e8 ebdbfa592ac8 2024-07-03T15:52:58.727 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi148.szomfl smithi148 running (90s) 70s ago 11m 13.0M - 19.0.0-2550-gc00506dc 9d49405603e8 27650f9fb39f 2024-07-03T15:52:58.727 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi148.xgpuld smithi148 running (107s) 70s ago 11m 14.6M - 19.0.0-2550-gc00506dc 9d49405603e8 f24ab04507dc 2024-07-03T15:52:58.727 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.smithi028.vtqrtz smithi028 *:8443,9283,8765 running (7m) 13s ago 17m 515M - 19.0.0-2550-gc00506dc 9d49405603e8 14d0363c5b77 2024-07-03T15:52:58.727 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.smithi148.rjzsar smithi148 *:8443,9283,8765 running (6m) 70s ago 15m 441M - 19.0.0-2550-gc00506dc 9d49405603e8 c0193aa8a5dd 2024-07-03T15:52:58.727 INFO:teuthology.orchestra.run.smithi028.stdout:mon.smithi028 smithi028 running (6m) 13s ago 18m 53.7M 2048M 19.0.0-2550-gc00506dc 9d49405603e8 5093f7bf47d7 2024-07-03T15:52:58.727 INFO:teuthology.orchestra.run.smithi028.stdout:mon.smithi148 smithi148 running (6m) 70s ago 15m 38.2M 2048M 19.0.0-2550-gc00506dc 9d49405603e8 d338d2c648da 2024-07-03T15:52:58.727 INFO:teuthology.orchestra.run.smithi028.stdout:node-exporter.smithi028 smithi028 *:9100 running (77s) 13s ago 16m 8484k - 1.5.0 0da6a335fe13 125dcba51db4 2024-07-03T15:52:58.728 INFO:teuthology.orchestra.run.smithi028.stdout:node-exporter.smithi148 smithi148 *:9100 running (72s) 70s ago 15m 3577k - 1.5.0 0da6a335fe13 edc710a27078 2024-07-03T15:52:58.728 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0 smithi028 running (5m) 13s ago 14m 111M 1558M 19.0.0-2550-gc00506dc 9d49405603e8 4454cecff8fc 2024-07-03T15:52:58.728 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1 smithi028 running (4m) 13s ago 14m 117M 1558M 19.0.0-2550-gc00506dc 9d49405603e8 af67f7088643 2024-07-03T15:52:58.728 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2 smithi028 running (4m) 13s ago 13m 101M 1558M 19.0.0-2550-gc00506dc 9d49405603e8 5d160a327dc2 2024-07-03T15:52:58.728 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3 smithi148 running (3m) 70s ago 13m 120M 2581M 19.0.0-2550-gc00506dc 9d49405603e8 1d622eba4b3b 2024-07-03T15:52:58.728 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4 smithi148 running (3m) 70s ago 13m 106M 2581M 19.0.0-2550-gc00506dc 9d49405603e8 546343d0f31f 2024-07-03T15:52:58.728 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5 smithi148 running (2m) 70s ago 12m 106M 2581M 19.0.0-2550-gc00506dc 9d49405603e8 a819fcc3b569 2024-07-03T15:52:58.728 INFO:teuthology.orchestra.run.smithi028.stdout:prometheus.smithi028 smithi028 *:9095 running (54s) 13s ago 16m 48.9M - 2.43.0 a07b618ecd1d 406cff811726 2024-07-03T15:52:58.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.723+0000 7f488bccb640 1 -- 172.21.15.28:0/149004336 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f486c06e0b0 msgr2=0x7f486c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:52:58.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.723+0000 7f488bccb640 1 --2- 172.21.15.28:0/149004336 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f486c06e0b0 0x7f486c070560 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4874001250 tx=0x7f4874009620 comp rx=0 tx=0).stop 2024-07-03T15:52:58.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.723+0000 7f488bccb640 1 -- 172.21.15.28:0/149004336 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4884102d30 msgr2=0x7f48840fc190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:52:58.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.723+0000 7f488bccb640 1 --2- 172.21.15.28:0/149004336 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4884102d30 0x7f48840fc190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4878032430 tx=0x7f4878009da0 comp rx=0 tx=0).stop 2024-07-03T15:52:58.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.723+0000 7f488bccb640 1 -- 172.21.15.28:0/149004336 shutdown_connections 2024-07-03T15:52:58.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.723+0000 7f488bccb640 1 --2- 172.21.15.28:0/149004336 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f486c06e0b0 0x7f486c070560 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:58.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.723+0000 7f488bccb640 1 --2- 172.21.15.28:0/149004336 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4884102d30 0x7f48840fc190 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:58.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.723+0000 7f488bccb640 1 --2- 172.21.15.28:0/149004336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4884102420 0x7f48840fbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:52:58.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.723+0000 7f488bccb640 1 -- 172.21.15.28:0/149004336 >> 172.21.15.28:0/149004336 conn(0x7f4884070a90 msgr2=0x7f4884105150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:52:58.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.724+0000 7f488bccb640 1 -- 172.21.15.28:0/149004336 shutdown_connections 2024-07-03T15:52:58.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:52:58.724+0000 7f488bccb640 1 -- 172.21.15.28:0/149004336 wait complete. 2024-07-03T15:52:59.461 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -e sha1=c00506dc10003928f8cb40769168571f611cbd67 -- bash -c 'ceph orch upgrade status' 2024-07-03T15:52:59.736 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:53:00.078 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:59 smithi028 ceph-mon[92858]: pgmap v235: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:53:00.079 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:52:59 smithi028 ceph-mon[92858]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:53:00.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:59 smithi148 ceph-mon[80490]: pgmap v235: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:53:00.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:52:59 smithi148 ceph-mon[80490]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:53:01.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.456+0000 7f5e781b7640 1 -- 172.21.15.28:0/1418200811 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e70101c50 msgr2=0x7f5e700fb2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:53:01.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.456+0000 7f5e781b7640 1 --2- 172.21.15.28:0/1418200811 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e70101c50 0x7f5e700fb2e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5e600048d0 tx=0x7f5e60009da0 comp rx=0 tx=0).stop 2024-07-03T15:53:01.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.456+0000 7f5e781b7640 1 -- 172.21.15.28:0/1418200811 shutdown_connections 2024-07-03T15:53:01.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.456+0000 7f5e781b7640 1 --2- 172.21.15.28:0/1418200811 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e70101c50 0x7f5e700fb2e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:53:01.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.456+0000 7f5e781b7640 1 --2- 172.21.15.28:0/1418200811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e701012b0 0x7f5e70101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:53:01.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.456+0000 7f5e781b7640 1 -- 172.21.15.28:0/1418200811 >> 172.21.15.28:0/1418200811 conn(0x7f5e700f6ea0 msgr2=0x7f5e700f92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:53:01.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.456+0000 7f5e781b7640 1 -- 172.21.15.28:0/1418200811 shutdown_connections 2024-07-03T15:53:01.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.456+0000 7f5e781b7640 1 -- 172.21.15.28:0/1418200811 wait complete. 2024-07-03T15:53:01.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.457+0000 7f5e781b7640 1 Processor -- start 2024-07-03T15:53:01.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.457+0000 7f5e781b7640 1 -- start start 2024-07-03T15:53:01.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.457+0000 7f5e781b7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e701012b0 0x7f5e70113140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:53:01.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.457+0000 7f5e781b7640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e70101c50 0x7f5e70113680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:53:01.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.457+0000 7f5e781b7640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e70113c80 con 0x7f5e70101c50 2024-07-03T15:53:01.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.457+0000 7f5e781b7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7010d0c0 con 0x7f5e701012b0 2024-07-03T15:53:01.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.457+0000 7f5e75752640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e70101c50 0x7f5e70113680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:53:01.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.457+0000 7f5e75752640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e70101c50 0x7f5e70113680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:60068/0 (socket says 172.21.15.28:60068) 2024-07-03T15:53:01.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.457+0000 7f5e75752640 1 -- 172.21.15.28:0/331652902 learned_addr learned my addr 172.21.15.28:0/331652902 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:53:01.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.457+0000 7f5e75f53640 1 --2- 172.21.15.28:0/331652902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e701012b0 0x7f5e70113140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:53:01.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.457+0000 7f5e75752640 1 -- 172.21.15.28:0/331652902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e701012b0 msgr2=0x7f5e70113140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:53:01.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.457+0000 7f5e75752640 1 --2- 172.21.15.28:0/331652902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e701012b0 0x7f5e70113140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:53:01.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.457+0000 7f5e75752640 1 -- 172.21.15.28:0/331652902 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e60005040 con 0x7f5e70101c50 2024-07-03T15:53:01.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.458+0000 7f5e75752640 1 --2- 172.21.15.28:0/331652902 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e70101c50 0x7f5e70113680 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5e600043d0 tx=0x7f5e60009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:53:01.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.458+0000 7f5e667fc640 1 -- 172.21.15.28:0/331652902 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e60038070 con 0x7f5e70101c50 2024-07-03T15:53:01.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.458+0000 7f5e667fc640 1 -- 172.21.15.28:0/331652902 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5e60032970 con 0x7f5e70101c50 2024-07-03T15:53:01.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.458+0000 7f5e781b7640 1 -- 172.21.15.28:0/331652902 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e7010d340 con 0x7f5e70101c50 2024-07-03T15:53:01.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.458+0000 7f5e667fc640 1 -- 172.21.15.28:0/331652902 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e6003da60 con 0x7f5e70101c50 2024-07-03T15:53:01.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.458+0000 7f5e781b7640 1 -- 172.21.15.28:0/331652902 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e7010d800 con 0x7f5e70101c50 2024-07-03T15:53:01.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.459+0000 7f5e781b7640 1 -- 172.21.15.28:0/331652902 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e700fca20 con 0x7f5e70101c50 2024-07-03T15:53:01.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.459+0000 7f5e667fc640 1 -- 172.21.15.28:0/331652902 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5e60030020 con 0x7f5e70101c50 2024-07-03T15:53:01.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.460+0000 7f5e667fc640 1 --2- 172.21.15.28:0/331652902 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f5e5c06e0b0 0x7f5e5c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:53:01.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.460+0000 7f5e75f53640 1 --2- 172.21.15.28:0/331652902 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f5e5c06e0b0 0x7f5e5c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:53:01.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.460+0000 7f5e667fc640 1 -- 172.21.15.28:0/331652902 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f5e600adcc0 con 0x7f5e70101c50 2024-07-03T15:53:01.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.461+0000 7f5e75f53640 1 --2- 172.21.15.28:0/331652902 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f5e5c06e0b0 0x7f5e5c070560 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c007fd0 tx=0x7f5e6c009e90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:53:01.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.465+0000 7f5e667fc640 1 -- 172.21.15.28:0/331652902 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f5e60076f90 con 0x7f5e70101c50 2024-07-03T15:53:01.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.666+0000 7f5e781b7640 1 -- 172.21.15.28:0/331652902 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e70108410 con 0x7f5e5c06e0b0 2024-07-03T15:53:01.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.667+0000 7f5e667fc640 1 -- 172.21.15.28:0/331652902 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f5e70108410 con 0x7f5e5c06e0b0 2024-07-03T15:53:01.671 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-07-03T15:53:01.671 INFO:teuthology.orchestra.run.smithi028.stdout: "target_image": null, 2024-07-03T15:53:01.671 INFO:teuthology.orchestra.run.smithi028.stdout: "in_progress": false, 2024-07-03T15:53:01.671 INFO:teuthology.orchestra.run.smithi028.stdout: "which": "", 2024-07-03T15:53:01.671 INFO:teuthology.orchestra.run.smithi028.stdout: "services_complete": [], 2024-07-03T15:53:01.671 INFO:teuthology.orchestra.run.smithi028.stdout: "progress": null, 2024-07-03T15:53:01.671 INFO:teuthology.orchestra.run.smithi028.stdout: "message": "", 2024-07-03T15:53:01.671 INFO:teuthology.orchestra.run.smithi028.stdout: "is_paused": false 2024-07-03T15:53:01.672 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-07-03T15:53:01.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.670+0000 7f5e781b7640 1 -- 172.21.15.28:0/331652902 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f5e5c06e0b0 msgr2=0x7f5e5c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:53:01.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.670+0000 7f5e781b7640 1 --2- 172.21.15.28:0/331652902 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f5e5c06e0b0 0x7f5e5c070560 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c007fd0 tx=0x7f5e6c009e90 comp rx=0 tx=0).stop 2024-07-03T15:53:01.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.670+0000 7f5e781b7640 1 -- 172.21.15.28:0/331652902 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e70101c50 msgr2=0x7f5e70113680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:53:01.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.670+0000 7f5e781b7640 1 --2- 172.21.15.28:0/331652902 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e70101c50 0x7f5e70113680 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5e600043d0 tx=0x7f5e60009da0 comp rx=0 tx=0).stop 2024-07-03T15:53:01.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.670+0000 7f5e781b7640 1 -- 172.21.15.28:0/331652902 shutdown_connections 2024-07-03T15:53:01.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.670+0000 7f5e781b7640 1 --2- 172.21.15.28:0/331652902 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f5e5c06e0b0 0x7f5e5c070560 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:53:01.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.670+0000 7f5e781b7640 1 --2- 172.21.15.28:0/331652902 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e70101c50 0x7f5e70113680 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:53:01.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.670+0000 7f5e781b7640 1 --2- 172.21.15.28:0/331652902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e701012b0 0x7f5e70113140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:53:01.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.670+0000 7f5e781b7640 1 -- 172.21.15.28:0/331652902 >> 172.21.15.28:0/331652902 conn(0x7f5e700f6ea0 msgr2=0x7f5e700f7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:53:01.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.670+0000 7f5e781b7640 1 -- 172.21.15.28:0/331652902 shutdown_connections 2024-07-03T15:53:01.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:01.670+0000 7f5e781b7640 1 -- 172.21.15.28:0/331652902 wait complete. 2024-07-03T15:53:02.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:53:01 smithi028 ceph-mon[92858]: pgmap v236: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:53:02.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:53:01 smithi148 ceph-mon[80490]: pgmap v236: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:53:02.294 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -e sha1=c00506dc10003928f8cb40769168571f611cbd67 -- bash -c 'ceph health detail' 2024-07-03T15:53:02.562 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:53:03.003 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:53:02 smithi028 ceph-mon[92858]: from='client.34510 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:53:03.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:53:02 smithi148 ceph-mon[80490]: from='client.34510 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:53:03.960 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:53:03 smithi028 ceph-mon[92858]: pgmap v237: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:53:04.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.183+0000 7f7dc5630640 1 -- 172.21.15.28:0/2605773252 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7dc00fdf60 msgr2=0x7f7dc00fe390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:53:04.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.183+0000 7f7dc5630640 1 --2- 172.21.15.28:0/2605773252 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7dc00fdf60 0x7f7dc00fe390 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7db40048d0 tx=0x7f7db4009da0 comp rx=0 tx=0).stop 2024-07-03T15:53:04.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.183+0000 7f7dc5630640 1 -- 172.21.15.28:0/2605773252 shutdown_connections 2024-07-03T15:53:04.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.183+0000 7f7dc5630640 1 --2- 172.21.15.28:0/2605773252 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7dc00fdf60 0x7f7dc00fe390 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:53:04.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.183+0000 7f7dc5630640 1 --2- 172.21.15.28:0/2605773252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7dc0065150 0x7f7dc0065520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:53:04.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.183+0000 7f7dc5630640 1 -- 172.21.15.28:0/2605773252 >> 172.21.15.28:0/2605773252 conn(0x7f7dc00721c0 msgr2=0x7f7dc00725c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:53:04.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.183+0000 7f7dc5630640 1 -- 172.21.15.28:0/2605773252 shutdown_connections 2024-07-03T15:53:04.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.183+0000 7f7dc5630640 1 -- 172.21.15.28:0/2605773252 wait complete. 2024-07-03T15:53:04.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.183+0000 7f7dc5630640 1 Processor -- start 2024-07-03T15:53:04.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.184+0000 7f7dc5630640 1 -- start start 2024-07-03T15:53:04.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.184+0000 7f7dc5630640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7dc0065150 0x7f7dc01130a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:53:04.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.184+0000 7f7dc5630640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7dc00fdf60 0x7f7dc01135e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:53:04.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.184+0000 7f7dc5630640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc0113be0 con 0x7f7dc0065150 2024-07-03T15:53:04.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.184+0000 7f7dc5630640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc010d020 con 0x7f7dc00fdf60 2024-07-03T15:53:04.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.184+0000 7f7dbeffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7dc0065150 0x7f7dc01130a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:53:04.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.184+0000 7f7dbeffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7dc0065150 0x7f7dc01130a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:60086/0 (socket says 172.21.15.28:60086) 2024-07-03T15:53:04.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.184+0000 7f7dbeffd640 1 -- 172.21.15.28:0/1134786962 learned_addr learned my addr 172.21.15.28:0/1134786962 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:53:04.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.184+0000 7f7dbe7fc640 1 --2- 172.21.15.28:0/1134786962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7dc00fdf60 0x7f7dc01135e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:53:04.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.184+0000 7f7dbeffd640 1 -- 172.21.15.28:0/1134786962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7dc00fdf60 msgr2=0x7f7dc01135e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:53:04.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.184+0000 7f7dbeffd640 1 --2- 172.21.15.28:0/1134786962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7dc00fdf60 0x7f7dc01135e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:53:04.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.184+0000 7f7dbeffd640 1 -- 172.21.15.28:0/1134786962 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db4005040 con 0x7f7dc0065150 2024-07-03T15:53:04.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.184+0000 7f7dbe7fc640 1 --2- 172.21.15.28:0/1134786962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7dc00fdf60 0x7f7dc01135e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-03T15:53:04.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.185+0000 7f7dbeffd640 1 --2- 172.21.15.28:0/1134786962 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7dc0065150 0x7f7dc01130a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7db000c830 tx=0x7f7db0004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:53:04.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.185+0000 7f7da77fe640 1 -- 172.21.15.28:0/1134786962 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7db000a820 con 0x7f7dc0065150 2024-07-03T15:53:04.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.185+0000 7f7dc5630640 1 -- 172.21.15.28:0/1134786962 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dc010d300 con 0x7f7dc0065150 2024-07-03T15:53:04.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.185+0000 7f7da77fe640 1 -- 172.21.15.28:0/1134786962 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7db000ae50 con 0x7f7dc0065150 2024-07-03T15:53:04.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.185+0000 7f7da77fe640 1 -- 172.21.15.28:0/1134786962 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7db00115e0 con 0x7f7dc0065150 2024-07-03T15:53:04.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.185+0000 7f7dc5630640 1 -- 172.21.15.28:0/1134786962 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dc010d8a0 con 0x7f7dc0065150 2024-07-03T15:53:04.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.186+0000 7f7dc5630640 1 -- 172.21.15.28:0/1134786962 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dc00fdb00 con 0x7f7dc0065150 2024-07-03T15:53:04.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.187+0000 7f7da77fe640 1 -- 172.21.15.28:0/1134786962 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7db0012400 con 0x7f7dc0065150 2024-07-03T15:53:04.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.187+0000 7f7da77fe640 1 --2- 172.21.15.28:0/1134786962 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f7da806e0b0 0x7f7da8070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:53:04.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.187+0000 7f7da77fe640 1 -- 172.21.15.28:0/1134786962 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f7db008dd80 con 0x7f7dc0065150 2024-07-03T15:53:04.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.187+0000 7f7dbe7fc640 1 --2- 172.21.15.28:0/1134786962 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f7da806e0b0 0x7f7da8070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:53:04.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.188+0000 7f7dbe7fc640 1 --2- 172.21.15.28:0/1134786962 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f7da806e0b0 0x7f7da8070560 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7db4004540 tx=0x7f7db4009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:53:04.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.191+0000 7f7da77fe640 1 -- 172.21.15.28:0/1134786962 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f7db00570d0 con 0x7f7dc0065150 2024-07-03T15:53:04.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:53:03 smithi148 ceph-mon[80490]: pgmap v237: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:53:04.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.448+0000 7f7dc5630640 1 -- 172.21.15.28:0/1134786962 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7dc0072cd0 con 0x7f7dc0065150 2024-07-03T15:53:04.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.449+0000 7f7da77fe640 1 -- 172.21.15.28:0/1134786962 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f7db005a790 con 0x7f7dc0065150 2024-07-03T15:53:04.453 INFO:teuthology.orchestra.run.smithi028.stdout:HEALTH_OK 2024-07-03T15:53:04.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.451+0000 7f7dc5630640 1 -- 172.21.15.28:0/1134786962 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f7da806e0b0 msgr2=0x7f7da8070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:53:04.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.451+0000 7f7dc5630640 1 --2- 172.21.15.28:0/1134786962 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f7da806e0b0 0x7f7da8070560 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7db4004540 tx=0x7f7db4009da0 comp rx=0 tx=0).stop 2024-07-03T15:53:04.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.451+0000 7f7dc5630640 1 -- 172.21.15.28:0/1134786962 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7dc0065150 msgr2=0x7f7dc01130a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:53:04.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.451+0000 7f7dc5630640 1 --2- 172.21.15.28:0/1134786962 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7dc0065150 0x7f7dc01130a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7db000c830 tx=0x7f7db0004430 comp rx=0 tx=0).stop 2024-07-03T15:53:04.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.451+0000 7f7dc5630640 1 -- 172.21.15.28:0/1134786962 shutdown_connections 2024-07-03T15:53:04.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.451+0000 7f7dc5630640 1 --2- 172.21.15.28:0/1134786962 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f7da806e0b0 0x7f7da8070560 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:53:04.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.451+0000 7f7dc5630640 1 --2- 172.21.15.28:0/1134786962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7dc00fdf60 0x7f7dc01135e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:53:04.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.451+0000 7f7dc5630640 1 --2- 172.21.15.28:0/1134786962 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7dc0065150 0x7f7dc01130a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:53:04.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.451+0000 7f7dc5630640 1 -- 172.21.15.28:0/1134786962 >> 172.21.15.28:0/1134786962 conn(0x7f7dc00721c0 msgr2=0x7f7dc0101080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:53:04.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.451+0000 7f7dc5630640 1 -- 172.21.15.28:0/1134786962 shutdown_connections 2024-07-03T15:53:04.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:04.451+0000 7f7dc5630640 1 -- 172.21.15.28:0/1134786962 wait complete. 2024-07-03T15:53:05.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:53:04 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/1134786962' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-03T15:53:05.148 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -e sha1=c00506dc10003928f8cb40769168571f611cbd67 -- bash -c 'ceph versions' 2024-07-03T15:53:05.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:53:04 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/1134786962' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-03T15:53:05.416 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:53:05.868 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:53:05 smithi028 ceph-mon[92858]: pgmap v238: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:53:06.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:53:05 smithi148 ceph-mon[80490]: pgmap v238: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:53:07.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:07.105+0000 7f043e7cf640 1 -- 172.21.15.28:0/2883503488 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0438101250 msgr2=0x7f0438101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:53:07.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:07.105+0000 7f043e7cf640 1 --2- 172.21.15.28:0/2883503488 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0438101250 0x7f0438101620 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f04280048d0 tx=0x7f0428009da0 comp rx=0 tx=0).stop 2024-07-03T15:53:07.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:07.105+0000 7f043e7cf640 1 -- 172.21.15.28:0/2883503488 shutdown_connections 2024-07-03T15:53:07.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:07.105+0000 7f043e7cf640 1 --2- 172.21.15.28:0/2883503488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0438101bf0 0x7f04380fb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:53:07.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:07.105+0000 7f043e7cf640 1 --2- 172.21.15.28:0/2883503488 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0438101250 0x7f0438101620 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:53:07.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:07.105+0000 7f043e7cf640 1 -- 172.21.15.28:0/2883503488 >> 172.21.15.28:0/2883503488 conn(0x7f04380f6e40 msgr2=0x7f04380f9250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:53:07.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:07.105+0000 7f043e7cf640 1 -- 172.21.15.28:0/2883503488 shutdown_connections 2024-07-03T15:53:07.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:07.105+0000 7f043e7cf640 1 -- 172.21.15.28:0/2883503488 wait complete. 2024-07-03T15:53:07.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:07.105+0000 7f043e7cf640 1 Processor -- start 2024-07-03T15:53:07.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:07.106+0000 7f043e7cf640 1 -- start start 2024-07-03T15:53:07.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:07.106+0000 7f043e7cf640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0438101250 0x7f043810e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:53:07.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:07.106+0000 7f043e7cf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0438101bf0 0x7f043810e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:53:07.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:07.106+0000 7f043e7cf640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043810eda0 con 0x7f0438101250 2024-07-03T15:53:07.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:07.106+0000 7f043e7cf640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043810ef10 con 0x7f0438101bf0 2024-07-03T15:53:07.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:07.106+0000 7f0437fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0438101250 0x7f043810e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:53:07.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:07.106+0000 7f0437fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0438101250 0x7f043810e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:60108/0 (socket says 172.21.15.28:60108) 2024-07-03T15:53:07.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:07.106+0000 7f0437fff640 1 -- 172.21.15.28:0/1790842795 learned_addr learned my addr 172.21.15.28:0/1790842795 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:53:07.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:07.106+0000 7f0437fff640 1 -- 172.21.15.28:0/1790842795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0438101bf0 msgr2=0x7f043810e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:53:07.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:07.106+0000 7f0437fff640 1 --2- 172.21.15.28:0/1790842795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0438101bf0 0x7f043810e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:53:07.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:07.106+0000 7f0437fff640 1 -- 172.21.15.28:0/1790842795 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0428005040 con 0x7f0438101250 2024-07-03T15:53:07.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:07.106+0000 7f0437fff640 1 --2- 172.21.15.28:0/1790842795 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0438101250 0x7f043810e160 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0428003080 tx=0x7f0428009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:53:07.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:07.107+0000 7f0434ff9640 1 -- 172.21.15.28:0/1790842795 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0428038070 con 0x7f0438101250 2024-07-03T15:53:07.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:07.107+0000 7f043e7cf640 1 -- 172.21.15.28:0/1790842795 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0438117230 con 0x7f0438101250 2024-07-03T15:53:07.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:07.107+0000 7f0434ff9640 1 -- 172.21.15.28:0/1790842795 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0428008ee0 con 0x7f0438101250 2024-07-03T15:53:07.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:07.107+0000 7f0434ff9640 1 -- 172.21.15.28:0/1790842795 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f042803d870 con 0x7f0438101250 2024-07-03T15:53:07.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:07.107+0000 7f043e7cf640 1 -- 172.21.15.28:0/1790842795 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04381176c0 con 0x7f0438101250 2024-07-03T15:53:07.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:07.108+0000 7f043e7cf640 1 -- 172.21.15.28:0/1790842795 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04380fc9c0 con 0x7f0438101250 2024-07-03T15:53:07.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:07.108+0000 7f0434ff9640 1 -- 172.21.15.28:0/1790842795 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0428030020 con 0x7f0438101250 2024-07-03T15:53:07.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:07.109+0000 7f0434ff9640 1 --2- 172.21.15.28:0/1790842795 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f042006e0b0 0x7f0420070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:53:07.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:07.109+0000 7f0434ff9640 1 -- 172.21.15.28:0/1790842795 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f04280ad8f0 con 0x7f0438101250 2024-07-03T15:53:07.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:07.109+0000 7f04377fe640 1 --2- 172.21.15.28:0/1790842795 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f042006e0b0 0x7f0420070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:53:07.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:07.109+0000 7f04377fe640 1 --2- 172.21.15.28:0/1790842795 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f042006e0b0 0x7f0420070560 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0438066020 tx=0x7f042c006d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:53:07.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:07.113+0000 7f0434ff9640 1 -- 172.21.15.28:0/1790842795 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0428076c40 con 0x7f0438101250 2024-07-03T15:53:07.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:07.370+0000 7f043e7cf640 1 -- 172.21.15.28:0/1790842795 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f043804b530 con 0x7f0438101250 2024-07-03T15:53:07.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:07.371+0000 7f0434ff9640 1 -- 172.21.15.28:0/1790842795 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7f042807a300 con 0x7f0438101250 2024-07-03T15:53:07.375 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-07-03T15:53:07.375 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": { 2024-07-03T15:53:07.375 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 2 2024-07-03T15:53:07.375 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:53:07.375 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": { 2024-07-03T15:53:07.375 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 2 2024-07-03T15:53:07.375 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:53:07.375 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": { 2024-07-03T15:53:07.375 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 6 2024-07-03T15:53:07.376 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:53:07.376 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": { 2024-07-03T15:53:07.376 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 4 2024-07-03T15:53:07.376 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:53:07.376 INFO:teuthology.orchestra.run.smithi028.stdout: "overall": { 2024-07-03T15:53:07.376 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 14 2024-07-03T15:53:07.376 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-07-03T15:53:07.376 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-07-03T15:53:07.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:07.373+0000 7f043e7cf640 1 -- 172.21.15.28:0/1790842795 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f042006e0b0 msgr2=0x7f0420070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:53:07.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:07.373+0000 7f043e7cf640 1 --2- 172.21.15.28:0/1790842795 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f042006e0b0 0x7f0420070560 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0438066020 tx=0x7f042c006d10 comp rx=0 tx=0).stop 2024-07-03T15:53:07.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:07.373+0000 7f043e7cf640 1 -- 172.21.15.28:0/1790842795 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0438101250 msgr2=0x7f043810e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:53:07.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:07.373+0000 7f043e7cf640 1 --2- 172.21.15.28:0/1790842795 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0438101250 0x7f043810e160 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0428003080 tx=0x7f0428009da0 comp rx=0 tx=0).stop 2024-07-03T15:53:07.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:07.373+0000 7f043e7cf640 1 -- 172.21.15.28:0/1790842795 shutdown_connections 2024-07-03T15:53:07.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:07.373+0000 7f043e7cf640 1 --2- 172.21.15.28:0/1790842795 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f042006e0b0 0x7f0420070560 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:53:07.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:07.373+0000 7f043e7cf640 1 --2- 172.21.15.28:0/1790842795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0438101bf0 0x7f043810e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:53:07.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:07.373+0000 7f043e7cf640 1 --2- 172.21.15.28:0/1790842795 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0438101250 0x7f043810e160 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:53:07.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:07.373+0000 7f043e7cf640 1 -- 172.21.15.28:0/1790842795 >> 172.21.15.28:0/1790842795 conn(0x7f04380f6e40 msgr2=0x7f04380f7dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:53:07.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:07.373+0000 7f043e7cf640 1 -- 172.21.15.28:0/1790842795 shutdown_connections 2024-07-03T15:53:07.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:53:07.373+0000 7f043e7cf640 1 -- 172.21.15.28:0/1790842795 wait complete. 2024-07-03T15:53:08.037 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:53:07 smithi028 ceph-mon[92858]: pgmap v239: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:53:08.038 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:53:07 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:53:08.038 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:53:07 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/1790842795' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:53:08.068 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -e sha1=c00506dc10003928f8cb40769168571f611cbd67 -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-07-03T15:53:08.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:53:07 smithi148 ceph-mon[80490]: pgmap v239: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:53:08.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:53:07 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:53:08.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:53:07 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/1790842795' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:53:08.339 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:53:09.980 INFO:teuthology.orchestra.run.smithi028.stdout:wait for servicemap items w/ changing names to refresh 2024-07-03T15:53:09.981 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:53:09 smithi028 ceph-mon[92858]: pgmap v240: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:53:10.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:53:09 smithi148 ceph-mon[80490]: pgmap v240: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:53:10.625 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -e sha1=c00506dc10003928f8cb40769168571f611cbd67 -- bash -c 'sleep 60' 2024-07-03T15:53:10.901 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:53:12.046 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:53:11 smithi028 ceph-mon[92858]: pgmap v241: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:53:12.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:53:11 smithi148 ceph-mon[80490]: pgmap v241: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:53:13.910 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:53:13 smithi028 ceph-mon[92858]: pgmap v242: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:53:14.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:53:13 smithi148 ceph-mon[80490]: pgmap v242: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:53:15.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:53:15 smithi028 ceph-mon[92858]: pgmap v243: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:53:16.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:53:15 smithi148 ceph-mon[80490]: pgmap v243: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:53:18.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:53:17 smithi028 ceph-mon[92858]: pgmap v244: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:53:18.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:53:17 smithi148 ceph-mon[80490]: pgmap v244: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:53:20.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:53:19 smithi028 ceph-mon[92858]: pgmap v245: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:53:20.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:53:19 smithi148 ceph-mon[80490]: pgmap v245: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:53:22.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:53:21 smithi028 ceph-mon[92858]: pgmap v246: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:53:22.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:53:21 smithi148 ceph-mon[80490]: pgmap v246: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:53:23.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:53:22 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:53:23.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:53:22 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:53:24.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:53:23 smithi028 ceph-mon[92858]: pgmap v247: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:53:24.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:53:23 smithi148 ceph-mon[80490]: pgmap v247: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:53:25.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:53:25 smithi028 ceph-mon[92858]: pgmap v248: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:53:26.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:53:25 smithi148 ceph-mon[80490]: pgmap v248: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:53:28.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:53:27 smithi028 ceph-mon[92858]: pgmap v249: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:53:28.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:53:27 smithi148 ceph-mon[80490]: pgmap v249: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:53:30.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:53:29 smithi028 ceph-mon[92858]: pgmap v250: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:53:30.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:53:29 smithi148 ceph-mon[80490]: pgmap v250: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:53:32.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:53:31 smithi028 ceph-mon[92858]: pgmap v251: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:53:32.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:53:31 smithi148 ceph-mon[80490]: pgmap v251: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:53:34.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:53:33 smithi028 ceph-mon[92858]: pgmap v252: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:53:34.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:53:33 smithi148 ceph-mon[80490]: pgmap v252: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:53:35.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:53:35 smithi028 ceph-mon[92858]: pgmap v253: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:53:36.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:53:35 smithi148 ceph-mon[80490]: pgmap v253: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:53:38.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:53:37 smithi028 ceph-mon[92858]: pgmap v254: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:53:38.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:53:37 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:53:38.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:53:37 smithi148 ceph-mon[80490]: pgmap v254: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:53:38.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:53:37 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:53:40.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:53:39 smithi028 ceph-mon[92858]: pgmap v255: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:53:40.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:53:39 smithi148 ceph-mon[80490]: pgmap v255: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:53:42.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:53:41 smithi028 ceph-mon[92858]: pgmap v256: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:53:42.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:53:41 smithi148 ceph-mon[80490]: pgmap v256: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:53:44.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:53:43 smithi028 ceph-mon[92858]: pgmap v257: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:53:44.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:53:43 smithi148 ceph-mon[80490]: pgmap v257: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:53:45.872 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:53:45 smithi028 ceph-mon[92858]: pgmap v258: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:53:46.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:53:45 smithi148 ceph-mon[80490]: pgmap v258: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:53:48.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:53:47 smithi028 ceph-mon[92858]: pgmap v259: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:53:48.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:53:47 smithi148 ceph-mon[80490]: pgmap v259: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:53:50.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:53:49 smithi028 ceph-mon[92858]: pgmap v260: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:53:50.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:53:49 smithi148 ceph-mon[80490]: pgmap v260: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:53:51.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:53:50 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:53:51.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:53:50 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:53:51.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:53:50 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:53:51.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:53:50 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:53:51.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:53:50 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-03T15:53:51.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:53:50 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-03T15:53:51.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:53:50 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-03T15:53:51.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:53:50 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' 2024-07-03T15:53:52.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:53:51 smithi028 ceph-mon[92858]: pgmap v261: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:53:52.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:53:51 smithi148 ceph-mon[80490]: pgmap v261: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:53:53.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:53:52 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:53:53.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:53:52 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:53:54.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:53:53 smithi028 ceph-mon[92858]: pgmap v262: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:53:54.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:53:53 smithi148 ceph-mon[80490]: pgmap v262: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:53:56.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:53:55 smithi028 ceph-mon[92858]: pgmap v263: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:53:56.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:53:55 smithi148 ceph-mon[80490]: pgmap v263: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:53:58.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:53:57 smithi028 ceph-mon[92858]: pgmap v264: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:53:58.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:53:57 smithi148 ceph-mon[80490]: pgmap v264: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:54:00.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:53:59 smithi028 ceph-mon[92858]: pgmap v265: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:54:00.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:53:59 smithi148 ceph-mon[80490]: pgmap v265: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:54:02.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:01 smithi148 ceph-mon[80490]: pgmap v266: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:54:02.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:01 smithi028 ceph-mon[92858]: pgmap v266: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:54:04.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:03 smithi148 ceph-mon[80490]: pgmap v267: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:54:04.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:03 smithi028 ceph-mon[92858]: pgmap v267: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:54:06.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:05 smithi148 ceph-mon[80490]: pgmap v268: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:54:06.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:05 smithi028 ceph-mon[92858]: pgmap v268: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:54:08.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:07 smithi148 ceph-mon[80490]: pgmap v269: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:54:08.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:07 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:54:08.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:07 smithi028 ceph-mon[92858]: pgmap v269: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:54:08.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:07 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:54:10.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:09 smithi148 ceph-mon[80490]: pgmap v270: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:54:10.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:09 smithi028 ceph-mon[92858]: pgmap v270: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:54:12.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:11 smithi148 ceph-mon[80490]: pgmap v271: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:54:12.371 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:11 smithi028 ceph-mon[92858]: pgmap v271: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:54:13.170 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -e sha1=c00506dc10003928f8cb40769168571f611cbd67 -- bash -c 'ceph orch ps' 2024-07-03T15:54:13.447 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:54:14.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:13 smithi028 ceph-mon[92858]: pgmap v272: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:54:14.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:13 smithi148 ceph-mon[80490]: pgmap v272: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:54:15.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.025+0000 7f51bd9de640 1 -- 172.21.15.28:0/3975273073 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f51b81012b0 msgr2=0x7f51b8101680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:15.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.025+0000 7f51bd9de640 1 --2- 172.21.15.28:0/3975273073 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f51b81012b0 0x7f51b8101680 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f51a80048d0 tx=0x7f51a8009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:15.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.025+0000 7f51bd9de640 1 -- 172.21.15.28:0/3975273073 shutdown_connections 2024-07-03T15:54:15.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.025+0000 7f51bd9de640 1 --2- 172.21.15.28:0/3975273073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51b8101c50 0x7f51b80fb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:15.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.025+0000 7f51bd9de640 1 --2- 172.21.15.28:0/3975273073 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f51b81012b0 0x7f51b8101680 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:15.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.025+0000 7f51bd9de640 1 -- 172.21.15.28:0/3975273073 >> 172.21.15.28:0/3975273073 conn(0x7f51b80f6ea0 msgr2=0x7f51b80f92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:15.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.025+0000 7f51bd9de640 1 -- 172.21.15.28:0/3975273073 shutdown_connections 2024-07-03T15:54:15.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.026+0000 7f51bd9de640 1 -- 172.21.15.28:0/3975273073 wait complete. 2024-07-03T15:54:15.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.026+0000 7f51bd9de640 1 Processor -- start 2024-07-03T15:54:15.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.026+0000 7f51bd9de640 1 -- start start 2024-07-03T15:54:15.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.026+0000 7f51bd9de640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f51b81012b0 0x7f51b810be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:15.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.026+0000 7f51bd9de640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51b8101c50 0x7f51b810c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:15.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.026+0000 7f51bd9de640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b810c980 con 0x7f51b81012b0 2024-07-03T15:54:15.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.026+0000 7f51bd9de640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b810cac0 con 0x7f51b8101c50 2024-07-03T15:54:15.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.026+0000 7f51b6ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f51b81012b0 0x7f51b810be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:15.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.027+0000 7f51b6ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f51b81012b0 0x7f51b810be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51606/0 (socket says 172.21.15.28:51606) 2024-07-03T15:54:15.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.027+0000 7f51b6ffd640 1 -- 172.21.15.28:0/150626604 learned_addr learned my addr 172.21.15.28:0/150626604 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:54:15.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.027+0000 7f51ae7fc640 1 --2- 172.21.15.28:0/150626604 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51b8101c50 0x7f51b810c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:15.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.027+0000 7f51b6ffd640 1 -- 172.21.15.28:0/150626604 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51b8101c50 msgr2=0x7f51b810c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:15.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.027+0000 7f51b6ffd640 1 --2- 172.21.15.28:0/150626604 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51b8101c50 0x7f51b810c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:15.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.027+0000 7f51b6ffd640 1 -- 172.21.15.28:0/150626604 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a8005040 con 0x7f51b81012b0 2024-07-03T15:54:15.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.027+0000 7f51b6ffd640 1 --2- 172.21.15.28:0/150626604 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f51b81012b0 0x7f51b810be70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f51a8003080 tx=0x7f51a8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:15.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.027+0000 7f51affff640 1 -- 172.21.15.28:0/150626604 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51a8038070 con 0x7f51b81012b0 2024-07-03T15:54:15.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.027+0000 7f51affff640 1 -- 172.21.15.28:0/150626604 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f51a8008ee0 con 0x7f51b81012b0 2024-07-03T15:54:15.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.027+0000 7f51bd9de640 1 -- 172.21.15.28:0/150626604 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51b81194e0 con 0x7f51b81012b0 2024-07-03T15:54:15.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.027+0000 7f51affff640 1 -- 172.21.15.28:0/150626604 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51a803d870 con 0x7f51b81012b0 2024-07-03T15:54:15.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.027+0000 7f51bd9de640 1 -- 172.21.15.28:0/150626604 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51b8119810 con 0x7f51b81012b0 2024-07-03T15:54:15.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.028+0000 7f51bd9de640 1 -- 172.21.15.28:0/150626604 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51b80fca20 con 0x7f51b81012b0 2024-07-03T15:54:15.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.029+0000 7f51affff640 1 -- 172.21.15.28:0/150626604 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f51a8030020 con 0x7f51b81012b0 2024-07-03T15:54:15.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.029+0000 7f51affff640 1 --2- 172.21.15.28:0/150626604 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f51a406e0b0 0x7f51a4070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:15.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.030+0000 7f51affff640 1 -- 172.21.15.28:0/150626604 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f51a80ad980 con 0x7f51b81012b0 2024-07-03T15:54:15.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.030+0000 7f51ae7fc640 1 --2- 172.21.15.28:0/150626604 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f51a406e0b0 0x7f51a4070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:15.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.030+0000 7f51ae7fc640 1 --2- 172.21.15.28:0/150626604 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f51a406e0b0 0x7f51a4070560 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f51b8066330 tx=0x7f51a0008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:15.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.034+0000 7f51affff640 1 -- 172.21.15.28:0/150626604 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f51a8076cd0 con 0x7f51b81012b0 2024-07-03T15:54:15.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.228+0000 7f51bd9de640 1 -- 172.21.15.28:0/150626604 --> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f51b8066030 con 0x7f51a406e0b0 2024-07-03T15:54:15.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.237+0000 7f51affff640 1 -- 172.21.15.28:0/150626604 <== mgr.34104 v2:172.21.15.28:6800/1205434373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f51b8066030 con 0x7f51a406e0b0 2024-07-03T15:54:15.241 INFO:teuthology.orchestra.run.smithi028.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-03T15:54:15.241 INFO:teuthology.orchestra.run.smithi028.stdout:alertmanager.smithi028 smithi028 *:9093,9094 running (115s) 90s ago 17m 16.5M - 0.25.0 c8568f914cd2 f997825b72e3 2024-07-03T15:54:15.241 INFO:teuthology.orchestra.run.smithi028.stdout:crash.smithi028 smithi028 running (7m) 90s ago 18m 6895k - 19.0.0-2550-gc00506dc 9d49405603e8 c62c2f45cabe 2024-07-03T15:54:15.241 INFO:teuthology.orchestra.run.smithi028.stdout:crash.smithi148 smithi148 running (7m) 2m ago 17m 6895k - 19.0.0-2550-gc00506dc 9d49405603e8 b1318dcaacb6 2024-07-03T15:54:15.241 INFO:teuthology.orchestra.run.smithi028.stdout:grafana.smithi028 smithi028 *:3000 running (94s) 90s ago 17m 44.8M - 9.4.12 2bacad6d85d8 c7e25b78cdeb 2024-07-03T15:54:15.241 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi028.rhpvve smithi028 running (3m) 90s ago 12m 84.3M - 19.0.0-2550-gc00506dc 9d49405603e8 5d528ea2a2f0 2024-07-03T15:54:15.241 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi028.wmbxew smithi028 running (3m) 90s ago 12m 82.0M - 19.0.0-2550-gc00506dc 9d49405603e8 ebdbfa592ac8 2024-07-03T15:54:15.241 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi148.szomfl smithi148 running (2m) 2m ago 12m 13.0M - 19.0.0-2550-gc00506dc 9d49405603e8 27650f9fb39f 2024-07-03T15:54:15.241 INFO:teuthology.orchestra.run.smithi028.stdout:mds.cephfs.smithi148.xgpuld smithi148 running (3m) 2m ago 12m 14.6M - 19.0.0-2550-gc00506dc 9d49405603e8 f24ab04507dc 2024-07-03T15:54:15.242 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.smithi028.vtqrtz smithi028 *:8443,9283,8765 running (9m) 90s ago 19m 515M - 19.0.0-2550-gc00506dc 9d49405603e8 14d0363c5b77 2024-07-03T15:54:15.242 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.smithi148.rjzsar smithi148 *:8443,9283,8765 running (8m) 2m ago 16m 441M - 19.0.0-2550-gc00506dc 9d49405603e8 c0193aa8a5dd 2024-07-03T15:54:15.242 INFO:teuthology.orchestra.run.smithi028.stdout:mon.smithi028 smithi028 running (8m) 90s ago 19m 53.7M 2048M 19.0.0-2550-gc00506dc 9d49405603e8 5093f7bf47d7 2024-07-03T15:54:15.242 INFO:teuthology.orchestra.run.smithi028.stdout:mon.smithi148 smithi148 running (7m) 2m ago 16m 38.2M 2048M 19.0.0-2550-gc00506dc 9d49405603e8 d338d2c648da 2024-07-03T15:54:15.242 INFO:teuthology.orchestra.run.smithi028.stdout:node-exporter.smithi028 smithi028 *:9100 running (2m) 90s ago 17m 8484k - 1.5.0 0da6a335fe13 125dcba51db4 2024-07-03T15:54:15.242 INFO:teuthology.orchestra.run.smithi028.stdout:node-exporter.smithi148 smithi148 *:9100 running (2m) 2m ago 16m 3577k - 1.5.0 0da6a335fe13 edc710a27078 2024-07-03T15:54:15.242 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0 smithi028 running (6m) 90s ago 16m 111M 1558M 19.0.0-2550-gc00506dc 9d49405603e8 4454cecff8fc 2024-07-03T15:54:15.242 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1 smithi028 running (6m) 90s ago 15m 117M 1558M 19.0.0-2550-gc00506dc 9d49405603e8 af67f7088643 2024-07-03T15:54:15.242 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2 smithi028 running (5m) 90s ago 15m 101M 1558M 19.0.0-2550-gc00506dc 9d49405603e8 5d160a327dc2 2024-07-03T15:54:15.242 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3 smithi148 running (4m) 2m ago 14m 120M 2581M 19.0.0-2550-gc00506dc 9d49405603e8 1d622eba4b3b 2024-07-03T15:54:15.242 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4 smithi148 running (4m) 2m ago 14m 106M 2581M 19.0.0-2550-gc00506dc 9d49405603e8 546343d0f31f 2024-07-03T15:54:15.242 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5 smithi148 running (3m) 2m ago 14m 106M 2581M 19.0.0-2550-gc00506dc 9d49405603e8 a819fcc3b569 2024-07-03T15:54:15.243 INFO:teuthology.orchestra.run.smithi028.stdout:prometheus.smithi028 smithi028 *:9095 running (2m) 90s ago 17m 48.9M - 2.43.0 a07b618ecd1d 406cff811726 2024-07-03T15:54:15.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.239+0000 7f51bd9de640 1 -- 172.21.15.28:0/150626604 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f51a406e0b0 msgr2=0x7f51a4070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:15.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.239+0000 7f51bd9de640 1 --2- 172.21.15.28:0/150626604 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f51a406e0b0 0x7f51a4070560 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f51b8066330 tx=0x7f51a0008040 comp rx=0 tx=0).stop 2024-07-03T15:54:15.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.239+0000 7f51bd9de640 1 -- 172.21.15.28:0/150626604 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f51b81012b0 msgr2=0x7f51b810be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:15.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.239+0000 7f51bd9de640 1 --2- 172.21.15.28:0/150626604 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f51b81012b0 0x7f51b810be70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f51a8003080 tx=0x7f51a8009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:15.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.239+0000 7f51bd9de640 1 -- 172.21.15.28:0/150626604 shutdown_connections 2024-07-03T15:54:15.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.239+0000 7f51bd9de640 1 --2- 172.21.15.28:0/150626604 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f51a406e0b0 0x7f51a4070560 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:15.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.239+0000 7f51bd9de640 1 --2- 172.21.15.28:0/150626604 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51b8101c50 0x7f51b810c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:15.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.239+0000 7f51bd9de640 1 --2- 172.21.15.28:0/150626604 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f51b81012b0 0x7f51b810be70 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:15.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.239+0000 7f51bd9de640 1 -- 172.21.15.28:0/150626604 >> 172.21.15.28:0/150626604 conn(0x7f51b80f6ea0 msgr2=0x7f51b80f7e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:15.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.240+0000 7f51bd9de640 1 -- 172.21.15.28:0/150626604 shutdown_connections 2024-07-03T15:54:15.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:15.240+0000 7f51bd9de640 1 -- 172.21.15.28:0/150626604 wait complete. 2024-07-03T15:54:16.008 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:15 smithi028 ceph-mon[92858]: pgmap v273: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:54:16.033 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -e sha1=c00506dc10003928f8cb40769168571f611cbd67 -- bash -c 'ceph versions' 2024-07-03T15:54:16.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:15 smithi148 ceph-mon[80490]: pgmap v273: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:54:16.310 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:54:17.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:16 smithi028 ceph-mon[92858]: from='client.34522 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:54:17.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:16 smithi148 ceph-mon[80490]: from='client.34522 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-03T15:54:18.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.056+0000 7f3f5918a640 1 -- 172.21.15.28:0/1871833133 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f540fd570 msgr2=0x7f3f540fd940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:18.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.056+0000 7f3f5918a640 1 --2- 172.21.15.28:0/1871833133 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f540fd570 0x7f3f540fd940 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3f440048d0 tx=0x7f3f44009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:18.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.056+0000 7f3f5918a640 1 -- 172.21.15.28:0/1871833133 shutdown_connections 2024-07-03T15:54:18.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.056+0000 7f3f5918a640 1 --2- 172.21.15.28:0/1871833133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f540fdf10 0x7f3f541063b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:18.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.056+0000 7f3f5918a640 1 --2- 172.21.15.28:0/1871833133 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f540fd570 0x7f3f540fd940 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:18.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.056+0000 7f3f5918a640 1 -- 172.21.15.28:0/1871833133 >> 172.21.15.28:0/1871833133 conn(0x7f3f540f6e40 msgr2=0x7f3f540f9250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:18.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.056+0000 7f3f5918a640 1 -- 172.21.15.28:0/1871833133 shutdown_connections 2024-07-03T15:54:18.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.056+0000 7f3f5918a640 1 -- 172.21.15.28:0/1871833133 wait complete. 2024-07-03T15:54:18.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.056+0000 7f3f5918a640 1 Processor -- start 2024-07-03T15:54:18.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.056+0000 7f3f5918a640 1 -- start start 2024-07-03T15:54:18.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.057+0000 7f3f5918a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f540fd570 0x7f3f540fb200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:18.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.057+0000 7f3f5918a640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f540fdf10 0x7f3f540fb740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:18.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.057+0000 7f3f5918a640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f540fbc80 con 0x7f3f540fdf10 2024-07-03T15:54:18.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.057+0000 7f3f5918a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f540fbdf0 con 0x7f3f540fd570 2024-07-03T15:54:18.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.057+0000 7f3f5259c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f540fdf10 0x7f3f540fb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:18.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.057+0000 7f3f5259c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f540fdf10 0x7f3f540fb740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51628/0 (socket says 172.21.15.28:51628) 2024-07-03T15:54:18.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.057+0000 7f3f5259c640 1 -- 172.21.15.28:0/2768079618 learned_addr learned my addr 172.21.15.28:0/2768079618 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:54:18.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.057+0000 7f3f52d9d640 1 --2- 172.21.15.28:0/2768079618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f540fd570 0x7f3f540fb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:18.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.057+0000 7f3f5259c640 1 -- 172.21.15.28:0/2768079618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f540fd570 msgr2=0x7f3f540fb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:18.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.057+0000 7f3f5259c640 1 --2- 172.21.15.28:0/2768079618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f540fd570 0x7f3f540fb200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:18.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.057+0000 7f3f5259c640 1 -- 172.21.15.28:0/2768079618 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f44005040 con 0x7f3f540fdf10 2024-07-03T15:54:18.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.057+0000 7f3f52d9d640 1 --2- 172.21.15.28:0/2768079618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f540fd570 0x7f3f540fb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-03T15:54:18.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.057+0000 7f3f5259c640 1 --2- 172.21.15.28:0/2768079618 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f540fdf10 0x7f3f540fb740 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3f4800dab0 tx=0x7f3f48004ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:18.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.057+0000 7f3f3b7fe640 1 -- 172.21.15.28:0/2768079618 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f48017070 con 0x7f3f540fdf10 2024-07-03T15:54:18.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.057+0000 7f3f3b7fe640 1 -- 172.21.15.28:0/2768079618 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3f4800f040 con 0x7f3f540fdf10 2024-07-03T15:54:18.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.057+0000 7f3f5918a640 1 -- 172.21.15.28:0/2768079618 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f540fc080 con 0x7f3f540fdf10 2024-07-03T15:54:18.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.057+0000 7f3f3b7fe640 1 -- 172.21.15.28:0/2768079618 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f48003a70 con 0x7f3f540fdf10 2024-07-03T15:54:18.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.057+0000 7f3f5918a640 1 -- 172.21.15.28:0/2768079618 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f541198b0 con 0x7f3f540fdf10 2024-07-03T15:54:18.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.058+0000 7f3f5918a640 1 -- 172.21.15.28:0/2768079618 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f540faaa0 con 0x7f3f540fdf10 2024-07-03T15:54:18.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.059+0000 7f3f3b7fe640 1 -- 172.21.15.28:0/2768079618 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3f48003ce0 con 0x7f3f540fdf10 2024-07-03T15:54:18.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.059+0000 7f3f3b7fe640 1 --2- 172.21.15.28:0/2768079618 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f3f3c06e0b0 0x7f3f3c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:18.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.060+0000 7f3f3b7fe640 1 -- 172.21.15.28:0/2768079618 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f3f4808de10 con 0x7f3f540fdf10 2024-07-03T15:54:18.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.060+0000 7f3f52d9d640 1 --2- 172.21.15.28:0/2768079618 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f3f3c06e0b0 0x7f3f3c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:18.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.060+0000 7f3f52d9d640 1 --2- 172.21.15.28:0/2768079618 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f3f3c06e0b0 0x7f3f3c070560 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009f80 tx=0x7f3f44009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:18.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.064+0000 7f3f3b7fe640 1 -- 172.21.15.28:0/2768079618 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f3f480570e0 con 0x7f3f540fdf10 2024-07-03T15:54:18.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:17 smithi028 ceph-mon[92858]: pgmap v274: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:54:18.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:17 smithi148 ceph-mon[80490]: pgmap v274: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:54:18.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.322+0000 7f3f5918a640 1 -- 172.21.15.28:0/2768079618 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3f54066140 con 0x7f3f540fdf10 2024-07-03T15:54:18.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.323+0000 7f3f3b7fe640 1 -- 172.21.15.28:0/2768079618 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7f3f4805a7a0 con 0x7f3f540fdf10 2024-07-03T15:54:18.326 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-07-03T15:54:18.326 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": { 2024-07-03T15:54:18.326 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 2 2024-07-03T15:54:18.326 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:54:18.326 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": { 2024-07-03T15:54:18.327 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 2 2024-07-03T15:54:18.327 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:54:18.327 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": { 2024-07-03T15:54:18.327 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 6 2024-07-03T15:54:18.327 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:54:18.327 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": { 2024-07-03T15:54:18.327 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 4 2024-07-03T15:54:18.327 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-07-03T15:54:18.327 INFO:teuthology.orchestra.run.smithi028.stdout: "overall": { 2024-07-03T15:54:18.327 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)": 14 2024-07-03T15:54:18.327 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-07-03T15:54:18.327 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-07-03T15:54:18.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.324+0000 7f3f5918a640 1 -- 172.21.15.28:0/2768079618 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f3f3c06e0b0 msgr2=0x7f3f3c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:18.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.324+0000 7f3f5918a640 1 --2- 172.21.15.28:0/2768079618 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f3f3c06e0b0 0x7f3f3c070560 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009f80 tx=0x7f3f44009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:18.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.324+0000 7f3f5918a640 1 -- 172.21.15.28:0/2768079618 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f540fdf10 msgr2=0x7f3f540fb740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:18.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.324+0000 7f3f5918a640 1 --2- 172.21.15.28:0/2768079618 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f540fdf10 0x7f3f540fb740 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3f4800dab0 tx=0x7f3f48004ee0 comp rx=0 tx=0).stop 2024-07-03T15:54:18.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.325+0000 7f3f5918a640 1 -- 172.21.15.28:0/2768079618 shutdown_connections 2024-07-03T15:54:18.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.325+0000 7f3f5918a640 1 --2- 172.21.15.28:0/2768079618 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f3f3c06e0b0 0x7f3f3c070560 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:18.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.325+0000 7f3f5918a640 1 --2- 172.21.15.28:0/2768079618 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f540fdf10 0x7f3f540fb740 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:18.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.325+0000 7f3f5918a640 1 --2- 172.21.15.28:0/2768079618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f540fd570 0x7f3f540fb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:18.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.325+0000 7f3f5918a640 1 -- 172.21.15.28:0/2768079618 >> 172.21.15.28:0/2768079618 conn(0x7f3f540f6e40 msgr2=0x7f3f540f8300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:18.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.325+0000 7f3f5918a640 1 -- 172.21.15.28:0/2768079618 shutdown_connections 2024-07-03T15:54:18.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:18.325+0000 7f3f5918a640 1 -- 172.21.15.28:0/2768079618 wait complete. 2024-07-03T15:54:18.967 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -e sha1=c00506dc10003928f8cb40769168571f611cbd67 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-07-03T15:54:19.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:18 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/2768079618' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:54:19.237 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:54:19.360 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:18 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/2768079618' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:54:20.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:19 smithi148 ceph-mon[80490]: pgmap v275: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:54:20.372 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:19 smithi028 ceph-mon[92858]: pgmap v275: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:54:20.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:20.988+0000 7fcb29259640 1 -- 172.21.15.28:0/3212787319 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcb24102370 msgr2=0x7fcb24102740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:20.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:20.988+0000 7fcb29259640 1 --2- 172.21.15.28:0/3212787319 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcb24102370 0x7fcb24102740 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcb14009c50 tx=0x7fcb14004d10 comp rx=0 tx=0).stop 2024-07-03T15:54:20.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:20.988+0000 7fcb29259640 1 -- 172.21.15.28:0/3212787319 shutdown_connections 2024-07-03T15:54:20.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:20.988+0000 7fcb29259640 1 --2- 172.21.15.28:0/3212787319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb24064df0 0x7fcb24065240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:20.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:20.988+0000 7fcb29259640 1 --2- 172.21.15.28:0/3212787319 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcb24102370 0x7fcb24102740 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:20.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:20.988+0000 7fcb29259640 1 -- 172.21.15.28:0/3212787319 >> 172.21.15.28:0/3212787319 conn(0x7fcb24071260 msgr2=0x7fcb24071660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:20.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:20.988+0000 7fcb29259640 1 -- 172.21.15.28:0/3212787319 shutdown_connections 2024-07-03T15:54:20.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:20.988+0000 7fcb29259640 1 -- 172.21.15.28:0/3212787319 wait complete. 2024-07-03T15:54:20.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:20.989+0000 7fcb29259640 1 Processor -- start 2024-07-03T15:54:20.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:20.989+0000 7fcb29259640 1 -- start start 2024-07-03T15:54:20.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:20.989+0000 7fcb29259640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcb24064df0 0x7fcb24113030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:20.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:20.990+0000 7fcb29259640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb24102370 0x7fcb24113570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:20.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:20.990+0000 7fcb29259640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb24113b70 con 0x7fcb24064df0 2024-07-03T15:54:20.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:20.990+0000 7fcb29259640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2410cfb0 con 0x7fcb24102370 2024-07-03T15:54:20.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:20.990+0000 7fcb22d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcb24064df0 0x7fcb24113030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:20.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:20.990+0000 7fcb22d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcb24064df0 0x7fcb24113030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39244/0 (socket says 172.21.15.28:39244) 2024-07-03T15:54:20.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:20.990+0000 7fcb22d9d640 1 -- 172.21.15.28:0/1022288199 learned_addr learned my addr 172.21.15.28:0/1022288199 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:54:20.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:20.990+0000 7fcb2259c640 1 --2- 172.21.15.28:0/1022288199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb24102370 0x7fcb24113570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:20.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:20.990+0000 7fcb22d9d640 1 -- 172.21.15.28:0/1022288199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb24102370 msgr2=0x7fcb24113570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:20.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:20.990+0000 7fcb22d9d640 1 --2- 172.21.15.28:0/1022288199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb24102370 0x7fcb24113570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:20.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:20.990+0000 7fcb22d9d640 1 -- 172.21.15.28:0/1022288199 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb140098b0 con 0x7fcb24064df0 2024-07-03T15:54:20.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:20.990+0000 7fcb22d9d640 1 --2- 172.21.15.28:0/1022288199 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcb24064df0 0x7fcb24113030 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcb14004f80 tx=0x7fcb14004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:20.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:20.990+0000 7fcb0b7fe640 1 -- 172.21.15.28:0/1022288199 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb14008de0 con 0x7fcb24064df0 2024-07-03T15:54:20.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:20.990+0000 7fcb29259640 1 -- 172.21.15.28:0/1022288199 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb2410d230 con 0x7fcb24064df0 2024-07-03T15:54:20.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:20.991+0000 7fcb29259640 1 -- 172.21.15.28:0/1022288199 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb2410d750 con 0x7fcb24064df0 2024-07-03T15:54:20.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:20.991+0000 7fcb0b7fe640 1 -- 172.21.15.28:0/1022288199 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcb14032c00 con 0x7fcb24064df0 2024-07-03T15:54:20.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:20.991+0000 7fcb0b7fe640 1 -- 172.21.15.28:0/1022288199 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb1403d8b0 con 0x7fcb24064df0 2024-07-03T15:54:20.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:20.991+0000 7fcb29259640 1 -- 172.21.15.28:0/1022288199 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb10005210 con 0x7fcb24064df0 2024-07-03T15:54:21.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:20.992+0000 7fcb0b7fe640 1 -- 172.21.15.28:0/1022288199 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fcb1402f030 con 0x7fcb24064df0 2024-07-03T15:54:21.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:20.993+0000 7fcb0b7fe640 1 --2- 172.21.15.28:0/1022288199 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fcb0c06e0b0 0x7fcb0c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:21.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:20.993+0000 7fcb0b7fe640 1 -- 172.21.15.28:0/1022288199 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fcb140aeaa0 con 0x7fcb24064df0 2024-07-03T15:54:21.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:20.993+0000 7fcb2259c640 1 --2- 172.21.15.28:0/1022288199 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fcb0c06e0b0 0x7fcb0c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:21.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:20.993+0000 7fcb2259c640 1 --2- 172.21.15.28:0/1022288199 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fcb0c06e0b0 0x7fcb0c070560 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcb2410e8f0 tx=0x7fcb18009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:21.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:20.997+0000 7fcb0b7fe640 1 -- 172.21.15.28:0/1022288199 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fcb14077d70 con 0x7fcb24064df0 2024-07-03T15:54:21.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:21.256+0000 7fcb29259640 1 -- 172.21.15.28:0/1022288199 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fcb10005cd0 con 0x7fcb24064df0 2024-07-03T15:54:21.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:21.257+0000 7fcb0b7fe640 1 -- 172.21.15.28:0/1022288199 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7fcb1407b430 con 0x7fcb24064df0 2024-07-03T15:54:21.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:21.259+0000 7fcb29259640 1 -- 172.21.15.28:0/1022288199 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fcb0c06e0b0 msgr2=0x7fcb0c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:21.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:21.259+0000 7fcb29259640 1 --2- 172.21.15.28:0/1022288199 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fcb0c06e0b0 0x7fcb0c070560 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcb2410e8f0 tx=0x7fcb18009040 comp rx=0 tx=0).stop 2024-07-03T15:54:21.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:21.259+0000 7fcb29259640 1 -- 172.21.15.28:0/1022288199 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcb24064df0 msgr2=0x7fcb24113030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:21.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:21.259+0000 7fcb29259640 1 --2- 172.21.15.28:0/1022288199 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcb24064df0 0x7fcb24113030 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcb14004f80 tx=0x7fcb14004d10 comp rx=0 tx=0).stop 2024-07-03T15:54:21.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:21.259+0000 7fcb29259640 1 -- 172.21.15.28:0/1022288199 shutdown_connections 2024-07-03T15:54:21.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:21.259+0000 7fcb29259640 1 --2- 172.21.15.28:0/1022288199 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fcb0c06e0b0 0x7fcb0c070560 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:21.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:21.259+0000 7fcb29259640 1 --2- 172.21.15.28:0/1022288199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb24102370 0x7fcb24113570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:21.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:21.259+0000 7fcb29259640 1 --2- 172.21.15.28:0/1022288199 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcb24064df0 0x7fcb24113030 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:21.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:21.259+0000 7fcb29259640 1 -- 172.21.15.28:0/1022288199 >> 172.21.15.28:0/1022288199 conn(0x7fcb24071260 msgr2=0x7fcb240f95d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:21.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:21.259+0000 7fcb29259640 1 -- 172.21.15.28:0/1022288199 shutdown_connections 2024-07-03T15:54:21.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:21.259+0000 7fcb29259640 1 -- 172.21.15.28:0/1022288199 wait complete. 2024-07-03T15:54:21.271 INFO:teuthology.orchestra.run.smithi028.stdout:true 2024-07-03T15:54:21.913 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -e sha1=c00506dc10003928f8cb40769168571f611cbd67 -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-07-03T15:54:22.191 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:54:22.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:21 smithi148 ceph-mon[80490]: pgmap v276: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:54:22.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:21 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/1022288199' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:54:22.315 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:21 smithi028 ceph-mon[92858]: pgmap v276: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:54:22.315 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:21 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/1022288199' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:54:23.185 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:22 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:54:23.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:22 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:54:23.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:23.907+0000 7f9217fe4640 1 -- 172.21.15.28:0/1187585028 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9210102370 msgr2=0x7f9210102740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:23.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:23.907+0000 7f9217fe4640 1 --2- 172.21.15.28:0/1187585028 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9210102370 0x7f9210102740 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f920c0048d0 tx=0x7f920c009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:23.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:23.907+0000 7f9217fe4640 1 -- 172.21.15.28:0/1187585028 shutdown_connections 2024-07-03T15:54:23.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:23.907+0000 7f9217fe4640 1 --2- 172.21.15.28:0/1187585028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9210102d10 0x7f92100fc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:23.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:23.907+0000 7f9217fe4640 1 --2- 172.21.15.28:0/1187585028 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9210102370 0x7f9210102740 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:23.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:23.907+0000 7f9217fe4640 1 -- 172.21.15.28:0/1187585028 >> 172.21.15.28:0/1187585028 conn(0x7f92100f7f00 msgr2=0x7f92100fa310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:23.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:23.908+0000 7f9217fe4640 1 -- 172.21.15.28:0/1187585028 shutdown_connections 2024-07-03T15:54:23.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:23.908+0000 7f9217fe4640 1 -- 172.21.15.28:0/1187585028 wait complete. 2024-07-03T15:54:23.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:23.908+0000 7f9217fe4640 1 Processor -- start 2024-07-03T15:54:23.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:23.908+0000 7f9217fe4640 1 -- start start 2024-07-03T15:54:23.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:23.908+0000 7f9217fe4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9210102370 0x7f921006f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:23.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:23.908+0000 7f9217fe4640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9210102d10 0x7f921006fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:23.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:23.908+0000 7f9217fe4640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92100702a0 con 0x7f9210102d10 2024-07-03T15:54:23.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:23.908+0000 7f9217fe4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9210070410 con 0x7f9210102370 2024-07-03T15:54:23.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:23.909+0000 7f921557f640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9210102d10 0x7f921006fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:23.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:23.909+0000 7f921557f640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9210102d10 0x7f921006fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39266/0 (socket says 172.21.15.28:39266) 2024-07-03T15:54:23.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:23.909+0000 7f921557f640 1 -- 172.21.15.28:0/1219996359 learned_addr learned my addr 172.21.15.28:0/1219996359 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:54:23.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:23.909+0000 7f9215d80640 1 --2- 172.21.15.28:0/1219996359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9210102370 0x7f921006f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:23.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:23.909+0000 7f921557f640 1 -- 172.21.15.28:0/1219996359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9210102370 msgr2=0x7f921006f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:23.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:23.909+0000 7f921557f640 1 --2- 172.21.15.28:0/1219996359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9210102370 0x7f921006f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:23.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:23.909+0000 7f921557f640 1 -- 172.21.15.28:0/1219996359 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f920c005040 con 0x7f9210102d10 2024-07-03T15:54:23.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:23.909+0000 7f9215d80640 1 --2- 172.21.15.28:0/1219996359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9210102370 0x7f921006f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-03T15:54:23.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:23.909+0000 7f921557f640 1 --2- 172.21.15.28:0/1219996359 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9210102d10 0x7f921006fd60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9200002a80 tx=0x7f9200004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:23.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:23.909+0000 7f92067fc640 1 -- 172.21.15.28:0/1219996359 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9200007c30 con 0x7f9210102d10 2024-07-03T15:54:23.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:23.909+0000 7f9217fe4640 1 -- 172.21.15.28:0/1219996359 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92100706a0 con 0x7f9210102d10 2024-07-03T15:54:23.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:23.909+0000 7f92067fc640 1 -- 172.21.15.28:0/1219996359 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f920000f040 con 0x7f9210102d10 2024-07-03T15:54:23.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:23.909+0000 7f92067fc640 1 -- 172.21.15.28:0/1219996359 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92000096b0 con 0x7f9210102d10 2024-07-03T15:54:23.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:23.909+0000 7f9217fe4640 1 -- 172.21.15.28:0/1219996359 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9210119a90 con 0x7f9210102d10 2024-07-03T15:54:23.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:23.911+0000 7f9217fe4640 1 -- 172.21.15.28:0/1219996359 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92100fdae0 con 0x7f9210102d10 2024-07-03T15:54:23.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:23.913+0000 7f92067fc640 1 -- 172.21.15.28:0/1219996359 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f920000d6b0 con 0x7f9210102d10 2024-07-03T15:54:23.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:23.914+0000 7f92067fc640 1 --2- 172.21.15.28:0/1219996359 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f91fc06e0b0 0x7f91fc070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:23.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:23.914+0000 7f92067fc640 1 -- 172.21.15.28:0/1219996359 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f920008d6a0 con 0x7f9210102d10 2024-07-03T15:54:23.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:23.914+0000 7f9215d80640 1 --2- 172.21.15.28:0/1219996359 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f91fc06e0b0 0x7f91fc070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:23.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:23.915+0000 7f9215d80640 1 --2- 172.21.15.28:0/1219996359 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f91fc06e0b0 0x7f91fc070560 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f920c032430 tx=0x7f920c035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:23.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:23.917+0000 7f92067fc640 1 -- 172.21.15.28:0/1219996359 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f9200056a90 con 0x7f9210102d10 2024-07-03T15:54:24.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:23 smithi028 ceph-mon[92858]: pgmap v277: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:54:24.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:24.182+0000 7f9217fe4640 1 -- 172.21.15.28:0/1219996359 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9210109510 con 0x7f9210102d10 2024-07-03T15:54:24.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:24.183+0000 7f92067fc640 1 -- 172.21.15.28:0/1219996359 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7f920005a150 con 0x7f9210102d10 2024-07-03T15:54:24.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:24.184+0000 7f9217fe4640 1 -- 172.21.15.28:0/1219996359 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f91fc06e0b0 msgr2=0x7f91fc070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:24.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:24.184+0000 7f9217fe4640 1 --2- 172.21.15.28:0/1219996359 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f91fc06e0b0 0x7f91fc070560 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f920c032430 tx=0x7f920c035040 comp rx=0 tx=0).stop 2024-07-03T15:54:24.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:24.184+0000 7f9217fe4640 1 -- 172.21.15.28:0/1219996359 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9210102d10 msgr2=0x7f921006fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:24.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:24.184+0000 7f9217fe4640 1 --2- 172.21.15.28:0/1219996359 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9210102d10 0x7f921006fd60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9200002a80 tx=0x7f9200004600 comp rx=0 tx=0).stop 2024-07-03T15:54:24.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:24.185+0000 7f9217fe4640 1 -- 172.21.15.28:0/1219996359 shutdown_connections 2024-07-03T15:54:24.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:24.185+0000 7f9217fe4640 1 --2- 172.21.15.28:0/1219996359 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f91fc06e0b0 0x7f91fc070560 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:24.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:24.185+0000 7f9217fe4640 1 --2- 172.21.15.28:0/1219996359 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9210102d10 0x7f921006fd60 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:24.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:24.185+0000 7f9217fe4640 1 --2- 172.21.15.28:0/1219996359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9210102370 0x7f921006f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:24.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:24.185+0000 7f9217fe4640 1 -- 172.21.15.28:0/1219996359 >> 172.21.15.28:0/1219996359 conn(0x7f92100f7f00 msgr2=0x7f92100f8e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:24.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:24.185+0000 7f9217fe4640 1 -- 172.21.15.28:0/1219996359 shutdown_connections 2024-07-03T15:54:24.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:24.185+0000 7f9217fe4640 1 -- 172.21.15.28:0/1219996359 wait complete. 2024-07-03T15:54:24.196 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph version 19.0.0-2550-gc00506dc (c00506dc10003928f8cb40769168571f611cbd67) squid (rc)" 2024-07-03T15:54:24.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:23 smithi148 ceph-mon[80490]: pgmap v277: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:54:24.845 DEBUG:teuthology.parallel:result is None 2024-07-03T15:54:24.845 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-07-03T15:54:24.857 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi028.front.sepia.ceph.com 2024-07-03T15:54:24.857 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a85e4be6-3951-11ef-bcab-c7b262605968 -- bash -c 'ceph fs dump' 2024-07-03T15:54:25.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:24 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/1219996359' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:54:25.128 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/mon.smithi028/config 2024-07-03T15:54:25.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:24 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/1219996359' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-03T15:54:26.207 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:25 smithi028 ceph-mon[92858]: pgmap v278: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:54:26.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:25 smithi148 ceph-mon[80490]: pgmap v278: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:54:26.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.749+0000 7f90219de640 1 -- 172.21.15.28:0/3082131098 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f901c1013c0 msgr2=0x7f901c101790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:26.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.749+0000 7f90219de640 1 --2- 172.21.15.28:0/3082131098 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f901c1013c0 0x7f901c101790 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f900c0048d0 tx=0x7f900c009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:26.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.749+0000 7f90219de640 1 -- 172.21.15.28:0/3082131098 shutdown_connections 2024-07-03T15:54:26.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.749+0000 7f90219de640 1 --2- 172.21.15.28:0/3082131098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f901c101d60 0x7f901c0fb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:26.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.749+0000 7f90219de640 1 --2- 172.21.15.28:0/3082131098 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f901c1013c0 0x7f901c101790 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:26.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.749+0000 7f90219de640 1 -- 172.21.15.28:0/3082131098 >> 172.21.15.28:0/3082131098 conn(0x7f901c0f6e40 msgr2=0x7f901c0f9250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:26.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.749+0000 7f90219de640 1 -- 172.21.15.28:0/3082131098 shutdown_connections 2024-07-03T15:54:26.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.749+0000 7f90219de640 1 -- 172.21.15.28:0/3082131098 wait complete. 2024-07-03T15:54:26.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.750+0000 7f90219de640 1 Processor -- start 2024-07-03T15:54:26.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.750+0000 7f90219de640 1 -- start start 2024-07-03T15:54:26.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.750+0000 7f90219de640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f901c1013c0 0x7f901c1120e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:26.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.750+0000 7f90219de640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f901c101d60 0x7f901c112620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:26.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.750+0000 7f90219de640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901c112c20 con 0x7f901c101d60 2024-07-03T15:54:26.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.750+0000 7f90219de640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901c10c060 con 0x7f901c1013c0 2024-07-03T15:54:26.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.750+0000 7f901a7fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f901c101d60 0x7f901c112620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:26.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.750+0000 7f901a7fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f901c101d60 0x7f901c112620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39286/0 (socket says 172.21.15.28:39286) 2024-07-03T15:54:26.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.750+0000 7f901a7fc640 1 -- 172.21.15.28:0/2233303518 learned_addr learned my addr 172.21.15.28:0/2233303518 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:54:26.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.751+0000 7f901affd640 1 --2- 172.21.15.28:0/2233303518 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f901c1013c0 0x7f901c1120e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:26.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.751+0000 7f901a7fc640 1 -- 172.21.15.28:0/2233303518 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f901c1013c0 msgr2=0x7f901c1120e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:26.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.751+0000 7f901a7fc640 1 --2- 172.21.15.28:0/2233303518 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f901c1013c0 0x7f901c1120e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:26.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.751+0000 7f901a7fc640 1 -- 172.21.15.28:0/2233303518 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f900c005040 con 0x7f901c101d60 2024-07-03T15:54:26.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.751+0000 7f901affd640 1 --2- 172.21.15.28:0/2233303518 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f901c1013c0 0x7f901c1120e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-03T15:54:26.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.751+0000 7f901a7fc640 1 --2- 172.21.15.28:0/2233303518 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f901c101d60 0x7f901c112620 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9010002a50 tx=0x7f9010004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:26.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.751+0000 7f90037fe640 1 -- 172.21.15.28:0/2233303518 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9010007c30 con 0x7f901c101d60 2024-07-03T15:54:26.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.751+0000 7f90219de640 1 -- 172.21.15.28:0/2233303518 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f901c10c340 con 0x7f901c101d60 2024-07-03T15:54:26.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.751+0000 7f90037fe640 1 -- 172.21.15.28:0/2233303518 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f901000f040 con 0x7f901c101d60 2024-07-03T15:54:26.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.751+0000 7f90037fe640 1 -- 172.21.15.28:0/2233303518 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f901000db50 con 0x7f901c101d60 2024-07-03T15:54:26.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.751+0000 7f90219de640 1 -- 172.21.15.28:0/2233303518 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f901c10c8e0 con 0x7f901c101d60 2024-07-03T15:54:26.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.752+0000 7f90219de640 1 -- 172.21.15.28:0/2233303518 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f901c0faaa0 con 0x7f901c101d60 2024-07-03T15:54:26.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.753+0000 7f90037fe640 1 -- 172.21.15.28:0/2233303518 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f901001c400 con 0x7f901c101d60 2024-07-03T15:54:26.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.753+0000 7f90037fe640 1 --2- 172.21.15.28:0/2233303518 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f900406e0b0 0x7f9004070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:26.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.753+0000 7f90037fe640 1 -- 172.21.15.28:0/2233303518 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f901008d4d0 con 0x7f901c101d60 2024-07-03T15:54:26.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.753+0000 7f901affd640 1 --2- 172.21.15.28:0/2233303518 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f900406e0b0 0x7f9004070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:26.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.754+0000 7f901affd640 1 --2- 172.21.15.28:0/2233303518 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f900406e0b0 0x7f9004070560 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f900c009f80 tx=0x7f900c009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:26.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.758+0000 7f90037fe640 1 -- 172.21.15.28:0/2233303518 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f90100567a0 con 0x7f901c101d60 2024-07-03T15:54:26.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.980+0000 7f90219de640 1 -- 172.21.15.28:0/2233303518 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f901c04b530 con 0x7f901c101d60 2024-07-03T15:54:26.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.981+0000 7f90037fe640 1 -- 172.21.15.28:0/2233303518 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1967 (secure 0 0 0) 0x7f9010059e60 con 0x7f901c101d60 2024-07-03T15:54:26.984 INFO:teuthology.orchestra.run.smithi028.stdout:e35 2024-07-03T15:54:26.984 INFO:teuthology.orchestra.run.smithi028.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-03T15:54:26.985 INFO:teuthology.orchestra.run.smithi028.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-03T15:54:26.985 INFO:teuthology.orchestra.run.smithi028.stdout:legacy client fscid: 1 2024-07-03T15:54:26.985 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:54:26.985 INFO:teuthology.orchestra.run.smithi028.stdout:Filesystem 'cephfs' (1) 2024-07-03T15:54:26.985 INFO:teuthology.orchestra.run.smithi028.stdout:fs_name cephfs 2024-07-03T15:54:26.985 INFO:teuthology.orchestra.run.smithi028.stdout:epoch 35 2024-07-03T15:54:26.985 INFO:teuthology.orchestra.run.smithi028.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-03T15:54:26.985 INFO:teuthology.orchestra.run.smithi028.stdout:created 2024-07-03T15:41:40.656298+0000 2024-07-03T15:54:26.985 INFO:teuthology.orchestra.run.smithi028.stdout:modified 2024-07-03T15:51:35.429732+0000 2024-07-03T15:54:26.985 INFO:teuthology.orchestra.run.smithi028.stdout:tableserver 0 2024-07-03T15:54:26.985 INFO:teuthology.orchestra.run.smithi028.stdout:root 0 2024-07-03T15:54:26.985 INFO:teuthology.orchestra.run.smithi028.stdout:session_timeout 60 2024-07-03T15:54:26.986 INFO:teuthology.orchestra.run.smithi028.stdout:session_autoclose 300 2024-07-03T15:54:26.986 INFO:teuthology.orchestra.run.smithi028.stdout:max_file_size 1099511627776 2024-07-03T15:54:26.986 INFO:teuthology.orchestra.run.smithi028.stdout:max_xattr_size 65536 2024-07-03T15:54:26.986 INFO:teuthology.orchestra.run.smithi028.stdout:required_client_features {} 2024-07-03T15:54:26.986 INFO:teuthology.orchestra.run.smithi028.stdout:last_failure 0 2024-07-03T15:54:26.986 INFO:teuthology.orchestra.run.smithi028.stdout:last_failure_osd_epoch 82 2024-07-03T15:54:26.986 INFO:teuthology.orchestra.run.smithi028.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-07-03T15:54:26.986 INFO:teuthology.orchestra.run.smithi028.stdout:max_mds 1 2024-07-03T15:54:26.986 INFO:teuthology.orchestra.run.smithi028.stdout:in 0 2024-07-03T15:54:26.986 INFO:teuthology.orchestra.run.smithi028.stdout:up {0=34376} 2024-07-03T15:54:26.986 INFO:teuthology.orchestra.run.smithi028.stdout:failed 2024-07-03T15:54:26.986 INFO:teuthology.orchestra.run.smithi028.stdout:damaged 2024-07-03T15:54:26.986 INFO:teuthology.orchestra.run.smithi028.stdout:stopped 2024-07-03T15:54:26.987 INFO:teuthology.orchestra.run.smithi028.stdout:data_pools [3] 2024-07-03T15:54:26.987 INFO:teuthology.orchestra.run.smithi028.stdout:metadata_pool 2 2024-07-03T15:54:26.987 INFO:teuthology.orchestra.run.smithi028.stdout:inline_data disabled 2024-07-03T15:54:26.987 INFO:teuthology.orchestra.run.smithi028.stdout:balancer 2024-07-03T15:54:26.987 INFO:teuthology.orchestra.run.smithi028.stdout:bal_rank_mask -1 2024-07-03T15:54:26.987 INFO:teuthology.orchestra.run.smithi028.stdout:standby_count_wanted 1 2024-07-03T15:54:26.987 INFO:teuthology.orchestra.run.smithi028.stdout:qdb_cluster leader: 34376 members: 34376 2024-07-03T15:54:26.987 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi028.rhpvve{0:34376} state up:active seq 16 join_fscid=1 addr [v2:172.21.15.28:6826/3127260162,v1:172.21.15.28:6827/3127260162] compat {c=[1],r=[1],i=[1fff]}] 2024-07-03T15:54:26.987 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi028.wmbxew{0:34408} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.28:6828/2908130327,v1:172.21.15.28:6829/2908130327] compat {c=[1],r=[1],i=[1fff]}] 2024-07-03T15:54:26.987 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:54:26.987 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:54:26.987 INFO:teuthology.orchestra.run.smithi028.stdout:Standby daemons: 2024-07-03T15:54:26.987 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:54:26.987 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi148.xgpuld{-1:44301} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/1480077002,v1:172.21.15.148:6825/1480077002] compat {c=[1],r=[1],i=[1fff]}] 2024-07-03T15:54:26.988 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.cephfs.smithi148.szomfl{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6826/3922163302,v1:172.21.15.148:6827/3922163302] compat {c=[1],r=[1],i=[1fff]}] 2024-07-03T15:54:26.988 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 35 2024-07-03T15:54:26.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.983+0000 7f90219de640 1 -- 172.21.15.28:0/2233303518 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f900406e0b0 msgr2=0x7f9004070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:26.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.983+0000 7f90219de640 1 --2- 172.21.15.28:0/2233303518 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f900406e0b0 0x7f9004070560 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f900c009f80 tx=0x7f900c009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:26.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.983+0000 7f90219de640 1 -- 172.21.15.28:0/2233303518 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f901c101d60 msgr2=0x7f901c112620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:26.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.983+0000 7f90219de640 1 --2- 172.21.15.28:0/2233303518 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f901c101d60 0x7f901c112620 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9010002a50 tx=0x7f9010004600 comp rx=0 tx=0).stop 2024-07-03T15:54:26.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.983+0000 7f90219de640 1 -- 172.21.15.28:0/2233303518 shutdown_connections 2024-07-03T15:54:26.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.983+0000 7f90219de640 1 --2- 172.21.15.28:0/2233303518 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f900406e0b0 0x7f9004070560 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:26.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.983+0000 7f90219de640 1 --2- 172.21.15.28:0/2233303518 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f901c101d60 0x7f901c112620 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:26.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.983+0000 7f90219de640 1 --2- 172.21.15.28:0/2233303518 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f901c1013c0 0x7f901c1120e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:26.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.983+0000 7f90219de640 1 -- 172.21.15.28:0/2233303518 >> 172.21.15.28:0/2233303518 conn(0x7f901c0f6e40 msgr2=0x7f901c0f7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:26.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.983+0000 7f90219de640 1 -- 172.21.15.28:0/2233303518 shutdown_connections 2024-07-03T15:54:26.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:26.983+0000 7f90219de640 1 -- 172.21.15.28:0/2233303518 wait complete. 2024-07-03T15:54:27.656 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-07-03T15:54:27.668 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-03T15:54:27.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.769+0000 7f3514a61640 1 -- 172.21.15.28:0/58391745 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35101011e0 msgr2=0x7f35101015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:27.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.769+0000 7f3514a61640 1 --2- 172.21.15.28:0/58391745 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35101011e0 0x7f35101015b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f35000048d0 tx=0x7f3500009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:27.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.769+0000 7f3514a61640 1 -- 172.21.15.28:0/58391745 shutdown_connections 2024-07-03T15:54:27.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.769+0000 7f3514a61640 1 --2- 172.21.15.28:0/58391745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3510101b80 0x7f35100fb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:27.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.769+0000 7f3514a61640 1 --2- 172.21.15.28:0/58391745 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35101011e0 0x7f35101015b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:27.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.770+0000 7f3514a61640 1 -- 172.21.15.28:0/58391745 >> 172.21.15.28:0/58391745 conn(0x7f35100f6d70 msgr2=0x7f35100f9180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:27.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.770+0000 7f3514a61640 1 -- 172.21.15.28:0/58391745 shutdown_connections 2024-07-03T15:54:27.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.770+0000 7f3514a61640 1 -- 172.21.15.28:0/58391745 wait complete. 2024-07-03T15:54:27.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.770+0000 7f3514a61640 1 Processor -- start 2024-07-03T15:54:27.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.770+0000 7f3514a61640 1 -- start start 2024-07-03T15:54:27.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.770+0000 7f3514a61640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35101011e0 0x7f3510111fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:27.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.770+0000 7f3514a61640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3510101b80 0x7f35101124f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:27.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.770+0000 7f3514a61640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3510112af0 con 0x7f3510101b80 2024-07-03T15:54:27.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.770+0000 7f3514a61640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f351010c140 con 0x7f35101011e0 2024-07-03T15:54:27.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.770+0000 7f350dd9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3510101b80 0x7f35101124f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:27.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.770+0000 7f350dd9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3510101b80 0x7f35101124f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39306/0 (socket says 172.21.15.28:39306) 2024-07-03T15:54:27.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.770+0000 7f350dd9b640 1 -- 172.21.15.28:0/3684611031 learned_addr learned my addr 172.21.15.28:0/3684611031 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:54:27.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.771+0000 7f350dd9b640 1 -- 172.21.15.28:0/3684611031 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35101011e0 msgr2=0x7f3510111fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:27.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.771+0000 7f350e59c640 1 --2- 172.21.15.28:0/3684611031 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35101011e0 0x7f3510111fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:27.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.771+0000 7f350dd9b640 1 --2- 172.21.15.28:0/3684611031 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35101011e0 0x7f3510111fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:27.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.771+0000 7f350dd9b640 1 -- 172.21.15.28:0/3684611031 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3500005040 con 0x7f3510101b80 2024-07-03T15:54:27.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.771+0000 7f350dd9b640 1 --2- 172.21.15.28:0/3684611031 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3510101b80 0x7f35101124f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f350400eb70 tx=0x7f350400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:27.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.771+0000 7f34feffd640 1 -- 172.21.15.28:0/3684611031 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3504007c30 con 0x7f3510101b80 2024-07-03T15:54:27.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.771+0000 7f34feffd640 1 -- 172.21.15.28:0/3684611031 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f350400f040 con 0x7f3510101b80 2024-07-03T15:54:27.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.771+0000 7f3514a61640 1 -- 172.21.15.28:0/3684611031 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f351010c3c0 con 0x7f3510101b80 2024-07-03T15:54:27.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.771+0000 7f34feffd640 1 -- 172.21.15.28:0/3684611031 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35040095b0 con 0x7f3510101b80 2024-07-03T15:54:27.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.771+0000 7f3514a61640 1 -- 172.21.15.28:0/3684611031 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f351010c8e0 con 0x7f3510101b80 2024-07-03T15:54:27.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.772+0000 7f3514a61640 1 -- 172.21.15.28:0/3684611031 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35100fc950 con 0x7f3510101b80 2024-07-03T15:54:27.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.773+0000 7f34feffd640 1 -- 172.21.15.28:0/3684611031 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3504009730 con 0x7f3510101b80 2024-07-03T15:54:27.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.773+0000 7f34feffd640 1 --2- 172.21.15.28:0/3684611031 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f34f806e0b0 0x7f34f8070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:27.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.773+0000 7f34feffd640 1 -- 172.21.15.28:0/3684611031 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f350408d970 con 0x7f3510101b80 2024-07-03T15:54:27.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.773+0000 7f350e59c640 1 --2- 172.21.15.28:0/3684611031 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f34f806e0b0 0x7f34f8070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:27.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.774+0000 7f350e59c640 1 --2- 172.21.15.28:0/3684611031 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f34f806e0b0 0x7f34f8070560 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3500003080 tx=0x7f3500009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:27.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.777+0000 7f34feffd640 1 -- 172.21.15.28:0/3684611031 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f3504056c40 con 0x7f3510101b80 2024-07-03T15:54:27.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.965+0000 7f3514a61640 1 -- 172.21.15.28:0/3684611031 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35100fe480 con 0x7f3510101b80 2024-07-03T15:54:27.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.966+0000 7f34feffd640 1 -- 172.21.15.28:0/3684611031 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+16768 (secure 0 0 0) 0x7f350405a300 con 0x7f3510101b80 2024-07-03T15:54:27.970 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:54:27.970 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":82,"fsid":"a85e4be6-3951-11ef-bcab-c7b262605968","created":"2024-07-03T15:34:56.026978+0000","modified":"2024-07-03T15:51:21.780577+0000","last_up_change":"2024-07-03T15:50:28.186628+0000","last_in_change":"2024-07-03T15:40:01.150891+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-03T15:39:08.833339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-07-03T15:41:39.583625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-07-03T15:41:40.084538+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7ecd90b5-3715-4ba7-a696-588843e3582c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":48,"up_from":52,"up_thru":77,"down_at":49,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":1756284953},{"type":"v1","addr":"172.21.15.28:6803","nonce":1756284953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":1756284953},{"type":"v1","addr":"172.21.15.28:6805","nonce":1756284953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1756284953},{"type":"v1","addr":"172.21.15.28:6809","nonce":1756284953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":1756284953},{"type":"v1","addr":"172.21.15.28:6807","nonce":1756284953}]},"public_addr":"172.21.15.28:6803/1756284953","cluster_addr":"172.21.15.28:6805/1756284953","heartbeat_back_addr":"172.21.15.28:6809/1756284953","heartbeat_front_addr":"172.21.15.28:6807/1756284953","state":["exists","up"]},{"osd":1,"uuid":"31c5ae0c-6c57-4615-902b-72bf49fa5b72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":53,"up_from":57,"up_thru":77,"down_at":54,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":662081515},{"type":"v1","addr":"172.21.15.28:6811","nonce":662081515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":662081515},{"type":"v1","addr":"172.21.15.28:6813","nonce":662081515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":662081515},{"type":"v1","addr":"172.21.15.28:6817","nonce":662081515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":662081515},{"type":"v1","addr":"172.21.15.28:6815","nonce":662081515}]},"public_addr":"172.21.15.28:6811/662081515","cluster_addr":"172.21.15.28:6813/662081515","heartbeat_back_addr":"172.21.15.28:6817/662081515","heartbeat_front_addr":"172.21.15.28:6815/662081515","state":["exists","up"]},{"osd":2,"uuid":"5ec8caa6-fb62-443f-bbd6-fbe52df5034c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":58,"up_from":62,"up_thru":77,"down_at":59,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2390177581},{"type":"v1","addr":"172.21.15.28:6819","nonce":2390177581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2390177581},{"type":"v1","addr":"172.21.15.28:6821","nonce":2390177581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":2390177581},{"type":"v1","addr":"172.21.15.28:6825","nonce":2390177581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2390177581},{"type":"v1","addr":"172.21.15.28:6823","nonce":2390177581}]},"public_addr":"172.21.15.28:6819/2390177581","cluster_addr":"172.21.15.28:6821/2390177581","heartbeat_back_addr":"172.21.15.28:6825/2390177581","heartbeat_front_addr":"172.21.15.28:6823/2390177581","state":["exists","up"]},{"osd":3,"uuid":"d3117e1c-a230-4fc8-9cfc-37e4dc7e3d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":63,"up_from":67,"up_thru":77,"down_at":64,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":660374222},{"type":"v1","addr":"172.21.15.148:6801","nonce":660374222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":660374222},{"type":"v1","addr":"172.21.15.148:6803","nonce":660374222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":660374222},{"type":"v1","addr":"172.21.15.148:6807","nonce":660374222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":660374222},{"type":"v1","addr":"172.21.15.148:6805","nonce":660374222}]},"public_addr":"172.21.15.148:6801/660374222","cluster_addr":"172.21.15.148:6803/660374222","heartbeat_back_addr":"172.21.15.148:6807/660374222","heartbeat_front_addr":"172.21.15.148:6805/660374222","state":["exists","up"]},{"osd":4,"uuid":"d338fd80-0a76-441e-9cba-9cfd27cccc46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":68,"up_from":72,"up_thru":77,"down_at":69,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3538420843},{"type":"v1","addr":"172.21.15.148:6809","nonce":3538420843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":3538420843},{"type":"v1","addr":"172.21.15.148:6811","nonce":3538420843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":3538420843},{"type":"v1","addr":"172.21.15.148:6815","nonce":3538420843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3538420843},{"type":"v1","addr":"172.21.15.148:6813","nonce":3538420843}]},"public_addr":"172.21.15.148:6809/3538420843","cluster_addr":"172.21.15.148:6811/3538420843","heartbeat_back_addr":"172.21.15.148:6815/3538420843","heartbeat_front_addr":"172.21.15.148:6813/3538420843","state":["exists","up"]},{"osd":5,"uuid":"5c87b168-85e8-4d65-9763-8f1626e474ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":73,"up_from":77,"up_thru":77,"down_at":74,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2178157137},{"type":"v1","addr":"172.21.15.148:6817","nonce":2178157137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2178157137},{"type":"v1","addr":"172.21.15.148:6819","nonce":2178157137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2178157137},{"type":"v1","addr":"172.21.15.148:6823","nonce":2178157137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":2178157137},{"type":"v1","addr":"172.21.15.148:6821","nonce":2178157137}]},"public_addr":"172.21.15.148:6817/2178157137","cluster_addr":"172.21.15.148:6819/2178157137","heartbeat_back_addr":"172.21.15.148:6823/2178157137","heartbeat_front_addr":"172.21.15.148:6821/2178157137","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-03T15:38:11.596686+0000","dead_epoch":48},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-03T15:38:38.765936+0000","dead_epoch":53},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-03T15:39:05.464618+0000","dead_epoch":58},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-03T15:39:28.180164+0000","dead_epoch":63},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-03T15:39:52.836379+0000","dead_epoch":68},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-03T15:40:17.711279+0000","dead_epoch":73}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.148:6826/4171231858":"2024-07-04T15:51:21.780541+0000","172.21.15.28:6800/1642892231":"2024-07-04T15:46:22.043750+0000","172.21.15.28:0/2594481782":"2024-07-04T15:46:22.043750+0000","172.21.15.28:0/1964492377":"2024-07-04T15:46:22.043750+0000","172.21.15.148:6827/4171231858":"2024-07-04T15:51:21.780541+0000","172.21.15.28:0/1079212302":"2024-07-04T15:46:22.043750+0000","172.21.15.148:0/2321782375":"2024-07-04T15:45:31.636243+0000","172.21.15.148:0/2875818850":"2024-07-04T15:45:31.636243+0000","172.21.15.28:0/1083276374":"2024-07-04T15:44:02.457424+0000","172.21.15.28:6801/1358048352":"2024-07-04T15:35:22.603924+0000","172.21.15.28:6800/1358048352":"2024-07-04T15:35:22.603924+0000","172.21.15.28:6800/577887491":"2024-07-04T15:37:00.336312+0000","172.21.15.28:0/2964865891":"2024-07-04T15:37:00.336312+0000","172.21.15.28:6826/1613936042":"2024-07-04T15:50:32.503548+0000","172.21.15.28:0/2733524057":"2024-07-04T15:46:22.043750+0000","172.21.15.28:0/3188588861":"2024-07-04T15:44:02.457424+0000","172.21.15.148:0/2297116482":"2024-07-04T15:45:31.636243+0000","172.21.15.148:0/2453903464":"2024-07-04T15:45:31.636243+0000","172.21.15.28:0/2748059179":"2024-07-04T15:35:22.603924+0000","172.21.15.28:6801/577887491":"2024-07-04T15:37:00.336312+0000","172.21.15.28:6800/421057178":"2024-07-04T15:44:02.457424+0000","172.21.15.28:6801/1434683766":"2024-07-04T15:35:52.741581+0000","172.21.15.28:0/1365299779":"2024-07-04T15:44:02.457424+0000","172.21.15.148:0/301347713":"2024-07-04T15:45:31.636243+0000","172.21.15.28:0/3813763982":"2024-07-04T15:35:52.741581+0000","172.21.15.28:0/1743142286":"2024-07-04T15:46:22.043750+0000","172.21.15.28:0/1009019065":"2024-07-04T15:35:22.603924+0000","172.21.15.28:6801/421057178":"2024-07-04T15:44:02.457424+0000","172.21.15.28:0/423260870":"2024-07-04T15:35:22.603924+0000","172.21.15.148:6828/4285739175":"2024-07-04T15:45:31.636243+0000","172.21.15.28:6827/1613936042":"2024-07-04T15:50:32.503548+0000","172.21.15.28:6801/1642892231":"2024-07-04T15:46:22.043750+0000","172.21.15.28:6800/1434683766":"2024-07-04T15:35:52.741581+0000","172.21.15.148:0/679918823":"2024-07-04T15:45:31.636243+0000","172.21.15.28:0/2750216161":"2024-07-04T15:35:52.741581+0000","172.21.15.148:6829/4285739175":"2024-07-04T15:45:31.636243+0000","172.21.15.28:0/3888547290":"2024-07-04T15:46:22.043750+0000","172.21.15.28:0/3219967940":"2024-07-04T15:37:00.336312+0000","172.21.15.28:0/4294623705":"2024-07-04T15:37:00.336312+0000","172.21.15.28:0/1938941600":"2024-07-04T15:44:02.457424+0000","172.21.15.28:0/390942453":"2024-07-04T15:35:52.741581+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-03T15:54:27.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.968+0000 7f3514a61640 1 -- 172.21.15.28:0/3684611031 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f34f806e0b0 msgr2=0x7f34f8070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:27.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.968+0000 7f3514a61640 1 --2- 172.21.15.28:0/3684611031 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f34f806e0b0 0x7f34f8070560 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3500003080 tx=0x7f3500009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:27.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.968+0000 7f3514a61640 1 -- 172.21.15.28:0/3684611031 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3510101b80 msgr2=0x7f35101124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:27.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.968+0000 7f3514a61640 1 --2- 172.21.15.28:0/3684611031 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3510101b80 0x7f35101124f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f350400eb70 tx=0x7f350400b040 comp rx=0 tx=0).stop 2024-07-03T15:54:27.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.968+0000 7f3514a61640 1 -- 172.21.15.28:0/3684611031 shutdown_connections 2024-07-03T15:54:27.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.968+0000 7f3514a61640 1 --2- 172.21.15.28:0/3684611031 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f34f806e0b0 0x7f34f8070560 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:27.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.968+0000 7f3514a61640 1 --2- 172.21.15.28:0/3684611031 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3510101b80 0x7f35101124f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:27.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.968+0000 7f3514a61640 1 --2- 172.21.15.28:0/3684611031 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35101011e0 0x7f3510111fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:27.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.968+0000 7f3514a61640 1 -- 172.21.15.28:0/3684611031 >> 172.21.15.28:0/3684611031 conn(0x7f35100f6d70 msgr2=0x7f35100f8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:27.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.968+0000 7f3514a61640 1 -- 172.21.15.28:0/3684611031 shutdown_connections 2024-07-03T15:54:27.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:27.969+0000 7f3514a61640 1 -- 172.21.15.28:0/3684611031 wait complete. 2024-07-03T15:54:27.982 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:27 smithi028 ceph-mon[92858]: pgmap v279: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:54:27.982 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:27 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/2233303518' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-03T15:54:27.984 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-03T15:39:08.833339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-07-03T15:41:39.583625+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-07-03T15:41:40.084538+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-07-03T15:54:27.985 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-07-03T15:54:28.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.084+0000 7f895ab5d640 1 -- 172.21.15.28:0/2274121934 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89541011c0 msgr2=0x7f8954101590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:28.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.084+0000 7f895ab5d640 1 --2- 172.21.15.28:0/2274121934 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89541011c0 0x7f8954101590 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f89440048d0 tx=0x7f8944009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:28.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.084+0000 7f895ab5d640 1 -- 172.21.15.28:0/2274121934 shutdown_connections 2024-07-03T15:54:28.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.084+0000 7f895ab5d640 1 --2- 172.21.15.28:0/2274121934 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89540fb190 0x7f89540fb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:28.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.084+0000 7f895ab5d640 1 --2- 172.21.15.28:0/2274121934 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89541011c0 0x7f8954101590 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:28.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.084+0000 7f895ab5d640 1 -- 172.21.15.28:0/2274121934 >> 172.21.15.28:0/2274121934 conn(0x7f8954071170 msgr2=0x7f8954071570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:28.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.084+0000 7f895ab5d640 1 -- 172.21.15.28:0/2274121934 shutdown_connections 2024-07-03T15:54:28.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.084+0000 7f895ab5d640 1 -- 172.21.15.28:0/2274121934 wait complete. 2024-07-03T15:54:28.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.085+0000 7f895ab5d640 1 Processor -- start 2024-07-03T15:54:28.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.085+0000 7f895ab5d640 1 -- start start 2024-07-03T15:54:28.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.085+0000 7f895ab5d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89540fb190 0x7f8954112d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:28.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.085+0000 7f895ab5d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89541011c0 0x7f89541132c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:28.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.085+0000 7f895ab5d640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89541138c0 con 0x7f89540fb190 2024-07-03T15:54:28.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.085+0000 7f895ab5d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8954113a30 con 0x7f89541011c0 2024-07-03T15:54:28.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.085+0000 7f89588f9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89540fb190 0x7f8954112d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:28.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.085+0000 7f89588f9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89540fb190 0x7f8954112d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39326/0 (socket says 172.21.15.28:39326) 2024-07-03T15:54:28.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.085+0000 7f89588f9640 1 -- 172.21.15.28:0/203600495 learned_addr learned my addr 172.21.15.28:0/203600495 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:54:28.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.086+0000 7f89588f9640 1 -- 172.21.15.28:0/203600495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89541011c0 msgr2=0x7f89541132c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:28.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.086+0000 7f89588f9640 1 --2- 172.21.15.28:0/203600495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89541011c0 0x7f89541132c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:28.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.086+0000 7f89588f9640 1 -- 172.21.15.28:0/203600495 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8944005040 con 0x7f89540fb190 2024-07-03T15:54:28.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.086+0000 7f89588f9640 1 --2- 172.21.15.28:0/203600495 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89540fb190 0x7f8954112d80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f89440041d0 tx=0x7f8944009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:28.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.086+0000 7f89517fa640 1 -- 172.21.15.28:0/203600495 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8944030870 con 0x7f89540fb190 2024-07-03T15:54:28.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.086+0000 7f895ab5d640 1 -- 172.21.15.28:0/203600495 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f895410d060 con 0x7f89540fb190 2024-07-03T15:54:28.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.086+0000 7f89517fa640 1 -- 172.21.15.28:0/203600495 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f89440092e0 con 0x7f89540fb190 2024-07-03T15:54:28.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.086+0000 7f89517fa640 1 -- 172.21.15.28:0/203600495 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8944043400 con 0x7f89540fb190 2024-07-03T15:54:28.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.086+0000 7f895ab5d640 1 -- 172.21.15.28:0/203600495 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f895410d580 con 0x7f89540fb190 2024-07-03T15:54:28.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.087+0000 7f895ab5d640 1 -- 172.21.15.28:0/203600495 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89540fc880 con 0x7f89540fb190 2024-07-03T15:54:28.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.088+0000 7f89517fa640 1 -- 172.21.15.28:0/203600495 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f894403b420 con 0x7f89540fb190 2024-07-03T15:54:28.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.088+0000 7f89517fa640 1 --2- 172.21.15.28:0/203600495 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f893c06e0b0 0x7f893c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:28.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.088+0000 7f89517fa640 1 -- 172.21.15.28:0/203600495 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f89440b0940 con 0x7f89540fb190 2024-07-03T15:54:28.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.088+0000 7f8953fff640 1 --2- 172.21.15.28:0/203600495 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f893c06e0b0 0x7f893c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:28.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.089+0000 7f8953fff640 1 --2- 172.21.15.28:0/203600495 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f893c06e0b0 0x7f893c070560 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f895410e680 tx=0x7f8948009c50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:28.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.092+0000 7f89517fa640 1 -- 172.21.15.28:0/203600495 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f8944079d50 con 0x7f89540fb190 2024-07-03T15:54:28.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:27 smithi148 ceph-mon[80490]: pgmap v279: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:54:28.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:27 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/2233303518' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-03T15:54:28.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.281+0000 7f895ab5d640 1 -- 172.21.15.28:0/203600495 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f895410d7a0 con 0x7f89540fb190 2024-07-03T15:54:28.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.281+0000 7f89517fa640 1 -- 172.21.15.28:0/203600495 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f894407d410 con 0x7f89540fb190 2024-07-03T15:54:28.285 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2024-07-03T15:54:28.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.283+0000 7f895ab5d640 1 -- 172.21.15.28:0/203600495 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f893c06e0b0 msgr2=0x7f893c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:28.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.283+0000 7f895ab5d640 1 --2- 172.21.15.28:0/203600495 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f893c06e0b0 0x7f893c070560 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f895410e680 tx=0x7f8948009c50 comp rx=0 tx=0).stop 2024-07-03T15:54:28.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.283+0000 7f895ab5d640 1 -- 172.21.15.28:0/203600495 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89540fb190 msgr2=0x7f8954112d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:28.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.283+0000 7f895ab5d640 1 --2- 172.21.15.28:0/203600495 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89540fb190 0x7f8954112d80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f89440041d0 tx=0x7f8944009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:28.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.283+0000 7f895ab5d640 1 -- 172.21.15.28:0/203600495 shutdown_connections 2024-07-03T15:54:28.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.283+0000 7f895ab5d640 1 --2- 172.21.15.28:0/203600495 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f893c06e0b0 0x7f893c070560 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:28.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.283+0000 7f895ab5d640 1 --2- 172.21.15.28:0/203600495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89541011c0 0x7f89541132c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:28.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.283+0000 7f895ab5d640 1 --2- 172.21.15.28:0/203600495 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89540fb190 0x7f8954112d80 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:28.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.283+0000 7f895ab5d640 1 -- 172.21.15.28:0/203600495 >> 172.21.15.28:0/203600495 conn(0x7f8954071170 msgr2=0x7f8954064440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:28.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.283+0000 7f895ab5d640 1 -- 172.21.15.28:0/203600495 shutdown_connections 2024-07-03T15:54:28.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.283+0000 7f895ab5d640 1 -- 172.21.15.28:0/203600495 wait complete. 2024-07-03T15:54:28.298 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-07-03T15:54:28.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.398+0000 7f3c4ed5d640 1 -- 172.21.15.28:0/1710925311 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c48102290 msgr2=0x7f3c48102660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:28.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.398+0000 7f3c4ed5d640 1 --2- 172.21.15.28:0/1710925311 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c48102290 0x7f3c48102660 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c0048d0 tx=0x7f3c3c009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:28.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.398+0000 7f3c4ed5d640 1 -- 172.21.15.28:0/1710925311 shutdown_connections 2024-07-03T15:54:28.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.398+0000 7f3c4ed5d640 1 --2- 172.21.15.28:0/1710925311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c48064430 0x7f3c48064880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:28.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.398+0000 7f3c4ed5d640 1 --2- 172.21.15.28:0/1710925311 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c48102290 0x7f3c48102660 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:28.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.398+0000 7f3c4ed5d640 1 -- 172.21.15.28:0/1710925311 >> 172.21.15.28:0/1710925311 conn(0x7f3c48071170 msgr2=0x7f3c48071570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:28.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.398+0000 7f3c4ed5d640 1 -- 172.21.15.28:0/1710925311 shutdown_connections 2024-07-03T15:54:28.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.398+0000 7f3c4ed5d640 1 -- 172.21.15.28:0/1710925311 wait complete. 2024-07-03T15:54:28.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.399+0000 7f3c4ed5d640 1 Processor -- start 2024-07-03T15:54:28.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.399+0000 7f3c4ed5d640 1 -- start start 2024-07-03T15:54:28.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.399+0000 7f3c4ed5d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c48064430 0x7f3c48112d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:28.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.399+0000 7f3c4ed5d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c48102290 0x7f3c48113270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:28.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.399+0000 7f3c4ed5d640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c48113870 con 0x7f3c48102290 2024-07-03T15:54:28.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.399+0000 7f3c4ed5d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4810cf10 con 0x7f3c48064430 2024-07-03T15:54:28.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.399+0000 7f3c47fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c48102290 0x7f3c48113270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:28.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.399+0000 7f3c47fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c48102290 0x7f3c48113270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39346/0 (socket says 172.21.15.28:39346) 2024-07-03T15:54:28.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.399+0000 7f3c47fff640 1 -- 172.21.15.28:0/2296399891 learned_addr learned my addr 172.21.15.28:0/2296399891 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:54:28.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.399+0000 7f3c4caf9640 1 --2- 172.21.15.28:0/2296399891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c48064430 0x7f3c48112d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:28.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.400+0000 7f3c47fff640 1 -- 172.21.15.28:0/2296399891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c48064430 msgr2=0x7f3c48112d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:28.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.400+0000 7f3c47fff640 1 --2- 172.21.15.28:0/2296399891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c48064430 0x7f3c48112d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:28.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.400+0000 7f3c47fff640 1 -- 172.21.15.28:0/2296399891 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c3c005040 con 0x7f3c48102290 2024-07-03T15:54:28.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.400+0000 7f3c4caf9640 1 --2- 172.21.15.28:0/2296399891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c48064430 0x7f3c48112d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-03T15:54:28.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.400+0000 7f3c47fff640 1 --2- 172.21.15.28:0/2296399891 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c48102290 0x7f3c48113270 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3c3400eb70 tx=0x7f3c3400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:28.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.400+0000 7f3c457fa640 1 -- 172.21.15.28:0/2296399891 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c34007c30 con 0x7f3c48102290 2024-07-03T15:54:28.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.400+0000 7f3c457fa640 1 -- 172.21.15.28:0/2296399891 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3c3400f040 con 0x7f3c48102290 2024-07-03T15:54:28.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.400+0000 7f3c4ed5d640 1 -- 172.21.15.28:0/2296399891 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c4810d190 con 0x7f3c48102290 2024-07-03T15:54:28.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.400+0000 7f3c457fa640 1 -- 172.21.15.28:0/2296399891 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c340095b0 con 0x7f3c48102290 2024-07-03T15:54:28.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.400+0000 7f3c4ed5d640 1 -- 172.21.15.28:0/2296399891 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c4810d6b0 con 0x7f3c48102290 2024-07-03T15:54:28.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.401+0000 7f3c4ed5d640 1 -- 172.21.15.28:0/2296399891 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c480fd990 con 0x7f3c48102290 2024-07-03T15:54:28.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.401+0000 7f3c457fa640 1 -- 172.21.15.28:0/2296399891 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3c34009730 con 0x7f3c48102290 2024-07-03T15:54:28.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.402+0000 7f3c457fa640 1 --2- 172.21.15.28:0/2296399891 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f3c3806e0b0 0x7f3c38070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:28.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.402+0000 7f3c457fa640 1 -- 172.21.15.28:0/2296399891 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f3c3408d970 con 0x7f3c48102290 2024-07-03T15:54:28.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.402+0000 7f3c4caf9640 1 --2- 172.21.15.28:0/2296399891 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f3c3806e0b0 0x7f3c38070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:28.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.402+0000 7f3c4caf9640 1 --2- 172.21.15.28:0/2296399891 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f3c3806e0b0 0x7f3c38070560 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c000c00 tx=0x7f3c3c009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:28.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.406+0000 7f3c457fa640 1 -- 172.21.15.28:0/2296399891 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f3c34056c40 con 0x7f3c48102290 2024-07-03T15:54:28.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.589+0000 7f3c4ed5d640 1 -- 172.21.15.28:0/2296399891 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f3c48109380 con 0x7f3c48102290 2024-07-03T15:54:28.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.590+0000 7f3c457fa640 1 -- 172.21.15.28:0/2296399891 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v82) v1 ==== 107+0+11 (secure 0 0 0) 0x7f3c3405a300 con 0x7f3c48102290 2024-07-03T15:54:28.593 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2024-07-03T15:54:28.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.592+0000 7f3c4ed5d640 1 -- 172.21.15.28:0/2296399891 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f3c3806e0b0 msgr2=0x7f3c38070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:28.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.592+0000 7f3c4ed5d640 1 --2- 172.21.15.28:0/2296399891 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f3c3806e0b0 0x7f3c38070560 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c000c00 tx=0x7f3c3c009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:28.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.592+0000 7f3c4ed5d640 1 -- 172.21.15.28:0/2296399891 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c48102290 msgr2=0x7f3c48113270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:28.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.592+0000 7f3c4ed5d640 1 --2- 172.21.15.28:0/2296399891 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c48102290 0x7f3c48113270 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3c3400eb70 tx=0x7f3c3400b040 comp rx=0 tx=0).stop 2024-07-03T15:54:28.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.592+0000 7f3c4ed5d640 1 -- 172.21.15.28:0/2296399891 shutdown_connections 2024-07-03T15:54:28.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.592+0000 7f3c4ed5d640 1 --2- 172.21.15.28:0/2296399891 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f3c3806e0b0 0x7f3c38070560 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:28.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.592+0000 7f3c4ed5d640 1 --2- 172.21.15.28:0/2296399891 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c48102290 0x7f3c48113270 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:28.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.592+0000 7f3c4ed5d640 1 --2- 172.21.15.28:0/2296399891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c48064430 0x7f3c48112d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:28.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.592+0000 7f3c4ed5d640 1 -- 172.21.15.28:0/2296399891 >> 172.21.15.28:0/2296399891 conn(0x7f3c48071170 msgr2=0x7f3c480f9520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:28.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.592+0000 7f3c4ed5d640 1 -- 172.21.15.28:0/2296399891 shutdown_connections 2024-07-03T15:54:28.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.592+0000 7f3c4ed5d640 1 -- 172.21.15.28:0/2296399891 wait complete. 2024-07-03T15:54:28.606 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-07-03T15:54:28.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.706+0000 7fe7b2d9d640 1 -- 172.21.15.28:0/727115021 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7ac101150 msgr2=0x7fe7ac101520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:28.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.706+0000 7fe7b2d9d640 1 --2- 172.21.15.28:0/727115021 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7ac101150 0x7fe7ac101520 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe7a00048d0 tx=0x7fe7a0009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:28.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.706+0000 7fe7b2d9d640 1 -- 172.21.15.28:0/727115021 shutdown_connections 2024-07-03T15:54:28.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.706+0000 7fe7b2d9d640 1 --2- 172.21.15.28:0/727115021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7ac101af0 0x7fe7ac0fb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:28.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.706+0000 7fe7b2d9d640 1 --2- 172.21.15.28:0/727115021 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7ac101150 0x7fe7ac101520 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:28.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.706+0000 7fe7b2d9d640 1 -- 172.21.15.28:0/727115021 >> 172.21.15.28:0/727115021 conn(0x7fe7ac0f6d60 msgr2=0x7fe7ac0f9170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:28.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.706+0000 7fe7b2d9d640 1 -- 172.21.15.28:0/727115021 shutdown_connections 2024-07-03T15:54:28.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.706+0000 7fe7b2d9d640 1 -- 172.21.15.28:0/727115021 wait complete. 2024-07-03T15:54:28.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.707+0000 7fe7b2d9d640 1 Processor -- start 2024-07-03T15:54:28.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.707+0000 7fe7b2d9d640 1 -- start start 2024-07-03T15:54:28.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.707+0000 7fe7b2d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7ac101150 0x7fe7ac112dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:28.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.707+0000 7fe7b2d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7ac101af0 0x7fe7ac113310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:28.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.707+0000 7fe7b2d9d640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac113910 con 0x7fe7ac101150 2024-07-03T15:54:28.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.707+0000 7fe7b2d9d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac10cf60 con 0x7fe7ac101af0 2024-07-03T15:54:28.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.707+0000 7fe7b0b39640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7ac101150 0x7fe7ac112dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:28.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.708+0000 7fe7b0b39640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7ac101150 0x7fe7ac112dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39360/0 (socket says 172.21.15.28:39360) 2024-07-03T15:54:28.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.708+0000 7fe7b0b39640 1 -- 172.21.15.28:0/3062990505 learned_addr learned my addr 172.21.15.28:0/3062990505 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:54:28.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.708+0000 7fe7abfff640 1 --2- 172.21.15.28:0/3062990505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7ac101af0 0x7fe7ac113310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:28.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.708+0000 7fe7b0b39640 1 -- 172.21.15.28:0/3062990505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7ac101af0 msgr2=0x7fe7ac113310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:28.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.708+0000 7fe7b0b39640 1 --2- 172.21.15.28:0/3062990505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7ac101af0 0x7fe7ac113310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:28.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.708+0000 7fe7b0b39640 1 -- 172.21.15.28:0/3062990505 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a0005040 con 0x7fe7ac101150 2024-07-03T15:54:28.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.708+0000 7fe7b0b39640 1 --2- 172.21.15.28:0/3062990505 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7ac101150 0x7fe7ac112dd0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0034040 tx=0x7fe7a0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:28.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.708+0000 7fe7a97fa640 1 -- 172.21.15.28:0/3062990505 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7a00092b0 con 0x7fe7ac101150 2024-07-03T15:54:28.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.708+0000 7fe7a97fa640 1 -- 172.21.15.28:0/3062990505 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe7a003b400 con 0x7fe7ac101150 2024-07-03T15:54:28.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.708+0000 7fe7b2d9d640 1 -- 172.21.15.28:0/3062990505 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7ac10d1e0 con 0x7fe7ac101150 2024-07-03T15:54:28.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.708+0000 7fe7a97fa640 1 -- 172.21.15.28:0/3062990505 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7a00435d0 con 0x7fe7ac101150 2024-07-03T15:54:28.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.708+0000 7fe7b2d9d640 1 -- 172.21.15.28:0/3062990505 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7ac10d6a0 con 0x7fe7ac101150 2024-07-03T15:54:28.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.709+0000 7fe7b2d9d640 1 -- 172.21.15.28:0/3062990505 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7ac0fc8c0 con 0x7fe7ac101150 2024-07-03T15:54:28.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.710+0000 7fe7a97fa640 1 -- 172.21.15.28:0/3062990505 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe7a002f070 con 0x7fe7ac101150 2024-07-03T15:54:28.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.710+0000 7fe7a97fa640 1 --2- 172.21.15.28:0/3062990505 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fe79c06e0b0 0x7fe79c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:28.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.711+0000 7fe7a97fa640 1 -- 172.21.15.28:0/3062990505 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fe7a00af890 con 0x7fe7ac101150 2024-07-03T15:54:28.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.711+0000 7fe7abfff640 1 --2- 172.21.15.28:0/3062990505 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fe79c06e0b0 0x7fe79c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:28.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.711+0000 7fe7abfff640 1 --2- 172.21.15.28:0/3062990505 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fe79c06e0b0 0x7fe79c070560 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac10e690 tx=0x7fe79800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:28.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.715+0000 7fe7a97fa640 1 -- 172.21.15.28:0/3062990505 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fe7a0078c10 con 0x7fe7ac101150 2024-07-03T15:54:28.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.897+0000 7fe7b2d9d640 1 -- 172.21.15.28:0/3062990505 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fe7ac04bd70 con 0x7fe7ac101150 2024-07-03T15:54:28.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.897+0000 7fe7a97fa640 1 -- 172.21.15.28:0/3062990505 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v82) v1 ==== 107+0+11 (secure 0 0 0) 0x7fe7a007c2d0 con 0x7fe7ac101150 2024-07-03T15:54:28.901 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2024-07-03T15:54:28.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.899+0000 7fe7b2d9d640 1 -- 172.21.15.28:0/3062990505 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fe79c06e0b0 msgr2=0x7fe79c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:28.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.899+0000 7fe7b2d9d640 1 --2- 172.21.15.28:0/3062990505 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fe79c06e0b0 0x7fe79c070560 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac10e690 tx=0x7fe79800b040 comp rx=0 tx=0).stop 2024-07-03T15:54:28.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.899+0000 7fe7b2d9d640 1 -- 172.21.15.28:0/3062990505 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7ac101150 msgr2=0x7fe7ac112dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:28.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.899+0000 7fe7b2d9d640 1 --2- 172.21.15.28:0/3062990505 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7ac101150 0x7fe7ac112dd0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0034040 tx=0x7fe7a0009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:28.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.899+0000 7fe7b2d9d640 1 -- 172.21.15.28:0/3062990505 shutdown_connections 2024-07-03T15:54:28.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.899+0000 7fe7b2d9d640 1 --2- 172.21.15.28:0/3062990505 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fe79c06e0b0 0x7fe79c070560 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:28.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.899+0000 7fe7b2d9d640 1 --2- 172.21.15.28:0/3062990505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7ac101af0 0x7fe7ac113310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:28.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.899+0000 7fe7b2d9d640 1 --2- 172.21.15.28:0/3062990505 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7ac101150 0x7fe7ac112dd0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:28.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.900+0000 7fe7b2d9d640 1 -- 172.21.15.28:0/3062990505 >> 172.21.15.28:0/3062990505 conn(0x7fe7ac0f6d60 msgr2=0x7fe7ac0f7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:28.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.900+0000 7fe7b2d9d640 1 -- 172.21.15.28:0/3062990505 shutdown_connections 2024-07-03T15:54:28.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:28.900+0000 7fe7b2d9d640 1 -- 172.21.15.28:0/3062990505 wait complete. 2024-07-03T15:54:28.914 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-03T15:54:29.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.011+0000 7fc67135d640 1 -- 172.21.15.28:0/3897794961 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc66c101170 msgr2=0x7fc66c101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:29.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.011+0000 7fc67135d640 1 --2- 172.21.15.28:0/3897794961 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc66c101170 0x7fc66c101540 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc65c0048d0 tx=0x7fc65c009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:29.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.011+0000 7fc67135d640 1 -- 172.21.15.28:0/3897794961 shutdown_connections 2024-07-03T15:54:29.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.011+0000 7fc67135d640 1 --2- 172.21.15.28:0/3897794961 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc66c101b10 0x7fc66c0fb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:29.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.011+0000 7fc67135d640 1 --2- 172.21.15.28:0/3897794961 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc66c101170 0x7fc66c101540 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:29.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.011+0000 7fc67135d640 1 -- 172.21.15.28:0/3897794961 >> 172.21.15.28:0/3897794961 conn(0x7fc66c071820 msgr2=0x7fc66c071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:29.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.011+0000 7fc67135d640 1 -- 172.21.15.28:0/3897794961 shutdown_connections 2024-07-03T15:54:29.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.011+0000 7fc67135d640 1 -- 172.21.15.28:0/3897794961 wait complete. 2024-07-03T15:54:29.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.012+0000 7fc67135d640 1 Processor -- start 2024-07-03T15:54:29.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.012+0000 7fc67135d640 1 -- start start 2024-07-03T15:54:29.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.012+0000 7fc67135d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc66c101170 0x7fc66c1150a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:29.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.012+0000 7fc67135d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc66c101b10 0x7fc66c1155e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:29.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.012+0000 7fc67135d640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c115be0 con 0x7fc66c101b10 2024-07-03T15:54:29.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.012+0000 7fc67135d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c115d50 con 0x7fc66c101170 2024-07-03T15:54:29.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.012+0000 7fc663fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc66c101b10 0x7fc66c1155e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:29.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.012+0000 7fc663fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc66c101b10 0x7fc66c1155e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39390/0 (socket says 172.21.15.28:39390) 2024-07-03T15:54:29.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.012+0000 7fc663fff640 1 -- 172.21.15.28:0/2720640211 learned_addr learned my addr 172.21.15.28:0/2720640211 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:54:29.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.012+0000 7fc663fff640 1 -- 172.21.15.28:0/2720640211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc66c101170 msgr2=0x7fc66c1150a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:29.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.012+0000 7fc663fff640 1 --2- 172.21.15.28:0/2720640211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc66c101170 0x7fc66c1150a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:29.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.012+0000 7fc663fff640 1 -- 172.21.15.28:0/2720640211 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc65c005040 con 0x7fc66c101b10 2024-07-03T15:54:29.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.013+0000 7fc663fff640 1 --2- 172.21.15.28:0/2720640211 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc66c101b10 0x7fc66c1155e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc654002840 tx=0x7fc654004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:29.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.013+0000 7fc6637fe640 1 -- 172.21.15.28:0/2720640211 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc654007c30 con 0x7fc66c101b10 2024-07-03T15:54:29.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.013+0000 7fc6637fe640 1 -- 172.21.15.28:0/2720640211 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc65400f040 con 0x7fc66c101b10 2024-07-03T15:54:29.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.013+0000 7fc67135d640 1 -- 172.21.15.28:0/2720640211 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc66c10f430 con 0x7fc66c101b10 2024-07-03T15:54:29.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.013+0000 7fc6637fe640 1 -- 172.21.15.28:0/2720640211 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc654009570 con 0x7fc66c101b10 2024-07-03T15:54:29.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.013+0000 7fc67135d640 1 -- 172.21.15.28:0/2720640211 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc66c10f950 con 0x7fc66c101b10 2024-07-03T15:54:29.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.014+0000 7fc67135d640 1 -- 172.21.15.28:0/2720640211 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc66c0fc9c0 con 0x7fc66c101b10 2024-07-03T15:54:29.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.014+0000 7fc6637fe640 1 -- 172.21.15.28:0/2720640211 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc65400dc80 con 0x7fc66c101b10 2024-07-03T15:54:29.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.015+0000 7fc6637fe640 1 --2- 172.21.15.28:0/2720640211 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc65806e0b0 0x7fc658070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:29.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.015+0000 7fc6637fe640 1 -- 172.21.15.28:0/2720640211 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fc65408d470 con 0x7fc66c101b10 2024-07-03T15:54:29.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.015+0000 7fc66affd640 1 --2- 172.21.15.28:0/2720640211 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc65806e0b0 0x7fc658070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:29.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.015+0000 7fc66affd640 1 --2- 172.21.15.28:0/2720640211 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc65806e0b0 0x7fc658070560 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc65c0043d0 tx=0x7fc65c009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:29.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.019+0000 7fc6637fe640 1 -- 172.21.15.28:0/2720640211 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc654056740 con 0x7fc66c101b10 2024-07-03T15:54:29.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:28 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/3684611031' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-03T15:54:29.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:28 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/203600495' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-03T15:54:29.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:28 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/2296399891' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-07-03T15:54:29.234 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:28 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/3062990505' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-07-03T15:54:29.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.244+0000 7fc67135d640 1 -- 172.21.15.28:0/2720640211 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc66c04bd70 con 0x7fc66c101b10 2024-07-03T15:54:29.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.244+0000 7fc6637fe640 1 -- 172.21.15.28:0/2720640211 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+5221 (secure 0 0 0) 0x7fc654059e00 con 0x7fc66c101b10 2024-07-03T15:54:29.247 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:54:29.248 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44301,"name":"cephfs.smithi148.xgpuld","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6825/1480077002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1480077002},{"type":"v1","addr":"172.21.15.148:6825","nonce":1480077002}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":44327,"name":"cephfs.smithi148.szomfl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6827/3922163302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3922163302},{"type":"v1","addr":"172.21.15.148:6827","nonce":3922163302}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-03T15:41:40.656298+0000","modified":"2024-07-03T15:51:35.429732+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34376},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34376":{"gid":34376,"name":"cephfs.smithi028.rhpvve","rank":0,"incarnation":31,"state":"up:active","state_seq":16,"addr":"172.21.15.28:6827/3127260162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":3127260162},{"type":"v1","addr":"172.21.15.28:6827","nonce":3127260162}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34408":{"gid":34408,"name":"cephfs.smithi028.wmbxew","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6829/2908130327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":2908130327},{"type":"v1","addr":"172.21.15.28:6829","nonce":2908130327}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34376,"qdb_cluster":[34376]},"id":1}]} 2024-07-03T15:54:29.248 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 35 2024-07-03T15:54:29.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.246+0000 7fc67135d640 1 -- 172.21.15.28:0/2720640211 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc65806e0b0 msgr2=0x7fc658070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:29.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.246+0000 7fc67135d640 1 --2- 172.21.15.28:0/2720640211 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc65806e0b0 0x7fc658070560 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc65c0043d0 tx=0x7fc65c009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:29.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.246+0000 7fc67135d640 1 -- 172.21.15.28:0/2720640211 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc66c101b10 msgr2=0x7fc66c1155e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:29.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.246+0000 7fc67135d640 1 --2- 172.21.15.28:0/2720640211 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc66c101b10 0x7fc66c1155e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc654002840 tx=0x7fc654004570 comp rx=0 tx=0).stop 2024-07-03T15:54:29.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.246+0000 7fc67135d640 1 -- 172.21.15.28:0/2720640211 shutdown_connections 2024-07-03T15:54:29.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.246+0000 7fc67135d640 1 --2- 172.21.15.28:0/2720640211 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc65806e0b0 0x7fc658070560 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:29.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.246+0000 7fc67135d640 1 --2- 172.21.15.28:0/2720640211 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc66c101b10 0x7fc66c1155e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:29.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.246+0000 7fc67135d640 1 --2- 172.21.15.28:0/2720640211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc66c101170 0x7fc66c1150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:29.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.246+0000 7fc67135d640 1 -- 172.21.15.28:0/2720640211 >> 172.21.15.28:0/2720640211 conn(0x7fc66c071820 msgr2=0x7fc66c0f8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:29.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.246+0000 7fc67135d640 1 -- 172.21.15.28:0/2720640211 shutdown_connections 2024-07-03T15:54:29.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.247+0000 7fc67135d640 1 -- 172.21.15.28:0/2720640211 wait complete. 2024-07-03T15:54:29.259 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:28 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/3684611031' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-03T15:54:29.260 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:28 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/203600495' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-03T15:54:29.260 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:28 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/2296399891' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-07-03T15:54:29.260 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:28 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/3062990505' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-07-03T15:54:29.261 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 16, 'max_mds': 1, 'flags': 50} 2024-07-03T15:54:29.261 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 17 2024-07-03T15:54:29.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.358+0000 7f7bb099d640 1 -- 172.21.15.28:0/2066302231 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7bac0fd260 msgr2=0x7f7bac0fd630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:29.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.358+0000 7f7bb099d640 1 --2- 172.21.15.28:0/2066302231 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7bac0fd260 0x7f7bac0fd630 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7b940048d0 tx=0x7f7b94009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:29.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.358+0000 7f7bb099d640 1 -- 172.21.15.28:0/2066302231 shutdown_connections 2024-07-03T15:54:29.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.358+0000 7f7bb099d640 1 --2- 172.21.15.28:0/2066302231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bac0fdc00 0x7f7bac106240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:29.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.358+0000 7f7bb099d640 1 --2- 172.21.15.28:0/2066302231 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7bac0fd260 0x7f7bac0fd630 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:29.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.358+0000 7f7bb099d640 1 -- 172.21.15.28:0/2066302231 >> 172.21.15.28:0/2066302231 conn(0x7f7bac0f6cb0 msgr2=0x7f7bac0f90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:29.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.358+0000 7f7bb099d640 1 -- 172.21.15.28:0/2066302231 shutdown_connections 2024-07-03T15:54:29.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.358+0000 7f7bb099d640 1 -- 172.21.15.28:0/2066302231 wait complete. 2024-07-03T15:54:29.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.359+0000 7f7bb099d640 1 Processor -- start 2024-07-03T15:54:29.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.359+0000 7f7bb099d640 1 -- start start 2024-07-03T15:54:29.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.359+0000 7f7bb099d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7bac0fd260 0x7f7bac112dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:29.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.359+0000 7f7bb099d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bac0fdc00 0x7f7bac113300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:29.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.359+0000 7f7bb099d640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bac113900 con 0x7f7bac0fd260 2024-07-03T15:54:29.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.359+0000 7f7bb099d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bac10cf50 con 0x7f7bac0fdc00 2024-07-03T15:54:29.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.359+0000 7f7baa59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7bac0fd260 0x7f7bac112dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:29.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.359+0000 7f7baa59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7bac0fd260 0x7f7bac112dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39410/0 (socket says 172.21.15.28:39410) 2024-07-03T15:54:29.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.359+0000 7f7baa59c640 1 -- 172.21.15.28:0/920403899 learned_addr learned my addr 172.21.15.28:0/920403899 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:54:29.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.359+0000 7f7baa59c640 1 -- 172.21.15.28:0/920403899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bac0fdc00 msgr2=0x7f7bac113300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:29.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.359+0000 7f7ba9d9b640 1 --2- 172.21.15.28:0/920403899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bac0fdc00 0x7f7bac113300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:29.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.359+0000 7f7baa59c640 1 --2- 172.21.15.28:0/920403899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bac0fdc00 0x7f7bac113300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:29.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.359+0000 7f7baa59c640 1 -- 172.21.15.28:0/920403899 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b94005040 con 0x7f7bac0fd260 2024-07-03T15:54:29.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.359+0000 7f7ba9d9b640 1 --2- 172.21.15.28:0/920403899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bac0fdc00 0x7f7bac113300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-03T15:54:29.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.359+0000 7f7baa59c640 1 --2- 172.21.15.28:0/920403899 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7bac0fd260 0x7f7bac112dc0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7b94004930 tx=0x7f7b94009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:29.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.360+0000 7f7ba2ffd640 1 -- 172.21.15.28:0/920403899 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b940094c0 con 0x7f7bac0fd260 2024-07-03T15:54:29.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.360+0000 7f7bb099d640 1 -- 172.21.15.28:0/920403899 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bac10d1d0 con 0x7f7bac0fd260 2024-07-03T15:54:29.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.360+0000 7f7ba2ffd640 1 -- 172.21.15.28:0/920403899 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7b94032e50 con 0x7f7bac0fd260 2024-07-03T15:54:29.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.360+0000 7f7ba2ffd640 1 -- 172.21.15.28:0/920403899 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b940433b0 con 0x7f7bac0fd260 2024-07-03T15:54:29.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.360+0000 7f7bb099d640 1 -- 172.21.15.28:0/920403899 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bac10d690 con 0x7f7bac0fd260 2024-07-03T15:54:29.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.360+0000 7f7bb099d640 1 -- 172.21.15.28:0/920403899 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bac0660f0 con 0x7f7bac0fd260 2024-07-03T15:54:29.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.361+0000 7f7ba2ffd640 1 -- 172.21.15.28:0/920403899 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7b9402f070 con 0x7f7bac0fd260 2024-07-03T15:54:29.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.361+0000 7f7ba2ffd640 1 --2- 172.21.15.28:0/920403899 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f7b9806e0b0 0x7f7b98070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:29.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.362+0000 7f7ba2ffd640 1 -- 172.21.15.28:0/920403899 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f7b940afa50 con 0x7f7bac0fd260 2024-07-03T15:54:29.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.362+0000 7f7ba9d9b640 1 --2- 172.21.15.28:0/920403899 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f7b9806e0b0 0x7f7b98070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:29.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.362+0000 7f7ba9d9b640 1 --2- 172.21.15.28:0/920403899 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f7b9806e0b0 0x7f7b98070560 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7bac10e680 tx=0x7f7b9c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:29.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.366+0000 7f7ba2ffd640 1 -- 172.21.15.28:0/920403899 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f7b94078d20 con 0x7f7bac0fd260 2024-07-03T15:54:29.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.588+0000 7f7bb099d640 1 -- 172.21.15.28:0/920403899 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 17, "format": "json"} v 0) v1 -- 0x7f7bac10e430 con 0x7f7bac0fd260 2024-07-03T15:54:29.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.588+0000 7f7ba2ffd640 1 -- 172.21.15.28:0/920403899 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 17, "format": "json"}]=0 dumped fsmap epoch 17 v35) v1 ==== 107+0+4869 (secure 0 0 0) 0x7f7b9407c3e0 con 0x7f7bac0fd260 2024-07-03T15:54:29.592 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:54:29.592 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24277,"name":"cephfs.smithi148.szomfl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.148:6827/4171231858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":4171231858},{"type":"v1","addr":"172.21.15.148:6827","nonce":4171231858}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":24281,"name":"cephfs.smithi148.xgpuld","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6825/2347536398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2347536398},{"type":"v1","addr":"172.21.15.148:6825","nonce":2347536398}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":16,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-03T15:41:40.656298+0000","modified":"2024-07-03T15:41:56.454984+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14450},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14450":{"gid":14450,"name":"cephfs.smithi028.rhpvve","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.28:6827/1613936042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1613936042},{"type":"v1","addr":"172.21.15.28:6827","nonce":1613936042}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14460":{"gid":14460,"name":"cephfs.smithi028.wmbxew","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.28:6829/1730336821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1730336821},{"type":"v1","addr":"172.21.15.28:6829","nonce":1730336821}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-07-03T15:54:29.592 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 17 2024-07-03T15:54:29.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.590+0000 7f7bb099d640 1 -- 172.21.15.28:0/920403899 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f7b9806e0b0 msgr2=0x7f7b98070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:29.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.590+0000 7f7bb099d640 1 --2- 172.21.15.28:0/920403899 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f7b9806e0b0 0x7f7b98070560 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7bac10e680 tx=0x7f7b9c00b040 comp rx=0 tx=0).stop 2024-07-03T15:54:29.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.590+0000 7f7bb099d640 1 -- 172.21.15.28:0/920403899 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7bac0fd260 msgr2=0x7f7bac112dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:29.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.590+0000 7f7bb099d640 1 --2- 172.21.15.28:0/920403899 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7bac0fd260 0x7f7bac112dc0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7b94004930 tx=0x7f7b94009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:29.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.590+0000 7f7bb099d640 1 -- 172.21.15.28:0/920403899 shutdown_connections 2024-07-03T15:54:29.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.590+0000 7f7bb099d640 1 --2- 172.21.15.28:0/920403899 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f7b9806e0b0 0x7f7b98070560 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:29.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.590+0000 7f7bb099d640 1 --2- 172.21.15.28:0/920403899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bac0fdc00 0x7f7bac113300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:29.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.590+0000 7f7bb099d640 1 --2- 172.21.15.28:0/920403899 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7bac0fd260 0x7f7bac112dc0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:29.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.590+0000 7f7bb099d640 1 -- 172.21.15.28:0/920403899 >> 172.21.15.28:0/920403899 conn(0x7f7bac0f6cb0 msgr2=0x7f7bac0f80f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:29.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.590+0000 7f7bb099d640 1 -- 172.21.15.28:0/920403899 shutdown_connections 2024-07-03T15:54:29.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.590+0000 7f7bb099d640 1 -- 172.21.15.28:0/920403899 wait complete. 2024-07-03T15:54:29.605 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 18 2024-07-03T15:54:29.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.703+0000 7f48a959d640 1 -- 172.21.15.28:0/1241385707 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48a40fb200 msgr2=0x7f48a40fb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:29.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.703+0000 7f48a959d640 1 --2- 172.21.15.28:0/1241385707 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48a40fb200 0x7f48a40fb670 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f48980048d0 tx=0x7f4898009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:29.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.704+0000 7f48a959d640 1 -- 172.21.15.28:0/1241385707 shutdown_connections 2024-07-03T15:54:29.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.704+0000 7f48a959d640 1 --2- 172.21.15.28:0/1241385707 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48a40fb200 0x7f48a40fb670 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:29.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.704+0000 7f48a959d640 1 --2- 172.21.15.28:0/1241385707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48a4101220 0x7f48a41015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:29.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.704+0000 7f48a959d640 1 -- 172.21.15.28:0/1241385707 >> 172.21.15.28:0/1241385707 conn(0x7f48a4072080 msgr2=0x7f48a4072480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:29.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.704+0000 7f48a959d640 1 -- 172.21.15.28:0/1241385707 shutdown_connections 2024-07-03T15:54:29.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.704+0000 7f48a959d640 1 -- 172.21.15.28:0/1241385707 wait complete. 2024-07-03T15:54:29.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.704+0000 7f48a959d640 1 Processor -- start 2024-07-03T15:54:29.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.704+0000 7f48a959d640 1 -- start start 2024-07-03T15:54:29.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.704+0000 7f48a959d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48a40fb200 0x7f48a410dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:29.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.704+0000 7f48a959d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48a4101220 0x7f48a410e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:29.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.704+0000 7f48a959d640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a40653c0 con 0x7f48a40fb200 2024-07-03T15:54:29.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.704+0000 7f48a959d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a410e990 con 0x7f48a4101220 2024-07-03T15:54:29.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.705+0000 7f48a2ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48a40fb200 0x7f48a410dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:29.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.705+0000 7f48a2ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48a40fb200 0x7f48a410dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39426/0 (socket says 172.21.15.28:39426) 2024-07-03T15:54:29.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.705+0000 7f48a2ffd640 1 -- 172.21.15.28:0/1181096454 learned_addr learned my addr 172.21.15.28:0/1181096454 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:54:29.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.705+0000 7f48a27fc640 1 --2- 172.21.15.28:0/1181096454 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48a4101220 0x7f48a410e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:29.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.705+0000 7f48a2ffd640 1 -- 172.21.15.28:0/1181096454 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48a4101220 msgr2=0x7f48a410e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:29.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.705+0000 7f48a2ffd640 1 --2- 172.21.15.28:0/1181096454 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48a4101220 0x7f48a410e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:29.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.705+0000 7f48a2ffd640 1 -- 172.21.15.28:0/1181096454 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4898005040 con 0x7f48a40fb200 2024-07-03T15:54:29.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.705+0000 7f48a2ffd640 1 --2- 172.21.15.28:0/1181096454 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48a40fb200 0x7f48a410dee0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f489400ece0 tx=0x7f489400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:29.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.705+0000 7f488b7fe640 1 -- 172.21.15.28:0/1181096454 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f489400ee40 con 0x7f48a40fb200 2024-07-03T15:54:29.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.705+0000 7f488b7fe640 1 -- 172.21.15.28:0/1181096454 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f489400f040 con 0x7f48a40fb200 2024-07-03T15:54:29.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.705+0000 7f48a959d640 1 -- 172.21.15.28:0/1181096454 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48a410ec70 con 0x7f48a40fb200 2024-07-03T15:54:29.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.705+0000 7f488b7fe640 1 -- 172.21.15.28:0/1181096454 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4894009750 con 0x7f48a40fb200 2024-07-03T15:54:29.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.705+0000 7f48a959d640 1 -- 172.21.15.28:0/1181096454 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48a4117200 con 0x7f48a40fb200 2024-07-03T15:54:29.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.706+0000 7f48a959d640 1 -- 172.21.15.28:0/1181096454 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48a40fc920 con 0x7f48a40fb200 2024-07-03T15:54:29.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.707+0000 7f488b7fe640 1 -- 172.21.15.28:0/1181096454 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4894007890 con 0x7f48a40fb200 2024-07-03T15:54:29.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.707+0000 7f488b7fe640 1 --2- 172.21.15.28:0/1181096454 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f488c06e0b0 0x7f488c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:29.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.707+0000 7f488b7fe640 1 -- 172.21.15.28:0/1181096454 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f489408e3b0 con 0x7f48a40fb200 2024-07-03T15:54:29.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.707+0000 7f48a27fc640 1 --2- 172.21.15.28:0/1181096454 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f488c06e0b0 0x7f488c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:29.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.708+0000 7f48a27fc640 1 --2- 172.21.15.28:0/1181096454 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f488c06e0b0 0x7f488c070560 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f48a4064c10 tx=0x7f4898009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:29.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.712+0000 7f488b7fe640 1 -- 172.21.15.28:0/1181096454 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f4894057700 con 0x7f48a40fb200 2024-07-03T15:54:29.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.938+0000 7f48a959d640 1 -- 172.21.15.28:0/1181096454 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 18, "format": "json"} v 0) v1 -- 0x7f48a40fe450 con 0x7f48a40fb200 2024-07-03T15:54:29.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.938+0000 7f488b7fe640 1 -- 172.21.15.28:0/1181096454 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 18, "format": "json"}]=0 dumped fsmap epoch 18 v35) v1 ==== 107+0+4093 (secure 0 0 0) 0x7f489405adc0 con 0x7f48a40fb200 2024-07-03T15:54:29.942 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:54:29.942 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24277,"name":"cephfs.smithi148.szomfl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.148:6827/4171231858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":4171231858},{"type":"v1","addr":"172.21.15.148:6827","nonce":4171231858}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":24281,"name":"cephfs.smithi148.xgpuld","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6825/2347536398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2347536398},{"type":"v1","addr":"172.21.15.148:6825","nonce":2347536398}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-03T15:41:40.656298+0000","modified":"2024-07-03T15:50:29.200315+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14450},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14450":{"gid":14450,"name":"cephfs.smithi028.rhpvve","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.28:6827/1613936042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1613936042},{"type":"v1","addr":"172.21.15.28:6827","nonce":1613936042}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14450,"qdb_cluster":[14450]},"id":1}]} 2024-07-03T15:54:29.942 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 18 2024-07-03T15:54:29.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.941+0000 7f48a959d640 1 -- 172.21.15.28:0/1181096454 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f488c06e0b0 msgr2=0x7f488c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:29.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.941+0000 7f48a959d640 1 --2- 172.21.15.28:0/1181096454 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f488c06e0b0 0x7f488c070560 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f48a4064c10 tx=0x7f4898009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:29.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.941+0000 7f48a959d640 1 -- 172.21.15.28:0/1181096454 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48a40fb200 msgr2=0x7f48a410dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:29.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.941+0000 7f48a959d640 1 --2- 172.21.15.28:0/1181096454 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48a40fb200 0x7f48a410dee0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f489400ece0 tx=0x7f489400b040 comp rx=0 tx=0).stop 2024-07-03T15:54:29.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.941+0000 7f48a959d640 1 -- 172.21.15.28:0/1181096454 shutdown_connections 2024-07-03T15:54:29.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.941+0000 7f48a959d640 1 --2- 172.21.15.28:0/1181096454 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f488c06e0b0 0x7f488c070560 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:29.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.941+0000 7f48a959d640 1 --2- 172.21.15.28:0/1181096454 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48a4101220 0x7f48a410e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:29.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.941+0000 7f48a959d640 1 --2- 172.21.15.28:0/1181096454 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48a40fb200 0x7f48a410dee0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:29.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.941+0000 7f48a959d640 1 -- 172.21.15.28:0/1181096454 >> 172.21.15.28:0/1181096454 conn(0x7f48a4072080 msgr2=0x7f48a40fa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:29.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.941+0000 7f48a959d640 1 -- 172.21.15.28:0/1181096454 shutdown_connections 2024-07-03T15:54:29.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:29.941+0000 7f48a959d640 1 -- 172.21.15.28:0/1181096454 wait complete. 2024-07-03T15:54:29.955 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 18 2024-07-03T15:54:29.956 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 19 2024-07-03T15:54:30.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.053+0000 7fbe9631d640 1 -- 172.21.15.28:0/1502293037 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe90102290 msgr2=0x7fbe90102660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:30.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.053+0000 7fbe9631d640 1 --2- 172.21.15.28:0/1502293037 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe90102290 0x7fbe90102660 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbe800048d0 tx=0x7fbe80009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:30.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.053+0000 7fbe9631d640 1 -- 172.21.15.28:0/1502293037 shutdown_connections 2024-07-03T15:54:30.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.053+0000 7fbe9631d640 1 --2- 172.21.15.28:0/1502293037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe90102c30 0x7fbe900fc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:30.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.053+0000 7fbe9631d640 1 --2- 172.21.15.28:0/1502293037 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe90102290 0x7fbe90102660 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:30.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.053+0000 7fbe9631d640 1 -- 172.21.15.28:0/1502293037 >> 172.21.15.28:0/1502293037 conn(0x7fbe900f7e60 msgr2=0x7fbe900fa270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:30.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.053+0000 7fbe9631d640 1 -- 172.21.15.28:0/1502293037 shutdown_connections 2024-07-03T15:54:30.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.053+0000 7fbe9631d640 1 -- 172.21.15.28:0/1502293037 wait complete. 2024-07-03T15:54:30.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.054+0000 7fbe9631d640 1 Processor -- start 2024-07-03T15:54:30.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.054+0000 7fbe9631d640 1 -- start start 2024-07-03T15:54:30.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.054+0000 7fbe9631d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe90102290 0x7fbe90112cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:30.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.054+0000 7fbe9631d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe90102c30 0x7fbe90113230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:30.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.054+0000 7fbe9631d640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe90113830 con 0x7fbe90102c30 2024-07-03T15:54:30.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.054+0000 7fbe9631d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9010ce80 con 0x7fbe90102290 2024-07-03T15:54:30.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.054+0000 7fbe8f7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe90102c30 0x7fbe90113230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:30.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.054+0000 7fbe8f7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe90102c30 0x7fbe90113230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47254/0 (socket says 172.21.15.28:47254) 2024-07-03T15:54:30.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.054+0000 7fbe8f7fe640 1 -- 172.21.15.28:0/1090333299 learned_addr learned my addr 172.21.15.28:0/1090333299 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:54:30.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.054+0000 7fbe8ffff640 1 --2- 172.21.15.28:0/1090333299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe90102290 0x7fbe90112cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:30.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.054+0000 7fbe8f7fe640 1 -- 172.21.15.28:0/1090333299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe90102290 msgr2=0x7fbe90112cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:30.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.054+0000 7fbe8f7fe640 1 --2- 172.21.15.28:0/1090333299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe90102290 0x7fbe90112cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:30.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.054+0000 7fbe8f7fe640 1 -- 172.21.15.28:0/1090333299 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe80005040 con 0x7fbe90102c30 2024-07-03T15:54:30.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.055+0000 7fbe8f7fe640 1 --2- 172.21.15.28:0/1090333299 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe90102c30 0x7fbe90113230 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbe84002a50 tx=0x7fbe84004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:30.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.055+0000 7fbe8cff9640 1 -- 172.21.15.28:0/1090333299 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe84007c30 con 0x7fbe90102c30 2024-07-03T15:54:30.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.055+0000 7fbe8cff9640 1 -- 172.21.15.28:0/1090333299 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe8400f040 con 0x7fbe90102c30 2024-07-03T15:54:30.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.055+0000 7fbe9631d640 1 -- 172.21.15.28:0/1090333299 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe9010d160 con 0x7fbe90102c30 2024-07-03T15:54:30.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.055+0000 7fbe8cff9640 1 -- 172.21.15.28:0/1090333299 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe8400dbe0 con 0x7fbe90102c30 2024-07-03T15:54:30.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.055+0000 7fbe9631d640 1 -- 172.21.15.28:0/1090333299 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe9010d680 con 0x7fbe90102c30 2024-07-03T15:54:30.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.056+0000 7fbe9631d640 1 -- 172.21.15.28:0/1090333299 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe900fda00 con 0x7fbe90102c30 2024-07-03T15:54:30.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.056+0000 7fbe8cff9640 1 -- 172.21.15.28:0/1090333299 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fbe8400dd40 con 0x7fbe90102c30 2024-07-03T15:54:30.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.057+0000 7fbe8cff9640 1 --2- 172.21.15.28:0/1090333299 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fbe7806e0b0 0x7fbe78070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:30.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.057+0000 7fbe8cff9640 1 -- 172.21.15.28:0/1090333299 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fbe8408dae0 con 0x7fbe90102c30 2024-07-03T15:54:30.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.057+0000 7fbe8ffff640 1 --2- 172.21.15.28:0/1090333299 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fbe7806e0b0 0x7fbe78070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:30.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.057+0000 7fbe8ffff640 1 --2- 172.21.15.28:0/1090333299 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fbe7806e0b0 0x7fbe78070560 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbe80003080 tx=0x7fbe80009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:30.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.061+0000 7fbe8cff9640 1 -- 172.21.15.28:0/1090333299 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fbe84056e30 con 0x7fbe90102c30 2024-07-03T15:54:30.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:29 smithi148 ceph-mon[80490]: pgmap v280: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:54:30.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:29 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/2720640211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-03T15:54:30.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:29 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/920403899' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-07-03T15:54:30.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.289+0000 7fbe9631d640 1 -- 172.21.15.28:0/1090333299 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7fbe90064400 con 0x7fbe90102c30 2024-07-03T15:54:30.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.290+0000 7fbe8cff9640 1 -- 172.21.15.28:0/1090333299 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v35) v1 ==== 107+0+4872 (secure 0 0 0) 0x7fbe8405a4f0 con 0x7fbe90102c30 2024-07-03T15:54:30.293 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:54:30.293 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24277,"name":"cephfs.smithi148.szomfl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.148:6827/4171231858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":4171231858},{"type":"v1","addr":"172.21.15.148:6827","nonce":4171231858}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":24281,"name":"cephfs.smithi148.xgpuld","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6825/2347536398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2347536398},{"type":"v1","addr":"172.21.15.148:6825","nonce":2347536398}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34372,"name":"cephfs.smithi028.wmbxew","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6829/1224242160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1224242160},{"type":"v1","addr":"172.21.15.28:6829","nonce":1224242160}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-03T15:41:40.656298+0000","modified":"2024-07-03T15:50:29.200315+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14450},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14450":{"gid":14450,"name":"cephfs.smithi028.rhpvve","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.28:6827/1613936042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1613936042},{"type":"v1","addr":"172.21.15.28:6827","nonce":1613936042}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14450,"qdb_cluster":[14450]},"id":1}]} 2024-07-03T15:54:30.294 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 19 2024-07-03T15:54:30.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.292+0000 7fbe9631d640 1 -- 172.21.15.28:0/1090333299 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fbe7806e0b0 msgr2=0x7fbe78070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:30.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.292+0000 7fbe9631d640 1 --2- 172.21.15.28:0/1090333299 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fbe7806e0b0 0x7fbe78070560 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbe80003080 tx=0x7fbe80009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:30.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.292+0000 7fbe9631d640 1 -- 172.21.15.28:0/1090333299 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe90102c30 msgr2=0x7fbe90113230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:30.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.292+0000 7fbe9631d640 1 --2- 172.21.15.28:0/1090333299 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe90102c30 0x7fbe90113230 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbe84002a50 tx=0x7fbe84004600 comp rx=0 tx=0).stop 2024-07-03T15:54:30.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.292+0000 7fbe9631d640 1 -- 172.21.15.28:0/1090333299 shutdown_connections 2024-07-03T15:54:30.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.292+0000 7fbe9631d640 1 --2- 172.21.15.28:0/1090333299 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fbe7806e0b0 0x7fbe78070560 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:30.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.292+0000 7fbe9631d640 1 --2- 172.21.15.28:0/1090333299 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe90102c30 0x7fbe90113230 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:30.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.292+0000 7fbe9631d640 1 --2- 172.21.15.28:0/1090333299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe90102290 0x7fbe90112cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:30.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.292+0000 7fbe9631d640 1 -- 172.21.15.28:0/1090333299 >> 172.21.15.28:0/1090333299 conn(0x7fbe900f7e60 msgr2=0x7fbe900f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:30.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.292+0000 7fbe9631d640 1 -- 172.21.15.28:0/1090333299 shutdown_connections 2024-07-03T15:54:30.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.292+0000 7fbe9631d640 1 -- 172.21.15.28:0/1090333299 wait complete. 2024-07-03T15:54:30.305 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:29 smithi028 ceph-mon[92858]: pgmap v280: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:54:30.305 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:29 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/2720640211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-03T15:54:30.305 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:29 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/920403899' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-07-03T15:54:30.306 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 19 2024-07-03T15:54:30.307 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 20 2024-07-03T15:54:30.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.404+0000 7f34c7c2e640 1 -- 172.21.15.28:0/917941849 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34c00fd280 msgr2=0x7f34c00fd650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:30.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.404+0000 7f34c7c2e640 1 --2- 172.21.15.28:0/917941849 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34c00fd280 0x7f34c00fd650 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f34bc0048d0 tx=0x7f34bc009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:30.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.404+0000 7f34c7c2e640 1 -- 172.21.15.28:0/917941849 shutdown_connections 2024-07-03T15:54:30.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.404+0000 7f34c7c2e640 1 --2- 172.21.15.28:0/917941849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34c00fdc20 0x7f34c0106260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:30.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.404+0000 7f34c7c2e640 1 --2- 172.21.15.28:0/917941849 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34c00fd280 0x7f34c00fd650 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:30.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.404+0000 7f34c7c2e640 1 -- 172.21.15.28:0/917941849 >> 172.21.15.28:0/917941849 conn(0x7f34c00f6d30 msgr2=0x7f34c00f9140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:30.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.404+0000 7f34c7c2e640 1 -- 172.21.15.28:0/917941849 shutdown_connections 2024-07-03T15:54:30.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.404+0000 7f34c7c2e640 1 -- 172.21.15.28:0/917941849 wait complete. 2024-07-03T15:54:30.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.404+0000 7f34c7c2e640 1 Processor -- start 2024-07-03T15:54:30.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.404+0000 7f34c7c2e640 1 -- start start 2024-07-03T15:54:30.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.405+0000 7f34c7c2e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34c00fd280 0x7f34c0112dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:30.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.405+0000 7f34c7c2e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34c00fdc20 0x7f34c0113310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:30.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.405+0000 7f34c7c2e640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c0113910 con 0x7f34c00fdc20 2024-07-03T15:54:30.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.405+0000 7f34c7c2e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c010cf60 con 0x7f34c00fd280 2024-07-03T15:54:30.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.405+0000 7f34c51c9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34c00fdc20 0x7f34c0113310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:30.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.405+0000 7f34c51c9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34c00fdc20 0x7f34c0113310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47276/0 (socket says 172.21.15.28:47276) 2024-07-03T15:54:30.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.405+0000 7f34c51c9640 1 -- 172.21.15.28:0/2177325776 learned_addr learned my addr 172.21.15.28:0/2177325776 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:54:30.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.405+0000 7f34c59ca640 1 --2- 172.21.15.28:0/2177325776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34c00fd280 0x7f34c0112dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:30.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.405+0000 7f34c51c9640 1 -- 172.21.15.28:0/2177325776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34c00fd280 msgr2=0x7f34c0112dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:30.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.405+0000 7f34c51c9640 1 --2- 172.21.15.28:0/2177325776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34c00fd280 0x7f34c0112dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:30.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.405+0000 7f34c51c9640 1 -- 172.21.15.28:0/2177325776 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34bc005040 con 0x7f34c00fdc20 2024-07-03T15:54:30.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.405+0000 7f34c51c9640 1 --2- 172.21.15.28:0/2177325776 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34c00fdc20 0x7f34c0113310 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f34b000c830 tx=0x7f34b0004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:30.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.405+0000 7f34b67fc640 1 -- 172.21.15.28:0/2177325776 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34b000a850 con 0x7f34c00fdc20 2024-07-03T15:54:30.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.406+0000 7f34b67fc640 1 -- 172.21.15.28:0/2177325776 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f34b000ae80 con 0x7f34c00fdc20 2024-07-03T15:54:30.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.406+0000 7f34c7c2e640 1 -- 172.21.15.28:0/2177325776 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34c010d240 con 0x7f34c00fdc20 2024-07-03T15:54:30.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.406+0000 7f34b67fc640 1 -- 172.21.15.28:0/2177325776 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34b0012ea0 con 0x7f34c00fdc20 2024-07-03T15:54:30.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.406+0000 7f34c7c2e640 1 -- 172.21.15.28:0/2177325776 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34c010d760 con 0x7f34c00fdc20 2024-07-03T15:54:30.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.406+0000 7f34c7c2e640 1 -- 172.21.15.28:0/2177325776 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34c00660f0 con 0x7f34c00fdc20 2024-07-03T15:54:30.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.407+0000 7f34b67fc640 1 -- 172.21.15.28:0/2177325776 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f34b0012400 con 0x7f34c00fdc20 2024-07-03T15:54:30.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.408+0000 7f34b67fc640 1 --2- 172.21.15.28:0/2177325776 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f34ac06e0b0 0x7f34ac070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:30.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.408+0000 7f34b67fc640 1 -- 172.21.15.28:0/2177325776 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f34b008dd60 con 0x7f34c00fdc20 2024-07-03T15:54:30.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.408+0000 7f34c59ca640 1 --2- 172.21.15.28:0/2177325776 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f34ac06e0b0 0x7f34ac070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:30.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.408+0000 7f34c59ca640 1 --2- 172.21.15.28:0/2177325776 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f34ac06e0b0 0x7f34ac070560 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f34bc031870 tx=0x7f34bc009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:30.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.412+0000 7f34b67fc640 1 -- 172.21.15.28:0/2177325776 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f34b00570b0 con 0x7f34c00fdc20 2024-07-03T15:54:30.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.639+0000 7f34c7c2e640 1 -- 172.21.15.28:0/2177325776 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7f34c01093c0 con 0x7f34c00fdc20 2024-07-03T15:54:30.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.639+0000 7f34b67fc640 1 -- 172.21.15.28:0/2177325776 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v35) v1 ==== 107+0+4075 (secure 0 0 0) 0x7f34b00126b0 con 0x7f34c00fdc20 2024-07-03T15:54:30.642 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:54:30.643 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24277,"name":"cephfs.smithi148.szomfl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.148:6827/4171231858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":4171231858},{"type":"v1","addr":"172.21.15.148:6827","nonce":4171231858}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":24281,"name":"cephfs.smithi148.xgpuld","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6825/2347536398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2347536398},{"type":"v1","addr":"172.21.15.148:6825","nonce":2347536398}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34372,"name":"cephfs.smithi028.wmbxew","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6829/1224242160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1224242160},{"type":"v1","addr":"172.21.15.28:6829","nonce":1224242160}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-03T15:41:40.656298+0000","modified":"2024-07-03T15:50:32.503823+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-07-03T15:54:30.643 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 20 2024-07-03T15:54:30.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.641+0000 7f34c7c2e640 1 -- 172.21.15.28:0/2177325776 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f34ac06e0b0 msgr2=0x7f34ac070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:30.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.641+0000 7f34c7c2e640 1 --2- 172.21.15.28:0/2177325776 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f34ac06e0b0 0x7f34ac070560 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f34bc031870 tx=0x7f34bc009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:30.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.641+0000 7f34c7c2e640 1 -- 172.21.15.28:0/2177325776 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34c00fdc20 msgr2=0x7f34c0113310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:30.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.641+0000 7f34c7c2e640 1 --2- 172.21.15.28:0/2177325776 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34c00fdc20 0x7f34c0113310 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f34b000c830 tx=0x7f34b0004430 comp rx=0 tx=0).stop 2024-07-03T15:54:30.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.641+0000 7f34c7c2e640 1 -- 172.21.15.28:0/2177325776 shutdown_connections 2024-07-03T15:54:30.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.641+0000 7f34c7c2e640 1 --2- 172.21.15.28:0/2177325776 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f34ac06e0b0 0x7f34ac070560 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:30.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.641+0000 7f34c7c2e640 1 --2- 172.21.15.28:0/2177325776 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34c00fdc20 0x7f34c0113310 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:30.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.641+0000 7f34c7c2e640 1 --2- 172.21.15.28:0/2177325776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34c00fd280 0x7f34c0112dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:30.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.641+0000 7f34c7c2e640 1 -- 172.21.15.28:0/2177325776 >> 172.21.15.28:0/2177325776 conn(0x7f34c00f6d30 msgr2=0x7f34c00f8130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:30.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.642+0000 7f34c7c2e640 1 -- 172.21.15.28:0/2177325776 shutdown_connections 2024-07-03T15:54:30.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.642+0000 7f34c7c2e640 1 -- 172.21.15.28:0/2177325776 wait complete. 2024-07-03T15:54:30.655 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 20 2024-07-03T15:54:30.656 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 21 2024-07-03T15:54:30.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.753+0000 7f7b4d46e640 1 -- 172.21.15.28:0/2382385112 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7b48102290 msgr2=0x7f7b48102660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:30.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.753+0000 7f7b4d46e640 1 --2- 172.21.15.28:0/2382385112 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7b48102290 0x7f7b48102660 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7b380048d0 tx=0x7f7b38009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:30.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.753+0000 7f7b4d46e640 1 -- 172.21.15.28:0/2382385112 shutdown_connections 2024-07-03T15:54:30.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.753+0000 7f7b4d46e640 1 --2- 172.21.15.28:0/2382385112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b48102c30 0x7f7b480fc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:30.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.753+0000 7f7b4d46e640 1 --2- 172.21.15.28:0/2382385112 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7b48102290 0x7f7b48102660 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:30.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.753+0000 7f7b4d46e640 1 -- 172.21.15.28:0/2382385112 >> 172.21.15.28:0/2382385112 conn(0x7f7b480f7e60 msgr2=0x7f7b480fa270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:30.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.753+0000 7f7b4d46e640 1 -- 172.21.15.28:0/2382385112 shutdown_connections 2024-07-03T15:54:30.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.753+0000 7f7b4d46e640 1 -- 172.21.15.28:0/2382385112 wait complete. 2024-07-03T15:54:30.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.754+0000 7f7b4d46e640 1 Processor -- start 2024-07-03T15:54:30.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.754+0000 7f7b4d46e640 1 -- start start 2024-07-03T15:54:30.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.754+0000 7f7b4d46e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b48102290 0x7f7b4806f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:30.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.754+0000 7f7b4d46e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7b48102c30 0x7f7b4806fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:30.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.754+0000 7f7b4d46e640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b48071700 con 0x7f7b48102c30 2024-07-03T15:54:30.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.754+0000 7f7b4d46e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b480702a0 con 0x7f7b48102290 2024-07-03T15:54:30.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.754+0000 7f7b467fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7b48102c30 0x7f7b4806fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:30.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.754+0000 7f7b467fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7b48102c30 0x7f7b4806fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47284/0 (socket says 172.21.15.28:47284) 2024-07-03T15:54:30.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.754+0000 7f7b467fc640 1 -- 172.21.15.28:0/1886767019 learned_addr learned my addr 172.21.15.28:0/1886767019 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:54:30.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.754+0000 7f7b46ffd640 1 --2- 172.21.15.28:0/1886767019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b48102290 0x7f7b4806f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:30.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.754+0000 7f7b467fc640 1 -- 172.21.15.28:0/1886767019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b48102290 msgr2=0x7f7b4806f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:30.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.754+0000 7f7b467fc640 1 --2- 172.21.15.28:0/1886767019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b48102290 0x7f7b4806f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:30.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.754+0000 7f7b467fc640 1 -- 172.21.15.28:0/1886767019 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b38005040 con 0x7f7b48102c30 2024-07-03T15:54:30.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.754+0000 7f7b467fc640 1 --2- 172.21.15.28:0/1886767019 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7b48102c30 0x7f7b4806fd60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c00c830 tx=0x7f7b3c004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:30.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.755+0000 7f7b2f7fe640 1 -- 172.21.15.28:0/1886767019 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b3c00a850 con 0x7f7b48102c30 2024-07-03T15:54:30.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.755+0000 7f7b2f7fe640 1 -- 172.21.15.28:0/1886767019 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7b3c00ae80 con 0x7f7b48102c30 2024-07-03T15:54:30.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.755+0000 7f7b4d46e640 1 -- 172.21.15.28:0/1886767019 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b48070580 con 0x7f7b48102c30 2024-07-03T15:54:30.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.755+0000 7f7b2f7fe640 1 -- 172.21.15.28:0/1886767019 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b3c012ea0 con 0x7f7b48102c30 2024-07-03T15:54:30.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.755+0000 7f7b4d46e640 1 -- 172.21.15.28:0/1886767019 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b48119410 con 0x7f7b48102c30 2024-07-03T15:54:30.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.756+0000 7f7b4d46e640 1 -- 172.21.15.28:0/1886767019 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b480fda00 con 0x7f7b48102c30 2024-07-03T15:54:30.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.756+0000 7f7b2f7fe640 1 -- 172.21.15.28:0/1886767019 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7b3c012400 con 0x7f7b48102c30 2024-07-03T15:54:30.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.757+0000 7f7b2f7fe640 1 --2- 172.21.15.28:0/1886767019 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f7b3006e0b0 0x7f7b30070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:30.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.757+0000 7f7b2f7fe640 1 -- 172.21.15.28:0/1886767019 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f7b3c08de40 con 0x7f7b48102c30 2024-07-03T15:54:30.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.757+0000 7f7b46ffd640 1 --2- 172.21.15.28:0/1886767019 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f7b3006e0b0 0x7f7b30070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:30.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.757+0000 7f7b46ffd640 1 --2- 172.21.15.28:0/1886767019 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f7b3006e0b0 0x7f7b30070560 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7b38003080 tx=0x7f7b38009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:30.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.761+0000 7f7b2f7fe640 1 -- 172.21.15.28:0/1886767019 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f7b3c057190 con 0x7f7b48102c30 2024-07-03T15:54:30.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.982+0000 7f7b4d46e640 1 -- 172.21.15.28:0/1886767019 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7f7b48064400 con 0x7f7b48102c30 2024-07-03T15:54:30.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.983+0000 7f7b2f7fe640 1 -- 172.21.15.28:0/1886767019 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v35) v1 ==== 107+0+4086 (secure 0 0 0) 0x7f7b3c0126b0 con 0x7f7b48102c30 2024-07-03T15:54:30.987 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:54:30.987 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24281,"name":"cephfs.smithi148.xgpuld","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6825/2347536398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2347536398},{"type":"v1","addr":"172.21.15.148:6825","nonce":2347536398}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34372,"name":"cephfs.smithi028.wmbxew","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6829/1224242160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1224242160},{"type":"v1","addr":"172.21.15.28:6829","nonce":1224242160}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-03T15:41:40.656298+0000","modified":"2024-07-03T15:50:32.508195+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24277},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24277":{"gid":24277,"name":"cephfs.smithi148.szomfl","rank":0,"incarnation":21,"state":"up:replay","state_seq":2,"addr":"172.21.15.148:6827/4171231858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":4171231858},{"type":"v1","addr":"172.21.15.148:6827","nonce":4171231858}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-07-03T15:54:30.987 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 21 2024-07-03T15:54:30.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.985+0000 7f7b4d46e640 1 -- 172.21.15.28:0/1886767019 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f7b3006e0b0 msgr2=0x7f7b30070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:30.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.985+0000 7f7b4d46e640 1 --2- 172.21.15.28:0/1886767019 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f7b3006e0b0 0x7f7b30070560 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7b38003080 tx=0x7f7b38009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:30.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.985+0000 7f7b4d46e640 1 -- 172.21.15.28:0/1886767019 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7b48102c30 msgr2=0x7f7b4806fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:30.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.985+0000 7f7b4d46e640 1 --2- 172.21.15.28:0/1886767019 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7b48102c30 0x7f7b4806fd60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c00c830 tx=0x7f7b3c004430 comp rx=0 tx=0).stop 2024-07-03T15:54:30.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.985+0000 7f7b4d46e640 1 -- 172.21.15.28:0/1886767019 shutdown_connections 2024-07-03T15:54:30.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.985+0000 7f7b4d46e640 1 --2- 172.21.15.28:0/1886767019 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f7b3006e0b0 0x7f7b30070560 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:30.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.985+0000 7f7b4d46e640 1 --2- 172.21.15.28:0/1886767019 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7b48102c30 0x7f7b4806fd60 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:30.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.985+0000 7f7b4d46e640 1 --2- 172.21.15.28:0/1886767019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b48102290 0x7f7b4806f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:30.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.985+0000 7f7b4d46e640 1 -- 172.21.15.28:0/1886767019 >> 172.21.15.28:0/1886767019 conn(0x7f7b480f7e60 msgr2=0x7f7b480f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:30.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.985+0000 7f7b4d46e640 1 -- 172.21.15.28:0/1886767019 shutdown_connections 2024-07-03T15:54:30.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:30.985+0000 7f7b4d46e640 1 -- 172.21.15.28:0/1886767019 wait complete. 2024-07-03T15:54:30.999 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 21 2024-07-03T15:54:30.999 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-07-03T15:54:31.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.097+0000 7f830071d640 1 -- 172.21.15.28:0/3885985175 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f82f8101ac0 msgr2=0x7f82f80fb150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:31.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.097+0000 7f830071d640 1 --2- 172.21.15.28:0/3885985175 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f82f8101ac0 0x7f82f80fb150 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f82e80048d0 tx=0x7f82e8009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:31.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.097+0000 7f830071d640 1 -- 172.21.15.28:0/3885985175 shutdown_connections 2024-07-03T15:54:31.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.097+0000 7f830071d640 1 --2- 172.21.15.28:0/3885985175 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f82f8101ac0 0x7f82f80fb150 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:31.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.097+0000 7f830071d640 1 --2- 172.21.15.28:0/3885985175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82f8101120 0x7f82f81014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:31.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.097+0000 7f830071d640 1 -- 172.21.15.28:0/3885985175 >> 172.21.15.28:0/3885985175 conn(0x7f82f80f6c90 msgr2=0x7f82f80f90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:31.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.097+0000 7f830071d640 1 -- 172.21.15.28:0/3885985175 shutdown_connections 2024-07-03T15:54:31.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.097+0000 7f830071d640 1 -- 172.21.15.28:0/3885985175 wait complete. 2024-07-03T15:54:31.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.098+0000 7f830071d640 1 Processor -- start 2024-07-03T15:54:31.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.098+0000 7f830071d640 1 -- start start 2024-07-03T15:54:31.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.098+0000 7f830071d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f82f8101120 0x7f82f8111c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:31.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.098+0000 7f830071d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82f8101ac0 0x7f82f81121a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:31.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.098+0000 7f830071d640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f81127a0 con 0x7f82f8101120 2024-07-03T15:54:31.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.098+0000 7f830071d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f810bdf0 con 0x7f82f8101ac0 2024-07-03T15:54:31.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.098+0000 7f82fe4b9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f82f8101120 0x7f82f8111c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:31.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.098+0000 7f82fe4b9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f82f8101120 0x7f82f8111c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47310/0 (socket says 172.21.15.28:47310) 2024-07-03T15:54:31.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.098+0000 7f82fe4b9640 1 -- 172.21.15.28:0/4153623391 learned_addr learned my addr 172.21.15.28:0/4153623391 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:54:31.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.098+0000 7f82fdcb8640 1 --2- 172.21.15.28:0/4153623391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82f8101ac0 0x7f82f81121a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:31.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.098+0000 7f82fe4b9640 1 -- 172.21.15.28:0/4153623391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82f8101ac0 msgr2=0x7f82f81121a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:31.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.098+0000 7f82fe4b9640 1 --2- 172.21.15.28:0/4153623391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82f8101ac0 0x7f82f81121a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:31.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.098+0000 7f82fe4b9640 1 -- 172.21.15.28:0/4153623391 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e8005040 con 0x7f82f8101120 2024-07-03T15:54:31.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.098+0000 7f82fe4b9640 1 --2- 172.21.15.28:0/4153623391 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f82f8101120 0x7f82f8111c60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f82f4002a50 tx=0x7f82f4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:31.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.099+0000 7f82eeffd640 1 -- 172.21.15.28:0/4153623391 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82f4007c30 con 0x7f82f8101120 2024-07-03T15:54:31.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.099+0000 7f830071d640 1 -- 172.21.15.28:0/4153623391 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82f810c0d0 con 0x7f82f8101120 2024-07-03T15:54:31.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.099+0000 7f82eeffd640 1 -- 172.21.15.28:0/4153623391 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f82f400f040 con 0x7f82f8101120 2024-07-03T15:54:31.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.099+0000 7f82eeffd640 1 -- 172.21.15.28:0/4153623391 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82f400db50 con 0x7f82f8101120 2024-07-03T15:54:31.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.099+0000 7f830071d640 1 -- 172.21.15.28:0/4153623391 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82f810c5c0 con 0x7f82f8101120 2024-07-03T15:54:31.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.100+0000 7f830071d640 1 -- 172.21.15.28:0/4153623391 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82f80fc890 con 0x7f82f8101120 2024-07-03T15:54:31.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.100+0000 7f82eeffd640 1 -- 172.21.15.28:0/4153623391 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f82f401c400 con 0x7f82f8101120 2024-07-03T15:54:31.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.101+0000 7f82eeffd640 1 --2- 172.21.15.28:0/4153623391 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f82e406e0b0 0x7f82e4070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:31.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.101+0000 7f82eeffd640 1 -- 172.21.15.28:0/4153623391 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f82f408d420 con 0x7f82f8101120 2024-07-03T15:54:31.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.101+0000 7f82fdcb8640 1 --2- 172.21.15.28:0/4153623391 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f82e406e0b0 0x7f82e4070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:31.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.101+0000 7f82fdcb8640 1 --2- 172.21.15.28:0/4153623391 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f82e406e0b0 0x7f82e4070560 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f82f80f9760 tx=0x7f82e8009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:31.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.105+0000 7f82eeffd640 1 -- 172.21.15.28:0/4153623391 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f82f40567a0 con 0x7f82f8101120 2024-07-03T15:54:31.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:30 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/1181096454' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-07-03T15:54:31.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:30 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/1090333299' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-07-03T15:54:31.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:30 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/2177325776' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-07-03T15:54:31.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.329+0000 7f830071d640 1 -- 172.21.15.28:0/4153623391 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7f82f810d2d0 con 0x7f82f8101120 2024-07-03T15:54:31.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.329+0000 7f82eeffd640 1 -- 172.21.15.28:0/4153623391 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v35) v1 ==== 107+0+4933 (secure 0 0 0) 0x7f82f4059e60 con 0x7f82f8101120 2024-07-03T15:54:31.333 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:54:31.333 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24281,"name":"cephfs.smithi148.xgpuld","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6825/2347536398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2347536398},{"type":"v1","addr":"172.21.15.148:6825","nonce":2347536398}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34372,"name":"cephfs.smithi028.wmbxew","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6829/1224242160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1224242160},{"type":"v1","addr":"172.21.15.28:6829","nonce":1224242160}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":34376,"name":"cephfs.smithi028.rhpvve","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6827/3127260162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":3127260162},{"type":"v1","addr":"172.21.15.28:6827","nonce":3127260162}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-03T15:41:40.656298+0000","modified":"2024-07-03T15:50:32.508195+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24277},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24277":{"gid":24277,"name":"cephfs.smithi148.szomfl","rank":0,"incarnation":21,"state":"up:replay","state_seq":2,"addr":"172.21.15.148:6827/4171231858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":4171231858},{"type":"v1","addr":"172.21.15.148:6827","nonce":4171231858}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-07-03T15:54:31.333 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 22 2024-07-03T15:54:31.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.332+0000 7f830071d640 1 -- 172.21.15.28:0/4153623391 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f82e406e0b0 msgr2=0x7f82e4070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:31.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.332+0000 7f830071d640 1 --2- 172.21.15.28:0/4153623391 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f82e406e0b0 0x7f82e4070560 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f82f80f9760 tx=0x7f82e8009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:31.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.332+0000 7f830071d640 1 -- 172.21.15.28:0/4153623391 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f82f8101120 msgr2=0x7f82f8111c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:31.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.332+0000 7f830071d640 1 --2- 172.21.15.28:0/4153623391 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f82f8101120 0x7f82f8111c60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f82f4002a50 tx=0x7f82f4004600 comp rx=0 tx=0).stop 2024-07-03T15:54:31.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.332+0000 7f830071d640 1 -- 172.21.15.28:0/4153623391 shutdown_connections 2024-07-03T15:54:31.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.332+0000 7f830071d640 1 --2- 172.21.15.28:0/4153623391 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f82e406e0b0 0x7f82e4070560 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:31.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.332+0000 7f830071d640 1 --2- 172.21.15.28:0/4153623391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82f8101ac0 0x7f82f81121a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:31.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.332+0000 7f830071d640 1 --2- 172.21.15.28:0/4153623391 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f82f8101120 0x7f82f8111c60 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:31.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.332+0000 7f830071d640 1 -- 172.21.15.28:0/4153623391 >> 172.21.15.28:0/4153623391 conn(0x7f82f80f6c90 msgr2=0x7f82f80f7c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:31.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.332+0000 7f830071d640 1 -- 172.21.15.28:0/4153623391 shutdown_connections 2024-07-03T15:54:31.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.332+0000 7f830071d640 1 -- 172.21.15.28:0/4153623391 wait complete. 2024-07-03T15:54:31.345 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:30 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/1181096454' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-07-03T15:54:31.345 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:30 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/1090333299' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-07-03T15:54:31.345 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:30 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/2177325776' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-07-03T15:54:31.346 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 22 2024-07-03T15:54:31.346 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-07-03T15:54:31.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.443+0000 7f1cb099d640 1 -- 172.21.15.28:0/1425509001 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1cac102c20 msgr2=0x7f1cac0fc2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:31.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.443+0000 7f1cb099d640 1 --2- 172.21.15.28:0/1425509001 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1cac102c20 0x7f1cac0fc2b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1c940048d0 tx=0x7f1c94009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:31.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.443+0000 7f1cb099d640 1 -- 172.21.15.28:0/1425509001 shutdown_connections 2024-07-03T15:54:31.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.443+0000 7f1cb099d640 1 --2- 172.21.15.28:0/1425509001 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1cac102c20 0x7f1cac0fc2b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:31.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.443+0000 7f1cb099d640 1 --2- 172.21.15.28:0/1425509001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cac102280 0x7f1cac102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:31.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.443+0000 7f1cb099d640 1 -- 172.21.15.28:0/1425509001 >> 172.21.15.28:0/1425509001 conn(0x7f1cac0f7e10 msgr2=0x7f1cac0fa220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:31.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.443+0000 7f1cb099d640 1 -- 172.21.15.28:0/1425509001 shutdown_connections 2024-07-03T15:54:31.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.443+0000 7f1cb099d640 1 -- 172.21.15.28:0/1425509001 wait complete. 2024-07-03T15:54:31.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.444+0000 7f1cb099d640 1 Processor -- start 2024-07-03T15:54:31.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.444+0000 7f1cb099d640 1 -- start start 2024-07-03T15:54:31.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.444+0000 7f1cb099d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1cac102280 0x7f1cac112d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:31.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.444+0000 7f1cb099d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cac102c20 0x7f1cac1132d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:31.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.444+0000 7f1cb099d640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac1138d0 con 0x7f1cac102280 2024-07-03T15:54:31.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.444+0000 7f1cb099d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac113a40 con 0x7f1cac102c20 2024-07-03T15:54:31.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.444+0000 7f1caa59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1cac102280 0x7f1cac112d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:31.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.444+0000 7f1caa59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1cac102280 0x7f1cac112d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47320/0 (socket says 172.21.15.28:47320) 2024-07-03T15:54:31.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.444+0000 7f1caa59c640 1 -- 172.21.15.28:0/985465244 learned_addr learned my addr 172.21.15.28:0/985465244 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:54:31.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.444+0000 7f1caa59c640 1 -- 172.21.15.28:0/985465244 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cac102c20 msgr2=0x7f1cac1132d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:31.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.444+0000 7f1caa59c640 1 --2- 172.21.15.28:0/985465244 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cac102c20 0x7f1cac1132d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:31.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.444+0000 7f1caa59c640 1 -- 172.21.15.28:0/985465244 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c94005040 con 0x7f1cac102280 2024-07-03T15:54:31.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.445+0000 7f1caa59c640 1 --2- 172.21.15.28:0/985465244 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1cac102280 0x7f1cac112d90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c00c790 tx=0x7f1c9c0043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:31.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.445+0000 7f1ca2ffd640 1 -- 172.21.15.28:0/985465244 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c9c00a820 con 0x7f1cac102280 2024-07-03T15:54:31.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.445+0000 7f1cb099d640 1 -- 172.21.15.28:0/985465244 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cac10d0d0 con 0x7f1cac102280 2024-07-03T15:54:31.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.445+0000 7f1ca2ffd640 1 -- 172.21.15.28:0/985465244 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1c9c00ae50 con 0x7f1cac102280 2024-07-03T15:54:31.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.445+0000 7f1ca2ffd640 1 -- 172.21.15.28:0/985465244 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c9c0115e0 con 0x7f1cac102280 2024-07-03T15:54:31.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.445+0000 7f1cb099d640 1 -- 172.21.15.28:0/985465244 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cac10d5f0 con 0x7f1cac102280 2024-07-03T15:54:31.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.446+0000 7f1cb099d640 1 -- 172.21.15.28:0/985465244 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cac0fd9f0 con 0x7f1cac102280 2024-07-03T15:54:31.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.446+0000 7f1ca2ffd640 1 -- 172.21.15.28:0/985465244 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1c9c012400 con 0x7f1cac102280 2024-07-03T15:54:31.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.447+0000 7f1ca2ffd640 1 --2- 172.21.15.28:0/985465244 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f1c9806e0b0 0x7f1c98070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:31.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.447+0000 7f1ca2ffd640 1 -- 172.21.15.28:0/985465244 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f1c9c08dd90 con 0x7f1cac102280 2024-07-03T15:54:31.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.447+0000 7f1ca9d9b640 1 --2- 172.21.15.28:0/985465244 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f1c9806e0b0 0x7f1c98070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:31.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.448+0000 7f1ca9d9b640 1 --2- 172.21.15.28:0/985465244 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f1c9806e0b0 0x7f1c98070560 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1c94003080 tx=0x7f1c94009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:31.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.451+0000 7f1ca2ffd640 1 -- 172.21.15.28:0/985465244 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f1c9c0570e0 con 0x7f1cac102280 2024-07-03T15:54:31.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.678+0000 7f1cb099d640 1 -- 172.21.15.28:0/985465244 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7f1cac064400 con 0x7f1cac102280 2024-07-03T15:54:31.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.679+0000 7f1ca2ffd640 1 -- 172.21.15.28:0/985465244 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v35) v1 ==== 107+0+4938 (secure 0 0 0) 0x7f1c9c05a7a0 con 0x7f1cac102280 2024-07-03T15:54:31.682 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:54:31.682 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24281,"name":"cephfs.smithi148.xgpuld","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6825/2347536398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2347536398},{"type":"v1","addr":"172.21.15.148:6825","nonce":2347536398}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34372,"name":"cephfs.smithi028.wmbxew","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6829/1224242160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1224242160},{"type":"v1","addr":"172.21.15.28:6829","nonce":1224242160}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":34376,"name":"cephfs.smithi028.rhpvve","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6827/3127260162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":3127260162},{"type":"v1","addr":"172.21.15.28:6827","nonce":3127260162}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-03T15:41:40.656298+0000","modified":"2024-07-03T15:50:39.478078+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24277},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24277":{"gid":24277,"name":"cephfs.smithi148.szomfl","rank":0,"incarnation":21,"state":"up:reconnect","state_seq":133,"addr":"172.21.15.148:6827/4171231858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":4171231858},{"type":"v1","addr":"172.21.15.148:6827","nonce":4171231858}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-07-03T15:54:31.682 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 23 2024-07-03T15:54:31.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.681+0000 7f1cb099d640 1 -- 172.21.15.28:0/985465244 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f1c9806e0b0 msgr2=0x7f1c98070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:31.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.681+0000 7f1cb099d640 1 --2- 172.21.15.28:0/985465244 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f1c9806e0b0 0x7f1c98070560 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1c94003080 tx=0x7f1c94009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:31.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.681+0000 7f1cb099d640 1 -- 172.21.15.28:0/985465244 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1cac102280 msgr2=0x7f1cac112d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:31.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.681+0000 7f1cb099d640 1 --2- 172.21.15.28:0/985465244 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1cac102280 0x7f1cac112d90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c00c790 tx=0x7f1c9c0043c0 comp rx=0 tx=0).stop 2024-07-03T15:54:31.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.681+0000 7f1cb099d640 1 -- 172.21.15.28:0/985465244 shutdown_connections 2024-07-03T15:54:31.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.681+0000 7f1cb099d640 1 --2- 172.21.15.28:0/985465244 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f1c9806e0b0 0x7f1c98070560 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:31.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.681+0000 7f1cb099d640 1 --2- 172.21.15.28:0/985465244 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cac102c20 0x7f1cac1132d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:31.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.681+0000 7f1cb099d640 1 --2- 172.21.15.28:0/985465244 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1cac102280 0x7f1cac112d90 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:31.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.681+0000 7f1cb099d640 1 -- 172.21.15.28:0/985465244 >> 172.21.15.28:0/985465244 conn(0x7f1cac0f7e10 msgr2=0x7f1cac0f8e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:31.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.681+0000 7f1cb099d640 1 -- 172.21.15.28:0/985465244 shutdown_connections 2024-07-03T15:54:31.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.681+0000 7f1cb099d640 1 -- 172.21.15.28:0/985465244 wait complete. 2024-07-03T15:54:31.695 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 23 2024-07-03T15:54:31.695 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-07-03T15:54:31.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.794+0000 7f93f195d640 1 -- 172.21.15.28:0/3835815044 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f93ec102310 msgr2=0x7f93ec1026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:31.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.794+0000 7f93f195d640 1 --2- 172.21.15.28:0/3835815044 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f93ec102310 0x7f93ec1026e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f93dc0048d0 tx=0x7f93dc009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:31.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.794+0000 7f93f195d640 1 -- 172.21.15.28:0/3835815044 shutdown_connections 2024-07-03T15:54:31.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.794+0000 7f93f195d640 1 --2- 172.21.15.28:0/3835815044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93ec102cb0 0x7f93ec0fa970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:31.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.794+0000 7f93f195d640 1 --2- 172.21.15.28:0/3835815044 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f93ec102310 0x7f93ec1026e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:31.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.794+0000 7f93f195d640 1 -- 172.21.15.28:0/3835815044 >> 172.21.15.28:0/3835815044 conn(0x7f93ec071890 msgr2=0x7f93ec071c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:31.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.794+0000 7f93f195d640 1 -- 172.21.15.28:0/3835815044 shutdown_connections 2024-07-03T15:54:31.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.794+0000 7f93f195d640 1 -- 172.21.15.28:0/3835815044 wait complete. 2024-07-03T15:54:31.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.795+0000 7f93f195d640 1 Processor -- start 2024-07-03T15:54:31.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.795+0000 7f93f195d640 1 -- start start 2024-07-03T15:54:31.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.795+0000 7f93f195d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f93ec102310 0x7f93ec112d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:31.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.795+0000 7f93f195d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93ec102cb0 0x7f93ec113250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:31.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.795+0000 7f93f195d640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec113850 con 0x7f93ec102310 2024-07-03T15:54:31.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.795+0000 7f93f195d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec10cea0 con 0x7f93ec102cb0 2024-07-03T15:54:31.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.795+0000 7f93eaffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f93ec102310 0x7f93ec112d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:31.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.795+0000 7f93eaffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f93ec102310 0x7f93ec112d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47334/0 (socket says 172.21.15.28:47334) 2024-07-03T15:54:31.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.795+0000 7f93eaffd640 1 -- 172.21.15.28:0/4011705026 learned_addr learned my addr 172.21.15.28:0/4011705026 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:54:31.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.795+0000 7f93ea7fc640 1 --2- 172.21.15.28:0/4011705026 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93ec102cb0 0x7f93ec113250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:31.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.795+0000 7f93eaffd640 1 -- 172.21.15.28:0/4011705026 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93ec102cb0 msgr2=0x7f93ec113250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:31.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.795+0000 7f93eaffd640 1 --2- 172.21.15.28:0/4011705026 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93ec102cb0 0x7f93ec113250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:31.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.795+0000 7f93eaffd640 1 -- 172.21.15.28:0/4011705026 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93dc005040 con 0x7f93ec102310 2024-07-03T15:54:31.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.796+0000 7f93eaffd640 1 --2- 172.21.15.28:0/4011705026 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f93ec102310 0x7f93ec112d10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f93dc004930 tx=0x7f93dc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:31.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.796+0000 7f93d37fe640 1 -- 172.21.15.28:0/4011705026 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93dc0094c0 con 0x7f93ec102310 2024-07-03T15:54:31.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.796+0000 7f93d37fe640 1 -- 172.21.15.28:0/4011705026 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f93dc032e50 con 0x7f93ec102310 2024-07-03T15:54:31.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.796+0000 7f93d37fe640 1 -- 172.21.15.28:0/4011705026 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93dc0433b0 con 0x7f93ec102310 2024-07-03T15:54:31.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.796+0000 7f93f195d640 1 -- 172.21.15.28:0/4011705026 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93ec10d120 con 0x7f93ec102310 2024-07-03T15:54:31.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.796+0000 7f93f195d640 1 -- 172.21.15.28:0/4011705026 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93ec10d5e0 con 0x7f93ec102310 2024-07-03T15:54:31.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.797+0000 7f93f195d640 1 -- 172.21.15.28:0/4011705026 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93ec0655e0 con 0x7f93ec102310 2024-07-03T15:54:31.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.797+0000 7f93d37fe640 1 -- 172.21.15.28:0/4011705026 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f93dc02f070 con 0x7f93ec102310 2024-07-03T15:54:31.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.798+0000 7f93d37fe640 1 --2- 172.21.15.28:0/4011705026 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f93d406e0b0 0x7f93d4070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:31.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.798+0000 7f93ea7fc640 1 --2- 172.21.15.28:0/4011705026 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f93d406e0b0 0x7f93d4070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:31.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.798+0000 7f93d37fe640 1 -- 172.21.15.28:0/4011705026 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f93dc0afaa0 con 0x7f93ec102310 2024-07-03T15:54:31.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.799+0000 7f93ea7fc640 1 --2- 172.21.15.28:0/4011705026 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f93d406e0b0 0x7f93d4070560 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f93e0007fd0 tx=0x7f93e0009e90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:31.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:31.803+0000 7f93d37fe640 1 -- 172.21.15.28:0/4011705026 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f93dc078d70 con 0x7f93ec102310 2024-07-03T15:54:32.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.031+0000 7f93f195d640 1 -- 172.21.15.28:0/4011705026 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7f93ec064400 con 0x7f93ec102310 2024-07-03T15:54:32.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.032+0000 7f93d37fe640 1 -- 172.21.15.28:0/4011705026 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v35) v1 ==== 107+0+4935 (secure 0 0 0) 0x7f93dc07c430 con 0x7f93ec102310 2024-07-03T15:54:32.035 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:54:32.035 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24281,"name":"cephfs.smithi148.xgpuld","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6825/2347536398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2347536398},{"type":"v1","addr":"172.21.15.148:6825","nonce":2347536398}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34372,"name":"cephfs.smithi028.wmbxew","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6829/1224242160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1224242160},{"type":"v1","addr":"172.21.15.28:6829","nonce":1224242160}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":34376,"name":"cephfs.smithi028.rhpvve","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6827/3127260162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":3127260162},{"type":"v1","addr":"172.21.15.28:6827","nonce":3127260162}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-03T15:41:40.656298+0000","modified":"2024-07-03T15:50:40.342009+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24277},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24277":{"gid":24277,"name":"cephfs.smithi148.szomfl","rank":0,"incarnation":21,"state":"up:rejoin","state_seq":134,"addr":"172.21.15.148:6827/4171231858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":4171231858},{"type":"v1","addr":"172.21.15.148:6827","nonce":4171231858}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-07-03T15:54:32.035 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 24 2024-07-03T15:54:32.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.034+0000 7f93f195d640 1 -- 172.21.15.28:0/4011705026 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f93d406e0b0 msgr2=0x7f93d4070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:32.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.034+0000 7f93f195d640 1 --2- 172.21.15.28:0/4011705026 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f93d406e0b0 0x7f93d4070560 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f93e0007fd0 tx=0x7f93e0009e90 comp rx=0 tx=0).stop 2024-07-03T15:54:32.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.034+0000 7f93f195d640 1 -- 172.21.15.28:0/4011705026 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f93ec102310 msgr2=0x7f93ec112d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:32.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.034+0000 7f93f195d640 1 --2- 172.21.15.28:0/4011705026 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f93ec102310 0x7f93ec112d10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f93dc004930 tx=0x7f93dc009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:32.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.034+0000 7f93f195d640 1 -- 172.21.15.28:0/4011705026 shutdown_connections 2024-07-03T15:54:32.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.034+0000 7f93f195d640 1 --2- 172.21.15.28:0/4011705026 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f93d406e0b0 0x7f93d4070560 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:32.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.034+0000 7f93f195d640 1 --2- 172.21.15.28:0/4011705026 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93ec102cb0 0x7f93ec113250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:32.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.034+0000 7f93f195d640 1 --2- 172.21.15.28:0/4011705026 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f93ec102310 0x7f93ec112d10 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:32.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.034+0000 7f93f195d640 1 -- 172.21.15.28:0/4011705026 >> 172.21.15.28:0/4011705026 conn(0x7f93ec071890 msgr2=0x7f93ec0f9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:32.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.034+0000 7f93f195d640 1 -- 172.21.15.28:0/4011705026 shutdown_connections 2024-07-03T15:54:32.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.034+0000 7f93f195d640 1 -- 172.21.15.28:0/4011705026 wait complete. 2024-07-03T15:54:32.047 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:31 smithi028 ceph-mon[92858]: pgmap v281: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:54:32.047 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:31 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/1886767019' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-07-03T15:54:32.047 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:31 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/4153623391' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-07-03T15:54:32.047 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:31 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/985465244' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-07-03T15:54:32.048 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 24 2024-07-03T15:54:32.048 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-07-03T15:54:32.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.145+0000 7fa690014640 1 -- 172.21.15.28:0/2289777826 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa688101150 msgr2=0x7fa688101520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:32.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.145+0000 7fa690014640 1 --2- 172.21.15.28:0/2289777826 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa688101150 0x7fa688101520 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa6780048d0 tx=0x7fa678009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:32.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.145+0000 7fa690014640 1 -- 172.21.15.28:0/2289777826 shutdown_connections 2024-07-03T15:54:32.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.145+0000 7fa690014640 1 --2- 172.21.15.28:0/2289777826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa688101af0 0x7fa6880fb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:32.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.145+0000 7fa690014640 1 --2- 172.21.15.28:0/2289777826 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa688101150 0x7fa688101520 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:32.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.145+0000 7fa690014640 1 -- 172.21.15.28:0/2289777826 >> 172.21.15.28:0/2289777826 conn(0x7fa6880f6d00 msgr2=0x7fa6880f9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:32.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.145+0000 7fa690014640 1 -- 172.21.15.28:0/2289777826 shutdown_connections 2024-07-03T15:54:32.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.145+0000 7fa690014640 1 -- 172.21.15.28:0/2289777826 wait complete. 2024-07-03T15:54:32.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.146+0000 7fa690014640 1 Processor -- start 2024-07-03T15:54:32.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.146+0000 7fa690014640 1 -- start start 2024-07-03T15:54:32.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.146+0000 7fa690014640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa688101150 0x7fa688112d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:32.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.146+0000 7fa690014640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa688101af0 0x7fa6881132b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:32.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.146+0000 7fa690014640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6881138b0 con 0x7fa688101150 2024-07-03T15:54:32.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.146+0000 7fa690014640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa68810cf00 con 0x7fa688101af0 2024-07-03T15:54:32.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.146+0000 7fa68ddb0640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa688101150 0x7fa688112d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:32.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.146+0000 7fa68ddb0640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa688101150 0x7fa688112d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47354/0 (socket says 172.21.15.28:47354) 2024-07-03T15:54:32.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.146+0000 7fa68ddb0640 1 -- 172.21.15.28:0/3560773116 learned_addr learned my addr 172.21.15.28:0/3560773116 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:54:32.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.146+0000 7fa68ddb0640 1 -- 172.21.15.28:0/3560773116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa688101af0 msgr2=0x7fa6881132b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:32.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.146+0000 7fa68ddb0640 1 --2- 172.21.15.28:0/3560773116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa688101af0 0x7fa6881132b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:32.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.146+0000 7fa68ddb0640 1 -- 172.21.15.28:0/3560773116 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa678005040 con 0x7fa688101150 2024-07-03T15:54:32.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.147+0000 7fa68ddb0640 1 --2- 172.21.15.28:0/3560773116 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa688101150 0x7fa688112d70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa678004930 tx=0x7fa678009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:32.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.147+0000 7fa67e7fc640 1 -- 172.21.15.28:0/3560773116 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6780094c0 con 0x7fa688101150 2024-07-03T15:54:32.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.147+0000 7fa67e7fc640 1 -- 172.21.15.28:0/3560773116 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa678032e50 con 0x7fa688101150 2024-07-03T15:54:32.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.147+0000 7fa67e7fc640 1 -- 172.21.15.28:0/3560773116 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6780433b0 con 0x7fa688101150 2024-07-03T15:54:32.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.147+0000 7fa690014640 1 -- 172.21.15.28:0/3560773116 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa68810d180 con 0x7fa688101150 2024-07-03T15:54:32.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.147+0000 7fa690014640 1 -- 172.21.15.28:0/3560773116 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa68810d5f0 con 0x7fa688101150 2024-07-03T15:54:32.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.148+0000 7fa690014640 1 -- 172.21.15.28:0/3560773116 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6880fc8c0 con 0x7fa688101150 2024-07-03T15:54:32.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.148+0000 7fa67e7fc640 1 -- 172.21.15.28:0/3560773116 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa67802f070 con 0x7fa688101150 2024-07-03T15:54:32.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.149+0000 7fa67e7fc640 1 --2- 172.21.15.28:0/3560773116 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fa67406e0b0 0x7fa674070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:32.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.149+0000 7fa67e7fc640 1 -- 172.21.15.28:0/3560773116 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fa6780afbf0 con 0x7fa688101150 2024-07-03T15:54:32.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.149+0000 7fa68d5af640 1 --2- 172.21.15.28:0/3560773116 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fa67406e0b0 0x7fa674070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:32.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.149+0000 7fa68d5af640 1 --2- 172.21.15.28:0/3560773116 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fa67406e0b0 0x7fa674070560 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa68810e630 tx=0x7fa684007cb0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:32.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.153+0000 7fa67e7fc640 1 -- 172.21.15.28:0/3560773116 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fa678078ec0 con 0x7fa688101150 2024-07-03T15:54:32.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:31 smithi148 ceph-mon[80490]: pgmap v281: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:54:32.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:31 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/1886767019' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-07-03T15:54:32.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:31 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/4153623391' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-07-03T15:54:32.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:31 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/985465244' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-07-03T15:54:32.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.375+0000 7fa690014640 1 -- 172.21.15.28:0/3560773116 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7fa6881082e0 con 0x7fa688101150 2024-07-03T15:54:32.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.375+0000 7fa67e7fc640 1 -- 172.21.15.28:0/3560773116 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v35) v1 ==== 107+0+4944 (secure 0 0 0) 0x7fa67807c580 con 0x7fa688101150 2024-07-03T15:54:32.379 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:54:32.379 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24281,"name":"cephfs.smithi148.xgpuld","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6825/2347536398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2347536398},{"type":"v1","addr":"172.21.15.148:6825","nonce":2347536398}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34372,"name":"cephfs.smithi028.wmbxew","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6829/1224242160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1224242160},{"type":"v1","addr":"172.21.15.28:6829","nonce":1224242160}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":34376,"name":"cephfs.smithi028.rhpvve","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6827/3127260162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":3127260162},{"type":"v1","addr":"172.21.15.28:6827","nonce":3127260162}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-03T15:41:40.656298+0000","modified":"2024-07-03T15:50:42.345494+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24277},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24277":{"gid":24277,"name":"cephfs.smithi148.szomfl","rank":0,"incarnation":21,"state":"up:active","state_seq":135,"addr":"172.21.15.148:6827/4171231858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":4171231858},{"type":"v1","addr":"172.21.15.148:6827","nonce":4171231858}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24277,"qdb_cluster":[24277]},"id":1}]} 2024-07-03T15:54:32.379 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 25 2024-07-03T15:54:32.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.377+0000 7fa690014640 1 -- 172.21.15.28:0/3560773116 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fa67406e0b0 msgr2=0x7fa674070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:32.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.377+0000 7fa690014640 1 --2- 172.21.15.28:0/3560773116 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fa67406e0b0 0x7fa674070560 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa68810e630 tx=0x7fa684007cb0 comp rx=0 tx=0).stop 2024-07-03T15:54:32.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.377+0000 7fa690014640 1 -- 172.21.15.28:0/3560773116 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa688101150 msgr2=0x7fa688112d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:32.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.377+0000 7fa690014640 1 --2- 172.21.15.28:0/3560773116 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa688101150 0x7fa688112d70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa678004930 tx=0x7fa678009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:32.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.377+0000 7fa690014640 1 -- 172.21.15.28:0/3560773116 shutdown_connections 2024-07-03T15:54:32.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.377+0000 7fa690014640 1 --2- 172.21.15.28:0/3560773116 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fa67406e0b0 0x7fa674070560 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:32.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.377+0000 7fa690014640 1 --2- 172.21.15.28:0/3560773116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa688101af0 0x7fa6881132b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:32.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.377+0000 7fa690014640 1 --2- 172.21.15.28:0/3560773116 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa688101150 0x7fa688112d70 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:32.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.377+0000 7fa690014640 1 -- 172.21.15.28:0/3560773116 >> 172.21.15.28:0/3560773116 conn(0x7fa6880f6d00 msgr2=0x7fa6880f7c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:32.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.377+0000 7fa690014640 1 -- 172.21.15.28:0/3560773116 shutdown_connections 2024-07-03T15:54:32.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.377+0000 7fa690014640 1 -- 172.21.15.28:0/3560773116 wait complete. 2024-07-03T15:54:32.392 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 25 2024-07-03T15:54:32.392 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-07-03T15:54:32.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.490+0000 7fdb526ae640 1 -- 172.21.15.28:0/3919459334 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb4c102c00 msgr2=0x7fdb4c0fc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:32.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.490+0000 7fdb526ae640 1 --2- 172.21.15.28:0/3919459334 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb4c102c00 0x7fdb4c0fc290 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fdb340048d0 tx=0x7fdb34009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:32.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.490+0000 7fdb526ae640 1 -- 172.21.15.28:0/3919459334 shutdown_connections 2024-07-03T15:54:32.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.490+0000 7fdb526ae640 1 --2- 172.21.15.28:0/3919459334 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb4c102c00 0x7fdb4c0fc290 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:32.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.490+0000 7fdb526ae640 1 --2- 172.21.15.28:0/3919459334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb4c102260 0x7fdb4c102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:32.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.490+0000 7fdb526ae640 1 -- 172.21.15.28:0/3919459334 >> 172.21.15.28:0/3919459334 conn(0x7fdb4c0f7e30 msgr2=0x7fdb4c0fa240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:32.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.490+0000 7fdb526ae640 1 -- 172.21.15.28:0/3919459334 shutdown_connections 2024-07-03T15:54:32.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.490+0000 7fdb526ae640 1 -- 172.21.15.28:0/3919459334 wait complete. 2024-07-03T15:54:32.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.490+0000 7fdb526ae640 1 Processor -- start 2024-07-03T15:54:32.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.490+0000 7fdb526ae640 1 -- start start 2024-07-03T15:54:32.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.491+0000 7fdb526ae640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb4c102260 0x7fdb4c112d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:32.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.491+0000 7fdb526ae640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb4c102c00 0x7fdb4c1132b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:32.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.491+0000 7fdb526ae640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c1138b0 con 0x7fdb4c102c00 2024-07-03T15:54:32.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.491+0000 7fdb526ae640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c10cf00 con 0x7fdb4c102260 2024-07-03T15:54:32.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.491+0000 7fdb43fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb4c102c00 0x7fdb4c1132b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:32.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.491+0000 7fdb43fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb4c102c00 0x7fdb4c1132b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47384/0 (socket says 172.21.15.28:47384) 2024-07-03T15:54:32.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.491+0000 7fdb43fff640 1 -- 172.21.15.28:0/90679994 learned_addr learned my addr 172.21.15.28:0/90679994 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:54:32.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.491+0000 7fdb4bfff640 1 --2- 172.21.15.28:0/90679994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb4c102260 0x7fdb4c112d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:32.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.491+0000 7fdb43fff640 1 -- 172.21.15.28:0/90679994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb4c102260 msgr2=0x7fdb4c112d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:32.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.491+0000 7fdb43fff640 1 --2- 172.21.15.28:0/90679994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb4c102260 0x7fdb4c112d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:32.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.491+0000 7fdb43fff640 1 -- 172.21.15.28:0/90679994 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb34005040 con 0x7fdb4c102c00 2024-07-03T15:54:32.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.491+0000 7fdb43fff640 1 --2- 172.21.15.28:0/90679994 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb4c102c00 0x7fdb4c1132b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdb34034040 tx=0x7fdb34009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:32.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.491+0000 7fdb497fa640 1 -- 172.21.15.28:0/90679994 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb340092b0 con 0x7fdb4c102c00 2024-07-03T15:54:32.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.491+0000 7fdb497fa640 1 -- 172.21.15.28:0/90679994 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdb3403b400 con 0x7fdb4c102c00 2024-07-03T15:54:32.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.491+0000 7fdb526ae640 1 -- 172.21.15.28:0/90679994 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb4c10d180 con 0x7fdb4c102c00 2024-07-03T15:54:32.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.491+0000 7fdb497fa640 1 -- 172.21.15.28:0/90679994 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb340435d0 con 0x7fdb4c102c00 2024-07-03T15:54:32.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.491+0000 7fdb526ae640 1 -- 172.21.15.28:0/90679994 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb4c10d640 con 0x7fdb4c102c00 2024-07-03T15:54:32.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.492+0000 7fdb526ae640 1 -- 172.21.15.28:0/90679994 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb4c0fd9d0 con 0x7fdb4c102c00 2024-07-03T15:54:32.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.493+0000 7fdb497fa640 1 -- 172.21.15.28:0/90679994 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fdb3402f070 con 0x7fdb4c102c00 2024-07-03T15:54:32.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.493+0000 7fdb497fa640 1 --2- 172.21.15.28:0/90679994 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fdb3806e0b0 0x7fdb38070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:32.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.494+0000 7fdb4bfff640 1 --2- 172.21.15.28:0/90679994 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fdb3806e0b0 0x7fdb38070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:32.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.494+0000 7fdb497fa640 1 -- 172.21.15.28:0/90679994 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fdb340af7f0 con 0x7fdb4c102c00 2024-07-03T15:54:32.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.494+0000 7fdb4bfff640 1 --2- 172.21.15.28:0/90679994 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fdb3806e0b0 0x7fdb38070560 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c009af0 tx=0x7fdb3c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:32.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.498+0000 7fdb497fa640 1 -- 172.21.15.28:0/90679994 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fdb34078bf0 con 0x7fdb4c102c00 2024-07-03T15:54:32.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.723+0000 7fdb526ae640 1 -- 172.21.15.28:0/90679994 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7fdb4c1093c0 con 0x7fdb4c102c00 2024-07-03T15:54:32.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.724+0000 7fdb497fa640 1 -- 172.21.15.28:0/90679994 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v35) v1 ==== 107+0+4165 (secure 0 0 0) 0x7fdb3407c2b0 con 0x7fdb4c102c00 2024-07-03T15:54:32.727 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:54:32.727 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24281,"name":"cephfs.smithi148.xgpuld","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6825/2347536398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2347536398},{"type":"v1","addr":"172.21.15.148:6825","nonce":2347536398}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34376,"name":"cephfs.smithi028.rhpvve","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6827/3127260162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":3127260162},{"type":"v1","addr":"172.21.15.28:6827","nonce":3127260162}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-03T15:41:40.656298+0000","modified":"2024-07-03T15:50:42.345494+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24277},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24277":{"gid":24277,"name":"cephfs.smithi148.szomfl","rank":0,"incarnation":21,"state":"up:active","state_seq":135,"addr":"172.21.15.148:6827/4171231858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":4171231858},{"type":"v1","addr":"172.21.15.148:6827","nonce":4171231858}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24277,"qdb_cluster":[24277]},"id":1}]} 2024-07-03T15:54:32.728 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 26 2024-07-03T15:54:32.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.726+0000 7fdb526ae640 1 -- 172.21.15.28:0/90679994 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fdb3806e0b0 msgr2=0x7fdb38070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:32.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.726+0000 7fdb526ae640 1 --2- 172.21.15.28:0/90679994 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fdb3806e0b0 0x7fdb38070560 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c009af0 tx=0x7fdb3c00b040 comp rx=0 tx=0).stop 2024-07-03T15:54:32.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.726+0000 7fdb526ae640 1 -- 172.21.15.28:0/90679994 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb4c102c00 msgr2=0x7fdb4c1132b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:32.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.726+0000 7fdb526ae640 1 --2- 172.21.15.28:0/90679994 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb4c102c00 0x7fdb4c1132b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdb34034040 tx=0x7fdb34009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:32.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.726+0000 7fdb526ae640 1 -- 172.21.15.28:0/90679994 shutdown_connections 2024-07-03T15:54:32.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.726+0000 7fdb526ae640 1 --2- 172.21.15.28:0/90679994 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fdb3806e0b0 0x7fdb38070560 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:32.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.726+0000 7fdb526ae640 1 --2- 172.21.15.28:0/90679994 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb4c102c00 0x7fdb4c1132b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:32.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.726+0000 7fdb526ae640 1 --2- 172.21.15.28:0/90679994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb4c102260 0x7fdb4c112d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:32.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.726+0000 7fdb526ae640 1 -- 172.21.15.28:0/90679994 >> 172.21.15.28:0/90679994 conn(0x7fdb4c0f7e30 msgr2=0x7fdb4c0f8d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:32.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.726+0000 7fdb526ae640 1 -- 172.21.15.28:0/90679994 shutdown_connections 2024-07-03T15:54:32.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.726+0000 7fdb526ae640 1 -- 172.21.15.28:0/90679994 wait complete. 2024-07-03T15:54:32.740 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 26 2024-07-03T15:54:32.741 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-07-03T15:54:32.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.838+0000 7f0d52d9d640 1 -- 172.21.15.28:0/2397978573 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d4c101220 msgr2=0x7f0d4c1015f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:32.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.838+0000 7f0d52d9d640 1 --2- 172.21.15.28:0/2397978573 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d4c101220 0x7f0d4c1015f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c0048d0 tx=0x7f0d3c009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:32.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.838+0000 7f0d52d9d640 1 -- 172.21.15.28:0/2397978573 shutdown_connections 2024-07-03T15:54:32.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.838+0000 7f0d52d9d640 1 --2- 172.21.15.28:0/2397978573 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d4c0fb200 0x7f0d4c0fb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:32.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.838+0000 7f0d52d9d640 1 --2- 172.21.15.28:0/2397978573 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d4c101220 0x7f0d4c1015f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:32.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.838+0000 7f0d52d9d640 1 -- 172.21.15.28:0/2397978573 >> 172.21.15.28:0/2397978573 conn(0x7f0d4c072080 msgr2=0x7f0d4c072480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:32.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.838+0000 7f0d52d9d640 1 -- 172.21.15.28:0/2397978573 shutdown_connections 2024-07-03T15:54:32.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.838+0000 7f0d52d9d640 1 -- 172.21.15.28:0/2397978573 wait complete. 2024-07-03T15:54:32.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.838+0000 7f0d52d9d640 1 Processor -- start 2024-07-03T15:54:32.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.838+0000 7f0d52d9d640 1 -- start start 2024-07-03T15:54:32.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.839+0000 7f0d52d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d4c0fb200 0x7f0d4c10dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:32.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.839+0000 7f0d52d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d4c101220 0x7f0d4c10e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:32.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.839+0000 7f0d52d9d640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4c0653a0 con 0x7f0d4c0fb200 2024-07-03T15:54:32.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.839+0000 7f0d52d9d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4c10e960 con 0x7f0d4c101220 2024-07-03T15:54:32.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.839+0000 7f0d50b39640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d4c0fb200 0x7f0d4c10dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:32.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.839+0000 7f0d50b39640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d4c0fb200 0x7f0d4c10dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47408/0 (socket says 172.21.15.28:47408) 2024-07-03T15:54:32.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.839+0000 7f0d50b39640 1 -- 172.21.15.28:0/2102845527 learned_addr learned my addr 172.21.15.28:0/2102845527 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:54:32.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.839+0000 7f0d4bfff640 1 --2- 172.21.15.28:0/2102845527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d4c101220 0x7f0d4c10e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:32.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.839+0000 7f0d50b39640 1 -- 172.21.15.28:0/2102845527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d4c101220 msgr2=0x7f0d4c10e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:32.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.839+0000 7f0d50b39640 1 --2- 172.21.15.28:0/2102845527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d4c101220 0x7f0d4c10e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:32.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.839+0000 7f0d50b39640 1 -- 172.21.15.28:0/2102845527 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d3c005040 con 0x7f0d4c0fb200 2024-07-03T15:54:32.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.839+0000 7f0d4bfff640 1 --2- 172.21.15.28:0/2102845527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d4c101220 0x7f0d4c10e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-03T15:54:32.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.839+0000 7f0d50b39640 1 --2- 172.21.15.28:0/2102845527 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d4c0fb200 0x7f0d4c10dee0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c004930 tx=0x7f0d3c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:32.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.839+0000 7f0d497fa640 1 -- 172.21.15.28:0/2102845527 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d3c0094c0 con 0x7f0d4c0fb200 2024-07-03T15:54:32.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.839+0000 7f0d52d9d640 1 -- 172.21.15.28:0/2102845527 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d4c10ebe0 con 0x7f0d4c0fb200 2024-07-03T15:54:32.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.839+0000 7f0d497fa640 1 -- 172.21.15.28:0/2102845527 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d3c032e50 con 0x7f0d4c0fb200 2024-07-03T15:54:32.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.839+0000 7f0d497fa640 1 -- 172.21.15.28:0/2102845527 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d3c0433b0 con 0x7f0d4c0fb200 2024-07-03T15:54:32.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.839+0000 7f0d52d9d640 1 -- 172.21.15.28:0/2102845527 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d4c117140 con 0x7f0d4c0fb200 2024-07-03T15:54:32.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.840+0000 7f0d52d9d640 1 -- 172.21.15.28:0/2102845527 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d4c0fc920 con 0x7f0d4c0fb200 2024-07-03T15:54:32.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.841+0000 7f0d497fa640 1 -- 172.21.15.28:0/2102845527 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0d3c02f070 con 0x7f0d4c0fb200 2024-07-03T15:54:32.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.841+0000 7f0d497fa640 1 --2- 172.21.15.28:0/2102845527 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f0d3406e0b0 0x7f0d34070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:32.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.841+0000 7f0d497fa640 1 -- 172.21.15.28:0/2102845527 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f0d3c0afa30 con 0x7f0d4c0fb200 2024-07-03T15:54:32.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.842+0000 7f0d4bfff640 1 --2- 172.21.15.28:0/2102845527 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f0d3406e0b0 0x7f0d34070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:32.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.842+0000 7f0d4bfff640 1 --2- 172.21.15.28:0/2102845527 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f0d3406e0b0 0x7f0d34070560 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0d40009af0 tx=0x7f0d4000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:32.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:32.846+0000 7f0d497fa640 1 -- 172.21.15.28:0/2102845527 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0d3c078d00 con 0x7f0d4c0fb200 2024-07-03T15:54:33.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.068+0000 7f0d52d9d640 1 -- 172.21.15.28:0/2102845527 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7f0d4c063fb0 con 0x7f0d4c0fb200 2024-07-03T15:54:33.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.068+0000 7f0d497fa640 1 -- 172.21.15.28:0/2102845527 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v35) v1 ==== 107+0+5012 (secure 0 0 0) 0x7f0d4c063fb0 con 0x7f0d4c0fb200 2024-07-03T15:54:33.072 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:54:33.072 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24281,"name":"cephfs.smithi148.xgpuld","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6825/2347536398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2347536398},{"type":"v1","addr":"172.21.15.148:6825","nonce":2347536398}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34376,"name":"cephfs.smithi028.rhpvve","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6827/3127260162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":3127260162},{"type":"v1","addr":"172.21.15.28:6827","nonce":3127260162}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":34408,"name":"cephfs.smithi028.wmbxew","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6829/2908130327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":2908130327},{"type":"v1","addr":"172.21.15.28:6829","nonce":2908130327}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-03T15:41:40.656298+0000","modified":"2024-07-03T15:50:42.345494+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24277},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24277":{"gid":24277,"name":"cephfs.smithi148.szomfl","rank":0,"incarnation":21,"state":"up:active","state_seq":135,"addr":"172.21.15.148:6827/4171231858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":4171231858},{"type":"v1","addr":"172.21.15.148:6827","nonce":4171231858}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24277,"qdb_cluster":[24277]},"id":1}]} 2024-07-03T15:54:33.072 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 27 2024-07-03T15:54:33.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.070+0000 7f0d52d9d640 1 -- 172.21.15.28:0/2102845527 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f0d3406e0b0 msgr2=0x7f0d34070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:33.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.070+0000 7f0d52d9d640 1 --2- 172.21.15.28:0/2102845527 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f0d3406e0b0 0x7f0d34070560 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0d40009af0 tx=0x7f0d4000b040 comp rx=0 tx=0).stop 2024-07-03T15:54:33.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.070+0000 7f0d52d9d640 1 -- 172.21.15.28:0/2102845527 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d4c0fb200 msgr2=0x7f0d4c10dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:33.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.070+0000 7f0d52d9d640 1 --2- 172.21.15.28:0/2102845527 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d4c0fb200 0x7f0d4c10dee0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c004930 tx=0x7f0d3c009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:33.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.071+0000 7f0d52d9d640 1 -- 172.21.15.28:0/2102845527 shutdown_connections 2024-07-03T15:54:33.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.071+0000 7f0d52d9d640 1 --2- 172.21.15.28:0/2102845527 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f0d3406e0b0 0x7f0d34070560 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:33.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.071+0000 7f0d52d9d640 1 --2- 172.21.15.28:0/2102845527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d4c101220 0x7f0d4c10e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:33.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.071+0000 7f0d52d9d640 1 --2- 172.21.15.28:0/2102845527 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d4c0fb200 0x7f0d4c10dee0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:33.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.071+0000 7f0d52d9d640 1 -- 172.21.15.28:0/2102845527 >> 172.21.15.28:0/2102845527 conn(0x7f0d4c072080 msgr2=0x7f0d4c0fa390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:33.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.071+0000 7f0d52d9d640 1 -- 172.21.15.28:0/2102845527 shutdown_connections 2024-07-03T15:54:33.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.071+0000 7f0d52d9d640 1 -- 172.21.15.28:0/2102845527 wait complete. 2024-07-03T15:54:33.084 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:32 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/4011705026' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-07-03T15:54:33.084 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:32 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/3560773116' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-07-03T15:54:33.084 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:32 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/90679994' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-07-03T15:54:33.085 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 27 2024-07-03T15:54:33.085 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-07-03T15:54:33.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.182+0000 7f2a16c6e640 1 -- 172.21.15.28:0/2023774528 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a10102c80 msgr2=0x7f2a100fa970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:33.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.182+0000 7f2a16c6e640 1 --2- 172.21.15.28:0/2023774528 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a10102c80 0x7f2a100fa970 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f29fc0048d0 tx=0x7f29fc009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:33.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.182+0000 7f2a16c6e640 1 -- 172.21.15.28:0/2023774528 shutdown_connections 2024-07-03T15:54:33.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.182+0000 7f2a16c6e640 1 --2- 172.21.15.28:0/2023774528 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a10102c80 0x7f2a100fa970 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:33.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.182+0000 7f2a16c6e640 1 --2- 172.21.15.28:0/2023774528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a101022e0 0x7f2a101026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:33.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.182+0000 7f2a16c6e640 1 -- 172.21.15.28:0/2023774528 >> 172.21.15.28:0/2023774528 conn(0x7f2a10071890 msgr2=0x7f2a10071c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:33.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.182+0000 7f2a16c6e640 1 -- 172.21.15.28:0/2023774528 shutdown_connections 2024-07-03T15:54:33.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.182+0000 7f2a16c6e640 1 -- 172.21.15.28:0/2023774528 wait complete. 2024-07-03T15:54:33.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.183+0000 7f2a16c6e640 1 Processor -- start 2024-07-03T15:54:33.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.183+0000 7f2a16c6e640 1 -- start start 2024-07-03T15:54:33.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.183+0000 7f2a16c6e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a101022e0 0x7f2a10112e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:33.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.183+0000 7f2a16c6e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a10102c80 0x7f2a10113340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:33.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.183+0000 7f2a16c6e640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a10113940 con 0x7f2a10102c80 2024-07-03T15:54:33.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.183+0000 7f2a16c6e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1010cf90 con 0x7f2a101022e0 2024-07-03T15:54:33.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.183+0000 7f2a0ffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a10102c80 0x7f2a10113340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:33.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.183+0000 7f2a0ffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a10102c80 0x7f2a10113340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47428/0 (socket says 172.21.15.28:47428) 2024-07-03T15:54:33.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.183+0000 7f2a0ffff640 1 -- 172.21.15.28:0/1082260137 learned_addr learned my addr 172.21.15.28:0/1082260137 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:54:33.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.183+0000 7f2a14a0a640 1 --2- 172.21.15.28:0/1082260137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a101022e0 0x7f2a10112e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:33.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.184+0000 7f2a0ffff640 1 -- 172.21.15.28:0/1082260137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a101022e0 msgr2=0x7f2a10112e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:33.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.184+0000 7f2a0ffff640 1 --2- 172.21.15.28:0/1082260137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a101022e0 0x7f2a10112e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:33.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.184+0000 7f2a0ffff640 1 -- 172.21.15.28:0/1082260137 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29fc005040 con 0x7f2a10102c80 2024-07-03T15:54:33.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.184+0000 7f2a0ffff640 1 --2- 172.21.15.28:0/1082260137 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a10102c80 0x7f2a10113340 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f29fc02bda0 tx=0x7f29fc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:33.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.184+0000 7f2a0d7fa640 1 -- 172.21.15.28:0/1082260137 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29fc0094c0 con 0x7f2a10102c80 2024-07-03T15:54:33.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.184+0000 7f2a16c6e640 1 -- 172.21.15.28:0/1082260137 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a1010d210 con 0x7f2a10102c80 2024-07-03T15:54:33.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.184+0000 7f2a0d7fa640 1 -- 172.21.15.28:0/1082260137 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f29fc032e50 con 0x7f2a10102c80 2024-07-03T15:54:33.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.184+0000 7f2a0d7fa640 1 -- 172.21.15.28:0/1082260137 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29fc03b400 con 0x7f2a10102c80 2024-07-03T15:54:33.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.184+0000 7f2a16c6e640 1 -- 172.21.15.28:0/1082260137 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a1010d730 con 0x7f2a10102c80 2024-07-03T15:54:33.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.185+0000 7f2a16c6e640 1 -- 172.21.15.28:0/1082260137 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a10064400 con 0x7f2a10102c80 2024-07-03T15:54:33.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.186+0000 7f2a0d7fa640 1 -- 172.21.15.28:0/1082260137 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f29fc02f070 con 0x7f2a10102c80 2024-07-03T15:54:33.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.186+0000 7f2a0d7fa640 1 --2- 172.21.15.28:0/1082260137 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f2a0006e0b0 0x7f2a00070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:33.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.186+0000 7f2a0d7fa640 1 -- 172.21.15.28:0/1082260137 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f29fc0aff40 con 0x7f2a10102c80 2024-07-03T15:54:33.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.186+0000 7f2a14a0a640 1 --2- 172.21.15.28:0/1082260137 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f2a0006e0b0 0x7f2a00070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:33.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.187+0000 7f2a14a0a640 1 --2- 172.21.15.28:0/1082260137 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f2a0006e0b0 0x7f2a00070560 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2a04004ed0 tx=0x7f2a0400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:33.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.190+0000 7f2a0d7fa640 1 -- 172.21.15.28:0/1082260137 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f29fc0e4850 con 0x7f2a10102c80 2024-07-03T15:54:33.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:32 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/4011705026' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-07-03T15:54:33.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:32 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/3560773116' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-07-03T15:54:33.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:32 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/90679994' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-07-03T15:54:33.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.413+0000 7f2a16c6e640 1 -- 172.21.15.28:0/1082260137 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7f2a1010e470 con 0x7f2a10102c80 2024-07-03T15:54:33.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.414+0000 7f2a0d7fa640 1 -- 172.21.15.28:0/1082260137 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v35) v1 ==== 107+0+4230 (secure 0 0 0) 0x7f29fc079210 con 0x7f2a10102c80 2024-07-03T15:54:33.417 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:54:33.417 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34376,"name":"cephfs.smithi028.rhpvve","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6827/3127260162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":3127260162},{"type":"v1","addr":"172.21.15.28:6827","nonce":3127260162}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":34408,"name":"cephfs.smithi028.wmbxew","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6829/2908130327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":2908130327},{"type":"v1","addr":"172.21.15.28:6829","nonce":2908130327}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-03T15:41:40.656298+0000","modified":"2024-07-03T15:50:42.345494+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24277},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24277":{"gid":24277,"name":"cephfs.smithi148.szomfl","rank":0,"incarnation":21,"state":"up:active","state_seq":135,"addr":"172.21.15.148:6827/4171231858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":4171231858},{"type":"v1","addr":"172.21.15.148:6827","nonce":4171231858}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24277,"qdb_cluster":[24277]},"id":1}]} 2024-07-03T15:54:33.417 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 28 2024-07-03T15:54:33.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.416+0000 7f2a16c6e640 1 -- 172.21.15.28:0/1082260137 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f2a0006e0b0 msgr2=0x7f2a00070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:33.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.416+0000 7f2a16c6e640 1 --2- 172.21.15.28:0/1082260137 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f2a0006e0b0 0x7f2a00070560 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2a04004ed0 tx=0x7f2a0400b040 comp rx=0 tx=0).stop 2024-07-03T15:54:33.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.416+0000 7f2a16c6e640 1 -- 172.21.15.28:0/1082260137 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a10102c80 msgr2=0x7f2a10113340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:33.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.416+0000 7f2a16c6e640 1 --2- 172.21.15.28:0/1082260137 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a10102c80 0x7f2a10113340 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f29fc02bda0 tx=0x7f29fc009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:33.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.416+0000 7f2a16c6e640 1 -- 172.21.15.28:0/1082260137 shutdown_connections 2024-07-03T15:54:33.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.416+0000 7f2a16c6e640 1 --2- 172.21.15.28:0/1082260137 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f2a0006e0b0 0x7f2a00070560 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:33.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.416+0000 7f2a16c6e640 1 --2- 172.21.15.28:0/1082260137 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a10102c80 0x7f2a10113340 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:33.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.416+0000 7f2a16c6e640 1 --2- 172.21.15.28:0/1082260137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a101022e0 0x7f2a10112e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:33.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.416+0000 7f2a16c6e640 1 -- 172.21.15.28:0/1082260137 >> 172.21.15.28:0/1082260137 conn(0x7f2a10071890 msgr2=0x7f2a100fa2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:33.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.416+0000 7f2a16c6e640 1 -- 172.21.15.28:0/1082260137 shutdown_connections 2024-07-03T15:54:33.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.416+0000 7f2a16c6e640 1 -- 172.21.15.28:0/1082260137 wait complete. 2024-07-03T15:54:33.430 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 28 2024-07-03T15:54:33.430 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-07-03T15:54:33.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.529+0000 7f8284b54640 1 -- 172.21.15.28:0/1954909833 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8280102c80 msgr2=0x7f82800fa970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:33.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.529+0000 7f8284b54640 1 --2- 172.21.15.28:0/1954909833 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8280102c80 0x7f82800fa970 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f82740048d0 tx=0x7f8274009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:33.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.529+0000 7f8284b54640 1 -- 172.21.15.28:0/1954909833 shutdown_connections 2024-07-03T15:54:33.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.529+0000 7f8284b54640 1 --2- 172.21.15.28:0/1954909833 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8280102c80 0x7f82800fa970 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:33.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.529+0000 7f8284b54640 1 --2- 172.21.15.28:0/1954909833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82801022e0 0x7f82801026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:33.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.529+0000 7f8284b54640 1 -- 172.21.15.28:0/1954909833 >> 172.21.15.28:0/1954909833 conn(0x7f8280071890 msgr2=0x7f8280071c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:33.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.529+0000 7f8284b54640 1 -- 172.21.15.28:0/1954909833 shutdown_connections 2024-07-03T15:54:33.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.529+0000 7f8284b54640 1 -- 172.21.15.28:0/1954909833 wait complete. 2024-07-03T15:54:33.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.530+0000 7f8284b54640 1 Processor -- start 2024-07-03T15:54:33.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.530+0000 7f8284b54640 1 -- start start 2024-07-03T15:54:33.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.530+0000 7f8284b54640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82801022e0 0x7f8280112de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:33.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.530+0000 7f8284b54640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8280102c80 0x7f8280113320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:33.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.530+0000 7f8284b54640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8280113920 con 0x7f8280102c80 2024-07-03T15:54:33.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.530+0000 7f8284b54640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828010cf70 con 0x7f82801022e0 2024-07-03T15:54:33.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.530+0000 7f827dd9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8280102c80 0x7f8280113320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:33.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.530+0000 7f827dd9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8280102c80 0x7f8280113320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47444/0 (socket says 172.21.15.28:47444) 2024-07-03T15:54:33.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.530+0000 7f827dd9b640 1 -- 172.21.15.28:0/1114529787 learned_addr learned my addr 172.21.15.28:0/1114529787 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:54:33.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.530+0000 7f827dd9b640 1 -- 172.21.15.28:0/1114529787 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82801022e0 msgr2=0x7f8280112de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:33.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.530+0000 7f827dd9b640 1 --2- 172.21.15.28:0/1114529787 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82801022e0 0x7f8280112de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:33.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.530+0000 7f827dd9b640 1 -- 172.21.15.28:0/1114529787 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8274005040 con 0x7f8280102c80 2024-07-03T15:54:33.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.530+0000 7f827dd9b640 1 --2- 172.21.15.28:0/1114529787 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8280102c80 0x7f8280113320 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f827402bda0 tx=0x7f8274009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:33.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.531+0000 7f826effd640 1 -- 172.21.15.28:0/1114529787 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82740094c0 con 0x7f8280102c80 2024-07-03T15:54:33.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.531+0000 7f826effd640 1 -- 172.21.15.28:0/1114529787 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8274032e50 con 0x7f8280102c80 2024-07-03T15:54:33.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.531+0000 7f8284b54640 1 -- 172.21.15.28:0/1114529787 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f828010d1f0 con 0x7f8280102c80 2024-07-03T15:54:33.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.531+0000 7f826effd640 1 -- 172.21.15.28:0/1114529787 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f827403b400 con 0x7f8280102c80 2024-07-03T15:54:33.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.531+0000 7f8284b54640 1 -- 172.21.15.28:0/1114529787 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f828010d710 con 0x7f8280102c80 2024-07-03T15:54:33.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.532+0000 7f8284b54640 1 -- 172.21.15.28:0/1114529787 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82800655e0 con 0x7f8280102c80 2024-07-03T15:54:33.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.532+0000 7f826effd640 1 -- 172.21.15.28:0/1114529787 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f827402f070 con 0x7f8280102c80 2024-07-03T15:54:33.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.533+0000 7f826effd640 1 --2- 172.21.15.28:0/1114529787 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f826806e0b0 0x7f8268070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:33.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.533+0000 7f826effd640 1 -- 172.21.15.28:0/1114529787 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f82740affb0 con 0x7f8280102c80 2024-07-03T15:54:33.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.533+0000 7f827e59c640 1 --2- 172.21.15.28:0/1114529787 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f826806e0b0 0x7f8268070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:33.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.533+0000 7f827e59c640 1 --2- 172.21.15.28:0/1114529787 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f826806e0b0 0x7f8268070560 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8270004c80 tx=0x7f8270009c50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:33.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.537+0000 7f826effd640 1 -- 172.21.15.28:0/1114529787 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f82740e4850 con 0x7f8280102c80 2024-07-03T15:54:33.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.760+0000 7f8284b54640 1 -- 172.21.15.28:0/1114529787 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7f8280109440 con 0x7f8280102c80 2024-07-03T15:54:33.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.761+0000 7f826effd640 1 -- 172.21.15.28:0/1114529787 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v35) v1 ==== 107+0+5080 (secure 0 0 0) 0x7f8274079260 con 0x7f8280102c80 2024-07-03T15:54:33.764 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:54:33.764 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34376,"name":"cephfs.smithi028.rhpvve","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6827/3127260162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":3127260162},{"type":"v1","addr":"172.21.15.28:6827","nonce":3127260162}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":34408,"name":"cephfs.smithi028.wmbxew","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6829/2908130327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":2908130327},{"type":"v1","addr":"172.21.15.28:6829","nonce":2908130327}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44301,"name":"cephfs.smithi148.xgpuld","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6825/1480077002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1480077002},{"type":"v1","addr":"172.21.15.148:6825","nonce":1480077002}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-03T15:41:40.656298+0000","modified":"2024-07-03T15:50:42.345494+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24277},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24277":{"gid":24277,"name":"cephfs.smithi148.szomfl","rank":0,"incarnation":21,"state":"up:active","state_seq":135,"addr":"172.21.15.148:6827/4171231858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":4171231858},{"type":"v1","addr":"172.21.15.148:6827","nonce":4171231858}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24277,"qdb_cluster":[24277]},"id":1}]} 2024-07-03T15:54:33.765 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 29 2024-07-03T15:54:33.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.763+0000 7f8284b54640 1 -- 172.21.15.28:0/1114529787 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f826806e0b0 msgr2=0x7f8268070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:33.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.763+0000 7f8284b54640 1 --2- 172.21.15.28:0/1114529787 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f826806e0b0 0x7f8268070560 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8270004c80 tx=0x7f8270009c50 comp rx=0 tx=0).stop 2024-07-03T15:54:33.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.763+0000 7f8284b54640 1 -- 172.21.15.28:0/1114529787 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8280102c80 msgr2=0x7f8280113320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:33.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.763+0000 7f8284b54640 1 --2- 172.21.15.28:0/1114529787 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8280102c80 0x7f8280113320 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f827402bda0 tx=0x7f8274009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:33.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.763+0000 7f8284b54640 1 -- 172.21.15.28:0/1114529787 shutdown_connections 2024-07-03T15:54:33.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.763+0000 7f8284b54640 1 --2- 172.21.15.28:0/1114529787 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f826806e0b0 0x7f8268070560 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:33.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.763+0000 7f8284b54640 1 --2- 172.21.15.28:0/1114529787 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8280102c80 0x7f8280113320 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:33.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.763+0000 7f8284b54640 1 --2- 172.21.15.28:0/1114529787 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82801022e0 0x7f8280112de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:33.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.763+0000 7f8284b54640 1 -- 172.21.15.28:0/1114529787 >> 172.21.15.28:0/1114529787 conn(0x7f8280071890 msgr2=0x7f82800fa2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:33.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.763+0000 7f8284b54640 1 -- 172.21.15.28:0/1114529787 shutdown_connections 2024-07-03T15:54:33.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.763+0000 7f8284b54640 1 -- 172.21.15.28:0/1114529787 wait complete. 2024-07-03T15:54:33.777 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 29 2024-07-03T15:54:33.778 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 30 2024-07-03T15:54:33.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.875+0000 7f6247e2e640 1 -- 172.21.15.28:0/2937774669 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6240101150 msgr2=0x7f6240101520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:33.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.875+0000 7f6247e2e640 1 --2- 172.21.15.28:0/2937774669 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6240101150 0x7f6240101520 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f623c0048d0 tx=0x7f623c009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:33.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.875+0000 7f6247e2e640 1 -- 172.21.15.28:0/2937774669 shutdown_connections 2024-07-03T15:54:33.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.875+0000 7f6247e2e640 1 --2- 172.21.15.28:0/2937774669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6240101af0 0x7f62400fb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:33.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.875+0000 7f6247e2e640 1 --2- 172.21.15.28:0/2937774669 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6240101150 0x7f6240101520 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:33.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.875+0000 7f6247e2e640 1 -- 172.21.15.28:0/2937774669 >> 172.21.15.28:0/2937774669 conn(0x7f62400f6d00 msgr2=0x7f62400f9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:33.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.875+0000 7f6247e2e640 1 -- 172.21.15.28:0/2937774669 shutdown_connections 2024-07-03T15:54:33.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.875+0000 7f6247e2e640 1 -- 172.21.15.28:0/2937774669 wait complete. 2024-07-03T15:54:33.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.875+0000 7f6247e2e640 1 Processor -- start 2024-07-03T15:54:33.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.875+0000 7f6247e2e640 1 -- start start 2024-07-03T15:54:33.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.876+0000 7f6247e2e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6240101150 0x7f624010df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:33.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.876+0000 7f6247e2e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6240101af0 0x7f624010e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:33.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.876+0000 7f6247e2e640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62400667b0 con 0x7f6240101af0 2024-07-03T15:54:33.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.876+0000 7f6247e2e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f624010e9b0 con 0x7f6240101150 2024-07-03T15:54:33.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.876+0000 7f62453c9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6240101af0 0x7f624010e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:33.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.876+0000 7f62453c9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6240101af0 0x7f624010e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47456/0 (socket says 172.21.15.28:47456) 2024-07-03T15:54:33.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.876+0000 7f62453c9640 1 -- 172.21.15.28:0/3710159357 learned_addr learned my addr 172.21.15.28:0/3710159357 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:54:33.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.876+0000 7f6245bca640 1 --2- 172.21.15.28:0/3710159357 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6240101150 0x7f624010df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:33.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.876+0000 7f62453c9640 1 -- 172.21.15.28:0/3710159357 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6240101150 msgr2=0x7f624010df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:33.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.876+0000 7f62453c9640 1 --2- 172.21.15.28:0/3710159357 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6240101150 0x7f624010df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:33.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.876+0000 7f62453c9640 1 -- 172.21.15.28:0/3710159357 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f623c005040 con 0x7f6240101af0 2024-07-03T15:54:33.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.876+0000 7f6245bca640 1 --2- 172.21.15.28:0/3710159357 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6240101150 0x7f624010df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-03T15:54:33.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.876+0000 7f62453c9640 1 --2- 172.21.15.28:0/3710159357 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6240101af0 0x7f624010e470 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6230002a50 tx=0x7f6230004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:33.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.877+0000 7f62367fc640 1 -- 172.21.15.28:0/3710159357 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6230007c30 con 0x7f6240101af0 2024-07-03T15:54:33.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.877+0000 7f6247e2e640 1 -- 172.21.15.28:0/3710159357 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f624010ec90 con 0x7f6240101af0 2024-07-03T15:54:33.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.877+0000 7f62367fc640 1 -- 172.21.15.28:0/3710159357 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f623000f040 con 0x7f6240101af0 2024-07-03T15:54:33.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.877+0000 7f62367fc640 1 -- 172.21.15.28:0/3710159357 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f623000db50 con 0x7f6240101af0 2024-07-03T15:54:33.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.877+0000 7f6247e2e640 1 -- 172.21.15.28:0/3710159357 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6240117250 con 0x7f6240101af0 2024-07-03T15:54:33.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.878+0000 7f6247e2e640 1 -- 172.21.15.28:0/3710159357 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62400fc8c0 con 0x7f6240101af0 2024-07-03T15:54:33.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.878+0000 7f62367fc640 1 -- 172.21.15.28:0/3710159357 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f623001c400 con 0x7f6240101af0 2024-07-03T15:54:33.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.879+0000 7f62367fc640 1 --2- 172.21.15.28:0/3710159357 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f622c06e0b0 0x7f622c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:33.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.879+0000 7f62367fc640 1 -- 172.21.15.28:0/3710159357 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f623008d4d0 con 0x7f6240101af0 2024-07-03T15:54:33.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.879+0000 7f6245bca640 1 --2- 172.21.15.28:0/3710159357 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f622c06e0b0 0x7f622c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:33.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.879+0000 7f6245bca640 1 --2- 172.21.15.28:0/3710159357 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f622c06e0b0 0x7f622c070560 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f623c000c00 tx=0x7f623c009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:33.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:33.883+0000 7f62367fc640 1 -- 172.21.15.28:0/3710159357 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f62300567a0 con 0x7f6240101af0 2024-07-03T15:54:34.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.110+0000 7f6247e2e640 1 -- 172.21.15.28:0/3710159357 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7f62401082b0 con 0x7f6240101af0 2024-07-03T15:54:34.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.111+0000 7f62367fc640 1 -- 172.21.15.28:0/3710159357 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v35) v1 ==== 107+0+4276 (secure 0 0 0) 0x7f6230059e60 con 0x7f6240101af0 2024-07-03T15:54:34.114 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:54:34.114 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34376,"name":"cephfs.smithi028.rhpvve","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6827/3127260162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":3127260162},{"type":"v1","addr":"172.21.15.28:6827","nonce":3127260162}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":34408,"name":"cephfs.smithi028.wmbxew","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6829/2908130327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":2908130327},{"type":"v1","addr":"172.21.15.28:6829","nonce":2908130327}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44301,"name":"cephfs.smithi148.xgpuld","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6825/1480077002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1480077002},{"type":"v1","addr":"172.21.15.148:6825","nonce":1480077002}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-03T15:41:40.656298+0000","modified":"2024-07-03T15:51:21.780823+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-07-03T15:54:34.115 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 30 2024-07-03T15:54:34.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.113+0000 7f6247e2e640 1 -- 172.21.15.28:0/3710159357 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f622c06e0b0 msgr2=0x7f622c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:34.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.113+0000 7f6247e2e640 1 --2- 172.21.15.28:0/3710159357 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f622c06e0b0 0x7f622c070560 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f623c000c00 tx=0x7f623c009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:34.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.113+0000 7f6247e2e640 1 -- 172.21.15.28:0/3710159357 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6240101af0 msgr2=0x7f624010e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:34.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.113+0000 7f6247e2e640 1 --2- 172.21.15.28:0/3710159357 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6240101af0 0x7f624010e470 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6230002a50 tx=0x7f6230004600 comp rx=0 tx=0).stop 2024-07-03T15:54:34.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.113+0000 7f6247e2e640 1 -- 172.21.15.28:0/3710159357 shutdown_connections 2024-07-03T15:54:34.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.113+0000 7f6247e2e640 1 --2- 172.21.15.28:0/3710159357 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f622c06e0b0 0x7f622c070560 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:34.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.113+0000 7f6247e2e640 1 --2- 172.21.15.28:0/3710159357 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6240101af0 0x7f624010e470 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:34.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.113+0000 7f6247e2e640 1 --2- 172.21.15.28:0/3710159357 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6240101150 0x7f624010df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:34.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.113+0000 7f6247e2e640 1 -- 172.21.15.28:0/3710159357 >> 172.21.15.28:0/3710159357 conn(0x7f62400f6d00 msgr2=0x7f62400f7c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:34.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.113+0000 7f6247e2e640 1 -- 172.21.15.28:0/3710159357 shutdown_connections 2024-07-03T15:54:34.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.113+0000 7f6247e2e640 1 -- 172.21.15.28:0/3710159357 wait complete. 2024-07-03T15:54:34.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:33 smithi028 ceph-mon[92858]: pgmap v282: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:54:34.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:33 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/2102845527' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-07-03T15:54:34.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:33 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/1082260137' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-07-03T15:54:34.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:33 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/1114529787' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-07-03T15:54:34.127 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 30 2024-07-03T15:54:34.127 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 31 2024-07-03T15:54:34.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.225+0000 7f809055d640 1 -- 172.21.15.28:0/2374283434 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8088102290 msgr2=0x7f8088102660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:34.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.225+0000 7f809055d640 1 --2- 172.21.15.28:0/2374283434 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8088102290 0x7f8088102660 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8078009c50 tx=0x7f8078004d10 comp rx=0 tx=0).stop 2024-07-03T15:54:34.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.225+0000 7f809055d640 1 -- 172.21.15.28:0/2374283434 shutdown_connections 2024-07-03T15:54:34.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.225+0000 7f809055d640 1 --2- 172.21.15.28:0/2374283434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8088102c30 0x7f80880fc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:34.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.225+0000 7f809055d640 1 --2- 172.21.15.28:0/2374283434 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8088102290 0x7f8088102660 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:34.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.225+0000 7f809055d640 1 -- 172.21.15.28:0/2374283434 >> 172.21.15.28:0/2374283434 conn(0x7f80880f7e60 msgr2=0x7f80880fa270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:34.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.225+0000 7f809055d640 1 -- 172.21.15.28:0/2374283434 shutdown_connections 2024-07-03T15:54:34.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.225+0000 7f809055d640 1 -- 172.21.15.28:0/2374283434 wait complete. 2024-07-03T15:54:34.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.226+0000 7f809055d640 1 Processor -- start 2024-07-03T15:54:34.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.226+0000 7f809055d640 1 -- start start 2024-07-03T15:54:34.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.226+0000 7f809055d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8088102290 0x7f8088112d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:34.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.226+0000 7f809055d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8088102c30 0x7f80881132b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:34.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.226+0000 7f809055d640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8088113940 con 0x7f8088102290 2024-07-03T15:54:34.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.226+0000 7f809055d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808810cf00 con 0x7f8088102c30 2024-07-03T15:54:34.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.226+0000 7f808e2f9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8088102290 0x7f8088112d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:34.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.226+0000 7f808e2f9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8088102290 0x7f8088112d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47482/0 (socket says 172.21.15.28:47482) 2024-07-03T15:54:34.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.226+0000 7f808e2f9640 1 -- 172.21.15.28:0/3427959384 learned_addr learned my addr 172.21.15.28:0/3427959384 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:54:34.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.226+0000 7f808e2f9640 1 -- 172.21.15.28:0/3427959384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8088102c30 msgr2=0x7f80881132b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:34.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.226+0000 7f808e2f9640 1 --2- 172.21.15.28:0/3427959384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8088102c30 0x7f80881132b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:34.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.226+0000 7f808e2f9640 1 -- 172.21.15.28:0/3427959384 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80780098b0 con 0x7f8088102290 2024-07-03T15:54:34.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.226+0000 7f808e2f9640 1 --2- 172.21.15.28:0/3427959384 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8088102290 0x7f8088112d70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f807802c1f0 tx=0x7f8078004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:34.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.227+0000 7f8086ffd640 1 -- 172.21.15.28:0/3427959384 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f807803a070 con 0x7f8088102290 2024-07-03T15:54:34.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.227+0000 7f8086ffd640 1 -- 172.21.15.28:0/3427959384 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8078009740 con 0x7f8088102290 2024-07-03T15:54:34.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.227+0000 7f809055d640 1 -- 172.21.15.28:0/3427959384 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f808810d180 con 0x7f8088102290 2024-07-03T15:54:34.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.227+0000 7f8086ffd640 1 -- 172.21.15.28:0/3427959384 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f807803eb40 con 0x7f8088102290 2024-07-03T15:54:34.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.227+0000 7f809055d640 1 -- 172.21.15.28:0/3427959384 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f808810d6a0 con 0x7f8088102290 2024-07-03T15:54:34.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:33 smithi148 ceph-mon[80490]: pgmap v282: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:54:34.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:33 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/2102845527' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-07-03T15:54:34.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:33 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/1082260137' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-07-03T15:54:34.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:33 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/1114529787' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-07-03T15:54:34.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.228+0000 7f809055d640 1 -- 172.21.15.28:0/3427959384 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80880fda00 con 0x7f8088102290 2024-07-03T15:54:34.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.228+0000 7f8086ffd640 1 -- 172.21.15.28:0/3427959384 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8078032070 con 0x7f8088102290 2024-07-03T15:54:34.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.229+0000 7f8086ffd640 1 --2- 172.21.15.28:0/3427959384 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f807006e0b0 0x7f8070070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:34.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.229+0000 7f808daf8640 1 --2- 172.21.15.28:0/3427959384 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f807006e0b0 0x7f8070070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:34.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.229+0000 7f8086ffd640 1 -- 172.21.15.28:0/3427959384 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f80780affa0 con 0x7f8088102290 2024-07-03T15:54:34.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.229+0000 7f808daf8640 1 --2- 172.21.15.28:0/3427959384 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f807006e0b0 0x7f8070070560 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f808810e630 tx=0x7f807c006d20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:34.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.233+0000 7f8086ffd640 1 -- 172.21.15.28:0/3427959384 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f8078079460 con 0x7f8088102290 2024-07-03T15:54:34.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.455+0000 7f809055d640 1 -- 172.21.15.28:0/3427959384 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7f8088064400 con 0x7f8088102290 2024-07-03T15:54:34.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.455+0000 7f8086ffd640 1 -- 172.21.15.28:0/3427959384 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v35) v1 ==== 107+0+4355 (secure 0 0 0) 0x7f807807cb20 con 0x7f8088102290 2024-07-03T15:54:34.459 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:54:34.459 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34408,"name":"cephfs.smithi028.wmbxew","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6829/2908130327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":2908130327},{"type":"v1","addr":"172.21.15.28:6829","nonce":2908130327}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44301,"name":"cephfs.smithi148.xgpuld","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6825/1480077002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1480077002},{"type":"v1","addr":"172.21.15.148:6825","nonce":1480077002}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-03T15:41:40.656298+0000","modified":"2024-07-03T15:51:21.785806+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34376},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34376":{"gid":34376,"name":"cephfs.smithi028.rhpvve","rank":0,"incarnation":31,"state":"up:replay","state_seq":1,"addr":"172.21.15.28:6827/3127260162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":3127260162},{"type":"v1","addr":"172.21.15.28:6827","nonce":3127260162}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-07-03T15:54:34.459 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 31 2024-07-03T15:54:34.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.457+0000 7f809055d640 1 -- 172.21.15.28:0/3427959384 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f807006e0b0 msgr2=0x7f8070070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:34.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.457+0000 7f809055d640 1 --2- 172.21.15.28:0/3427959384 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f807006e0b0 0x7f8070070560 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f808810e630 tx=0x7f807c006d20 comp rx=0 tx=0).stop 2024-07-03T15:54:34.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.457+0000 7f809055d640 1 -- 172.21.15.28:0/3427959384 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8088102290 msgr2=0x7f8088112d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:34.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.457+0000 7f809055d640 1 --2- 172.21.15.28:0/3427959384 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8088102290 0x7f8088112d70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f807802c1f0 tx=0x7f8078004d10 comp rx=0 tx=0).stop 2024-07-03T15:54:34.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.457+0000 7f809055d640 1 -- 172.21.15.28:0/3427959384 shutdown_connections 2024-07-03T15:54:34.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.457+0000 7f809055d640 1 --2- 172.21.15.28:0/3427959384 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f807006e0b0 0x7f8070070560 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:34.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.457+0000 7f809055d640 1 --2- 172.21.15.28:0/3427959384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8088102c30 0x7f80881132b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:34.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.457+0000 7f809055d640 1 --2- 172.21.15.28:0/3427959384 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8088102290 0x7f8088112d70 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:34.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.457+0000 7f809055d640 1 -- 172.21.15.28:0/3427959384 >> 172.21.15.28:0/3427959384 conn(0x7f80880f7e60 msgr2=0x7f80880f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:34.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.457+0000 7f809055d640 1 -- 172.21.15.28:0/3427959384 shutdown_connections 2024-07-03T15:54:34.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.457+0000 7f809055d640 1 -- 172.21.15.28:0/3427959384 wait complete. 2024-07-03T15:54:34.471 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 31 2024-07-03T15:54:34.471 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 32 2024-07-03T15:54:34.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.568+0000 7f169ca21640 1 -- 172.21.15.28:0/1286951551 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1698101af0 msgr2=0x7f16980fb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:34.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.568+0000 7f169ca21640 1 --2- 172.21.15.28:0/1286951551 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1698101af0 0x7f16980fb180 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f168c0048d0 tx=0x7f168c009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:34.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.568+0000 7f169ca21640 1 -- 172.21.15.28:0/1286951551 shutdown_connections 2024-07-03T15:54:34.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.568+0000 7f169ca21640 1 --2- 172.21.15.28:0/1286951551 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1698101af0 0x7f16980fb180 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:34.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.568+0000 7f169ca21640 1 --2- 172.21.15.28:0/1286951551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1698101150 0x7f1698101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:34.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.568+0000 7f169ca21640 1 -- 172.21.15.28:0/1286951551 >> 172.21.15.28:0/1286951551 conn(0x7f16980f6d00 msgr2=0x7f16980f9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:34.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.568+0000 7f169ca21640 1 -- 172.21.15.28:0/1286951551 shutdown_connections 2024-07-03T15:54:34.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.568+0000 7f169ca21640 1 -- 172.21.15.28:0/1286951551 wait complete. 2024-07-03T15:54:34.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.569+0000 7f169ca21640 1 Processor -- start 2024-07-03T15:54:34.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.569+0000 7f169ca21640 1 -- start start 2024-07-03T15:54:34.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.569+0000 7f169ca21640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1698101150 0x7f1698112d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:34.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.569+0000 7f169ca21640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1698101af0 0x7f1698113270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:34.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.569+0000 7f169ca21640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1698113870 con 0x7f1698101150 2024-07-03T15:54:34.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.569+0000 7f169ca21640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169810cec0 con 0x7f1698101af0 2024-07-03T15:54:34.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.569+0000 7f169659c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1698101150 0x7f1698112d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:34.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.569+0000 7f169659c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1698101150 0x7f1698112d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47502/0 (socket says 172.21.15.28:47502) 2024-07-03T15:54:34.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.569+0000 7f169659c640 1 -- 172.21.15.28:0/3788758644 learned_addr learned my addr 172.21.15.28:0/3788758644 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:54:34.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.569+0000 7f169659c640 1 -- 172.21.15.28:0/3788758644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1698101af0 msgr2=0x7f1698113270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:34.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.569+0000 7f169659c640 1 --2- 172.21.15.28:0/3788758644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1698101af0 0x7f1698113270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:34.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.570+0000 7f169659c640 1 -- 172.21.15.28:0/3788758644 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f168c005040 con 0x7f1698101150 2024-07-03T15:54:34.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.570+0000 7f169659c640 1 --2- 172.21.15.28:0/3788758644 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1698101150 0x7f1698112d30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1688002a50 tx=0x7f1688004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:34.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.570+0000 7f1686ffd640 1 -- 172.21.15.28:0/3788758644 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1688007c30 con 0x7f1698101150 2024-07-03T15:54:34.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.570+0000 7f169ca21640 1 -- 172.21.15.28:0/3788758644 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f169810d1a0 con 0x7f1698101150 2024-07-03T15:54:34.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.570+0000 7f1686ffd640 1 -- 172.21.15.28:0/3788758644 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f168800f040 con 0x7f1698101150 2024-07-03T15:54:34.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.570+0000 7f1686ffd640 1 -- 172.21.15.28:0/3788758644 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f168800db50 con 0x7f1698101150 2024-07-03T15:54:34.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.570+0000 7f169ca21640 1 -- 172.21.15.28:0/3788758644 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f169810d6c0 con 0x7f1698101150 2024-07-03T15:54:34.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.571+0000 7f169ca21640 1 -- 172.21.15.28:0/3788758644 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16980fc8c0 con 0x7f1698101150 2024-07-03T15:54:34.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.571+0000 7f1686ffd640 1 -- 172.21.15.28:0/3788758644 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f168801c400 con 0x7f1698101150 2024-07-03T15:54:34.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.572+0000 7f1686ffd640 1 --2- 172.21.15.28:0/3788758644 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f168006e0b0 0x7f1680070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:34.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.572+0000 7f1686ffd640 1 -- 172.21.15.28:0/3788758644 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f168808d4d0 con 0x7f1698101150 2024-07-03T15:54:34.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.572+0000 7f1695d9b640 1 --2- 172.21.15.28:0/3788758644 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f168006e0b0 0x7f1680070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:34.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.572+0000 7f1695d9b640 1 --2- 172.21.15.28:0/3788758644 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f168006e0b0 0x7f1680070560 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f168c0043d0 tx=0x7f168c009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:34.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.576+0000 7f1686ffd640 1 -- 172.21.15.28:0/3788758644 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f16880567a0 con 0x7f1698101150 2024-07-03T15:54:34.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.807+0000 7f169ca21640 1 -- 172.21.15.28:0/3788758644 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7f1698064400 con 0x7f1698101150 2024-07-03T15:54:34.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.807+0000 7f1686ffd640 1 -- 172.21.15.28:0/3788758644 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v35) v1 ==== 107+0+5209 (secure 0 0 0) 0x7f1688059e60 con 0x7f1698101150 2024-07-03T15:54:34.811 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:54:34.811 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34408,"name":"cephfs.smithi028.wmbxew","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6829/2908130327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":2908130327},{"type":"v1","addr":"172.21.15.28:6829","nonce":2908130327}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44301,"name":"cephfs.smithi148.xgpuld","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6825/1480077002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1480077002},{"type":"v1","addr":"172.21.15.148:6825","nonce":1480077002}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":44327,"name":"cephfs.smithi148.szomfl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6827/3922163302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3922163302},{"type":"v1","addr":"172.21.15.148:6827","nonce":3922163302}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-03T15:41:40.656298+0000","modified":"2024-07-03T15:51:28.691379+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34376},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34376":{"gid":34376,"name":"cephfs.smithi028.rhpvve","rank":0,"incarnation":31,"state":"up:reconnect","state_seq":14,"addr":"172.21.15.28:6827/3127260162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":3127260162},{"type":"v1","addr":"172.21.15.28:6827","nonce":3127260162}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-07-03T15:54:34.811 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 32 2024-07-03T15:54:34.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.809+0000 7f169ca21640 1 -- 172.21.15.28:0/3788758644 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f168006e0b0 msgr2=0x7f1680070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:34.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.809+0000 7f169ca21640 1 --2- 172.21.15.28:0/3788758644 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f168006e0b0 0x7f1680070560 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f168c0043d0 tx=0x7f168c009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:34.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.809+0000 7f169ca21640 1 -- 172.21.15.28:0/3788758644 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1698101150 msgr2=0x7f1698112d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:34.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.809+0000 7f169ca21640 1 --2- 172.21.15.28:0/3788758644 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1698101150 0x7f1698112d30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1688002a50 tx=0x7f1688004600 comp rx=0 tx=0).stop 2024-07-03T15:54:34.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.809+0000 7f169ca21640 1 -- 172.21.15.28:0/3788758644 shutdown_connections 2024-07-03T15:54:34.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.809+0000 7f169ca21640 1 --2- 172.21.15.28:0/3788758644 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f168006e0b0 0x7f1680070560 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:34.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.809+0000 7f169ca21640 1 --2- 172.21.15.28:0/3788758644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1698101af0 0x7f1698113270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:34.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.809+0000 7f169ca21640 1 --2- 172.21.15.28:0/3788758644 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1698101150 0x7f1698112d30 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:34.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.809+0000 7f169ca21640 1 -- 172.21.15.28:0/3788758644 >> 172.21.15.28:0/3788758644 conn(0x7f16980f6d00 msgr2=0x7f16980f7c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:34.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.810+0000 7f169ca21640 1 -- 172.21.15.28:0/3788758644 shutdown_connections 2024-07-03T15:54:34.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.810+0000 7f169ca21640 1 -- 172.21.15.28:0/3788758644 wait complete. 2024-07-03T15:54:34.823 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 32 2024-07-03T15:54:34.824 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 33 2024-07-03T15:54:34.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.921+0000 7f537a707640 1 -- 172.21.15.28:0/1577280770 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5374102c80 msgr2=0x7f53740fa970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:34.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.921+0000 7f537a707640 1 --2- 172.21.15.28:0/1577280770 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5374102c80 0x7f53740fa970 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f53680048d0 tx=0x7f5368009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:34.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.921+0000 7f537a707640 1 -- 172.21.15.28:0/1577280770 shutdown_connections 2024-07-03T15:54:34.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.921+0000 7f537a707640 1 --2- 172.21.15.28:0/1577280770 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5374102c80 0x7f53740fa970 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:34.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.921+0000 7f537a707640 1 --2- 172.21.15.28:0/1577280770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53741022e0 0x7f53741026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:34.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.921+0000 7f537a707640 1 -- 172.21.15.28:0/1577280770 >> 172.21.15.28:0/1577280770 conn(0x7f5374071890 msgr2=0x7f5374071c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:34.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.921+0000 7f537a707640 1 -- 172.21.15.28:0/1577280770 shutdown_connections 2024-07-03T15:54:34.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.921+0000 7f537a707640 1 -- 172.21.15.28:0/1577280770 wait complete. 2024-07-03T15:54:34.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.922+0000 7f537a707640 1 Processor -- start 2024-07-03T15:54:34.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.922+0000 7f537a707640 1 -- start start 2024-07-03T15:54:34.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.922+0000 7f537a707640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53741022e0 0x7f5374112e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:34.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.922+0000 7f537a707640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5374102c80 0x7f5374113340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:34.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.922+0000 7f537a707640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5374113940 con 0x7f5374102c80 2024-07-03T15:54:34.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.922+0000 7f537a707640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537410cf90 con 0x7f53741022e0 2024-07-03T15:54:34.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.922+0000 7f53737fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5374102c80 0x7f5374113340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:34.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.922+0000 7f53737fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5374102c80 0x7f5374113340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47510/0 (socket says 172.21.15.28:47510) 2024-07-03T15:54:34.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.922+0000 7f53737fe640 1 -- 172.21.15.28:0/3976767942 learned_addr learned my addr 172.21.15.28:0/3976767942 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:54:34.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.923+0000 7f5373fff640 1 --2- 172.21.15.28:0/3976767942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53741022e0 0x7f5374112e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:34.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.923+0000 7f53737fe640 1 -- 172.21.15.28:0/3976767942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53741022e0 msgr2=0x7f5374112e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:34.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.923+0000 7f53737fe640 1 --2- 172.21.15.28:0/3976767942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53741022e0 0x7f5374112e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:34.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.923+0000 7f53737fe640 1 -- 172.21.15.28:0/3976767942 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5368005040 con 0x7f5374102c80 2024-07-03T15:54:34.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.923+0000 7f5373fff640 1 --2- 172.21.15.28:0/3976767942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53741022e0 0x7f5374112e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-03T15:54:34.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.923+0000 7f53737fe640 1 --2- 172.21.15.28:0/3976767942 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5374102c80 0x7f5374113340 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f536802bda0 tx=0x7f5368009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:34.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.923+0000 7f5370ff9640 1 -- 172.21.15.28:0/3976767942 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53680094c0 con 0x7f5374102c80 2024-07-03T15:54:34.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.923+0000 7f537a707640 1 -- 172.21.15.28:0/3976767942 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f537410d210 con 0x7f5374102c80 2024-07-03T15:54:34.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.923+0000 7f5370ff9640 1 -- 172.21.15.28:0/3976767942 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5368032e50 con 0x7f5374102c80 2024-07-03T15:54:34.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.923+0000 7f5370ff9640 1 -- 172.21.15.28:0/3976767942 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f536803b400 con 0x7f5374102c80 2024-07-03T15:54:34.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.923+0000 7f537a707640 1 -- 172.21.15.28:0/3976767942 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f537410d730 con 0x7f5374102c80 2024-07-03T15:54:34.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.924+0000 7f537a707640 1 -- 172.21.15.28:0/3976767942 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53740655e0 con 0x7f5374102c80 2024-07-03T15:54:34.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.925+0000 7f5370ff9640 1 -- 172.21.15.28:0/3976767942 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f536802f070 con 0x7f5374102c80 2024-07-03T15:54:34.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.925+0000 7f5370ff9640 1 --2- 172.21.15.28:0/3976767942 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f535c06e0b0 0x7f535c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:34.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.925+0000 7f5370ff9640 1 -- 172.21.15.28:0/3976767942 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f53680afff0 con 0x7f5374102c80 2024-07-03T15:54:34.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.925+0000 7f5373fff640 1 --2- 172.21.15.28:0/3976767942 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f535c06e0b0 0x7f535c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:34.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.926+0000 7f5373fff640 1 --2- 172.21.15.28:0/3976767942 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f535c06e0b0 0x7f535c070560 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5364004d80 tx=0x7f536400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:34.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:34.929+0000 7f5370ff9640 1 -- 172.21.15.28:0/3976767942 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f53680792c0 con 0x7f5374102c80 2024-07-03T15:54:35.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:34 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/3710159357' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-07-03T15:54:35.121 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:34 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/3427959384' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-07-03T15:54:35.122 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:34 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/3788758644' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-07-03T15:54:35.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.152+0000 7f537a707640 1 -- 172.21.15.28:0/3976767942 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7f5374109460 con 0x7f5374102c80 2024-07-03T15:54:35.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.153+0000 7f5370ff9640 1 -- 172.21.15.28:0/3976767942 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v35) v1 ==== 107+0+5206 (secure 0 0 0) 0x7f536807c980 con 0x7f5374102c80 2024-07-03T15:54:35.156 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:54:35.156 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34408,"name":"cephfs.smithi028.wmbxew","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6829/2908130327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":2908130327},{"type":"v1","addr":"172.21.15.28:6829","nonce":2908130327}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44301,"name":"cephfs.smithi148.xgpuld","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6825/1480077002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1480077002},{"type":"v1","addr":"172.21.15.148:6825","nonce":1480077002}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":44327,"name":"cephfs.smithi148.szomfl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6827/3922163302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3922163302},{"type":"v1","addr":"172.21.15.148:6827","nonce":3922163302}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-03T15:41:40.656298+0000","modified":"2024-07-03T15:51:29.415687+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34376},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34376":{"gid":34376,"name":"cephfs.smithi028.rhpvve","rank":0,"incarnation":31,"state":"up:rejoin","state_seq":15,"addr":"172.21.15.28:6827/3127260162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":3127260162},{"type":"v1","addr":"172.21.15.28:6827","nonce":3127260162}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-07-03T15:54:35.156 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 33 2024-07-03T15:54:35.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.155+0000 7f537a707640 1 -- 172.21.15.28:0/3976767942 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f535c06e0b0 msgr2=0x7f535c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:35.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.155+0000 7f537a707640 1 --2- 172.21.15.28:0/3976767942 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f535c06e0b0 0x7f535c070560 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5364004d80 tx=0x7f536400b040 comp rx=0 tx=0).stop 2024-07-03T15:54:35.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.155+0000 7f537a707640 1 -- 172.21.15.28:0/3976767942 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5374102c80 msgr2=0x7f5374113340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:35.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.155+0000 7f537a707640 1 --2- 172.21.15.28:0/3976767942 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5374102c80 0x7f5374113340 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f536802bda0 tx=0x7f5368009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:35.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.155+0000 7f537a707640 1 -- 172.21.15.28:0/3976767942 shutdown_connections 2024-07-03T15:54:35.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.155+0000 7f537a707640 1 --2- 172.21.15.28:0/3976767942 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f535c06e0b0 0x7f535c070560 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:35.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.155+0000 7f537a707640 1 --2- 172.21.15.28:0/3976767942 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5374102c80 0x7f5374113340 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:35.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.155+0000 7f537a707640 1 --2- 172.21.15.28:0/3976767942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53741022e0 0x7f5374112e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:35.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.155+0000 7f537a707640 1 -- 172.21.15.28:0/3976767942 >> 172.21.15.28:0/3976767942 conn(0x7f5374071890 msgr2=0x7f53740fa2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:35.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.155+0000 7f537a707640 1 -- 172.21.15.28:0/3976767942 shutdown_connections 2024-07-03T15:54:35.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.155+0000 7f537a707640 1 -- 172.21.15.28:0/3976767942 wait complete. 2024-07-03T15:54:35.169 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 33 2024-07-03T15:54:35.169 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 34 2024-07-03T15:54:35.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:34 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/3710159357' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-07-03T15:54:35.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:34 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/3427959384' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-07-03T15:54:35.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:34 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/3788758644' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-07-03T15:54:35.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.266+0000 7f631706e640 1 -- 172.21.15.28:0/3805785049 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6310102c30 msgr2=0x7f63100fc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:35.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.266+0000 7f631706e640 1 --2- 172.21.15.28:0/3805785049 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6310102c30 0x7f63100fc2c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f62f80048d0 tx=0x7f62f8009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:35.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.266+0000 7f631706e640 1 -- 172.21.15.28:0/3805785049 shutdown_connections 2024-07-03T15:54:35.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.266+0000 7f631706e640 1 --2- 172.21.15.28:0/3805785049 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6310102c30 0x7f63100fc2c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:35.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.266+0000 7f631706e640 1 --2- 172.21.15.28:0/3805785049 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6310102290 0x7f6310102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:35.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.266+0000 7f631706e640 1 -- 172.21.15.28:0/3805785049 >> 172.21.15.28:0/3805785049 conn(0x7f63100f7e60 msgr2=0x7f63100fa270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:35.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.266+0000 7f631706e640 1 -- 172.21.15.28:0/3805785049 shutdown_connections 2024-07-03T15:54:35.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.266+0000 7f631706e640 1 -- 172.21.15.28:0/3805785049 wait complete. 2024-07-03T15:54:35.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.266+0000 7f631706e640 1 Processor -- start 2024-07-03T15:54:35.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.267+0000 7f631706e640 1 -- start start 2024-07-03T15:54:35.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.267+0000 7f631706e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6310102290 0x7f631006f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:35.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.267+0000 7f631706e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6310102c30 0x7f631006fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:35.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.267+0000 7f631706e640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6310071700 con 0x7f6310102290 2024-07-03T15:54:35.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.267+0000 7f631706e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63100702a0 con 0x7f6310102c30 2024-07-03T15:54:35.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.267+0000 7f6314e0a640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6310102290 0x7f631006f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:35.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.267+0000 7f6314e0a640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6310102290 0x7f631006f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47532/0 (socket says 172.21.15.28:47532) 2024-07-03T15:54:35.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.267+0000 7f6314e0a640 1 -- 172.21.15.28:0/463837734 learned_addr learned my addr 172.21.15.28:0/463837734 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:54:35.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.267+0000 7f6314e0a640 1 -- 172.21.15.28:0/463837734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6310102c30 msgr2=0x7f631006fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:35.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.267+0000 7f6314e0a640 1 --2- 172.21.15.28:0/463837734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6310102c30 0x7f631006fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:35.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.267+0000 7f6314e0a640 1 -- 172.21.15.28:0/463837734 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62f8005040 con 0x7f6310102290 2024-07-03T15:54:35.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.267+0000 7f6314e0a640 1 --2- 172.21.15.28:0/463837734 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6310102290 0x7f631006f820 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f630000c830 tx=0x7f6300004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:35.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.268+0000 7f630d7fa640 1 -- 172.21.15.28:0/463837734 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f630000a850 con 0x7f6310102290 2024-07-03T15:54:35.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.268+0000 7f630d7fa640 1 -- 172.21.15.28:0/463837734 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f630000ae80 con 0x7f6310102290 2024-07-03T15:54:35.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.268+0000 7f630d7fa640 1 -- 172.21.15.28:0/463837734 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6300012ea0 con 0x7f6310102290 2024-07-03T15:54:35.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.268+0000 7f631706e640 1 -- 172.21.15.28:0/463837734 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6310070580 con 0x7f6310102290 2024-07-03T15:54:35.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.268+0000 7f631706e640 1 -- 172.21.15.28:0/463837734 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6310119410 con 0x7f6310102290 2024-07-03T15:54:35.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.269+0000 7f631706e640 1 -- 172.21.15.28:0/463837734 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63100fda00 con 0x7f6310102290 2024-07-03T15:54:35.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.269+0000 7f630d7fa640 1 -- 172.21.15.28:0/463837734 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6300012400 con 0x7f6310102290 2024-07-03T15:54:35.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.270+0000 7f630d7fa640 1 --2- 172.21.15.28:0/463837734 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f62fc06e0b0 0x7f62fc070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:35.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.270+0000 7f630d7fa640 1 -- 172.21.15.28:0/463837734 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f630008de00 con 0x7f6310102290 2024-07-03T15:54:35.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.270+0000 7f630ffff640 1 --2- 172.21.15.28:0/463837734 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f62fc06e0b0 0x7f62fc070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:35.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.270+0000 7f630ffff640 1 --2- 172.21.15.28:0/463837734 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f62fc06e0b0 0x7f62fc070560 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f62f80043d0 tx=0x7f62f8009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:35.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.274+0000 7f630d7fa640 1 -- 172.21.15.28:0/463837734 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f6300057130 con 0x7f6310102290 2024-07-03T15:54:35.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.496+0000 7f631706e640 1 -- 172.21.15.28:0/463837734 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7f6310109460 con 0x7f6310102290 2024-07-03T15:54:35.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.497+0000 7f630d7fa640 1 -- 172.21.15.28:0/463837734 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v35) v1 ==== 107+0+5215 (secure 0 0 0) 0x7f63000126b0 con 0x7f6310102290 2024-07-03T15:54:35.500 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:54:35.500 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34408,"name":"cephfs.smithi028.wmbxew","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6829/2908130327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":2908130327},{"type":"v1","addr":"172.21.15.28:6829","nonce":2908130327}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44301,"name":"cephfs.smithi148.xgpuld","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6825/1480077002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1480077002},{"type":"v1","addr":"172.21.15.148:6825","nonce":1480077002}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":44327,"name":"cephfs.smithi148.szomfl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6827/3922163302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3922163302},{"type":"v1","addr":"172.21.15.148:6827","nonce":3922163302}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-03T15:41:40.656298+0000","modified":"2024-07-03T15:51:31.417269+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34376},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34376":{"gid":34376,"name":"cephfs.smithi028.rhpvve","rank":0,"incarnation":31,"state":"up:active","state_seq":16,"addr":"172.21.15.28:6827/3127260162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":3127260162},{"type":"v1","addr":"172.21.15.28:6827","nonce":3127260162}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34376,"qdb_cluster":[34376]},"id":1}]} 2024-07-03T15:54:35.501 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 34 2024-07-03T15:54:35.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.499+0000 7f631706e640 1 -- 172.21.15.28:0/463837734 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f62fc06e0b0 msgr2=0x7f62fc070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:35.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.499+0000 7f631706e640 1 --2- 172.21.15.28:0/463837734 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f62fc06e0b0 0x7f62fc070560 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f62f80043d0 tx=0x7f62f8009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:35.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.499+0000 7f631706e640 1 -- 172.21.15.28:0/463837734 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6310102290 msgr2=0x7f631006f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:35.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.499+0000 7f631706e640 1 --2- 172.21.15.28:0/463837734 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6310102290 0x7f631006f820 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f630000c830 tx=0x7f6300004430 comp rx=0 tx=0).stop 2024-07-03T15:54:35.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.499+0000 7f631706e640 1 -- 172.21.15.28:0/463837734 shutdown_connections 2024-07-03T15:54:35.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.499+0000 7f631706e640 1 --2- 172.21.15.28:0/463837734 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f62fc06e0b0 0x7f62fc070560 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:35.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.499+0000 7f631706e640 1 --2- 172.21.15.28:0/463837734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6310102c30 0x7f631006fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:35.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.499+0000 7f631706e640 1 --2- 172.21.15.28:0/463837734 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6310102290 0x7f631006f820 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:35.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.499+0000 7f631706e640 1 -- 172.21.15.28:0/463837734 >> 172.21.15.28:0/463837734 conn(0x7f63100f7e60 msgr2=0x7f63100f9bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:35.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.499+0000 7f631706e640 1 -- 172.21.15.28:0/463837734 shutdown_connections 2024-07-03T15:54:35.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.499+0000 7f631706e640 1 -- 172.21.15.28:0/463837734 wait complete. 2024-07-03T15:54:35.514 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 34 2024-07-03T15:54:35.514 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-07-03T15:54:35.525 INFO:tasks.kclient:Unmounting kernel clients... 2024-07-03T15:54:35.525 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-07-03T15:54:35.525 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-03T15:54:35.544 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-07-03T15:54:35.544 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-03T15:54:35.601 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-03T15:54:35.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.737+0000 7f9ea0418640 1 -- 172.21.15.28:0/1446472165 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9e981011b0 msgr2=0x7f9e98101580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:35.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.737+0000 7f9ea0418640 1 --2- 172.21.15.28:0/1446472165 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9e981011b0 0x7f9e98101580 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9e940048d0 tx=0x7f9e94009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:35.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.738+0000 7f9ea0418640 1 -- 172.21.15.28:0/1446472165 shutdown_connections 2024-07-03T15:54:35.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.738+0000 7f9ea0418640 1 --2- 172.21.15.28:0/1446472165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e98101b50 0x7f9e980fb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:35.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.738+0000 7f9ea0418640 1 --2- 172.21.15.28:0/1446472165 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9e981011b0 0x7f9e98101580 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:35.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.738+0000 7f9ea0418640 1 -- 172.21.15.28:0/1446472165 >> 172.21.15.28:0/1446472165 conn(0x7f9e98071270 msgr2=0x7f9e98071670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:35.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.738+0000 7f9ea0418640 1 -- 172.21.15.28:0/1446472165 shutdown_connections 2024-07-03T15:54:35.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.738+0000 7f9ea0418640 1 -- 172.21.15.28:0/1446472165 wait complete. 2024-07-03T15:54:35.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.738+0000 7f9ea0418640 1 Processor -- start 2024-07-03T15:54:35.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.738+0000 7f9ea0418640 1 -- start start 2024-07-03T15:54:35.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.738+0000 7f9ea0418640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9e981011b0 0x7f9e98112e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:35.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.738+0000 7f9ea0418640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e98101b50 0x7f9e98113350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:35.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.738+0000 7f9ea0418640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e98113950 con 0x7f9e981011b0 2024-07-03T15:54:35.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.738+0000 7f9ea0418640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9810cfa0 con 0x7f9e98101b50 2024-07-03T15:54:35.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.738+0000 7f9e9e1b4640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9e981011b0 0x7f9e98112e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:35.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.738+0000 7f9e9e1b4640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9e981011b0 0x7f9e98112e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47536/0 (socket says 172.21.15.28:47536) 2024-07-03T15:54:35.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.738+0000 7f9e9e1b4640 1 -- 172.21.15.28:0/2129734729 learned_addr learned my addr 172.21.15.28:0/2129734729 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:54:35.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.739+0000 7f9e9e1b4640 1 -- 172.21.15.28:0/2129734729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e98101b50 msgr2=0x7f9e98113350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:35.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.739+0000 7f9e9d9b3640 1 --2- 172.21.15.28:0/2129734729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e98101b50 0x7f9e98113350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:35.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.739+0000 7f9e9e1b4640 1 --2- 172.21.15.28:0/2129734729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e98101b50 0x7f9e98113350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:35.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.739+0000 7f9e9e1b4640 1 -- 172.21.15.28:0/2129734729 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e94005040 con 0x7f9e981011b0 2024-07-03T15:54:35.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.739+0000 7f9e9d9b3640 1 --2- 172.21.15.28:0/2129734729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e98101b50 0x7f9e98113350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-03T15:54:35.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.739+0000 7f9e9e1b4640 1 --2- 172.21.15.28:0/2129734729 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9e981011b0 0x7f9e98112e10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9e94004930 tx=0x7f9e94009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:35.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.739+0000 7f9e8effd640 1 -- 172.21.15.28:0/2129734729 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e940094c0 con 0x7f9e981011b0 2024-07-03T15:54:35.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.739+0000 7f9e8effd640 1 -- 172.21.15.28:0/2129734729 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9e94032e50 con 0x7f9e981011b0 2024-07-03T15:54:35.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.739+0000 7f9e8effd640 1 -- 172.21.15.28:0/2129734729 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e940433b0 con 0x7f9e981011b0 2024-07-03T15:54:35.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.739+0000 7f9ea0418640 1 -- 172.21.15.28:0/2129734729 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e9810d220 con 0x7f9e981011b0 2024-07-03T15:54:35.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.739+0000 7f9ea0418640 1 -- 172.21.15.28:0/2129734729 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e9810d6e0 con 0x7f9e981011b0 2024-07-03T15:54:35.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.740+0000 7f9ea0418640 1 -- 172.21.15.28:0/2129734729 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e980fc930 con 0x7f9e981011b0 2024-07-03T15:54:35.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.741+0000 7f9e8effd640 1 -- 172.21.15.28:0/2129734729 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9e9402f070 con 0x7f9e981011b0 2024-07-03T15:54:35.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.741+0000 7f9e8effd640 1 --2- 172.21.15.28:0/2129734729 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f9e8406e0b0 0x7f9e84070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:35.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.741+0000 7f9e8effd640 1 -- 172.21.15.28:0/2129734729 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f9e940afb80 con 0x7f9e981011b0 2024-07-03T15:54:35.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.741+0000 7f9e9d9b3640 1 --2- 172.21.15.28:0/2129734729 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f9e8406e0b0 0x7f9e84070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:35.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.742+0000 7f9e9d9b3640 1 --2- 172.21.15.28:0/2129734729 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f9e8406e0b0 0x7f9e84070560 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9e9810e6d0 tx=0x7f9e88009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:35.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.746+0000 7f9e8effd640 1 -- 172.21.15.28:0/2129734729 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f9e94078e50 con 0x7f9e981011b0 2024-07-03T15:54:35.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.933+0000 7f9ea0418640 1 -- 172.21.15.28:0/2129734729 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e9810e480 con 0x7f9e981011b0 2024-07-03T15:54:35.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.934+0000 7f9e8effd640 1 -- 172.21.15.28:0/2129734729 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+16768 (secure 0 0 0) 0x7f9e9407c510 con 0x7f9e981011b0 2024-07-03T15:54:35.938 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:54:35.938 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":82,"fsid":"a85e4be6-3951-11ef-bcab-c7b262605968","created":"2024-07-03T15:34:56.026978+0000","modified":"2024-07-03T15:51:21.780577+0000","last_up_change":"2024-07-03T15:50:28.186628+0000","last_in_change":"2024-07-03T15:40:01.150891+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-03T15:39:08.833339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-07-03T15:41:39.583625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-07-03T15:41:40.084538+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7ecd90b5-3715-4ba7-a696-588843e3582c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":48,"up_from":52,"up_thru":77,"down_at":49,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":1756284953},{"type":"v1","addr":"172.21.15.28:6803","nonce":1756284953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":1756284953},{"type":"v1","addr":"172.21.15.28:6805","nonce":1756284953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1756284953},{"type":"v1","addr":"172.21.15.28:6809","nonce":1756284953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":1756284953},{"type":"v1","addr":"172.21.15.28:6807","nonce":1756284953}]},"public_addr":"172.21.15.28:6803/1756284953","cluster_addr":"172.21.15.28:6805/1756284953","heartbeat_back_addr":"172.21.15.28:6809/1756284953","heartbeat_front_addr":"172.21.15.28:6807/1756284953","state":["exists","up"]},{"osd":1,"uuid":"31c5ae0c-6c57-4615-902b-72bf49fa5b72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":53,"up_from":57,"up_thru":77,"down_at":54,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":662081515},{"type":"v1","addr":"172.21.15.28:6811","nonce":662081515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":662081515},{"type":"v1","addr":"172.21.15.28:6813","nonce":662081515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":662081515},{"type":"v1","addr":"172.21.15.28:6817","nonce":662081515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":662081515},{"type":"v1","addr":"172.21.15.28:6815","nonce":662081515}]},"public_addr":"172.21.15.28:6811/662081515","cluster_addr":"172.21.15.28:6813/662081515","heartbeat_back_addr":"172.21.15.28:6817/662081515","heartbeat_front_addr":"172.21.15.28:6815/662081515","state":["exists","up"]},{"osd":2,"uuid":"5ec8caa6-fb62-443f-bbd6-fbe52df5034c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":58,"up_from":62,"up_thru":77,"down_at":59,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2390177581},{"type":"v1","addr":"172.21.15.28:6819","nonce":2390177581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2390177581},{"type":"v1","addr":"172.21.15.28:6821","nonce":2390177581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":2390177581},{"type":"v1","addr":"172.21.15.28:6825","nonce":2390177581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2390177581},{"type":"v1","addr":"172.21.15.28:6823","nonce":2390177581}]},"public_addr":"172.21.15.28:6819/2390177581","cluster_addr":"172.21.15.28:6821/2390177581","heartbeat_back_addr":"172.21.15.28:6825/2390177581","heartbeat_front_addr":"172.21.15.28:6823/2390177581","state":["exists","up"]},{"osd":3,"uuid":"d3117e1c-a230-4fc8-9cfc-37e4dc7e3d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":63,"up_from":67,"up_thru":77,"down_at":64,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":660374222},{"type":"v1","addr":"172.21.15.148:6801","nonce":660374222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":660374222},{"type":"v1","addr":"172.21.15.148:6803","nonce":660374222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":660374222},{"type":"v1","addr":"172.21.15.148:6807","nonce":660374222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":660374222},{"type":"v1","addr":"172.21.15.148:6805","nonce":660374222}]},"public_addr":"172.21.15.148:6801/660374222","cluster_addr":"172.21.15.148:6803/660374222","heartbeat_back_addr":"172.21.15.148:6807/660374222","heartbeat_front_addr":"172.21.15.148:6805/660374222","state":["exists","up"]},{"osd":4,"uuid":"d338fd80-0a76-441e-9cba-9cfd27cccc46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":68,"up_from":72,"up_thru":77,"down_at":69,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3538420843},{"type":"v1","addr":"172.21.15.148:6809","nonce":3538420843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":3538420843},{"type":"v1","addr":"172.21.15.148:6811","nonce":3538420843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":3538420843},{"type":"v1","addr":"172.21.15.148:6815","nonce":3538420843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3538420843},{"type":"v1","addr":"172.21.15.148:6813","nonce":3538420843}]},"public_addr":"172.21.15.148:6809/3538420843","cluster_addr":"172.21.15.148:6811/3538420843","heartbeat_back_addr":"172.21.15.148:6815/3538420843","heartbeat_front_addr":"172.21.15.148:6813/3538420843","state":["exists","up"]},{"osd":5,"uuid":"5c87b168-85e8-4d65-9763-8f1626e474ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":73,"up_from":77,"up_thru":77,"down_at":74,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2178157137},{"type":"v1","addr":"172.21.15.148:6817","nonce":2178157137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2178157137},{"type":"v1","addr":"172.21.15.148:6819","nonce":2178157137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2178157137},{"type":"v1","addr":"172.21.15.148:6823","nonce":2178157137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":2178157137},{"type":"v1","addr":"172.21.15.148:6821","nonce":2178157137}]},"public_addr":"172.21.15.148:6817/2178157137","cluster_addr":"172.21.15.148:6819/2178157137","heartbeat_back_addr":"172.21.15.148:6823/2178157137","heartbeat_front_addr":"172.21.15.148:6821/2178157137","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-03T15:38:11.596686+0000","dead_epoch":48},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-03T15:38:38.765936+0000","dead_epoch":53},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-03T15:39:05.464618+0000","dead_epoch":58},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-03T15:39:28.180164+0000","dead_epoch":63},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-03T15:39:52.836379+0000","dead_epoch":68},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-03T15:40:17.711279+0000","dead_epoch":73}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.148:6826/4171231858":"2024-07-04T15:51:21.780541+0000","172.21.15.28:6800/1642892231":"2024-07-04T15:46:22.043750+0000","172.21.15.28:0/2594481782":"2024-07-04T15:46:22.043750+0000","172.21.15.28:0/1964492377":"2024-07-04T15:46:22.043750+0000","172.21.15.148:6827/4171231858":"2024-07-04T15:51:21.780541+0000","172.21.15.28:0/1079212302":"2024-07-04T15:46:22.043750+0000","172.21.15.148:0/2321782375":"2024-07-04T15:45:31.636243+0000","172.21.15.148:0/2875818850":"2024-07-04T15:45:31.636243+0000","172.21.15.28:0/1083276374":"2024-07-04T15:44:02.457424+0000","172.21.15.28:6801/1358048352":"2024-07-04T15:35:22.603924+0000","172.21.15.28:6800/1358048352":"2024-07-04T15:35:22.603924+0000","172.21.15.28:6800/577887491":"2024-07-04T15:37:00.336312+0000","172.21.15.28:0/2964865891":"2024-07-04T15:37:00.336312+0000","172.21.15.28:6826/1613936042":"2024-07-04T15:50:32.503548+0000","172.21.15.28:0/2733524057":"2024-07-04T15:46:22.043750+0000","172.21.15.28:0/3188588861":"2024-07-04T15:44:02.457424+0000","172.21.15.148:0/2297116482":"2024-07-04T15:45:31.636243+0000","172.21.15.148:0/2453903464":"2024-07-04T15:45:31.636243+0000","172.21.15.28:0/2748059179":"2024-07-04T15:35:22.603924+0000","172.21.15.28:6801/577887491":"2024-07-04T15:37:00.336312+0000","172.21.15.28:6800/421057178":"2024-07-04T15:44:02.457424+0000","172.21.15.28:6801/1434683766":"2024-07-04T15:35:52.741581+0000","172.21.15.28:0/1365299779":"2024-07-04T15:44:02.457424+0000","172.21.15.148:0/301347713":"2024-07-04T15:45:31.636243+0000","172.21.15.28:0/3813763982":"2024-07-04T15:35:52.741581+0000","172.21.15.28:0/1743142286":"2024-07-04T15:46:22.043750+0000","172.21.15.28:0/1009019065":"2024-07-04T15:35:22.603924+0000","172.21.15.28:6801/421057178":"2024-07-04T15:44:02.457424+0000","172.21.15.28:0/423260870":"2024-07-04T15:35:22.603924+0000","172.21.15.148:6828/4285739175":"2024-07-04T15:45:31.636243+0000","172.21.15.28:6827/1613936042":"2024-07-04T15:50:32.503548+0000","172.21.15.28:6801/1642892231":"2024-07-04T15:46:22.043750+0000","172.21.15.28:6800/1434683766":"2024-07-04T15:35:52.741581+0000","172.21.15.148:0/679918823":"2024-07-04T15:45:31.636243+0000","172.21.15.28:0/2750216161":"2024-07-04T15:35:52.741581+0000","172.21.15.148:6829/4285739175":"2024-07-04T15:45:31.636243+0000","172.21.15.28:0/3888547290":"2024-07-04T15:46:22.043750+0000","172.21.15.28:0/3219967940":"2024-07-04T15:37:00.336312+0000","172.21.15.28:0/4294623705":"2024-07-04T15:37:00.336312+0000","172.21.15.28:0/1938941600":"2024-07-04T15:44:02.457424+0000","172.21.15.28:0/390942453":"2024-07-04T15:35:52.741581+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-03T15:54:35.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.936+0000 7f9ea0418640 1 -- 172.21.15.28:0/2129734729 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f9e8406e0b0 msgr2=0x7f9e84070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:35.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.936+0000 7f9ea0418640 1 --2- 172.21.15.28:0/2129734729 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f9e8406e0b0 0x7f9e84070560 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9e9810e6d0 tx=0x7f9e88009040 comp rx=0 tx=0).stop 2024-07-03T15:54:35.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.936+0000 7f9ea0418640 1 -- 172.21.15.28:0/2129734729 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9e981011b0 msgr2=0x7f9e98112e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:35.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.936+0000 7f9ea0418640 1 --2- 172.21.15.28:0/2129734729 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9e981011b0 0x7f9e98112e10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9e94004930 tx=0x7f9e94009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:35.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.936+0000 7f9ea0418640 1 -- 172.21.15.28:0/2129734729 shutdown_connections 2024-07-03T15:54:35.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.936+0000 7f9ea0418640 1 --2- 172.21.15.28:0/2129734729 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f9e8406e0b0 0x7f9e84070560 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:35.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.936+0000 7f9ea0418640 1 --2- 172.21.15.28:0/2129734729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e98101b50 0x7f9e98113350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:35.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.936+0000 7f9ea0418640 1 --2- 172.21.15.28:0/2129734729 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9e981011b0 0x7f9e98112e10 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:35.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.936+0000 7f9ea0418640 1 -- 172.21.15.28:0/2129734729 >> 172.21.15.28:0/2129734729 conn(0x7f9e98071270 msgr2=0x7f9e980faaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:35.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.936+0000 7f9ea0418640 1 -- 172.21.15.28:0/2129734729 shutdown_connections 2024-07-03T15:54:35.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:35.936+0000 7f9ea0418640 1 -- 172.21.15.28:0/2129734729 wait complete. 2024-07-03T15:54:35.951 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-03T15:39:08.833339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-07-03T15:41:39.583625+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-07-03T15:41:40.084538+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-07-03T15:54:35.951 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-07-03T15:54:36.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.052+0000 7faac3421640 1 -- 172.21.15.28:0/2022941879 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faabc0fba00 msgr2=0x7faabc0fbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:36.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.052+0000 7faac3421640 1 --2- 172.21.15.28:0/2022941879 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faabc0fba00 0x7faabc0fbe50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7faaac0048d0 tx=0x7faaac009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:36.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.052+0000 7faac3421640 1 -- 172.21.15.28:0/2022941879 shutdown_connections 2024-07-03T15:54:36.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.052+0000 7faac3421640 1 --2- 172.21.15.28:0/2022941879 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faabc0fba00 0x7faabc0fbe50 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:36.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.052+0000 7faac3421640 1 --2- 172.21.15.28:0/2022941879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faabc1012e0 0x7faabc1016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:36.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.052+0000 7faac3421640 1 -- 172.21.15.28:0/2022941879 >> 172.21.15.28:0/2022941879 conn(0x7faabc0717b0 msgr2=0x7faabc071bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:36.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.052+0000 7faac3421640 1 -- 172.21.15.28:0/2022941879 shutdown_connections 2024-07-03T15:54:36.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.052+0000 7faac3421640 1 -- 172.21.15.28:0/2022941879 wait complete. 2024-07-03T15:54:36.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.052+0000 7faac3421640 1 Processor -- start 2024-07-03T15:54:36.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.052+0000 7faac3421640 1 -- start start 2024-07-03T15:54:36.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.053+0000 7faac3421640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faabc0fba00 0x7faabc112e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:36.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.053+0000 7faac3421640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faabc1012e0 0x7faabc113340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:36.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.053+0000 7faac3421640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faabc113940 con 0x7faabc1012e0 2024-07-03T15:54:36.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.053+0000 7faac3421640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faabc10cf90 con 0x7faabc0fba00 2024-07-03T15:54:36.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.053+0000 7faac09bc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faabc1012e0 0x7faabc113340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:36.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.053+0000 7faac09bc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faabc1012e0 0x7faabc113340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47556/0 (socket says 172.21.15.28:47556) 2024-07-03T15:54:36.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.053+0000 7faac09bc640 1 -- 172.21.15.28:0/2412932557 learned_addr learned my addr 172.21.15.28:0/2412932557 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:54:36.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.053+0000 7faac11bd640 1 --2- 172.21.15.28:0/2412932557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faabc0fba00 0x7faabc112e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:36.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.053+0000 7faac09bc640 1 -- 172.21.15.28:0/2412932557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faabc0fba00 msgr2=0x7faabc112e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:36.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.053+0000 7faac09bc640 1 --2- 172.21.15.28:0/2412932557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faabc0fba00 0x7faabc112e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:36.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.053+0000 7faac09bc640 1 -- 172.21.15.28:0/2412932557 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaac005040 con 0x7faabc1012e0 2024-07-03T15:54:36.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.053+0000 7faac09bc640 1 --2- 172.21.15.28:0/2412932557 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faabc1012e0 0x7faabc113340 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7faaac004930 tx=0x7faaac009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:36.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.053+0000 7faab1ffb640 1 -- 172.21.15.28:0/2412932557 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faaac0094c0 con 0x7faabc1012e0 2024-07-03T15:54:36.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.053+0000 7faab1ffb640 1 -- 172.21.15.28:0/2412932557 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faaac032e50 con 0x7faabc1012e0 2024-07-03T15:54:36.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.053+0000 7faab1ffb640 1 -- 172.21.15.28:0/2412932557 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faaac0433b0 con 0x7faabc1012e0 2024-07-03T15:54:36.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.053+0000 7faac3421640 1 -- 172.21.15.28:0/2412932557 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faabc10d210 con 0x7faabc1012e0 2024-07-03T15:54:36.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.054+0000 7faac3421640 1 -- 172.21.15.28:0/2412932557 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faabc10d680 con 0x7faabc1012e0 2024-07-03T15:54:36.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.055+0000 7faac3421640 1 -- 172.21.15.28:0/2412932557 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faabc0fca10 con 0x7faabc1012e0 2024-07-03T15:54:36.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.055+0000 7faab1ffb640 1 -- 172.21.15.28:0/2412932557 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7faaac02f070 con 0x7faabc1012e0 2024-07-03T15:54:36.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.056+0000 7faab1ffb640 1 --2- 172.21.15.28:0/2412932557 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7faaa806e0b0 0x7faaa8070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:36.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.056+0000 7faab1ffb640 1 -- 172.21.15.28:0/2412932557 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7faaac0afaf0 con 0x7faabc1012e0 2024-07-03T15:54:36.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.056+0000 7faac11bd640 1 --2- 172.21.15.28:0/2412932557 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7faaa806e0b0 0x7faaa8070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:36.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.057+0000 7faac11bd640 1 --2- 172.21.15.28:0/2412932557 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7faaa806e0b0 0x7faaa8070560 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7faab8009fd0 tx=0x7faab8009e50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:36.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.061+0000 7faab1ffb640 1 -- 172.21.15.28:0/2412932557 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7faaac078d20 con 0x7faabc1012e0 2024-07-03T15:54:36.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:35 smithi148 ceph-mon[80490]: pgmap v283: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:54:36.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:35 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/3976767942' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-07-03T15:54:36.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:35 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/463837734' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-07-03T15:54:36.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:35 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/2129734729' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-03T15:54:36.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.249+0000 7faac3421640 1 -- 172.21.15.28:0/2412932557 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faabc1083b0 con 0x7faabc1012e0 2024-07-03T15:54:36.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.250+0000 7faab1ffb640 1 -- 172.21.15.28:0/2412932557 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7faaac07c3e0 con 0x7faabc1012e0 2024-07-03T15:54:36.253 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2024-07-03T15:54:36.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.252+0000 7faac3421640 1 -- 172.21.15.28:0/2412932557 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7faaa806e0b0 msgr2=0x7faaa8070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:36.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.252+0000 7faac3421640 1 --2- 172.21.15.28:0/2412932557 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7faaa806e0b0 0x7faaa8070560 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7faab8009fd0 tx=0x7faab8009e50 comp rx=0 tx=0).stop 2024-07-03T15:54:36.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.252+0000 7faac3421640 1 -- 172.21.15.28:0/2412932557 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faabc1012e0 msgr2=0x7faabc113340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:36.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.252+0000 7faac3421640 1 --2- 172.21.15.28:0/2412932557 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faabc1012e0 0x7faabc113340 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7faaac004930 tx=0x7faaac009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:36.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.252+0000 7faac3421640 1 -- 172.21.15.28:0/2412932557 shutdown_connections 2024-07-03T15:54:36.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.252+0000 7faac3421640 1 --2- 172.21.15.28:0/2412932557 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7faaa806e0b0 0x7faaa8070560 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:36.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.252+0000 7faac3421640 1 --2- 172.21.15.28:0/2412932557 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faabc1012e0 0x7faabc113340 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:36.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.252+0000 7faac3421640 1 --2- 172.21.15.28:0/2412932557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faabc0fba00 0x7faabc112e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:36.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.252+0000 7faac3421640 1 -- 172.21.15.28:0/2412932557 >> 172.21.15.28:0/2412932557 conn(0x7faabc0717b0 msgr2=0x7faabc0fb180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:36.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.252+0000 7faac3421640 1 -- 172.21.15.28:0/2412932557 shutdown_connections 2024-07-03T15:54:36.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.252+0000 7faac3421640 1 -- 172.21.15.28:0/2412932557 wait complete. 2024-07-03T15:54:36.265 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:35 smithi028 ceph-mon[92858]: pgmap v283: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-03T15:54:36.265 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:35 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/3976767942' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-07-03T15:54:36.265 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:35 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/463837734' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-07-03T15:54:36.265 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:35 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/2129734729' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-03T15:54:36.266 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-07-03T15:54:36.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.365+0000 7f6de6b5d640 1 -- 172.21.15.28:0/3127315481 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6de0102bd0 msgr2=0x7f6de00fc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:36.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.365+0000 7f6de6b5d640 1 --2- 172.21.15.28:0/3127315481 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6de0102bd0 0x7f6de00fc260 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0009c50 tx=0x7f6dd0004d10 comp rx=0 tx=0).stop 2024-07-03T15:54:36.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.366+0000 7f6de6b5d640 1 -- 172.21.15.28:0/3127315481 shutdown_connections 2024-07-03T15:54:36.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.366+0000 7f6de6b5d640 1 --2- 172.21.15.28:0/3127315481 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6de0102bd0 0x7f6de00fc260 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:36.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.366+0000 7f6de6b5d640 1 --2- 172.21.15.28:0/3127315481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6de0102230 0x7f6de0102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:36.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.366+0000 7f6de6b5d640 1 -- 172.21.15.28:0/3127315481 >> 172.21.15.28:0/3127315481 conn(0x7f6de00f7e60 msgr2=0x7f6de00fa270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:36.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.366+0000 7f6de6b5d640 1 -- 172.21.15.28:0/3127315481 shutdown_connections 2024-07-03T15:54:36.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.366+0000 7f6de6b5d640 1 -- 172.21.15.28:0/3127315481 wait complete. 2024-07-03T15:54:36.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.366+0000 7f6de6b5d640 1 Processor -- start 2024-07-03T15:54:36.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.366+0000 7f6de6b5d640 1 -- start start 2024-07-03T15:54:36.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.366+0000 7f6de6b5d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6de0102230 0x7f6de0112d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:36.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.366+0000 7f6de6b5d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6de0102bd0 0x7f6de01132c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:36.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.366+0000 7f6de6b5d640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de01138c0 con 0x7f6de0102bd0 2024-07-03T15:54:36.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.366+0000 7f6de6b5d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de010cf10 con 0x7f6de0102230 2024-07-03T15:54:36.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.367+0000 7f6ddffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6de0102bd0 0x7f6de01132c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:36.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.367+0000 7f6ddffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6de0102bd0 0x7f6de01132c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47570/0 (socket says 172.21.15.28:47570) 2024-07-03T15:54:36.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.367+0000 7f6ddffff640 1 -- 172.21.15.28:0/2298810576 learned_addr learned my addr 172.21.15.28:0/2298810576 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:54:36.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.367+0000 7f6de48f9640 1 --2- 172.21.15.28:0/2298810576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6de0102230 0x7f6de0112d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:36.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.367+0000 7f6ddffff640 1 -- 172.21.15.28:0/2298810576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6de0102230 msgr2=0x7f6de0112d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:36.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.367+0000 7f6ddffff640 1 --2- 172.21.15.28:0/2298810576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6de0102230 0x7f6de0112d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:36.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.367+0000 7f6ddffff640 1 -- 172.21.15.28:0/2298810576 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd00098b0 con 0x7f6de0102bd0 2024-07-03T15:54:36.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.367+0000 7f6ddffff640 1 --2- 172.21.15.28:0/2298810576 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6de0102bd0 0x7f6de01132c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6dd002c1f0 tx=0x7f6dd0004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:36.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.367+0000 7f6ddd7fa640 1 -- 172.21.15.28:0/2298810576 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6dd003a070 con 0x7f6de0102bd0 2024-07-03T15:54:36.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.367+0000 7f6ddd7fa640 1 -- 172.21.15.28:0/2298810576 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6dd0004610 con 0x7f6de0102bd0 2024-07-03T15:54:36.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.367+0000 7f6de6b5d640 1 -- 172.21.15.28:0/2298810576 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6de010d190 con 0x7f6de0102bd0 2024-07-03T15:54:36.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.367+0000 7f6ddd7fa640 1 -- 172.21.15.28:0/2298810576 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6dd003ebe0 con 0x7f6de0102bd0 2024-07-03T15:54:36.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.367+0000 7f6de6b5d640 1 -- 172.21.15.28:0/2298810576 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de010d730 con 0x7f6de0102bd0 2024-07-03T15:54:36.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.368+0000 7f6de6b5d640 1 -- 172.21.15.28:0/2298810576 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6de00fd9a0 con 0x7f6de0102bd0 2024-07-03T15:54:36.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.369+0000 7f6ddd7fa640 1 -- 172.21.15.28:0/2298810576 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6dd0032070 con 0x7f6de0102bd0 2024-07-03T15:54:36.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.370+0000 7f6ddd7fa640 1 --2- 172.21.15.28:0/2298810576 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f6dcc06e0b0 0x7f6dcc070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:36.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.370+0000 7f6ddd7fa640 1 -- 172.21.15.28:0/2298810576 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f6dd00b0030 con 0x7f6de0102bd0 2024-07-03T15:54:36.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.370+0000 7f6de48f9640 1 --2- 172.21.15.28:0/2298810576 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f6dcc06e0b0 0x7f6dcc070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:36.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.370+0000 7f6de48f9640 1 --2- 172.21.15.28:0/2298810576 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f6dcc06e0b0 0x7f6dcc070560 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8004eb0 tx=0x7f6dd800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:36.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.375+0000 7f6ddd7fa640 1 -- 172.21.15.28:0/2298810576 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f6dd00793b0 con 0x7f6de0102bd0 2024-07-03T15:54:36.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.558+0000 7f6de6b5d640 1 -- 172.21.15.28:0/2298810576 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f6de0109370 con 0x7f6de0102bd0 2024-07-03T15:54:36.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.558+0000 7f6ddd7fa640 1 -- 172.21.15.28:0/2298810576 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v82) v1 ==== 107+0+11 (secure 0 0 0) 0x7f6dd007ca70 con 0x7f6de0102bd0 2024-07-03T15:54:36.562 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2024-07-03T15:54:36.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.560+0000 7f6de6b5d640 1 -- 172.21.15.28:0/2298810576 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f6dcc06e0b0 msgr2=0x7f6dcc070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:36.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.560+0000 7f6de6b5d640 1 --2- 172.21.15.28:0/2298810576 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f6dcc06e0b0 0x7f6dcc070560 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8004eb0 tx=0x7f6dd800b040 comp rx=0 tx=0).stop 2024-07-03T15:54:36.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.560+0000 7f6de6b5d640 1 -- 172.21.15.28:0/2298810576 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6de0102bd0 msgr2=0x7f6de01132c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:36.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.560+0000 7f6de6b5d640 1 --2- 172.21.15.28:0/2298810576 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6de0102bd0 0x7f6de01132c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6dd002c1f0 tx=0x7f6dd0004d10 comp rx=0 tx=0).stop 2024-07-03T15:54:36.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.560+0000 7f6de6b5d640 1 -- 172.21.15.28:0/2298810576 shutdown_connections 2024-07-03T15:54:36.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.560+0000 7f6de6b5d640 1 --2- 172.21.15.28:0/2298810576 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f6dcc06e0b0 0x7f6dcc070560 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:36.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.560+0000 7f6de6b5d640 1 --2- 172.21.15.28:0/2298810576 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6de0102bd0 0x7f6de01132c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:36.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.560+0000 7f6de6b5d640 1 --2- 172.21.15.28:0/2298810576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6de0102230 0x7f6de0112d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:36.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.560+0000 7f6de6b5d640 1 -- 172.21.15.28:0/2298810576 >> 172.21.15.28:0/2298810576 conn(0x7f6de00f7e60 msgr2=0x7f6de00f8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:36.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.560+0000 7f6de6b5d640 1 -- 172.21.15.28:0/2298810576 shutdown_connections 2024-07-03T15:54:36.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.560+0000 7f6de6b5d640 1 -- 172.21.15.28:0/2298810576 wait complete. 2024-07-03T15:54:36.575 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-07-03T15:54:36.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.674+0000 7fe78a6ae640 1 -- 172.21.15.28:0/1144056570 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe784101ae0 msgr2=0x7fe7840fb170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:36.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.674+0000 7fe78a6ae640 1 --2- 172.21.15.28:0/1144056570 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe784101ae0 0x7fe7840fb170 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe7780048d0 tx=0x7fe778009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:36.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.674+0000 7fe78a6ae640 1 -- 172.21.15.28:0/1144056570 shutdown_connections 2024-07-03T15:54:36.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.674+0000 7fe78a6ae640 1 --2- 172.21.15.28:0/1144056570 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe784101ae0 0x7fe7840fb170 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:36.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.674+0000 7fe78a6ae640 1 --2- 172.21.15.28:0/1144056570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe784101140 0x7fe784101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:36.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.674+0000 7fe78a6ae640 1 -- 172.21.15.28:0/1144056570 >> 172.21.15.28:0/1144056570 conn(0x7fe7840f6cd0 msgr2=0x7fe7840f90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:36.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.674+0000 7fe78a6ae640 1 -- 172.21.15.28:0/1144056570 shutdown_connections 2024-07-03T15:54:36.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.674+0000 7fe78a6ae640 1 -- 172.21.15.28:0/1144056570 wait complete. 2024-07-03T15:54:36.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.674+0000 7fe78a6ae640 1 Processor -- start 2024-07-03T15:54:36.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.675+0000 7fe78a6ae640 1 -- start start 2024-07-03T15:54:36.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.675+0000 7fe78a6ae640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe784101140 0x7fe784112d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:36.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.675+0000 7fe78a6ae640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe784101ae0 0x7fe7841132a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:36.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.675+0000 7fe78a6ae640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7841138a0 con 0x7fe784101140 2024-07-03T15:54:36.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.675+0000 7fe78a6ae640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe784113a10 con 0x7fe784101ae0 2024-07-03T15:54:36.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.675+0000 7fe783fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe784101140 0x7fe784112d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:36.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.675+0000 7fe783fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe784101140 0x7fe784112d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47586/0 (socket says 172.21.15.28:47586) 2024-07-03T15:54:36.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.675+0000 7fe783fff640 1 -- 172.21.15.28:0/3973091190 learned_addr learned my addr 172.21.15.28:0/3973091190 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:54:36.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.675+0000 7fe7837fe640 1 --2- 172.21.15.28:0/3973091190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe784101ae0 0x7fe7841132a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:36.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.675+0000 7fe783fff640 1 -- 172.21.15.28:0/3973091190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe784101ae0 msgr2=0x7fe7841132a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:36.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.675+0000 7fe783fff640 1 --2- 172.21.15.28:0/3973091190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe784101ae0 0x7fe7841132a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:36.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.675+0000 7fe783fff640 1 -- 172.21.15.28:0/3973091190 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe778005040 con 0x7fe784101140 2024-07-03T15:54:36.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.675+0000 7fe783fff640 1 --2- 172.21.15.28:0/3973091190 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe784101140 0x7fe784112d60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe77400c830 tx=0x7fe774004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:36.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.676+0000 7fe780ff9640 1 -- 172.21.15.28:0/3973091190 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe77400a850 con 0x7fe784101140 2024-07-03T15:54:36.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.676+0000 7fe780ff9640 1 -- 172.21.15.28:0/3973091190 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe77400ae80 con 0x7fe784101140 2024-07-03T15:54:36.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.676+0000 7fe78a6ae640 1 -- 172.21.15.28:0/3973091190 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe78410d0a0 con 0x7fe784101140 2024-07-03T15:54:36.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.676+0000 7fe780ff9640 1 -- 172.21.15.28:0/3973091190 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe774012ea0 con 0x7fe784101140 2024-07-03T15:54:36.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.676+0000 7fe78a6ae640 1 -- 172.21.15.28:0/3973091190 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe78410d5c0 con 0x7fe784101140 2024-07-03T15:54:36.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.677+0000 7fe78a6ae640 1 -- 172.21.15.28:0/3973091190 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7840fc8b0 con 0x7fe784101140 2024-07-03T15:54:36.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.677+0000 7fe780ff9640 1 -- 172.21.15.28:0/3973091190 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe774012400 con 0x7fe784101140 2024-07-03T15:54:36.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.678+0000 7fe780ff9640 1 --2- 172.21.15.28:0/3973091190 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fe76c06e0b0 0x7fe76c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:36.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.678+0000 7fe780ff9640 1 -- 172.21.15.28:0/3973091190 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fe77408de40 con 0x7fe784101140 2024-07-03T15:54:36.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.678+0000 7fe7837fe640 1 --2- 172.21.15.28:0/3973091190 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fe76c06e0b0 0x7fe76c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:36.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.678+0000 7fe7837fe640 1 --2- 172.21.15.28:0/3973091190 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fe76c06e0b0 0x7fe76c070560 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe7780047f0 tx=0x7fe778009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:36.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.682+0000 7fe780ff9640 1 -- 172.21.15.28:0/3973091190 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fe774057190 con 0x7fe784101140 2024-07-03T15:54:36.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.870+0000 7fe78a6ae640 1 -- 172.21.15.28:0/3973091190 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fe78404bd70 con 0x7fe784101140 2024-07-03T15:54:36.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.870+0000 7fe780ff9640 1 -- 172.21.15.28:0/3973091190 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v82) v1 ==== 107+0+11 (secure 0 0 0) 0x7fe7740126b0 con 0x7fe784101140 2024-07-03T15:54:36.873 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2024-07-03T15:54:36.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.872+0000 7fe78a6ae640 1 -- 172.21.15.28:0/3973091190 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fe76c06e0b0 msgr2=0x7fe76c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:36.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.872+0000 7fe78a6ae640 1 --2- 172.21.15.28:0/3973091190 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fe76c06e0b0 0x7fe76c070560 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe7780047f0 tx=0x7fe778009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:36.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.872+0000 7fe78a6ae640 1 -- 172.21.15.28:0/3973091190 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe784101140 msgr2=0x7fe784112d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:36.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.872+0000 7fe78a6ae640 1 --2- 172.21.15.28:0/3973091190 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe784101140 0x7fe784112d60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe77400c830 tx=0x7fe774004430 comp rx=0 tx=0).stop 2024-07-03T15:54:36.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.872+0000 7fe78a6ae640 1 -- 172.21.15.28:0/3973091190 shutdown_connections 2024-07-03T15:54:36.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.872+0000 7fe78a6ae640 1 --2- 172.21.15.28:0/3973091190 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fe76c06e0b0 0x7fe76c070560 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:36.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.872+0000 7fe78a6ae640 1 --2- 172.21.15.28:0/3973091190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe784101ae0 0x7fe7841132a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:36.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.872+0000 7fe78a6ae640 1 --2- 172.21.15.28:0/3973091190 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe784101140 0x7fe784112d60 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:36.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.872+0000 7fe78a6ae640 1 -- 172.21.15.28:0/3973091190 >> 172.21.15.28:0/3973091190 conn(0x7fe7840f6cd0 msgr2=0x7fe7840655a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:36.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.873+0000 7fe78a6ae640 1 -- 172.21.15.28:0/3973091190 shutdown_connections 2024-07-03T15:54:36.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.873+0000 7fe78a6ae640 1 -- 172.21.15.28:0/3973091190 wait complete. 2024-07-03T15:54:36.887 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-07-03T15:54:36.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.985+0000 7fc288e2e640 1 -- 172.21.15.28:0/2369760192 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc284101170 msgr2=0x7fc284101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:36.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.985+0000 7fc288e2e640 1 --2- 172.21.15.28:0/2369760192 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc284101170 0x7fc284101540 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc2740048d0 tx=0x7fc274009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:36.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.985+0000 7fc288e2e640 1 -- 172.21.15.28:0/2369760192 shutdown_connections 2024-07-03T15:54:36.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.985+0000 7fc288e2e640 1 --2- 172.21.15.28:0/2369760192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc284101b10 0x7fc2840fb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:36.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.985+0000 7fc288e2e640 1 --2- 172.21.15.28:0/2369760192 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc284101170 0x7fc284101540 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:36.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.985+0000 7fc288e2e640 1 -- 172.21.15.28:0/2369760192 >> 172.21.15.28:0/2369760192 conn(0x7fc2840f6d60 msgr2=0x7fc2840f9170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:36.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.985+0000 7fc288e2e640 1 -- 172.21.15.28:0/2369760192 shutdown_connections 2024-07-03T15:54:36.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.985+0000 7fc288e2e640 1 -- 172.21.15.28:0/2369760192 wait complete. 2024-07-03T15:54:36.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.986+0000 7fc288e2e640 1 Processor -- start 2024-07-03T15:54:36.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.986+0000 7fc288e2e640 1 -- start start 2024-07-03T15:54:36.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.986+0000 7fc288e2e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc284101170 0x7fc284111cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:36.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.986+0000 7fc288e2e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc284101b10 0x7fc2841121f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:36.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.986+0000 7fc288e2e640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2841127f0 con 0x7fc284101b10 2024-07-03T15:54:36.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.986+0000 7fc288e2e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28410be40 con 0x7fc284101170 2024-07-03T15:54:36.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.986+0000 7fc27bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc284101b10 0x7fc2841121f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:36.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.986+0000 7fc27bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc284101b10 0x7fc2841121f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47602/0 (socket says 172.21.15.28:47602) 2024-07-03T15:54:36.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.986+0000 7fc27bfff640 1 -- 172.21.15.28:0/1862924612 learned_addr learned my addr 172.21.15.28:0/1862924612 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:54:36.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.986+0000 7fc27bfff640 1 -- 172.21.15.28:0/1862924612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc284101170 msgr2=0x7fc284111cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:36.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.986+0000 7fc27bfff640 1 --2- 172.21.15.28:0/1862924612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc284101170 0x7fc284111cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:36.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.986+0000 7fc27bfff640 1 -- 172.21.15.28:0/1862924612 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc274005040 con 0x7fc284101b10 2024-07-03T15:54:36.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.987+0000 7fc27bfff640 1 --2- 172.21.15.28:0/1862924612 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc284101b10 0x7fc2841121f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc26c002a50 tx=0x7fc26c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:36.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.987+0000 7fc27affd640 1 -- 172.21.15.28:0/1862924612 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc26c007c30 con 0x7fc284101b10 2024-07-03T15:54:36.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.987+0000 7fc27affd640 1 -- 172.21.15.28:0/1862924612 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc26c00f040 con 0x7fc284101b10 2024-07-03T15:54:36.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.987+0000 7fc27affd640 1 -- 172.21.15.28:0/1862924612 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc26c00db50 con 0x7fc284101b10 2024-07-03T15:54:36.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.987+0000 7fc288e2e640 1 -- 172.21.15.28:0/1862924612 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc28410c120 con 0x7fc284101b10 2024-07-03T15:54:36.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.987+0000 7fc288e2e640 1 -- 172.21.15.28:0/1862924612 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc28410c640 con 0x7fc284101b10 2024-07-03T15:54:36.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.988+0000 7fc288e2e640 1 -- 172.21.15.28:0/1862924612 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2840fc8e0 con 0x7fc284101b10 2024-07-03T15:54:36.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.988+0000 7fc27affd640 1 -- 172.21.15.28:0/1862924612 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc26c00dcf0 con 0x7fc284101b10 2024-07-03T15:54:36.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.989+0000 7fc27affd640 1 --2- 172.21.15.28:0/1862924612 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc27006e0b0 0x7fc270070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:36.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.989+0000 7fc27affd640 1 -- 172.21.15.28:0/1862924612 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fc26c08d430 con 0x7fc284101b10 2024-07-03T15:54:36.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.989+0000 7fc28259c640 1 --2- 172.21.15.28:0/1862924612 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc27006e0b0 0x7fc270070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:36.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.989+0000 7fc28259c640 1 --2- 172.21.15.28:0/1862924612 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc27006e0b0 0x7fc270070560 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc2740043d0 tx=0x7fc274009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:36.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:36.993+0000 7fc27affd640 1 -- 172.21.15.28:0/1862924612 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc26c056780 con 0x7fc284101b10 2024-07-03T15:54:37.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.195+0000 7fc288e2e640 1 -- 172.21.15.28:0/1862924612 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc28404bd70 con 0x7fc284101b10 2024-07-03T15:54:37.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.195+0000 7fc27affd640 1 -- 172.21.15.28:0/1862924612 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 41 entries v82) v1 ==== 81+0+2428 (secure 0 0 0) 0x7fc26c059e40 con 0x7fc284101b10 2024-07-03T15:54:37.198 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.148:6826/4171231858 2024-07-04T15:51:21.780541+0000 2024-07-03T15:54:37.198 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6800/1642892231 2024-07-04T15:46:22.043750+0000 2024-07-03T15:54:37.198 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:0/2594481782 2024-07-04T15:46:22.043750+0000 2024-07-03T15:54:37.199 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:0/1964492377 2024-07-04T15:46:22.043750+0000 2024-07-03T15:54:37.199 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.148:6827/4171231858 2024-07-04T15:51:21.780541+0000 2024-07-03T15:54:37.199 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:0/1079212302 2024-07-04T15:46:22.043750+0000 2024-07-03T15:54:37.199 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.148:0/2321782375 2024-07-04T15:45:31.636243+0000 2024-07-03T15:54:37.199 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.148:0/2875818850 2024-07-04T15:45:31.636243+0000 2024-07-03T15:54:37.199 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:0/1083276374 2024-07-04T15:44:02.457424+0000 2024-07-03T15:54:37.199 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6801/1358048352 2024-07-04T15:35:22.603924+0000 2024-07-03T15:54:37.199 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6800/1358048352 2024-07-04T15:35:22.603924+0000 2024-07-03T15:54:37.199 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6800/577887491 2024-07-04T15:37:00.336312+0000 2024-07-03T15:54:37.199 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:0/2964865891 2024-07-04T15:37:00.336312+0000 2024-07-03T15:54:37.199 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6826/1613936042 2024-07-04T15:50:32.503548+0000 2024-07-03T15:54:37.199 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:0/2733524057 2024-07-04T15:46:22.043750+0000 2024-07-03T15:54:37.199 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:0/3188588861 2024-07-04T15:44:02.457424+0000 2024-07-03T15:54:37.200 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.148:0/2297116482 2024-07-04T15:45:31.636243+0000 2024-07-03T15:54:37.200 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.148:0/2453903464 2024-07-04T15:45:31.636243+0000 2024-07-03T15:54:37.200 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:0/2748059179 2024-07-04T15:35:22.603924+0000 2024-07-03T15:54:37.200 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6801/577887491 2024-07-04T15:37:00.336312+0000 2024-07-03T15:54:37.200 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6800/421057178 2024-07-04T15:44:02.457424+0000 2024-07-03T15:54:37.200 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6801/1434683766 2024-07-04T15:35:52.741581+0000 2024-07-03T15:54:37.200 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:0/1365299779 2024-07-04T15:44:02.457424+0000 2024-07-03T15:54:37.200 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.148:0/301347713 2024-07-04T15:45:31.636243+0000 2024-07-03T15:54:37.200 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:0/3813763982 2024-07-04T15:35:52.741581+0000 2024-07-03T15:54:37.200 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:0/1743142286 2024-07-04T15:46:22.043750+0000 2024-07-03T15:54:37.200 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:0/1009019065 2024-07-04T15:35:22.603924+0000 2024-07-03T15:54:37.200 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6801/421057178 2024-07-04T15:44:02.457424+0000 2024-07-03T15:54:37.200 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:0/423260870 2024-07-04T15:35:22.603924+0000 2024-07-03T15:54:37.201 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.148:6828/4285739175 2024-07-04T15:45:31.636243+0000 2024-07-03T15:54:37.201 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6827/1613936042 2024-07-04T15:50:32.503548+0000 2024-07-03T15:54:37.201 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6801/1642892231 2024-07-04T15:46:22.043750+0000 2024-07-03T15:54:37.201 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6800/1434683766 2024-07-04T15:35:52.741581+0000 2024-07-03T15:54:37.201 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.148:0/679918823 2024-07-04T15:45:31.636243+0000 2024-07-03T15:54:37.201 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:0/2750216161 2024-07-04T15:35:52.741581+0000 2024-07-03T15:54:37.201 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.148:6829/4285739175 2024-07-04T15:45:31.636243+0000 2024-07-03T15:54:37.201 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:0/3888547290 2024-07-04T15:46:22.043750+0000 2024-07-03T15:54:37.201 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:0/3219967940 2024-07-04T15:37:00.336312+0000 2024-07-03T15:54:37.201 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:0/4294623705 2024-07-04T15:37:00.336312+0000 2024-07-03T15:54:37.202 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:0/1938941600 2024-07-04T15:44:02.457424+0000 2024-07-03T15:54:37.202 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:0/390942453 2024-07-04T15:35:52.741581+0000 2024-07-03T15:54:37.202 INFO:teuthology.orchestra.run.smithi028.stderr:listed 41 entries 2024-07-03T15:54:37.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.197+0000 7fc288e2e640 1 -- 172.21.15.28:0/1862924612 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc27006e0b0 msgr2=0x7fc270070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:37.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.197+0000 7fc288e2e640 1 --2- 172.21.15.28:0/1862924612 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc27006e0b0 0x7fc270070560 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc2740043d0 tx=0x7fc274009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:37.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.197+0000 7fc288e2e640 1 -- 172.21.15.28:0/1862924612 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc284101b10 msgr2=0x7fc2841121f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:37.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.197+0000 7fc288e2e640 1 --2- 172.21.15.28:0/1862924612 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc284101b10 0x7fc2841121f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc26c002a50 tx=0x7fc26c004600 comp rx=0 tx=0).stop 2024-07-03T15:54:37.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.198+0000 7fc288e2e640 1 -- 172.21.15.28:0/1862924612 shutdown_connections 2024-07-03T15:54:37.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.198+0000 7fc288e2e640 1 --2- 172.21.15.28:0/1862924612 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fc27006e0b0 0x7fc270070560 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:37.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.198+0000 7fc288e2e640 1 --2- 172.21.15.28:0/1862924612 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc284101b10 0x7fc2841121f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:37.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.198+0000 7fc288e2e640 1 --2- 172.21.15.28:0/1862924612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc284101170 0x7fc284111cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:37.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.198+0000 7fc288e2e640 1 -- 172.21.15.28:0/1862924612 >> 172.21.15.28:0/1862924612 conn(0x7fc2840f6d60 msgr2=0x7fc2840f7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:37.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.198+0000 7fc288e2e640 1 -- 172.21.15.28:0/1862924612 shutdown_connections 2024-07-03T15:54:37.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.198+0000 7fc288e2e640 1 -- 172.21.15.28:0/1862924612 wait complete. 2024-07-03T15:54:37.215 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:36 smithi028 ceph-mon[92858]: pgmap v284: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:54:37.215 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:36 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/2412932557' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-03T15:54:37.215 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:36 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/2298810576' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-07-03T15:54:37.215 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:36 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/3973091190' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-07-03T15:54:37.216 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-07-03T15:54:37.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-03T15:54:37.216 DEBUG:teuthology.orchestra.run.smithi028:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-07-03T15:54:37.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:36 smithi148 ceph-mon[80490]: pgmap v284: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-07-03T15:54:37.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:36 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/2412932557' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-03T15:54:37.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:36 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/2298810576' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-07-03T15:54:37.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:36 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/3973091190' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-07-03T15:54:37.243 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-07-03T15:54:37.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-03T15:54:37.243 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-07-03T15:54:37.315 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-07-03T15:54:37.316 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-03T15:54:37.335 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-07-03T15:54:37.335 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-03T15:54:37.393 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-03T15:54:37.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.490+0000 7ff726b5d640 1 -- 172.21.15.28:0/2963288177 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff720101b00 msgr2=0x7ff7200fb190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:37.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.490+0000 7ff726b5d640 1 --2- 172.21.15.28:0/2963288177 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff720101b00 0x7ff7200fb190 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff70c0048d0 tx=0x7ff70c009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:37.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.490+0000 7ff726b5d640 1 -- 172.21.15.28:0/2963288177 shutdown_connections 2024-07-03T15:54:37.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.490+0000 7ff726b5d640 1 --2- 172.21.15.28:0/2963288177 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff720101b00 0x7ff7200fb190 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:37.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.490+0000 7ff726b5d640 1 --2- 172.21.15.28:0/2963288177 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff720101160 0x7ff720101530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:37.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.490+0000 7ff726b5d640 1 -- 172.21.15.28:0/2963288177 >> 172.21.15.28:0/2963288177 conn(0x7ff7200f6d50 msgr2=0x7ff7200f9160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:37.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.490+0000 7ff726b5d640 1 -- 172.21.15.28:0/2963288177 shutdown_connections 2024-07-03T15:54:37.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.490+0000 7ff726b5d640 1 -- 172.21.15.28:0/2963288177 wait complete. 2024-07-03T15:54:37.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.491+0000 7ff726b5d640 1 Processor -- start 2024-07-03T15:54:37.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.491+0000 7ff726b5d640 1 -- start start 2024-07-03T15:54:37.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.491+0000 7ff726b5d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff720101160 0x7ff72010df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:37.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.491+0000 7ff726b5d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff720101b00 0x7ff72010e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:37.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.491+0000 7ff726b5d640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff720066520 con 0x7ff720101160 2024-07-03T15:54:37.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.491+0000 7ff726b5d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72010e9f0 con 0x7ff720101b00 2024-07-03T15:54:37.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.491+0000 7ff7248f9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff720101160 0x7ff72010df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:37.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.491+0000 7ff7248f9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff720101160 0x7ff72010df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47624/0 (socket says 172.21.15.28:47624) 2024-07-03T15:54:37.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.491+0000 7ff7248f9640 1 -- 172.21.15.28:0/3161099552 learned_addr learned my addr 172.21.15.28:0/3161099552 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:54:37.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.491+0000 7ff7248f9640 1 -- 172.21.15.28:0/3161099552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff720101b00 msgr2=0x7ff72010e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:37.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.491+0000 7ff7248f9640 1 --2- 172.21.15.28:0/3161099552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff720101b00 0x7ff72010e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:37.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.491+0000 7ff7248f9640 1 -- 172.21.15.28:0/3161099552 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff70c005040 con 0x7ff720101160 2024-07-03T15:54:37.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.492+0000 7ff7248f9640 1 --2- 172.21.15.28:0/3161099552 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff720101160 0x7ff72010df70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff714002a50 tx=0x7ff714004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:37.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.492+0000 7ff71d7fa640 1 -- 172.21.15.28:0/3161099552 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff714007c30 con 0x7ff720101160 2024-07-03T15:54:37.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.492+0000 7ff71d7fa640 1 -- 172.21.15.28:0/3161099552 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff71400f040 con 0x7ff720101160 2024-07-03T15:54:37.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.492+0000 7ff71d7fa640 1 -- 172.21.15.28:0/3161099552 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff71400db50 con 0x7ff720101160 2024-07-03T15:54:37.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.492+0000 7ff726b5d640 1 -- 172.21.15.28:0/3161099552 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff72010ecd0 con 0x7ff720101160 2024-07-03T15:54:37.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.492+0000 7ff726b5d640 1 -- 172.21.15.28:0/3161099552 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff720117290 con 0x7ff720101160 2024-07-03T15:54:37.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.493+0000 7ff726b5d640 1 -- 172.21.15.28:0/3161099552 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7200fc8d0 con 0x7ff720101160 2024-07-03T15:54:37.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.493+0000 7ff71d7fa640 1 -- 172.21.15.28:0/3161099552 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff71400dcf0 con 0x7ff720101160 2024-07-03T15:54:37.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.494+0000 7ff71d7fa640 1 --2- 172.21.15.28:0/3161099552 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7ff71006e0b0 0x7ff710070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:37.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.494+0000 7ff71d7fa640 1 -- 172.21.15.28:0/3161099552 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7ff71408d420 con 0x7ff720101160 2024-07-03T15:54:37.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.494+0000 7ff71ffff640 1 --2- 172.21.15.28:0/3161099552 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7ff71006e0b0 0x7ff710070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:37.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.494+0000 7ff71ffff640 1 --2- 172.21.15.28:0/3161099552 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7ff71006e0b0 0x7ff710070560 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff70c003080 tx=0x7ff70c009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:37.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.498+0000 7ff71d7fa640 1 -- 172.21.15.28:0/3161099552 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff714056770 con 0x7ff720101160 2024-07-03T15:54:37.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.682+0000 7ff726b5d640 1 -- 172.21.15.28:0/3161099552 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7201174d0 con 0x7ff720101160 2024-07-03T15:54:37.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.683+0000 7ff71d7fa640 1 -- 172.21.15.28:0/3161099552 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+16768 (secure 0 0 0) 0x7ff714059e30 con 0x7ff720101160 2024-07-03T15:54:37.687 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-07-03T15:54:37.687 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":82,"fsid":"a85e4be6-3951-11ef-bcab-c7b262605968","created":"2024-07-03T15:34:56.026978+0000","modified":"2024-07-03T15:51:21.780577+0000","last_up_change":"2024-07-03T15:50:28.186628+0000","last_in_change":"2024-07-03T15:40:01.150891+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-03T15:39:08.833339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-07-03T15:41:39.583625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-07-03T15:41:40.084538+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7ecd90b5-3715-4ba7-a696-588843e3582c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":48,"up_from":52,"up_thru":77,"down_at":49,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":1756284953},{"type":"v1","addr":"172.21.15.28:6803","nonce":1756284953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":1756284953},{"type":"v1","addr":"172.21.15.28:6805","nonce":1756284953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1756284953},{"type":"v1","addr":"172.21.15.28:6809","nonce":1756284953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":1756284953},{"type":"v1","addr":"172.21.15.28:6807","nonce":1756284953}]},"public_addr":"172.21.15.28:6803/1756284953","cluster_addr":"172.21.15.28:6805/1756284953","heartbeat_back_addr":"172.21.15.28:6809/1756284953","heartbeat_front_addr":"172.21.15.28:6807/1756284953","state":["exists","up"]},{"osd":1,"uuid":"31c5ae0c-6c57-4615-902b-72bf49fa5b72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":53,"up_from":57,"up_thru":77,"down_at":54,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":662081515},{"type":"v1","addr":"172.21.15.28:6811","nonce":662081515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":662081515},{"type":"v1","addr":"172.21.15.28:6813","nonce":662081515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":662081515},{"type":"v1","addr":"172.21.15.28:6817","nonce":662081515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":662081515},{"type":"v1","addr":"172.21.15.28:6815","nonce":662081515}]},"public_addr":"172.21.15.28:6811/662081515","cluster_addr":"172.21.15.28:6813/662081515","heartbeat_back_addr":"172.21.15.28:6817/662081515","heartbeat_front_addr":"172.21.15.28:6815/662081515","state":["exists","up"]},{"osd":2,"uuid":"5ec8caa6-fb62-443f-bbd6-fbe52df5034c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":58,"up_from":62,"up_thru":77,"down_at":59,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2390177581},{"type":"v1","addr":"172.21.15.28:6819","nonce":2390177581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2390177581},{"type":"v1","addr":"172.21.15.28:6821","nonce":2390177581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":2390177581},{"type":"v1","addr":"172.21.15.28:6825","nonce":2390177581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2390177581},{"type":"v1","addr":"172.21.15.28:6823","nonce":2390177581}]},"public_addr":"172.21.15.28:6819/2390177581","cluster_addr":"172.21.15.28:6821/2390177581","heartbeat_back_addr":"172.21.15.28:6825/2390177581","heartbeat_front_addr":"172.21.15.28:6823/2390177581","state":["exists","up"]},{"osd":3,"uuid":"d3117e1c-a230-4fc8-9cfc-37e4dc7e3d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":63,"up_from":67,"up_thru":77,"down_at":64,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":660374222},{"type":"v1","addr":"172.21.15.148:6801","nonce":660374222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":660374222},{"type":"v1","addr":"172.21.15.148:6803","nonce":660374222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":660374222},{"type":"v1","addr":"172.21.15.148:6807","nonce":660374222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":660374222},{"type":"v1","addr":"172.21.15.148:6805","nonce":660374222}]},"public_addr":"172.21.15.148:6801/660374222","cluster_addr":"172.21.15.148:6803/660374222","heartbeat_back_addr":"172.21.15.148:6807/660374222","heartbeat_front_addr":"172.21.15.148:6805/660374222","state":["exists","up"]},{"osd":4,"uuid":"d338fd80-0a76-441e-9cba-9cfd27cccc46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":68,"up_from":72,"up_thru":77,"down_at":69,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3538420843},{"type":"v1","addr":"172.21.15.148:6809","nonce":3538420843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":3538420843},{"type":"v1","addr":"172.21.15.148:6811","nonce":3538420843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":3538420843},{"type":"v1","addr":"172.21.15.148:6815","nonce":3538420843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3538420843},{"type":"v1","addr":"172.21.15.148:6813","nonce":3538420843}]},"public_addr":"172.21.15.148:6809/3538420843","cluster_addr":"172.21.15.148:6811/3538420843","heartbeat_back_addr":"172.21.15.148:6815/3538420843","heartbeat_front_addr":"172.21.15.148:6813/3538420843","state":["exists","up"]},{"osd":5,"uuid":"5c87b168-85e8-4d65-9763-8f1626e474ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":73,"up_from":77,"up_thru":77,"down_at":74,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2178157137},{"type":"v1","addr":"172.21.15.148:6817","nonce":2178157137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2178157137},{"type":"v1","addr":"172.21.15.148:6819","nonce":2178157137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2178157137},{"type":"v1","addr":"172.21.15.148:6823","nonce":2178157137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":2178157137},{"type":"v1","addr":"172.21.15.148:6821","nonce":2178157137}]},"public_addr":"172.21.15.148:6817/2178157137","cluster_addr":"172.21.15.148:6819/2178157137","heartbeat_back_addr":"172.21.15.148:6823/2178157137","heartbeat_front_addr":"172.21.15.148:6821/2178157137","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-03T15:38:11.596686+0000","dead_epoch":48},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-03T15:38:38.765936+0000","dead_epoch":53},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-03T15:39:05.464618+0000","dead_epoch":58},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-03T15:39:28.180164+0000","dead_epoch":63},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-03T15:39:52.836379+0000","dead_epoch":68},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-03T15:40:17.711279+0000","dead_epoch":73}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.148:6826/4171231858":"2024-07-04T15:51:21.780541+0000","172.21.15.28:6800/1642892231":"2024-07-04T15:46:22.043750+0000","172.21.15.28:0/2594481782":"2024-07-04T15:46:22.043750+0000","172.21.15.28:0/1964492377":"2024-07-04T15:46:22.043750+0000","172.21.15.148:6827/4171231858":"2024-07-04T15:51:21.780541+0000","172.21.15.28:0/1079212302":"2024-07-04T15:46:22.043750+0000","172.21.15.148:0/2321782375":"2024-07-04T15:45:31.636243+0000","172.21.15.148:0/2875818850":"2024-07-04T15:45:31.636243+0000","172.21.15.28:0/1083276374":"2024-07-04T15:44:02.457424+0000","172.21.15.28:6801/1358048352":"2024-07-04T15:35:22.603924+0000","172.21.15.28:6800/1358048352":"2024-07-04T15:35:22.603924+0000","172.21.15.28:6800/577887491":"2024-07-04T15:37:00.336312+0000","172.21.15.28:0/2964865891":"2024-07-04T15:37:00.336312+0000","172.21.15.28:6826/1613936042":"2024-07-04T15:50:32.503548+0000","172.21.15.28:0/2733524057":"2024-07-04T15:46:22.043750+0000","172.21.15.28:0/3188588861":"2024-07-04T15:44:02.457424+0000","172.21.15.148:0/2297116482":"2024-07-04T15:45:31.636243+0000","172.21.15.148:0/2453903464":"2024-07-04T15:45:31.636243+0000","172.21.15.28:0/2748059179":"2024-07-04T15:35:22.603924+0000","172.21.15.28:6801/577887491":"2024-07-04T15:37:00.336312+0000","172.21.15.28:6800/421057178":"2024-07-04T15:44:02.457424+0000","172.21.15.28:6801/1434683766":"2024-07-04T15:35:52.741581+0000","172.21.15.28:0/1365299779":"2024-07-04T15:44:02.457424+0000","172.21.15.148:0/301347713":"2024-07-04T15:45:31.636243+0000","172.21.15.28:0/3813763982":"2024-07-04T15:35:52.741581+0000","172.21.15.28:0/1743142286":"2024-07-04T15:46:22.043750+0000","172.21.15.28:0/1009019065":"2024-07-04T15:35:22.603924+0000","172.21.15.28:6801/421057178":"2024-07-04T15:44:02.457424+0000","172.21.15.28:0/423260870":"2024-07-04T15:35:22.603924+0000","172.21.15.148:6828/4285739175":"2024-07-04T15:45:31.636243+0000","172.21.15.28:6827/1613936042":"2024-07-04T15:50:32.503548+0000","172.21.15.28:6801/1642892231":"2024-07-04T15:46:22.043750+0000","172.21.15.28:6800/1434683766":"2024-07-04T15:35:52.741581+0000","172.21.15.148:0/679918823":"2024-07-04T15:45:31.636243+0000","172.21.15.28:0/2750216161":"2024-07-04T15:35:52.741581+0000","172.21.15.148:6829/4285739175":"2024-07-04T15:45:31.636243+0000","172.21.15.28:0/3888547290":"2024-07-04T15:46:22.043750+0000","172.21.15.28:0/3219967940":"2024-07-04T15:37:00.336312+0000","172.21.15.28:0/4294623705":"2024-07-04T15:37:00.336312+0000","172.21.15.28:0/1938941600":"2024-07-04T15:44:02.457424+0000","172.21.15.28:0/390942453":"2024-07-04T15:35:52.741581+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-03T15:54:37.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.686+0000 7ff726b5d640 1 -- 172.21.15.28:0/3161099552 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7ff71006e0b0 msgr2=0x7ff710070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:37.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.686+0000 7ff726b5d640 1 --2- 172.21.15.28:0/3161099552 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7ff71006e0b0 0x7ff710070560 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff70c003080 tx=0x7ff70c009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:37.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.686+0000 7ff726b5d640 1 -- 172.21.15.28:0/3161099552 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff720101160 msgr2=0x7ff72010df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:37.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.686+0000 7ff726b5d640 1 --2- 172.21.15.28:0/3161099552 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff720101160 0x7ff72010df70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff714002a50 tx=0x7ff714004600 comp rx=0 tx=0).stop 2024-07-03T15:54:37.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.686+0000 7ff726b5d640 1 -- 172.21.15.28:0/3161099552 shutdown_connections 2024-07-03T15:54:37.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.686+0000 7ff726b5d640 1 --2- 172.21.15.28:0/3161099552 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7ff71006e0b0 0x7ff710070560 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:37.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.686+0000 7ff726b5d640 1 --2- 172.21.15.28:0/3161099552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff720101b00 0x7ff72010e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:37.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.686+0000 7ff726b5d640 1 --2- 172.21.15.28:0/3161099552 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff720101160 0x7ff72010df70 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:37.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.686+0000 7ff726b5d640 1 -- 172.21.15.28:0/3161099552 >> 172.21.15.28:0/3161099552 conn(0x7ff7200f6d50 msgr2=0x7ff7200f7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:37.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.686+0000 7ff726b5d640 1 -- 172.21.15.28:0/3161099552 shutdown_connections 2024-07-03T15:54:37.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.686+0000 7ff726b5d640 1 -- 172.21.15.28:0/3161099552 wait complete. 2024-07-03T15:54:37.700 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-03T15:39:08.833339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-07-03T15:41:39.583625+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-07-03T15:41:40.084538+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-07-03T15:54:37.701 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-07-03T15:54:37.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.799+0000 7fddf7f9d640 1 -- 172.21.15.28:0/2965268910 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fddf00fd080 msgr2=0x7fddf00fd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:37.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.799+0000 7fddf7f9d640 1 --2- 172.21.15.28:0/2965268910 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fddf00fd080 0x7fddf00fd450 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fddec0048d0 tx=0x7fddec009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:37.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.799+0000 7fddf7f9d640 1 -- 172.21.15.28:0/2965268910 shutdown_connections 2024-07-03T15:54:37.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.799+0000 7fddf7f9d640 1 --2- 172.21.15.28:0/2965268910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fddf00fa8f0 0x7fddf00fad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:37.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.799+0000 7fddf7f9d640 1 --2- 172.21.15.28:0/2965268910 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fddf00fd080 0x7fddf00fd450 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:37.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.799+0000 7fddf7f9d640 1 -- 172.21.15.28:0/2965268910 >> 172.21.15.28:0/2965268910 conn(0x7fddf00717b0 msgr2=0x7fddf0071bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:37.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.800+0000 7fddf7f9d640 1 -- 172.21.15.28:0/2965268910 shutdown_connections 2024-07-03T15:54:37.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.800+0000 7fddf7f9d640 1 -- 172.21.15.28:0/2965268910 wait complete. 2024-07-03T15:54:37.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.800+0000 7fddf7f9d640 1 Processor -- start 2024-07-03T15:54:37.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.800+0000 7fddf7f9d640 1 -- start start 2024-07-03T15:54:37.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.800+0000 7fddf7f9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fddf00fa8f0 0x7fddf0114f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:37.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.800+0000 7fddf7f9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fddf00fd080 0x7fddf0115490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:37.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.800+0000 7fddf7f9d640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf0115a90 con 0x7fddf00fa8f0 2024-07-03T15:54:37.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.800+0000 7fddf7f9d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf010f0e0 con 0x7fddf00fd080 2024-07-03T15:54:37.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.800+0000 7fddf5d39640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fddf00fa8f0 0x7fddf0114f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:37.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.800+0000 7fddf5d39640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fddf00fa8f0 0x7fddf0114f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47638/0 (socket says 172.21.15.28:47638) 2024-07-03T15:54:37.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.800+0000 7fddf5d39640 1 -- 172.21.15.28:0/3964743870 learned_addr learned my addr 172.21.15.28:0/3964743870 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:54:37.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.801+0000 7fddf5538640 1 --2- 172.21.15.28:0/3964743870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fddf00fd080 0x7fddf0115490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:37.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.801+0000 7fddf5d39640 1 -- 172.21.15.28:0/3964743870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fddf00fd080 msgr2=0x7fddf0115490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:37.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.801+0000 7fddf5d39640 1 --2- 172.21.15.28:0/3964743870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fddf00fd080 0x7fddf0115490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:37.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.801+0000 7fddf5d39640 1 -- 172.21.15.28:0/3964743870 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddec005040 con 0x7fddf00fa8f0 2024-07-03T15:54:37.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.801+0000 7fddf5d39640 1 --2- 172.21.15.28:0/3964743870 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fddf00fa8f0 0x7fddf0114f50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fddec004930 tx=0x7fddec009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:37.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.801+0000 7fdde67fc640 1 -- 172.21.15.28:0/3964743870 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fddec0094c0 con 0x7fddf00fa8f0 2024-07-03T15:54:37.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.801+0000 7fdde67fc640 1 -- 172.21.15.28:0/3964743870 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fddec032e50 con 0x7fddf00fa8f0 2024-07-03T15:54:37.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.801+0000 7fddf7f9d640 1 -- 172.21.15.28:0/3964743870 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddf010f360 con 0x7fddf00fa8f0 2024-07-03T15:54:37.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.801+0000 7fdde67fc640 1 -- 172.21.15.28:0/3964743870 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fddec0433b0 con 0x7fddf00fa8f0 2024-07-03T15:54:37.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.801+0000 7fddf7f9d640 1 -- 172.21.15.28:0/3964743870 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddf010f820 con 0x7fddf00fa8f0 2024-07-03T15:54:37.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.802+0000 7fddf7f9d640 1 -- 172.21.15.28:0/3964743870 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddf00fb900 con 0x7fddf00fa8f0 2024-07-03T15:54:37.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.803+0000 7fdde67fc640 1 -- 172.21.15.28:0/3964743870 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fddec02f070 con 0x7fddf00fa8f0 2024-07-03T15:54:37.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.803+0000 7fdde67fc640 1 --2- 172.21.15.28:0/3964743870 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fdddc06e0b0 0x7fdddc070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:37.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.803+0000 7fdde67fc640 1 -- 172.21.15.28:0/3964743870 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fddec0afaa0 con 0x7fddf00fa8f0 2024-07-03T15:54:37.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.803+0000 7fddf5538640 1 --2- 172.21.15.28:0/3964743870 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fdddc06e0b0 0x7fdddc070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:37.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.804+0000 7fddf5538640 1 --2- 172.21.15.28:0/3964743870 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fdddc06e0b0 0x7fdddc070560 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdde0007fd0 tx=0x7fdde0009e90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:37.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.807+0000 7fdde67fc640 1 -- 172.21.15.28:0/3964743870 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fddec078d70 con 0x7fddf00fa8f0 2024-07-03T15:54:37.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.992+0000 7fddf7f9d640 1 -- 172.21.15.28:0/3964743870 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fddf004bd70 con 0x7fddf00fa8f0 2024-07-03T15:54:37.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.993+0000 7fdde67fc640 1 -- 172.21.15.28:0/3964743870 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7fddec07c430 con 0x7fddf00fa8f0 2024-07-03T15:54:37.996 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2024-07-03T15:54:37.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.995+0000 7fddf7f9d640 1 -- 172.21.15.28:0/3964743870 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fdddc06e0b0 msgr2=0x7fdddc070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:37.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.995+0000 7fddf7f9d640 1 --2- 172.21.15.28:0/3964743870 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fdddc06e0b0 0x7fdddc070560 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdde0007fd0 tx=0x7fdde0009e90 comp rx=0 tx=0).stop 2024-07-03T15:54:37.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.995+0000 7fddf7f9d640 1 -- 172.21.15.28:0/3964743870 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fddf00fa8f0 msgr2=0x7fddf0114f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:37.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.995+0000 7fddf7f9d640 1 --2- 172.21.15.28:0/3964743870 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fddf00fa8f0 0x7fddf0114f50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fddec004930 tx=0x7fddec009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:37.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.995+0000 7fddf7f9d640 1 -- 172.21.15.28:0/3964743870 shutdown_connections 2024-07-03T15:54:37.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.995+0000 7fddf7f9d640 1 --2- 172.21.15.28:0/3964743870 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fdddc06e0b0 0x7fdddc070560 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:37.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.995+0000 7fddf7f9d640 1 --2- 172.21.15.28:0/3964743870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fddf00fd080 0x7fddf0115490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:37.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.995+0000 7fddf7f9d640 1 --2- 172.21.15.28:0/3964743870 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fddf00fa8f0 0x7fddf0114f50 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:37.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.995+0000 7fddf7f9d640 1 -- 172.21.15.28:0/3964743870 >> 172.21.15.28:0/3964743870 conn(0x7fddf00717b0 msgr2=0x7fddf00f88e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:37.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.995+0000 7fddf7f9d640 1 -- 172.21.15.28:0/3964743870 shutdown_connections 2024-07-03T15:54:37.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:37.995+0000 7fddf7f9d640 1 -- 172.21.15.28:0/3964743870 wait complete. 2024-07-03T15:54:38.009 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-07-03T15:54:38.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.109+0000 7f538719d640 1 -- 172.21.15.28:0/2659324586 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53800fa8f0 msgr2=0x7f53800fad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:38.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.109+0000 7f538719d640 1 --2- 172.21.15.28:0/2659324586 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53800fa8f0 0x7f53800fad60 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f53740048d0 tx=0x7f5374009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:38.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.109+0000 7f538719d640 1 -- 172.21.15.28:0/2659324586 shutdown_connections 2024-07-03T15:54:38.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.109+0000 7f538719d640 1 --2- 172.21.15.28:0/2659324586 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53800fa8f0 0x7f53800fad60 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:38.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.109+0000 7f538719d640 1 --2- 172.21.15.28:0/2659324586 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53801022f0 0x7f53801026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:38.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.109+0000 7f538719d640 1 -- 172.21.15.28:0/2659324586 >> 172.21.15.28:0/2659324586 conn(0x7f53800717b0 msgr2=0x7f5380071bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:38.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.109+0000 7f538719d640 1 -- 172.21.15.28:0/2659324586 shutdown_connections 2024-07-03T15:54:38.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.109+0000 7f538719d640 1 -- 172.21.15.28:0/2659324586 wait complete. 2024-07-03T15:54:38.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.109+0000 7f538719d640 1 Processor -- start 2024-07-03T15:54:38.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.109+0000 7f538719d640 1 -- start start 2024-07-03T15:54:38.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.110+0000 7f538719d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53800fa8f0 0x7f5380112d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:38.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.110+0000 7f538719d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53801022f0 0x7f53801132d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:38.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.110+0000 7f538719d640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53801138d0 con 0x7f53800fa8f0 2024-07-03T15:54:38.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.110+0000 7f538719d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538010cf20 con 0x7f53801022f0 2024-07-03T15:54:38.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.110+0000 7f5384f39640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53800fa8f0 0x7f5380112d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:38.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.110+0000 7f5384f39640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53800fa8f0 0x7f5380112d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47660/0 (socket says 172.21.15.28:47660) 2024-07-03T15:54:38.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.110+0000 7f5384f39640 1 -- 172.21.15.28:0/2297056165 learned_addr learned my addr 172.21.15.28:0/2297056165 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:54:38.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.110+0000 7f5384f39640 1 -- 172.21.15.28:0/2297056165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53801022f0 msgr2=0x7f53801132d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:38.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.110+0000 7f537ffff640 1 --2- 172.21.15.28:0/2297056165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53801022f0 0x7f53801132d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:38.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.110+0000 7f5384f39640 1 --2- 172.21.15.28:0/2297056165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53801022f0 0x7f53801132d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:38.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.110+0000 7f5384f39640 1 -- 172.21.15.28:0/2297056165 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5374005040 con 0x7f53800fa8f0 2024-07-03T15:54:38.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.110+0000 7f5384f39640 1 --2- 172.21.15.28:0/2297056165 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53800fa8f0 0x7f5380112d90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5370002840 tx=0x7f5370004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:38.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.110+0000 7f537d7fa640 1 -- 172.21.15.28:0/2297056165 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5370007c30 con 0x7f53800fa8f0 2024-07-03T15:54:38.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.110+0000 7f537d7fa640 1 -- 172.21.15.28:0/2297056165 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f537000f040 con 0x7f53800fa8f0 2024-07-03T15:54:38.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.110+0000 7f538719d640 1 -- 172.21.15.28:0/2297056165 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f538010d200 con 0x7f53800fa8f0 2024-07-03T15:54:38.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.111+0000 7f537d7fa640 1 -- 172.21.15.28:0/2297056165 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5370009570 con 0x7f53800fa8f0 2024-07-03T15:54:38.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.111+0000 7f538719d640 1 -- 172.21.15.28:0/2297056165 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f538010d720 con 0x7f53800fa8f0 2024-07-03T15:54:38.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.112+0000 7f538719d640 1 -- 172.21.15.28:0/2297056165 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5380065360 con 0x7f53800fa8f0 2024-07-03T15:54:38.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.112+0000 7f537d7fa640 1 -- 172.21.15.28:0/2297056165 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f537000dc80 con 0x7f53800fa8f0 2024-07-03T15:54:38.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.112+0000 7f537d7fa640 1 --2- 172.21.15.28:0/2297056165 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f536806e0b0 0x7f5368070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:38.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.113+0000 7f537d7fa640 1 -- 172.21.15.28:0/2297056165 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f537008db80 con 0x7f53800fa8f0 2024-07-03T15:54:38.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.113+0000 7f537ffff640 1 --2- 172.21.15.28:0/2297056165 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f536806e0b0 0x7f5368070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:38.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.113+0000 7f537ffff640 1 --2- 172.21.15.28:0/2297056165 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f536806e0b0 0x7f5368070560 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5374000c00 tx=0x7f5374009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:38.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.117+0000 7f537d7fa640 1 -- 172.21.15.28:0/2297056165 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f537005a980 con 0x7f53800fa8f0 2024-07-03T15:54:38.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:37 smithi148 ceph-mon[80490]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:54:38.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:37 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/1862924612' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-07-03T15:54:38.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:37 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/3161099552' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-03T15:54:38.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.300+0000 7f538719d640 1 -- 172.21.15.28:0/2297056165 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f538010e400 con 0x7f53800fa8f0 2024-07-03T15:54:38.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.300+0000 7f537d7fa640 1 -- 172.21.15.28:0/2297056165 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v82) v1 ==== 107+0+11 (secure 0 0 0) 0x7f537005a510 con 0x7f53800fa8f0 2024-07-03T15:54:38.303 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2024-07-03T15:54:38.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.302+0000 7f538719d640 1 -- 172.21.15.28:0/2297056165 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f536806e0b0 msgr2=0x7f5368070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:38.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.302+0000 7f538719d640 1 --2- 172.21.15.28:0/2297056165 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f536806e0b0 0x7f5368070560 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5374000c00 tx=0x7f5374009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:38.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.302+0000 7f538719d640 1 -- 172.21.15.28:0/2297056165 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53800fa8f0 msgr2=0x7f5380112d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:38.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.302+0000 7f538719d640 1 --2- 172.21.15.28:0/2297056165 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53800fa8f0 0x7f5380112d90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5370002840 tx=0x7f5370004570 comp rx=0 tx=0).stop 2024-07-03T15:54:38.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.303+0000 7f538719d640 1 -- 172.21.15.28:0/2297056165 shutdown_connections 2024-07-03T15:54:38.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.303+0000 7f538719d640 1 --2- 172.21.15.28:0/2297056165 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f536806e0b0 0x7f5368070560 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:38.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.303+0000 7f538719d640 1 --2- 172.21.15.28:0/2297056165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53801022f0 0x7f53801132d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:38.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.303+0000 7f538719d640 1 --2- 172.21.15.28:0/2297056165 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53800fa8f0 0x7f5380112d90 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:38.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.303+0000 7f538719d640 1 -- 172.21.15.28:0/2297056165 >> 172.21.15.28:0/2297056165 conn(0x7f53800717b0 msgr2=0x7f53800f9a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:38.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.303+0000 7f538719d640 1 -- 172.21.15.28:0/2297056165 shutdown_connections 2024-07-03T15:54:38.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.303+0000 7f538719d640 1 -- 172.21.15.28:0/2297056165 wait complete. 2024-07-03T15:54:38.315 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:37 smithi028 ceph-mon[92858]: from='mgr.34104 172.21.15.28:0/991258935' entity='mgr.smithi028.vtqrtz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-03T15:54:38.316 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:37 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/1862924612' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-07-03T15:54:38.316 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:37 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/3161099552' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-03T15:54:38.317 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-07-03T15:54:38.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.415+0000 7fb5f9c6e640 1 -- 172.21.15.28:0/3372515174 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5f40fb190 msgr2=0x7fb5f40fb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:38.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.415+0000 7fb5f9c6e640 1 --2- 172.21.15.28:0/3372515174 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5f40fb190 0x7fb5f40fb5e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb5e80048d0 tx=0x7fb5e8009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:38.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.416+0000 7fb5f9c6e640 1 -- 172.21.15.28:0/3372515174 shutdown_connections 2024-07-03T15:54:38.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.416+0000 7fb5f9c6e640 1 --2- 172.21.15.28:0/3372515174 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5f40fb190 0x7fb5f40fb5e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:38.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.416+0000 7fb5f9c6e640 1 --2- 172.21.15.28:0/3372515174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5f41011c0 0x7fb5f4101590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:38.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.416+0000 7fb5f9c6e640 1 -- 172.21.15.28:0/3372515174 >> 172.21.15.28:0/3372515174 conn(0x7fb5f4071170 msgr2=0x7fb5f4071570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:38.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.416+0000 7fb5f9c6e640 1 -- 172.21.15.28:0/3372515174 shutdown_connections 2024-07-03T15:54:38.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.416+0000 7fb5f9c6e640 1 -- 172.21.15.28:0/3372515174 wait complete. 2024-07-03T15:54:38.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.416+0000 7fb5f9c6e640 1 Processor -- start 2024-07-03T15:54:38.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.416+0000 7fb5f9c6e640 1 -- start start 2024-07-03T15:54:38.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.416+0000 7fb5f9c6e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5f40fb190 0x7fb5f4111c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:38.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.417+0000 7fb5f9c6e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5f41011c0 0x7fb5f41121a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:38.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.417+0000 7fb5f9c6e640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f41127a0 con 0x7fb5f40fb190 2024-07-03T15:54:38.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.417+0000 7fb5f9c6e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f410bdf0 con 0x7fb5f41011c0 2024-07-03T15:54:38.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.417+0000 7fb5f37fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5f40fb190 0x7fb5f4111c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:38.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.417+0000 7fb5f37fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5f40fb190 0x7fb5f4111c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47676/0 (socket says 172.21.15.28:47676) 2024-07-03T15:54:38.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.417+0000 7fb5f37fe640 1 -- 172.21.15.28:0/2318066950 learned_addr learned my addr 172.21.15.28:0/2318066950 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:54:38.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.417+0000 7fb5f2ffd640 1 --2- 172.21.15.28:0/2318066950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5f41011c0 0x7fb5f41121a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:38.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.417+0000 7fb5f37fe640 1 -- 172.21.15.28:0/2318066950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5f41011c0 msgr2=0x7fb5f41121a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:38.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.417+0000 7fb5f37fe640 1 --2- 172.21.15.28:0/2318066950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5f41011c0 0x7fb5f41121a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:38.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.417+0000 7fb5f37fe640 1 -- 172.21.15.28:0/2318066950 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e8005040 con 0x7fb5f40fb190 2024-07-03T15:54:38.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.417+0000 7fb5f37fe640 1 --2- 172.21.15.28:0/2318066950 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5f40fb190 0x7fb5f4111c60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb5e400c810 tx=0x7fb5e40043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:38.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.417+0000 7fb5dbfff640 1 -- 172.21.15.28:0/2318066950 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5e400a820 con 0x7fb5f40fb190 2024-07-03T15:54:38.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.417+0000 7fb5f9c6e640 1 -- 172.21.15.28:0/2318066950 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5f410c0d0 con 0x7fb5f40fb190 2024-07-03T15:54:38.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.417+0000 7fb5dbfff640 1 -- 172.21.15.28:0/2318066950 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb5e400ae50 con 0x7fb5f40fb190 2024-07-03T15:54:38.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.417+0000 7fb5dbfff640 1 -- 172.21.15.28:0/2318066950 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5e40115c0 con 0x7fb5f40fb190 2024-07-03T15:54:38.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.417+0000 7fb5f9c6e640 1 -- 172.21.15.28:0/2318066950 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5f410c5c0 con 0x7fb5f40fb190 2024-07-03T15:54:38.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.418+0000 7fb5f9c6e640 1 -- 172.21.15.28:0/2318066950 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5f40fc880 con 0x7fb5f40fb190 2024-07-03T15:54:38.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.419+0000 7fb5dbfff640 1 -- 172.21.15.28:0/2318066950 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb5e400a980 con 0x7fb5f40fb190 2024-07-03T15:54:38.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.419+0000 7fb5dbfff640 1 --2- 172.21.15.28:0/2318066950 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fb5dc06e0b0 0x7fb5dc070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:38.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.420+0000 7fb5f2ffd640 1 --2- 172.21.15.28:0/2318066950 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fb5dc06e0b0 0x7fb5dc070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:38.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.420+0000 7fb5dbfff640 1 -- 172.21.15.28:0/2318066950 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fb5e408e110 con 0x7fb5f40fb190 2024-07-03T15:54:38.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.420+0000 7fb5f2ffd640 1 --2- 172.21.15.28:0/2318066950 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fb5dc06e0b0 0x7fb5dc070560 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb5f40f9430 tx=0x7fb5e8009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:38.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.424+0000 7fb5dbfff640 1 -- 172.21.15.28:0/2318066950 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fb5e40573e0 con 0x7fb5f40fb190 2024-07-03T15:54:38.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.607+0000 7fb5f9c6e640 1 -- 172.21.15.28:0/2318066950 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fb5f41082b0 con 0x7fb5f40fb190 2024-07-03T15:54:38.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.608+0000 7fb5dbfff640 1 -- 172.21.15.28:0/2318066950 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v82) v1 ==== 107+0+11 (secure 0 0 0) 0x7fb5e405aaa0 con 0x7fb5f40fb190 2024-07-03T15:54:38.611 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2024-07-03T15:54:38.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.610+0000 7fb5f9c6e640 1 -- 172.21.15.28:0/2318066950 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fb5dc06e0b0 msgr2=0x7fb5dc070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:38.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.610+0000 7fb5f9c6e640 1 --2- 172.21.15.28:0/2318066950 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fb5dc06e0b0 0x7fb5dc070560 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb5f40f9430 tx=0x7fb5e8009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:38.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.610+0000 7fb5f9c6e640 1 -- 172.21.15.28:0/2318066950 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5f40fb190 msgr2=0x7fb5f4111c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:38.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.610+0000 7fb5f9c6e640 1 --2- 172.21.15.28:0/2318066950 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5f40fb190 0x7fb5f4111c60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb5e400c810 tx=0x7fb5e40043c0 comp rx=0 tx=0).stop 2024-07-03T15:54:38.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.610+0000 7fb5f9c6e640 1 -- 172.21.15.28:0/2318066950 shutdown_connections 2024-07-03T15:54:38.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.610+0000 7fb5f9c6e640 1 --2- 172.21.15.28:0/2318066950 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7fb5dc06e0b0 0x7fb5dc070560 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:38.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.610+0000 7fb5f9c6e640 1 --2- 172.21.15.28:0/2318066950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5f41011c0 0x7fb5f41121a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:38.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.610+0000 7fb5f9c6e640 1 --2- 172.21.15.28:0/2318066950 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5f40fb190 0x7fb5f4111c60 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:38.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.610+0000 7fb5f9c6e640 1 -- 172.21.15.28:0/2318066950 >> 172.21.15.28:0/2318066950 conn(0x7fb5f4071170 msgr2=0x7fb5f40fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:38.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.610+0000 7fb5f9c6e640 1 -- 172.21.15.28:0/2318066950 shutdown_connections 2024-07-03T15:54:38.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.610+0000 7fb5f9c6e640 1 -- 172.21.15.28:0/2318066950 wait complete. 2024-07-03T15:54:38.625 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-07-03T15:54:38.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.722+0000 7f495ab9d640 1 -- 172.21.15.28:0/3555798310 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4954102c10 msgr2=0x7f49540fa960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:38.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.722+0000 7f495ab9d640 1 --2- 172.21.15.28:0/3555798310 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4954102c10 0x7f49540fa960 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f49400048d0 tx=0x7f4940009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:38.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.722+0000 7f495ab9d640 1 -- 172.21.15.28:0/3555798310 shutdown_connections 2024-07-03T15:54:38.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.722+0000 7f495ab9d640 1 --2- 172.21.15.28:0/3555798310 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4954102c10 0x7f49540fa960 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:38.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.722+0000 7f495ab9d640 1 --2- 172.21.15.28:0/3555798310 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4954102270 0x7f4954102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:38.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.722+0000 7f495ab9d640 1 -- 172.21.15.28:0/3555798310 >> 172.21.15.28:0/3555798310 conn(0x7f4954070940 msgr2=0x7f4954070d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:38.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.722+0000 7f495ab9d640 1 -- 172.21.15.28:0/3555798310 shutdown_connections 2024-07-03T15:54:38.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.722+0000 7f495ab9d640 1 -- 172.21.15.28:0/3555798310 wait complete. 2024-07-03T15:54:38.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.723+0000 7f495ab9d640 1 Processor -- start 2024-07-03T15:54:38.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.723+0000 7f495ab9d640 1 -- start start 2024-07-03T15:54:38.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.723+0000 7f495ab9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4954102270 0x7f49540fbb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:38.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.723+0000 7f495ab9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4954102c10 0x7f49540fc080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:38.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.723+0000 7f495ab9d640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49540fda20 con 0x7f4954102270 2024-07-03T15:54:38.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.723+0000 7f495ab9d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49540fc5c0 con 0x7f4954102c10 2024-07-03T15:54:38.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.723+0000 7f4958939640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4954102270 0x7f49540fbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:38.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.723+0000 7f4958939640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4954102270 0x7f49540fbb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47698/0 (socket says 172.21.15.28:47698) 2024-07-03T15:54:38.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.723+0000 7f4958939640 1 -- 172.21.15.28:0/1429544112 learned_addr learned my addr 172.21.15.28:0/1429544112 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-07-03T15:54:38.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.723+0000 7f4953fff640 1 --2- 172.21.15.28:0/1429544112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4954102c10 0x7f49540fc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:38.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.724+0000 7f4958939640 1 -- 172.21.15.28:0/1429544112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4954102c10 msgr2=0x7f49540fc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:38.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.724+0000 7f4958939640 1 --2- 172.21.15.28:0/1429544112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4954102c10 0x7f49540fc080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:38.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.724+0000 7f4958939640 1 -- 172.21.15.28:0/1429544112 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4940005040 con 0x7f4954102270 2024-07-03T15:54:38.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.724+0000 7f4953fff640 1 --2- 172.21.15.28:0/1429544112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4954102c10 0x7f49540fc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-03T15:54:38.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.724+0000 7f4958939640 1 --2- 172.21.15.28:0/1429544112 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4954102270 0x7f49540fbb40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4948002a50 tx=0x7f4948004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:38.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.724+0000 7f49517fa640 1 -- 172.21.15.28:0/1429544112 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4948007c30 con 0x7f4954102270 2024-07-03T15:54:38.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.724+0000 7f49517fa640 1 -- 172.21.15.28:0/1429544112 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f494800f040 con 0x7f4954102270 2024-07-03T15:54:38.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.724+0000 7f495ab9d640 1 -- 172.21.15.28:0/1429544112 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49540fc8a0 con 0x7f4954102270 2024-07-03T15:54:38.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.724+0000 7f49517fa640 1 -- 172.21.15.28:0/1429544112 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f494800db50 con 0x7f4954102270 2024-07-03T15:54:38.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.724+0000 7f495ab9d640 1 -- 172.21.15.28:0/1429544112 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4954119510 con 0x7f4954102270 2024-07-03T15:54:38.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.725+0000 7f495ab9d640 1 -- 172.21.15.28:0/1429544112 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4954065350 con 0x7f4954102270 2024-07-03T15:54:38.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.725+0000 7f49517fa640 1 -- 172.21.15.28:0/1429544112 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 37) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f494800dcf0 con 0x7f4954102270 2024-07-03T15:54:38.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.726+0000 7f49517fa640 1 --2- 172.21.15.28:0/1429544112 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f494406e0b0 0x7f4944070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-03T15:54:38.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.726+0000 7f49517fa640 1 -- 172.21.15.28:0/1429544112 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f494808d410 con 0x7f4954102270 2024-07-03T15:54:38.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.726+0000 7f4953fff640 1 --2- 172.21.15.28:0/1429544112 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f494406e0b0 0x7f4944070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-03T15:54:38.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.726+0000 7f4953fff640 1 --2- 172.21.15.28:0/1429544112 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f494406e0b0 0x7f4944070560 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f49400048a0 tx=0x7f4940009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-03T15:54:38.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.730+0000 7f49517fa640 1 -- 172.21.15.28:0/1429544112 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f4948056760 con 0x7f4954102270 2024-07-03T15:54:38.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.929+0000 7f495ab9d640 1 -- 172.21.15.28:0/1429544112 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f495404bd70 con 0x7f4954102270 2024-07-03T15:54:38.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.930+0000 7f49517fa640 1 -- 172.21.15.28:0/1429544112 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 41 entries v82) v1 ==== 81+0+2428 (secure 0 0 0) 0x7f4948059e20 con 0x7f4954102270 2024-07-03T15:54:38.933 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.148:6826/4171231858 2024-07-04T15:51:21.780541+0000 2024-07-03T15:54:38.933 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6800/1642892231 2024-07-04T15:46:22.043750+0000 2024-07-03T15:54:38.934 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:0/2594481782 2024-07-04T15:46:22.043750+0000 2024-07-03T15:54:38.934 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:0/1964492377 2024-07-04T15:46:22.043750+0000 2024-07-03T15:54:38.934 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.148:6827/4171231858 2024-07-04T15:51:21.780541+0000 2024-07-03T15:54:38.934 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:0/1079212302 2024-07-04T15:46:22.043750+0000 2024-07-03T15:54:38.934 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.148:0/2321782375 2024-07-04T15:45:31.636243+0000 2024-07-03T15:54:38.934 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.148:0/2875818850 2024-07-04T15:45:31.636243+0000 2024-07-03T15:54:38.934 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:0/1083276374 2024-07-04T15:44:02.457424+0000 2024-07-03T15:54:38.934 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6801/1358048352 2024-07-04T15:35:22.603924+0000 2024-07-03T15:54:38.934 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6800/1358048352 2024-07-04T15:35:22.603924+0000 2024-07-03T15:54:38.934 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6800/577887491 2024-07-04T15:37:00.336312+0000 2024-07-03T15:54:38.934 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:0/2964865891 2024-07-04T15:37:00.336312+0000 2024-07-03T15:54:38.935 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6826/1613936042 2024-07-04T15:50:32.503548+0000 2024-07-03T15:54:38.935 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:0/2733524057 2024-07-04T15:46:22.043750+0000 2024-07-03T15:54:38.935 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:0/3188588861 2024-07-04T15:44:02.457424+0000 2024-07-03T15:54:38.935 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.148:0/2297116482 2024-07-04T15:45:31.636243+0000 2024-07-03T15:54:38.935 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.148:0/2453903464 2024-07-04T15:45:31.636243+0000 2024-07-03T15:54:38.935 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:0/2748059179 2024-07-04T15:35:22.603924+0000 2024-07-03T15:54:38.935 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6801/577887491 2024-07-04T15:37:00.336312+0000 2024-07-03T15:54:38.935 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6800/421057178 2024-07-04T15:44:02.457424+0000 2024-07-03T15:54:38.935 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6801/1434683766 2024-07-04T15:35:52.741581+0000 2024-07-03T15:54:38.935 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:0/1365299779 2024-07-04T15:44:02.457424+0000 2024-07-03T15:54:38.935 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.148:0/301347713 2024-07-04T15:45:31.636243+0000 2024-07-03T15:54:38.935 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:0/3813763982 2024-07-04T15:35:52.741581+0000 2024-07-03T15:54:38.935 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:0/1743142286 2024-07-04T15:46:22.043750+0000 2024-07-03T15:54:38.936 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:0/1009019065 2024-07-04T15:35:22.603924+0000 2024-07-03T15:54:38.936 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6801/421057178 2024-07-04T15:44:02.457424+0000 2024-07-03T15:54:38.936 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:0/423260870 2024-07-04T15:35:22.603924+0000 2024-07-03T15:54:38.936 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.148:6828/4285739175 2024-07-04T15:45:31.636243+0000 2024-07-03T15:54:38.936 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6827/1613936042 2024-07-04T15:50:32.503548+0000 2024-07-03T15:54:38.936 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6801/1642892231 2024-07-04T15:46:22.043750+0000 2024-07-03T15:54:38.936 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6800/1434683766 2024-07-04T15:35:52.741581+0000 2024-07-03T15:54:38.936 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.148:0/679918823 2024-07-04T15:45:31.636243+0000 2024-07-03T15:54:38.936 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:0/2750216161 2024-07-04T15:35:52.741581+0000 2024-07-03T15:54:38.936 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.148:6829/4285739175 2024-07-04T15:45:31.636243+0000 2024-07-03T15:54:38.936 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:0/3888547290 2024-07-04T15:46:22.043750+0000 2024-07-03T15:54:38.936 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:0/3219967940 2024-07-04T15:37:00.336312+0000 2024-07-03T15:54:38.936 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:0/4294623705 2024-07-04T15:37:00.336312+0000 2024-07-03T15:54:38.937 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:0/1938941600 2024-07-04T15:44:02.457424+0000 2024-07-03T15:54:38.937 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:0/390942453 2024-07-04T15:35:52.741581+0000 2024-07-03T15:54:38.937 INFO:teuthology.orchestra.run.smithi028.stderr:listed 41 entries 2024-07-03T15:54:38.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.932+0000 7f495ab9d640 1 -- 172.21.15.28:0/1429544112 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f494406e0b0 msgr2=0x7f4944070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:38.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.932+0000 7f495ab9d640 1 --2- 172.21.15.28:0/1429544112 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f494406e0b0 0x7f4944070560 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f49400048a0 tx=0x7f4940009da0 comp rx=0 tx=0).stop 2024-07-03T15:54:38.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.932+0000 7f495ab9d640 1 -- 172.21.15.28:0/1429544112 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4954102270 msgr2=0x7f49540fbb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-03T15:54:38.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.932+0000 7f495ab9d640 1 --2- 172.21.15.28:0/1429544112 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4954102270 0x7f49540fbb40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4948002a50 tx=0x7f4948004600 comp rx=0 tx=0).stop 2024-07-03T15:54:38.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.932+0000 7f495ab9d640 1 -- 172.21.15.28:0/1429544112 shutdown_connections 2024-07-03T15:54:38.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.932+0000 7f495ab9d640 1 --2- 172.21.15.28:0/1429544112 >> [v2:172.21.15.28:6800/1205434373,v1:172.21.15.28:6801/1205434373] conn(0x7f494406e0b0 0x7f4944070560 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:38.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.932+0000 7f495ab9d640 1 --2- 172.21.15.28:0/1429544112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4954102c10 0x7f49540fc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:38.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.932+0000 7f495ab9d640 1 --2- 172.21.15.28:0/1429544112 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4954102270 0x7f49540fbb40 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-03T15:54:38.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.932+0000 7f495ab9d640 1 -- 172.21.15.28:0/1429544112 >> 172.21.15.28:0/1429544112 conn(0x7f4954070940 msgr2=0x7f4954105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-03T15:54:38.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.932+0000 7f495ab9d640 1 -- 172.21.15.28:0/1429544112 shutdown_connections 2024-07-03T15:54:38.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-07-03T15:54:38.932+0000 7f495ab9d640 1 -- 172.21.15.28:0/1429544112 wait complete. 2024-07-03T15:54:38.946 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-07-03T15:54:38.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-03T15:54:38.946 DEBUG:teuthology.orchestra.run.smithi148:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-07-03T15:54:38.982 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2024-07-03T15:54:38.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-03T15:54:38.982 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-07-03T15:54:39.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-03T15:54:39.124 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2024-07-03T15:54:39.141 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-07-03T15:54:39.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-03T15:54:39.141 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-03T15:54:39.203 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:38 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/3964743870' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-03T15:54:39.203 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:38 smithi028 ceph-mon[92858]: pgmap v285: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:54:39.203 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:38 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/2297056165' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-07-03T15:54:39.203 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:38 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/2318066950' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-07-03T15:54:39.203 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:38 smithi028 ceph-mon[92858]: from='client.? 172.21.15.28:0/1429544112' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-07-03T15:54:39.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-03T15:54:39.207 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link delete ceph-brx 2024-07-03T15:54:39.231 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:38 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/3964743870' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-03T15:54:39.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:38 smithi148 ceph-mon[80490]: pgmap v285: 65 pgs: 65 active+clean; 193 MiB data, 792 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-07-03T15:54:39.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:38 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/2297056165' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-07-03T15:54:39.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:38 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/2318066950' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-07-03T15:54:39.232 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:38 smithi148 ceph-mon[80490]: from='client.? 172.21.15.28:0/1429544112' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-07-03T15:54:39.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-03T15:54:39.302 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-07-03T15:54:39.321 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-07-03T15:54:39.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-03T15:54:39.321 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-03T15:54:39.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-03T15:54:39.391 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link delete ceph-brx 2024-07-03T15:54:39.496 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-07-03T15:54:39.506 INFO:tasks.cephadm:Teardown begin 2024-07-03T15:54:39.507 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-07-03T15:54:39.536 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-07-03T15:54:39.564 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-07-03T15:54:39.564 DEBUG:teuthology.orchestra.run.smithi028:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-07-03T15:54:39.592 DEBUG:teuthology.orchestra.run.smithi148:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-07-03T15:54:39.620 INFO:tasks.cephadm:Stopping all daemons... 2024-07-03T15:54:39.620 INFO:tasks.cephadm.mon.smithi028:Stopping mon.smithi028... 2024-07-03T15:54:39.620 DEBUG:teuthology.orchestra.run.smithi028:> sudo systemctl stop ceph-a85e4be6-3951-11ef-bcab-c7b262605968@mon.smithi028 2024-07-03T15:54:39.871 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:39 smithi028 systemd[1]: Stopping Ceph mon.smithi028 for a85e4be6-3951-11ef-bcab-c7b262605968... 2024-07-03T15:54:40.204 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:39 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-mon-smithi028[92854]: 2024-07-03T15:54:39.916+0000 7f3b91185640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi028 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-07-03T15:54:40.204 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:39 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-mon-smithi028[92854]: 2024-07-03T15:54:39.916+0000 7f3b91185640 -1 mon.smithi028@0(leader) e3 *** Got Signal Terminated *** 2024-07-03T15:54:40.204 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:39 smithi028 podman[145965]: 2024-07-03 15:54:39.968153268 +0000 UTC m=+0.146442833 container died 5093f7bf47d7d127fd7bce60c3a831acac39c7f908d95617cea2dfb1ac776bb4 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-mon-smithi028, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240625, CEPH_POINT_RELEASE=, RELEASE=squid-release-c00506d, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-07-03T15:54:40.612 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:40 smithi028 podman[145965]: 2024-07-03 15:54:40.350100662 +0000 UTC m=+0.528390269 container cleanup 5093f7bf47d7d127fd7bce60c3a831acac39c7f908d95617cea2dfb1ac776bb4 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-mon-smithi028, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-release-c00506d, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240625, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, ceph=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-07-03T15:54:40.612 INFO:journalctl@ceph.mon.smithi028.smithi028.stdout:Jul 03 15:54:40 smithi028 bash[145965]: ceph-a85e4be6-3951-11ef-bcab-c7b262605968-mon-smithi028 2024-07-03T15:54:40.791 DEBUG:teuthology.orchestra.run.smithi028:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a85e4be6-3951-11ef-bcab-c7b262605968@mon.smithi028.service' 2024-07-03T15:54:40.839 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-03T15:54:40.839 INFO:tasks.cephadm.mon.smithi028:Stopped mon.smithi028 2024-07-03T15:54:40.840 INFO:tasks.cephadm.mon.smithi148:Stopping mon.smithi148... 2024-07-03T15:54:40.840 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-a85e4be6-3951-11ef-bcab-c7b262605968@mon.smithi148 2024-07-03T15:54:41.126 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:40 smithi148 systemd[1]: Stopping Ceph mon.smithi148 for a85e4be6-3951-11ef-bcab-c7b262605968... 2024-07-03T15:54:41.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:41 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-mon-smithi148[80484]: 2024-07-03T15:54:41.122+0000 7fc37bab5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi148 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-07-03T15:54:41.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:41 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-mon-smithi148[80484]: 2024-07-03T15:54:41.122+0000 7fc37bab5640 -1 mon.smithi148@1(peon) e3 *** Got Signal Terminated *** 2024-07-03T15:54:41.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:41 smithi148 podman[116100]: 2024-07-03 15:54:41.347568644 +0000 UTC m=+0.314375690 container died d338d2c648dab4f59e621f5c94b28fea53c4543e6132d5f525d1b021a3a18467 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-mon-smithi148, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-release-c00506d, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240625) 2024-07-03T15:54:42.069 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:41 smithi148 podman[116100]: 2024-07-03 15:54:41.806838506 +0000 UTC m=+0.773645555 container cleanup d338d2c648dab4f59e621f5c94b28fea53c4543e6132d5f525d1b021a3a18467 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-mon-smithi148, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-release-c00506d, org.label-schema.build-date=20240625, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0) 2024-07-03T15:54:42.069 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Jul 03 15:54:41 smithi148 bash[116100]: ceph-a85e4be6-3951-11ef-bcab-c7b262605968-mon-smithi148 2024-07-03T15:54:42.204 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a85e4be6-3951-11ef-bcab-c7b262605968@mon.smithi148.service' 2024-07-03T15:54:42.254 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-03T15:54:42.254 INFO:tasks.cephadm.mon.smithi148:Stopped mon.smithi148 2024-07-03T15:54:42.254 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-07-03T15:54:42.254 DEBUG:teuthology.orchestra.run.smithi028:> sudo systemctl stop ceph-a85e4be6-3951-11ef-bcab-c7b262605968@osd.0 2024-07-03T15:54:42.583 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:54:42 smithi028 systemd[1]: Stopping Ceph osd.0 for a85e4be6-3951-11ef-bcab-c7b262605968... 2024-07-03T15:54:42.871 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:54:42 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-0[101308]: 2024-07-03T15:54:42.579+0000 7fe9398d3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-03T15:54:42.871 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:54:42 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-0[101308]: 2024-07-03T15:54:42.579+0000 7fe9398d3640 -1 osd.0 82 *** Got signal Terminated *** 2024-07-03T15:54:42.872 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:54:42 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-0[101308]: 2024-07-03T15:54:42.579+0000 7fe9398d3640 -1 osd.0 82 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-03T15:54:48.325 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:54:47 smithi028 podman[146100]: 2024-07-03 15:54:47.96531788 +0000 UTC m=+5.509277470 container died 4454cecff8fcd7ef64517202475d69270ead15f9771e4e2f283fdb197351ff30 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-0, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240625, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-release-c00506d, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0) 2024-07-03T15:54:48.622 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:54:48 smithi028 podman[146100]: 2024-07-03 15:54:48.413095776 +0000 UTC m=+5.957055364 container cleanup 4454cecff8fcd7ef64517202475d69270ead15f9771e4e2f283fdb197351ff30 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-release-c00506d, org.label-schema.build-date=20240625, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-07-03T15:54:48.622 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:54:48 smithi028 bash[146100]: ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-0 2024-07-03T15:54:49.061 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:54:48 smithi028 podman[146120]: 2024-07-03 15:54:48.631113751 +0000 UTC m=+0.658278670 container remove 4454cecff8fcd7ef64517202475d69270ead15f9771e4e2f283fdb197351ff30 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240625, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.36.0, RELEASE=squid-release-c00506d, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-07-03T15:54:49.338 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:54:49 smithi028 podman[146207]: 2024-07-03 15:54:48.961641674 +0000 UTC m=+0.017992150 image pull 9d49405603e8445bc510362cdeb05f243619be76b0f3897dd4c152b00cffe45f quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2 2024-07-03T15:54:49.338 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:54:49 smithi028 podman[146207]: 2024-07-03 15:54:49.073664241 +0000 UTC m=+0.130014692 container create 0e9fd33102cdda7e08e5148b9dcd6ebef767e928e5b8e40b2dff7239890a3c26 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.build-date=20240625, RELEASE=squid-release-c00506d, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True) 2024-07-03T15:54:49.338 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:54:49 smithi028 podman[146207]: 2024-07-03 15:54:49.276475359 +0000 UTC m=+0.332825826 container init 0e9fd33102cdda7e08e5148b9dcd6ebef767e928e5b8e40b2dff7239890a3c26 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240625, org.label-schema.vendor=CentOS, RELEASE=squid-release-c00506d, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-07-03T15:54:49.338 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:54:49 smithi028 podman[146207]: 2024-07-03 15:54:49.279919519 +0000 UTC m=+0.336269974 container start 0e9fd33102cdda7e08e5148b9dcd6ebef767e928e5b8e40b2dff7239890a3c26 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.build-date=20240625, ceph=True, RELEASE=squid-release-c00506d, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-07-03T15:54:49.622 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:54:49 smithi028 podman[146207]: 2024-07-03 15:54:49.368958732 +0000 UTC m=+0.425309201 container attach 0e9fd33102cdda7e08e5148b9dcd6ebef767e928e5b8e40b2dff7239890a3c26 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, org.label-schema.build-date=20240625, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-release-c00506d, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-07-03T15:54:49.622 INFO:journalctl@ceph.osd.0.smithi028.stdout:Jul 03 15:54:49 smithi028 podman[146207]: 2024-07-03 15:54:49.545149516 +0000 UTC m=+0.601499969 container died 0e9fd33102cdda7e08e5148b9dcd6ebef767e928e5b8e40b2dff7239890a3c26 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-release-c00506d, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240625, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0) 2024-07-03T15:54:50.141 DEBUG:teuthology.orchestra.run.smithi028:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a85e4be6-3951-11ef-bcab-c7b262605968@osd.0.service' 2024-07-03T15:54:50.186 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-03T15:54:50.186 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-07-03T15:54:50.186 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-07-03T15:54:50.186 DEBUG:teuthology.orchestra.run.smithi028:> sudo systemctl stop ceph-a85e4be6-3951-11ef-bcab-c7b262605968@osd.1 2024-07-03T15:54:50.426 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:54:50 smithi028 systemd[1]: Stopping Ceph osd.1 for a85e4be6-3951-11ef-bcab-c7b262605968... 2024-07-03T15:54:50.872 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:54:50 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-1[106705]: 2024-07-03T15:54:50.557+0000 7f161da68640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-03T15:54:50.872 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:54:50 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-1[106705]: 2024-07-03T15:54:50.557+0000 7f161da68640 -1 osd.1 82 *** Got signal Terminated *** 2024-07-03T15:54:50.872 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:54:50 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-1[106705]: 2024-07-03T15:54:50.557+0000 7f161da68640 -1 osd.1 82 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-03T15:54:56.219 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:54:55 smithi028 podman[146311]: 2024-07-03 15:54:55.959306712 +0000 UTC m=+5.521991157 container died af67f7088643df8759891f308a39b9da0d69befd1c684013a744d1d883f00ece (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-1, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240625, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-release-c00506d) 2024-07-03T15:54:56.512 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:54:56 smithi028 podman[146311]: 2024-07-03 15:54:56.299595379 +0000 UTC m=+5.862279830 container cleanup af67f7088643df8759891f308a39b9da0d69befd1c684013a744d1d883f00ece (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-1, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.build-date=20240625, CEPH_POINT_RELEASE=, RELEASE=squid-release-c00506d, ceph=True) 2024-07-03T15:54:56.512 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:54:56 smithi028 bash[146311]: ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-1 2024-07-03T15:54:56.871 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:54:56 smithi028 podman[146338]: 2024-07-03 15:54:56.509316079 +0000 UTC m=+0.550789570 container remove af67f7088643df8759891f308a39b9da0d69befd1c684013a744d1d883f00ece (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-1, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.build-date=20240625, ceph=True, RELEASE=squid-release-c00506d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.vendor=CentOS) 2024-07-03T15:54:57.198 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:54:56 smithi028 podman[146429]: 2024-07-03 15:54:56.843366019 +0000 UTC m=+0.017814634 image pull 9d49405603e8445bc510362cdeb05f243619be76b0f3897dd4c152b00cffe45f quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2 2024-07-03T15:54:57.198 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:54:56 smithi028 podman[146429]: 2024-07-03 15:54:56.960178646 +0000 UTC m=+0.134627253 container create 956669fba56e47ec0e583e41f5f10bdb6dd8a412ee2b30563b17c32cdcb7d236 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-release-c00506d, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240625, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-03T15:54:57.198 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:54:57 smithi028 podman[146429]: 2024-07-03 15:54:57.195718213 +0000 UTC m=+0.370166832 container init 956669fba56e47ec0e583e41f5f10bdb6dd8a412ee2b30563b17c32cdcb7d236 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-release-c00506d, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.build-date=20240625, CEPH_POINT_RELEASE=) 2024-07-03T15:54:57.464 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:54:57 smithi028 podman[146429]: 2024-07-03 15:54:57.199561479 +0000 UTC m=+0.374010089 container start 956669fba56e47ec0e583e41f5f10bdb6dd8a412ee2b30563b17c32cdcb7d236 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240625, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-release-c00506d, org.label-schema.vendor=CentOS) 2024-07-03T15:54:57.464 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:54:57 smithi028 podman[146429]: 2024-07-03 15:54:57.287488192 +0000 UTC m=+0.461936811 container attach 956669fba56e47ec0e583e41f5f10bdb6dd8a412ee2b30563b17c32cdcb7d236 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-release-c00506d, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240625, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD) 2024-07-03T15:54:57.760 INFO:journalctl@ceph.osd.1.smithi028.stdout:Jul 03 15:54:57 smithi028 podman[146429]: 2024-07-03 15:54:57.461854094 +0000 UTC m=+0.636302713 container died 956669fba56e47ec0e583e41f5f10bdb6dd8a412ee2b30563b17c32cdcb7d236 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-release-c00506d, org.label-schema.build-date=20240625, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-07-03T15:54:58.063 DEBUG:teuthology.orchestra.run.smithi028:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a85e4be6-3951-11ef-bcab-c7b262605968@osd.1.service' 2024-07-03T15:54:58.108 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-03T15:54:58.109 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-07-03T15:54:58.109 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-07-03T15:54:58.109 DEBUG:teuthology.orchestra.run.smithi028:> sudo systemctl stop ceph-a85e4be6-3951-11ef-bcab-c7b262605968@osd.2 2024-07-03T15:54:58.371 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:54:58 smithi028 systemd[1]: Stopping Ceph osd.2 for a85e4be6-3951-11ef-bcab-c7b262605968... 2024-07-03T15:54:58.871 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:54:58 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-2[112153]: 2024-07-03T15:54:58.477+0000 7ff895ebf640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-03T15:54:58.872 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:54:58 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-2[112153]: 2024-07-03T15:54:58.477+0000 7ff895ebf640 -1 osd.2 82 *** Got signal Terminated *** 2024-07-03T15:54:58.872 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:54:58 smithi028 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-2[112153]: 2024-07-03T15:54:58.477+0000 7ff895ebf640 -1 osd.2 82 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-03T15:55:04.072 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:55:03 smithi028 podman[146530]: 2024-07-03 15:55:03.816101022 +0000 UTC m=+5.463276900 container died 5d160a327dc2c08da25282c52d408154d36dc13ce6ca73d0e36fe860691ede4d (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240625, org.label-schema.license=GPLv2, RELEASE=squid-release-c00506d) 2024-07-03T15:55:04.364 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:55:04 smithi028 podman[146530]: 2024-07-03 15:55:04.152192339 +0000 UTC m=+5.799368216 container cleanup 5d160a327dc2c08da25282c52d408154d36dc13ce6ca73d0e36fe860691ede4d (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-2, ceph=True, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240625, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-release-c00506d, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-07-03T15:55:04.364 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:55:04 smithi028 bash[146530]: ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-2 2024-07-03T15:55:04.621 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:55:04 smithi028 podman[146547]: 2024-07-03 15:55:04.361697135 +0000 UTC m=+0.548216743 container remove 5d160a327dc2c08da25282c52d408154d36dc13ce6ca73d0e36fe860691ede4d (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=squid-release-c00506d, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240625) 2024-07-03T15:55:05.050 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:55:04 smithi028 podman[146637]: 2024-07-03 15:55:04.779009774 +0000 UTC m=+0.106573982 container create 40894fbe64c19e654a68e1137d6a3dd42e6c0d4bae9560316b63a92cd98c4883 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-release-c00506d, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240625) 2024-07-03T15:55:05.050 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:55:04 smithi028 podman[146637]: 2024-07-03 15:55:04.690233949 +0000 UTC m=+0.017798173 image pull 9d49405603e8445bc510362cdeb05f243619be76b0f3897dd4c152b00cffe45f quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2 2024-07-03T15:55:05.051 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:55:05 smithi028 podman[146637]: 2024-07-03 15:55:05.048228826 +0000 UTC m=+0.375793044 container init 40894fbe64c19e654a68e1137d6a3dd42e6c0d4bae9560316b63a92cd98c4883 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240625, GIT_CLEAN=True, io.buildah.version=1.36.0, RELEASE=squid-release-c00506d, GIT_BRANCH=HEAD, ceph=True) 2024-07-03T15:55:05.318 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:55:05 smithi028 podman[146637]: 2024-07-03 15:55:05.051886303 +0000 UTC m=+0.379450518 container start 40894fbe64c19e654a68e1137d6a3dd42e6c0d4bae9560316b63a92cd98c4883 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240625, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-release-c00506d, ceph=True, io.buildah.version=1.36.0) 2024-07-03T15:55:05.318 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:55:05 smithi028 podman[146637]: 2024-07-03 15:55:05.148449017 +0000 UTC m=+0.476013237 container attach 40894fbe64c19e654a68e1137d6a3dd42e6c0d4bae9560316b63a92cd98c4883 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.36.0, ceph=True, org.label-schema.build-date=20240625, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-release-c00506d, GIT_BRANCH=HEAD) 2024-07-03T15:55:05.594 INFO:journalctl@ceph.osd.2.smithi028.stdout:Jul 03 15:55:05 smithi028 podman[146637]: 2024-07-03 15:55:05.316507063 +0000 UTC m=+0.644071282 container died 40894fbe64c19e654a68e1137d6a3dd42e6c0d4bae9560316b63a92cd98c4883 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-release-c00506d, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240625, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-07-03T15:55:05.931 DEBUG:teuthology.orchestra.run.smithi028:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a85e4be6-3951-11ef-bcab-c7b262605968@osd.2.service' 2024-07-03T15:55:05.970 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-03T15:55:05.971 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-07-03T15:55:05.971 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-07-03T15:55:05.971 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-a85e4be6-3951-11ef-bcab-c7b262605968@osd.3 2024-07-03T15:55:06.329 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:55:06 smithi148 systemd[1]: Stopping Ceph osd.3 for a85e4be6-3951-11ef-bcab-c7b262605968... 2024-07-03T15:55:06.731 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:55:06 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3[89973]: 2024-07-03T15:55:06.326+0000 7f62530a8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-03T15:55:06.731 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:55:06 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3[89973]: 2024-07-03T15:55:06.326+0000 7f62530a8640 -1 osd.3 82 *** Got signal Terminated *** 2024-07-03T15:55:06.732 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:55:06 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3[89973]: 2024-07-03T15:55:06.326+0000 7f62530a8640 -1 osd.3 82 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-03T15:55:09.982 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:55:09 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5[100022]: 2024-07-03T15:55:09.663+0000 7f5b55c8b640 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-07-03T15:54:47.478399+0000 front 2024-07-03T15:54:47.478379+0000 (oldest deadline 2024-07-03T15:55:09.178253+0000) 2024-07-03T15:55:10.981 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:55:10 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5[100022]: 2024-07-03T15:55:10.686+0000 7f5b55c8b640 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-07-03T15:54:47.478399+0000 front 2024-07-03T15:54:47.478379+0000 (oldest deadline 2024-07-03T15:55:09.178253+0000) 2024-07-03T15:55:11.982 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:55:11 smithi148 podman[116234]: 2024-07-03 15:55:11.726328406 +0000 UTC m=+5.555324044 container died 1d622eba4b3b59f5a22317e0a65a6343794549b0b33a510f362445651e06f686 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=squid-release-c00506d, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240625, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-03T15:55:11.982 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:55:11 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5[100022]: 2024-07-03T15:55:11.688+0000 7f5b55c8b640 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-07-03T15:54:47.478399+0000 front 2024-07-03T15:54:47.478379+0000 (oldest deadline 2024-07-03T15:55:09.178253+0000) 2024-07-03T15:55:12.363 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:55:12 smithi148 podman[116234]: 2024-07-03 15:55:12.167822333 +0000 UTC m=+5.996817963 container cleanup 1d622eba4b3b59f5a22317e0a65a6343794549b0b33a510f362445651e06f686 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-release-c00506d, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, org.label-schema.build-date=20240625, CEPH_POINT_RELEASE=) 2024-07-03T15:55:12.364 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:55:12 smithi148 bash[116234]: ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3 2024-07-03T15:55:12.732 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:55:12 smithi148 podman[116250]: 2024-07-03 15:55:12.361185594 +0000 UTC m=+0.635242344 container remove 1d622eba4b3b59f5a22317e0a65a6343794549b0b33a510f362445651e06f686 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.build-date=20240625, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, RELEASE=squid-release-c00506d, maintainer=Guillaume Abrioux ) 2024-07-03T15:55:13.058 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:55:12 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5[100022]: 2024-07-03T15:55:12.734+0000 7f5b55c8b640 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-07-03T15:54:47.478399+0000 front 2024-07-03T15:54:47.478379+0000 (oldest deadline 2024-07-03T15:55:09.178253+0000) 2024-07-03T15:55:13.059 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:55:12 smithi148 podman[116346]: 2024-07-03 15:55:12.720726314 +0000 UTC m=+0.017241486 image pull 9d49405603e8445bc510362cdeb05f243619be76b0f3897dd4c152b00cffe45f quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2 2024-07-03T15:55:13.059 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:55:12 smithi148 podman[116346]: 2024-07-03 15:55:12.83722875 +0000 UTC m=+0.133743922 container create 9e9c000dd4df4dc4bb71b1bd5bcfa0a741f9a12617ab1bb274a7bb511bb4937a (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-release-c00506d, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240625, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_BRANCH=HEAD) 2024-07-03T15:55:13.059 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:55:13 smithi148 podman[116346]: 2024-07-03 15:55:13.056297358 +0000 UTC m=+0.352812530 container init 9e9c000dd4df4dc4bb71b1bd5bcfa0a741f9a12617ab1bb274a7bb511bb4937a (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240625, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, RELEASE=squid-release-c00506d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-03T15:55:13.325 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:55:13 smithi148 podman[116346]: 2024-07-03 15:55:13.059884282 +0000 UTC m=+0.356399464 container start 9e9c000dd4df4dc4bb71b1bd5bcfa0a741f9a12617ab1bb274a7bb511bb4937a (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.build-date=20240625, CEPH_POINT_RELEASE=, RELEASE=squid-release-c00506d) 2024-07-03T15:55:13.326 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:55:13 smithi148 podman[116346]: 2024-07-03 15:55:13.148136096 +0000 UTC m=+0.444651286 container attach 9e9c000dd4df4dc4bb71b1bd5bcfa0a741f9a12617ab1bb274a7bb511bb4937a (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240625, GIT_CLEAN=True, RELEASE=squid-release-c00506d, io.buildah.version=1.36.0) 2024-07-03T15:55:13.611 INFO:journalctl@ceph.osd.3.smithi148.stdout:Jul 03 15:55:13 smithi148 podman[116346]: 2024-07-03 15:55:13.323520806 +0000 UTC m=+0.620035988 container died 9e9c000dd4df4dc4bb71b1bd5bcfa0a741f9a12617ab1bb274a7bb511bb4937a (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240625, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-release-c00506d, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-07-03T15:55:13.980 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:55:13 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4[95224]: 2024-07-03T15:55:13.735+0000 7fed763bf640 -1 osd.4 82 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-07-03T15:54:46.736422+0000 front 2024-07-03T15:54:46.736492+0000 (oldest deadline 2024-07-03T15:55:13.136680+0000) 2024-07-03T15:55:13.980 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:55:13 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5[100022]: 2024-07-03T15:55:13.753+0000 7f5b55c8b640 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-07-03T15:54:47.478399+0000 front 2024-07-03T15:54:47.478379+0000 (oldest deadline 2024-07-03T15:55:09.178253+0000) 2024-07-03T15:55:13.999 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a85e4be6-3951-11ef-bcab-c7b262605968@osd.3.service' 2024-07-03T15:55:14.039 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-03T15:55:14.039 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-07-03T15:55:14.039 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-07-03T15:55:14.040 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-a85e4be6-3951-11ef-bcab-c7b262605968@osd.4 2024-07-03T15:55:14.231 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:55:14 smithi148 systemd[1]: Stopping Ceph osd.4 for a85e4be6-3951-11ef-bcab-c7b262605968... 2024-07-03T15:55:14.723 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:55:14 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4[95224]: 2024-07-03T15:55:14.363+0000 7fed795b9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-03T15:55:14.723 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:55:14 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4[95224]: 2024-07-03T15:55:14.363+0000 7fed795b9640 -1 osd.4 82 *** Got signal Terminated *** 2024-07-03T15:55:14.724 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:55:14 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4[95224]: 2024-07-03T15:55:14.363+0000 7fed795b9640 -1 osd.4 82 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-03T15:55:14.982 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:55:14 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5[100022]: 2024-07-03T15:55:14.720+0000 7f5b55c8b640 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-07-03T15:54:47.478399+0000 front 2024-07-03T15:54:47.478379+0000 (oldest deadline 2024-07-03T15:55:09.178253+0000) 2024-07-03T15:55:14.982 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:55:14 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4[95224]: 2024-07-03T15:55:14.751+0000 7fed763bf640 -1 osd.4 82 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-07-03T15:54:46.736422+0000 front 2024-07-03T15:54:46.736492+0000 (oldest deadline 2024-07-03T15:55:13.136680+0000) 2024-07-03T15:55:15.982 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:55:15 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5[100022]: 2024-07-03T15:55:15.672+0000 7f5b55c8b640 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-07-03T15:54:47.478399+0000 front 2024-07-03T15:54:47.478379+0000 (oldest deadline 2024-07-03T15:55:09.178253+0000) 2024-07-03T15:55:15.982 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:55:15 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4[95224]: 2024-07-03T15:55:15.712+0000 7fed763bf640 -1 osd.4 82 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-07-03T15:54:46.736422+0000 front 2024-07-03T15:54:46.736492+0000 (oldest deadline 2024-07-03T15:55:13.136680+0000) 2024-07-03T15:55:16.981 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:55:16 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5[100022]: 2024-07-03T15:55:16.632+0000 7f5b55c8b640 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-07-03T15:54:47.478399+0000 front 2024-07-03T15:54:47.478379+0000 (oldest deadline 2024-07-03T15:55:09.178253+0000) 2024-07-03T15:55:16.982 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:55:16 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4[95224]: 2024-07-03T15:55:16.737+0000 7fed763bf640 -1 osd.4 82 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-07-03T15:54:46.736422+0000 front 2024-07-03T15:54:46.736492+0000 (oldest deadline 2024-07-03T15:55:13.136680+0000) 2024-07-03T15:55:16.982 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:55:16 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4[95224]: 2024-07-03T15:55:16.737+0000 7fed763bf640 -1 osd.4 82 heartbeat_check: no reply from 172.21.15.28:6814 osd.1 since back 2024-07-03T15:54:53.137137+0000 front 2024-07-03T15:54:53.137159+0000 (oldest deadline 2024-07-03T15:55:16.636958+0000) 2024-07-03T15:55:17.982 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:55:17 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4[95224]: 2024-07-03T15:55:17.770+0000 7fed763bf640 -1 osd.4 82 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-07-03T15:54:46.736422+0000 front 2024-07-03T15:54:46.736492+0000 (oldest deadline 2024-07-03T15:55:13.136680+0000) 2024-07-03T15:55:17.982 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:55:17 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4[95224]: 2024-07-03T15:55:17.770+0000 7fed763bf640 -1 osd.4 82 heartbeat_check: no reply from 172.21.15.28:6814 osd.1 since back 2024-07-03T15:54:53.137137+0000 front 2024-07-03T15:54:53.137159+0000 (oldest deadline 2024-07-03T15:55:16.636958+0000) 2024-07-03T15:55:17.982 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:55:17 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5[100022]: 2024-07-03T15:55:17.655+0000 7f5b55c8b640 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-07-03T15:54:47.478399+0000 front 2024-07-03T15:54:47.478379+0000 (oldest deadline 2024-07-03T15:55:09.178253+0000) 2024-07-03T15:55:17.982 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:55:17 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5[100022]: 2024-07-03T15:55:17.655+0000 7f5b55c8b640 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.28:6814 osd.1 since back 2024-07-03T15:54:53.179582+0000 front 2024-07-03T15:54:53.179560+0000 (oldest deadline 2024-07-03T15:55:16.679622+0000) 2024-07-03T15:55:18.982 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:55:18 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4[95224]: 2024-07-03T15:55:18.808+0000 7fed763bf640 -1 osd.4 82 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-07-03T15:54:46.736422+0000 front 2024-07-03T15:54:46.736492+0000 (oldest deadline 2024-07-03T15:55:13.136680+0000) 2024-07-03T15:55:18.982 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:55:18 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4[95224]: 2024-07-03T15:55:18.808+0000 7fed763bf640 -1 osd.4 82 heartbeat_check: no reply from 172.21.15.28:6814 osd.1 since back 2024-07-03T15:54:53.137137+0000 front 2024-07-03T15:54:53.137159+0000 (oldest deadline 2024-07-03T15:55:16.636958+0000) 2024-07-03T15:55:18.982 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:55:18 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5[100022]: 2024-07-03T15:55:18.700+0000 7f5b55c8b640 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-07-03T15:54:47.478399+0000 front 2024-07-03T15:54:47.478379+0000 (oldest deadline 2024-07-03T15:55:09.178253+0000) 2024-07-03T15:55:18.982 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:55:18 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5[100022]: 2024-07-03T15:55:18.700+0000 7f5b55c8b640 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.28:6814 osd.1 since back 2024-07-03T15:54:53.179582+0000 front 2024-07-03T15:54:53.179560+0000 (oldest deadline 2024-07-03T15:55:16.679622+0000) 2024-07-03T15:55:20.134 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:55:19 smithi148 podman[116451]: 2024-07-03 15:55:19.824929088 +0000 UTC m=+5.560129757 container died 546343d0f31f6ddd8b4fe56c03a6832414698b9328b6824154bffa3d1af74df3 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240625, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-release-c00506d, io.buildah.version=1.36.0, GIT_BRANCH=HEAD) 2024-07-03T15:55:20.134 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:55:19 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5[100022]: 2024-07-03T15:55:19.735+0000 7f5b55c8b640 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-07-03T15:54:47.478399+0000 front 2024-07-03T15:54:47.478379+0000 (oldest deadline 2024-07-03T15:55:09.178253+0000) 2024-07-03T15:55:20.134 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:55:19 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5[100022]: 2024-07-03T15:55:19.735+0000 7f5b55c8b640 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.28:6814 osd.1 since back 2024-07-03T15:54:53.179582+0000 front 2024-07-03T15:54:53.179560+0000 (oldest deadline 2024-07-03T15:55:16.679622+0000) 2024-07-03T15:55:20.426 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:55:20 smithi148 podman[116451]: 2024-07-03 15:55:20.272085468 +0000 UTC m=+6.007286178 container cleanup 546343d0f31f6ddd8b4fe56c03a6832414698b9328b6824154bffa3d1af74df3 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=squid-release-c00506d, org.label-schema.build-date=20240625, CEPH_POINT_RELEASE=) 2024-07-03T15:55:20.426 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:55:20 smithi148 bash[116451]: ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4 2024-07-03T15:55:20.732 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:55:20 smithi148 podman[116466]: 2024-07-03 15:55:20.42343703 +0000 UTC m=+0.593023818 container remove 546343d0f31f6ddd8b4fe56c03a6832414698b9328b6824154bffa3d1af74df3 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-release-c00506d, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240625, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-07-03T15:55:20.997 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:55:20 smithi148 podman[116555]: 2024-07-03 15:55:20.747841109 +0000 UTC m=+0.017104526 image pull 9d49405603e8445bc510362cdeb05f243619be76b0f3897dd4c152b00cffe45f quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2 2024-07-03T15:55:20.997 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:55:20 smithi148 podman[116555]: 2024-07-03 15:55:20.868920177 +0000 UTC m=+0.138183637 container create 94d26d019bd74012a20fe551b3ecbb8a538ca2448a72c4c957452c9f4449cf2c (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-release-c00506d, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240625, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux ) 2024-07-03T15:55:20.997 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:55:20 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5[100022]: 2024-07-03T15:55:20.743+0000 7f5b55c8b640 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-07-03T15:54:47.478399+0000 front 2024-07-03T15:54:47.478379+0000 (oldest deadline 2024-07-03T15:55:09.178253+0000) 2024-07-03T15:55:20.998 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:55:20 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5[100022]: 2024-07-03T15:55:20.743+0000 7f5b55c8b640 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.28:6814 osd.1 since back 2024-07-03T15:54:53.179582+0000 front 2024-07-03T15:54:53.179560+0000 (oldest deadline 2024-07-03T15:55:16.679622+0000) 2024-07-03T15:55:21.344 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:55:21 smithi148 podman[116555]: 2024-07-03 15:55:21.076846296 +0000 UTC m=+0.346109713 container init 94d26d019bd74012a20fe551b3ecbb8a538ca2448a72c4c957452c9f4449cf2c (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, ceph=True, RELEASE=squid-release-c00506d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240625, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-03T15:55:21.345 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:55:21 smithi148 podman[116555]: 2024-07-03 15:55:21.080340493 +0000 UTC m=+0.349603910 container start 94d26d019bd74012a20fe551b3ecbb8a538ca2448a72c4c957452c9f4449cf2c (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4-deactivate, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, RELEASE=squid-release-c00506d, org.label-schema.build-date=20240625, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-07-03T15:55:21.345 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:55:21 smithi148 podman[116555]: 2024-07-03 15:55:21.160239446 +0000 UTC m=+0.429502880 container attach 94d26d019bd74012a20fe551b3ecbb8a538ca2448a72c4c957452c9f4449cf2c (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-release-c00506d, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, org.label-schema.build-date=20240625, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-03T15:55:21.640 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:55:21 smithi148 podman[116555]: 2024-07-03 15:55:21.342551766 +0000 UTC m=+0.611815192 container died 94d26d019bd74012a20fe551b3ecbb8a538ca2448a72c4c957452c9f4449cf2c (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-release-c00506d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240625, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.36.0) 2024-07-03T15:55:21.891 INFO:journalctl@ceph.osd.4.smithi148.stdout:Jul 03 15:55:21 smithi148 podman[116578]: 2024-07-03 15:55:21.872042065 +0000 UTC m=+0.523665862 container remove 94d26d019bd74012a20fe551b3ecbb8a538ca2448a72c4c957452c9f4449cf2c (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240625, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=squid-release-c00506d, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-07-03T15:55:21.891 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:55:21 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5[100022]: 2024-07-03T15:55:21.751+0000 7f5b55c8b640 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-07-03T15:54:47.478399+0000 front 2024-07-03T15:54:47.478379+0000 (oldest deadline 2024-07-03T15:55:09.178253+0000) 2024-07-03T15:55:21.891 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:55:21 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5[100022]: 2024-07-03T15:55:21.751+0000 7f5b55c8b640 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.28:6814 osd.1 since back 2024-07-03T15:54:53.179582+0000 front 2024-07-03T15:54:53.179560+0000 (oldest deadline 2024-07-03T15:55:16.679622+0000) 2024-07-03T15:55:21.895 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a85e4be6-3951-11ef-bcab-c7b262605968@osd.4.service' 2024-07-03T15:55:21.934 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-03T15:55:21.934 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-07-03T15:55:21.935 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-07-03T15:55:21.935 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-a85e4be6-3951-11ef-bcab-c7b262605968@osd.5 2024-07-03T15:55:22.232 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:55:22 smithi148 systemd[1]: Stopping Ceph osd.5 for a85e4be6-3951-11ef-bcab-c7b262605968... 2024-07-03T15:55:22.661 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:55:22 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5[100022]: 2024-07-03T15:55:22.266+0000 7f5b5ce8d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-03T15:55:22.661 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:55:22 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5[100022]: 2024-07-03T15:55:22.266+0000 7f5b5ce8d640 -1 osd.5 82 *** Got signal Terminated *** 2024-07-03T15:55:22.661 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:55:22 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5[100022]: 2024-07-03T15:55:22.266+0000 7f5b5ce8d640 -1 osd.5 82 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-03T15:55:22.982 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:55:22 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5[100022]: 2024-07-03T15:55:22.745+0000 7f5b55c8b640 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-07-03T15:54:47.478399+0000 front 2024-07-03T15:54:47.478379+0000 (oldest deadline 2024-07-03T15:55:09.178253+0000) 2024-07-03T15:55:22.982 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:55:22 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5[100022]: 2024-07-03T15:55:22.745+0000 7f5b55c8b640 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.28:6814 osd.1 since back 2024-07-03T15:54:53.179582+0000 front 2024-07-03T15:54:53.179560+0000 (oldest deadline 2024-07-03T15:55:16.679622+0000) 2024-07-03T15:55:24.232 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:55:23 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5[100022]: 2024-07-03T15:55:23.774+0000 7f5b55c8b640 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-07-03T15:54:47.478399+0000 front 2024-07-03T15:54:47.478379+0000 (oldest deadline 2024-07-03T15:55:09.178253+0000) 2024-07-03T15:55:24.232 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:55:23 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5[100022]: 2024-07-03T15:55:23.774+0000 7f5b55c8b640 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.28:6814 osd.1 since back 2024-07-03T15:54:53.179582+0000 front 2024-07-03T15:54:53.179560+0000 (oldest deadline 2024-07-03T15:55:16.679622+0000) 2024-07-03T15:55:25.232 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:55:24 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5[100022]: 2024-07-03T15:55:24.810+0000 7f5b55c8b640 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-07-03T15:54:47.478399+0000 front 2024-07-03T15:54:47.478379+0000 (oldest deadline 2024-07-03T15:55:09.178253+0000) 2024-07-03T15:55:25.232 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:55:24 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5[100022]: 2024-07-03T15:55:24.810+0000 7f5b55c8b640 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.28:6814 osd.1 since back 2024-07-03T15:54:53.179582+0000 front 2024-07-03T15:54:53.179560+0000 (oldest deadline 2024-07-03T15:55:16.679622+0000) 2024-07-03T15:55:26.232 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:55:25 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5[100022]: 2024-07-03T15:55:25.823+0000 7f5b55c8b640 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-07-03T15:54:47.478399+0000 front 2024-07-03T15:54:47.478379+0000 (oldest deadline 2024-07-03T15:55:09.178253+0000) 2024-07-03T15:55:26.232 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:55:25 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5[100022]: 2024-07-03T15:55:25.823+0000 7f5b55c8b640 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.28:6814 osd.1 since back 2024-07-03T15:54:53.179582+0000 front 2024-07-03T15:54:53.179560+0000 (oldest deadline 2024-07-03T15:55:16.679622+0000) 2024-07-03T15:55:27.232 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:55:26 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5[100022]: 2024-07-03T15:55:26.868+0000 7f5b55c8b640 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-07-03T15:54:47.478399+0000 front 2024-07-03T15:54:47.478379+0000 (oldest deadline 2024-07-03T15:55:09.178253+0000) 2024-07-03T15:55:27.232 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:55:26 smithi148 ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5[100022]: 2024-07-03T15:55:26.868+0000 7f5b55c8b640 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.28:6814 osd.1 since back 2024-07-03T15:54:53.179582+0000 front 2024-07-03T15:54:53.179560+0000 (oldest deadline 2024-07-03T15:55:16.679622+0000) 2024-07-03T15:55:28.103 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:55:27 smithi148 podman[116657]: 2024-07-03 15:55:27.802196693 +0000 UTC m=+5.640370223 container died a819fcc3b5696d337d6bfb3999e88dc58da6d2c415f1a7636f84873cbbcd9f3f (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240625, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-release-c00506d) 2024-07-03T15:55:28.482 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:55:28 smithi148 podman[116657]: 2024-07-03 15:55:28.275309301 +0000 UTC m=+6.113482822 container cleanup a819fcc3b5696d337d6bfb3999e88dc58da6d2c415f1a7636f84873cbbcd9f3f (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5, CEPH_POINT_RELEASE=, RELEASE=squid-release-c00506d, org.label-schema.build-date=20240625, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-03T15:55:28.482 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:55:28 smithi148 bash[116657]: ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5 2024-07-03T15:55:28.840 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:55:28 smithi148 podman[116672]: 2024-07-03 15:55:28.509993401 +0000 UTC m=+0.708878449 container remove a819fcc3b5696d337d6bfb3999e88dc58da6d2c415f1a7636f84873cbbcd9f3f (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240625, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, RELEASE=squid-release-c00506d, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0) 2024-07-03T15:55:29.134 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:55:28 smithi148 podman[116761]: 2024-07-03 15:55:28.837616024 +0000 UTC m=+0.101619591 container create c288029049916e7c6389c2200fe2655317cff263eaa69de66dc62ec892ce1635 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240625, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-release-c00506d, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0) 2024-07-03T15:55:29.134 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:55:28 smithi148 podman[116761]: 2024-07-03 15:55:28.753551153 +0000 UTC m=+0.017554678 image pull 9d49405603e8445bc510362cdeb05f243619be76b0f3897dd4c152b00cffe45f quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2 2024-07-03T15:55:29.134 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:55:29 smithi148 podman[116761]: 2024-07-03 15:55:29.050106973 +0000 UTC m=+0.314110506 container init c288029049916e7c6389c2200fe2655317cff263eaa69de66dc62ec892ce1635 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, RELEASE=squid-release-c00506d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240625, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-07-03T15:55:29.135 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:55:29 smithi148 podman[116761]: 2024-07-03 15:55:29.053926301 +0000 UTC m=+0.317929822 container start c288029049916e7c6389c2200fe2655317cff263eaa69de66dc62ec892ce1635 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240625, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=squid-release-c00506d, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-07-03T15:55:29.135 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:55:29 smithi148 podman[116761]: 2024-07-03 15:55:29.13166537 +0000 UTC m=+0.395668900 container attach c288029049916e7c6389c2200fe2655317cff263eaa69de66dc62ec892ce1635 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240625, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-release-c00506d) 2024-07-03T15:55:29.482 INFO:journalctl@ceph.osd.5.smithi148.stdout:Jul 03 15:55:29 smithi148 podman[116761]: 2024-07-03 15:55:29.315391393 +0000 UTC m=+0.579394915 container died c288029049916e7c6389c2200fe2655317cff263eaa69de66dc62ec892ce1635 (image=quay.ceph.io/ceph-ci/ceph@sha256:3f957f352f90933ee70f469925dc807e01e986f1554d2223755948c2c1bb1fb2, name=ceph-a85e4be6-3951-11ef-bcab-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, RELEASE=squid-release-c00506d, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240625, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-07-03T15:55:29.961 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a85e4be6-3951-11ef-bcab-c7b262605968@osd.5.service' 2024-07-03T15:55:29.999 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-03T15:55:29.999 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-07-03T15:55:30.000 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a85e4be6-3951-11ef-bcab-c7b262605968 --force --keep-logs 2024-07-03T15:55:52.344 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a85e4be6-3951-11ef-bcab-c7b262605968 --force --keep-logs 2024-07-03T15:56:06.647 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-07-03T15:56:06.682 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-07-03T15:56:06.710 INFO:tasks.cephadm:Archiving crash dumps... 2024-07-03T15:56:06.711 DEBUG:teuthology.misc:Transferring archived files from smithi028:/var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-07-03_14:50:01-orch:cephadm-squid-release-distro-default-smithi/7785842/remote/smithi028/crash 2024-07-03T15:56:06.712 DEBUG:teuthology.orchestra.run.smithi028:> sudo tar c -f - -C /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/crash -- . 2024-07-03T15:56:06.753 INFO:teuthology.orchestra.run.smithi028.stderr:tar: /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/crash: Cannot open: No such file or directory 2024-07-03T15:56:06.753 INFO:teuthology.orchestra.run.smithi028.stderr:tar: Error is not recoverable: exiting now 2024-07-03T15:56:06.755 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-07-03_14:50:01-orch:cephadm-squid-release-distro-default-smithi/7785842/remote/smithi148/crash 2024-07-03T15:56:06.756 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/crash -- . 2024-07-03T15:56:06.780 INFO:teuthology.orchestra.run.smithi148.stderr:tar: /var/lib/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/crash: Cannot open: No such file or directory 2024-07-03T15:56:06.780 INFO:teuthology.orchestra.run.smithi148.stderr:tar: Error is not recoverable: exiting now 2024-07-03T15:56:06.781 INFO:tasks.cephadm:Checking cluster log for badness... 2024-07-03T15:56:06.782 DEBUG:teuthology.orchestra.run.smithi028:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-07-03T15:56:06.834 INFO:teuthology.orchestra.run.smithi028.stdout:2024-07-03T15:50:00.000121+0000 mon.smithi028 (mon.0) 474 : cluster 3 [WRN] PG_DEGRADED: Degraded data redundancy: 31/219 objects degraded (14.155%), 13 pgs degraded 2024-07-03T15:56:06.834 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-07-03T15:56:06.835 DEBUG:teuthology.orchestra.run.smithi028:> sudo egrep '\[SEC\]' /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-07-03T15:56:06.910 DEBUG:teuthology.orchestra.run.smithi028:> sudo egrep '\[ERR\]' /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-07-03T15:56:06.986 DEBUG:teuthology.orchestra.run.smithi028:> sudo egrep '\[WRN\]' /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-07-03T15:56:07.061 INFO:teuthology.orchestra.run.smithi028.stdout:2024-07-03T15:50:00.000121+0000 mon.smithi028 (mon.0) 474 : cluster 3 [WRN] PG_DEGRADED: Degraded data redundancy: 31/219 objects degraded (14.155%), 13 pgs degraded 2024-07-03T15:56:07.062 INFO:tasks.cephadm:Compressing logs... 2024-07-03T15:56:07.062 DEBUG:teuthology.orchestra.run.smithi028:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-07-03T15:56:07.106 DEBUG:teuthology.orchestra.run.smithi148:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-07-03T15:56:07.129 INFO:teuthology.orchestra.run.smithi028.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-07-03T15:56:07.130 INFO:teuthology.orchestra.run.smithi028.stderr:: No such file or directory 2024-07-03T15:56:07.130 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-mgr.smithi028.vtqrtz.log 2024-07-03T15:56:07.130 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose/var/log/ceph/cephadm.log: -- /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph.log 2024-07-03T15:56:07.130 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-mgr.smithi028.vtqrtz.log: gzip -5 --verbose -- /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-osd.2.log 2024-07-03T15:56:07.131 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph.audit.log 2024-07-03T15:56:07.131 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-osd.1.log 2024-07-03T15:56:07.132 INFO:teuthology.orchestra.run.smithi148.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-07-03T15:56:07.133 INFO:teuthology.orchestra.run.smithi148.stderr:: No such file or directory 2024-07-03T15:56:07.133 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-mds.cephfs.smithi148.szomfl.log 2024-07-03T15:56:07.133 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-mon.smithi148.log 2024-07-03T15:56:07.133 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph.log 2024-07-03T15:56:07.133 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-mds.cephfs.smithi148.szomfl.log: /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-mon.smithi148.log: gzip -5 --verbose -- /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-mgr.smithi148.rjzsar.log 2024-07-03T15:56:07.134 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-mds.cephfs.smithi028.rhpvve.log 2024-07-03T15:56:07.134 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-osd.0.log 2024-07-03T15:56:07.134 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-mds.cephfs.smithi028.rhpvve.log: 87.2% -- replaced with /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph.log.gz 2024-07-03T15:56:07.134 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph.cephadm.log 2024-07-03T15:56:07.136 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph.audit.log 2024-07-03T15:56:07.136 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph.log: /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-mgr.smithi148.rjzsar.log: gzip -5 --verbose -- /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-osd.5.log 2024-07-03T15:56:07.136 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-osd.3.log 2024-07-03T15:56:07.136 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-osd.5.log: gzip -5 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-07-03T15:56:07.137 INFO:teuthology.orchestra.run.smithi148.stderr: --verbose -- /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-osd.4.log 2024-07-03T15:56:07.137 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph.cephadm.log 2024-07-03T15:56:07.137 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-osd.3.log: /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-osd.4.log: 87.2% -- replaced with /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph.log.gz 2024-07-03T15:56:07.138 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-osd.0.log: 91.1% -- replaced with /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph.audit.log.gz 2024-07-03T15:56:07.139 INFO:teuthology.orchestra.run.smithi148.stderr: 91.2% -- replaced with /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph.audit.log.gz 2024-07-03T15:56:07.139 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-mds.cephfs.smithi148.xgpuld.log 2024-07-03T15:56:07.139 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-volume.log 2024-07-03T15:56:07.139 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-mds.cephfs.smithi148.xgpuld.log: 85.4% -- replaced with /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph.cephadm.log.gz 2024-07-03T15:56:07.146 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-volume.log: 91.9% -- replaced with /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-mds.cephfs.smithi148.xgpuld.log.gz 2024-07-03T15:56:07.154 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-mon.smithi028.log 2024-07-03T15:56:07.155 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph.cephadm.log: 86.4% -- replaced with /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph.cephadm.log.gz 2024-07-03T15:56:07.161 INFO:teuthology.orchestra.run.smithi148.stderr: 91.1% -- replaced with /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-volume.log.gz 2024-07-03T15:56:07.162 INFO:teuthology.orchestra.run.smithi028.stderr: 92.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-07-03T15:56:07.163 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-mds.cephfs.smithi028.wmbxew.log 2024-07-03T15:56:07.163 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-mon.smithi028.log: gzip -5 --verbose -- /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-volume.log 2024-07-03T15:56:07.198 INFO:teuthology.orchestra.run.smithi148.stderr: 89.6% -- replaced with /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-mgr.smithi148.rjzsar.log.gz 2024-07-03T15:56:07.229 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-mds.cephfs.smithi028.wmbxew.log: /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-volume.log: 91.0% -- replaced with /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-volume.log.gz 2024-07-03T15:56:07.372 INFO:teuthology.orchestra.run.smithi148.stderr: 92.5% -- replaced with /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-mon.smithi148.log.gz 2024-07-03T15:56:07.489 INFO:teuthology.orchestra.run.smithi028.stderr: 88.4% -- replaced with /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-mgr.smithi028.vtqrtz.log.gz 2024-07-03T15:56:08.167 INFO:teuthology.orchestra.run.smithi028.stderr: 90.6% -- replaced with /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-mon.smithi028.log.gz 2024-07-03T15:56:11.317 INFO:teuthology.orchestra.run.smithi148.stderr: 93.4% -- replaced with /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-osd.4.log.gz 2024-07-03T15:56:12.039 INFO:teuthology.orchestra.run.smithi028.stderr: 93.6% -- replaced with /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-osd.2.log.gz 2024-07-03T15:56:12.293 INFO:teuthology.orchestra.run.smithi148.stderr: 94.9% -- replaced with /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-mds.cephfs.smithi148.szomfl.log.gz 2024-07-03T15:56:12.660 INFO:teuthology.orchestra.run.smithi028.stderr: 93.4% -- replaced with /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-osd.0.log.gz 2024-07-03T15:56:12.869 INFO:teuthology.orchestra.run.smithi148.stderr: 93.5% -- replaced with /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-osd.5.log.gz 2024-07-03T15:56:13.138 INFO:teuthology.orchestra.run.smithi028.stderr: 93.5% -- replaced with /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-osd.1.log.gz 2024-07-03T15:56:13.492 INFO:teuthology.orchestra.run.smithi148.stderr: 93.5% -- replaced with /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-osd.3.log.gz 2024-07-03T15:56:13.494 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-07-03T15:56:13.494 INFO:teuthology.orchestra.run.smithi148.stderr:real 0m6.373s 2024-07-03T15:56:13.494 INFO:teuthology.orchestra.run.smithi148.stderr:user 0m20.975s 2024-07-03T15:56:13.494 INFO:teuthology.orchestra.run.smithi148.stderr:sys 0m0.792s 2024-07-03T15:56:21.939 INFO:teuthology.orchestra.run.smithi028.stderr: 95.0% -- replaced with /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-mds.cephfs.smithi028.wmbxew.log.gz 2024-07-03T15:57:47.630 INFO:teuthology.orchestra.run.smithi028.stderr: 92.9% -- replaced with /var/log/ceph/a85e4be6-3951-11ef-bcab-c7b262605968/ceph-mds.cephfs.smithi028.rhpvve.log.gz 2024-07-03T15:57:47.632 INFO:teuthology.orchestra.run.smithi028.stderr: 2024-07-03T15:57:47.632 INFO:teuthology.orchestra.run.smithi028.stderr:real 1m40.512s 2024-07-03T15:57:47.632 INFO:teuthology.orchestra.run.smithi028.stderr:user 2m9.127s 2024-07-03T15:57:47.632 INFO:teuthology.orchestra.run.smithi028.stderr:sys 0m3.693s 2024-07-03T15:57:47.633 INFO:tasks.cephadm:Archiving logs... 2024-07-03T15:57:47.634 DEBUG:teuthology.misc:Transferring archived files from smithi028:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-07-03_14:50:01-orch:cephadm-squid-release-distro-default-smithi/7785842/remote/smithi028/log 2024-07-03T15:57:47.635 DEBUG:teuthology.orchestra.run.smithi028:> sudo tar c -f - -C /var/log/ceph -- . 2024-07-03T15:58:05.446 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-07-03_14:50:01-orch:cephadm-squid-release-distro-default-smithi/7785842/remote/smithi148/log 2024-07-03T15:58:05.447 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /var/log/ceph -- . 2024-07-03T15:58:08.425 INFO:tasks.cephadm:Removing cluster... 2024-07-03T15:58:08.425 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a85e4be6-3951-11ef-bcab-c7b262605968 --force 2024-07-03T15:58:09.067 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a85e4be6-3951-11ef-bcab-c7b262605968 --force 2024-07-03T15:58:09.572 INFO:tasks.cephadm:Removing cephadm ... 2024-07-03T15:58:09.572 DEBUG:teuthology.orchestra.run.smithi028:> rm -rf /home/ubuntu/cephtest/cephadm 2024-07-03T15:58:09.589 DEBUG:teuthology.orchestra.run.smithi148:> rm -rf /home/ubuntu/cephtest/cephadm 2024-07-03T15:58:09.605 INFO:tasks.cephadm:Teardown complete 2024-07-03T15:58:09.605 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-07-03T15:58:09.617 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-07-03T15:58:09.617 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-07-03T15:58:09.634 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-07-03T15:58:09.712 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-07-03T15:58:09.723 INFO:teuthology.task.clock:Checking final clock skew... 2024-07-03T15:58:09.723 DEBUG:teuthology.orchestra.run.smithi028:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-03T15:58:09.725 DEBUG:teuthology.orchestra.run.smithi148:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-03T15:58:09.739 INFO:teuthology.orchestra.run.smithi028.stderr:bash: line 1: ntpq: command not found 2024-07-03T15:58:09.739 INFO:teuthology.orchestra.run.smithi148.stderr:bash: line 1: ntpq: command not found 2024-07-03T15:58:09.744 INFO:teuthology.orchestra.run.smithi028.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-03T15:58:09.744 INFO:teuthology.orchestra.run.smithi028.stdout:=============================================================================== 2024-07-03T15:58:09.744 INFO:teuthology.orchestra.run.smithi028.stdout:^* hv01.front.sepia.ceph.com 3 6 377 40 -174us[ -194us] +/- 47ms 2024-07-03T15:58:09.744 INFO:teuthology.orchestra.run.smithi028.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 39 +2822us[+2822us] +/- 82ms 2024-07-03T15:58:09.744 INFO:teuthology.orchestra.run.smithi028.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 40 -2208us[-2208us] +/- 69ms 2024-07-03T15:58:09.744 INFO:teuthology.orchestra.run.smithi028.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-07-03T15:58:09.746 INFO:teuthology.orchestra.run.smithi148.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-03T15:58:09.746 INFO:teuthology.orchestra.run.smithi148.stdout:=============================================================================== 2024-07-03T15:58:09.746 INFO:teuthology.orchestra.run.smithi148.stdout:^* hv01.front.sepia.ceph.com 3 6 377 39 -148us[ -162us] +/- 47ms 2024-07-03T15:58:09.746 INFO:teuthology.orchestra.run.smithi148.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 39 +2799us[+2799us] +/- 82ms 2024-07-03T15:58:09.746 INFO:teuthology.orchestra.run.smithi148.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 40 -2205us[-2219us] +/- 69ms 2024-07-03T15:58:09.746 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-07-03T15:58:09.746 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-07-03T15:58:09.757 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-07-03T15:58:09.757 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-07-03T15:58:09.772 DEBUG:teuthology.orchestra.run.smithi028:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-07-03T15:58:09.870 DEBUG:teuthology.orchestra.run.smithi148:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-07-03T15:58:09.943 DEBUG:teuthology.orchestra.run.smithi028:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-03T15:58:09.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-03T15:58:09.973 DEBUG:teuthology.orchestra.run.smithi148:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-03T15:58:10.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-03T15:58:10.001 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-07-03T15:58:10.012 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-07-03T15:58:10.035 INFO:teuthology.task.internal:Duration was 1869.620412 seconds 2024-07-03T15:58:10.036 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-07-03T15:58:10.046 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-07-03T15:58:10.046 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-07-03T15:58:10.049 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-07-03T15:58:10.087 INFO:teuthology.orchestra.run.smithi148.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-03T15:58:10.088 INFO:teuthology.orchestra.run.smithi028.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-03T15:58:10.523 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-07-03T15:58:10.523 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi028.front.sepia.ceph.com 2024-07-03T15:58:10.523 DEBUG:teuthology.orchestra.run.smithi028:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-07-03T15:58:10.544 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi148.front.sepia.ceph.com 2024-07-03T15:58:10.545 DEBUG:teuthology.orchestra.run.smithi148:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-07-03T15:58:10.591 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-07-03T15:58:10.592 DEBUG:teuthology.orchestra.run.smithi028:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-07-03T15:58:10.594 DEBUG:teuthology.orchestra.run.smithi148:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-07-03T15:58:10.659 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-07-03T15:58:10.659 DEBUG:teuthology.orchestra.run.smithi028:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-07-03T15:58:10.662 DEBUG:teuthology.orchestra.run.smithi148:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-07-03T15:58:11.286 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-07-03T15:58:11.298 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-07-03T15:58:11.298 DEBUG:teuthology.orchestra.run.smithi028:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-07-03T15:58:11.325 DEBUG:teuthology.orchestra.run.smithi148:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-07-03T15:58:11.350 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-07-03T15:58:11.362 DEBUG:teuthology.orchestra.run.smithi028:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-07-03T15:58:11.367 DEBUG:teuthology.orchestra.run.smithi148:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-07-03T15:58:11.389 INFO:teuthology.orchestra.run.smithi028.stdout:kernel.core_pattern = core 2024-07-03T15:58:11.414 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern = core 2024-07-03T15:58:11.427 DEBUG:teuthology.orchestra.run.smithi028:> test -e /home/ubuntu/cephtest/archive/coredump 2024-07-03T15:58:11.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-03T15:58:11.475 DEBUG:teuthology.orchestra.run.smithi148:> test -e /home/ubuntu/cephtest/archive/coredump 2024-07-03T15:58:11.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-03T15:58:11.490 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-07-03T15:58:11.501 INFO:teuthology.task.internal:Transferring archived files... 2024-07-03T15:58:11.502 DEBUG:teuthology.misc:Transferring archived files from smithi028:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-07-03_14:50:01-orch:cephadm-squid-release-distro-default-smithi/7785842/remote/smithi028 2024-07-03T15:58:11.502 DEBUG:teuthology.orchestra.run.smithi028:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-07-03T15:58:11.564 DEBUG:teuthology.misc:Transferring archived files from smithi148:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-07-03_14:50:01-orch:cephadm-squid-release-distro-default-smithi/7785842/remote/smithi148 2024-07-03T15:58:11.564 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-07-03T15:58:11.602 INFO:teuthology.task.internal:Removing archive directory... 2024-07-03T15:58:11.602 DEBUG:teuthology.orchestra.run.smithi028:> rm -rf -- /home/ubuntu/cephtest/archive 2024-07-03T15:58:11.605 DEBUG:teuthology.orchestra.run.smithi148:> rm -rf -- /home/ubuntu/cephtest/archive 2024-07-03T15:58:11.650 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-07-03T15:58:11.662 INFO:teuthology.task.internal:Not uploading archives. 2024-07-03T15:58:11.662 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-07-03T15:58:11.679 INFO:teuthology.task.internal:Tidying up after the test... 2024-07-03T15:58:11.679 DEBUG:teuthology.orchestra.run.smithi028:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-07-03T15:58:11.682 DEBUG:teuthology.orchestra.run.smithi148:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-07-03T15:58:11.695 INFO:teuthology.orchestra.run.smithi028.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 3 15:58 /home/ubuntu/cephtest 2024-07-03T15:58:11.704 INFO:teuthology.orchestra.run.smithi148.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 3 15:58 /home/ubuntu/cephtest 2024-07-03T15:58:11.705 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-07-03T15:58:11.717 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-07-03T15:58:11.749 INFO:teuthology.run:Summary data: description: orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 1869.6204118728638 failure_reason: '"2024-07-03T15:50:00.000121+0000 mon.smithi028 (mon.0) 474 : cluster 3 [WRN] PG_DEGRADED: Degraded data redundancy: 31/219 objects degraded (14.155%), 13 pgs degraded" in cluster log' flavor: default owner: scheduled_gabrioux@teuthology success: false 2024-07-03T15:58:11.749 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-07-03T15:58:11.835 INFO:teuthology.run:FAIL